Well, this isn't going to win me any credits, but I always wear a PFD when I'm in my sea kayaks, but seldom in my rec. yaks. I always have one along and right close by.
Not to rationalize my casual adherence to convention, but I am a very good swimmer and when I'm in my rec. kayaks I don't get too far from shore.