I think if drag was an issue with rudders, we wouldn't see them on K-1's. I am a definite fan of the rudder, and I use it for steering weather I am racing or paddling for fun. As pointed out earlier, long distance paddlers swear by them.

Edit: I should point out I fall into the "explorer" category, and prefer paddling longer distances. My Sunday AM fitness paddle is 36 miles, and when I paddle all day can be up to twice that. I can see how a rudder might not be preferred in surf and rock gardens.