What is a "roller?" Do you mean a breaking wave, or a swell?
I don't blame you a bit for being disappointed with the boat if 2 to 3 footers are a problem. That would make real waves downright undoable. It sounds like the skeg might be the real problem and if it is one of those little high aspect thingies--I've wondered just how effective they would be when you really need them. Maybe a regular skeg could be fitted to the boat without much difficulty.
I have a CD Sirocco that really does a nice job in waves. I haven't had my new boat in very big waves yet, but I'm looking forward to seeing if it does as well as the Sirocco. It is a bit longer and lighter and specializes in going straight, but I'm going to take it slow and careful until I see how it is going to handle. It has no skeg, nor rudder.
This may sound a little strage, but my recreation kayak is a great surfing boat. I surfed it on a very large, steep tugboat wave one time and I couldn't believe how well it did. The boat gets all the credit though, because I was basically out of control. I meant to surf down the wave, but got turned a little and went diagonally down the wave in the direction the tug was traveling. The boat stayed on the wave for a long ways--just like a surf board and it was really flying. I actually was catching up with the tug. There's a little more to the story, but the point is that I wish I were good enough to do that sort of thing on purpose.