If you are bouncing either you have way to high a cadence or you are not spinning in smooth circles. Probably the latter.
Another thing you might want to check is your saddle height - if your saddle is set too high your hips will rock which will make you bounce a bit as you pedal, and it gets more pronounced as you up your cadence. But like V suggested, if you're not spinning in smooth circles you'll bounce at a high cadence, even if your saddle is set correctly.