Two common causes of knee pain while riding can be:

Saddle either too low or too high (most often it's too low, especially with new riders). A too-low saddle typically causes pain near the front lower part of your kneecap.
or...
If you use clipless pedals, I've read that sometimes the angle at which your foot is set in can cause knee pain if it's not right for you. (I can't speak personally about this since I don't use clipless.)