Cloudy,
If you really are a new bike rider, then it could take a couple of dozen rides or more for your body and muscles to adapt to the new positions and strains you are putting it in. As Knot said, try to 'float' on your bike and keep all your joint loose and light and not locked into place. In time, your muscles will strengthen more to help hold your body in the biking position.
Muscle soreness is expected- actual pain can be either normal and temporary, or not normal- depending.