If your pain is decreasing, there's a good chance that your body is just adjusting to riding and it will go away on it's own. I had a few aches and pains like that when I started. I also had a few that didn't go away without making changes, but these got worse over time, not better. I say keep riding and see if it resolves on its own.

As far as posture, you don't want to slouch or deliberately round your back, and you should keep your arms slightly bent and your shoulders out of your ears. Also, you should stretch after every ride.