When I first started riding I was terrified going 7mph and if someone was approaching me I would shout out "New rider, absolutely no controle! Pass at your own risk!"

And my hands ached. I first got gloves which helped a lot. Then I put gel pads under the bar tape to make it bigger (I found that winding my hand tightly (too tightly) around the smaller size made my hands hurt a lot, but with a larger diameter it was relieved). Just recently I've been able to relax my hands enough to not have this be an issue.

I think that you will get over it with experience, but in the mean time taking steps to relieve the pain is important. As you learn to relax your hands you will find yourself going in a straighter line.

Have you thought about finding a bike club? Clubs that have various levels of riders can be a great way to learn and get over some of the 'newbie' issues.

Barbara