Thanks, everyone, you've all been helpful.

Kano, I think you are on to something with that core work. Mine is not terrible, but not great, either. Today I was really paying attention to my posture/position and what was causing things to go wrong. I really do think I have too long of a reach to my handlebars. They are flat bars on a mountain bike. I'm constantly trying to push myself back in the saddle (and the saddle is level) and even though I can't tell it by looking at myself, I just feel like I'm reaching too far forwards. I think something like moustache handlebars might be enough get me by until I can afford to buy something better.

However, I did notice that when I tried to straighten up to where I thought I should be, that the core muscles needed to be there (and weren't as much as they should be).

Still not sure about the neck and shoulders...I still can't imagine leaning over with my neck cranked up will ever be comfortable for a long ride, but I think that's a little bit different problem than "too much weight on my shoulders because I'm stretching too far". Maybe correcting one problem will at least lessen the other...only trying out bikes that fit will tell me.

And I do totally get buying a bike from a bike shop. I never did before...but then I only road around the neighborhood a bit, so it probably didn't matter, anyway. Now I'm trying to get recommendations for good ones (in a different thread).