What is the impact of my very upright riding position? I certainly have more wind resistance than someone in a more aerodynamic position, and I won't break any speed records. To me it is more important to be able to ride as long as I want to and to not have pain afterwards.

I concur with Goldfinch, whatever you decide to do, that you should ramp up your mileage base/time in the saddle slowly. Listen to your body more than to a schedule.
I'm not very upright, but I do bend forward slightly. I've been cycling like this..for past 20 yrs. and have never cycled in dropped handlebars. I've done several 1,000 km bike touring trips with my panniers @60-100 km. per day.

Some years I was piling up nearly 7,000 km. annually. We're car-free in lifestyle.

It's more important at this stage, to keep on cycling and being healthy, rather than worry about optimum aerodynamic position or being quite fast (which I realize you aren't that focused on.)

I have 4 bikes. None of my bikes have dropped handlebars.
Not a big deal to me but I know for whatever reason, it's important to others who don't race, but still want to be more aerodynamic.