Recumbent?
I ask what Maggie does. What are your goals? Racing? Fitness? Just getting out on the road?
I know you don't necessarily want to change the bike, but you could try a recumbent. They really are easier on the back, and some of them are quite fast. If what you want/need is to go screaming down the road, you could do worse than a nice 'bent.
As for your back itself, strengthening the core couldn't hurt, if you start slowly. I think.
Give big space to the festive dog that make sport in the roadway. Avoid entanglement with your wheel spoke.
(Sign in Japan)
1978 Raleigh Gran Prix
2003 EZ Sport AX