I've had lower back problems and I "ditto" the inversions--I even have an inversion table.
I also recommend yoga, but agree you need to be careful and find a reputable studio for beginners. The stretches you don't like or can't do are probably the one you need the most.
Another thought--don't think only "lower back stretch." Tight hamstrings, pelvis and hip flexors can cause back pain.
I have a great stretching routine I do every morning and evening, but also hesitate to give specifics since it was designed by a physical therapist for my specific problems.
"Well-behaved women seldom make history." --Laurel Thatcher Ulrich
'09 Trek WSD 2.1 with a Brooks B-68 saddle
'11 Trek WSD Madone 5.2 with Brooks B-17