Tennis balls in a sock! (seriously). Put two tennis balls in a sock and tie it off. Lie on the floor with the tennis balls under your back - stretch over them in several places up and down your back as is tolerable. You can use them to hit the sore spot in your shoulder too. This was the only thing I found that really, really helped me with a problem that sounds just like yours.

Mine was from computer mousing with really bad, non-erganomic desks for years, but cycling and a bad bike fit was really aggravating it. The tennis ball trick took care of the acute problem and now I never use a mouse pad - I always mouse on a clip board placed in my lap.