Trading handlebars from straight to swept back improved my comfort immensely. I tried a few bike shops before I found someone who was willing to experiment. I have tourist type handlebars traded from an old Schwinn. There are many names and styles of handlebars that can suit you:
moustache, sparrow, tourist, priest, etc. If you look at the Velo Orange website you can click on pictures of some of these.

Some of the handlebars have rise and sweep which can put you in a more upright position.