Not knowing what type of bike you have makes it hard to give you suggestions - if you post pics of your bike how it is set up now and/or pics of you on the bike, we can try to help.

If there's any resale value of your bike on craigslist - you could sell it and buy another used one on craigslist that's more suitable to you.

Yes, your core is probably weak - the first summer I started cycling I had lots of wrist/fit issues - and a lot of those were resolved as my core grew stronger. I'm quite happy bending over much further now than I was the first year. So to a certain extent - you are just gonna be uncomfortable for a while.