The problem is most likely that your soles are too soft and your foot doesn't have enough support (but it sounds like you know that already). When I bought my bike two years ago, I,too, had a hard time spending more money than it already cost. Therefore, I used eBay to buy almost everything I needed! I bought used road shoes (really nice and cheap), the pedals and cleats (but had to borrow a pedal wrench from a friend to put them on), and cycling jerseys a plenty. The only things I bought new at first was the helmet and a pair of bike shorts. Try eBay for some inexpensive cycling shoes and pedals/cleats. I'll bet that'll solve your foot/knee pain in a jiff.