About the shirts and socks--you can find inexpensive Coolmax (wicking fabric) clothes at lots of general sporting goods stores these days. Just make sure the shirts are long enough for coverage in the back and don't have any extra bells and whistles (buttons, collars, etc) that might be irritating while riding. I think you can get Coolmax socks at Walmart.

I second what Eden said about shoes. They should at least have a nice stiff sole--otherwise, you'll be working harder than you need to and your feet may hurt after a while.

Have fun!