Don't bother with toe clips/cages. They are not as safe as clipless IMO. Besides where are you going to get a cage? and the cleats for the cage? Don't know if you can even get a such a thing...

Go with a clipless. Some are not that expensive and do a descent job. Time, Look, Speedplay...

Get the bike you want. you'll be happier and more content. And in the long run, it will be cheaper.