A lot of people start off with SPD pedals. They allow you to walk normally and a lot of the shoes that are compatible look more like hiking/tennis shoes. The pedals themselves (there are various models) are pretty cheap, as are some of the compatible shoes. The downside to SPDS for some is that the pedals hold your knees in a fixed position. For that reason, it's important that they're set up correctly. One of the other downsides for some is that SPD compatible shoes are often not quite as stiff as road shoes. That's bothersome to some.

There are a handful of other options that allow for easy walking but also offer float. Speedplay Frogs come to mind. They offer unlimited float (meaning your knee has a full range of movement) and I know of at least a couple of people on TE who really like them.

That leaves a bunch of road options. It seems like the two most common ones are Speedplay and Keo Look (or Look style) pedas. I have the latter. There's a learning curve with them, but with the tension set low, they're pretty easy to get in and out of. They hold up well, and provide for float (how much depends on which cleat you get). Plus, the cleats now come with a bit of rubber on them, which makes stoppind and walking a bit easier. I like SIDI road shoes myself, too (they're the only cycling shoes that really fit me), which makes the Keos a better choice for me. But a lot of road shoes aren't cheap and it can sometimes be hard to find a pair that fits.

So, my advice is to start with the shoes. Find a pair that fits. You don't want them too tight (because your foot may swell) but your heal should stay put in them. The shoes may dictate what kind of pedal is available to you, since not every pedal is compatible for every shoe.

But in the end, if the shoe fits and you give yourself a chance to learn how to use the pedals and to feel comfortable, you're good to go. Don't get analysis paralysis with it!