Riders choose their gearing depending on the terrain they'll be riding in, their strength, and the type of riding they plan to do. So if you're noticing that more experienced riders have lower gearing than you, they may be on the right track.

All the other advice you've got here is right on, but you might also want to ask your bike shop about changing to lower gearing. Unless you have higher end drive train components, OR unless your bike has pretty low gearing already (doubtful if most other riders have lower!), you should be able to put on a wider-range cassette for $50 or $60 US.

The tradeoff is that your gears will be "farther apart," and you may find that you have gear ranges where you can't maintain a cadence you like at certain speeds.