Inspect your helmet periodically. Look for signs of wear. As it ages little cracks will start to appear. If it is exposed to a lot of sun and sweat and temperatures, it will age faster than if it is sitting in a closet. I've heard 3 to 5 years, but I bike every day in all weather, and perhaps my sweat is toxic besides, so I've ended up replacing mine every 2 years.
Of course replace it if it is involved in a crash. Even if it looks fine there can be cracks inside that you can't see.
2009 Trek 7.2FX WSD, brooks Champion Flyer S, commuter bike