Inspect your helmet periodically. Look for signs of wear. As it ages little cracks will start to appear. If it is exposed to a lot of sun and sweat and temperatures, it will age faster than if it is sitting in a closet. I've heard 3 to 5 years, but I bike every day in all weather, and perhaps my sweat is toxic besides, so I've ended up replacing mine every 2 years.

Of course replace it if it is involved in a crash. Even if it looks fine there can be cracks inside that you can't see.