I don't know the real name for it- I just call it a headset cap. Yep, you can replace it. My MTB has one that says "Pedal faster, fool!"

Check on Ebay for all sorts of interesting caps.