I do it every time I oil the chain.

I use an oily rag to wipe everything, so it all gets a little cleaning then as well.