The cleaning question is a puzzling one to me too.
I finally took my bike to a shop and said "Please show me how to clean it."

The one trick I could never figure out from reading was how to clean the cogs. I don't know if I can explain it any better, I should probably take a picture of it. I've heard it described as flossing between the cogs or earlier in this thread as a shoe shine.
First, find a shop rag that has a nice hem. I 'acquired' a couple from the local pizza shop which uses them as cloth napkins. The hem is nice because it keeps threads from catching on the teeth.
Slide the edge of the rag between two cogs, one hand on each corner of that side, and floss back & forth. Don't fold or bunch up the rag, it won't fit.
You do have to shift to move the chain out of your way to get all the cogs.

I also wondered how often, and discovered that question answered itself: When it isn't shifting smoothly. When I've been out on the trail or out in the rain--but usually those events make it not shift smoothly. I just got fenders on, and noticed the difference right away.

My biggest newbie mistake was not wiping away the excess lube. That makes it dirtier than if you'd never cleaned & lubed it! Some of these lubes are a combination of oil & water or alcohol, which thins it so that the oil works its way into the chain, but you should give it time to do that and for the water/alcohol to evaporate before you wipe off the excess. It is easy to wipe off the excess (same as wiping it down before you lube). Take your shop rag, hold the sides of the chain gently, and spin the pedals so the chain slides through the rag.

This is one of the few things I can do myself and I love doing it because I feel so empowered.