IME, loose isn't a problem, as long as it's not so loose that it falls off of your nose.

Fogging is inevitable, especially if you have to stop. Un-covering your nose helps to vent the hot air away from your glasses. I will roll up to a stop sign/red light, pulling my Buff down as I brake, and I'll pull it back up once I get rolling again.