All the time. No problem.

Most of my riding is for real-world stuff and not playing-and-sports, so I wear real-world clothes most of the time on the bike.