Yes, you could say the same thing about cycling here in New England, although I have seen some change in diversity in the last 2 years or so, at least among recreational cyclists.
We noticed it a lot when we took our son to races a few years ago. There was no diversity at all in the teams. And while professional cycling is often seen as a route "up and out" in Europe, it is definitely an upper middle class sport here.