What the others said. I live in the Chicago area, so I understand the winds. Don't you love how they just shift on you? I've done rides in Iowa and Wisconsin, and by far the hardest part of riding in the open is the wind. That will certainly slow anyone down.
My average speed is faster than yours, but so what? I train to be fast, so I ride with clubs that hammer and sprint. I guess I just like to go fast. I'm not a racer, but I am competitive. On a flat ride, like my commute to school, I average about 20mph. On a hilly route, I average 15 if I'm lucky. If there's a good headwind, I drop to about 17-18mph on a flat route. On club rides we cruise at 22mph with sprints to 30mph. When I ride with my friend who's just getting into cycling, I go 11mph to stay with him. So I guess it all depends on what I'm doing. I think what's most important is that you're having fun. One thing I've noticed since going slower is I take time to see what's around me.
My $.02.
"Only the meek get pinched, the bold survive"