Haha, I have that problem, and thats without any compression bra!

I think you should go for it. Once you're racing you won't even remember that you were worried because you'll be so focused!