Brake rubbing happens to all of us at some point so not to fret. The positive side is you got a really good work out.

Do go ride with your brother and just to be fair, I think you should tweak his brakes a bit so that its rubbing. Then you can really smoke him.