I would do what your PT says because s/he knows your situation way better than we would.

Swimming (as your PT suggested) sounds good, because you can not stress muscles and such if you vary strokes and aren't too intense.

Good luck with your injury.