Rinse your clothes out, roll them in your towel to get most of the water out (as you would after handwashing them), and lay them out inside the car as you're doing now. Then you can close the towel in the car door with all but the edge on the outside, so it will dry and be held securely.

Alternately, do you share your office bathroom with few enough people that you could hang them there?