Quit worrying about being called a *****. Do not deliberately act like one, sure, but don't worry about it, even if it happens.

I also suggest more assertive behaviour. And, by the way, it's never too late to change seat.

Good luck!