Try putting your hand on the top of the bars near the stem when signaling with the other hand. This should give you more balance and keep you from accidentally turning the bars.