Batteries are a notorious source of magnetic interference and one of the most difficult to compensate for in compassing (their magnetic signature changes).

Is this a digital or analog (waving hand) compass? If it's digital, there is probably a calibration routine you can use to account for any magnetic interference (it most likely would involve riding in a circle) from your bike.