Why do you turn it off at all? I don't talk on the phone a lot, in general, but the point of Bluetooth is to be able to receive calls when you are in the car. When I answer one, it is work related and it has saved me invaluable time. I am not making social calls! Before I had Bluetooth, I would have to stop, pull over and call the person back. I know the information about being distracted and using Bluetooth, but I keep the calls short and most people respect it when I say I'm in the car, just give me the essentials.
The only problem I've ever had is after we put my phone onto DH's Bluetooth. We were driving home from our cycling trip in VT last Sept. when I had several client phone calls of an emergency nature and we couldn't turn off the Bluetooth quickly enough while we were driving. I had to tell the person to hang up and let me call her back, so her voice wasn't coming out of the Bluetooth, where DH could hear! It was a pain, so we took me off permanently.