Oakleaf, I do drive to ride to some of the group rides I do in the summer, as they tend to be "beach" rides on the North Shore or even Cape Cod. I have a zippered pocket on the inside of the flap of my seat bag and that is where I put my key. I took it off my key ring long ago, since we have a keypad on the front door, and don't need any other keys.
I guess I really don't care about my phone. It's a flip phone, nothing special and I hardly use it. Also, I don't ride in the rain, unless it happens "unplanned." Last Sunday was the first time in 2 years I got rained on.