I *just* did this the other day because my straps were smelling really gross. I just got the straps wet, put some Dreft on my fingers, rubbed it all around on the straps, and then rinsed the straps under the tap. (I tried not to get a lot of soap on the helmet itself. Not sure if that matters, it's just how I did it.) The straps smell nice and clean now!

I'm baby-free, but I've been using Dreft on my gentle and hand-wash laundry ever since Woolite did strange things to some of my nice clothes. It works well.