I went down there for it a few years ago but got rained out. DH has done it a couple of times. It's a hard ride for flatlanders, but it's doable with decent training--which is often hard to get by the first week in May. Rain is often an issue that time of year, too. Descents in the rain are just not my cup of tea. In fact, the year we were down there, a friend of ours got in a accident when another rider's brakes failed on a descent. For me, I want a better guarantee of descent weather if I'm going to travel 5-6 hours for a ride.
A ride you might enjoy, Catrin, is the Harpeth River Ride. It's outside of Franklin, Tennesee in June (I think). It's not nearly as hilly or as crowded, but it's a nice ride nevertheless. I did the ride in 2008 and combined it with a 40-mile ride along the Natchez Trace. It was a nice weekend. Franklin's a nice town, too, with a lovely little downtown.