Seersucker is supposedly a lot cooler than regular cotton. Worth a try.
My husband has been talking about getting a seersucker shirt for summer riding, either from Riv or elsewhere. He wears baggy tops when riding, either loose-fitting wool jerseys (lighter weight ones in the summer) or mtn bike style jerseys. He can't stand anything snug on top. Plus loose tops camouflage his belly 
He still wears the lycra shorts though; he didn't ride with lycra shorts for quite some time, but then once he tried them he liked 'em and now always wears the lycra shorts for any sort of distance ride.
2009 Lynskey R230 Houseblend - Brooks Team Pro
2007 Rivendell Bleriot - Rivet Pearl