Hello Lesley,
I have not ridden either but looking and comparing the specs of the 2 bikes.. they reflect their intended uses.
The Allez will likely feel a bit "rougher" and more aggressive. If you compare the geometry of the two. the Allez has a shorter head tube (putting you in a more leaned over racer position), slightly shorter wheelbase and more aggressive fork rake.. all of which will usually contribute to a more race like experience on the bike.
That's not to say it isn't comfortable or won't fit you but I thought I'd add a comment to your inquiry 
The secteur is built to be a slightly more comfortable bike over the long haul and its specs won't slow it down much, but you might find it a bit more comfortable since you won't be quite so leaned over and the rake and wheelbase are tweaked to be slightly more stable.
But really in the end.. these are all just specs, as you probably already know. It's really about your experience and interaction with the bike.
But .. I did want reply to your thread. Best of luck.
Push the pedal down watch the world around fly by us