I have a Giant OCR 3 in an XS that I love. LBS just had to make a few adjustments to my handlebars and I was good to go.

When I was looking, the Specialized dealer treated me the same way, showing me only the Specialized Dolce & telling me that it was the only womean specific bike made. I honestly don't know if he thought I was an idiot, or if he didn't think I was really interested in making a purchase. Either way he lost the sale.

Definitely don' let others talk you into something, do what feels right for you!