I have a proportionately long torso for my height and yet I ride a WSD. I also have short legs and short arms, so for me, it's really about the overall geometry of the bike.

Coincidentally, I'm 5'4" with a 28" inseam and I ride a 19" Terry Isis. My guess is that at your height with the same inseam as me, you don't have a long torso. You may have longer arms or a more flexible upper body than I do...so it is possible that you do need a longer stem. Currently, I have a 10 cm stem on my Isis (and on my 19" Terry Classic), but again, keep in mind that my torso is longer than yours.

I agree that you should do a few more rides with the bike as is...it may just be your body adjusting to the road position.