interesting, so perhaps the chamois is slightly thicker than the fleece putting pressure on your girlie parts.
You may be a good candidate for a saddle with a cut out. I am a leather saddle fan so ride a Rivet Pearl and a Brooks B17 Imperial
Does you LBS offer the ability to go ride a saddle for 40 miles before purchase? Ask them and if so try some with a cut out.