I guess we could do the math... 500 yards is about 457 meters or .28 miles. You could find a triathlon with 1/4 mi swims and see how you compare.
I loaded up the Danskin results for Seattle, since I know it's big and has a lot of first-timers. The slowest swim times - it's about 800 meters/.5 miles - were over an hour, some well over. There are a lot of swim times between 35-45 minutes. Considering you have 350 meters to add to get to that distance, you'd probably be right in there somewhere.
So, no, you're not laughably slow, and you have plenty of time to train. Consider a couple of lessons/coach sessions, or just stick with it for a while until you can swim the distance you want to swim essentially non stop (at least, no major rest stops). A friend of mine went from zero "real" swimming to completing an intimidating and not entirely newb-friendly sprint tri in 3 months. And she wasn't last out of the water either!