Any time you do something new there is a learning curve- for both your mind and your body.
Yes you are going to be plenty sore- you are using all your muscles in a whole new way. Biking is not like jogging.
Yes it will get easier (way easier) as you do it regularly.
Yes it seems like everyone is on the street laughing at you- but in reality they likely are not even thinking about you at all.
Yes 4 miles seems like 100 miles. After 6 months it'll probably seem more like 1 mile.
Yes you will slowly gain confidence and one day you'll be riding on the road instead of the sidewalk and you'll feel fine about it.

Good for you in doing this!