You do get better at looking around, but still need to keep the focus on the trail

What does happen is that you start enjoying the trail more as you get more relaxed. One of the secrets is to look ahead, pick your line and not concentrate on the stuff that is already under your wheels.