I think I'm lucky in that my left eye is very much dominant. My right (bad) eye barely factors into my visual perception - and with how we've chosen to treat my vision, that's been an encouraged development. I don't notice that I can't see clearly in that eye when I'm running/biking, though I imagine it affects the peripheral vision on my right side. I can function almost perfectly with my left eye half-corrected doing everything except driving at night and reading. We've talked about making me a pair of glasses with just the astigmatism correction so I can wear them over my contacts to see road signs in low light but I haven't taken that step yet - but it might be an option for you if the toric is unbearable but normal lenses are ok? I wonder if your brain is still trying to rely too much on input from your left eye and won't accept that it's an unreliable source. I also wonder if they might have a problem matching the focal point of your astigmatism prescription to your visual focal point? If your astigmatism isn't standard but is more unevenly distributed (if your eyeball is shaped really wonky) I could see that being a result. The toric might help but might not, I found them to move more than normal contacts and it was sometimes difficult to keep them in the right position on my eye. I'm curious now though, your depth perception seems to be an odd symptom for just astigmatism, more of a source data/computational analysis miscommunication. I wonder if you might not be helped more by some sort of exercise that teaches your brain how to better interpret the visual input it's gonna get.

(For reference, my distance prescription for contacts is about -6.5 in each eye, my bad eye also should have an astigmatism correction to -5.5 or -5.75. But it's bad enough that if it's not corrected, it doesn't register as visual input.)