Hi David,
I think I was the same with work - always a low level of anxiety, but generally able to cope. What tipped things for me was being given a management position - something I didn't actively seek, and which I don't think I'm totally suited to. I kept going for five years, but in the end I had to call it quits.
The good news is that, not so long after, the chance for my current job came along (plus pregabalin). So here I am in Japan, and pretty happy (most of the time).
So keep fighting. You never know the twists and turns that life can take.
Hanshan