Learning to drive with OCD?

For a few months I have stopped learning to drive and this is because of some practical reasons (like costs, trouble sleeping) but mainly about the strong anxiety and OCD I experienced learning to drive, also because of my aspergers.

I kept on having intrusive thinking like going to the other side of the road or the intense fear that I would cause an accident even with my instructor by me.
I had over a dozen lessons and the little further I got the worse it seemed to get and the last lesson I just broke down after being stuck in traffic because I really thought I would lose control and cause an accident. I also found driving difficult because I found it very difficult to concentrate and multitask and doing everything all at once in the vehicle whilst concerning the safety of others (I think this where my aspergers comes in such as judging the speed of other vehicles)

At the moment I feel comfortable I'm not driving but the same time comparing myself to others that drive and concerned about my future like getting a job, taking people out etc. And I keep predicting the future like even if I do pass the test that I will cause something to go terribly wrong and put myself and others at risk.

I'm having CBT but I can't see myself learning to drive again, but everyone keeps saying about how much a necessity it is. But should I try again sooner rather than later? I just really struggled with concentration because of so much anxiety and just stuck in my head.