Fibroids and pregnancy worried sick!

Hi everyone I have just found out that im 2-3weeks pregnant but last November I had to have a 5cm cyst removed from one of my ovaries but when I came around they told me I had a uterine fibroid that they couldn’t take it because of it’s location. I am so worried because I’ve heard so many horror stories and I keep thinking with this fibroid will it make my pregnancy hard or I keep fearing the worse! Has anyone had a pregnancy with fibroids? Thank you so much ❤️