How do I move forward in my business after making a lot of mistakes?

This is my first business I created and am embarrassed about the experience I have given my customers up to this point. The biggest mistakes being lack of communication and delivering the product late to them because I genuinely thought I would be able to get it to them sooner.

I have some very unsatisfied customers and am so embarrassed with the way I dealt with things, especially because my business has a lot of word of mouth referrals and it is very important to give the client a good experience. I have tried to follow up with them and they will not respond.

These mistakes and dealings with upset costumers have put a stagnation in the growth of my business because of my mindset. I am scared to create a Google business where they could write a review or even post on social media out of fear they will say something negative or to really put myself out there in any way now. While that is not the experience of every customer – I have other customers who absolutely love my product and write the most amazing reviews about their experience – I feel a lot of shame about how I went about it with those customers.

How do I overcome this? Is it too late to fix or should I just start a new business now that I know the best way to do things?