How do I feel closer to my partner?

I want to feel closer to my partner but I have found out recently that he has been betraying me in many different ways for the past four and a half years. He sometimes acknowledges this and apologizes but I feel like it still hasn’t been enough.

He doesn’t believe that there should be consequences for anything so he is frustrated that I still get upset at the betrayals sometimes and that I don’t have any trust in him. But at the end of the day, all these thoughts I am having just frustrate me and I feel like I do genuinely want to be close with him. I guess I feel like if I let my guard down around him that he will just betray me again. How can I move forward?