Boundary with my boyfriend

I find certain things my boyfriend does annoying me, examples are: he’d promise to call/come over/do certain things together, but didn’t end up happening fairly last minute because work/family came up. It seems he prioritises a lot other things in life more than me. I have had chats with him in the past, he’d feel sorry but nothing changed.

I understand the fact i get annoyed is because of my thoughts; and most of times I can change the thoughts to be less annoyed and let these things go (because I love him and would want this relationship to work).

However I wonder, what is the boundary, that I should leave and not tolerate behaviours that I don’t like in a partner? Am I being a doormat and allow him to treat me like im at the bottom of his priority list?
I feel like I’m confused on this is because I dont really know my self-worth and I’d like to be more certain of myself/ my thoughts / my boundary but also unsure where to start?