Want matches

I understand the concept of want matches and not believing that one person (like a romantic partner) has to meet all of them in order to have a great relationship. My question is what happens when you don’t share a want match with you partner and it’s an area where finding that match with someone else would be a boundary violation or deal breaker in the relationship. For example, physical intimacy and affection. I’m not sure how to proceed, my partner and I don’t have a want match in this area and finding physical intimacy and affection outside of our relationship would be something we’ve both agreed not to do. I see the options as manage my mind and try to be ok with the lack of physical intimacy and affection, not manage my mind/stay anyway and continue to feel frustrated, cheat/lie or leave the relationship. I’ve discussed this with my partner and he doesn’t see it as a problem and in fact says he gets turned off when I bring it up. Guidance appreciated!