Help with marriage conflict

My husband and I have been so much happier since I joined you and quit drinking and flour and sugar. But there is a recurring problem I don’t know how to get over: I want guests and he does not. We are recently retired and live 2.5 hours from most of our city friends and love living here in this beautiful home. I am trying to make new friends and build a different life and community and trying not to judge him that he does not, it is up to him how he fills his time and I try not to feel guilty when I leave him alone to lunch with a friend.
My son wants to come with two of his buddies for the long weekend and I would love love for them to come, they came a year ago and were so pleasant and respectful and did their own thing almost all the time. But my husband did not speak to them or me all weekend, it didn’t bother them but the pain was excruciating for me as I tried to navigate everyone to happiness. Now they have asked again to come and again he is angry and silent.
I just don’t know what to do. This is the life I want to live, where my kids can come and bring friends and enjoy our home and I can enjoy being in their orbit occasionally. My kids are 25 and 28, but it is my son (who is unattached) he has the most difficulty with, probably because my daughter wouldn’t even ask if she could bring friends. She brings her boyfriend and that is ok (but not encouraged). My husband has a daughter, 40, and two grandchildren and I like her and do my best when they come to visit and try to entertain his elderly mom when she comes. He is ok if my parents come but he certainly discourages such a visit where he can. Parties I don’t even bother to ask anymore.
I am trying to let him be angry and uncertain and not get angry myself. He will freak out and that is ok. But is it ok that I insist on having my son and his two friends? This is what I want, is it ok to get my own way even if it isn’t what he wants. This confuses me and is an issue that is not going away.
My husband technically owns our home and at times like these he likes to let me know that. He says it is his home and not a family cottage. I say what is the point in having this huge house if we can’t share it with our family.