Niche question

I am just starting up with my business. Still in planning phase really. As I am a 46 year old Transgender Woman, I want to work with the transgender community for weight loss. I personally have a strong connection to this community.

This feels specific enough to me but I am wondering if I shouldn’t narrow it even more. My concern is that I will be eliminating clients. I don’t know how I feel about narrowing it down to working with only women or only men or only non-conforming. In this community, gender is an important thing to consider as with transgender people, gender is everything.