How do I discern whether a person is just a “challenging relationship” or I just flat don’t like them?

In one of your podcasts you mentioned that a “challenging relationship” is good because being around the person forces you to grow and be uncomfortable.

But at what point can you decide that you just don’t like that person and choose to distance yourself from them?

Does it depend on how much (and what kind) of thought work your doing around them?

Thanks so much,
Sorry if this was an odd question! I hope I explained it correctly.
Debbie