How do I handle my job while I work towards my dream job?

I’ve been in my current job for the last 4 to 5 years and I’ve been wanting to change for the last 3 years. Now this summer, I had a sparkle in my mind, I figured that what I truly wanted to do was flipping houses and I am taking actions to reach this goal.

During these last 3 years the scope of projects that my boss gave me narrowed at several times to the benefit of my colleagues. When realising this I always went through a big period of doubt but somehow managed to overcome the negative feelings by putting into perspective that my goal was somewhere else.
Recently it happened again, in a rather brutal manner. After giving me a project he sent an email to the whole team saying that my colleague was in charge.

I can always tell myself that this will give me more time to work on my dream job but this reaction doesn’t seem to be helpful in the end as I always come back to what seems to be a rush forward thinking : less projects > feeling of being stupid, humiliated, unimportant and confused > it’s not that important because I should really be somewhere else > less projects.

I figure my full transition towards my dream work will take 3 years, that means I will keep having an office job for quite a long time.
What can I do to get out of this loop?