Does a Joy Eat delay fat adaptation?

I’m a few weeks in and still struggling to adjust mentally and physically to no sugar, no flour. It stands to reason that the more flour and sugar in the form of Joy Eats or off-protocol eating that my body has, the longer it will take to become fat adapted and make real progress. Is this accurate? I can go about two days before I catch myself thinking things like “I can’t live like this” or “It’s not working.” Yes, I model those thoughts and I’m doing the work. Including working on allowing this journey to be about more than just my body and enjoying the process. I have A LOT of work to do.
Thanks in advance,