Should I stop joy eats?

I did no sugar, no flour for 2 weeks. Then I started doing planned joy eats with sugar and flour.

After a joy eat, I fell back in to the habit of giving in to urges. Should I avoid joy eats because my brain is so wired for binging? Or just expect to have urges after a joy eat, and make that part of the management of a joy eat after affect? Or just never have a sugar or flour joy eat? Thank you!