Question about non refined flours


I have been off sugar for two weeks, and it’s going well, I feel less attracted by sugar.
For now, I didn’t not quit flour, but I am eat just non refined bread/flours. Could you precise why all flours should be evoided ? Is it to break the habit for eating bread/pasta etc. or are they also bad for hormones ?