Breastfeeding And Weight Loss

Hello! I am currently exclusively nursing my four month old and am trying to lose 25 pounds. I see positive results for completely eliminating sugar and flour from my diet while nursing but I am hesitant to begin fasting while nursing. Is it safe for me to fast while nursing? Also, does adding almond milk to coffee technically go against fasting? Thank you!