Lose Weight Or Exercise

I have lost 45 pounds successfully with the Stop Overeating program.

As I stand on the threshold of the last 10 pounds I want to lose, I’ve decided to start exercising and while I maintain a very precise protocol, I noticed that for the past month I gain 4 pounds and lose the same 4 pounds with no additional weight loss to bring me to my goal weight.

I’ve decided to search the answer as to why actually we gain weight when we start exercising. The common short answer I always heard was that muscle weighs more than fat. Alas, I found another issue I was unaware of:

A new exercise regimen puts stress on our muscle fibers. This causes small micro tears and inflammation.
Those two conditions in the muscle fibers are the reason we gain some weigh, as our body responds to the micro tears and inflammation in a way that causes temporary water weight gain.

Armed with this knowledge, which basically tells me that nothing has gone wrong with my weight plateau, I am having a dilemma as to whether I should continue working out or cease until I lose all the weight I want.

Working out gives me a boosted sense of energy throughout the day, and tones my arms and butt :-). When I don’t do certain exercise I like less my slim and loose body.
Not working out though, give me a much slimmer figure, and a lighter feeling overall.

Learning from you that there is no right or wrong decision, still, I wanted your take on how can I help myself deciding what would be right for me.