Workout: How to deal with a flu?

I enjoy doing sports (yoga, walking, ergometer riding, weights…). But I stop, when I get ill (cold or flu with fever), because you can get heart muscle inflammation if you keep exercising (Doctors of a health and performance center once brought this home to me forcefully). So I have to pause till I´m healthy again – this time 1,5 weeks. What is the best way to approach this, since Brooke is pointing out, how important it is, not to interrupt the work-out? Especially when I have to take a longer break (3 weeks or so) and loose shape a bit.
Thanks in advance for your advice!