As the case may be:
If fitness is anaerobic training and swimming is best aerobic, then fitness first and then swimming;
If fitness is aerobic training, swimming is best without oxygen, or swimming after fitness;
In short, putting the anaerobic side behind is, in layman's terms, putting strenuous exercise behind.