First of all, the four of them have different identities and can enjoy different benefits naturally. As we all know, among these four people, Whitebeard is a pirate who spends a lot of time wandering on the sea, while Warlord and Cap are senior officers of the Navy since more than 20 years ago. Except for some special circumstances, they didn't need to go out on missions often and just stayed at the Navy Command.
And it was precisely because Warlord and Karp, more than 20 years ago, had become senior officers who didn't need to go out to sea frequently and only needed to stay at the naval headquarters. As such, they could enjoy the various perks (such as body maintenance and disease prevention) offered by the World Government and Naval Headquarters while they were on standby at Naval Headquarters until they were ready to go out on some special mission.
But unlike Warlord and Cap, who stayed at the Navy HQ for a long time and received all kinds of treatments from the World Government and the Navy HQ, Whitebeard, because they were pirates hunted by the Navy and wanted by the World Government. This makes him not only unable to enjoy all kinds of treatment like Warlord and Karp, but also endure the bitter life of not being able to stay on land all year round, but only drifting in the sea. So it also creates a huge difference in physical fitness even though we look at Warlord and Karp and Whitebeard, who are obviously about the same age.
As for Zephyr, he's (ex)/Crocker-3/, and his health would be made worse by another special circumstance. Zephyr retired from his position as "Admiral" to train new Navy recruits. However, as a former naval officer, he was treated as an "Admiral" by the World Government and Naval Headquarters.
However, a year after the end of the war, Zephyr lost an arm in a battle, and the old man began to weaken rapidly. Especially when Zephyr quit the Navy to form his own "NEO Marines" organization, he began a year or so of wandering the seas, which made Zephyr's already poor health due to his injuries even more debilitating.
So while (former) Admiral Zephyr could receive the same treatment from the World Government and Navy Headquarters as Sengoku and Cap, he hadn't been getting the rest and care he deserved since his injuries a year ago, and so he'd end up letting the (former) Admiral off the leash, and in just one year, his health would drop to extremely poor levels.