In 1934 he painted The Fleet's In! while working for the Public Works of Art Project of the WPA. This painting, featuring carousing sailors, women, and a homosexual man, was the subject of a public outcry and was removed from exhibition at the Corcoran Gallery. The publicity helped to launch his career. He worked in commercial illustration as well, but Jared French, another tempera artist who befriended him and became his lover, convinced him to devote himself completely to fine art.
He lived with his companion of 35 years, Jon Anderson, who was a subject of many of his works.
In 1999 he died in his home in Weston, Connecticut due to advanced age, just five days short of his 95th birthday.
Cadmus's sister, Fidelma, was the wife of philanthropist and arts patron Lincoln Kirstein.