I think here are the ones who had retirements longer than 40 years. Years in retirement in parentheses:
First ladies:
Frances Cleveland - 50 years (1897-1947)
Julia Gardiner Tyler - 44 years (1845-1889)
Rosalynn Carter - 42 years (1981-present)
Sarah Polk - 42 years (1849-1891)
Edith Wilson - 40 years (1821-1861)
Acting first ladies:
Sarah Yorke Jackson (Andrew Jackson's daughter-in-law) - 50 years (1837-1887)
Harriet Lane (James Buchanan's niece) - 42 years (1861-1903)
Notable that James K. Polk only had 3 months of retirement before dying of cholera. Sarah never remarried.