The idea of the federalist party as being antislavery is pretty nonsensical when you consider Hamilton literally tried to rig a presidential election for Charles Pickney
Slavery wasn't as big of an issue then and Adams and Hamilton despised each other. It isn't so much that the Federalists were anti-slavery as they weren't pro-plantation.
While the Republican's were overall more pro-slavery, by no means was the Federalist Party free of the the Peculiar Institution. South Carolina's Lowcountry, by far the area of the early U.S. most dependent on slavery, was a major Federalist stronghold. There was also quite a few northern abolitionist Republicans.