That Senate vote says it was from Dec 17, 1805. Does not seem to be quite the same legislation that the House voted on in 1807.
We have to come up for a moment in December of 1805, when a New Englander gets conspicuous again. Stephen Row Bradley, of Vermont, gets up in the Senate and suggests they get cracking on a bill to prohibit slave imports, effective the first of 1808. The time might have seemed ripe, with states calling for a constitutional amendment to permit banning slave imports then and there not that long before, but Bradley’s proposal was taken as too soon all the same. There’s an undercurrent of doubt in all of this as to whether it’s proper for Congress to even consider an import ban before 1808, let alone years in advance.
https://freedmenspatrol.wordpress.com/2016/08/22/new-england-and-the-slave-trade-to-1808/That's an interesting article: talks about prior acts aiming to restrict the trade, and how the New England slave trade was already mostly focused outside of the US, between Africa and the Caribbean.
I think there must have been something about the acts which some New Englanders were dissatisfied with, not that there was a desire to continue the trade. After all, these opposing votes do not even come from Rhode Island, where you might expect. Maybe those who opposed passage in the north were holding out for something stronger.