A political party is perfectly fine to expel members who act against the interests of the wider party - the point of a political party is meant to be a machine to elect people who have broadly similar aims and goals. Since Felder is sitting with the other party in the State Senate then the Democrats have a right to expel him since, as far as I am concerned, that is breaking rule number 1 of being a member of a political party. He can still run as an Independent or as a Republican or whatever: it means that in the General Elections voters will actually have the opportunity to vote for a real alternative and that's a positive thing.
Felder has every right to be a Democrat, period. We have free agency in this country.
free agency also allows a group to remove people who they feel act contrary to the aims of said group. I don't know about how but sitting with the opposing political party when your party has a majority strikes me as meeting that definition.
