I think there is a distinction between having relations with cousins and immediate family members. The former is generally socially acceptable. The latter is not and we are very fortunate that is the case, for a variety of reasons (biological, psychological, etc.). It might not cause immediate birth defects (statistically speaking) but it opens up the door to abusive situations, decreased genetic diversity, etc. For that reason it should not be tolerated.
But this is all statistically speaking and nowadays we can test for whether there genetic problems. would you support a ban on sex amongst unrelated people whose genetic makeup shows a high propensity for defects/illnesses and vice versa?