Why do people support this?
There's a separation of religion and state, and Congress is literally one of the 3 branches of the state. The state shouldn't have any religious stuff whatsoever. That includes Congress, the White House and the Supreme Court I guess, as well as arguably state and local governments.
And that's without analysing the potential security argument. Which I guess may or may not apply anymore but it's still there.
Our First Amendment protects religious freedom
and separation of church and state. It does not , however, enforce an official Läicité policy. As long as no religious favoritisms enters legislation, personal religious traditions observed visually are protected.