If it's completely consensual (and I'm talking both polyandry and polygyny), I honestly don't care. The idea that "normal" relationships can be monogamous only is without a basis.
That being said, I wouldn't be interested in polygamy myself (I'm not interested in marriage at all).
Even if it is completely consensual, there's the damaging psychological effects on any children that are raised by parents practicing polygamy. And of course, pretty much every time polygamy has been legalized, women have suffered as a result.