My ancestors (from the volunteer state) actually did volunteer to fight for the Union. But I only believe in the use of violence in self-defense, so obviously I couldn't fight.
I've absolutely never understood pacifism or things that look like it. Is it a doctrine of simple and complete surrender to the evils of the world? Obviously, you wouldn't believe in violence to force someone into slavery, but you would also deprive yourself of some of the necessary tools to right that wrong. And why limit it to specifically "self" defense? What about defense of your wife or your children or your family as a whole? What about your neighbor, innocents in general, or your country? I'm not attempting to prescribe that one decide to take up a state of constant war against what they perceive as evil, but limiting your ability to impose force against others to only self defense can be quite harmful.