Are you saying that you believe that tens of millions of people having equal rights, the right to vote, the right to live their lives without facing institutionalized, state-supported discrimination and harassment, all of that liberty, is not worth the liberty that some (and, until you give us concrete examples, hypothetical) white business owners had to surrender?
Did those business owners really have to surrender those rights in order for tens of millions of people to have other rights? Why was that part necessary?
Otherwise I would have to say that yes you have a point. However, I believe it was a very low move to include the provision about business owners, even if it meant no more state sponsored discrimination and harrassment.
Believe me, issues like Civil Rights aren't black and white to me, I actually do think about the outcomes. The fact that the Civil Rights Act of 1964 had a net good effect in regards to state sponsored discrimination doesn't change the fact that it was wrong of the government to force individual business owners to give up their rights to conduct business the way they please.
I can say that if I were in Congress I would've cast an "abstain" vote simply because to this day I don't know what exactly to think of the CRA of 1964. Sometimes I'm pro, sometimes I'm against, right now I'm in the middle.
Other libertarians probably disagree.