Agreed. And to some extent, not all food should be tax exempt. Milk should be tax exempt, but not soda. Sliced bread should be, but not Italian bread sticks. Etc....
That makes it complicated. How are you going to determine whether each and every item is taxable or not?
As Dibble said, it's complicated (and no, I'm not for this system). It can be done through a national UPC system (not hard to assign products various coding when being labelled), but you would have to ensure that each store has up-to-date tax information. Additionally, your mom and pop shops which do everything by hand would have to know what gets taxed and what doesn't get taxed.
But, if we were to go to a system of not taxing food to protect the poor, I don't think the poor are going to be buying the most expensive Grade A steak around because it's tax exempt. They are going to buy what they can afford and need to survive. So, only the basics like bread, eggs, milk, etc would need to be tax exempt while the rest carries a tax so the other 90% of society gets their fair hit at the registers.