Do you mean something like a gay marriage case (before the Oberfell case) in a state court in a state that had a gay marriage ban?
I think the State Court has to treat the US Constitution as controlling over the State Constitution. But, that sort of case wouldn't usually be in a state court, right? If the crux of the case is Federal law, wouldn't it probably be in Federal Court?
I think a couple state high courts ruled after
Windsor that the U.S. Constitution required marriage equality and thus struck down their state's statutory gay marriage ban, but I don't know that any actually struck down provisions of their state
constitutions on that basis. Actually, an Arkansas court did, but the decision was appealed to the Arkansas Supreme Court which dragged its feet until SCOTUS made it a moot point.
Yeah, if you're trying to get part of a state constitution struck down as a violation of the U.S. Constitution, you're almost certainly better off bringing your suit in federal court.