The need for visitation by a caseworker and the welfare assistance provided by the government are directly related.
A search is not reasonable merely because the government feels that it might serve somebody's "paramount needs." The requirement for probable cause is made clear by the Fourth Amendment itself.
Probable cause does not apply because criminal conduct is not alleged for justification of the search. The justification is that the government has a need to ensure that its money is being spent properly. Note that James was never accused of a crime. Note also that her refusal to allow the search was not met with police officers forcing a search. The only "punishment" James received was termination of benefits.
What if this had been a defense contractor that received funding from the government to built jets. Could the government send inspectors to check that quality manufacturing practices were employed, or would they need a search warrant. If a search warrant is required, how would they go about getting one? Would the government be justified in canceling the contract and demanding its money refunded if the company refused to allow inspections?