Question regarding a defeated president refusing to leave office

[President Johnson]

Well, the presidential term expires on January 20 at 12 a.m. EST. At this time, the old president loses his authority over the federal government and is nothing more than a private citizen without any formal power. The new president, as commander-in-chief and head of the executive branch, could simply order the Secret Service to remove his predecessor from the grounds of the White House if he refuses to do so. Any orders or formal acts by the old president would have zero effect and authorities wouldn't abide by them. His signature on an issued order would be be worth not more than mine or yours.

If the former president committs any crime with his actions, he could of course be indicted.

Between the election and inauguration (early November to January 20), the old president is still in charge with all the powers, but I don't see that he can do anything to stay in office beyond January 20. Unlike in parliamentary systems, the term of the US president is fixed. By comparison, the German chancellor could stay in as acting head of government for an unlimited amount of time if no successor is elected. Just like Angela Merkel was acting chancellor from October 2017, when her term expired, to March 2018, when the Bundestag reelected her.

[President Johnson]

But isn't the Secret Service part of the Department of Homeland Security? Wouldn't the (acting) Secretary be the person ordering at that point?

The president, as the head of the executive branch, can overrule any cabinet secretary and/or ultimately dismiss him or her from office and install someone who abides by presidential orders.