Why? Executive privilege is not an enumerated power of the executive. To the degree it has been inferred as an executive power, the reasoning has been that executive power requires being able to receive confidential advice on what to do when in office from his subordinate officials, so long as that advice concerns the lawful exercise of his executive powers. If that confidentiality becomes subject to the discretion of a successor, who may well be a political adversary, it is far less secure and effective and thus weakens the value of executive privilege. I realize that our constitution is extremely naive when it comes to politics, and for that reason, I don't consider your argument entirely wrong, but since the whole scope of executive privilege is one of those constitutional penumbras inferred from the text, I find it runs counter to the whole rationale for having executive privilege in the first place.
However, since the information being sought neither concerns the lawful exercise of executive powers nor advice from a subordinate official, I agree that executive privilege doesn't apply in this case. That would be so, even were Trump still President. Hence, there is no need at this time for SCOTUS to decide if ex-Presidents can invoke executive privilege.
In any case, it would be wise to keep in mind that any decision concerning the invocation of executive privilege by ex-Presidents will not affect only Trump, but all ex-Presidents, including Democratic ones, if a future Republican Congress seeks information actually covered by executive privilege but a future Republican President declines to invoke executive privilege.
It is my belief that all executive power derives from Article II of the Constitution and what Congress allows through a matter of law. Inferring it from the executive power granted by the Constitution, the power lies with the President of the United States. An ex-President, a private individual, is not the President of the United States. That individual, and all other private individuals, have no claim on the powers granted under Article II. As for the rationale of executive privilege itself, I would indeed argue that it has gone far beyond the powers granted to the Presidency. Subordinate officials are almost all creations of law and established to execute the law as proscribed by law. I feel your argument runs against
United States v. Nixon.
I have no concern as to how this works politically. If you've seen my posts in this board, you'd know that I generally have a narrow view of executive powers and privilege. That does not change based on the party balance. Additionally, I would say that the political doctrine at the Supreme Court has been used far more than it should be. I think it has legitimacy in some cases (such as impeachment), but most seem to be a cop-out to resolve the real and serious issues.