I know you're not a believer in either 4 or 5 but how is 5 so much more ridiculous than 4?
Because 4 at least is a combination of factors we know to be
possible, even if extremely improbable. It is possible life has developed outside of earth (but very hard to put a probability on). It is possible that such life would be able to develop technology for space travel - after all, we've done it. Though very unlikely, it is at least possible that one such spacefaring civilization could reach Earth; and finally, we know that it is possible to construct craft that's at least reasonably good at evading or at least delaying detection by specific radars etc. Again, the idea that there is in fact a civilization with all of this which is
also regularly doing stuff on Earth without making their presence known is ridiculously unlikely; but we still in theory know that the mechanisms of it are physically possible.
On the other hand, the other stuff requires a
much more dramatic leap of faith to assume they could be happening right now.
Maybe I shouldn't have conflated time travelers with alternate dimensions but quantum physics and the string theory essentially has proved the concept of the multiverse and why is it so much of a stretch from there to believe that there are portals between our universes?
Careful where you're treading here. The "concept of a multiverse" has certainly not been 'proven', and really falls more under the philosophy of physics than actual physics. Certainly many physicists favor the idea of the multiverse, but many do not as well, and to claim there is any "proof" for the idea is a gross exaggeration.
Quantum mechanics has provided some with an attractive multiverse hypothesis (involving the universal wavefunction etc.) that some find attractive, but it is still nothing more than that. Again, "proven" is a gross stretch. Claiming something is "proven" by string theory makes even less sense since one of the main criticisms of it is that it hasn't been able to prove/solve very much; I believe (though I could be wrong on this) that string theory on the whole has been on a rather precipitous footing for the last several decades, especially after the LHC was never able to achieve any of the results/readings that it was hoped to.
Flying spaceships at light speed across the galaxy seems a lot harder than just slipping through a portal (and it would explain some of the stranger parts like UFOs flat out disappearing/aliens seeming to float through walls).
So I will note that I was not assuming Option #4 involved lightspeed or greater-than-light speed. I'd regard that as pretty much the same as Option 5 (maybe slightly more likely than the inter dimensional stuff since it requires other dimensions to actually exist in a fashion that they can be interacted with). In a relativistic world time travel and faster than light travel are exactly the same thing, so I wouldn't put very much in between them.
For any of those three things to be possible we'd need something completely unknown to us/our understanding of the world. The most we have, to my knowledge, is a few wormhole models which have been shown to satisfy the equations of general relativity. But even from there we get to several sets of Herculean leaps; that they actually exist and aren't just a theoretical possibility, that they actually exist
in close proximity to the Earth, that they can be created and destroyed at all, that there is a way for a group of travelers to create such objects or even affect their creations, that they are able to do so on-demand for human-scale objects and beings, and finally that they are able to do all this without a significant transfer of energy or anything else that would be obviously detectable.
Given all of this, such explanations make less sense by leaps and bounds than even the standard 'aliens idea'. The notion that even one of these sightings/events is caused by this is a gargantuan stretch and is quite frankly a ridiculous conclusion to come to - even when compared to just 'it's aliens'.
TL;DR: 4 consists of "doing things we have done, just much much better, plus an astronomical amount of coincidence and convenience". 5 requires a whole lot more than that; stuff that we have not come close to proving possible or even having a concrete conception of.
full disclosure. Am not a physicist; may have gotten some of the details wrong here, but I'm pretty sure the gist of this is correct. If someone here is more knowledgeable in this area please feel free to correct any of it.