Watched them recently so I can remember them pretty well right now, here’s my list from worst to best:
Fantastic Beast and the Crimes of Grindewald
Fantastic Beasts and Where to Find Them
Harry Potter and the Goblet of Fire
Harry Potter and the Chamber of Secrets
Harry Potter and the Deathly Hallows Part 1
Harry Potter and the Order of the Phoenix
Harry Potter and the Sorcerer’s Stone
Harry Potter and the Half Blood Prince
Harry Potter and the Deathly Hallows Part 2
Harry Potter and the Prisoner of Azkaban
Prisoner of Azkaban is just head and shoulders above the rest, and if it weren’t for him directing Children of Men instead, I would say Alfonso Cuaron should have done all of them. The other 6 movies I enjoy quite a bit, though I think Goblet of Fire badly handled the source material (which I consider the best book). The Fantastic Beasts movies are just... bad, especially the second one.
I agree completely with you on this.
Goblet of Fire (the movie) had so much potential - the book was by far the best - but director Mike Newell turned it into a mediocrity of a movie. (He didn't even read the book before directing the movie, citing its length!)