In a nut shell, it's a millennial generation take on unrequited love. Guy develops feelings for a female friend, desires to take it to the next level but is rebuffed with "let's just stay friends." or at least a presumption that such a thing would happen.
More accurately, it's an invention designed by our blubbering, socially incompetent contemporaries to blame their inability to cope with rejection and letdown on society and trivialized versions of people instead of themselves.
More often, it's actually not rejection so much as the fear of rejection that keeps people in the so-called "friend zone". See Snowstalker's example.
It might be because I am just as (if not more) awkward as Snowstalker, but from my perspective his attitude in that situation makes perfect sense. Sure, he could go on and make his feelings known, but why should he if nothing good can possibly come from that?
To get it out of his system and be able to move on.