Is artificial intelligence an existential threat to humanity?

[Aurelius]

Yes, very urgently so.

[GregTheGreat657]

It's not yet developed enough to be an existential threat to humanity

[Ferguson97]

No, The Terminator and The Matrix are just movies.

[Boobs]

I think you meant to write “climate change”?

[Since I'm the mad scientist proclaimed by myself]

Artificial intelligence has the potential to both severely damage our world and make it much better, depending on how we play our cards.

[Middle-aged Europe]

We must stop Skynet from becoming operational... again.

[John Dule]

Only in the sense that it will automate away many jobs (including mine).

[Aurelius]

I think you meant to write “climate change”?

"At least I'm not dying of climate change," the dying man says, as a swarm of nanobots vacuums the iron out of his body to assemble paperclips, shortly before dying of hypoxia due to total loss of hemoglobin.

[Aurelius]

Of these statements:

1. AI capability is advancing at a very fast rate.
2. As AIs get smarter, they will be able to iterate and get even smarter faster and faster. Once they get smarter than a human, this will happen exponentially faster.
3. The alignment problem - having an AI that is given a task, how do you get it to only consider solutions to that task that fit within human moral standards? - is extremely difficult, and despite a tremendous amount of research remains completely unsolved.
4. As the number of AIs increases (probably exponentially, as with basically every other technological advance), even if the probability of any one AI going rogue is extremely low, the probability of some AI somewhere going rogue increases exponentially.
5. Protein folding is a solved problem. This means a sufficiently computationally powerful AI can bootstrap a nanofactory very straightforwardly.

Which one is false or a non-sequitur and why?

----

 In all seriousness, those of us who are firmly convinced of the existential threat of runaway AI seriously need to  Start working on explanations of this, in an accessible manner, in a way that is understandable to normal people. 99.9% of people  I'll tune out when you tell them "read these 50 blog blog posts on less wrong. com, and then read then read these other 100 blog post add blog posts explaining the jargon in everything you just read, and" and justifiably so.

[Pedocon Theory is not a theory]

Of these statements:

1. AI capability is advancing at a very fast rate.
2. As AIs get smarter, they will be able to iterate and get even smarter faster and faster. Once they get smarter than a human, this will happen exponentially faster.
3. The alignment problem - having an AI that is given a task, how do you get it to only consider solutions to that task that fit within human moral standards? - is extremely difficult, and despite a tremendous amount of research remains completely unsolved.
4. As the number of AIs increases (probably exponentially, as with basically every other technological advance), even if the probability of any one AI going rogue is extremely low, the probability of some AI somewhere going rogue increases exponentially.
5. Protein folding is a solved problem. This means a sufficiently computationally powerful AI can bootstrap a nanofactory very straightforwardly.

Which one is false or a non-sequitur and why?

----

 In all seriousness, those of us who are firmly convinced of the existential threat of runaway AI seriously need to  Start working on explanations of this, in an accessible manner, in a way that is understandable to normal people. 99.9% of people  I'll tune out when you tell them "read these 50 blog blog posts on less wrong. com, and then read then read these other 100 blog post add blog posts explaining the jargon in everything you just read, and" and justifiably so.

All five.

Anyone going ballistic over the existential threat 'posed' by today's AI has been deeply deluded about much of modern computing, particularly regarding the likelihood of death via the mass paperclip-ification of humanity.

Please try to read some other science fiction, preferably something that doesn't involve paperclips.

[Dr. MB]

Depends. At its current level of capability no but in the future you can't predict. Who knows?

[Boobs]

I think you meant to write “climate change”?

"At least I'm not dying of climate change," the dying man says, as a swarm of nanobots vacuums the iron out of his body to assemble paperclips, shortly before dying of hypoxia due to total loss of hemoglobin.

First of all, thank you. Thank you endlessly for giving me the gift of laughter. I haven't laughed this hard at an atlas post in a good while.

Second, the only thing funnier than your post is that you wrote it believing that you are the one coming off as smart, reasonable, and generally not-a-lunatic.

[Santander]

Human stupidity is a far more acute existential threat.

[beaver2.0]

Not right now, but I could see it happening.  Probably by the time any of us know it will be too late to do anything.

[John Dule]

I think you meant to write “climate change”?

"At least I'm not dying of climate change," the dying man says, as a swarm of nanobots vacuums the iron out of his body to assemble paperclips, shortly before dying of hypoxia due to total loss of hemoglobin.

First of all, thank you. Thank you endlessly for giving me the gift of laughter. I haven't laughed this hard at an atlas post in a good while.

Second, the only thing funnier than your post is that you wrote it believing that you are the one coming off as smart, reasonable, and generally not-a-lunatic.

This is one of those instances where the line between parody and sincerity is completely obscured by the internet.

[Devout Centrist]

I think you meant to write “climate change”?

"At least I'm not dying of climate change," the dying man says, as a swarm of nanobots vacuums the iron out of his body to assemble paperclips, shortly before dying of hypoxia due to total loss of hemoglobin.

lmao haha get fu.cked, I yell at my broken iphone as I lay dying from dehydration

[Aurelius]

I think you meant to write “climate change”?

"At least I'm not dying of climate change," the dying man says, as a swarm of nanobots vacuums the iron out of his body to assemble paperclips, shortly before dying of hypoxia due to total loss of hemoglobin.

First of all, thank you. Thank you endlessly for giving me the gift of laughter. I haven't laughed this hard at an atlas post in a good while.

Second, the only thing funnier than your post is that you wrote it believing that you are the one coming off as smart, reasonable, and generally not-a-lunatic.

...you're welcome. Upon re-reading my post, I do realize that it sounds almost Time-Cube levels of insane to most people. This is often what happens when people who've spent a lot of time thinking about AI dangers sound like when talking about it to others, and a lesson to me on spouting narrow technical jargon to someone unfamiliar with it.

I have a computer science background, and rogue AI is something I've been thinking and worrying about for years. Until the last couple days, this wasn't something I was worrying too much about, because I assumed that sufficiently capable AI was 40-50 years away. The recent news out of Google made me realize that I haven't been paying close enough attention to the progress of AI in the last year or two while I've been doing other stuff, and that current AI capabilities are at least 2-3 years ahead of where I thought they were. This means the rate of development is much faster than I had thought, and it's forced me to revise my estimate of the arrival of sufficiently capable AI to 20-30 years from now, if not less. This has made the problem much more urgent in my mind, and is forcing me to think about the risks more intensely than before, hence my freaking out on here over the past couple days.

The "paperclip maximizer" is the go-to example, among those who talk about AI risks, of an AI programmed to do one specific thing, that overly narrowly focuses on that one thing to the exclusion of literally everyone else. So yes, talk of being turned into paperclips is commonplace in those circles. But yes, it definitely comes across as bad scifi to the uninitiated, now that I think about it.

  Probably by the time any of us know it will be too late to do anything.

This is exactly the concern. As AI get smarter, they will not only accelerate in their capability improvement rate (as I mentioned above), they will also get more adept at lying to human handlers if they decide that serves their goal.

Generally, AIs work by doing millions and millions of matrix multiplications over some search space to find an output that best satisfies some mathematical function based on the input. Currently, this process is incredibly opaque and we don't really understand what the numbers in these matrices "mean". In other words, we are unable to track the AI's "thought process" in terms of how it gets from input to output. This obviously has enormous implications in terms of our ability to detect whether AI systems are lying to human operators, or whether they are on the verge of doing something incredibly dangerous.

[Kleine Scheiße]

Elon musk needs to buy google. He has to step in.

[Kleine Scheiße]

I have a computer science background

That settles it. Boobs pwned epic style

[Boobs]

"At least I'm not dying of climate change," the dying man says, as a swarm of nanobots vacuums the iron out of his body to assemble paperclips, shortly before dying of hypoxia due to total loss of hemoglobin.

a lesson to me on spouting narrow technical jargon to someone unfamiliar with it.

[Virginiá]

The thing that worries me about AI is that there are so many nations, corporations and other groups racing to develop one that I worry we're rushing into something we don't fully understand nor will be able to control.

But we're a long ways away from a superintelligent or even general AI, and the technology to let an AI manifest massive nanobot swarms to suck the iron out of our blood doesn't even exist yet. There will be many more major steps before we create an AI that can threaten humanity.

[Farmlands]

100 percent. I really don't understand how one can think climate change, a long process which hasn't been predicted by any scientist to possibly wipe out human life is one, but exponential and generally overlooked AI evolution is of no concern. In the future, yes, but some more oversight is definitely needed.

[Antonio the Sixth]

100 percent. I really don't understand how one can think climate change, a long process which hasn't been predicted by any scientist to possibly wipe out human life is one, but exponential and generally overlooked AI evolution is of no concern. In the future, yes, but some more oversight is definitely needed.

One is an actual, proven, and at least partially inevitable fact of the world. The other is a series of conjectures resting on conjectures resting on conjectures.

There's definitely much to be potentially worried about if we create conscious AI, but to jump from that to "OMG LITERALLY TERMINATOR" is a symptom of severe popculture brainrot. Most likely, such AI will likely become a tool (as well as potentially a victim) of the same oppressive systems that shackle humanity today. Men with machines enslaving other men, as Frank Herbert envisioned. I mean, that's clearly the direction the tech industry is going right now.

[PSOL]

AI that is both sentient and can actually feel as a person, as claimed to be the case by the new Google AI, is where I draw the red line and say that under no circumstances should this exist. By all accounts this is a bad idea on both abusing the new persons and ruining an already tough labor market where wages are immensely depressed and there not being enough non-temp positions most can live well on. Dealing with new people entering the job market like this, and the chance for immense abuse that will drag down every laborer just as free labor was during times of slavery, is a hellscape to be made.

[Person Man]

We don't know. It might never become a bad thing even when the "singularity" is well in the rear mirror or it could serious and immediate problem without it advancing much more than it already has.