The Exciting, Perilous Journey Toward AGI | Ilya Sutskever | TED

You've all experienced the progress of artificial intelligence. Many of you may have spoken with a computer, and a computer understood you and spoke back to you. With the rate of progress being what it is, it's not difficult to imagine that at some point in the future, our intelligent computers will become as smart or smarter than people. And it's also not difficult to imagine that when that happens, the impact of such artificial intelligence is going to be truly, truly vast. And you may wonder, is it going to be okay when technology is so impactful? And here, my goal is to point out the existence of a force that many of you may have not noticed that gives me hope that indeed we will be happy with the result.

你们都经历了人工智能的进步。你们中的许多人可能已经与计算机对话，计算机理解了你们并且回答了你们。考虑到进步的速度，不难想象，在未来的某个时刻，我们的智能计算机将变得和人类一样聪明或者更聪明。同样不难想象，当这种情况发生时，这样的人工智能将会产生真正巨大的影响。你们可能会想，当技术如此有影响力时，情况会好吗？在这里，我的目标是指出一种许多人可能没有注意到的力量，这种力量让我希望我们确实会对结果感到满意。

So, artificial intelligence, what is it, and how does it work? Well, it turns out that it's very easy to explain how artificial intelligence works, just one sentence. Artificial intelligence is nothing but digital brains inside large computers. That's what artificial intelligence is. Every single interest in AI that you've seen is based on this idea. Over the decades, scientists and engineers have been figuring out how such digital brains should work and how to build them, how to engineer them.

那么，人工智能是什么，它是如何工作的呢？其实，解释人工智能的工作原理非常简单，只需要一句话。人工智能不过是大型计算机内的数字大脑。这就是人工智能。你所看到的关于人工智能的每一个兴趣点都基于这个概念。几十年来，科学家和工程师一直在研究这样的数字大脑应该如何工作，以及如何构建它们，如何对它们进行工程化。

Now, I find it interesting that the seat of intelligence in human beings is our biological brain. It is fitting that the seat of intelligence in artificial intelligence is an artificial brain. Here, I'd like to take a digression and tell you about how I got into AI. There were three forces that pulled me into it. The first one was that when I was a little child, at around the age of five or six, I was very struck by my own conscious experience, by the fact that I am me, and I am experiencing things. That when I look at things, I see them. This feeling, over time, went away, though by simply mentioning it to you right now, it comes back. But this feeling of that I am me, that you are you, I found it very strange and very disturbing, almost. And so, when I learned about artificial intelligence, I thought, wow, if we could build a computer that is intelligent, maybe we will learn something about ourselves, about our own consciousness. That was my first motivation that pulled me towards AI.

现在，我觉得有趣的是，人类智能的核心是我们的生物大脑。同样地，人工智能的智能核心是一个人造大脑。在这里，我想岔开一下，告诉你们我是如何进入人工智能领域的。有三股力量将我拉入这个领域。第一股力量是当我还是个小孩子，大约五六岁的时候，我对自己的意识体验感到非常震撼，对于我是我，我在体验事物这个事实感到非常震撼。当我看东西时，我能看到它们。这种感觉随着时间的推移消失了，尽管现在我只是提一下，它又回来了。但这种我是我，你是你的感觉，我发现它非常奇怪，甚至有点令人不安。因此，当我了解到人工智能时，我想，哇，如果我们能够构建一个智能的计算机，也许我们会对我们自己，对我们自己的意识了解更多。这是我被吸引到人工智能领域的第一个动机。

The second motivation was more pedestrian, in a way. I was simply curious about how intelligence works. And when I was a teenager, an early teenager in the late '90s, the sense that I got is that science simply did not know how intelligence worked. There was also a third reason, which is that it was clear to me back then that artificial intelligence, if it worked, it would be incredibly impactful. Now, it wasn't at all obvious that it will be possible to make progress in artificial intelligence, but if it were possible to make progress in artificial intelligence, that would be incredibly impactful. So these were the three reasons that pulled me towards AI.

第二个动机在某种程度上更为平凡。我只是出于好奇想要了解智能是如何运作的。在90年代末，我还是个十几岁的少年时，我感觉到科学简直不知道智能是如何工作的。还有第三个原因，那就是对我来说很明显，如果人工智能能够实现，它将产生巨大的影响。当时并不明显能否在人工智能上取得进展，但如果能够在人工智能上取得进展，那将是极其深远的。所以这就是吸引我走向人工智能的三个原因。

That's why I thought that's a great area to spend all my efforts on. So now, let's come back to our artificial intelligence, the digital brains. Today, these digital brains are far less smart than our biological brains. When you speak to an AI chatbot, you very quickly see that it's not all there, that it's, you know, it understands mostly, sort of, but you can clearly see that there are so many things it cannot do and that there are some strange gaps. But this situation, I claim, is temporary. As researchers and engineers continue to work on AI, the day will come when the digital brains that live inside our computers will become as good, and even better than our own biological brains. Computers will become smarter than us. We call such an AI, an AGI, artificial general intelligence, when we can say that the level at which we can teach the AI to do anything that, for example, I can do, or someone else.

这就是为什么我认为那是一个值得我全力以赴的绝佳领域。所以现在，让我们回到我们的人工智能，这些数字大脑。今天，这些数字大脑远不如我们的生物大脑聪明。当你和一个AI聊天机器人交谈时，你会很快发现它并不完全在那里，它知道的大多数事情，有点儿，但你可以清楚地看到它不能做的很多事情，以及一些奇怪的空白。但我声称，这种情况是暂时的。随着研究人员和工程师继续研究AI，总有一天，存在于我们计算机中的数字大脑将变得和我们自己的生物大脑一样好，甚至更好。计算机将变得比我们更聪明。我们称这样的AI为AGI，人工通用智能，当我们可以说我们能教AI做任何事情，比如，我能做的，或其他人能做的。

So although AGI does not exist today, we can still gain a little bit of an insight into the impact of AGI once it's built. It is completely obvious that such an AGI will have a dramatic impact on every area of life, of human activity and society. And I want to go over a quick case study. This is a narrow example of a very, very broad technology. The example I want to present is healthcare. Many, many of you may have had the experience of trying to go to a doctor. You need to wait for many months sometimes, and then when you do get to see a doctor, you get a small, very limited amount of time with the doctor. And furthermore, the doctor, being only human, can have only limited knowledge of all this, all the medical knowledge that exists. And then by the end of it, you get a very large bill. Well, if you have an intelligent computer, an AGI, that is built to be a doctor, it will have complete and

尽管如今通用人工智能（AGI）尚未存在，我们仍然可以稍微洞察一下一旦构建出AGI，它将对我们的影响。显而易见，这样的AGI将对生活的每一个领域、人类活动和社会产生巨大的影响。我想快速地讲一个案例研究。这是一个非常狭窄的例子，用来说明一项非常广泛的技术。我想要呈现的例子是医疗保健。你们中的许多人可能有过尝试去看医生的经历。有时你需要等待好几个月，等你真的能见到医生时，你与医生的交流时间又少得可怜。而且，医生毕竟也是人，他们的知识有限，不可能完全掌握所有存在的医学知识。到最后，你还得支付一笔很大的账单。好吧，如果你有一个智能计算机，一个被设计成医生的AGI，它将拥有完整和

exhaustive knowledge of all medical literature. It will have billions of hours of clinical experience, and it will be always available and extremely cheap. When this happens, we will look back at today's healthcare similarly to how we look at 16th-century dentistry, when, you know, when they tie people with belts and then have this drill. That's how today's healthcare will look like. And again, to emphasize, this is just one example. AGI will have dramatic and incredible impact on every single area of human activity.

对所有医学文献的详尽知识。它将拥有数十亿小时的临床经验，并且始终可用且极为便宜。当这一切发生时，我们会回顾今天的医疗保健，就像我们看待16世纪的牙科一样，你知道的，那时候他们用皮带绑住人，然后使用这种钻头。今天的医疗保健将会是这样的。再次强调，这只是一个例子。AGI将对人类活动的每一个领域产生巨大而不可思议的影响。

But when you see impact this large, you may wonder, gosh, isn't this technology too impactful? And indeed, for every positive application of AGI, there will be a negative application as well. This technology is also going to be different from technologies that we are used to because it will have the ability to improve itself. It is possible to build an AGI that will work on the next generation of AGI. The closest analog we had to this kind of rapid technological improvements was when the Industrial Revolution has taken place, where human, the material condition of human society was very, very constant, and then it was a rapid increase, rapid growth. With AGI, something like this could happen again, but on a shorter time scale. And then furthermore, there are concerns around if an AGI ever becomes very, very powerful, which is possible, maybe it will want to go rogue, being that it is an agent. So, this is a concern that exists with this unprecedented, not yet existing technology. And indeed, you look at all the positive potential of AGI and all the concerning possibilities of AGI as well, and you may say, gosh, like, where is this all headed?

但当你看到这么大的影响时，你可能会想，天哪，这项技术是不是太有影响力了？的确，对于人工通用智能（AGI）的每一个积极应用，也会有一个负面应用。这项技术与我们习惯的技术也会有所不同，因为它将具有自我改进的能力。有可能构建一个AGI，它将致力于下一代AGI的开发。我们所经历的与这种快速技术进步最相似的是工业革命时期，那时人类社会的物质条件非常非常稳定，然后迅速增长，快速发展。有了AGI，类似的事情可能再次发生，但时间尺度会更短。而且，还有人担心，如果AGI变得非常非常强大，这是有可能的，它可能会想要变得不受控制，因为它是一个代理体。所以，这是一个关于这项前所未有、尚未存在的技术的担忧。的确，你看到了AGI所有的积极潜力和所有令人担忧的可能性，你可能会说，天哪，这一切到底是怎么了？

One of my motivations in creating OpenAI was, in addition to developing this technology, was also to address the questions that are posed by AGI, the difficult questions, the concerns that we raised. In addition to working with governments and helping them understand what is coming and prepare for it, we also doing a lot of research on addressing the technological side of things, so that the AI will never want to go rogue. And this is something which I'm working on as well. But I think the thing to note, because AI and AGI is really the only area of the economy where there is a lot of excitement, a lot of investment, everyone is working on it. There's a huge number of labs in the world trying to build the same thing. Even if OpenAI takes these desirable steps that I mentioned, what about the rest of the companies and the rest of the world? And this is where I want to make my observation about the force that exists. And this observation is this: consider the world one year ago. As recently as one year ago, people weren't really talking about AI, not in the same way at all. What happened? We all experienced what it's like to talk to a computer and to be understood. The idea that computers will become really intelligent and eventually more intelligent than us is becoming widespread. It used to be a niche idea that only a few enthusiasts and hobbyists and people who were very into AI were thinking about. But now, everyone is thinking about it. And as AI continues to make progress, as technology continues to advance, as more and more people see what AI can do and where it is headed towards, then it will become clear just how dramatic, incredible, and almost fantastical AGI is going to be, and how much trepidation is appropriate. And what I claim will happen is that people will start to act in unprecedentedly collaborative ways out of their own self-interest. It's already happening right now. You see the leading AGI companies starting to collaborate, for a specific example, through the Frontiers model forum. And we will expect that companies that are competitors will share technical information to make their AI safe. We may even see governments do this.

我创建OpenAI的动机之一，除了开发这项技术外，还是要解决由AGI提出的问题，那些困难的问题，我们提出的担忧。除了与政府合作，帮助他们理解即将到来的事物并为之做准备外，我们还在做大量研究，解决技术层面的问题，以确保AI永远不会变得失控。这也是我正在努力的事情。但我认为需要注意的是，因为AI和AGI实际上是经济中唯一一个充满激情和投资的领域，每个人都在研究它。世界上有大量实验室试图构建相同的东西。即使OpenAI采取了我提到的这些理想步骤，其他公司和世界其他地方又如何呢？这就是我想要表达的关于存在的力量的观察。这个观察是这样的：想象一下一年前的世界。就在一年前，人们并没有真正谈论AI，至少不是以同样的方式。发生了什么？我们都经历了与计算机对话并被理解的感觉。 计算机将变得非常智能，最终比我们更智能的想法正在变得普遍。这曾是一个小众观点，只有少数热情者、业余爱好者和非常热衷于人工智能的人才会考虑。但现在，每个人都在思考这个问题。随着人工智能继续取得进展，技术不断发展，越来越多的人看到人工智能能做什么以及它的发展方向，那么通用人工智能将会有多么戏剧性、不可思议和几乎是幻想般的，以及适当的恐惧感将变得多么明显。我所声称的将会发生的是，人们将出于自身利益，以前所未有的协作方式开始行动。这已经在发生了。你看到领先的通用人工智能公司开始通过前沿模型论坛这样的特定例子进行合作。我们预计，即使是竞争对手的公司也会分享技术信息以确保他们的人工智能安全。我们甚至可能看到政府这样做。

For another example, at OpenAI, we really believed in how dramatic AGI is going to be. So, one of the ideas that we were operating by, and it's been written on our website for 5 years now, is that when technology gets such that we are very, very close to AGI, to computers smarter than humans, and if some other company is far ahead of us, then rather than compete with them, we will help them out, join them, in a sense. And why do that? Because we feel we appreciate how incredibly dramatic AGI is going to be. And my claim is that with each generation of capability advancements, as AI gets better, and as all of you experience what AI can do, as people who run AI efforts and AGI efforts, and people who work on them will experience it as well, this will change the way we see AI and AGI, and that will change collective behavior. And this is an important reason why I'm hopeful that despite the great challenges that's posed by this technology, we will overcome them. Thank you.

再举一个例子，在OpenAI，我们真的相信AGI将会带来多么巨大的变化。因此，我们运作的一个理念，现在已经在我们的网站上写了5年，就是当技术发展到我们非常非常接近AGI，接近比人类更聪明的计算机时，如果其他公司遥遥领先于我们，那么我们不会与他们竞争，相反，我们会帮助他们，从某种意义上加入他们。为什么要这么做呢？因为我们感觉到AGI将会带来多么不可思议的巨变。我的观点是，随着每一代能力的提升，随着AI变得更好，随着你们所有人体验到AI能做什么，作为运营AI和AGI项目的人，以及在这些项目上工作的人也会有所体验，这将改变我们看待AI和AGI的方式，进而改变集体行为。这也是我为什么对于尽管这项技术带来了巨大挑战，我们仍将克服它们而感到希望的一个重要原因。谢谢。