Roman Yampolsky
speaker
257 appearances
1 recordings
1 series
first heard Jun 2024
last heard Jun 2024
Roman Yampolsky’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
What's the timeframe?
So the problem of controlling AGI or superintelligence, in my opinion, is like a problem of creating a perpetual safety machine. By analogy with perpetual motion machine, it's impossible. Yeah, we may succeed and do a good job with GPT-5, 6, 7, but they just keep improving, learning, eventually self-modifying, interacting with the environment, interacting with malevolent actors.
The difference between cybersecurity, narrow AI safety, and safety for general AI for superintelligence is that we don't get a second chance. With cybersecurity, somebody hacks your account, what's the big deal? You get a new password, new credit card, you move on. Here, if we're talking about existential risks, you only get one chance.
So you're really asking me, what are the chances that we'll create the most complex software ever on the first try with zero bugs, and it will continue to have zero bugs for 100 years or more?
I don't think we so far have made any system safe. At the level of capability they display, they already have made mistakes. We had accidents. They've been jailbroken. I don't think there is a single large language model today which no one was successful at making do something developers didn't intend it to do.
Exactly. But the systems we have today have capability of causing X amount of damage. So when they fail, that's all we get. If we develop systems capable of impacting all of humanity, all of universe, the damage is proportionate.
That's obviously a wonderful question. So one of the chapters in my new book is about unpredictability. I argue that we cannot predict what a smarter system will do. So you're really not asking me how superintelligence will kill everyone. You're asking me how I would do it. And I think it's not that interesting. I can tell you about the standard nanotech, synthetic, bio, nuclear.
Superintelligence will come up with something completely new, completely super. We may not even recognize that as a possible path to achieve that goal.
They are limited by how imaginative we are. If you are that much smarter, that much more creative, you are capable of thinking across multiple domains, do novel research in physics and biology, you may not be limited by those tools. If squirrels were planning to kill humans, they would have a set of possible ways of doing it, but they would never consider things we can come up with.
I think about a lot of things. So there is X risk, existential risk, everyone's dead. There is S risk, suffering risks, where everyone wishes they were dead. We have also idea for I risk, ikigai risks, where we lost our meaning. The systems can be more creative. They can do all the jobs. It's not obvious what you have to contribute to a world where superintelligence exists.
Of course, you can have all the variants you mentioned where we are safe, we are kept alive, but we are not in control. We are not deciding anything. We are like animals in a zoo. Possibilities we can come up with as very smart humans, and then possibilities something a thousand times smarter can come up with for reasons we cannot comprehend.
So Japanese concept of ikigai, you find something which allows you to make money, you are good at it, and the society says we need it. So like you have this awesome job, you are a podcaster, gives you a lot of meaning, you have a good life, I assume you're happy. That's what we want most people to find, to have. For many intellectuals, it is their occupation which gives them a lot of meaning.
I am a researcher, philosopher, scholar. That means something to me. In a world where an artist is not feeling appreciated because his art is just not competitive with what is produced by machines, or a writer or scientist will lose a lot of that. And at the lower level, we're talking about complete technological unemployment. We're not losing 10% of jobs, we're losing all jobs.
What do people do with all that free time? What happens then? Everything society is built on is completely modified in one generation. It's not a slow process where we get to kind of figure out how to live that new lifestyle, but it's pretty quick.
It's an option. I have a paper where I try to solve the value alignment problem for multiple agents. And the solution to avoid compromise is to give everyone a personal virtual universe. You can do whatever you want in that world. You could be king, you could be slave, you decide what happens.
So it's basically a glorified video game where you get to enjoy yourself and someone else takes care of your needs and the substrate alignment is the only thing we need to solve. We don't have to get 8 billion humans to agree on anything.
Some people say that's what happened. We're in a simulation.
And some people choose to play on a more difficult level with more constraints. Some say, okay, I'm just going to enjoy the game, high privilege level. Absolutely.
Personal universes. Personal universes.
In order to solve value alignment problem, I'm trying to formalize it a little better. Usually, we're talking about getting AIs to do what we want, which is not well-defined. We're talking about creator of the system, owner of that AI, humanity as a whole, but we don't agree on much. There is no universally accepted ethics, morals across cultures, religions.
Showing 1–20 of 257 · page 1 of 13
Next →