Roman Yampolsky
speaker
257 appearances
1 recordings
1 series
first heard Jun 2024
last heard Jun 2024
Roman Yampolsky’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
I cannot make a case that he's right. He's wrong in so many ways, it's difficult for me to remember all of them. He's a Facebook buddy, so I have a lot of fun having those little debates with him. So I'm trying to remember the arguments. So one, he says we are not... gifted this intelligence from aliens. We are designing it, we are making decisions about it. That's not true.
It was true when we had expert systems, symbolic AI, decision trees. Today, you set up parameters for a model and you water this plant. You give it data, you give it compute, and it grows. And after it's finished growing into this alien plant, you start testing it to find out what capabilities it has. And it takes years to figure out, even for existing models.
If it's trained for six months, it will take you two, three years to figure out basic capabilities of that system. We still discover new capabilities in systems which are already out there. So that's not the case.
Absolutely. That's what makes it so successful. Then we had to painstakingly hard code in everything. We didn't have much progress. Now, just spend more money and more compute and it's a lot more capable.
Let's say there is a ceiling. It's not guaranteed to be at the level which is competitive with us. It may be greatly superior to ours.
Historically, he's completely right. Open source software is wonderful. It's tested by the community. It's debugged, but we're switching from tools to agents. Now you're giving open source weapons to psychopaths. Do we want to open source nuclear weapons? biological weapons.
It's not safe to give technology so powerful to those who may misalign it, even if you are successful at somehow getting it to work in the first place in a friendly manner.
It also sets a very wrong precedent. So we open sourced model one, model two, model three, nothing ever bad happened. So obviously we're gonna do it with model four. It's just gradual improvement.
So I have a paper which collects accidents through history of AI, and they always are proportional to capabilities of that system. So if you have tic-tac-toe playing AI, it will fail to properly play and loses the game which it should draw. Trivial. Your spell checker will misspell a word, so on.
I stopped collecting those because there are just too many examples of AIs failing at what they are capable of. We haven't had... terrible accidents in the sense of billion people get killed. Absolutely true. But in another paper, I argue that those accidents do not actually prevent people from continuing with research. And actually, they kind of serve like vaccines.
A vaccine makes your body a little bit sick, so you can handle the big disease later much better. It's the same here. People will point out, you know that accident, AI accident we had where 12 people died? Everyone's still here. 12 people is less than smoking kills. It's not a big deal. So we continue. So in a way, it will actually be kind of confirming that it's not that bad.
So you bring up example of cars. Yes, cars were slowly developed and integrated. If we had no cars, and somebody came around and said, I invented this thing. It's called cars. It's awesome. It kills like 100,000 Americans every year. Let's deploy it. Would we deploy that?
You need data. You need to know. But if I'm right and it's unpredictable, unexplainable, uncontrollable, you cannot make this decision, we're gaining $10 trillion of wealth, but we're losing, we don't know how many people. You basically have to perform an experiment on 8 billion humans without their consent.
And even if they want to give you consent, they can't because they cannot give informed consent. They don't understand those things.
We're literally doing it. The previous model we learned about after we finished training it, what it was capable of. Let's say we stop GPT-4 training run around human capability, hypothetically. We start training GPT-5, and I have no knowledge of insider training runs or anything. And we start at that point of about human, and we train it for the next nine months.
Maybe two months in, it becomes super intelligent. We continue training it. At the time when we start testing it, It is already a dangerous system. How dangerous? I have no idea. But neither people training it.
If we had capability of ahead of the run, before the training run, to register exactly what capabilities that next model will have at the end of the training run, and we accurately guessed all of them, I would say, you're right, we can definitely go ahead with this run. We don't have that capability.
We're not talking just about capabilities, specific tasks. We're talking about general capability to learn. Maybe like a child at the time of testing and deployment, it is still not extremely capable, but as it is exposed to more data, real world, it can be trained to become much more dangerous and capable.
So I think at some point it becomes capable of getting out of control. For game theoretic reasons, it may decide not to do anything right away and for a long time just collect more resources, accumulate strategic advantage. Right away, it may be kind of still young, weak superintelligence. Give it a decade, it's in charge of a lot more resources. It had time to make backups.
So it's not obvious to me that it will strike as soon as it can.
Showing 61–80 of 257 · page 4 of 13
← Previous
Next →