Rob Wiblin

I hope we don't end up in this situation.

Like Jack, I think recursive self-improvement is too dangerous and humanity would be crazy to dive in headfirst the very moment it first becomes possible.

We should make it illegal in my view, but if we don't, I at least hope Mita has a red teamer in there trying hard to figure out what bad stuff an evil clawed mythos for would be able to get away with if it were so inclined.

And on that note, I'll speak with you again soon.

Today, I'm speaking with Joshua Bengio.

He is the scientific director at Law Zero, a Turing Award winner in 2018, the most cited computer scientist of all time, and as it happens, also the most cited scientist of any type that is still alive.

Thanks so much for coming on the show, Joshua.

Thanks for having me.

You think you found the right approach to build a safe, super-intelligent AI.

What's the approach?

So what does the new training process look like and how is it different from the models that people are familiar with?

So you'd be building a model where you would feed in a statement, and it would basically tell you what probability it assigns to that statement being true.

Yes, in context.

Yes.

Hey, listeners, Rob jumping in here.

Joshua is naturally pitching this in a way that's ideal for staff at frontier AI companies.

And they're obviously a particularly important audience for this proposal.

But I'm confident with just a few minutes of plain language explanation, everyone else will be able to follow the rest of the conversation as well.

So bear with me or skip ahead about four minutes if you feel very at home with this sort of material already.

As you probably know, in their first stage of training, today's large language models are taught to predict the word that's most likely to come next, at least the token that's most likely to come next.