Sholto Douglas

Maybe this is one of the things that the people who were here working on AI research and I think each of the companies is trying to define this for themselves, but it's actually something that broader society can participate in.

If you take as premise that in a few years we're going to have something that's human level intelligence and you want to imbue that with a certain set of values.

What should those values be is a question that everyone should be participating in and offering a perspective on.

Like in the constitutionally high paper, it's not just flourishing.

It's like there's a lot of strictures and there's a lot of like dot points there.

Yeah.

But it's not an easy question.

I think at the beginning, the hill to climb.

So the reason why people hill climbed Hendrix Math for so long was that there's five levels of problem.

And it starts off reasonably easy.

And so you can both get some initial signal of, are you improving?

And then you have this quite continuous signal, which is important.

Something like Frontier Math is actually

only makes sense to introduce after you've got something like Hendrix math, that you can max out Hendrix math, and they go, OK, now it's time for frontier math.

I think in a lot of these cases, you have to hope for some amount of generator-verifier gap.

You need it to be easier to judge, did you just output a million extraneous files than it is to...

generate solutions in and of itself.

That needs to be a very easy to verify thing.

So Sloppy's hot.

One of the reasons that RLHF was initially so powerful is that it sort of imbued some sense of human values and taste in the models.