Sholto Douglas

OK, I've raised all this money.

Do I spend it along this axis, or do I spend it on this axis?

And currently, the companies are spending more on compute than they are on humans.

Otherwise, scale AI's revenue would be like $10 billion.

Look at it.

NVIDIA's revenue is much higher than scale AI's revenue.

And so currently, the equation is compute over data.

Like, that will evolve in some way over time, but... Yeah, interesting.

But even in light of all of this- The language result is really cool.

You should talk about the language result.

You know, how smaller models have separate neurons for different languages, whereas larger models end up sharing more and more like an abstract space.

But, like, strikingly, that is more so the case in larger models, where you'd think, like, actually larger models have more space, so they could, like, separate things out more.

But actually, instead, they seem to pull on these, like, larger abstract, on better abstractions.

Yeah.

Which is very interesting.

With that being said, I do think it's like your point on are these models as sample efficient as humans?

Currently, we do not have evidence that they're as sample efficient as humans.

I think we have evidence of like total complexity ceiling.

Like there are currently nothing that provides you have a clean enough signal you can't teach them.

But we don't have evidence that like we can teach them as fast as humans do.