Ilya Sutskever

In terms of utility, though, I think there is a thing where, you know, there is this complexity, robustness trade off, where complex things can be very useful,

But simple things are very useful in a very broad range of situations.

And so I think one way to interpret what we are seeing is that we've got these emotions that essentially evolved mostly from our mammal ancestors and then fine-tuned a little bit while we were hominids, just a bit.

We do have like a decent amount of social emotions though, which mammals may lack.

but they're not very sophisticated.

And because they're not sophisticated, they serve us so well in this very different world compared to the one that we've been living in.

Actually, they also make mistakes.

For example, our emotions, well, I don't know, does hunger count as an emotion?

It's debatable, but I think, for example, our intuitive feeling of hunger

is not succeeding in guiding us correctly in this world with an abundance of food.

So... The thing... So here is a perspective.

Here's a perspective that I think might be true.

So...

The way ML used to work is that people would just tinker with stuff and try to get interesting results.

That's what's been going on in the past.

Then the scaling insight arrived, right?

Scaling laws, GPT-3, and suddenly everyone realized we should scale.

And this is an example of how language affects thought.

Scaling is just one word, but it's such a powerful word because it informs people what to do.

They say, okay, let's try to scale things.