Grant Harvey

Yeah.

So Ilya, when you published that in 2017, did you and the team anticipate that Transformers would become the foundation of everything today?

And what do you think about where we are now?

Like looking at it from the perspective of 2017 to today, do you think we're in the early innings or are we approaching some kind of inflection point?

What's your take?

We are.

Yeah.

I mean, that begs the question.

I mean, you know, there's a debate about whether or not we're hitting limits on scaling laws.

Personally, I've been tracking the switch to everyone spinning up RL environments for everything now.

But I've also seen compelling evidence that, you know, maybe there's new architectures that we need at some level.

I really don't know anything about that compared to you, I'm sure, in that space.

But...

Um, do you think that there could be, do you think we need some sort of new architecture or from your perspective of someone who helped create this foundation?

I mean, what's the bottleneck from where we are now to where we go from here, like to superhuman AI?

Yes.

Yes.

Literally.

And if you could solve it and if you could solve it on this podcast, that would be great.

Because the AI is basically doing the same process as the researchers in that case, where it's going out and it's trying to try a million different things until it works, right?