Jeff Dean

Yep.

And then it's AI complete.

If you can do a great job of that, then you can pretty much do anything.

There were definitely some... There was this neural Turing machine.

So...

Yeah, a bunch of ideas around this attention slash there's like having these key value stores that could be useful in neural networks to kind of focus on things.

So, yeah, I think in some sense in the air and in some sense, you know, you need some group to go do it.

I don't know what you... I am also excited to see where we go.

And, you know, yeah, I think definitely the organizing information, you know, is clearly like a, you know, trillion-dollar opportunity.

But, you know, a trillion dollars is not cool anymore.

What's cool is a quadrillion dollars.

I mean, and obviously the...

The idea is not to just pile up some giant pile of money, but it's to create value in the world.

And so much more value can be created when these systems can actually go and do something for you, write your code, or figure out problems that you wouldn't have been able to figure out yourself and to do that at scale.

we're going to have to be very, very flexible and dynamic as we improve the capabilities of these models.

Yeah.

I mean...

Yeah, the beautiful thing about model parameters is they are

quite memory efficient at sort of memorizing facts.

Maybe you can probably memorize order of one fact or something per model parameter, whereas if you have some token in context, there are lots of keys and values at every layer.