Andrej Karpathy

Yeah.

I think it's just like the hardware for long-term memory is just not fully developed.

Sure.

I kind of feel like the first few years of, uh, uh, of infants is not actually like learning.

It's brain maturing.

Yeah.

Um, we're born premature.

Um, there's a theory along those lines because of the birth canal and the swelling of the brain.

And so we're born premature.

And then the first few years were just the brains maturing.

Uh, and then there's some learning eventually.

Um,

That's my current view on it.

Yes, but I don't know to what extent it will be explicitly constructed.

It might take unintuitive forms where you are telling the GPT, like, hey, you have a declarative memory bank to which you can store and retrieve data from, and whenever you encounter some information that you find useful, just save it to your memory bank.

And here's an example of something you have retrieved, and here's how you say it, and here's how you load from it.

You just say, load...

whatever, you teach it in text, in English, and then it might learn to use a memory bank from that.

It's not just text, right?

You're giving it gadgets and gizmos.