Andrej Karpathy

Yeah, and just stuff to work with.

I mean, I'm guessing it would be harder to, if I was a dolphin, I mean, how do you do, you can't have fire, for example, and stuff like that.

I mean, probably like the universe of things you can do in water, like inside water, is probably lower than what you can do on land.

just chemically.

Yeah, I do agree with this viewpoint of these niches and what's being incentivized.

I still find it kind of miraculous that I don't, I would have maybe expected things to get stuck on like animals with bigger muscles, you know?

Yeah.

Like going through intelligence is actually a really fascinating breaking point.

Yeah, exactly.

You have to incentivize some kind of adaptability.

You actually want environments that are unpredictable.

So evolution can't bake your algorithms into your weights.

A lot of animals are basically pre-baked in this sense.

And so humans have to figure it out at test time when they get born.

And so maybe you actually want these kinds of environments that actually change really rapidly or something like that where you can't foresee what will work well.

And so you actually put all that intelligence, you create intelligence to figure it out at test time.

Yes and no, because LLMs don't really have the equivalent of culture.

And maybe we're giving them way too much and incentivizing not to create it or something like that.

But I guess like the mention of culture and of written record and of like passing down notes between each other, I don't think there's an equivalent of that with LLMs right now.

So LLMs don't really have culture right now.