Daniel Kokotajlo

What I found really surprising is that the help is bigger where it's less like autocomplete and more like a novel contribution.

It's like a more significant productivity improvement there.

Isn't this a good opportunity to discuss a certain question I asked Dario that you responded to?

What are you thinking of?

Well, I asked this question where, as you say, they know all this stuff.

I don't know if you saw this.

I asked this question where I said, look, these models know all this stuff.

And if a human knew every single thing a human has ever written down on the Internet, they'd be able to make all these interesting connections between different ideas and maybe even find medical cures or scientific discoveries as a result.

there was some guy who noticed that magnesium deficiency causes something in the brain that is similar to what happens when you get a migraine.

And so he just said, give you magnesium supplements that cured a lot of migraines.

So why aren't LLMs able to leverage this enormous asymmetric advantage they have to make a single new discovery like this?

I will actually disagree with this.

We know that humans can do... We have examples of humans doing this.

I agree that we don't have logical omniscience because there is a combinatorial explosion.

But we are able to leverage our intelligence to... Actually, one of my favorite examples of this is David Anthony, the guy who wrote The Horse, The Wheel, and Language.

He made this...

It was a super impressive discovery before we had the genetic evidence for it, like a decade before, where he said, look, if I look at all these languages in India and Europe, they all share the same etymology.

I mean, literally what you're talking about, the same etymology for words like wheel and cart and horse.

And these are technologies that have only been around for the last 6,000 years, which must mean that there was some group that—

these groups are all at least linguistically descended from.