Gwern Branwen

And if it wasn't written down, then it isn't written down.

The thing that I swing back and forth on the most is the relationship between human intelligence and neural network intelligence.

It's just, it's not clear in what sense they are two sides of the same coin or one is like an inferior version of the other.

This is something that I constantly go back and forth on.

One day I'll be like, humans are awesome.

And then the next time I'm like, no, neural networks are awesome.

Or no, both suck.

Or maybe I'll say both are awesome, just in different ways.

So every day I find that I'm arguing with myself a little bit about why each one is good or bad or how.

What's, you know, the whole deal there with things like GBD4 memorization, but not being creative.

Why do humans not remember anything, but we still seem to be so smart?

One day I'll argue that language models are sample efficient compared to humans.

The next day I feel like I'm arguing the opposite.

Yeah, I'm not sure that it was necessary.

When I think about others who are correct, like Shane Legge or Dario Amadai, they don't seem to be all that polymathic.

They just have broad intellectual curiosity, broad general understanding, you know, absolutely.

But I don't think they are absurdly polymathic.

You know, clearly you could get to the correct view without being polymathic.

That's just how I happened to come to it at this point and the connection that I'm kind of like making post-hoc.

it wasn't like I was using primatology to kind of justify scaling to myself, right?