Gwern Branwen

I think otherwise there's kind of, you know, there's no general master algorithm and there's no special intelligence fluid.

It's just a tremendous number of special cases that we learn and then encode into our brains.

When we talk about more or less intelligence, it's just that they have more compute in order to do search over more Turing machines for longer.

I don't think there's like anything else other than that.

So from any learned brain, you could extract small solutions to specific problems, but because all the large brain is doing with the compute is finding it.

And that's why you never kind of are going to find any IQ gland.

There's nowhere in the brain where if you hit it, you eliminate fluid intelligence.

Mm-hmm.

I just think that, you know, it'll turn out that this doesn't exist because what your brain is doing is a lot of learning individual specialized problems.

And then once those individual problems are learned, then they get recombined for fluid intelligence.

And that's just, you know, like intelligence.

typically with a large neural network model, you can always pull out kind of a small model, which does a specific task equally well, because that's all the large model is, right?

It's just a gigantic ensemble of small models tailored to the ever-escalating number of tiny problems that you have been feeding them.

Not really.

I would actually just say that it helps explain why human-level intelligence isn't such a great idea and so rare to evolve.

Because any small Turing machine could always be encoded more directly by your genes with sufficient evolution.

You have these organisms where their entire neural network is just hard-coded by the genes.

So if you could do that, obviously that's way better than some sort of colossally expensive, unreliable, glitchy search process like what humans implement, right?

Which takes whole days in some cases to learn.

Whereas, you know, it could be hardwired in right from birth.