Trenton Bricken

Maybe three.

So one, like, in the alpha fold paper, one of the transformer modules, they have a few, and the architecture is, like, very intricate.

But they do, I think, five forwards passes through it.

and will gradually like refine their solution as a result.

You can also kind of think of the residual stream.

I mean, Shelter alluded to kind of the read write operations as like a poor man's adaptive compute, where it's like, I'm just gonna give you all these layers.

And like, if you wanna use them, great.

If you don't, then that's also fine.

And then people will be like, oh, well the brain is recurrent and you can like do however many loops through it

And I think to a certain extent, that's right.

Like if I ask you a hard question, you'll spend more time thinking about it.

And that would correspond to more forward passes.

But I think there's a finite number of forward passes that you can do.

It's kind of with language as well.

People are like, oh, well, human language can have like infinite recursion in it, like infinite nested statements.

Of like the boy jumped over the bear that was doing this, that had done this, that had done that.

But like empirically, you'll only see five to seven levels of recursion, which kind of relates to whatever that magic number of like how many things you can hold in working memory at any given time is.

And so.

Yeah, it's not infinitely recursive, but like, does that matter in the regime of human intelligence?

And like, can you not just add more layers?