Sholto Douglas

the thing that connects the two halves of the brain.

And then, yeah, the speech half is on the left side, so it's not connected to the part that decides to do a movement.

And so if the other side decides to do something, the speech part will just make something up, and the person will think that's legit the reason they did it.

What will this landscape of models communicating to themselves in ways we don't understand, how does that change with AI agents?

Because then these things will, it's not just like the model itself with its previous caches, but like other instances of the model.

How much more effective do you think the models would be if they could share the residual streams versus just text?

Hard to know.

So for the audience, you would project the residual stream into this larger space where we know what each dimension actually corresponds to and then back into the next agents or whatever.

So your claim is that we'll get AI agents when these things can...

are more reliable and so forth.

When that happens, do you expect that it will be multiple copies of models talking to each other?

Or will it be just an adaptive computer solved and the thing just like runs bigger, like more compute when it needs to do a kind of thing that a whole firm needs to do?

And I ask this because there's two things that make me wonder about like whether agents is the right way to think about what will happen in the future.

One is with longer contexts,

these models are able to ingest and consider the information that no human can.

And therefore we need like one engineer who's thinking about the front-end code and one engineer who's thinking about the back-end code, where this thing can just ingest the whole thing.

This sort of like Hayekian problem of specialization goes away.

Second, these models are just very general.

You're not using different types of GPT-4 to do different kinds of things.

You're using the exact same model, right?