Dwarkesh

Even after AlexNet, the amount of compute that was used to run experiments kept increasing, and the size of frontier systems kept increasing.

And do you think now that this era of research will still require tremendous amounts of compute?

do you think it will require going back into the archives and reading old papers?

What is, maybe what was the vibe of like, you were at Google and OpenAI and Stanford, these places when there was like a more of a vibe of research, what kind of things should we be expecting in the community?

Yeah, I guess while it was possible to develop these ideas, I'm asking you for the history because you were actually there.

I'm not sure what actually happened.

But it sounds like it was possible to develop these ideas using minimal amounts of compute.

But the transformer didn't immediately become famous.

It became the thing everybody started doing and then started experimenting on top of and building on top of because it was validated at higher and higher levels of compute.

Correct.

Yeah.

And if you at SSI have 50 different ideas, how will you know which one is the next transformer and which one is, you know, brittle without having the kinds of compute that other frontier labs have?

There's been public estimates that companies like OpenAI spend on the order of $5-6 billion a year, just so far, on experiments.

This is separate from the amount of money they're spending on inference and so forth.

So it seems like they're spending more a year running research experiments than you guys have in total funding.

I think it's a question of what you do with it.

But then, so why is your default plan to straight shot superintelligence?

Because it sounds like, you know, OpenAI, Anthropic, all these other companies, their explicit thinking is, look, we have weaker and weaker intelligences that the public can get used to and prepare for.

And why is it potentially better to build a superintelligence directly?

Well, I guess not even just that you can communicate the idea, but... Communicate the AI, not the idea.