Scott Alexander

kind of do that but unreliably and that if you actually like tried to use that to run your life it would make some hilarious mistakes that would appear on Twitter and go viral but that like the MVP of it will probably exist by this year like there'll be like some Twitter thread about someone being like I plugged in this agent to like run my

Yeah, I think in general, most people following the field have underestimated the pace of AI progress and underestimated the pace of AI diffusion into the world.

For example, Robin Hanson famously made a bet about less than a billion dollars of revenue, I think, by 2025.

But he's a smart guy, you know?

So I think that the aggregate opinion has been underestimating the pace of both technical progress and deployment.

I agree that there have been plenty of people who have been more bullish than me and have been already proven wrong.

But they're not being... Wait a second.

Yeah, that is interesting.

I imagine what's going on there is that a lot of the process when you're unfamiliar with a domain is like Googling around and learning more about the domain and language models are excellent because they've already read the whole internet and know all the details.

I'll add some more things to that.

So I think there's a long and sordid history of people looking at some limitation of the current LLMs and then making grand claims about how the whole paradigm is doomed because they'll never overcome this limitation.

And then like a year or two later, the new LLMs overcome that limitation.

And I would say that like,

With respect to this thing of why haven't they made these interesting scientific discoveries by combining the knowledge they already have and noticing interesting connections, I would say, first of all, have we seriously tried to build scaffolding to make them do this?

And I think the answer is mostly no.

I think Google DeepMind tried this, right?

Maybe.

So maybe.

Second thing, have you tried making the model bigger?

They've made it a bit bigger over the last couple years, and it hasn't worked so far.