Sean Carroll

Like people are not humble enough about the fields they're not expert in.

And that's not only true for people on the streets who get their ideas from podcasts.

It's true for the best intellectuals we have in the world also.

So that is a feature of the modern world that I think we could certainly do a lot better at.

OK, I'm going to group a couple questions together.

Rob says, has your credence that advanced AI could become an existential risk to humanity increased in recent years?

I'm especially curious whether developments in autonomous AI agents, including systems that can coordinate complex tasks and even hire humans online, have updated your view meaningfully.

John Plasterer says, you've developed some well-formed opinions on the limitations of language models and articulated them on your show.

With the recent result from OpenAI disproving a well-known central conjecture in discrete geometry, does this change your view on language models directly contributing to new physics?

If not, what would change your opinion?

So both about the capabilities of LLMs and other AI models and things like that.

So for John's question first, the recent discovery or proof, I guess, of a conjecture by an LLM, I don't even know which one this is referring to because there's been several of them.

There's been impressive work with using LLMs to be able to either disprove or prove mathematical conjectures.

Actually, to me, this is not very surprising at all.

And I'd like to emphasize I have been surprised.

I'm overall very surprised at the improvement in LLMs from 10 years ago to today by a lot.

They're much more capable in many ways than I thought they would be.

But given where they are now, the fact that they can prove or disprove well-formed mathematical conjectures isn't that surprising to me.

Just like I was not surprised when AI became good at chess or Go or protein folding.

Those are, to me, exactly what computers should be good at.