Dwarkesh Patel

And Ben Thompson writes, quote,

If nuclear weapons were developed by a private company, the U.S.

would absolutely be incentivized to destroy that company.

And honestly, safety-aligned people have made a similar point.

Leopold Aschenbrenner, who is a former guest and, full disclosure, a good friend, wrote in his 2014 memo, Situational Awareness, quote, I find it an insane proposition that the U.S.

government will let a random SF startup develop superintelligence.

Imagine if we had developed atomic bombs by letting Uber just improvise.

And my response to Leopold's argument at the time and Ben's argument now is, while they're right that it's crazy that we're entrusting private companies with the development of this world historical technology, I just don't think it's an improvement to give that authority to the government.

Nobody's qualified to be the stewards of superintelligence.

It's a terrifying, unprecedented thing that our species is doing right now.

The fact that private companies aren't the ideal institutions to deal with this does not mean that the Pentagon or the White House is.

Yes, if a single private company were the only entity capable of building nuclear weapons, the government would not tolerate it having a veto power over how those weapons are used.

But I think this is a terrible analogy for the current situation with AI for at least two important reasons.

First, AI is not some self-contained weapon like a nuclear bomb, which only does one thing.

Rather, it is more like the process of industrialization itself.

which is a general-purpose transformation of the whole economy with thousands of applications across every single sector.

If you applied Ben Thompson or Leopold Lash and Brenner's logic to the Industrial Revolution, which is also world-historically important, it would imply the government had the right to requisition any factory it wanted or destroy any business it wanted and punish and coerce anybody who refused to comply.

But this is just not how free societies handle the process of industrialization.

And it's also not how they should handle AI.

Now, people will say, well, AI will develop unprecedentedly powerful superweapons, superhuman hackers, superhuman bioweapons researchers.