Nathaniel Whittemore

This is cause for optimism.

The incentives of capitalism are working.

The training wheels are coming off, but at least we are the ones removing them, as opposed to our enemies.

Perhaps we can be the first to learn to bike for real.

The first step would be to get beyond all the low-fidelity, underspecified, pimply little fights of AI policy's prepubescent era.

That goes for me, too.

What hath God wrought, wrote the first telegram.

What, indeed.

In this case, the answer is still up to us.

I think one of the things that's important to remember is that we are living in the world of double-edged swords.

The same capabilities that theoretically make this model incredibly powerful for exploiting cyber vulnerabilities are also the most powerful tool that security professionals have ever had.

AI security researcher Nicholas Carlini said, I've found more bugs in the last few weeks with Mythos than in the rest of my entire life combined.

As always, the gasping, incentivized, horrified, and fearful first reactions of social media, which are, of course, the ones incentivized by the social media algorithms, are, I think, much less useful than the nuance that unfortunately tends to get buried.

Daniel Jeffries points out, My best understanding is that Anthropic did not train the model to be an exploit wizard.

They trained it to be the best coder in the world.

If you're the best doctor in the world, you know lots of ways to poison people.

If you're the best coder in the world, you have the capability to be a great hacker.

But the difference is intention.

Now where Daniel differs from the anthropic team is on the right approach from here.

He writes, I do not believe in giant, centrally controlled, centrally planned approach to security or economics.