Rob Wiblin

Companies basically, they mostly don't think that this sort of collusion is an issue yet.

So they're not doing that kind of testing at scale, not so far.

But in future, they could start if collusion became a live concern.

Anyway, what was the red teamers bottom line on opportunity?

In short, anthropics monitoring isn't sufficient to have a high probability of picking up a deeply misaligned Claude taking rogue actions in any specific case.

But to get anything meaningful done, you have to take a lot of actions.

And soon enough, the model would get busted.

David estimates within a few days or at most weeks.

Before it was caught, the model might be able to take some small early steps like inserting exploits or vulnerabilities that make larger sabotage actions easier later on.

But once it was caught, these would, for the most part, be identified and rolled back.

So there's a window of opportunity, but it's not a wide open window.

And just to be clear, you shouldn't read this as suggesting Anthropic is somehow uniquely vulnerable here.

They're just the first ones to invite an external analyst in to stress test their systems.

Okay, that's opportunity.

How about means?

Are the models good enough to pull off an attack like this?

Well, the models are good enough to grab roughly the amount of compute the human user can access, set up machines to run agents using that compute, and hide that from casual user oversight.

That's why the report concludes that internal agents have the means to start minimal rogue deployments at all companies.

But how about expanding those deployments, spreading across computers, avoiding detection, or even evading shutdown once detected?

I would have thought AR models would find that kind of thing pretty easy.