Dr. Suelet Dreyfus

So Anthropic is a company and it manufactures, if you will, not widgets, but rather AI systems.

Its selling point is that the systems are safe and insteerable, that is, it does what humans actually intend, or so it says.

So this new model is a new large language model or LLM.

And what it allegedly can do is autonomously identify and exploit computer security vulnerabilities across every major operating system.

So your Mac, your Windows, your Linux, and every major web browser.

Now, autonomously means that it's able, allegedly, to do this on its own.

It doesn't have someone leaning over going, break that or poke this in order to get in.

And so the reason it's considered to be such a serious shift in the threat landscape is that usually that sort of attack requires...

humans with a rare and highly skilled background, and there are not that many of them.

This would do it at scale, and someone who was not particularly skilled, not an engineer, could do it at scale.

That's the risk.

Yeah, it's not just financial systems and markets, it's all our SCADA systems.

So if you think about the fact that you could have this AI go rogue and break into, without authorization, the systems that control the water flow out of dams, the systems that control the electricity systems for all our power being on or our gas flowing.

Or it could affect financial markets.

Let's say a set of these AIs come together and collude to manipulate the market, to go to all short a particular set of companies at a particular time.

And all of those things have the opportunity to really disrupt the systems that we depend on in life.

So the risk here is that the human, even a badly intentioned human who might do bad things, for example, extortion, blackmail, if you don't do this, I'll let it loose on your electricity system for the city of Melbourne.

It's actually potentially worse than that because if they go rogue, even the bad actor may no longer be able to control them.

That's why there's so much concern among the institutions about what the risk is here.

Yeah, I mean, it could be.