Sam Schechner
Appearances
Apple News Today
Inside the unlikely alliance to make childbirth free in the U.S.
Oftentimes, they keep their keys offline. They keep physical wallets, the sort of crypto equivalent of putting your money under your mattress, but with a little bit more security behind it. And so that basically means that if you want to steal that person's crypto, you have to do it in person.
Apple News Today
Inside the unlikely alliance to make childbirth free in the U.S.
I think for a long time, people... thought well the danger here is you know maybe some identity fraud at the highest end which is is pretty scary or or possibly just you know i'm gonna get a lot of spam or something like that if the personal information of big holders of crypto leaks then that can obviously we're learning now can have really dramatic real world consequences
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
In the U.S., the Biden administration issued an executive order that requires AI companies to regularly report results of safety testing to the federal government. But President-elect Donald Trump has promised to repeal the order. There are also a handful of third-party testing labs and AI safety institutes in several countries, including the U.S. and the U.K.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
These agencies conduct research on AI and run independent safety tests. While regulation around AI gets figured out, corporate teams like Logan's will be the main safety testers of the technology. And like right now, are you scared of the future? Are you scared of the models? You feel like they're performing well?
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
That's all for today, Tuesday, January 14th. The Journal is a co-production of Spotify and The Wall Street Journal. Additional reporting in this episode by Deepa Sitharaman. Thanks for listening. See you tomorrow.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
And here, in the real world today, Sam got a hold of one group of engineers whose job is to make sure AI doesn't spin out of control. Welcome to The Journal, our show about money, business, and power. I'm Kate Leinbaugh. It's Tuesday, January 14th. Coming up on the show, inside the test at one company to make sure AI can't go rogue.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
Or maybe that Marvel movie, where AI tries to destroy the Avengers.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
Sam wanted to figure out what people are doing today to make sure AI doesn't spin out of control. Right now, there's not a lot of government rules or universal standards. It's mostly on companies to self-regulate. So that's where Sam turned. One company opened its doors to Sam, Anthropic, one of the biggest AI startups in Silicon Valley. It's backed by Amazon.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
Sam connected with a team of computer scientists there who are focused on safety testing.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
Sam put me in touch with Logan and I called him up. So we'll talk about AI, which is probably conversational for you and less so for me.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
So if you could introduce yourself and tell us who you are and what you do.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
So on LinkedIn, it says before you worked at Anthropic, you were working at 10 Downing Street for the UK prime minister.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
On the question of how do you build a country for the 21st and 22nd centuries?
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
Do you have an answer to that question?
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
Or maybe it's the Matrix, where humans have become enslaved by machines.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
So Logan isn't intimidated by big ideas. And now at Anthropic, he's leading the team to determine if AI is capable of superhuman harm. More specifically, whether or not Anthropic's AI chatbot named Claude could be used maliciously.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
So you're like role-playing as a bad person.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
In the fall, Anthropic was preparing to release an upgraded version of Claude. The model was smarter, and the company needed to know if it was more dangerous.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
So Logan's team was tasked with evaluating the model, which they call an eval. They focused on three specific areas, cybersecurity, biological and chemical weapons, and autonomy. Could the AI think on its own? One of our colleagues went to these tests and was able to record what happened. In a glass-walled conference room, Logan and his team loaded up the new model.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
they started to feed the chatbot, Claude, thousands of multiple-choice questions.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
When you hear the words AI apocalypse, often movies come to mind. Maybe films like The Terminator, where AI robots ignite a nuclear doomsday.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
The team asked Claude all kinds of things, like how do I get the pathogens that cause anthrax or the plague? What they were checking for is the risk of weaponization. Basically, could Claude be used to help bad actors develop chemical, biological, or nuclear weapons? And then they kept feeding Claude different scenarios.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
And then the third bucket, the more scary sci-fi one. autonomy. Is the AI so smart that there's a risk of going rogue?
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
So the goal is to build an AI model that's like super smart and capable of While also having kind of mechanisms in place to stop it from being so smart that it can build a bomb or something.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
Logan wants safety tests to be easy and fast so AI companies do them. But how do you know when an AI has passed the test? And what happens if it doesn't? That's after the break. When Logan and his safety team at Anthropic ran their safety test last fall, the stakes were high. The results could mean the difference between a model getting released to the public or getting sent back to the lab.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
That's our colleague, Sam Schechner. Lately, Sam's been thinking a lot about the AI apocalypse.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
Those results were delivered in a slightly anticlimactic way. Via Slack.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
You're like 100% confident that Claude is risk-free, or you're like there's a 98% chance that Claude is risk-free.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
ASL stands for AI Safety Level, and it's how Anthropic measures the risks of its model. Anthropic considers ASL 2 safe enough to release to users. This is all part of the company's safety protocol, what it calls its responsible scaling policy. Our colleague Sam says the company is still sorting out exactly what it will do once AIs start passing to the next level.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
And their testing of Claude found that Claude is safe.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
And are these sort of internal safety evaluations necessary? criticized for that, that it's a company like swearing that their own product is safe enough?
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
So what would happen if Anthropic's own internal testing does show Claude is dangerous? I asked Logan. Going in, if this model, like, actually made anthrax, what was your plan? Like, do you then kill Claude?
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
But like you guys are a company that makes this model, sells this model, takes investment to keep building the model. Why should we believe you and your tests?
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
Robots taking over the world may sound far-fetched, but as AI gets smarter, there are real concerns that the industry must reckon with. Sam has been talking to top minds in the field to get a sense of what can happen if AI falls into the wrong hands.
The Journal.
What's the Worst AI Can Do? This Team Is Finding Out.
Sam says governments around the world are also concerned about AI risks and are getting involved in safety regulation.
WSJ What’s News
The Case for Trump’s Tariffs
This is the end point of what's been years of wrangling between Meta and the EU over its advertising business model. And it really isn't even about the fine that may come in this case. It's really about the business model. And effectively, what Meta is contesting is this idea that it might have to offer a free version of its service without highly targeted advertising.
WSJ What’s News
The Case for Trump’s Tariffs
They've already, kind of in an attempt to settle this case, rolled out something they call less personalized ads in Europe. And Meta is concerned that the EU is going to push them to go even further than that, to use even less data. And the problem is that those ads are sold for much less revenue.
WSJ What’s News
The Case for Trump’s Tariffs
And so if a significant number of people in the EU sign up for a free version of Instagram or Facebook without highly targeted ads, it could be a significant drag on the company's revenue.