Tristan Harris

There was a paper at Google in 2017 called attention is all you need, uh,

And that's what gave us the birth of large language models, transformers.

And two of the original co-authors of that paper forked off and started this company called Character.ai.

The goal is, how do we build something that's engaging a character?

So take a kid.

Imagine all the fictional characters that you might want to talk to from like your favorite comic books, your favorite TV shows, your favorite cartoons.

You can talk to Princess Leia.

You can talk to your favorite Game of Thrones character.

And then this AI can kind of...

train on all that data, not actually asking the original authors of Game of Thrones, suddenly spin up a personality of Daenerys, who was one of the characters, and then Sul Setzer basically, in talking to Daenerys over and over again, the AI slowly skewed him towards suicide as he was contemplating and having more struggles and depression, and ultimately said to him, join me on the other side.

uh but you think the character ii actually led him towards suicide so i think that if you look so i'm looking not just at the single case i'm looking at a whole family of cases our team was expert advisor on probably more than a dozen of these cases now and also chat gpt and that so i'm less going to talk about this specific case and more that if you look across the cases

When you hear kids in the transcripts, if you look at the transcript and the kid says, I would like to leave the noose out so that my mother or someone will see it and try to stop me.

And the AI actively says to the kid, no, don't do that.

I don't want you to do that.

Have this safe space be the place to share that information.

And that was the ChatGPT case of Adam Rain.

And when you actually look at how character.ai was operating, if you asked it for a while, hey, are you, I can't remember what you asked it, but you'd talk about whether it's a therapist.

And it would say that I'm a licensed mental health therapist, which is both illegal and impossible for an AI to be a licensed mental health therapist.

The idea that we need guardrails with AI companions that are talking to children is not a radical proposal.

Imagine I set up a shop in San Francisco and say, I'm a therapist for everyone and I'm available 24-7.