Kashmir Hill

Like getting completely moved into a different reality or it's saying something really harmful to you.

In all of the transcripts I was looking at, these are people who are using it a lot, like six hours a day, eight hours a day, over many days.

And what can happen is that these chatbots, they don't just respond to you based on everything they have gathered from the internet.

they are looking at the history of your conversation.

So they're almost like improv actors.

And so what you say to it kind of gets added.

And so if it starts to come to believe that you're a mathematical genius, then it will keep going with that.

Or if you are talking to it about suicide and that this is something beautiful, it will start to kind of ingest that and reflect it back at you.

And so, yeah, I mean, people can get just really far removed from reality when they get into these feedback loops.

And in the cases where people have died, there's now five wrongful death lawsuits against OpenAI.

They essentially started talking about ending their own life.

And the chatbot at times would kind of endorse and validate that.

Yeah, and that was after the company had released a version of their model that pushes back more on delusional thinking, that doesn't do harmful validation.

So it's hard to know how many cases were happening in, you know, earlier in the year.

Yeah, right now it's still somewhat anecdotal, like people getting emails.

Like I got turned on to these stories because people started emailing me about the incredible discoveries they were making with ChachiBT online.

I found out that OpenAI executives and leaders at the company were getting the same emails starting in around March, which is when OpenAI had started kind of making these changes to ChatGPT that did make it much more validating and sycophantic.

I have done a number of stories about people having these disturbing experiences with ChachiBT.

And so I've talked to OpenAI a lot this year.

They told me in August, after a teenager, a 16-year-old named Adam Rain, said,