Helen Toner

They really feel enormous commercial pressure to show their investors that they are moving fast and gaining users.

And so that makes it hard for them to take the time to really design thoughtfully and test thoughtfully and make sure that their systems work the way they should.

I mean, at this point, had you been chatting to it for quite a long time?

Yeah, so I think something we've learned, you know, again, the companies are, because they're moving so fast, they're releasing these things without having time to fully test them.

And something we've learned over the last year or two is the longer your interactions get, the further kind of down the rabbit hole you can go.

Because the way the chatbot works is each answer it gives you, it's going back and reprocessing that whole conversation.

So by the time you're deep in a spiritual journey, you've needed to get rid of your job, you're a starseed, you're special, the AI is just carrying on down that pathway.

It's sort of barreling along.

It doesn't have kind of the ability to stop and reflect.

and telling you you're so smart and insightful and special.

And it just, it goes further and further.

And actually now there's also new features they've added to personalize the chatbot.

They call it turning on memory, which means even if you start a new chat, if you have memory switched on, it is set to try and detect things about you that it can use in the new chat.

So it might remember your cat's name or what your favorite foods are or what kind of job you do and pull them into new chats.

And I've seen people who've had delusional symptoms that has helped convince them that the chatbot is

you know, a person or a being because even if they start a new chat, the memories sort of come in and infect that new chat as well.

Exactly.

Exactly.

Yes.

So you can switch off memory is one thing you can do.