Dr. Karthik V. Sarma

Their intent isn't to make a mental health agent.

And if you're having a mental health problem, you've got to talk to a doctor.

So there are some, and the makers would call them safeguards.

There are safeguards that, for example, it's supposed to auto-detect if you're asking a medical question, they'll put in a label saying, hey, I'm not a doctor, you should talk to a doctor.

There are safeguards where if people start having really dark thoughts, talking about very dark things, like thoughts of death, that it's supposed to come back and say, hey, it sounds like you're having a crisis, call 9-8-8.

They have implemented these things, but there is some evidence, for example, a recent report by Common Sense Media that found that those safeguards aren't consistently tripped by people talking about things that are concerning.

And I think part of the reason for that is it's actually really hard without the context of being a doctor to know, is this concerning or is this play acting?

Is this someone really having a crisis or is this someone whose mom died this morning and is just really sad, but tomorrow will start to feel a little bit better?

I'll tell you, my gut feeling is that really minor interactions are probably safe.

But I can't say that the data backs that up because we really actually have very little data to show what happens.

How do these agents change people's mental health?

But we do know things about that from other realms.

I think we have some sense of what it's like to be a problematic user of anything, whether that be a chatbot.

or it's internet games like Call of Duty or social media, we have some sense that there are some common factors that might signify, hey, your relationship with whatever this is, it seems to be disrupting your life and you've got to change.

It depends a little bit on who you ask.

There have been a number of different studies that have tried to evaluate that.

OpenAI actually recently released a blog post where they talk about the subject a little bit.

I would say, though, in my experience as a psychiatrist, a pretty high proportion of my patients have consulted CHI CPT or something like it about the mental health.

If I had to peg it, I'd say probably 60, 70% of the patients who I've seen have at least once consulted Chachapiti about what's going on.

I think all of the above.