Sam Schechner

For instance, if a user described just a man and his daughter entering a room, the AI would, we were told, a quote, uncomfortable amount of the time, proceed to depict a scenario involving incest.

Yeah.

Now, I mean, for me, any amount of the time would be uncomfortable.

Yeah.

It's hard to put a figure on that, but it's definitely a tendency that chatbots trained on internet content have.

They didn't have tools to moderate content where they could draw clear lines between types of erotica that might be totally cool and things that were very disturbing.

So they just basically said that they weren't going to allow any erotica on the platform.

there are a certain number of power users who can become very emotionally engaged with it.

That can have multiple potentially bad impacts on them.

For instance, it can help push out relationships they have with real humans and lead them to become emotionally over-reliant on the chatbot.

The fear is that for those subset of people and potentially for other people as well, that when you mix in sexual content,

literally tickling the parts of the brain that govern attachment and love and devotion, that you could just pour fuel on that fire.

There are people inside OpenAI who think, listen, this is something that people want.

And who are we to say this or that fetish or interest is or isn't okay?

That's the same logic that you might use to ban gay content a generation ago.

And so who are we to ban this?

And maybe we should even open this up and allow more of this potentially pornographic content.

So there is this idea that we shouldn't be telling people what to do.

It's big for their business, too.

You know, are you going to maintain the growth that you've seen in ChatGPT as there's more competition?