Karen Hao

This is one of the big challenges. There was this really great article in The Atlantic written by a former colleague of mine, Matteo Wong, who says that the fact that pangram is significantly more accurate than other AI detection software potentially makes the situation worse, because it garners more trust in the tool, even when the tool is still wrong. It can absolutely be wrong, and the errors...

go back to the way that this technology works in the first place. It's not like eventually the technology will improve so much that it will be correct 100% of the time. No, it will always have errors. And the reason is because of the way that Pangram actually trains its detection software. It trains it just like actually companies train large language models. It gives this detection tool...

Tons of examples of AI-generated text and then tons of examples of human written text. And the reason why its accuracy, the company says, is higher than other types of detection tools is because of the ways that it really tries to give particularly tough examples to its AI software during training. So it will take human-generated text and ask ChatGBT, Gemini, Claude, like the most

frequent commercial AI chatbots to then generate a passage that is on the same exact topic so that it has a very, very close AI-generated pair to the human written text. And that is part of the reason why it's been able to drive its accuracy higher and higher.

But at the end of the day, when you think about it, first of all, these are probabilistic tools. So it's only basing the differences between human written and AI generated with statistical patterns that it's calculating.

So it's literally never going to be deterministic. It's never going to be 100% accurate. And second of all, the tools that give us AI-generated text were literally trained on human written writing. So the Venn diagram for human writing and AI-generated writing has a massive overlap in the middle. And no matter what you do to try and train an AI detection tool,

It just will not be able to discern. You know, some humans will write like AI because AI is intended to write like humans.

Tämä on tärkeä ongelma, että nämä humanisoitajat eivät oikeastaan humanisoida kirjoja. He ovat oikeastaan vain rajoittamassa kirjoja AI-generoivasta kirjoista, mikä oikeastaan vain tekee sen pahemmin. Kirjoja AI-generoivasta kirjoista.

Yksi syy, miksi AI-tietoja on olemassa, on se, että se tarkoittaa uudistamista ihmisen kirjoittamista. Se tarkoittaa, että ihmiset, jotka eivät käyttäneet omia mielisiänsä, eivät käyttäneet omia mielisiänsä,

But it's actually just adding another layer onto AI influencing our writing overall. You know, we're being influenced both by some people using AI tools to generate the writing. We're being influenced by people actually just like

Whether intentionally or unintentionally absorbing the language and rhythm of AI writing into our own writing. And then detection tools themselves are causing, you know, these poor students from this listener that wrote to us to literally edit their essays to sound actually less human, less proficient in English to try to avoid AI detection software.

Joten kaikkien näiden tavoitteiden jälkeen, se vaikuttaa ihmisen kirjoitteluun ja tekee sitä vielä vaikeampaa, jotta me voimme oikeasti ottaa todella tarkkaan kontrollin maailmalle, mitä me haluamme sanoa ja esittää.

I want to read you this quote that I really liked from an essay written by a researcher and artist named Eric Salvaggio. He says at the end of his essay where he's reflecting on this pangram problem, if using AI to write is at its worst an industrialization of the mind, then AI detection at its worst becomes a surveillance system for thought. I think this goes exactly to what both of you are saying, which is that

We keep just trying in all of these ways to superficially fix or patch fundamental structural issues that AI is actually revealing in our society. It's revealing the inequities in education. It's revealing how the model of how we teach our students was some...

Not always actually designed around facilitating their learning. It was more just trying to mechanize the process of knowledge induction itself.

And what I often say when I'm speaking to journalists or speaking to educators or any institution or sector that now feels like they're facing an existential crisis with AI is now is actually the moment to ask, why do you exist in the first place? And if AI is really beginning to steamroll some of the processes or procedures that you have developed over time, maybe that's actually...

an indictment of how you were doing things, like how you started to do things, because it might not actually be getting at the core or the essence of what you were meant to be doing in the first place. And so rather than using all of these different new AI, yet more AI tools to try to patch that over, use this as an opportunity to revert back to the fundamentals and reinvent how you should be doing journalism, doing teaching, doing education in and of itself.

It was a non-profit.

It was a non-profit.

Which seems hilarious, no?