Kate Klonick
👤 PersonPodcast Appearances
I mean, that's what we go to the movies for.
That's what we turn into like certain types of things for, right?
It's I'm not in the mood for, you know, a horror film.
This kind of approach is much easier to moderate.
Because there is not as much reactive content moderation to do.
You don't have to employ hundreds of people in call centers to review every report of something that's been flagged.
And so this has kind of become the new standard.
No, I mean, I've always liked the mall metaphor and it has a weird squirrely little place in First Amendment law in a bunch of cases.
But I want to hear what you kind of want to hear what yours is.
You know, you can shadow ban or take down or limit the reach, but it doesn't even have to be that subtle.
Like Elon Musk always showing up in my feed, even though I don't follow Elon Musk, is like having Rupert Murdoch in like the interstitial spaces before every commercial break at Fox News, you know, like directly telling me what I should think.
Like that is the other thing about this that is maybe the scariest part of the last couple of months is that none of it even is super pretextual.
We're not even hiding behind algorithms anymore.
It is just the owner of the platform saying the thing out loud and forcing everyone to see it if they're on his platform.
You know, I think that if you're going to all of these different platform islands, the other thing is, like, how do we change those?
To use regulatory regimes to try to control how they speak is obviously a problematic thing by any type of measure.
We don't want governments controlling speech for the exact reason of all of the authoritarianism we've just discussed.
And so I think that there's... It's very hard... Sorry, if I can jump in there, though, but it does feel like...
No, I mean, like every Western state has some type of media regulator specifically to avoid maybe like two or three people controlling all of media.
But all of a sudden we're like on the internet and yes, there is an infinite amount of content on the internet, but is it so infinite?
Like if there are, if we're talking about like the same three main places that people are going to for their news, people are going to for like their, for their daily interactions, people are going to, to feel like they're part of a conversation, their water cooler, their public square, whatever it is.
If that is like three people and they're all friends of the president, like that's a problem.
And maybe even more importantly, journalists, they go to X, they go to Blue Sky, they go to YouTube, they go to TikTok.
And they report things that are happening in those places as if they're real places that things are happening.
But they're also controlled by these individuals.
And so they're not reflective necessarily of real world, yet they are being reported on as if they were reflective of real world.
I just think that what you see in the last five years is an industry understand the power that it holds in content moderation, that it's so not a customer service issue, that it is actually like a huge, huge force
And now it is a problem of, okay, how do we stop billionaires and authoritarian governments from twisting these platforms into censorship machines or political propaganda?
for shaping public opinion and that that has exponential value to political parties and governments.
It's like as valuable as oil and guns because how you push things, what you keep up, what you take down.
I mean, this is how you can basically create
You know, the rise and fall of presidencies, if you want to, or political parties.
And they know how to market them to you, no matter how niche you are.
And so, like, it's a way to make a lot of money, and then it's a way to control a lot of minds.
You know, it wouldn't be the first time that someone has told me that in some way I'm a useful idiot to Facebook or in some type of capacity.
I feel as if a lot of people, and a lot of what we've said today, people will be like, of course this is what happened.
But it wasn't fait accompli when we talked about it.
Every single one of these solutions has the same...
flaw at the end of the day to it which is that these are for-profit companies that do what they want to do and things change um as things settle so i don't know okay well so then like is content moderation sort of dead i just um
Yeah, this is like a very controversial thing.
It really depends on what you mean by that question.
There has been a lot of controversy around, like, are they going to invest in these huge cost centers of trust and safety?
Are they going to care about this type of issue?
if they can, TikTokify everything and just send you down these rabbit holes of endlessly drooly, like, eye-glaze-over, like, WALL-E kind of scene where you're on the couch with your Slurpee, like, Barca lounger or whatever, like, watching things.
Is that what they're basically going to do and are they going to have to keep moderating?
And I mean, I think that, like, the answer is that we're going to increasingly see a automated content moderation system.
the edges of society and the range of voice that we had at the beginnings of the internet.
And that we are going to kind of see a productification of speech.
Maybe you can ask AI to do it for you if you're really busy.
So the main thing, the main thing from the last time we talked that has really, truly changed from like 2020 to 2025 is the rise of TikTok.
I mean, if you will remember, like in two short years, it had basically caught up with 12 years of Facebook's growth.
And I mean, TikTok has a different way that they run their content moderation.
Well, when we spoke in these past episodes, one of the assumptions of content moderation when it was getting off the ground, via Facebook or Instagram or YouTube, was that we don't want to censor people unnecessarily.
And so you would keep content up until it was reported as being harmful.
And then you would make rules that would limit and try to preserve voice
That was like the industry term for free speech, voice.
But generally, like, it was a keep it up unless we have to take it down type of thing.
TikTok comes from obviously China and it comes from a censorship kind of authoritarian CCP culture.
And I mean, I believe the Chinese kind of approach to speech is very reflected in the algorithm that TikTok uses.
This is a free world and people have a right to say whatever they want, even if it's a private platform.
It is a we get to determine what people see and say.
TikTok, it prescreens such a volume of content that it determines to not be outside of certain political parameters.
And so they're less likely to cause negative interaction effects, to put kind of an economic term on it.
And they push things up that are very milquetoast, very like happy, make you feel good, very apolitical.
And so this is basically downranking or shadow banning.
The idea that you're going to manipulate the algorithm to not delete the content, but not promote it.
And in addition to that, the algorithm is constantly improving and iterating on all the behavioral signals that you give it.
And so it's able to provide a very addictive and...
I mean, there's no way I'm like almost an experience, but I'm like, yeah, it's kind of, but it's not, it's, it's, I don't know what it is.
Well, I have like rules for some of these things.
But, you know, I study online speech for a living, so it seems kind of crazy.
But I don't need to actually be on TikTok for TikTok to be all over my life.
Oh, I mean, I think that it's actually fascinating.
You know, what they figured out is it is a format, a video that people are hooked by.
You will find yourself often watching things that you didn't know you were interested in, but like you're just compelled by certain types of couples that like look very different from each other doing any type of like interaction.
Yes, that's like one way of thinking about it.
I mean, like advertisers have been doing this forever.
Like this is, you know, it's just a very different business model.
But it's also in like a certain way is even more dangerous because like the ultimate in censorship in American First Amendment law is really prior restraint.
Don't get me saying the F word again because last time my parents yelled at me.
Prior restraint is censorship before something goes up or is ever published.
And it's important because the existence of this redaction, the proof that it was removed from Facebook, is actually evidence that censorship has happened, right?
Whereas with TikTok, you never even know what you missed.
You never even know what you were kept from seeing.
And that is really, unfortunately, what we're staring down at this moment because in the last five years...
American social media has moved towards TikTok's approach to content moderation.
Yeah, they're like, Kate, you're an adult now.
But the biggest sea change is the one that you're thinking of.
which is the one that happened on January 7th of this year, 2025, when Mark Zuckerberg announced the end of the fact-checking program.
And that he was going to try to move towards a community notes-based system of content moderation.
And I mean, I think that, like, it was and it wasn't a sea change.
Okay, so not much, which is why this was a really... Okay.
Such a frustrating announcement, and it was frustrating that the media focused on it so much.
The fact-checking was like a commitment to fact-checking because there had been so much clamor about mis- or disinformation.
But they were removing posts days after they were flagged, and, like, it was very small.
And so to watch it go on the chopping block was really more of a signal.
To a very particular person and to a very particular party that felt like big tech censorship was coming for them.
And like, you know, we can get into a whole kind of conversation about whether or not that was reality based, but that was kind of the complaint.
Reporting lays out purported emails between Hunter Biden and a Ukrainian businessman.
New York Post, they broke the story and links to that were taken off Facebook and Twitter.
Well, that was happening a couple weeks before the 2020 election.
And so what had been the huge concern for Facebook and all these other companies was how social media impacted the 2016 election.
And one of them was just kind of like, we're not going to allow things that could possibly be foreign influence to stay up because this is exactly what we got yelled at in 2016.
And I think in hindsight, it was a really hard call and maybe probably the wrong one.
And then you extend that to the Wuhan lab leak.
It's not like they were that censored, unlike going to, say, China, where it's like you're like, oh, you know, Tank Man.
My honest belief, I can't predict the future, but my honest belief is this administration would very quickly put the platforms in line.
Yeah, I think that there would be no hesitation to do this because I don't think that this was ever about free speech.
And that is really what you're unfortunately seeing right now.
There is no recognizable free speech notions coming out of this current administration.
And with the TikTokification of social media, people have seen the vector for power that is in content moderation.
Yeah, I think that basically what you're seeing is the power over what appears in your feed or doesn't appear in your feed or the types of new content that you're recommended or the first commenters that you see on a video that you just watched.
That type of control is an ability that we've never seen before.
I remember when I was first writing about this in like 2017, 2018, presenting my research, one of the things that people were so concerned with was filter bubbles.
Well, we're going to be in these filter bubbles fed to us by the algorithm.
And as it turns out, that was one very true that that would happen.
But also, even maybe more disturbingly, we don't even need filter bubbles anymore.
People are just choosing platforms based on the types of content that they expect to find there.