Max Tegmark

Yeah, you can teach it all it needs to know about how to cure cancer and stuff like that, but don't let it read Daniel Kahneman's book about cognitive biases and all that.

And then

oops, LOL, let's invent social media recommender algorithms, which do exactly that.

They get so good at knowing us and pressing our buttons that we're starting to create a world now where we just have ever more hatred because they figured out that these algorithms, not out of evil, but just to make money on advertising, that the best way to get more engagement, the euphemism,

get people glued to their little rectangles.

It's just to make them pissed off.

Exactly.

And that is not something you think is in humanity's interest.

Yes.

Right now, it's mainly letting some humans manipulate other humans for profit and power, which already caused a lot of damage.

And eventually that's a sort of...

skill that can make AIs persuade humans to let them escape whatever safety precautions we put.

You know, there was a really nice article in the New York Times recently by Yuval Noah Harari and two co-authors, including Tristan Harris from The Social Dilemma.

They have this phrase in there I love.

Humanity's first contact with advanced AI was social media.

And we lost that one.

We now live in a country where there's much more hate, in a world where there's much more hate, in fact.

And in our democracy that we're having this conversation and people can't even agree on who won the last election.

And we humans often point fingers at other humans and say it's their fault, but it's really Moloch and these AI algorithms.

We got the algorithms and then Moloch