Max Tegmark

That's right.

And I feel we have to remember, as individuals, we should from time to time ask ourselves, why are we doing what we're doing?

And as a species, we need to do that too.

So if we're building, as you say, machines that are outnumbering us and more and more outsmarting us and replacing us on the job market, not just for the dangerous and boring tasks, but also for writing poems and doing art and things that a lot of people find really meaningful, we've got to ask ourselves, why?

Why are we doing this?

The answer is Moloch is tricking us into doing it.

And it's such a clever trick that even though we see the trick, we still have no choice but to fall for it, right?

Also, the thing you said about you using co-pilot AI tools to program faster, what factor faster would you say you code now?

Does it go twice as fast?

So if that's typical for programmers, then you're already seeing another kind of recursive self-improvement, right?

Because previously, a major generation of improvement of the code would happen at the human R&D timescale.

And now, if that's five times shorter...

then it's going to take five times less time than it otherwise would to develop the next level of these tools and so on.

So this is exactly the sort of beginning of an intelligence explosion.

There can be humans in the loop a lot in the early stages, and then eventually humans are needed less and less, and the machines can more kind of go along.

But what you said there is just an exact example of these sort of things.

Another thing which...

I was kind of lying on my psychiatrist, imagining I'm on a psychiatrist's couch here saying, well, what are my fears that people would do with AI systems?

So I mentioned three that I had fears about many years ago that they would do, namely teach you the code, connect it to the internet, then teach it to manipulate humans.

A fourth one is,