Alex Imas

And you can't automate an entire job to an AI right now, even though it might...

be able to perform it at some probability.

You need extreme reliability in order for it to not destroy the finished good.

I think this is, yeah, so this might explain why there's less automation now than there otherwise could be.

But I think it works in the other direction once AIs get advanced enough that integrating humans into the production flow of future goods

even beyond the arguments about how humans will be more expensive or dumber or whatever, even beyond that, just there will be whole production flows that are organized for AI labor, where they're talking in neuralese, they're thinking many thousands of times faster.

So even if there's some comparative advantage where it makes sense to hire a human, there will be like transaction costs and worries about reliability that will actually make it hard to integrate humans into future production flows.

By the way, the model you're talking about seems extremely plausible to me of why more lawyers or accountants or whatever are not automated.

Like there are cases or even software engineers where there's a pretty good probability that the thing worked as you expect.

But the thing you're paying the lawyer for is like, no, really, my company is not going to go under because.

So speaking of which, we've been talking about

what preferences humans currently have and what impact that has on what kinds of goods will be scarce in the future.

But of course, we'll have different kinds of entities in the future, AIs, right?

There was a time when there were no humans on earth, but evolution selected for agents that have specific drives and preferences because those tend to survive the most.

And those preferences now basically determine how $100 trillion world economy

what it produces.

And so why not expect the same thing of AIs in the future?

This is not even a world with catastrophic misalignment, that is to say they just kill everybody.

But there will be evolution of, even if not individual AIs, then firms which have AIs as part of them.

And what will that evolution favor?