L. Rudolph L

The Possessed Machines is one of the most important AI microsites.

It was published anonymously by an ex-Lab employee, and does not seem to have spread very far, likely at least partly due to this anonymity, for example there is no less wrong discussion at the time I'm posting this.

This post is my attempt to fix that.

I do not agree with everything in the piece, but I think cultural critiques of the LAGI uniparty are vastly undersupplied and incredibly important in modelling and fixing the current trajectory.

The piece is a long but worthwhile analysis of some of the cultural and psychological failures of the AGI industry.

The frame is Dostoevsky's Demons, alternatively translated The Possessed, a novel about ruin in a small provincial town.

The author argues it's best read as a detailed description of earnest people causing a catastrophe by following tracks laid down by the surrounding culture that have gotten corrupted.

Quote

What I know is that Dostoevsky, looking at his own time, saw something true about how intelligent societies destroy themselves.

He saw that the destruction comes from the best as well as the worst, from the idealists as well as the cynics, from the people who believe they are saving humanity as well as those who want to burn it down.

End quote.

The piece is rich in good shorthands for important concepts, many taken from Dostoevsky, which I try to summarize below.

first how to generalize from fictional evidence correctly the author argues for literature as a source of limited but valuable insight into questions of culture and moral intuition quote literature cannot tell us what to do it cannot provide policy prescriptions or technical solutions it cannot predict the future or settle empirical questions

the person who reads Dostoevsky looking for an alignment technique will be disappointed.

What literature can do is reshape perception.

It can make visible patterns that were invisible, make felt truths that were merely known, make urgent realities that were abstract.

It can serve as a kind of training data for moral intuition, presenting scenarios that expand the range of situations one has experienced and therefore the range of situations one can respond to wisely.

Dostoevsky's particular value is that he was obsessed with exactly the questions that matter most for AI development.

What happens when intelligence develops faster than wisdom?

What happens when the capacity for reasoning outstrips the capacity for feeling?