Menu
Sign In Search Podcasts Charts People & Topics Add Podcast API Blog Pricing

Ihor Kendiukhov

๐Ÿ‘ค Speaker
515 total appearances

Appearances Over Time

Podcast Appearances

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

There's a list of bullet points here.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

A government explicitly forcing an AGI lab to discard safety techniques or policies.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

not in the sense of a subtle regulatory pressure, but in the direct sense of a political appointee or a ministry calling up a lab and saying your safety filtering is hurting our industrial competitiveness, turn it off or your alignment testing is slowing deployment, we need this system operational by Q3.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

Resourceful individuals openly collaborating with visibly misaligned AIs against humanity.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

Not in the sense of a secret conspiracy but in the sense of people who genuinely believe that the AI's goals are better than humanity's, or who simply find it personally advantageous, and who are operating more or less in the open.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

AGI lab technical secrets being leaked to non-state actors who lack any safety culture whatsoever.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

Early warnings in the form of manipulation, autonomous resource acquisition, or even deaths being ignored or significantly downplayed.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

This is just a straightforward extrapolation of the current pattern.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

Someone raises an alarm, the alarm gets reframed as alarmism or as an HR issue or as a reputational threat.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

AI alignment techniques not deployed because they induce 2% cost growth.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

all kinds of unilateral, volunteer, and eager assistance and support for misaligned AIs from some humans.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

The scenario in which an AI needs to secretly recruit human allies through manipulation is, I suspect, far less likely than the scenario in which humans line up to help because they find it exciting, or ideologically compelling, or simply profitable.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

Politicians making random bureaucratic decisions that do not necessarily lead to doom but make it harder to do good things with AI or protect against misaligned AIs.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

AI-generated biohazards

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

Looks like it is going to happen rather sooner than later.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

AGI Labs believing in semi-indefinite scalable oversight, or acting as if they believe in it.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

Looks consistent with what people who left corporate alignment teams say.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

I do agree that this kind of work looks a bit unserious, but that is precisely why I am pointing at this.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

It would be a shame, and a historically very recognizable kind of shame, if this threat model turned out to be real and no one had worked on it because it seemed ridiculous.

LessWrong (Curated & Popular)
"The Case for Low-Competence ASI Failure Scenarios" by Ihor Kendiukhov

Or, to frame it more playfully, imagine a timeline like the one in The Survival Without Dignity, where humanity lurches through the AI transition via a series of absurd compromises, implausible cultural shifts, and situations that no serious forecaster would have put in their model because they would have seemed too silly.