“HIA and X-risk part 2: Why it hurts” by TsviBT

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded08 Jan 2026

Publisherfeeds.type3.audio ↗

Handlerec:15192197

Show notes

Crosspost from my blog. PDF version. berkeleygenomics.org. Crosspost from my blog. PDF version. berkeleygenomics.org. Context Previously, in "HIA and X-risk part 1: Why it helps", I laid out the reasons I think human intelligence amplification would decrease existential risk from AGI. Here I'll give all the reasons I can currently think of that HIA might plausibly increase AGI X-risk. Questions for the reader Did I miss any important reasons to think that HIA would increase existential risk from AGI? Which reasons seem most worrisome to you (e.g. demand more investigation, demand efforts to avert)? Which reasons, if any, are cruxy for you, i.e. they might make you think human intelligence amplification is net negative in expectation? Up for a live discussion / debate? Caveats The world is very complicated and chaotic and I can't plausibly predict even important questions like "what actual effect would such and such have". I can't even plausibly resolve much uncertainty, and the world is full of agents who will adaptively do surprising things. So the actual search procedure is something like: What is a way, or reason to think, that HIA might increase [...] ---Outline:(00:26) Context(00:46) Questions for the reader(01:15) Caveats(02:31) What is HIA?(03:18) Vague definitions of intelligence and HIA(05:37) HIA as a general access good(07:39) HIA and reprogenetics(08:21) AGI X-risk(08:24) Background assumptions(09:29) Red vs. Blue AGI capabilities research(11:03) An ontology of effects of interventions on world processes(12:21) The meaning of acceleration(13:58) Effects of HIA on a single process(17:24) Effects of HIA involving multiple processes(19:00) Processes(20:20) Some plausible bad effects of HIA on processes(20:34) Speeding up Blue research(22:27) Speeding up Red research(24:56) Less speeding up legal and social regulation(26:21) Nonlinear / race-condition regulatory escape(26:58) Alignment loses the race anyway(27:37) Intrinsic regulatory escape(28:34) Disrupting regulatory systems(31:02) Social values favor following local incentives(32:44) Less speeding up change towards better values(34:03) Alignment harnesses added brainpower much less effectively than capabilities research does(39:32) HIA people may tend to be transgressive(41:59) Other arguments(42:08) Concentration of power(43:06) HIA is unpredictable and therefore risky(43:35) More capable but not wiser(44:38) Acknowledgements --- First published: January 7th, 2026 Source: https://www.lesswrong.com/posts/K4K6ikQtHxcG49Tcn/hia-and-x-risk-part-2-why-it-hurts --- Narrated by TYPE III AUDIO.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.