Tobes (author/narrator of the LessWrong linkpost)

No one in my org puts money in their pension.

It's summer 2022, I am at a party in central London.

I am talking to an AI alignment researcher in the kitchen.

He is upbeat and is excitedly talking about some ideas that might ensure the alignment of superintelligent AI with human values.

We just need to create this small AI that we can fully understand and verify is aligned, he explains.

And then we use that AI to validate that the super huge massive superintelligence is aligned with our values.

It's a bit like creating Godzilla to fight Mega Godzilla.

That is not a reassuring metaphor, I say.

Oh yeah, we're not super hopeful, but we're trying, he states, somehow still cheerfully.

Later in the conversation, he will explain that despite the government incentive systems, no one at his place of work takes anything above the minimum possible pension.

They all doubt they will need it.

Over the next few weeks, I will wonder if I should be putting less money into my pension.

it is february 2023 i now have a vague sense of impending doom most mornings i'm not sure why

It might be the combination of my pre-existing beliefs about the difficulty of AI alignment, and the holy shit rapid advances vibe that open AI has been producing.

It might be that I have been reading post after post talking about the timelines to transformative AI and the probability of doom.

Or all the people talking about how AI alignment seems really hard, and how there is a now a malokian race to the bottom between AI labs to produce the biggest intelligence as quickly as possible.

The doom feeling might also be the midwinter darkness and cold, alongside my general tendency towards anxious thoughts and feelings.

It could also be that I am yet to secure future funding for the project that is my current livelihood.

All I know is that I regularly feel hopeless.

Each morning, I go to the window of my East London flat.