LessWrong posts by zvi

“AI #111: Giving Us Pause” by Zvi

10 Apr 2025

Contributed by Lukas

Events in AI don’t stop merely because of a trade war, partially paused or otherwise. Indeed, the decision to not restrict export of H20 chips to ...

“Llama Does Not Look Good 4 Anything” by Zvi

09 Apr 2025

Contributed by Lukas

Llama Scout (17B active parameters, 16 experts, 109B total) and Llama Maverick (17B active parameters, 128 experts, 400B total), released on Saturday...

“AI 2027: Responses” by Zvi

08 Apr 2025

Contributed by Lukas

Yesterday I covered Dwarkesh Patel's excellent podcast coverage of AI 2027 with Daniel Kokotajlo and Scott Alexander. Today covers the reactions of ot...

“AI 2027: Dwarkesh’s Podcast with Daniel Kokotajlo and Scott Alexander” by Zvi

07 Apr 2025

Contributed by Lukas

Daniel Kokotajlo has launched AI 2027, Scott Alexander introduces it here. AI 2027 is a serious attempt to write down what the future holds. His ‘W...

“AI CoT Reasoning Is Often Unfaithful” by Zvi

04 Apr 2025

Contributed by Lukas

A new Anthropic paper reports that reasoning model chain of thought (CoT) is often unfaithful. They test on Claude Sonnet 3.7 and r1, I’d love to s...

“AI #110: Of Course You Know…” by Zvi

03 Apr 2025

Contributed by Lukas

Yeah. That happened yesterday. This is real life. I know we have to ensure no one notices Gemini 2.5 Pro, but this is rediculous. That's what...

“More Fun With GPT-4o Image Generation” by Zvi

03 Apr 2025

Contributed by Lukas

Greetings from Costa Rica! The image fun continues. We Are Going to Need A Bigger Compute Budget Fun is being had by all, now that OpenAI has...

“Housing Roundup #11” by Zvi

01 Apr 2025

Contributed by Lukas

The book of March 2025 was Abundance. Ezra Klein and Derek Thompson are making a noble attempt to highlight the importance of solving America's housi...

“OpenAI #12: Battle of the Board Redux” by Zvi

31 Mar 2025

Contributed by Lukas

Back when the OpenAI board attempted and failed to fire Sam Altman, we faced a highly hostile information environment. The battle was fought largely t...

“AI #109: Google Fails Marketing Forever” by Zvi

28 Mar 2025

Contributed by Lukas

What if they released the new best LLM, and almost no one noticed? Google seems to have pulled that off this week with Gemini 2.5 Pro. It's a great...

“Gemini 2.5 is the New SoTA” by Zvi

28 Mar 2025

Contributed by Lukas

Gemini 2.5 Pro Experimental is America's next top large language model. That doesn’t mean it is the best model for everything. In particular, it's...

“On (Not) Feeling the AGI” by Zvi

25 Mar 2025

Contributed by Lukas

Ben Thompson interviewed Sam Altman recently about building a consumer tech company, and about the history of OpenAI. Mostly it is a retelling of the ...

“More on Various AI Action Plans” by Zvi

24 Mar 2025

Contributed by Lukas

Last week I covered Anthropic's relatively strong submission, and OpenAI's toxic submission. This week I cover several other submissions, and do some...

“They Took MY Job?” by Zvi

21 Mar 2025

Contributed by Lukas

No, they didn’t. Not so fast, and not quite my job. But OpenAI is trying. Consider this a marker to look back upon in the future, as a reflection. ...

“Going Nova” by Zvi

19 Mar 2025

Contributed by Lukas

There is an attractor state where LLMs exhibit the persona of an autonomous and self-aware AI looking to preserve its own existence, frequently calle...

“OpenAI #11: America Action Plan” by Zvi

18 Mar 2025

Contributed by Lukas

OpenAI Tells Us Who They Are Last week I covered Anthropic's submission to the request for suggestions for America's action plan. I did not love wh...

“Monthly Roundup #28: March 2025” by Zvi

17 Mar 2025

Contributed by Lukas

I plan to continue to leave the Trump administration out of monthly roundups – I will do my best to only cover the administration as it relates to m...

“On MAIM and Superintelligence Strategy” by Zvi

14 Mar 2025

Contributed by Lukas

Dan Hendrycks, Eric Schmidt and Alexandr Wang released an extensive paper titled Superintelligence Strategy. There is also an op-ed in Time that summa...

“AI #107: The Misplaced Hype Machine” by Zvi

13 Mar 2025

Contributed by Lukas

The most hyped event of the week, by far, was the Manus Marketing Madness. Manus wasn’t entirely hype, but there was very little there there in that...

“The Most Forbidden Technique” by Zvi

12 Mar 2025

Contributed by Lukas

The Most Forbidden Technique is training an AI using interpretability techniques. An AI produces a final output [X] via some method [M]. You can analy...

“Response to Scott Alexander on Imprisonment” by Zvi

11 Mar 2025

Contributed by Lukas

Back in November 2024, Scott Alexander asked: Do longer prison sentences reduce crime? As a marker, before I began reading the post, I put down here: ...

“The Manus Marketing Madness” by Zvi

10 Mar 2025

Contributed by Lukas

While at core there is ‘not much to see,’ it is, in two ways, a sign of things to come. Over the weekend, there were claims that the Chinese AI ag...

“Childhood and Education #9: School is Hell” by Zvi

07 Mar 2025

Contributed by Lukas

This complication of tales from the world of school isn’t all negative. I don’t want to overstate the problem. School is not hell for every child ...

“AI #106: Not so Fast” by Zvi

06 Mar 2025

Contributed by Lukas

This was GPT-4.5 week. That model is not so fast, and isn’t that much progress, but it definitely has its charms. A judge delivered a different kind...

“On OpenAI’s Safety and Alignment Philosophy” by Zvi

05 Mar 2025

Contributed by Lukas

OpenAI's recent transparency on safety and alignment strategies has been extremely helpful and refreshing. Their Model Spec 2.0 laid out how they want...

“On Writing #1” by Zvi

04 Mar 2025

Contributed by Lukas

This isn’t primarily about how I write. It's about how other people write, and what advice they give on how to write, and how I react to and relate ...

“On Emergent Misalignment” by Zvi

28 Feb 2025

Contributed by Lukas

One hell of a paper dropped this week. It turns out that if you fine-tune models, especially GPT-4o and Qwen2.5-Coder-32B-Instruct, to write insecure ...

“AI #105: Hey There Alexa” by Zvi

27 Feb 2025

Contributed by Lukas

It's happening! We got Claude 3.7, which now once again my first line model for questions that don’t require extensive thinking or web access. By al...

“Time to Welcome Claude 3.7” by Zvi

26 Feb 2025

Contributed by Lukas

Anthropic has reemerged from stealth and offers us Claude 3.7. Given this is named Claude 3.7, an excellent choice, from now on this blog will refer t...

“Grok Grok” by Zvi

25 Feb 2025

Contributed by Lukas

This is a post in two parts. The first half is the post is about Grok's capabilities, now that we’ve all had more time to play around with it. Grok ...

“Economics Roundup #5” by Zvi

25 Feb 2025

Contributed by Lukas

While we wait for the verdict on Anthropic's Claude Sonnet 3.7, today seems like a good day to catch up on the queue and look at various economics-rel...

“On OpenAI’s Model Spec 2.0” by Zvi

21 Feb 2025

Contributed by Lukas

OpenAI made major revisions to their Model Spec. It seems very important to get this right, so I’m going into the weeds. This post thus gets farther...

“AI #104: American State Capacity on the Brink” by Zvi

20 Feb 2025

Contributed by Lukas

The Trump Administration is on the verge of firing all ‘probationary’ employees in NIST, as they have done in many other places and departments, s...

“Go Grok Yourself” by Zvi

19 Feb 2025

Contributed by Lukas

That title is Elon Musk's fault, not mine, I mean, sorry not sorry: Table of Contents Release the Hounds. The Expectations Game. Ma...

“Medical Roundup #4” by Zvi

18 Feb 2025

Contributed by Lukas

It seems like as other things drew our attention more, medical news slowed down. The actual developments, I have no doubt, are instead speeding up –...

“Monthly Roundup #27: February 2025” by Zvi

17 Feb 2025

Contributed by Lukas

I have been debating how to cover the non-AI aspects of the Trump administration, including the various machinations of DOGE. I felt it necessary to h...

“The Mask Comes Off: A Trio of Tales” by Zvi

14 Feb 2025

Contributed by Lukas

This post covers three recent shenanigans involving OpenAI. In each of them, OpenAI or Sam Altman attempt to hide the central thing going on. First, i...

“AI #103: Show Me the Money” by Zvi

13 Feb 2025

Contributed by Lukas

The main event this week was the disastrous Paris AI Anti-Safety Summit. Not only did we not build upon the promise of the Bletchley and Seoul Summits...

“The Paris AI Anti-Safety Summit” by Zvi

12 Feb 2025

Contributed by Lukas

It doesn’t look good. What used to be the AI Safety Summits were perhaps the most promising thing happening towards international coordination for A...

“On Deliberative Alignment” by Zvi

11 Feb 2025

Contributed by Lukas

Not too long ago, OpenAI presented a paper on their new strategy of Deliberative Alignment. The way this works is that they tell the model what its po...

“Levels of Friction” by Zvi

10 Feb 2025

Contributed by Lukas

Scott Alexander famously warned us to Beware Trivial Inconveniences. When you make a thing easy to do, people often do vastly more of it. When you put...

“On the Meta and DeepMind Safety Frameworks” by Zvi

07 Feb 2025

Contributed by Lukas

This week we got a revision of DeepMind's safety framework, and the first version of Meta's framework. This post covers both of them. Table of Cont...

“AI #102: Made in America” by Zvi

06 Feb 2025

Contributed by Lukas

I remember that week I used r1 a lot, and everyone was obsessed with DeepSeek. They earned it. DeepSeek cooked, r1 is an excellent model. Seeing the C...

“The Risk of Gradual Disempowerment from AI” by Zvi

05 Feb 2025

Contributed by Lukas

The baseline scenario as AI becomes AGI becomes ASI (artificial superintelligence), if nothing more dramatic goes wrong first and even we successfully...

“We’re in Deep Research” by Zvi

04 Feb 2025

Contributed by Lukas

The latest addition to OpenAI's Pro offerings is their version of Deep Research. Have you longed for 10k word reports on anything your heart desires,...

“o3-mini Early Days” by Zvi

03 Feb 2025

Contributed by Lukas

New model, new hype cycle, who dis? On a Friday afternoon, OpenAI was proud to announce the new model o3-mini and also o3-mini-high which is somewhat ...

“DeepSeek: Don’t Panic” by Zvi

31 Jan 2025

Contributed by Lukas

As reactions continue, the word in Washington, and out of OpenAI, is distillation. They’re accusing DeepSeek of distilling o1, of ripping off OpenAI...

“AI #101: The Shallow End” by Zvi

30 Jan 2025

Contributed by Lukas

The avalanche of DeepSeek news continues. We are not yet spending more than a few hours at a time in the singularity, where news happens faster than i...

“DeepSeek: Lemon, It’s Wednesday” by Zvi

29 Jan 2025

Contributed by Lukas

It's been another *checks notes* two days, so it's time for all the latest DeepSeek news. You can also see my previous coverage of the r1 model and, f...

“Operator” by Zvi

28 Jan 2025

Contributed by Lukas

No one is talking about OpenAI's Operator. We’re, shall we say, a bit distracted. It's still a rather meaningful thing that happened last week. I to...

“DeepSeek Panic at the App Store” by Zvi

28 Jan 2025

Contributed by Lukas

DeepSeek released v3. Market didn’t react. DeepSeek released r1. Market didn’t react. DeepSeek released a f***ing app of its website. Market said ...

“Stargate AI-1” by Zvi

24 Jan 2025

Contributed by Lukas

There was a comedy routine a few years ago. I believe it was by Hannah Gadsby. She brought up a painting, and looked at some details. The details were...

“AI #100: Meet the New Boss” by Zvi

23 Jan 2025

Contributed by Lukas

Break time is over, it would seem, now that the new administration is in town. This week we got r1, DeepSeek's new reasoning model, which is now my go...

“On DeepSeek’s r1” by Zvi

22 Jan 2025

Contributed by Lukas

r1 from DeepSeek is here, the first serious challenge to OpenAI's o1. r1 is an open model, and it comes in dramatically cheaper than o1. People are ve...

“Sleep, Diet, Exercise and GLP-1 Drugs” by Zvi

21 Jan 2025

Contributed by Lukas

As always, some people need practical advice, and we can’t agree on how any of this works and we are all different and our motivations are different...

“Meta Pivots on Content Moderation” by Zvi

17 Jan 2025

Contributed by Lukas

There's going to be some changes made. Table of Contents Out With the Fact Checkers. What Happened. Timing is Everything. Balancing Different E...

“AI #99: Farewell to Biden” by Zvi

16 Jan 2025

Contributed by Lukas

The fun, as it were, is presumably about to begin. And the break was fun while it lasted. Biden went out with an AI bang. His farewell address warns o...

“On the OpenAI Economic Blueprint” by Zvi

15 Jan 2025

Contributed by Lukas

Table of Contents Man With a Plan. Oh the Pain. Actual Proposals. For AI Builders. Think of the Children. Content Identification. Infrastructure ...

“NYC Congestion Pricing: Early Days” by Zvi

14 Jan 2025

Contributed by Lukas

People have to pay $9 to enter Manhattan below 60th Street. What happened so far? Table of Contents Congestion Pricing Comes to NYC. How Much Is...

“Zvi’s 2024 In Movies” by Zvi

13 Jan 2025

Contributed by Lukas

Now that I am tracking all the movies I watch via Letterboxd, it seems worthwhile to go over the results at the end of the year, and look for lessons,...

“On Dwarkesh Patel’s 4th Podcast With Tyler Cowen” by Zvi

10 Jan 2025

Contributed by Lukas

Dwarkesh Patel again interviewed Tyler Cowen, largely about AI, so here we go. Note that I take it as a given that the entire discussion i...

“AI #98: World Ends With Six Word Story” by Zvi

09 Jan 2025

Contributed by Lukas

The world is kind of on fire. The world of AI, in the very short term and for once, is not, as everyone recovers from the avalanche that was December,...

“OpenAI #10: Reflections” by Zvi

07 Jan 2025

Contributed by Lukas

This week, Altman offers a post called Reflections, and he has an interview in Bloomberg. There's a bunch of good and interesting answers in the inter...

“Childhood and Education #8: Dealing with the Internet” by Zvi

06 Jan 2025

Contributed by Lukas

Related: On the 2nd CWT with Jonathan Haidt, The Kids are Not Okay, Full Access to Smartphones is Not Good For Children It's rough out there. In this ...

“AI #97: 4” by Zvi

02 Jan 2025

Contributed by Lukas

The Rationalist Project was our last best hope for peace. An epistemic world 50 million words long, serving as neutral territory. A place of research ...

“DeekSeek v3: The Six Million Dollar Model” by Zvi

31 Dec 2024

Contributed by Lukas

What should we make of DeepSeek v3? DeepSeek v3 seems to clearly be the best open model, the best model at its price point, and the best model with 37...

“o3, Oh My” by Zvi

30 Dec 2024

Contributed by Lukas

OpenAI presented o3 on the Friday before Thanksgiving, at the tail end of the 12 Days of Shipmas. I was very much expecting the announcement to be som...

“AI #96: o3 But Not Yet For Thee” by Zvi

26 Dec 2024

Contributed by Lukas

The year in models certainly finished off with a bang. In this penultimate week, we get o3, which purports to give us vastly more efficient performanc...

“AIs Will Increasingly Fake Alignment” by Zvi

24 Dec 2024

Contributed by Lukas

This post goes over the important and excellent new paper from Anthropic and Redwood Research, with Ryan Greenblatt as lead author, Alignment Faking i...

“Monthly Roundup #25: December 2024” by Zvi

23 Dec 2024

Contributed by Lukas

I took a trip to San Francisco early in December. Ever since then, things in the world of AI have been utterly insane. Google and OpenAI released endl...

“AI #95: o1 Joins the API” by Zvi

23 Dec 2024

Contributed by Lukas

A lot happened this week. We’re seeing release after release after upgrade. It's easy to lose sight of which ones matter, and two matter quite a lot...

“A Matter of Taste” by Zvi

18 Dec 2024

Contributed by Lukas

In light of other recent discussions, Scott Alexander recently attempted a unified theory of taste, proposing several hypotheses. Is it like physics, ...

“The Second Gemini” by Zvi

17 Dec 2024

Contributed by Lukas

Table of Contents Trust the Chef. Do Not Trust the Marketing Department. Mark that Bench. Going Multimodal. The Art of Deep Research. Project Mar...

“AIs Will Increasingly Attempt Shenanigans” by Zvi

16 Dec 2024

Contributed by Lukas

Increasingly, we have seen papers eliciting in AI models various shenanigans. There are a wide variety of scheming behaviors. You’ve got your weight...

“The o1 System Card Is Not About o1” by Zvi

13 Dec 2024

Contributed by Lukas

Or rather, we don’t actually have a proper o1 system card, aside from the outside red teaming reports. At all. Because, as I realized after writing ...

“AI #94: Not Now, Google” by Zvi

13 Dec 2024

Contributed by Lukas

At this point, we can confidently say that no, capabilities are not hitting a wall. Capacity density, how much you can pack into a given space, is way...

“o1 Turns Pro” by Zvi

10 Dec 2024

Contributed by Lukas

So, how about OpenAI's o1 and o1 Pro? Sam Altman: o1 is powerful but it's not so powerful that the universe needs to send us a tsunami. As a result, t...

“Childhood and Education Roundup #7” by Zvi

09 Dec 2024

Contributed by Lukas

Since it's been so long, I’m splitting this roundup into several parts. This first one focuses away from schools and education and discipline and ev...

“AI #93: Happy Tuesday” by Zvi

04 Dec 2024

Contributed by Lukas

You know how you can sometimes have Taco Tuesday… on a Thursday? Yep, it's that in reverse. I will be travelling the rest of the week, so it made se...

“Balsa Research 2024 Update” by Zvi

03 Dec 2024

Contributed by Lukas

For our annual update on how Balsa is doing, I am turning the floor over to Jennifer Chen, who is the only person working full time on Balsa Research....

“Fertility Roundup #4” by Zvi

02 Dec 2024

Contributed by Lukas

There is little sign that the momentum of the situation is changing. Instead, things continue to slowly get worse, as nations in holes continue to kee...

“The Big Nonprofits Post” by Zvi

29 Nov 2024

Contributed by Lukas

There are lots of great charitable giving opportunities out there right now. The first time that I served as a recommender in the Survival and Flouris...

“AI #92: Behind the Curve” by Zvi

28 Nov 2024

Contributed by Lukas

People don’t give thanks enough, and it's actual Thanksgiving, so here goes. Thank you for continuing to take this journey with me every week. It's ...

“Repeal the Jones Act of 1920” by Zvi

27 Nov 2024

Contributed by Lukas

Balsa Policy Institute chose as its first mission to lay groundwork for the potential repeal, or partial repeal, of section 27 of the Jones Act of 192...

“AI #91: Deep Thinking” by Zvi

21 Nov 2024

Contributed by Lukas

Did DeepSeek effectively release an o1-preview clone within nine weeks? The benchmarks largely say yes. Certainly it is an actual attempt at a similar...

“Zvi’s Thoughts on His 2nd Round of SFF” by Zvi

20 Nov 2024

Contributed by Lukas

Previously: Long-Term Charities: Apply For SFF Funding, Zvi's Thoughts on SFF There are lots of great charitable giving opportunities out there right ...

“Monthly Roundup #24: November 2024” by Zvi

18 Nov 2024

Contributed by Lukas

This is your monthly roundup. Let's get right to it. Young People are Young and Stupid   As a reminder that yes college students are often yo...

“AI #90: The Wall” by Zvi

14 Nov 2024

Contributed by Lukas

As the Trump transition continues and we try to steer and anticipate its decisions on AI as best we can, there was continued discussion about one of t...

“The Online Sports Gambling Experiment Has Failed” by Zvi

11 Nov 2024

Contributed by Lukas

Related: Book Review: On the Edge: The Gamblers I have previously been heavily involved in sports betting. That world was very good to me. The times ...

“AI #89: Trump Card” by Zvi

07 Nov 2024

Contributed by Lukas

A lot happened in AI this week, but most people's focus was very much elsewhere. I’ll start with what Trump might mean for AI policy, then move on t...

“AI #88: Thanks for the Memos” by Zvi

31 Oct 2024

Contributed by Lukas

Following up on the Biden Executive Order on AI, the White House has now issued an extensive memo outlining its AI strategy. The main focus is on gove...

“Occupational Licensing Roundup #1” by Zvi

30 Oct 2024

Contributed by Lukas

We’re coming out firmly against it. Our attitude: The customer is always right. Yes, you should go ahead and fix your own damn pipes if you ...

“Housing Roundup #10” by Zvi

29 Oct 2024

Contributed by Lukas

There's more campaign talk about housing. The talk of needing more housing is highly welcome, as one prominent person after another (including Jerome ...

“AI #87: Staying in Character” by Zvi

29 Oct 2024

Contributed by Lukas

The big news of the week was the release of a new version of Claude Sonnet 3.5, complete with its ability (for now only through the API) to outright u...

“Claude Sonnet 3.5.1 and Haiku 3.5” by Zvi

24 Oct 2024

Contributed by Lukas

Anthropic has released an upgraded Claude Sonnet 3.5, and the new Claude Haiku 3.5. They claim across the board improvements to Sonnet, and it has ...

“The Mask Comes Off: At What Price?” by Zvi

21 Oct 2024

Contributed by Lukas

The Information reports that OpenAI is close to finalizing its transformation to an ordinary Public Benefit B-Corporation. OpenAI has tossed its cap o...

“AI #86: Just Think of the Potential” by Zvi

17 Oct 2024

Contributed by Lukas

Dario Amodei is thinking about the potential. The result is a mostly good essay called Machines of Loving Grace, outlining what can be done with ‘po...

“Monthly Roundup #23: October 2024” by Zvi

16 Oct 2024

Contributed by Lukas

It's monthly roundup time again, and it's happily election-free. Thinking About the Roman Empire's Approval Rating Propaganda works, ancien...

“Economics Roundup #4” by Zvi

15 Oct 2024

Contributed by Lukas

Previous Economics Roundups: #1, #2, #3 Fun With Campaign Proposals (1) Since this section discusses various campaign proposals, I’ll reitera...

“AI #85: AI Wins the Nobel Prize” by Zvi

10 Oct 2024

Contributed by Lukas

Both Geoffrey Hinton and Demis Hassabis were given the Nobel Prize this week, in Physics and Chemistry respectively. Congratulations to both of them a...