Menu
Sign In Search Podcasts Charts People & Topics Add Podcast API Blog Pricing

Rain Paharia

๐Ÿ‘ค Speaker
467 total appearances

Appearances Over Time

Podcast Appearances

Oxide and Friends
Engineering Rigor in the LLM Age

Right.

Oxide and Friends
Engineering Rigor in the LLM Age

I found that, okay, there were a few things that was unsatisfied, but so I went back and like changed the guide, updated the guide.

Oxide and Friends
Engineering Rigor in the LLM Age

I kind of started from scratch.

Oxide and Friends
Engineering Rigor in the LLM Age

So, you know, I like iterated on it.

Oxide and Friends
Engineering Rigor in the LLM Age

I want to say like overall, like this guide kind of went through maybe a couple dozen iterations of like me looking at the LLM output and being like, okay, you know, this is great or this is not good and so on.

Oxide and Friends
Engineering Rigor in the LLM Age

And, you know, kind of basically ended up converging on something that is like this.

Oxide and Friends
Engineering Rigor in the LLM Age

clear, very reproducible set of instructions that are simply way too complicated to capture in any deterministic algorithm, right?

Oxide and Friends
Engineering Rigor in the LLM Age

So this is like, you know, there's like enough judgment here.

Oxide and Friends
Engineering Rigor in the LLM Age

And it's just like this really complicated set of things that, I mean, you know, there is no way I can write like a migration tool to do this.

Oxide and Friends
Engineering Rigor in the LLM Age

I mean, maybe someone smarter than me could do that.

Oxide and Friends
Engineering Rigor in the LLM Age

I don't think I can.

Oxide and Friends
Engineering Rigor in the LLM Age

But what the LLM let me do is it kind of, again, let me design this guide once and then apply it everywhere.

Oxide and Friends
Engineering Rigor in the LLM Age

So it was funny because there was one morning where I just rapidly put up three PRs

Oxide and Friends
Engineering Rigor in the LLM Age

where like you know the first one was like a thousand lines of code the second one was two thousand lines of code and the third one was three thousand lines of code and i got like all three of those done in like an hour and that was just wild like you know and this is like one of those things where like it turns out that llms are really really good

Oxide and Friends
Engineering Rigor in the LLM Age

at following instructions that are like, you know, clearly written and are written in a way that, you know, the LLM kind of works well with.

Oxide and Friends
Engineering Rigor in the LLM Age

So I had a, like, this is again, one of those things where like, it sounds like so like, you know, mid priority, right?

Oxide and Friends
Engineering Rigor in the LLM Age

It's like, how are we going to, you know, migrate like 40,000 lines of code and like rearrange the types, right?

Oxide and Friends
Engineering Rigor in the LLM Age

This is the kind of thing that just falls, you know, people just don't do or like,

Oxide and Friends
Engineering Rigor in the LLM Age

We might do it in the future and there's this long migration period or like, you know, this is the kind of thing that you do in like tech debt week, but this would be more like tech debt month, right?

Oxide and Friends
Engineering Rigor in the LLM Age

But like, you know, and LLM just like, as I said, it just nailed three different APIs in like one hour.