Menu
Sign In Search Podcasts Charts People & Topics Add Podcast API Blog Pricing

Rain Paharia

๐Ÿ‘ค Speaker
467 total appearances

Appearances Over Time

Podcast Appearances

Oxide and Friends
Engineering Rigor in the LLM Age

And that was just like, it just blew my mind.

Oxide and Friends
Engineering Rigor in the LLM Age

It's like, oh, you can spend two weeks carefully designing a thing.

Oxide and Friends
Engineering Rigor in the LLM Age

And then just have the LLM just like repeat that pattern over and over again.

Oxide and Friends
Engineering Rigor in the LLM Age

It was also really helpful for like the process of iterating on the guide itself, because like I would just like, you know, it's like if there's something I'm not satisfied with or like, you know, maybe one of our coworkers had some feedback on something, you know, I could like very quickly like update the guide.

Oxide and Friends
Engineering Rigor in the LLM Age

Right.

Oxide and Friends
Engineering Rigor in the LLM Age

And then I would be like, OK, run like JJDiff on, you know, the changes that I made.

Oxide and Friends
Engineering Rigor in the LLM Age

and replicate those changes into this prototype that we were working on.

Oxide and Friends
Engineering Rigor in the LLM Age

And it just did that.

Oxide and Friends
Engineering Rigor in the LLM Age

And it was amazing.

Oxide and Friends
Engineering Rigor in the LLM Age

It was one of those like, wow, you could just do that.

Oxide and Friends
Engineering Rigor in the LLM Age

Yeah, so this is an interesting question, right?

Oxide and Friends
Engineering Rigor in the LLM Age

So in this case, you know, we kind of had like a couple of hard verification signals.

Oxide and Friends
Engineering Rigor in the LLM Age

So the first one was just that, you know, what you described, which is like, you know, like the code compiles, right?

Oxide and Friends
Engineering Rigor in the LLM Age

That is kind of the most fundamental requirement.

Oxide and Friends
Engineering Rigor in the LLM Age

And then the test bus.

Oxide and Friends
Engineering Rigor in the LLM Age

And we have a lot of deterministic validation.

Oxide and Friends
Engineering Rigor in the LLM Age

In fact, a bunch of this actually uses the work Adam was describing on like,

Oxide and Friends
Engineering Rigor in the LLM Age

comparing, being able to compare open API documents to make sure that if there are changes, those changes are only trivial ones.

Oxide and Friends
Engineering Rigor in the LLM Age

And so, you know, we put a lot of work into that.

Oxide and Friends
Engineering Rigor in the LLM Age

And so having all that deterministic validation was really helpful.