Menu
Sign In Search Podcasts Charts People & Topics Add Podcast API Blog Pricing

Adam Leventhal

๐Ÿ‘ค Speaker
2711 total appearances

Appearances Over Time

Podcast Appearances

Oxide and Friends
Engineering Rigor in the LLM Age

And I'll drop a link to the actual bug itself.

Oxide and Friends
Engineering Rigor in the LLM Age

Lumos 17.8.16.

Oxide and Friends
Engineering Rigor in the LLM Age

So I'll drop a link in for that.

Oxide and Friends
Engineering Rigor in the LLM Age

And so you can see exactly what the problem was at hand.

Oxide and Friends
Engineering Rigor in the LLM Age

pretty straightforward.

Oxide and Friends
Engineering Rigor in the LLM Age

Now, I was gonna use, like very deliberately, not using it, I'm definitely not closing the loop, not vibe coding it, not one-shotting it, but really, because in particular, like I am not, I'm not even gonna let it build anything, right?

Oxide and Friends
Engineering Rigor in the LLM Age

I'm gonna let it, we're gonna go into the source base and I just wanted to see how it did.

Oxide and Friends
Engineering Rigor in the LLM Age

And it really did, like it did remarkably well.

Oxide and Friends
Engineering Rigor in the LLM Age

One thing that was really interesting

Oxide and Friends
Engineering Rigor in the LLM Age

And I would, but definitely not perfectly and had some subtle issues that needed to be resolved, but we got those resolved pretty quickly.

Oxide and Friends
Engineering Rigor in the LLM Age

And I think I would say like it had two subtle issues, but it had also did not have a subtle issue that it could, it made a subtle discovery as well.

Oxide and Friends
Engineering Rigor in the LLM Age

And the thing that was really interesting to me about it is I was unleashing it on like a pretty big source base in terms of Illumos.

Oxide and Friends
Engineering Rigor in the LLM Age

And it was really interesting to watch it effectively read block comments to understand how subsystems worked.

Oxide and Friends
Engineering Rigor in the LLM Age

And to understand... And so reading not just code, but also comments.

Oxide and Friends
Engineering Rigor in the LLM Age

And all in all, it was really pretty impressive.

Oxide and Friends
Engineering Rigor in the LLM Age

You know, it definitely understood... I mean, it's... We're talking almost here.

Oxide and Friends
Engineering Rigor in the LLM Age

So it's like, this is not... Like, anything that you have trained on that is...

Oxide and Friends
Engineering Rigor in the LLM Age

the Linux kernel or the BSD kernel is like literally not going to apply.

Oxide and Friends
Engineering Rigor in the LLM Age

It would be very easy for you to create arguments to functions that didn't exist.

Oxide and Friends
Engineering Rigor in the LLM Age

I'm talking about the Kstat facility, which is a facility that doesn't exist.