Menu
Sign In Search Podcasts Libraries Charts People & Topics Add Podcast API Blog Pricing

David Crespo

πŸ‘€ Speaker
169 total appearances

Appearances Over Time

Podcast Appearances

Oxide and Friends
Engineering Rigor in the LLM Age

The two months ago being several years ago, especially in the really complicated one was that, you know, something was it was a mutex lock that was like not being taken at the right time.

Oxide and Friends
Engineering Rigor in the LLM Age

And so there was like a conflict in.

Oxide and Friends
Engineering Rigor in the LLM Age

And so, you know, reasoning about that was pretty tough for me, not understanding how the code worked.

Oxide and Friends
Engineering Rigor in the LLM Age

But it was pretty impressive that the model was able to see it, you know, like this is where you should have taken a lock and you didn't.

Oxide and Friends
Engineering Rigor in the LLM Age

It's worth looking at Ngozi specifically.

Oxide and Friends
Engineering Rigor in the LLM Age

They have a quite clear LLM disclosure policy.

Oxide and Friends
Engineering Rigor in the LLM Age

So Mitchell has been pretty open that he uses LLM's tooling, but they really want upfront disclosure.

Oxide and Friends
Engineering Rigor in the LLM Age

So they made it easy by telling me exactly what to do.

Oxide and Friends
Engineering Rigor in the LLM Age

I was more worried about sort of the embarrassment if my issue was fake for me to be like one of those guys posting issues that are fake that the LLM told them, you know, was a bug.

Oxide and Friends
Engineering Rigor in the LLM Age

Yeah, well, I thought this was such a clear-cut case where it was obviously not my expertise that was operative here because I didn't have any.

Oxide and Friends
Engineering Rigor in the LLM Age

You know, I had, like, there was a high level, like, I could tell that it felt legitimate.

Oxide and Friends
Engineering Rigor in the LLM Age

And I think there may have been one or two things where it came up with something that I was like, that doesn't sound real.

Oxide and Friends
Engineering Rigor in the LLM Age

But, you know, the amount of guidance that I actually provided in the process was a very small proportion of what actually took place.

Oxide and Friends
Engineering Rigor in the LLM Age

That was what I think felt legitimate.

Oxide and Friends
Engineering Rigor in the LLM Age

unsettling about it.

Oxide and Friends
Engineering Rigor in the LLM Age

And, you know, the guidance that I did provide also didn't feel, you know, that ineffable human taste that people love to attribute to themselves.

Oxide and Friends
Engineering Rigor in the LLM Age

It really wasn't that.

Oxide and Friends
Engineering Rigor in the LLM Age

It was like finding the rust port of the stack trace, you know, symbolicate or whatever.

Oxide and Friends
Engineering Rigor in the LLM Age

Well, if it had turned out to be fake, I wouldn't have been.

Oxide and Friends
Engineering Rigor in the LLM Age

If my diagnosis was wrong, then that was just creating work for them.