Walden Yan
speaker
387 appearances
1 recordings
1 series
first heard May 2026
last heard 28 May
Walden Yan’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in May 2026 with 1.
Appearances
This can get arbitrarily hard.
Maybe you have to be an admin, maybe a certain thing has to be feature flagged on, maybe you have to run two sessions and then send us a very specific word into one of them to trigger specific behavior.
And figuring out how do you do that requires a lot of code-based context, requires a lot of orchestration that we've specifically done.
And in some cases, we found that actually no one frontier model can actually do this full end-to-end task itself.
We've seen cases where we actually had to orchestrate different frontier models together to kind of solve this problem together.
That is where we spend most of our time when we think about this testing problem, not so much the computer use part.
Computer use, for what it's worth, has gotten a lot better with recent models, and it's made that part of the job certainly easier.
Having evals for all these as well is something that takes a while to build up, and having the evals be right is tricky as well.
Do you ever see clients who are building their own agents have to start standing up evals to make sure things don't regress?
Maybe in your future.
Yeah.
Yeah, the annotations at the bottom was also a big difference for me when I added those.
Yeah, it's just like, what am I looking at?
What are you trying to demonstrate?
Exactly.
There's a surprising long tail of small details that ends up making a big difference for this kind of end metric of how fast you actually merge the code in.
one experience that we spent a lot of time tuning early on was what is the right experience on GitHub for these tools?
Because I think most tools out there when you build the agent, you'll think about like it'll create the PR for you.
We try to take that a step further and say, oh, like, what if we actually made sure you could interact, Devin, with Devin directly on GitHub?
And so we made sure that you can comment on GitHub and Devin would actually receive those comments and address them back.
Showing 101–120 of 387 · page 6 of 20
← Previous
Next →