Eiso Kant
speaker
1,158 appearances
1 recordings
1 series
first heard Jul 2026
last heard 23 Jul
Eiso Kant’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in Jul 2026 with 1.
Appearances
Maybe we should have just forked this repo.
But then when we solved it, I still remember at the time we were like five people in the company.
When we solved it, we're like, oh, we can do things.
Like if we were just willing to work hard.
And I think that culture with a very strong engineering bias has kind of helped us like get to where we were.
And so there's this kind of notion of open source and talent and these things.
I think we just took different decisions from a different starting point.
And I think we are lucky.
I do want to definitely call it lucky.
And also a lot of our team that now like that's starting to kind of show up in results.
So the bug, you're going to test my memory here.
But I think I can recall.
So if you look at, so if you take like Atom as an optimizer, you have epsilon, which is like in the denominator.
Exactly, in the denominator.
And at the time, if I recall, you looked at like the early Lama papers and things like that.
people were juicing epsilon like quite a bit like they were like adding i don't know if it was e minus four or whatever like a high value for epsilon and if you think about this during training it's kind of like a bit weird and counterintuitive that we're adding noise to our optimizer by just adding effectively like a random number in the denominator uh right like behind the decimal point
And I don't recall the exact bug, but what I remember is once we solved it, we no longer had to choose epsilon as much as like was happening in the Lama paper and other places.
And it was like one of those fundamental moments where we had trusted this paper that was out there.
And we're like, oh, no, it has to be this way.
It has to have this high value of epsilon.
Showing 141–160 of 1,158 · page 8 of 58
← Previous
Next →