Sebastian Raschka
speaker
1,024 appearances
1 recordings
1 series
first heard Feb 2026
last heard 1 Feb
Sebastian Raschka’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in Feb 2026 with 1.
Appearances
And you see all the components in like a human readable, I don't know, 100 lines of config file.
And then you start, let's say, with your GPT-2 model and add these things, you know.
And the cool thing here is you can then load the pre-trained weights and see if they work in your model.
And you want to match the same output that you get with a transformer model.
And then you can use that as a basically as a verifiable reward to make your architecture correct.
And then it's kind of sometimes it takes me a day to with almost three.
The challenge was rope for the position embeddings.
They had a yarn extension and there was some custom work.
scaling there and i couldn't quite match these things and in this struggle you kind of understand things but the cool thing is at the end you know you have it correct because you can unit test it you can check against the reference implementation and i think that's maybe one of the best ways to learn really like to basically reverse engineer something yeah
Yeah, I think you can't try to do it all because it would be very overwhelming and you would burn out if you try to keep up with everything.
For me, for example, I haven't kept up with computer vision in a long time, just focused on LMs.
But coming back to your book, for example, I think this is also a really great book and a really good bang for the buck because you want to learn about RLHF.
I wouldn't go out there and read RLHF papers because you would be spending two years.
Makes sense.
I do something like that.
I did that the other day for video games, for example.
I sometimes, for my pastime, play video games.
Like, I like video games with puzzles.
So, you know, like Zelda and Metroid.
And there's this new game where I got stuck.
Showing 541–560 of 1,024 · page 28 of 52
← Previous
Next →