Sebastian Raschka

speaker

1,024 appearances 1 recordings 1 series first heard Feb 2026 last heard 1 Feb

Sebastian Raschka’s voice in public audio — every appearance, attributed to the second.

Trend

recordings per month · last 12 months

Recordings per month over the last 12 months — 1 in all, peaking in Feb 2026 with 1.

newest first · ▶ plays the moment

So the Hugging Face transformer library is great.

But if you want to learn about LLMs, I think that's not the best place to start because the code is so complex, because it has so full...

It has to fit so many use cases.

Also, some people use it in production.

It has to be really sophisticated, and it's really intertwined and really hard.

It's not linear to read.

And all Frontier Labs that have open weight models have a Hugging Face Transformers version of it, like from DeepSeq to GPT-OSS.

That's like the canonical weight that you can load there.

But again, also even Transformers, the library is not used in production.

People use then SGLang or VLLM and it adds another layer of complexity.

So it's a one library that tries to implement a lot of LLMs.

And so you have a huge code base, basically.

It's like huge.

It's like, it's, I don't know, maybe a million, hundreds of thousands of lines of code.

And it's like understanding the part that you want to understand is finding the needle in the haystack.

But what's beautiful about it is you have a working implementation.

And so you can work backwards from it.

What I would recommend doing, or what I also do is, if I want to understand, for example, how OMO3 is implemented,

I would look at the weights in the model hub, the config file, and then you can see, oh, they used so many layers.

They use, let's say, group query attention or multi-head attention in that case.

Showing 521–540 of 1,024 · page 27 of 52