Justin Johnson
speaker
345 appearances
1 recordings
1 series
first heard Nov 2025
last heard 25 Nov
Justin Johnson’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in Nov 2025 with 1.
Appearances
It means the technology actually worked.
But that means the expectations around what we should be doing as academics shifts a little bit.
And it shouldn't be about trying to train the biggest model and scaling up the biggest thing.
It should be about trying wacky ideas and new ideas and crazy ideas, most of which won't work.
And I think there's a lot to be done there.
If anything, I'm worried that too many people in academia are hyper-focused on this no-
notion of trying to pretend like we can train the biggest models or or treating it as almost a vocational training program to then graduate and go to a big lab and then be able to play with all the GPUs.
I think there's just so much crazy stuff you can do around like new algorithms, new architectures, like new systems that, you know, there's a lot you can do as as one person.
Oh, like I had I had this idea that I kept pitching to my students uh at at Michigan, which is that I I really like hardware and I really like like new kinds of hardware coming online.
Um and in some sense the the emergence of the neural networks that you use we use today and transformers are really based around matrix multiplication, because matrix multiplication fits really well with GPUs.
But if we think about how GPUs are gonna scale how how hardware is likely to scale in the future, I don't think the current system that we have, like the GPUs
Like hardware design is gonna scale infinitely.
And that we start to see that even now, that like the unit of compute is not the single device anymore.
It's this whole cluster of devices.
So if you imagine a node.
Node.
Yeah, it's a whole node or a whole cluster.
It's full mode or something.
But the way we talk about neural networks is still as if they are a monolithic thing that could be coded like in one GPU in PyTorch.
Um, but then in practice, they get distributed over thousands of devices.
Showing 41–60 of 345 · page 3 of 18
← Previous
Next →