Ethan He
speaker
722 appearances
1 recordings
1 series
first heard Jun 2026
last heard 1 Jun
Ethan He’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsRecordings per month over the last 12 months — 1 in all, peaking in Jun 2026 with 1.
Appearances
For example, you train a billion images and there's a mapping from the text to the image and the cost to train the same, like a billion text to a billion videos, that's much more expensive because videos naturally have more tokens than images.
Because the diffusion models, their understanding of language purely comes from this mapping.
So if you don't have enough mapping, say if you only train on like 10 million videos or something, you might not see enough language tokens in your training.
So your model does not understand human intention enough.
So that's why you really, you first train this image diffusion models, and then you bootstrap the video model from there.
There are a few different approaches.
Let's say first, you want to just directly use MP4 compression and use that as the tokens for the transformers to train, right?
So...
People actually have tried that, but the main challenge is the latent space for the MP4 tokens were not very comprehensible for the models.
It's extremely hard to train on that.
And there's a
So that's why we created VAEs, which creates more continuous latent space.
So the models can understand that latent space and learn from it much easier.
Even within the VAEs, there are different difficulties of the latent space.
So you can imagine something that the simplest, the most naive VAE is like you have an image.
and you just shuffle all of the images into a vector.
So you don't need to train any of these, right?
But that little space is extremely hard for models to train on top of.
That's why there's some debate on how do you compress the tokens.
So you mentioned you can compress frame by frame.
Showing 141–160 of 722 · page 8 of 37
← Previous
Next →