Tim Sweeney
speaker
640 appearances
1 recordings
1 series
first heard Apr 2025
last heard Apr 2025
Tim Sweeney’s voice in public audio — every appearance, attributed to the second.
Trend
recordings per month · last 12 monthsNo recordings in the last 12 months.Older appearances are listed below; set an alert to hear about the next one.
Appearances
One part is capturing humans, and so they've got really advanced dedicated hardware that puts a human in a capture sphere with dozens of cameras in them taking high resolution, high frame rate video of them as they go through a range of motions.
And then capturing the human face is complicated because the nuanced detail of our faces and how all the muscles and sinews and fat work together to give us different expressions. So it's not only about the shape of a person's face, but it's also about the entire range of motion that they might go through.
Capturing one human requires a few hours of capture work in a dedicated environment like that, then thousands of hours of processing work to capture one. a precise and real-time replicatable version of that human in the environment. And so one of the things that's done is just capturing an actor or actress in the real world and then using them in a video game.
But the much more interesting thing going on is capturing thousands of humans to form a data set whose goal is to encompass the entire range of faces in all of humanity. So going around every culture, every continent, every age, and every face of variety, and capturing representative people. So the entire range of faces is represented.
And then being able to combine and merge those together to enable recreating an arbitrary face that the system's never seen before. So, you know, one of the ideas is capture giant amounts of this high-precision data, and then you use it to reconstruct a face at a consumer level.
Like maybe, you know, take an iPhone photo of somebody's face and then capture a very accurate depiction of that, not by synthesizing it then and there on that device, but by combining all the known details of human faces to accurately capture the most accurate representation of that image. So that's the data problem. There's a lot of other problems of computer graphics.
There's technology for rendering hair, which is really hard because you can't render every... Again, we know the laws of physics. It would be easy to just render every hair. It would just be a billion times too slow. So you need approximations that capture the net effect of hair on rendering and on pixels without calculating every single interaction of every light with every strand of hair.
That's one part of it. There's detailed features for different parts of faces. There's subsurface scattering because... We think of humans as opaque, but really our skin as light travels through it. It's not completely opaque, and the way in which light travels through skin has a huge impact on our appearance. This is why there's no way you can paint a mannequin to look realistic for a human.
It's just a solid surface, and we'll never have the sort of detail you see.
That's right. Getting faces right requires the interplay of literally dozens of different systems and aspects of computer graphics, and if any one of them is wrong, your eye is completely drawn to that, and you find it on the wrong side of Uncanny Valley. So the level of perfection needed in this area is vastly, vastly higher than world rendering or grass or any of these other things.
If the shadow is on a on a work of architecture slightly wrong, you're pretty perfect game, actually. Your brain doesn't really care that much, but if anything wrong with the human, it's totally jarring.
There are two main tools. Compared to the old days where every face was created by hand by an artist from scratch, one is the MetaHuman Creator tool for creating faces, where you have a huge number of parameters you can adjust to create a unique human by adjusting all the different capabilities of them.
You can then get that out of MetaHuman Creator into Unreal Engine, and then you can add all kinds of computer graphics features there in the engine. You could add clothing using the cloth simulation system, and you can adjust the hair and all these other parameters on the thing.
And then there's MetaHuman Animator, a tool for animating a human based on a facial capture, which can be done on a device as simple as like an iPhone. and transfers the captured animation to the human you want, which is not straightforward.
If the actor has one face shape and the character on screen has another face shape, the translation that needs to be done from the actor to the face is actually really sophisticated and non-obvious. And if you just applied it literally, then it would be completely wrong from your point of view. So those are the main tools that people are using now.
And then, went in the Unreal Engine, then you have a face and you can do absolutely anything you want to it. And you could also, if you decide to go outside of the metahuman geometry pipeline, you could build your own face, like any creature of any sort, and then use the animation tools to animate it.
But yeah, this is 30 years into a project that's probably like 50 years in total to get to absolute photorealism and controllability for absolutely everything. So there's vast amounts of work still to do. And we don't feel like we've solved the problem at all. We've just given artists a big productivity multiplier and a quality multiplier. But this is not in a state that we would say is done.
Yeah, that's right. When you have an artist or actor in your studio and you're recording a specific performance, you can just capture their facial motion and apply it. But if all you have is a voice recording or you're generating a voice recording or it's parametric or procedural or AI generated, yeah, then you need...
the system to translate that speech, not only to movement of the mouth and lips, but also to facial expressions and the whole intent. When we're speaking, it's our whole face that's active and emoting in different ways. It's not just a mechanical motion of the pieces.
Lumen is a system for global illumination, meaning it's supposed to calculate the interaction of light with the entire scene in a way that mimics reality. The first generation of engines that did lighting just said, well, the light casts light and the surfaces it hits are lit and the surfaces it doesn't directly hit are dark. And that's just all the techniques we have. So you'd have...
Showing 241–260 of 640 · page 13 of 32
← Previous
Next →