Andrew (Google Beam lead)

I didn't think it's possible.

Hey, Lex.

My name is Andrew.

I lead the Google Beam team and we're going to be excited to show you a demo.

We're going to show you, I think, a glimpse of something new.

So that's the idea.

A way to connect, a way to feel present from anywhere with anybody you care about.

Here is Google Beam.

This is a development platform that we've built.

So there's a prototype here of Google Beam.

There's one right down the hallway.

I'm going to go down and turn that on in a second.

We're going to experience it together.

We'll be back in the same room.

Whoa, okay.

Here we are.

This is real.

Good to see you.

This is Google Beam.

We're trying to make it feel like you and I could be anywhere in the world, but when these magic windows open, we're back together.

I see you exactly the same way you see me.

It's almost like we're sitting at the table sharing a table together.

I could learn from you, talk to you, share a meal with you, get to know you.

So you could feel the depth of this.

Yeah, great to meet you.

For me, it looks real to you.

We quickly believe, once we're in Veeam, that we're just together.

You settle into it, you're naturally attuned to seeing the world like this, and you just get used to seeing people this way.

But literally from anywhere in the world with these magic screens.

This is incredible.

It's a neat technology.

You know, it kind of fades it from my room into yours.

Of course, yeah.

Wow.

It feels like you... Try this.

Try giving me a high five.

And there's almost a sensation of being in touch.

Yeah.

Almost feel.

Because you're so attuned to, you know, that should be a high five, it feeling like you could connect with somebody that way.

So it's kind of a magical experience.

How much does it cost?

We've got a lot of companies testing it.

We just announced that we're going to be bringing it to offices soon as a set of products.

We've got some companies helping us build these screens.

But eventually, I think this will be in almost every screen.

Yeah.

The audio is spatialized.

So if I'm talking from here, of course it sounds like I'm talking from here.

You know, if I move to the other side of the room.

Wow.

So these little subtle cues, these really matter to bring people together.

All the nonverbals, all the emotion, the things that are lost today.

Here it is.

We put it back into the system.

You pulled this off.

Yeah, we've got a bunch of things.

Let me show you a couple kind of cool things.

Let's do a little bit of work together.

Maybe we could...

critique one of your latest.

You and I work together, so of course we're in the same room, but with this superpower, I can bring other things in here with me.

It's nice.

It's like we could sit together, we could watch something, we could work.

We've shared meals as a team together in this system, but once you do the presence aspect of this, you want to bring some other superpowers to it.

Yeah, yeah, exactly.

I've got some slides I'm working on.

You know, maybe you could help me with this.

Keep your eyes on me for a second.

I'll slide back into the center.

I didn't really move, but the system just kind of puts us in the right spot and knows where we need to be.

Kind of morphs the room to put things in the spot that they need to be in.

Everything has a place in the room.

Everything has a sense of presence or spatial consistency.

And that kind of makes it feel like we're together with us and other things.

Let me tell you how this works.

You probably already have the premise of it, but there's two things, two really hard things that we put together.

One is an AI video model.

So there's a set of cameras.

You asked kind of about those earlier.

There's six color cameras, just like webcams that we have today, taking video streams and feeding them into our AI model and turning that into a 3D video of you and I. It's effectively a light field.

So it's kind of an interactive 3D video that you can see from any perspective.

That's transmitted over to the second thing, and that's a light field display.

And it's happening bidirectionally.

I see you and you see me both in our light field displays.

These are effectively flat televisions or flat displays, but they have the sense of dimensionality, depth, size is correct.

You can see shadows and lighting are correct.

And everything's correct from your vantage point.

So if you move around ever so slightly and I hold still,

you see a different perspective here.

You see kind of things that were occluded become revealed.

You see shadows that, you know, move in the way they should move.

All of that's computed and generated using our AI video model for you.

It's based on your eye position.

Where does the right scene need to be placed in this light field display for you just to feel present?

No, no, I hope not.

I think it's you and I together, real time.

That's what you need for real communication.

And at a quality level, this is awesome.

realistic.

Yeah.

Let me, let me kind of show you.

So if, if she enters the room with us, you can see her, you can see me.

And if we had more people, you eventually lose the sense of presence.

You kind of shrink people down, you lose the sense of scale.

So think of it as the window fits a certain number of people.

If you want to fit a big group of people, you know, the boardroom or the big room, you need like a much wider window.

If you want to see, you know, just grandma and the kids, you can do smaller windows.

So everybody has a seat at the table or everybody has a sense of where they belong and there's kind of a sense of presence that's obeyed.

If you have too many people, you kind of go back to like 2D metaphors that we're used to.

People in tiles placed anywhere.

I mean, I see you without being scanned.

So it's just so much easier if you don't have to wear anything.

You don't have to pre-scan.

You just...

Do it the way it's supposed to happen without anybody having to learn anything or put anything on.

It's just vision.

It's video.

Yeah, we're not trying to kind of make an approximation of you because everything you do every day matters.

You know, I cut myself shaving.

I put on a pin.

All the little kind of, you know, aspects of you, those just happen.

We don't have the time to scan or kind of capture those or dress avatars.

We kind of appear as we appear.

And so all that's transmitted truthfully at the top of it.