Marc Raibert

We use the springiness in our legs, our muscles and our tendons and things like that. As part of the story, the energy circulates. We don't just throw it away every time. I'm not sure I understood all that when I first thought, but I definitely got inspired to say, let's try the opposite. I didn't have a clue as to how to make a hopping robot work, not balance in 3D.

We use the springiness in our legs, our muscles and our tendons and things like that. As part of the story, the energy circulates. We don't just throw it away every time. I'm not sure I understood all that when I first thought, but I definitely got inspired to say, let's try the opposite. I didn't have a clue as to how to make a hopping robot work, not balance in 3D.

In fact, when I started, it was all just about the energy of bouncing. And I was going to have a springy thing in the leg and some actuator so that you could get an energy regime going of bouncing. And the idea that balance was an important part of it didn't come until a little later. And then I made the one-legged, the pogo stick robots. Now I think that we need to do that in manipulation.

In fact, when I started, it was all just about the energy of bouncing. And I was going to have a springy thing in the leg and some actuator so that you could get an energy regime going of bouncing. And the idea that balance was an important part of it didn't come until a little later. And then I made the one-legged, the pogo stick robots. Now I think that we need to do that in manipulation.

If you look at robot manipulation, we, a community, has been working on it for 50 years. We're nowhere near human levels of manipulation. I mean, we can, you know, it's come along, but I think it's all too safe.

If you look at robot manipulation, we, a community, has been working on it for 50 years. We're nowhere near human levels of manipulation. I mean, we can, you know, it's come along, but I think it's all too safe.

And I think trying to break out of that safety thing of static grasping, you know, if you look at a lot of work that goes on, it's about the geometry of the part, and then you figure out how to move your hand so that you can position it with respect to that, and then you grasp it carefully, and then you move it. That's not anything like how people and animals work. We juggle in our hands.

And I think trying to break out of that safety thing of static grasping, you know, if you look at a lot of work that goes on, it's about the geometry of the part, and then you figure out how to move your hand so that you can position it with respect to that, and then you grasp it carefully, and then you move it. That's not anything like how people and animals work. We juggle in our hands.

We hog multiple objects and can sort them. Now, to be fair, being more aggressive is going to mean things aren't going to work very well for a while. It's a longer-term approach to the problem. That's just theory now. Maybe that won't pay off, but that's how I'm trying to think about it, trying to encourage our group to go at it.

We hog multiple objects and can sort them. Now, to be fair, being more aggressive is going to mean things aren't going to work very well for a while. It's a longer-term approach to the problem. That's just theory now. Maybe that won't pay off, but that's how I'm trying to think about it, trying to encourage our group to go at it.

I don't know if you know my friend Matt Mason, who is the director of the Robotics Institute at Carnegie Mellon. And we go back to graduate school together. But he analyzed... a movie of Julia Childs doing a cooking thing. And she did, I think he said something like there were 40 different ways that she handled a thing and none of them was grasping.

I don't know if you know my friend Matt Mason, who is the director of the Robotics Institute at Carnegie Mellon. And we go back to graduate school together. But he analyzed... a movie of Julia Childs doing a cooking thing. And she did, I think he said something like there were 40 different ways that she handled a thing and none of them was grasping.

She would nudge, roll, flatten with her knife, things like that, and none of them was grasping.

She would nudge, roll, flatten with her knife, things like that, and none of them was grasping.

First of all, the Leg Lab actually started at Carnegie Mellon. I was a professor there starting in 1980, about 1986. And so that's where the first hopping machines were built. I guess we got the first one working in about 1982, something like that. That was a simplified one. Then we got a three-dimensional one in 1983.

First of all, the Leg Lab actually started at Carnegie Mellon. I was a professor there starting in 1980, about 1986. And so that's where the first hopping machines were built. I guess we got the first one working in about 1982, something like that. That was a simplified one. Then we got a three-dimensional one in 1983.

The quadruped that we built at the Leg Lab, the first version, was built in about 1984 or 5 and really only got going about 86 or so. It took years of development to get it to work.

The quadruped that we built at the Leg Lab, the first version, was built in about 1984 or 5 and really only got going about 86 or so. It took years of development to get it to work.

Well, I'm going to start on the, not the technical side, but the, I guess we could call it the motivational side or the funding side. So before Carnegie Mellon, I was actually at JPL, at the Jet Propulsion Lab for three years.

Well, I'm going to start on the, not the technical side, but the, I guess we could call it the motivational side or the funding side. So before Carnegie Mellon, I was actually at JPL, at the Jet Propulsion Lab for three years.

And while I was there, I connected up with Ivan Sutherland, who is sometimes regarded as the father of computer graphics because of work he did both at MIT and then University of Utah and Evans and Sutherland.

And while I was there, I connected up with Ivan Sutherland, who is sometimes regarded as the father of computer graphics because of work he did both at MIT and then University of Utah and Evans and Sutherland.

Anyway, um, I got to know him and at one point he said, uh, he encouraged me to, uh, do some kind of project, uh, at Caltech, even though I was at JPL, you know, those are kind of related institutions. And, uh, So I thought about it, and I made up a list of three possible projects. And I purposely made the top one and the bottom one really boring sounding.

Anyway, um, I got to know him and at one point he said, uh, he encouraged me to, uh, do some kind of project, uh, at Caltech, even though I was at JPL, you know, those are kind of related institutions. And, uh, So I thought about it, and I made up a list of three possible projects. And I purposely made the top one and the bottom one really boring sounding.

And in the middle, I put Pogo Stick Robot. When he looked at it, Ivan is a brilliant guy, brilliant engineer, and a real cultivator of people. He looked at it and knew right away what the thing that was worth doing. He had an endowed chair, so he had about $3,000 that he gave me to build the first model for

And in the middle, I put Pogo Stick Robot. When he looked at it, Ivan is a brilliant guy, brilliant engineer, and a real cultivator of people. He looked at it and knew right away what the thing that was worth doing. He had an endowed chair, so he had about $3,000 that he gave me to build the first model for

which I went to the shop and with my own hands made a first model, which didn't work and was just a beginning shot at it. Ivan and I took that to Washington. In those days, you could just walk into DARPA and walk down the hallway and see who's there. Ivan, who had been there in his previous life, We walked around and we looked in the offices. Of course, I didn't know anything.

which I went to the shop and with my own hands made a first model, which didn't work and was just a beginning shot at it. Ivan and I took that to Washington. In those days, you could just walk into DARPA and walk down the hallway and see who's there. Ivan, who had been there in his previous life, We walked around and we looked in the offices. Of course, I didn't know anything.

I was basically a kid, but Ivan knew his way around. We found Craig Fields in his office. Craig later became the director of DARPA, but in those days, he was a program manager. We went in. I had a little Samsonite suitcase. We opened and it had just the skeleton of this one-legged hopping robot. We showed it to him. And you could almost see the drool going down his chin of excitement.

I was basically a kid, but Ivan knew his way around. We found Craig Fields in his office. Craig later became the director of DARPA, but in those days, he was a program manager. We went in. I had a little Samsonite suitcase. We opened and it had just the skeleton of this one-legged hopping robot. We showed it to him. And you could almost see the drool going down his chin of excitement.

And he sent me $250,000. He said, okay, I want to fund this. And I was between institutions. I was just about to leave JPL, and I hadn't decided yet where I was going next. And then when I landed at CMU, he sent $250,000, which in 1980 was a lot of research money.

And he sent me $250,000. He said, okay, I want to fund this. And I was between institutions. I was just about to leave JPL, and I hadn't decided yet where I was going next. And then when I landed at CMU, he sent $250,000, which in 1980 was a lot of research money.

Like, all the fundamentals are there. Yeah, I mean, I think that was the motivation to try and get more at the fundamentals of how animals work. But the idea that it would result in, you know, machines that were anything like practical... like we're making now. That wasn't anywhere in my head, no.

Like, all the fundamentals are there. Yeah, I mean, I think that was the motivation to try and get more at the fundamentals of how animals work. But the idea that it would result in, you know, machines that were anything like practical... like we're making now. That wasn't anywhere in my head, no.

As an academic, I was mostly just trying to do the next thing, make some progress, impress my colleagues if I could.

As an academic, I was mostly just trying to do the next thing, make some progress, impress my colleagues if I could.

Well, in the very early days, I needed some better engineering than I could do myself. And I hired Ben Brown. We each had our way of contributing to the design. And we came up with a thing that could start to work. I had some stupid ideas about how the actuation system should work. And we sorted that out.

Well, in the very early days, I needed some better engineering than I could do myself. And I hired Ben Brown. We each had our way of contributing to the design. And we came up with a thing that could start to work. I had some stupid ideas about how the actuation system should work. And we sorted that out.

It wasn't that hard to make it balanced once you get the physical machine to be working well enough and have enough control over the degrees of freedom. And then we very quickly, you know, we started out by having it floating on an inclined air table. And then that only gave us like six foot of travel.

It wasn't that hard to make it balanced once you get the physical machine to be working well enough and have enough control over the degrees of freedom. And then we very quickly, you know, we started out by having it floating on an inclined air table. And then that only gave us like six foot of travel.

So once it started working, we switched to a thing that could run around the room on another device. It's hard to explain these without you seeing them, but you probably know what I'm talking about, a planarizer. And then the next big step was to make it work in 3D, which that was really the scary part.

So once it started working, we switched to a thing that could run around the room on another device. It's hard to explain these without you seeing them, but you probably know what I'm talking about, a planarizer. And then the next big step was to make it work in 3D, which that was really the scary part.

With these simple things, you know, people had inverted pendulums at the time for years and they could control them by driving a cart back and forth. But could you make it work in three dimensions while it's bouncing and all that? But it turned out, you know, not to be that hard to do, at least at the level of performance we achieved at the time.

With these simple things, you know, people had inverted pendulums at the time for years and they could control them by driving a cart back and forth. But could you make it work in three dimensions while it's bouncing and all that? But it turned out, you know, not to be that hard to do, at least at the level of performance we achieved at the time.

Yes.

Yes.

The simple story is that there's three things going on. There's something making it bounce. We had a system that was estimating how high the robot was off the ground. Using that, there's energy that can be in three places in a pogo stick. One is in the spring, one is in the altitude, and the other is in the velocity. And so when at the top of the hop, it's all in the height.

The simple story is that there's three things going on. There's something making it bounce. We had a system that was estimating how high the robot was off the ground. Using that, there's energy that can be in three places in a pogo stick. One is in the spring, one is in the altitude, and the other is in the velocity. And so when at the top of the hop, it's all in the height.

And so you could just measure how high you're going and thereby have an idea of a lot about the cycle, and you could decide whether to put more energy in or less. So that is one element. Then there's a part that you decide where to put the foot. And if you think when you're landing on the ground with respect to the center of mass, so if you think of a pole vaulter,

And so you could just measure how high you're going and thereby have an idea of a lot about the cycle, and you could decide whether to put more energy in or less. So that is one element. Then there's a part that you decide where to put the foot. And if you think when you're landing on the ground with respect to the center of mass, so if you think of a pole vaulter,

The key thing the pole vaulter has to do is get its body to the right place when the pole gets stuck. If they're too far forward, they kind of get thrown backwards. If they're too far back, they go over. And what they need to do is get it so that they go mostly up to get over the thing. And high jumpers is the same kind of thing.

The key thing the pole vaulter has to do is get its body to the right place when the pole gets stuck. If they're too far forward, they kind of get thrown backwards. If they're too far back, they go over. And what they need to do is get it so that they go mostly up to get over the thing. And high jumpers is the same kind of thing.

So there's a calculation about where to put the foot and we did something, you know, relatively simple. And then there's a third part to keep the body at an attitude that's upright. Cause if it gets too far, you know, you could hop and just keep rotating around, but if it gets too far, then you run out of motion of the joints at the hips. So you have to do that.

So there's a calculation about where to put the foot and we did something, you know, relatively simple. And then there's a third part to keep the body at an attitude that's upright. Cause if it gets too far, you know, you could hop and just keep rotating around, but if it gets too far, then you run out of motion of the joints at the hips. So you have to do that.

And we did that by applying a torque between the legs and the body. Every time the foot's on the ground, you only can do it while the foot's on the ground in the air. You know, it, it, the physics don't work out.

And we did that by applying a torque between the legs and the body. Every time the foot's on the ground, you only can do it while the foot's on the ground in the air. You know, it, it, the physics don't work out.

Well, you're asking an interesting question because... In those days, we didn't actually optimize things. And they probably could have gone much further than we did and then had higher performance. And we just kind of got a sketch of a solution and worked on that.

Well, you're asking an interesting question because... In those days, we didn't actually optimize things. And they probably could have gone much further than we did and then had higher performance. And we just kind of got a sketch of a solution and worked on that.

And then in years since, some people working for us, some people working for others, people came up with all kinds of equations or algorithms for how to do a better job, be able to go faster. One of my students worked on getting things to go faster. Another one worked on... climbing over obstacles. Because when you're running, on the open ground, it's one thing.

And then in years since, some people working for us, some people working for others, people came up with all kinds of equations or algorithms for how to do a better job, be able to go faster. One of my students worked on getting things to go faster. Another one worked on... climbing over obstacles. Because when you're running, on the open ground, it's one thing.

If you're running up a stair, you have to adjust where you are. Otherwise, things don't work out right. You land your foot on the edge of the step. So there's other degrees of freedom to control if you're getting to more realistic, practical situations.

If you're running up a stair, you have to adjust where you are. Otherwise, things don't work out right. You land your foot on the edge of the step. So there's other degrees of freedom to control if you're getting to more realistic, practical situations.

Probably the smartest thing I ever did is to find the other people. I mean, when I look at it now, I look at Boston Dynamics and all the really excellent engineering there, people who really make stuff work. I'm only the dreamer.

Probably the smartest thing I ever did is to find the other people. I mean, when I look at it now, I look at Boston Dynamics and all the really excellent engineering there, people who really make stuff work. I'm only the dreamer.

Did you experience a lot of people around you kind of... I don't know if they doubted whether it was possible, but I think they thought it was a waste of time.

Did you experience a lot of people around you kind of... I don't know if they doubted whether it was possible, but I think they thought it was a waste of time.

I think for a lot of people. I think it's been both, though. Some people, I felt like they were saying, oh, why are you wasting your time on this stupid problem? But then I've been at many things where people have told me it's been an inspiration to go out and attack these harder things. And I think it has turned out, I think legged locomotion has turned out to be a useful thing.

I think for a lot of people. I think it's been both, though. Some people, I felt like they were saying, oh, why are you wasting your time on this stupid problem? But then I've been at many things where people have told me it's been an inspiration to go out and attack these harder things. And I think it has turned out, I think legged locomotion has turned out to be a useful thing.

I mean, at first, I wasn't an enthusiast for the humanoids because, again, it goes back to saying, what's the functionality? And the form wasn't as important as the functionality. And also, there's an aspect to humanoid robots that's about functionality. all about the cosmetics, where there isn't really other functionality, and that kind of is off-putting for me.

I mean, at first, I wasn't an enthusiast for the humanoids because, again, it goes back to saying, what's the functionality? And the form wasn't as important as the functionality. And also, there's an aspect to humanoid robots that's about functionality. all about the cosmetics, where there isn't really other functionality, and that kind of is off-putting for me.

As a roboticist, I think the functionality really matters. So probably that's why I avoided humanoid robots to start with. I'll tell you, after we started working on him, you could see the connection and the impact with other people, whether they're lay people or even other technical people.

As a roboticist, I think the functionality really matters. So probably that's why I avoided humanoid robots to start with. I'll tell you, after we started working on him, you could see the connection and the impact with other people, whether they're lay people or even other technical people.

There's a special thing that goes on, even though most of the humanoid robots aren't that much like a person.

There's a special thing that goes on, even though most of the humanoid robots aren't that much like a person.

I'll tell you, I go around giving talks and take Spot to a lot of them. And it's amazing. The media likes to say that they're terrifying and that people are afraid. And And YouTube commenters like to say that it's frightening.

I'll tell you, I go around giving talks and take Spot to a lot of them. And it's amazing. The media likes to say that they're terrifying and that people are afraid. And And YouTube commenters like to say that it's frightening.

But when you take a spot out there, now maybe it's self-selecting, but you get a crowd of people who want to take pictures, want to pose for selfies, want to operate the robot, want to pet it, want to put clothes on it. It's amazing.

But when you take a spot out there, now maybe it's self-selecting, but you get a crowd of people who want to take pictures, want to pose for selfies, want to operate the robot, want to pet it, want to put clothes on it. It's amazing.

What the connection was is that at that point, Boston Dynamics was mostly a physics-based simulation company. When I left MIT to start Boston Dynamics, there was a few years of overlap, but the concept wasn't to start a robot company. The concept was to use this dynamic simulation tool that we developed to do robotics for other things.

What the connection was is that at that point, Boston Dynamics was mostly a physics-based simulation company. When I left MIT to start Boston Dynamics, there was a few years of overlap, but the concept wasn't to start a robot company. The concept was to use this dynamic simulation tool that we developed to do robotics for other things.

But working with Sony, we got back into robotics by doing the AIBO Runner. We made some tools for programming Curio, which was a humanoid this big that could do some dancing and other kinds of fun stuff. And I don't think it ever reached the market, even though they did show it. When I look back, I say that we got us back where we belonged. Yeah.

But working with Sony, we got back into robotics by doing the AIBO Runner. We made some tools for programming Curio, which was a humanoid this big that could do some dancing and other kinds of fun stuff. And I don't think it ever reached the market, even though they did show it. When I look back, I say that we got us back where we belonged. Yeah.

That's right.

That's right.

One of the robots that we built wasn't actually a robot. It was a surgical simulator, but it had force feedback. So it had all the techniques of robotics. And you look down into this... mirror it actually was. And it looked like you were looking down onto the body you were working on. Your hands were underneath the mirror, so they were where you were looking.

One of the robots that we built wasn't actually a robot. It was a surgical simulator, but it had force feedback. So it had all the techniques of robotics. And you look down into this... mirror it actually was. And it looked like you were looking down onto the body you were working on. Your hands were underneath the mirror, so they were where you were looking.

And you had tools in your hands that were connected up to these force feedback devices made by another MIT spin-out, Sensible Technologies. So they made the force feedback device. We attached the tools and we wrote all the software and did all the graphics. So we had 3D computer graphics.

And you had tools in your hands that were connected up to these force feedback devices made by another MIT spin-out, Sensible Technologies. So they made the force feedback device. We attached the tools and we wrote all the software and did all the graphics. So we had 3D computer graphics.

It was in the old days when, this was in the late 90s, when you had a silicon graphics computer that was about this big. It was the heater in the office, basically. And we were doing surgical operations, anastomosis, which was stitching tubes together, tubes like blood vessels or other things in their body. And you could feel and you could see the tissues move. And it was really exciting.

It was in the old days when, this was in the late 90s, when you had a silicon graphics computer that was about this big. It was the heater in the office, basically. And we were doing surgical operations, anastomosis, which was stitching tubes together, tubes like blood vessels or other things in their body. And you could feel and you could see the tissues move. And it was really exciting.

And the idea was to make a trainer to teach surgeons how to do stuff. We built a scoring system because we interviewed surgeons that told us what you're supposed to do and what you're not supposed to do. You're not supposed to tear the tissue. You're not supposed to touch it in any place except for where you're trying to engage. There were a bunch of rules.

And the idea was to make a trainer to teach surgeons how to do stuff. We built a scoring system because we interviewed surgeons that told us what you're supposed to do and what you're not supposed to do. You're not supposed to tear the tissue. You're not supposed to touch it in any place except for where you're trying to engage. There were a bunch of rules.

So we built this thing and took it to a trade show, a surgical trade show. And the surgeons were practically lined up. Well, we kept the score and we posted their scores like on a video game. And those guys are so competitive that they really, really love doing it. And they would come around and they see someone's score was higher there. So they would come back.

So we built this thing and took it to a trade show, a surgical trade show. And the surgeons were practically lined up. Well, we kept the score and we posted their scores like on a video game. And those guys are so competitive that they really, really love doing it. And they would come around and they see someone's score was higher there. So they would come back.

But we figured out shortly after that we thought surgeons were going to pay us to get trained on these things. And the surgeons thought we should pay them in order to, so they could teach us about the thing. And there was no money from the surgeons. And we looked at it and thought, well, maybe we could sell it to hospitals that would teach, train their surgeons.

But we figured out shortly after that we thought surgeons were going to pay us to get trained on these things. And the surgeons thought we should pay them in order to, so they could teach us about the thing. And there was no money from the surgeons. And we looked at it and thought, well, maybe we could sell it to hospitals that would teach, train their surgeons.

And then we said, well, we're this, at the time we were probably a 12-person company or maybe 15 people, I don't remember. There's no way we could go after a marketing activity. You know, the company was all bootstrapped in those years. We never had investors until Google bought us, which was after 20 years. So we didn't have any resources to go after hospitals.

And then we said, well, we're this, at the time we were probably a 12-person company or maybe 15 people, I don't remember. There's no way we could go after a marketing activity. You know, the company was all bootstrapped in those years. We never had investors until Google bought us, which was after 20 years. So we didn't have any resources to go after hospitals.

So at one day, Rob and I were looking at that and we said, we'd built another simulator for knee arthroscopy. And we said, this isn't going to work. And we killed it. And we moved on, and that was really a milestone in the company because we sort of understood who we were and what would work and what wouldn't, even though technically it was really a fascinating thing.

So at one day, Rob and I were looking at that and we said, we'd built another simulator for knee arthroscopy. And we said, this isn't going to work. And we killed it. And we moved on, and that was really a milestone in the company because we sort of understood who we were and what would work and what wouldn't, even though technically it was really a fascinating thing.

It just always felt right once we did it, you know?

It just always felt right once we did it, you know?

Well, there was the AIBO Runner, but it wasn't even a whole robot. It was just legs that we, we took off the legs on AIBOs and attached the legs we'd made. We got that working and showed it to the Sony people. We worked pretty closely with Sony in those years. One of the interesting things is that it was before the internet and Zoom and anything like that.

Well, there was the AIBO Runner, but it wasn't even a whole robot. It was just legs that we, we took off the legs on AIBOs and attached the legs we'd made. We got that working and showed it to the Sony people. We worked pretty closely with Sony in those years. One of the interesting things is that it was before the internet and Zoom and anything like that.

So we had six ISDN lines installed, and we would have a telecon every week that worked at very low frame rates, something like 10 hertz. You know, English across the boundary with Japan was a challenge, trying to understand what each of us was saying and have meetings every week. for several years doing that. And it was a pleasure working with them. They were really supporters.

So we had six ISDN lines installed, and we would have a telecon every week that worked at very low frame rates, something like 10 hertz. You know, English across the boundary with Japan was a challenge, trying to understand what each of us was saying and have meetings every week. for several years doing that. And it was a pleasure working with them. They were really supporters.

They seemed to like us and what we were doing. That was the real transition from us being a simulation company into being a robotics company again.

They seemed to like us and what we were doing. That was the real transition from us being a simulation company into being a robotics company again.

Yeah, no, four legs, yeah.

Yeah, no, four legs, yeah.

Mostly we learned that something that small doesn't look very exciting when it's running. It's like it's scampering. And you had to watch a slow-mo for it to look like it was interesting. If you watch it fast, it was just like a... That's funny. One of my things was to show stuff in video, even from the very early days of the hopping machines.

Mostly we learned that something that small doesn't look very exciting when it's running. It's like it's scampering. And you had to watch a slow-mo for it to look like it was interesting. If you watch it fast, it was just like a... That's funny. One of my things was to show stuff in video, even from the very early days of the hopping machines.

And so I was always focused on how is this going to look through the viewfinder. And running AIBO didn't look so cool through the viewfinder.

And so I was always focused on how is this going to look through the viewfinder. And running AIBO didn't look so cool through the viewfinder.

I mean, you got to say that big dog was, you know, sort of put us on the map and got our heads really pulled together. We scaled up the company. Big dog was the result of, uh, Alan Rudolph at DARPA, uh, starting a biodynotics program and he put out a, you know, a request for proposals and, uh, I think there were 42 proposals written and three got funded. One was Big Dog.

I mean, you got to say that big dog was, you know, sort of put us on the map and got our heads really pulled together. We scaled up the company. Big dog was the result of, uh, Alan Rudolph at DARPA, uh, starting a biodynotics program and he put out a, you know, a request for proposals and, uh, I think there were 42 proposals written and three got funded. One was Big Dog.

One was a climbing robot, Rise. That put things in motion. We hired Martin Buehler. He was a professor in Montreal at McGill. He was incredibly important for getting Big Dog started. Out of the lab and into the mud, which is a key step to really be willing to go out there and build it, break it, fix it, which is sort of one of our mottos at the company.

One was a climbing robot, Rise. That put things in motion. We hired Martin Buehler. He was a professor in Montreal at McGill. He was incredibly important for getting Big Dog started. Out of the lab and into the mud, which is a key step to really be willing to go out there and build it, break it, fix it, which is sort of one of our mottos at the company.

Well, it's the first thing that worked. So let's see, if we go back to the leg lab, we built a quadruped that could do many of the things that Big Dog did, but it had a hydraulic pump sitting in the room with hoses connected to the robot. Mm-hmm. It had a VAX computer in the next room. It needed its own room because it was this giant thing with air conditioning.

Well, it's the first thing that worked. So let's see, if we go back to the leg lab, we built a quadruped that could do many of the things that Big Dog did, but it had a hydraulic pump sitting in the room with hoses connected to the robot. Mm-hmm. It had a VAX computer in the next room. It needed its own room because it was this giant thing with air conditioning.

And it had this very complicated bus connected to the robot. And the robot itself just had the actuators. It had gyroscopes for sensing and some other sensors. But all the power and computing was off-board. Big Dog had all that stuff integrated on the platform. It had a gasoline engine for power, which was a very complicated thing to undertake.

And it had this very complicated bus connected to the robot. And the robot itself just had the actuators. It had gyroscopes for sensing and some other sensors. But all the power and computing was off-board. Big Dog had all that stuff integrated on the platform. It had a gasoline engine for power, which was a very complicated thing to undertake.

It had to convert the rotation of the engine into hydraulic power, which is how we actuated

It had to convert the rotation of the engine into hydraulic power, which is how we actuated

uh it so there was a lot of learning just on the uh you know building the physical robot and the system integration for that and then there was the controls uh of it so for big dog you brought it all together onto one platform right and then so you could you could take it out in the woods yeah and you did we did we spent a lot of time down at the uh marine corps base in quantico where there was a trail

uh it so there was a lot of learning just on the uh you know building the physical robot and the system integration for that and then there was the controls uh of it so for big dog you brought it all together onto one platform right and then so you could you could take it out in the woods yeah and you did we did we spent a lot of time down at the uh marine corps base in quantico where there was a trail

called the guadalcanal trail and our uh milestone that darpa had specified was that we could go on this one particular trail that involved you know a lot of challenge and we spent a lot of time our team spent a lot of time down there those were fun days hiking with the robot what did you learn about like what it takes to balance a robot like that on a trail

called the guadalcanal trail and our uh milestone that darpa had specified was that we could go on this one particular trail that involved you know a lot of challenge and we spent a lot of time our team spent a lot of time down there those were fun days hiking with the robot what did you learn about like what it takes to balance a robot like that on a trail

Yeah. As challenging as the woods were, working inside of a home or in an office is really harder. Because when you're in the woods, you can actually take any path up the hill. All you have to do is avoid the obstacles. There's no such thing as damaging the woods, at least to first order. Whereas if you're in a house, you can't leave scuff marks. You can't bang into the walls.

Yeah. As challenging as the woods were, working inside of a home or in an office is really harder. Because when you're in the woods, you can actually take any path up the hill. All you have to do is avoid the obstacles. There's no such thing as damaging the woods, at least to first order. Whereas if you're in a house, you can't leave scuff marks. You can't bang into the walls.

The robots aren't very comfortable bumping into the walls, especially in the early days. So I think those were actually bigger challenges once we faced them. It was mostly getting the systems to work well enough together, the hardware systems to work, and the controls. In those days, we did have a human operator who did all the visual perception going up the Guadalcanal Trail.

The robots aren't very comfortable bumping into the walls, especially in the early days. So I think those were actually bigger challenges once we faced them. It was mostly getting the systems to work well enough together, the hardware systems to work, and the controls. In those days, we did have a human operator who did all the visual perception going up the Guadalcanal Trail.

There was an operator who was right there, who was very skilled at Even though the robot was balancing itself and placing its own feet, if the operator didn't do the right thing, it wouldn't go.

There was an operator who was right there, who was very skilled at Even though the robot was balancing itself and placing its own feet, if the operator didn't do the right thing, it wouldn't go.

But years later, we went back with one of the electric, the precursor to Spot, and we had advanced the controls and everything so much that an amateur, complete amateur, could operate the robot the first time up and down and up and down, whereas it had taken us years to get there in the previous robots.

But years later, we went back with one of the electric, the precursor to Spot, and we had advanced the controls and everything so much that an amateur, complete amateur, could operate the robot the first time up and down and up and down, whereas it had taken us years to get there in the previous robots.

So Big Dog became LS3, which is the big load carrying one.

So Big Dog became LS3, which is the big load carrying one.

It was designed to carry 400, but we had it carrying about 1,000 pounds. Of course you did.

It was designed to carry 400, but we had it carrying about 1,000 pounds. Of course you did.

We had one carrying the other one. We had two of them. So we had one carrying the other one. There's a little clip of that. We should put that out somewhere. That's from like 20 years ago. Wow.

We had one carrying the other one. We had two of them. So we had one carrying the other one. There's a little clip of that. We should put that out somewhere. That's from like 20 years ago. Wow.

So, Big Dog and LS3 had engine power and hydraulic actuation. Then we made a robot that was Electric power, so there's a battery driving a motor, driving a pump, but still hydraulic actuation. Larry sort of asked us, could you make something that weighed 60 pounds that would not be so intimidating if you had it in a house where there were people?

So, Big Dog and LS3 had engine power and hydraulic actuation. Then we made a robot that was Electric power, so there's a battery driving a motor, driving a pump, but still hydraulic actuation. Larry sort of asked us, could you make something that weighed 60 pounds that would not be so intimidating if you had it in a house where there were people?

And that was the inspiration behind the spot, pretty much as it exists today. We did a prototype the same size that was the first all-electric thing Non-hydraulic robot.

And that was the inspiration behind the spot, pretty much as it exists today. We did a prototype the same size that was the first all-electric thing Non-hydraulic robot.

I mean, it was almost as simple as what I just said. You know, we were having a meeting. He said, yeah, geez, do you think you could make a smaller one that wouldn't be so intimidating, like a big dog, if it was in your house? And I said, yeah, we could do that. And we started and did.

I mean, it was almost as simple as what I just said. You know, we were having a meeting. He said, yeah, geez, do you think you could make a smaller one that wouldn't be so intimidating, like a big dog, if it was in your house? And I said, yeah, we could do that. And we started and did.

I had been in love with hydraulics and still love hydraulics. It's a great technology. It's too bad that somehow the world out there looks at it like it's old-fashioned or that it's icky. It's true that you do. It is very hard to keep it from having some amount of dripping from time to time. But if you look at the performance, how strong you can get in a lightweight package.

I had been in love with hydraulics and still love hydraulics. It's a great technology. It's too bad that somehow the world out there looks at it like it's old-fashioned or that it's icky. It's true that you do. It is very hard to keep it from having some amount of dripping from time to time. But if you look at the performance, how strong you can get in a lightweight package.

And of course, we did a huge amount of innovation. Most of hydraulic control, that is the valve that controls the flow of oil, had been designed in the 50s for airplanes. It had been made robust enough, safe enough that you could count on it so that humans could fly in airplanes. And very little innovation had happened. You know, that might not be fair to the people who make the valves.

And of course, we did a huge amount of innovation. Most of hydraulic control, that is the valve that controls the flow of oil, had been designed in the 50s for airplanes. It had been made robust enough, safe enough that you could count on it so that humans could fly in airplanes. And very little innovation had happened. You know, that might not be fair to the people who make the valves.

I'm sure that they did innovate. But the basic design had stayed the same. And there was so much more you could do. And so our engineers designed valves, the ones that are in Atlas, for instance, that had new kinds of circuits. They sort of did some of the computing that could get you much more efficient use. They were much smaller and lighter so that the whole robot could be smaller and lighter.

I'm sure that they did innovate. But the basic design had stayed the same. And there was so much more you could do. And so our engineers designed valves, the ones that are in Atlas, for instance, that had new kinds of circuits. They sort of did some of the computing that could get you much more efficient use. They were much smaller and lighter so that the whole robot could be smaller and lighter.

We made a hydraulic power supply that had a bunch of components integrated in this tiny package. It's about this big, the size of a football. It weighs five kilograms and it produces five kilowatts of power. Of course, it has to have a battery operating, but it's got a motor, a pump, filters, heat exchanger to keep it cool, some valves, all in this tiny little package. Hydraulics

We made a hydraulic power supply that had a bunch of components integrated in this tiny package. It's about this big, the size of a football. It weighs five kilograms and it produces five kilowatts of power. Of course, it has to have a battery operating, but it's got a motor, a pump, filters, heat exchanger to keep it cool, some valves, all in this tiny little package. Hydraulics

you know, could still have a ways to go.

you know, could still have a ways to go.

Well, I think having good hardware is part of the story, and people who think you don't need to innovate hardware anymore are wrong, in my opinion. So I think one of the things, certainly in the early years for me, taking a dynamic approach where you think about what's the evolution of the motion of the thing going to be,

Well, I think having good hardware is part of the story, and people who think you don't need to innovate hardware anymore are wrong, in my opinion. So I think one of the things, certainly in the early years for me, taking a dynamic approach where you think about what's the evolution of the motion of the thing going to be,

in the future and having a prediction of that that's used at the time that you're giving signals to it, as opposed to it all being servoing, which is servoing is sort of backward looking. It says, okay, where am I now? I'm going to try and adjust for that. But you really need to think about what's coming.

in the future and having a prediction of that that's used at the time that you're giving signals to it, as opposed to it all being servoing, which is servoing is sort of backward looking. It says, okay, where am I now? I'm going to try and adjust for that. But you really need to think about what's coming.

It's interesting. I think that the number is only a couple of seconds for a spot. So there's a limited horizon type approach where you're recalculating, assuming what's going to happen in the next second or second and a half. And then you keep iterating. At the next, even though a tenth of a second later, you'll say, okay, let's do that again and see what's happening.

It's interesting. I think that the number is only a couple of seconds for a spot. So there's a limited horizon type approach where you're recalculating, assuming what's going to happen in the next second or second and a half. And then you keep iterating. At the next, even though a tenth of a second later, you'll say, okay, let's do that again and see what's happening.

And you're looking at what the obstacles are, where the feet are going to be placed, and how to, you know, you have to coordinate a lot of things if you have obstacles and you're balancing at the same time. And it's that limited horizon type calculation that's doing a lot of that. But if you're doing something like a somersault, you're looking out a lot further, right?

And you're looking at what the obstacles are, where the feet are going to be placed, and how to, you know, you have to coordinate a lot of things if you have obstacles and you're balancing at the same time. And it's that limited horizon type calculation that's doing a lot of that. But if you're doing something like a somersault, you're looking out a lot further, right?

If you want to stick the landing, you have to get the, you know, you have to, at the time of launch, have, you know, momentum and rotation, all those things coordinated so that a landing is within reach.

If you want to stick the landing, you have to get the, you know, you have to, at the time of launch, have, you know, momentum and rotation, all those things coordinated so that a landing is within reach.

If you look at the first time we ever made a robot do a somersault, it was in a planar robot. It had a boom. It was restricted to the surface of a sphere. We call that planar. It could move fore and aft. It could go up and down, and it could rotate. The calculation of what you need to do to stick a landing isn't all that complicated.

If you look at the first time we ever made a robot do a somersault, it was in a planar robot. It had a boom. It was restricted to the surface of a sphere. We call that planar. It could move fore and aft. It could go up and down, and it could rotate. The calculation of what you need to do to stick a landing isn't all that complicated.

You have to get time to make the rotation, so how high you jump gives you time. You look at how quickly you can rotate. And so if you get those two right, then when you land, you have the feet in the right place. And you have to get rid of all that rotational and linear momentum. But that's not too hard to figure out.

You have to get time to make the rotation, so how high you jump gives you time. You look at how quickly you can rotate. And so if you get those two right, then when you land, you have the feet in the right place. And you have to get rid of all that rotational and linear momentum. But that's not too hard to figure out.

And we made, back in about 1985 or 6, I can't remember, we had a simple robot doing somersaults. To do it in 3D, really the calculation is the same. You just have to be balancing in the other degrees of freedom. If you're just doing a somersault, it's just a planar thing.

And we made, back in about 1985 or 6, I can't remember, we had a simple robot doing somersaults. To do it in 3D, really the calculation is the same. You just have to be balancing in the other degrees of freedom. If you're just doing a somersault, it's just a planar thing.

When Rob was my graduate student and we were at MIT, which is when we made a two-legged robot do a 3D somersault for the first time. There, in order to get enough rotation rate, you needed to do tucking also. You know, withdraw the legs in order to accelerate it. And he did some really fascinating work on how you stabilize more complicated maneuvers.

When Rob was my graduate student and we were at MIT, which is when we made a two-legged robot do a 3D somersault for the first time. There, in order to get enough rotation rate, you needed to do tucking also. You know, withdraw the legs in order to accelerate it. And he did some really fascinating work on how you stabilize more complicated maneuvers.

You remember he was a gymnast, a champion gymnast before he'd come to me. So he had the physical abilities. And he was an engineer, so he could translate some of that into the math and the algorithms that you need to do that.

You remember he was a gymnast, a champion gymnast before he'd come to me. So he had the physical abilities. And he was an engineer, so he could translate some of that into the math and the algorithms that you need to do that.

Unfortunately, though, humans don't really know how they do it, right? We're coached. We have ways of learning, but do we really understand in a physics way what we're doing? Probably most gymnasts and athletes don't know.

Unfortunately, though, humans don't really know how they do it, right? We're coached. We have ways of learning, but do we really understand in a physics way what we're doing? Probably most gymnasts and athletes don't know.

Atlas still doesn't walk like a person, and it still doesn't walk quite as gracefully as a person, even though it's been getting closer and closer. The running might be close to a human, but the walking is still a challenge.

Atlas still doesn't walk like a person, and it still doesn't walk quite as gracefully as a person, even though it's been getting closer and closer. The running might be close to a human, but the walking is still a challenge.

And something weird about the knee, that you can kind of do this folding and unfolding, And get it to work out just, a human can get it to work out just right. There's compliances. Compliance means springiness in the design that are important to how it all works. Well, we used to have a motto at Boston Dynamics in the early days, which was, you have to run before you can walk.

And something weird about the knee, that you can kind of do this folding and unfolding, And get it to work out just, a human can get it to work out just right. There's compliances. Compliance means springiness in the design that are important to how it all works. Well, we used to have a motto at Boston Dynamics in the early days, which was, you have to run before you can walk.

For a quadruped, probably. Of course, it was probably the loudest, too. So we had this little racing go-kart engine on it, and we would get people from three buildings away sending us, complaining about how loud it was.

For a quadruped, probably. Of course, it was probably the loudest, too. So we had this little racing go-kart engine on it, and we would get people from three buildings away sending us, complaining about how loud it was.

Well, it's always a balance between wanting to get where you really want to get and what's practical to do based on your resources or what you know and all that. So, I mean, the whole idea of the pogo stick was to do simplification. Obviously, it didn't look like a human.

Well, it's always a balance between wanting to get where you really want to get and what's practical to do based on your resources or what you know and all that. So, I mean, the whole idea of the pogo stick was to do simplification. Obviously, it didn't look like a human.

I think a technical scientist could appreciate that we were capturing some of the things that are important in human locomotion without it looking complicated. like it without having a knee and ankle. I'll tell you the first sketch that Ben Brown made when we were talking about building this thing was a very complicated thing with zillions of springs, lots of joints.

I think a technical scientist could appreciate that we were capturing some of the things that are important in human locomotion without it looking complicated. like it without having a knee and ankle. I'll tell you the first sketch that Ben Brown made when we were talking about building this thing was a very complicated thing with zillions of springs, lots of joints.

It looked much more like a kangaroo or an ostrich or something like that, things we were paying a lot of attention to at the time. Um, you know, so my job was to, uh, say, okay, well let's do something simpler to get started and maybe we'll get there at some point.

It looked much more like a kangaroo or an ostrich or something like that, things we were paying a lot of attention to at the time. Um, you know, so my job was to, uh, say, okay, well let's do something simpler to get started and maybe we'll get there at some point.

Oh yeah. We, we did, uh, we filmed and digitized, uh, data from horses. I did a dissection of a ostrich at one point, which has absolutely remarkable legs.

Oh yeah. We, we did, uh, we filmed and digitized, uh, data from horses. I did a dissection of a ostrich at one point, which has absolutely remarkable legs.

Most of it's up in the feathers, but there's a huge amount going on in the feathers, including a knee joint. The knee joints way up there. The thing that's halfway down the leg that looks like a backwards knee is actually the ankle. The thing on the ground, which looks like the foot, is actually the toes. It's an extended toe. Fascinating.

Most of it's up in the feathers, but there's a huge amount going on in the feathers, including a knee joint. The knee joints way up there. The thing that's halfway down the leg that looks like a backwards knee is actually the ankle. The thing on the ground, which looks like the foot, is actually the toes. It's an extended toe. Fascinating.

But, you know, the basic morphology is the same in all these animals.

But, you know, the basic morphology is the same in all these animals.

You know, the slow-mos of cheetahs running are incredible. You know, they're they, there's so much back motion and, uh, you know, grace. And of course they're moving very fast. Uh, The animals running away from the cheetah are pretty exciting.

You know, the slow-mos of cheetahs running are incredible. You know, they're they, there's so much back motion and, uh, you know, grace. And of course they're moving very fast. Uh, The animals running away from the cheetah are pretty exciting.

You know, the pronghorn, which, you know, they do this all four legs at once jump called the prong to kind of confuse the, especially if there's a group of them, to confuse whoever's chasing them. So they do like a misdirection type of thing? Yep, they do a misdirection thing.

You know, the pronghorn, which, you know, they do this all four legs at once jump called the prong to kind of confuse the, especially if there's a group of them, to confuse whoever's chasing them. So they do like a misdirection type of thing? Yep, they do a misdirection thing.

The front on views of the cheetahs running fast where the tail is whipping around to help in the turns, to help us stabilize in the turns. That's pretty exciting.

The front on views of the cheetahs running fast where the tail is whipping around to help in the turns, to help us stabilize in the turns. That's pretty exciting.

But they also turn very fast.

But they also turn very fast.

Everything in the body is probably helping turn, because they're chasing something that's trying to get away that's also zigzagging around. But I would be remiss if I didn't say humans are pretty good, too. You watch gymnasts, especially these days, they're doing just incredible stuff.

Everything in the body is probably helping turn, because they're chasing something that's trying to get away that's also zigzagging around. But I would be remiss if I didn't say humans are pretty good, too. You watch gymnasts, especially these days, they're doing just incredible stuff.

In Big Dog. Big Dog. Yeah, Big Dog came first, and then Little Dog was later. And you know, there was a... There's a big compromise there. Human knees have multiple muscles, and you could argue that there's, I mean, it's a technical thing about negative work. When you're contracting a joint, but you're pushing out, that's negative work.

In Big Dog. Big Dog. Yeah, Big Dog came first, and then Little Dog was later. And you know, there was a... There's a big compromise there. Human knees have multiple muscles, and you could argue that there's, I mean, it's a technical thing about negative work. When you're contracting a joint, but you're pushing out, that's negative work.

And if you don't have a place to store that, it can be very expensive to do negative work. And in Big Dog, there was no place to store negative work in the knees. But Big Dog also had pogo stick springs down below. So part of the action was to comply in a bouncing motion. Later on in Spot, we took that out. As we got further and further away from the leg lab, we had more energy-driven controls.

And if you don't have a place to store that, it can be very expensive to do negative work. And in Big Dog, there was no place to store negative work in the knees. But Big Dog also had pogo stick springs down below. So part of the action was to comply in a bouncing motion. Later on in Spot, we took that out. As we got further and further away from the leg lab, we had more energy-driven controls.

Sure. There's this idea called passive dynamics, which says that although you can use computers and actuators to make a motion, a mechanical system can make a motion just by itself if it gets stimulated the right way. Uh, so, uh, Tad McGeer in the, uh, I think in the mid eighties, uh, maybe it was in the late eighties starting to, started to work on that.

Sure. There's this idea called passive dynamics, which says that although you can use computers and actuators to make a motion, a mechanical system can make a motion just by itself if it gets stimulated the right way. Uh, so, uh, Tad McGeer in the, uh, I think in the mid eighties, uh, maybe it was in the late eighties starting to, started to work on that.

And he made this, uh, legged system that could walk down an inclined plane where the legs folded and unfolded and swung forward, you know, do the whole walk, walking motion where the only thing, there was no computer, uh, There were some adjustments to the mechanics so that there were dampers and springs in some places that helped the mechanical action happen.

And he made this, uh, legged system that could walk down an inclined plane where the legs folded and unfolded and swung forward, you know, do the whole walk, walking motion where the only thing, there was no computer, uh, There were some adjustments to the mechanics so that there were dampers and springs in some places that helped the mechanical action happen.

It was essentially a mechanical computer. The interesting idea there is That it's not all about the brain dictating to the body what the body should do. The body is a participant in the motion.

It was essentially a mechanical computer. The interesting idea there is That it's not all about the brain dictating to the body what the body should do. The body is a participant in the motion.

I think that these days most robots aren't doing that. Most robots are... are basically using the computer to govern the motion. Now, the brain, though, is taking into account what the mechanical thing can do and how it's going to behave.

I think that these days most robots aren't doing that. Most robots are... are basically using the computer to govern the motion. Now, the brain, though, is taking into account what the mechanical thing can do and how it's going to behave.

Otherwise, it would have to really forcefully move everything around all the time, which probably some solutions do, but I think you end up with a more efficient and more graceful thing if you're taking into account what the machine wants to do.

Otherwise, it would have to really forcefully move everything around all the time, which probably some solutions do, but I think you end up with a more efficient and more graceful thing if you're taking into account what the machine wants to do.

I like to talk about intelligence having two parts, an athletic part and a cognitive part. And I think Boston Dynamics, in my view, has sort of set the standard for what athletic intelligence can be. And it has to do with all the things we've been talking about, the The mechanical design, the real-time control, the energetics, and that kind of stuff.

I like to talk about intelligence having two parts, an athletic part and a cognitive part. And I think Boston Dynamics, in my view, has sort of set the standard for what athletic intelligence can be. And it has to do with all the things we've been talking about, the The mechanical design, the real-time control, the energetics, and that kind of stuff.

But obviously, people have another kind of intelligence. And animals have another kind of intelligence. We can make a plan. Our meeting started at 9.30. I looked up on Google Maps how long it took to walk over here. It was 20 minutes. So I decided, okay, I'd leave my house at 9.00. which is what I did. You know, simple intelligence, but we use that kind of stuff all the time.

But obviously, people have another kind of intelligence. And animals have another kind of intelligence. We can make a plan. Our meeting started at 9.30. I looked up on Google Maps how long it took to walk over here. It was 20 minutes. So I decided, okay, I'd leave my house at 9.00. which is what I did. You know, simple intelligence, but we use that kind of stuff all the time.

It's sort of what we think of as going on in our heads. And I think that's in short supply for robots. Most robots are pretty dumb. And as a result, it takes a lot of skilled people to program them to do everything they do. And it takes a long time. And if robots are gonna satisfy our dreams, they need to be smarter.

It's sort of what we think of as going on in our heads. And I think that's in short supply for robots. Most robots are pretty dumb. And as a result, it takes a lot of skilled people to program them to do everything they do. And it takes a long time. And if robots are gonna satisfy our dreams, they need to be smarter.

So the AI Institute is designed to combine that physicality of the athletic side with the cognitive side. So, for instance, we're trying to make robots that can watch a human do a task, understand what it's seeing, and then do the task itself. So sort of OJT, on-the-job training for robots, as a paradigm. Now, you know, that's pretty hard...

So the AI Institute is designed to combine that physicality of the athletic side with the cognitive side. So, for instance, we're trying to make robots that can watch a human do a task, understand what it's seeing, and then do the task itself. So sort of OJT, on-the-job training for robots, as a paradigm. Now, you know, that's pretty hard...

It's sort of science fiction, but our idea is to work on a longer timeframe and work on solving those kinds of problems. I have a whole list of things that are in that vein.

It's sort of science fiction, but our idea is to work on a longer timeframe and work on solving those kinds of problems. I have a whole list of things that are in that vein.

And using our hands, you know. Using your hands. The mechanics of interacting with all these things.

And using our hands, you know. Using your hands. The mechanics of interacting with all these things.

these two things you know you've never touched those things before right well i've touched ones like this okay look at all the things i can do right i can juggle and i'm rotating this way i can rotate it without looking i could fetch these things out of my pocket and figure out which one was which and all that kind of stuff yeah and uh i don't think we have much of a clue how all that works yet right and that's i really like putting that under the banner of athletic uh intelligence

these two things you know you've never touched those things before right well i've touched ones like this okay look at all the things i can do right i can juggle and i'm rotating this way i can rotate it without looking i could fetch these things out of my pocket and figure out which one was which and all that kind of stuff yeah and uh i don't think we have much of a clue how all that works yet right and that's i really like putting that under the banner of athletic uh intelligence

I mean, one question you could ask, it isn't my question, but, you know, are they commercially viable? Will they increase productivity? And I think... You know, we're getting very close to that. I don't think we're quite there still. You know, most of the robotics companies, it's a struggle.

I mean, one question you could ask, it isn't my question, but, you know, are they commercially viable? Will they increase productivity? And I think... You know, we're getting very close to that. I don't think we're quite there still. You know, most of the robotics companies, it's a struggle.

It's really the lack of the cognitive side that probably is the biggest barrier at the moment, even for the physically successful robots. But your question is, I mean, you can always do a thing that's more efficient, lighter, more reliable. I'd say reliability. I know that Spot, they've been working very hard on getting the the tail of the reliability curve up and they've made huge progress.

It's really the lack of the cognitive side that probably is the biggest barrier at the moment, even for the physically successful robots. But your question is, I mean, you can always do a thing that's more efficient, lighter, more reliable. I'd say reliability. I know that Spot, they've been working very hard on getting the the tail of the reliability curve up and they've made huge progress.

So the robots, you know, there's 1,500 of them out there now, many of them being used in practical applications day in and day out where, you know, where they have to work reliably. And, you know, it's very exciting that they've done that, but it takes a huge effort to get that kind of reliability in the robot. There's cost, too. You'd like to get the cost down.

So the robots, you know, there's 1,500 of them out there now, many of them being used in practical applications day in and day out where, you know, where they have to work reliably. And, you know, it's very exciting that they've done that, but it takes a huge effort to get that kind of reliability in the robot. There's cost, too. You'd like to get the cost down.

Spots are still pretty expensive, and I don't think that they have to be, but it takes a different kind of activity to do that. Now that... I think that Boston Dynamics is owned primarily by Hyundai now, and I think that the skills of Hyundai in making cars can be brought to bear in making robots that are less expensive and more reliable and those kinds of things.

Spots are still pretty expensive, and I don't think that they have to be, but it takes a different kind of activity to do that. Now that... I think that Boston Dynamics is owned primarily by Hyundai now, and I think that the skills of Hyundai in making cars can be brought to bear in making robots that are less expensive and more reliable and those kinds of things.

That's a good question. I think we're using the paradigm called stepping stones to moonshots. I don't believe, that was in my original proposal for the Institute, stepping stones to moonshots. I think if you go more than a year without seeing a tangible status report of where you are, which is the stepping stone, It could be a simplification.

That's a good question. I think we're using the paradigm called stepping stones to moonshots. I don't believe, that was in my original proposal for the Institute, stepping stones to moonshots. I think if you go more than a year without seeing a tangible status report of where you are, which is the stepping stone, It could be a simplification.

You don't necessarily have to solve all the problems of your target goal, even though your target goal is going to take several years. Those stepping stone results give you feedback, give motivation, because usually there's some success in there. That's the mantra we've been working on. That's pretty much how I'd say Boston Dynamics has worked, where you make progress and show it as you go.

You don't necessarily have to solve all the problems of your target goal, even though your target goal is going to take several years. Those stepping stone results give you feedback, give motivation, because usually there's some success in there. That's the mantra we've been working on. That's pretty much how I'd say Boston Dynamics has worked, where you make progress and show it as you go.

Show it to yourself, if not to the world.

Show it to yourself, if not to the world.

Well, with Watch, Understand, Do, the project I mentioned before, we've broken that down into getting some progress with what is meaningfully watching something mean, breaking down an observation of a person doing something into the components, segmenting. You watch me do something. I'm going to pick up this thing and put it down here and stack this on it.

Well, with Watch, Understand, Do, the project I mentioned before, we've broken that down into getting some progress with what is meaningfully watching something mean, breaking down an observation of a person doing something into the components, segmenting. You watch me do something. I'm going to pick up this thing and put it down here and stack this on it.

Well, it's not obvious if you just look at the raw data. what the sequence of acts are. It's really a creative, intelligent act for you to break that down into the pieces and understand them in a way so you could say, okay, what skill do I need to accomplish each of those things?

Well, it's not obvious if you just look at the raw data. what the sequence of acts are. It's really a creative, intelligent act for you to break that down into the pieces and understand them in a way so you could say, okay, what skill do I need to accomplish each of those things?

So we're working on the front end of that kind of a problem where we observe and translate the, if it may be video, it may be live, into

So we're working on the front end of that kind of a problem where we observe and translate the, if it may be video, it may be live, into

a description of what we think is going on and then try and map that into skills to accomplish that and we've been developing skills as well so you know we have kind of multiple stabs at the pieces of of doing that and this is usually video of humans manipulating objects with their hands kind of thing we're starting out with bicycle repair some simple bicycle repair oh no that seems complicated that seems really complicated it is but but there's some parts of it that aren't

a description of what we think is going on and then try and map that into skills to accomplish that and we've been developing skills as well so you know we have kind of multiple stabs at the pieces of of doing that and this is usually video of humans manipulating objects with their hands kind of thing we're starting out with bicycle repair some simple bicycle repair oh no that seems complicated that seems really complicated it is but but there's some parts of it that aren't

Like putting the seat in, you know, into the, you know, you have a tube that goes inside of another tube and there's a latch. You know, that should be within range.

Like putting the seat in, you know, into the, you know, you have a tube that goes inside of another tube and there's a latch. You know, that should be within range.

I think it is. And I think that's the kind of thing that people don't recognize. Let me translate it to navigation. Mm-hmm. I think the basic paradigm for navigating a space is to get some kind of sensor that tells you where an obstacle is and what's open, build a map, and then go through the space.

I think it is. And I think that's the kind of thing that people don't recognize. Let me translate it to navigation. Mm-hmm. I think the basic paradigm for navigating a space is to get some kind of sensor that tells you where an obstacle is and what's open, build a map, and then go through the space.

But if we were doing on-the-job training where I was giving you a task, I wouldn't have to say anything about the room. We came in here, all we did is adjust the chair, but we didn't say anything about the room, and we could navigate it. So I think there's opportunities to build that kind of navigation skill into robots. And we're hoping to be able to do that.

But if we were doing on-the-job training where I was giving you a task, I wouldn't have to say anything about the room. We came in here, all we did is adjust the chair, but we didn't say anything about the room, and we could navigate it. So I think there's opportunities to build that kind of navigation skill into robots. And we're hoping to be able to do that.

Yeah, and lack of specification.

Yeah, and lack of specification.

I mean, that's what sort of intelligence is, right? Kind of dealing with, understanding a situation even though it wasn't explained.

I mean, that's what sort of intelligence is, right? Kind of dealing with, understanding a situation even though it wasn't explained.

You know, Since ChatGBT, which is a year ago, basically, there's a huge interest in that and a huge optimism about it. And I think that there's a lot of things that machine learning, that kind of machine learning. Now, of course, there's lots of different kinds of machine learning. I think there's a lot of interest and optimism about it.

You know, Since ChatGBT, which is a year ago, basically, there's a huge interest in that and a huge optimism about it. And I think that there's a lot of things that machine learning, that kind of machine learning. Now, of course, there's lots of different kinds of machine learning. I think there's a lot of interest and optimism about it.

I think the facts on the ground are that doing physical things with physical robots is a little bit different than language. The tokens sort of don't exist. Pixel values aren't like words. But I think that there's a lot that can be done there. We have... We have several people working on machine learning approaches.

I think the facts on the ground are that doing physical things with physical robots is a little bit different than language. The tokens sort of don't exist. Pixel values aren't like words. But I think that there's a lot that can be done there. We have... We have several people working on machine learning approaches.

I don't know if you know, but we opened an office in Zurich recently, and Marco Hutter, who's one of the real leaders in reinforcement learning for robots, is the director of that office. He's still half-time at ETH now.

I don't know if you know, but we opened an office in Zurich recently, and Marco Hutter, who's one of the real leaders in reinforcement learning for robots, is the director of that office. He's still half-time at ETH now.

the university there where he has an unbelievably fantastic lab and then he's half time leading will be leading off efforts in the Zurich office so we have a healthy learning component but there's part of me that still says if you look out in the world at what the most impressive performances are

the university there where he has an unbelievably fantastic lab and then he's half time leading will be leading off efforts in the Zurich office so we have a healthy learning component but there's part of me that still says if you look out in the world at what the most impressive performances are

They're still pretty much, I hate to use the word traditional, but that's what everybody's calling it, traditional controls, like model predictive control. The Atlas performances that you've seen are mostly model predictive control. They've started to do some learning stuff that's really incredible. I don't know if it's all been shown yet, but you'll see it over time.

They're still pretty much, I hate to use the word traditional, but that's what everybody's calling it, traditional controls, like model predictive control. The Atlas performances that you've seen are mostly model predictive control. They've started to do some learning stuff that's really incredible. I don't know if it's all been shown yet, but you'll see it over time.

And then Marco has done some great stuff and others.

And then Marco has done some great stuff and others.

I think we're going to find a mating of the two and we'll have the best of both worlds. And we're working on that at the Institute too.

I think we're going to find a mating of the two and we'll have the best of both worlds. And we're working on that at the Institute too.

Sure, technical fearlessness means being willing to take on a problem that you don't know how to solve. Study it, figure out an entry point, maybe a simplified version or a simplified solution or something. Learn from the stepping stone and go back and eventually make a solution that meets your goals. And I think that's really important.

Sure, technical fearlessness means being willing to take on a problem that you don't know how to solve. Study it, figure out an entry point, maybe a simplified version or a simplified solution or something. Learn from the stepping stone and go back and eventually make a solution that meets your goals. And I think that's really important.

Yeah, and you don't know how to do it. There's easier stuff to do in life. I mean, I don't know. Watch, understand, do. It's a mountain of a challenge.

Yeah, and you don't know how to do it. There's easier stuff to do in life. I mean, I don't know. Watch, understand, do. It's a mountain of a challenge.

Yeah. I mean, we have others like that. We have one called Inspect, Diagnose, Fix. You call up the Maytag repairman. Okay, he's the one who you don't have to call, but you call up the dishwasher repair person, and they come to your house, and they look at your machine,

Yeah. I mean, we have others like that. We have one called Inspect, Diagnose, Fix. You call up the Maytag repairman. Okay, he's the one who you don't have to call, but you call up the dishwasher repair person, and they come to your house, and they look at your machine,

It's already been actually figured out that something doesn't work, but they have to kind of examine it and figure out what's wrong and then fix it. And I think robots should be able to do that. Boston Dynamics already has spot robots collecting data on machines, things like thermal data, reading the gauges, listening to them, getting sounds.

It's already been actually figured out that something doesn't work, but they have to kind of examine it and figure out what's wrong and then fix it. And I think robots should be able to do that. Boston Dynamics already has spot robots collecting data on machines, things like thermal data, reading the gauges, listening to them, getting sounds.

And that data are used to determine whether they're healthy or not. But the interpretation isn't done by the robots yet. And certainly the fixing, the diagnosing and the fixing isn't done yet. But I think it could be. And that's bringing the AI and combining it with the physical skills to do it.

And that data are used to determine whether they're healthy or not. But the interpretation isn't done by the robots yet. And certainly the fixing, the diagnosing and the fixing isn't done yet. But I think it could be. And that's bringing the AI and combining it with the physical skills to do it.

You mean convincing you it's okay that the dishwasher's broken?

You mean convincing you it's okay that the dishwasher's broken?

Why is diligence important? Well, if you want a real robot solution, it can't be a very narrow solution that's going to break at the first variation in what the robot does or the environment if it wasn't exactly as you expected it. So how do you get there? I think having an approach that leaves you unsatisfied until you've embraced the bigger problem is the diligence I'm talking about.

Why is diligence important? Well, if you want a real robot solution, it can't be a very narrow solution that's going to break at the first variation in what the robot does or the environment if it wasn't exactly as you expected it. So how do you get there? I think having an approach that leaves you unsatisfied until you've embraced the bigger problem is the diligence I'm talking about.

Again, I'll point at Boss Dynamics. Some of the videos that we had showing the engineer making it hard for the robot to do its task. spot opening a door, and then the guy gets there and pushes on the door so it doesn't open the way it's supposed to, pulling on the rope that's attached to the robot so its navigation has been screwed up.

Again, I'll point at Boss Dynamics. Some of the videos that we had showing the engineer making it hard for the robot to do its task. spot opening a door, and then the guy gets there and pushes on the door so it doesn't open the way it's supposed to, pulling on the rope that's attached to the robot so its navigation has been screwed up.

We have one where the robot's climbing stairs and an engineer is tugging on a rope that's pulling it back down the stairs. That's totally different than just the robot seeing the stairs, making a model, putting its feet carefully on each step. But that's what probably robotics needs to succeed, and having that broader

We have one where the robot's climbing stairs and an engineer is tugging on a rope that's pulling it back down the stairs. That's totally different than just the robot seeing the stairs, making a model, putting its feet carefully on each step. But that's what probably robotics needs to succeed, and having that broader

That broader idea that you want to come up with a robust solution is what I meant by diligence.

That broader idea that you want to come up with a robust solution is what I meant by diligence.

I learned something very early on with the first three-dimensional hopping machine. If you just show a video of it hopping It's a so what. If you show it falling over a couple of times and you can see how easily and fast it falls over, then you appreciate what the robot's doing when it's doing its thing.

I learned something very early on with the first three-dimensional hopping machine. If you just show a video of it hopping It's a so what. If you show it falling over a couple of times and you can see how easily and fast it falls over, then you appreciate what the robot's doing when it's doing its thing.

So I think the reaction you just gave to the robot getting kind of interfered with or tested while it's going through the door, it's showing you the scope of the solution.

So I think the reaction you just gave to the robot getting kind of interfered with or tested while it's going through the door, it's showing you the scope of the solution.

Well, I was the final edit for most of the videos up until until about three years ago or four years ago. And my theory of the video is no explanation. If they can't see it, then it's not the right thing. And if you do something worth showing, then let them see it. Don't interfere with a bunch of titles that slow you down or a bunch of distraction. Just do something worth showing and then show it.

Well, I was the final edit for most of the videos up until until about three years ago or four years ago. And my theory of the video is no explanation. If they can't see it, then it's not the right thing. And if you do something worth showing, then let them see it. Don't interfere with a bunch of titles that slow you down or a bunch of distraction. Just do something worth showing and then show it.

That's brilliant. It's hard though for people to buy into that.

That's brilliant. It's hard though for people to buy into that.

People have criticized, especially the big dog videos where there's a human driving the robot. And I understand the criticism now. At the time, we wanted to just show, look, this thing's using its legs to get up the hill. So we focused on showing that, which was, we thought, the story.

People have criticized, especially the big dog videos where there's a human driving the robot. And I understand the criticism now. At the time, we wanted to just show, look, this thing's using its legs to get up the hill. So we focused on showing that, which was, we thought, the story.

The fact that there was a human, so they were thinking about autonomy, whereas we were thinking about the mobility. And so, you know, we've adjusted to a lot of things that we see that people care about, trying to be honest. We've always tried to be honest.

The fact that there was a human, so they were thinking about autonomy, whereas we were thinking about the mobility. And so, you know, we've adjusted to a lot of things that we see that people care about, trying to be honest. We've always tried to be honest.

I mean, it might be the most important ingredient, and that is robotics is hard. It's not gonna work right, right away. So don't be discouraged is all it really means. So usually when I talk about these things, I show videos and I show a long string of outtakes. And you have to have courage to be intrepid when you work so hard, you built your machine,

I mean, it might be the most important ingredient, and that is robotics is hard. It's not gonna work right, right away. So don't be discouraged is all it really means. So usually when I talk about these things, I show videos and I show a long string of outtakes. And you have to have courage to be intrepid when you work so hard, you built your machine,

Then you're trying and it just doesn't do what you thought it would do, what you want it to do.

Then you're trying and it just doesn't do what you thought it would do, what you want it to do.

There's a video of Atlas climbing three big steps, and it's very dynamic, and it's really exciting, a real accomplishment. It took 109 tries, and we have video of every one of them. We shoot everything. Again, we, this is at Boston Dynamics. So it took 109 tries. But once it did it, it had a high percentage of success.

There's a video of Atlas climbing three big steps, and it's very dynamic, and it's really exciting, a real accomplishment. It took 109 tries, and we have video of every one of them. We shoot everything. Again, we, this is at Boston Dynamics. So it took 109 tries. But once it did it, it had a high percentage of success.

So it's not like we're cheating by just showing the best one, but we do show the evolved performance, not everything along the way. But everything along the way is informative, and it shows sort of there's stupid things that go wrong, like the robot just when you say go and it collapses right there on the start. That doesn't have to do with the steps.

So it's not like we're cheating by just showing the best one, but we do show the evolved performance, not everything along the way. But everything along the way is informative, and it shows sort of there's stupid things that go wrong, like the robot just when you say go and it collapses right there on the start. That doesn't have to do with the steps.

Or the perception didn't work right, so you miss the target when you jump, or something breaks and there's oil flying everywhere. Yeah. But that's fun. Yeah.

Or the perception didn't work right, so you miss the target when you jump, or something breaks and there's oil flying everywhere. Yeah. But that's fun. Yeah.

Lots of control of evolution during that time. I think it took six weeks to get those 109 trials. Because there was programming going on. It was actually robot learning, but there were humans in the loop helping with the learning. So all data-driven.

Lots of control of evolution during that time. I think it took six weeks to get those 109 trials. Because there was programming going on. It was actually robot learning, but there were humans in the loop helping with the learning. So all data-driven.

It's remarkable. One of the accomplishments of Atlas is that the engineers have made a machine that's robust enough that it can take that kind of testing where it's falling and stuff, and it doesn't break every time. It still breaks. Part of the paradigm is to have people to repair stuff. You got to figure that in if you're going to do this kind of work.

It's remarkable. One of the accomplishments of Atlas is that the engineers have made a machine that's robust enough that it can take that kind of testing where it's falling and stuff, and it doesn't break every time. It still breaks. Part of the paradigm is to have people to repair stuff. You got to figure that in if you're going to do this kind of work.

I sometimes criticize the people who have their gold-plated thing and they keep it on the shelf and they're afraid to use it. I don't think you can make progress if you're working that way. You need to be ready to have it break and go in there and fix it. It's part of the thing. Plan your budget so you have spare parts and a crew and all that stuff.

I sometimes criticize the people who have their gold-plated thing and they keep it on the shelf and they're afraid to use it. I don't think you can make progress if you're working that way. You need to be ready to have it break and go in there and fix it. It's part of the thing. Plan your budget so you have spare parts and a crew and all that stuff.

I think it applies to everything anybody tries to do that's worth doing.

I think it applies to everything anybody tries to do that's worth doing.

Fun. Technical fun, I usually say. I have technical fun. I think that life as an engineer... is really satisfying. To some degree, it can be like crafts work where you get to do things with your own hands or your own design or whatever your media is. It's very satisfying to be able to just do the work, unlike a lot of people who have to do something that they don't like doing.

Fun. Technical fun, I usually say. I have technical fun. I think that life as an engineer... is really satisfying. To some degree, it can be like crafts work where you get to do things with your own hands or your own design or whatever your media is. It's very satisfying to be able to just do the work, unlike a lot of people who have to do something that they don't like doing.

I think engineers typically get to do something that they like, and there's a lot of satisfaction from that. In many cases, you can have impact on the world somehow because you've done something that other people admire, which is different from just the craft fun of building a thing. So that's a second thing. way that being an engineer is good.

I think engineers typically get to do something that they like, and there's a lot of satisfaction from that. In many cases, you can have impact on the world somehow because you've done something that other people admire, which is different from just the craft fun of building a thing. So that's a second thing. way that being an engineer is good.

I think the third thing is that if you're lucky to be working in a team where you're getting the benefit of other people's skills that are helping you do your thing, none of us has all the skills needed to do most of these projects. If you have a team where you're working well with the others, that can be very satisfying. And then if you're an engineer, you also usually get paid.

I think the third thing is that if you're lucky to be working in a team where you're getting the benefit of other people's skills that are helping you do your thing, none of us has all the skills needed to do most of these projects. If you have a team where you're working well with the others, that can be very satisfying. And then if you're an engineer, you also usually get paid.

And so you kind of get paid four times in my view of the world. So what could be better than that?

And so you kind of get paid four times in my view of the world. So what could be better than that?

I think it's both being a scientist or getting to use science at the same time as being kind of an artist or a creator. Scientists only get to study what's out there, and engineers get to make stuff that didn't exist before.

I think it's both being a scientist or getting to use science at the same time as being kind of an artist or a creator. Scientists only get to study what's out there, and engineers get to make stuff that didn't exist before.

It's really, I think, a higher calling, even though I think most of the public out there think science is top and engineering is somehow secondary, but I think it's the other way around.

It's really, I think, a higher calling, even though I think most of the public out there think science is top and engineering is somehow secondary, but I think it's the other way around.

That is the art. Bringing metal to life or a machine to life is kind of... It was fun doing the dancing videos where it got a huge public response. We're going to do more. We're doing some at the Institute and we'll do more.

That is the art. Bringing metal to life or a machine to life is kind of... It was fun doing the dancing videos where it got a huge public response. We're going to do more. We're doing some at the Institute and we'll do more.

But I think sort of with it going back to the pieces of that, you design a linkage that turns out to be half the weight and just as strong. That's very satisfying. There are people who do that, and it's a creative act.

But I think sort of with it going back to the pieces of that, you design a linkage that turns out to be half the weight and just as strong. That's very satisfying. There are people who do that, and it's a creative act.

I think having the robots move in a way that's evocative of life is pretty exciting. So the elegance of movement. Or if it's a high-performance act where it's doing it faster, bigger than other robots. Usually we're not doing it bigger, faster than people, but we're getting there in a few narrow dimensions.

I think having the robots move in a way that's evocative of life is pretty exciting. So the elegance of movement. Or if it's a high-performance act where it's doing it faster, bigger than other robots. Usually we're not doing it bigger, faster than people, but we're getting there in a few narrow dimensions.

I mean, I'd like to do dancing that starts... You know, we're nowhere near the dancing capabilities of a human. We've been having a ballerina in who's kind of a well-known ballerina, and she's been programming the robot.

I mean, I'd like to do dancing that starts... You know, we're nowhere near the dancing capabilities of a human. We've been having a ballerina in who's kind of a well-known ballerina, and she's been programming the robot.

We've been working on the tools that can make it so that she can use her way of talking, you know, way of doing a choreography or something like that more accessible to get the robot to do things. And she's starting to produce some interesting stuff.

We've been working on the tools that can make it so that she can use her way of talking, you know, way of doing a choreography or something like that more accessible to get the robot to do things. And she's starting to produce some interesting stuff.

There is.

There is.

We hope to take that forward and make it more tuned to how the dance world wants to talk, wants to communicate, and get better performances. I mean, we've done a lot, but there's still a lot possible. And I'd like to have performances where the robots are dancing with people. So right now, almost everything that we've done on dancing is to a fixed time base.

We hope to take that forward and make it more tuned to how the dance world wants to talk, wants to communicate, and get better performances. I mean, we've done a lot, but there's still a lot possible. And I'd like to have performances where the robots are dancing with people. So right now, almost everything that we've done on dancing is to a fixed time base.

So once you press go, the robot does its thing and plays that thing. It's not listening, it's not watching, but I think it should do those things.

So once you press go, the robot does its thing and plays that thing. It's not listening, it's not watching, but I think it should do those things.

One of the cool things going on, you know that there's a class at Brown University called Choreo Robotics. Sidney Skybetter is a dancer, choreographer, and he teamed up with Stephanie Tellex, who's a computer science professor, and they taught this class, and I think they have some graduate students helping teach it. where they have two spots and people come in.

One of the cool things going on, you know that there's a class at Brown University called Choreo Robotics. Sidney Skybetter is a dancer, choreographer, and he teamed up with Stephanie Tellex, who's a computer science professor, and they taught this class, and I think they have some graduate students helping teach it. where they have two spots and people come in.

I think it's 50-50 of computer science people and dance people. And they program performances that are very interesting. I show some of them sometimes when I give a talk.

I think it's 50-50 of computer science people and dance people. And they program performances that are very interesting. I show some of them sometimes when I give a talk.

But it's also a challenge. We get asked to have our robots perform with famous dancers. And they have 200 degrees of freedom or something, right? Every little ripple and thing, and they have all this head and neck and shoulders and stuff. And the robots mostly don't have all that stuff. And it's a daunting challenge to not look stupid, physically stupid next to them.

But it's also a challenge. We get asked to have our robots perform with famous dancers. And they have 200 degrees of freedom or something, right? Every little ripple and thing, and they have all this head and neck and shoulders and stuff. And the robots mostly don't have all that stuff. And it's a daunting challenge to not look stupid, physically stupid next to them.

So we've pretty much avoided that kind of performance, but we'll get to it.

So we've pretty much avoided that kind of performance, but we'll get to it.

And we can reverse things. If you watch a human do robot animation, which is a dance style where you jerk around and you pop and lock and all that stuff, I think the robots could show up the humans by doing unstable oscillations and things that are faster than a person. So that's sort of on my... you know, my plan, but we haven't quite gotten there yet.

And we can reverse things. If you watch a human do robot animation, which is a dance style where you jerk around and you pop and lock and all that stuff, I think the robots could show up the humans by doing unstable oscillations and things that are faster than a person. So that's sort of on my... you know, my plan, but we haven't quite gotten there yet.

I think you need to have an environment where interesting engineers, well, you know, it's a chicken and egg. If you have an environment where interesting engineering is going on, then engineers want to work there. I think it took a long time to develop that at Boston Dynamics.

I think you need to have an environment where interesting engineers, well, you know, it's a chicken and egg. If you have an environment where interesting engineering is going on, then engineers want to work there. I think it took a long time to develop that at Boston Dynamics.

In fact, when we started, although I had the experience of building things in the leg lab, both at CMU and at MIT, we weren't that sophisticated of an engineering thing compared to what Boston Dynamics is now. But it was our ambition to do that. And Sarkos was another robot company. So I always thought of us as being this much on the computing side and this much on the hardware side.

In fact, when we started, although I had the experience of building things in the leg lab, both at CMU and at MIT, we weren't that sophisticated of an engineering thing compared to what Boston Dynamics is now. But it was our ambition to do that. And Sarkos was another robot company. So I always thought of us as being this much on the computing side and this much on the hardware side.

And they were like this. And then over the years, I think we achieved the same or better levels of engineering. Meanwhile, Sarkos got acquired and then they went through all kinds of changes and I don't know exactly what their current status is, but... So it took many years is part of the answer. I think you got to find people who love it.

And they were like this. And then over the years, I think we achieved the same or better levels of engineering. Meanwhile, Sarkos got acquired and then they went through all kinds of changes and I don't know exactly what their current status is, but... So it took many years is part of the answer. I think you got to find people who love it.

In the early days, we paid a little less, so we only got people who were doing it because they really loved it. We also hired people who might not have professional degrees, people who were building bicycles and building kayaks. We have some people who come from that kind of the maker world, and that's really important. for the kind of work we do to have that be part of the mix.

In the early days, we paid a little less, so we only got people who were doing it because they really loved it. We also hired people who might not have professional degrees, people who were building bicycles and building kayaks. We have some people who come from that kind of the maker world, and that's really important. for the kind of work we do to have that be part of the mix.

People who repaired the cars or motorcycles or whatever in their garages when they were kids.

People who repaired the cars or motorcycles or whatever in their garages when they were kids.

Talk about being happy. There used to be a time when I was doing the machine shop work myself, back in those JPL and Caltech days, when if I came home smelling like the machine shop, because it's an oily place, my wife would say, oh, you had a good day today. Because you could tell that that's where I'd been.

Talk about being happy. There used to be a time when I was doing the machine shop work myself, back in those JPL and Caltech days, when if I came home smelling like the machine shop, because it's an oily place, my wife would say, oh, you had a good day today. Because you could tell that that's where I'd been.

Oh, you know, at Boston Dynamics, it put us on the map. I remember interviewing some sales guy and he was from a company and he said, well, no one's ever heard of my company, but we have products, you know, really good products. You guys, everybody knows who you are, but you don't have any products at all, which was true. So it was, and you know, we thank YouTube for that.

Oh, you know, at Boston Dynamics, it put us on the map. I remember interviewing some sales guy and he was from a company and he said, well, no one's ever heard of my company, but we have products, you know, really good products. You guys, everybody knows who you are, but you don't have any products at all, which was true. So it was, and you know, we thank YouTube for that.

YouTube came, we caught the YouTube wave and it had a huge impact on our company.

YouTube came, we caught the YouTube wave and it had a huge impact on our company.

I really admire Elon as a technologist. I think that what he did with Tesla is just totally mind-boggling, that he could go from this totally niche area that less than 1% of anybody seemed to be interested to making it, so that essentially every car company in the world is trying to do what he's done. So you got to give it to him. Then look at SpaceX. He's basically replaced NASA, if you could.

I really admire Elon as a technologist. I think that what he did with Tesla is just totally mind-boggling, that he could go from this totally niche area that less than 1% of anybody seemed to be interested to making it, so that essentially every car company in the world is trying to do what he's done. So you got to give it to him. Then look at SpaceX. He's basically replaced NASA, if you could.

That might be a little exaggeration, but not by much. So, you know, you got to admire the guy. And, you know, I wouldn't count him out for anything. You know, I don't think Optimus today is where Atlas is, for instance. I don't know. It's a little hard to compare him to the other companies. You know, I visited Figure. I think they're doing well and they have a good team.

That might be a little exaggeration, but not by much. So, you know, you got to admire the guy. And, you know, I wouldn't count him out for anything. You know, I don't think Optimus today is where Atlas is, for instance. I don't know. It's a little hard to compare him to the other companies. You know, I visited Figure. I think they're doing well and they have a good team.

I've visited Aptronic, and I think they have a good team and they're doing well. But Elon has a lot of resources. He has a lot of ambition. I'd like to take some credit for his ambition. I think if I read between the lines, it's hard not to think that him seeing what Atlas is doing is a little bit of an inspiration. I hope so.

I've visited Aptronic, and I think they have a good team and they're doing well. But Elon has a lot of resources. He has a lot of ambition. I'd like to take some credit for his ambition. I think if I read between the lines, it's hard not to think that him seeing what Atlas is doing is a little bit of an inspiration. I hope so.

I would love to host that. Now that I'm not at Boston Dynamics, I'm not officially connected. I am on the board, but I'm not officially connected. I would love to host a- Robot meetups? A robot meetup, yeah.

I would love to host that. Now that I'm not at Boston Dynamics, I'm not officially connected. I am on the board, but I'm not officially connected. I would love to host a- Robot meetups? A robot meetup, yeah.

We have a bunch of different robots. We bought everything we could buy. So we have spots. I think we have a good size fleet of them. I don't know how many it is, but a good size fleet. We have a couple of animal robots. You know, Animal is a company founded by Marco Hutter, even though he's not that involved anymore, but we have a couple of those.

We have a bunch of different robots. We bought everything we could buy. So we have spots. I think we have a good size fleet of them. I don't know how many it is, but a good size fleet. We have a couple of animal robots. You know, Animal is a company founded by Marco Hutter, even though he's not that involved anymore, but we have a couple of those.

We have a bunch of arms like, you know, Frank is and... U.S. robotics. Because even though we have ambitions to build stuff, and we are starting to build stuff, day one, getting off the ground, we just bought stuff. I love this robot playground you've built. You can come over and take a look if you want.

We have a bunch of arms like, you know, Frank is and... U.S. robotics. Because even though we have ambitions to build stuff, and we are starting to build stuff, day one, getting off the ground, we just bought stuff. I love this robot playground you've built. You can come over and take a look if you want.

it doesn't feel that much like, well, there's some areas that feel like a playground, but it's not like they're all frolic together.

it doesn't feel that much like, well, there's some areas that feel like a playground, but it's not like they're all frolic together.

I think that, um, I don't think about competition that much. I'm not a commercial guy. I think for the many years I was at Boston Dynamics, we didn't think about competition. We were just kind of doing our thing. It wasn't like there were products out there that we were competing with. Maybe there was some competition for DARPA products.

I think that, um, I don't think about competition that much. I'm not a commercial guy. I think for the many years I was at Boston Dynamics, we didn't think about competition. We were just kind of doing our thing. It wasn't like there were products out there that we were competing with. Maybe there was some competition for DARPA products.

funding, which we got, you know, got a lot of, got very good at, at getting, but even there, uh, in, in a couple of cases where we might've competed, we ended up just being the robot provider. That is for the little dog program. You know, we, we just made the robots. We didn't participate as developers except for developing the robot. And in the DARPA robotics challenge, we didn't compete.

funding, which we got, you know, got a lot of, got very good at, at getting, but even there, uh, in, in a couple of cases where we might've competed, we ended up just being the robot provider. That is for the little dog program. You know, we, we just made the robots. We didn't participate as developers except for developing the robot. And in the DARPA robotics challenge, we didn't compete.

We, uh, provided the robots. So, uh, In the AI world now, now that we're working on cognitive stuff, it feels much more like a competition. The entry requirements in terms of computing hardware and the skills of the team and hiring talent, it's a much tougher place. So I think much more about competition now on the cognitive side.

We, uh, provided the robots. So, uh, In the AI world now, now that we're working on cognitive stuff, it feels much more like a competition. The entry requirements in terms of computing hardware and the skills of the team and hiring talent, it's a much tougher place. So I think much more about competition now on the cognitive side.

On the physical side, it doesn't feel like it's that much about competition yet. Obviously, with 10 humanoid companies out there, 10 or 12, I mean, there's probably others that I don't know about, they're definitely in competition, will be in competition.

On the physical side, it doesn't feel like it's that much about competition yet. Obviously, with 10 humanoid companies out there, 10 or 12, I mean, there's probably others that I don't know about, they're definitely in competition, will be in competition.

I think there's a huge way to go. I don't think we've seen the bottom of it or the bottom in terms of lower prices. I think you should be totally optimistic that at Asymptote, things don't have to be anything like as expensive as they are now. Back to competition, I wanted to say one thing.

I think there's a huge way to go. I don't think we've seen the bottom of it or the bottom in terms of lower prices. I think you should be totally optimistic that at Asymptote, things don't have to be anything like as expensive as they are now. Back to competition, I wanted to say one thing.

I think in the quadruped space, having other people selling quadrupeds is a great thing for Boston Dynamics. Because the question, I believe the question in the user's minds is, which quadruped do I want? It's not, oh, do I want a quadruped? Can a quadruped do my job? It's much more like that, which is a great place for it to be. Then you're just doing quadrupeds.

I think in the quadruped space, having other people selling quadrupeds is a great thing for Boston Dynamics. Because the question, I believe the question in the user's minds is, which quadruped do I want? It's not, oh, do I want a quadruped? Can a quadruped do my job? It's much more like that, which is a great place for it to be. Then you're just doing quadrupeds.

doing the things you normally do to make your product better and compete and selling and all that stuff. And that'll be the way it is with humanoids at some point.

doing the things you normally do to make your product better and compete and selling and all that stuff. And that'll be the way it is with humanoids at some point.

You're not really... You're creating the category. Yeah. Or the category is happening. I mean, right now, the use cases, that's the key thing, having realistic use cases that are money-making and... in robotics is a big challenge. There's the warehouse use case. That's probably the only thing that makes anybody any money in robotics at this point.

You're not really... You're creating the category. Yeah. Or the category is happening. I mean, right now, the use cases, that's the key thing, having realistic use cases that are money-making and... in robotics is a big challenge. There's the warehouse use case. That's probably the only thing that makes anybody any money in robotics at this point.

There's old-fashioned robotics. I mean, there's fixed arms doing manufacturing. I don't want to say that they're not making money.

There's old-fashioned robotics. I mean, there's fixed arms doing manufacturing. I don't want to say that they're not making money.

But it's also true that the companies making those things, there have been a lot of failures in recent times, right? There's that one year when I think three of them went under. So it's not that easy to do that, right? Getting performance, safety, and cost all to be where they need to be at the same time, that's hard.

But it's also true that the companies making those things, there have been a lot of failures in recent times, right? There's that one year when I think three of them went under. So it's not that easy to do that, right? Getting performance, safety, and cost all to be where they need to be at the same time, that's hard.

Yeah, it has two or three or four.

Yeah, it has two or three or four.

Well, I was always a builder from a young age. I was lucky. My father was a frustrated engineer. And by that, I mean he wanted to be an aerospace engineer, but his mom from the old country thought that that would be like a grease monkey. And so she said no. So he became an accountant. But the result of that was our basement was always full of tools and equipment and electronics.

Well, I was always a builder from a young age. I was lucky. My father was a frustrated engineer. And by that, I mean he wanted to be an aerospace engineer, but his mom from the old country thought that that would be like a grease monkey. And so she said no. So he became an accountant. But the result of that was our basement was always full of tools and equipment and electronics.

I think that intelligence is a lot of different things. I think some things, computers are already smarter than people. Some things, they're not even close. I think you'd need a menu of detailed categories to come up with that. I also think that the conversation that seems to be happening about AGI puzzles me. It's sort of So I ask you a question.

I think that intelligence is a lot of different things. I think some things, computers are already smarter than people. Some things, they're not even close. I think you'd need a menu of detailed categories to come up with that. I also think that the conversation that seems to be happening about AGI puzzles me. It's sort of So I ask you a question.

Do you think there's anybody smarter than you in the world?

Do you think there's anybody smarter than you in the world?

Do you find that threatening?

Do you find that threatening?

So I don't understand, even if computers were smarter than people, why we should assume that that's a threat. Especially since they could easily be smarter but still available to us or under our control, which is basically how computers generally are.

So I don't understand, even if computers were smarter than people, why we should assume that that's a threat. Especially since they could easily be smarter but still available to us or under our control, which is basically how computers generally are.

It's a little bit like that line in the Oppenheimer movie where they contemplate whether the first time they set off a reaction, all matter on Earth is going to go up. I don't remember what the verb they used was for the chain reaction, right? Yeah, I guess... It's possible, but I personally don't think it's worth worrying about that. It's balancing opportunities and risk.

It's a little bit like that line in the Oppenheimer movie where they contemplate whether the first time they set off a reaction, all matter on Earth is going to go up. I don't remember what the verb they used was for the chain reaction, right? Yeah, I guess... It's possible, but I personally don't think it's worth worrying about that. It's balancing opportunities and risk.

I think if you take any technology, there's opportunity and risk. I'll point at the car. They pollute and about 1.25 million people get killed every year around the world because of them. Despite that, I think they're a boon to humankind, very useful. Many of us love them. Those technical problems can be solved. I think they are becoming safer. I think they're becoming less polluting.

I think if you take any technology, there's opportunity and risk. I'll point at the car. They pollute and about 1.25 million people get killed every year around the world because of them. Despite that, I think they're a boon to humankind, very useful. Many of us love them. Those technical problems can be solved. I think they are becoming safer. I think they're becoming less polluting.

At least some of them are. Every technology you can name has a story like that in my opinion.

At least some of them are. Every technology you can name has a story like that in my opinion.

It was born of me being a contrarian. Yes. It's a symbol. Someone told me once that I was wearing one when I only had one or two. And they said, oh, those things are so old-fashioned. You can't wear that, Mark. And I stopped wearing them for about a week. And then I said, I'm not going to let them tell me what to do. And so every day since, pretty much. That was years ago. That was 20 years ago.

It was born of me being a contrarian. Yes. It's a symbol. Someone told me once that I was wearing one when I only had one or two. And they said, oh, those things are so old-fashioned. You can't wear that, Mark. And I stopped wearing them for about a week. And then I said, I'm not going to let them tell me what to do. And so every day since, pretty much. That was years ago. That was 20 years ago.

15 years ago, probably.

15 years ago, probably.

It took me a while to realize I was a contrarian. But, you know, it can be a useful tool.

It took me a while to realize I was a contrarian. But, you know, it can be a useful tool.

I'd rather talk about, there's a guy, when we were doing a lot of DARPA work, there was a Marine, Ed Tovar, who's still around, who his, what he would always say is when someone would say, oh, you can't do that, he'd say, why not? And it's a great question. I ask all the time when I'm thinking, oh, we're not going to do that. And I say, why not? And I give him credit for opening my eyes to

I'd rather talk about, there's a guy, when we were doing a lot of DARPA work, there was a Marine, Ed Tovar, who's still around, who his, what he would always say is when someone would say, oh, you can't do that, he'd say, why not? And it's a great question. I ask all the time when I'm thinking, oh, we're not going to do that. And I say, why not? And I give him credit for opening my eyes to

to resisting that.

to resisting that.

And from a young age, I would watch him assembling a kit, an Ico kit or something like that. I still have a couple of his old Ico kits. But it was really during graduate school when I followed a professor back from class. It was Bertolt Horn at MIT. And I was taking a an interim class. It's IAP, independent activities period. And I followed him back to his lab.

And from a young age, I would watch him assembling a kit, an Ico kit or something like that. I still have a couple of his old Ico kits. But it was really during graduate school when I followed a professor back from class. It was Bertolt Horn at MIT. And I was taking a an interim class. It's IAP, independent activities period. And I followed him back to his lab.

When I was teaching at MIT, for a while I had undergraduate advisees where people would have to meet with me once a semester or something. And they frequently would ask what they should do. And I think the advice I used to give was something like, well, if you had no constraints on you, No resource constraints, no opportunity constraints, and no skill constraints. What could you imagine doing?

When I was teaching at MIT, for a while I had undergraduate advisees where people would have to meet with me once a semester or something. And they frequently would ask what they should do. And I think the advice I used to give was something like, well, if you had no constraints on you, No resource constraints, no opportunity constraints, and no skill constraints. What could you imagine doing?

And I said, well, start there and see how close you can get. What's realistic for how close you can get? The other version of that is try and figure out what you want to do and do that. A lot of people think that they're in a channel, right, and there's only limited opportunities, but it's usually wider than they think.

And I said, well, start there and see how close you can get. What's realistic for how close you can get? The other version of that is try and figure out what you want to do and do that. A lot of people think that they're in a channel, right, and there's only limited opportunities, but it's usually wider than they think.

Some people think I'm in a rut, right? Why don't I do it? And in fact, some of the inspiration for the AI Institute is to say, okay, I've been working on locomotion for however many years it was. Let's do something else.

Some people think I'm in a rut, right? Why don't I do it? And in fact, some of the inspiration for the AI Institute is to say, okay, I've been working on locomotion for however many years it was. Let's do something else.

Just about to start showing some stuff off, yeah. I hope we have a YouTube channel. I mean, one of the challenges is it's one thing to show athletic skills on YouTube. Showing cognitive function is a lot harder and I haven't quite figured out yet how that's going to work.

Just about to start showing some stuff off, yeah. I hope we have a YouTube channel. I mean, one of the challenges is it's one thing to show athletic skills on YouTube. Showing cognitive function is a lot harder and I haven't quite figured out yet how that's going to work.

I think you're right.

I think you're right.

I don't know. I think you have to have fun while you're here. That's about all I know. It would be a waste not to, right?

I don't know. I think you have to have fun while you're here. That's about all I know. It would be a waste not to, right?

Somehow it spurred my imagination. I was in the brain and cognitive sciences department as a graduate student doing neurophysiology. I'd been an electrical engineer as an undergrad at Northeastern. The neurophysiology wasn't really working for me. It wasn't conceptual enough.

Somehow it spurred my imagination. I was in the brain and cognitive sciences department as a graduate student doing neurophysiology. I'd been an electrical engineer as an undergrad at Northeastern. The neurophysiology wasn't really working for me. It wasn't conceptual enough.

I couldn't see really how by looking at single neurons you were going to get to a place where you could understand control systems or thought or anything like that. And, you know, the AI lab was always an appealing, this was before CSAIL, right? This was in the 70s. So the AI lab was always an appealing idea.

I couldn't see really how by looking at single neurons you were going to get to a place where you could understand control systems or thought or anything like that. And, you know, the AI lab was always an appealing, this was before CSAIL, right? This was in the 70s. So the AI lab was always an appealing idea.

And so when I went back to the AI lab with, you know, following him and I saw the arm, I just thought, you know, this is it.

And so when I went back to the AI lab with, you know, following him and I saw the arm, I just thought, you know, this is it.

Well, BCS is now morphed a bit, right? They have the Center for Brains, Minds, and Machines, which is trying to bridge that gap. And even when I was there, you know, David Marr was in the AI lab. David Marr had models of the brain that were appealing both to biologists but also to computer people. So he was a visitor in the AI lab at the time, and I guess he became full-time there.

Well, BCS is now morphed a bit, right? They have the Center for Brains, Minds, and Machines, which is trying to bridge that gap. And even when I was there, you know, David Marr was in the AI lab. David Marr had models of the brain that were appealing both to biologists but also to computer people. So he was a visitor in the AI lab at the time, and I guess he became full-time there.

So that was the first time a bridge was made between those two groups. Then the bridge kind of went away, and then there was another time in the 80s. And then recently, the last five or so years, there's been a stronger connection.

So that was the first time a bridge was made between those two groups. Then the bridge kind of went away, and then there was another time in the 80s. And then recently, the last five or so years, there's been a stronger connection.

I mean, we were just doing gadgets when we were kids. I had a friend, we were I don't know if everybody remembers, but fluorescent lights had this little aluminum cylinder. I can't even remember what it's called now. You needed a starter, I think it was. We would take those apart, fill them with match heads, put a tail on it, and make it into little rockets.

I mean, we were just doing gadgets when we were kids. I had a friend, we were I don't know if everybody remembers, but fluorescent lights had this little aluminum cylinder. I can't even remember what it's called now. You needed a starter, I think it was. We would take those apart, fill them with match heads, put a tail on it, and make it into little rockets.

I think it's still a balance between those two. There was a time though when I was, I guess I was probably already a professor or maybe late in graduate school when I thought that function was everything and that mobility, dexterity, perception, and intelligence, those are sort of the key functionalities for robotics, that that's what mattered and nothing else mattered.

I think it's still a balance between those two. There was a time though when I was, I guess I was probably already a professor or maybe late in graduate school when I thought that function was everything and that mobility, dexterity, perception, and intelligence, those are sort of the key functionalities for robotics, that that's what mattered and nothing else mattered.

And I even had kind of this platonic ideal that if you just looked at a robot and it wasn't doing anything, it would look like a pile of junk, which a lot of my robots looked like in those days. But then when it started moving, you'd get the idea that it had some kind of life or some kind of interest in its movement.

And I even had kind of this platonic ideal that if you just looked at a robot and it wasn't doing anything, it would look like a pile of junk, which a lot of my robots looked like in those days. But then when it started moving, you'd get the idea that it had some kind of life or some kind of interest in its movement.

And I think we purposely even designed the machines, not worrying about the aesthetics of the structure itself. But then it turns out that the aesthetics of the thing itself add and combine with the lifelike things that the robots can do. But the heart of it is making them do things that are interesting.

And I think we purposely even designed the machines, not worrying about the aesthetics of the structure itself. But then it turns out that the aesthetics of the thing itself add and combine with the lifelike things that the robots can do. But the heart of it is making them do things that are interesting.

Well, let me tell you about how I got started on legs at all. When I was still a graduate student, I went to a conference. It was a biological legged locomotion conference. I think it was in Philadelphia. So it was all biomechanics people, researchers who would look at muscle and maybe neurons and things like that. They weren't so much computational people, but they were more biomechanics.

Well, let me tell you about how I got started on legs at all. When I was still a graduate student, I went to a conference. It was a biological legged locomotion conference. I think it was in Philadelphia. So it was all biomechanics people, researchers who would look at muscle and maybe neurons and things like that. They weren't so much computational people, but they were more biomechanics.

And maybe there were a thousand people there. And I went to a talk. One of the talks, all the talks were about the body of either animals or people and respiration, things like that. But one talk was by a robotics guy, and he showed a six-legged robot that walked very slowly.

And maybe there were a thousand people there. And I went to a talk. One of the talks, all the talks were about the body of either animals or people and respiration, things like that. But one talk was by a robotics guy, and he showed a six-legged robot that walked very slowly.

It always had at least three feet on the ground, so it worked like a table or a chair with tripod stability, and it moved really slowly. And I just looked at that and said, Wow, that's wrong. That's not anything like how people and animals work. Because we bounce and fly, we have to predict what's going to happen in order to keep our balance when we're taking a running step or something like that.

It always had at least three feet on the ground, so it worked like a table or a chair with tripod stability, and it moved really slowly. And I just looked at that and said, Wow, that's wrong. That's not anything like how people and animals work. Because we bounce and fly, we have to predict what's going to happen in order to keep our balance when we're taking a running step or something like that.

We use the springiness in our legs, our muscles and our tendons and things like that. As part of the story, the energy circulates. We don't just throw it away every time. I'm not sure I understood all that when I first thought, but I definitely got inspired to say, let's try the opposite. I didn't have a clue as to how to make a hopping robot work, not balance in 3D.

In fact, when I started, it was all just about the energy of bouncing. And I was going to have a springy thing in the leg and some actuator so that you could get an energy regime going of bouncing. And the idea that balance was an important part of it didn't come until a little later. And then I made the one-legged, the pogo stick robots. Now I think that we need to do that in manipulation.

If you look at robot manipulation, we, a community, has been working on it for 50 years. We're nowhere near human levels of manipulation. I mean, we can, you know, it's come along, but I think it's all too safe.

And I think trying to break out of that safety thing of static grasping, you know, if you look at a lot of work that goes on, it's about the geometry of the part, and then you figure out how to move your hand so that you can position it with respect to that, and then you grasp it carefully, and then you move it. That's not anything like how people and animals work. We juggle in our hands.

We hog multiple objects and can sort them. Now, to be fair, being more aggressive is going to mean things aren't going to work very well for a while. It's a longer-term approach to the problem. That's just theory now. Maybe that won't pay off, but that's how I'm trying to think about it, trying to encourage our group to go at it.

I don't know if you know my friend Matt Mason, who is the director of the Robotics Institute at Carnegie Mellon. And we go back to graduate school together. But he analyzed... a movie of Julia Childs doing a cooking thing. And she did, I think he said something like there were 40 different ways that she handled a thing and none of them was grasping.

She would nudge, roll, flatten with her knife, things like that, and none of them was grasping.

First of all, the Leg Lab actually started at Carnegie Mellon. I was a professor there starting in 1980, about 1986. And so that's where the first hopping machines were built. I guess we got the first one working in about 1982, something like that. That was a simplified one. Then we got a three-dimensional one in 1983.

The quadruped that we built at the Leg Lab, the first version, was built in about 1984 or 5 and really only got going about 86 or so. It took years of development to get it to work.

Well, I'm going to start on the, not the technical side, but the, I guess we could call it the motivational side or the funding side. So before Carnegie Mellon, I was actually at JPL, at the Jet Propulsion Lab for three years.

And while I was there, I connected up with Ivan Sutherland, who is sometimes regarded as the father of computer graphics because of work he did both at MIT and then University of Utah and Evans and Sutherland.

Anyway, um, I got to know him and at one point he said, uh, he encouraged me to, uh, do some kind of project, uh, at Caltech, even though I was at JPL, you know, those are kind of related institutions. And, uh, So I thought about it, and I made up a list of three possible projects. And I purposely made the top one and the bottom one really boring sounding.

And in the middle, I put Pogo Stick Robot. When he looked at it, Ivan is a brilliant guy, brilliant engineer, and a real cultivator of people. He looked at it and knew right away what the thing that was worth doing. He had an endowed chair, so he had about $3,000 that he gave me to build the first model for

which I went to the shop and with my own hands made a first model, which didn't work and was just a beginning shot at it. Ivan and I took that to Washington. In those days, you could just walk into DARPA and walk down the hallway and see who's there. Ivan, who had been there in his previous life, We walked around and we looked in the offices. Of course, I didn't know anything.

I was basically a kid, but Ivan knew his way around. We found Craig Fields in his office. Craig later became the director of DARPA, but in those days, he was a program manager. We went in. I had a little Samsonite suitcase. We opened and it had just the skeleton of this one-legged hopping robot. We showed it to him. And you could almost see the drool going down his chin of excitement.

And he sent me $250,000. He said, okay, I want to fund this. And I was between institutions. I was just about to leave JPL, and I hadn't decided yet where I was going next. And then when I landed at CMU, he sent $250,000, which in 1980 was a lot of research money.

Like, all the fundamentals are there. Yeah, I mean, I think that was the motivation to try and get more at the fundamentals of how animals work. But the idea that it would result in, you know, machines that were anything like practical... like we're making now. That wasn't anywhere in my head, no.

As an academic, I was mostly just trying to do the next thing, make some progress, impress my colleagues if I could.

Well, in the very early days, I needed some better engineering than I could do myself. And I hired Ben Brown. We each had our way of contributing to the design. And we came up with a thing that could start to work. I had some stupid ideas about how the actuation system should work. And we sorted that out.

It wasn't that hard to make it balanced once you get the physical machine to be working well enough and have enough control over the degrees of freedom. And then we very quickly, you know, we started out by having it floating on an inclined air table. And then that only gave us like six foot of travel.

So once it started working, we switched to a thing that could run around the room on another device. It's hard to explain these without you seeing them, but you probably know what I'm talking about, a planarizer. And then the next big step was to make it work in 3D, which that was really the scary part.

With these simple things, you know, people had inverted pendulums at the time for years and they could control them by driving a cart back and forth. But could you make it work in three dimensions while it's bouncing and all that? But it turned out, you know, not to be that hard to do, at least at the level of performance we achieved at the time.

Yes.

The simple story is that there's three things going on. There's something making it bounce. We had a system that was estimating how high the robot was off the ground. Using that, there's energy that can be in three places in a pogo stick. One is in the spring, one is in the altitude, and the other is in the velocity. And so when at the top of the hop, it's all in the height.

And so you could just measure how high you're going and thereby have an idea of a lot about the cycle, and you could decide whether to put more energy in or less. So that is one element. Then there's a part that you decide where to put the foot. And if you think when you're landing on the ground with respect to the center of mass, so if you think of a pole vaulter,

The key thing the pole vaulter has to do is get its body to the right place when the pole gets stuck. If they're too far forward, they kind of get thrown backwards. If they're too far back, they go over. And what they need to do is get it so that they go mostly up to get over the thing. And high jumpers is the same kind of thing.

So there's a calculation about where to put the foot and we did something, you know, relatively simple. And then there's a third part to keep the body at an attitude that's upright. Cause if it gets too far, you know, you could hop and just keep rotating around, but if it gets too far, then you run out of motion of the joints at the hips. So you have to do that.

And we did that by applying a torque between the legs and the body. Every time the foot's on the ground, you only can do it while the foot's on the ground in the air. You know, it, it, the physics don't work out.

Well, you're asking an interesting question because... In those days, we didn't actually optimize things. And they probably could have gone much further than we did and then had higher performance. And we just kind of got a sketch of a solution and worked on that.

And then in years since, some people working for us, some people working for others, people came up with all kinds of equations or algorithms for how to do a better job, be able to go faster. One of my students worked on getting things to go faster. Another one worked on... climbing over obstacles. Because when you're running, on the open ground, it's one thing.

If you're running up a stair, you have to adjust where you are. Otherwise, things don't work out right. You land your foot on the edge of the step. So there's other degrees of freedom to control if you're getting to more realistic, practical situations.

Probably the smartest thing I ever did is to find the other people. I mean, when I look at it now, I look at Boston Dynamics and all the really excellent engineering there, people who really make stuff work. I'm only the dreamer.

Did you experience a lot of people around you kind of... I don't know if they doubted whether it was possible, but I think they thought it was a waste of time.

I think for a lot of people. I think it's been both, though. Some people, I felt like they were saying, oh, why are you wasting your time on this stupid problem? But then I've been at many things where people have told me it's been an inspiration to go out and attack these harder things. And I think it has turned out, I think legged locomotion has turned out to be a useful thing.

I mean, at first, I wasn't an enthusiast for the humanoids because, again, it goes back to saying, what's the functionality? And the form wasn't as important as the functionality. And also, there's an aspect to humanoid robots that's about functionality. all about the cosmetics, where there isn't really other functionality, and that kind of is off-putting for me.

As a roboticist, I think the functionality really matters. So probably that's why I avoided humanoid robots to start with. I'll tell you, after we started working on him, you could see the connection and the impact with other people, whether they're lay people or even other technical people.

There's a special thing that goes on, even though most of the humanoid robots aren't that much like a person.

I'll tell you, I go around giving talks and take Spot to a lot of them. And it's amazing. The media likes to say that they're terrifying and that people are afraid. And And YouTube commenters like to say that it's frightening.

But when you take a spot out there, now maybe it's self-selecting, but you get a crowd of people who want to take pictures, want to pose for selfies, want to operate the robot, want to pet it, want to put clothes on it. It's amazing.

What the connection was is that at that point, Boston Dynamics was mostly a physics-based simulation company. When I left MIT to start Boston Dynamics, there was a few years of overlap, but the concept wasn't to start a robot company. The concept was to use this dynamic simulation tool that we developed to do robotics for other things.

But working with Sony, we got back into robotics by doing the AIBO Runner. We made some tools for programming Curio, which was a humanoid this big that could do some dancing and other kinds of fun stuff. And I don't think it ever reached the market, even though they did show it. When I look back, I say that we got us back where we belonged. Yeah.

That's right.

One of the robots that we built wasn't actually a robot. It was a surgical simulator, but it had force feedback. So it had all the techniques of robotics. And you look down into this... mirror it actually was. And it looked like you were looking down onto the body you were working on. Your hands were underneath the mirror, so they were where you were looking.

And you had tools in your hands that were connected up to these force feedback devices made by another MIT spin-out, Sensible Technologies. So they made the force feedback device. We attached the tools and we wrote all the software and did all the graphics. So we had 3D computer graphics.

It was in the old days when, this was in the late 90s, when you had a silicon graphics computer that was about this big. It was the heater in the office, basically. And we were doing surgical operations, anastomosis, which was stitching tubes together, tubes like blood vessels or other things in their body. And you could feel and you could see the tissues move. And it was really exciting.

And the idea was to make a trainer to teach surgeons how to do stuff. We built a scoring system because we interviewed surgeons that told us what you're supposed to do and what you're not supposed to do. You're not supposed to tear the tissue. You're not supposed to touch it in any place except for where you're trying to engage. There were a bunch of rules.

So we built this thing and took it to a trade show, a surgical trade show. And the surgeons were practically lined up. Well, we kept the score and we posted their scores like on a video game. And those guys are so competitive that they really, really love doing it. And they would come around and they see someone's score was higher there. So they would come back.

But we figured out shortly after that we thought surgeons were going to pay us to get trained on these things. And the surgeons thought we should pay them in order to, so they could teach us about the thing. And there was no money from the surgeons. And we looked at it and thought, well, maybe we could sell it to hospitals that would teach, train their surgeons.

And then we said, well, we're this, at the time we were probably a 12-person company or maybe 15 people, I don't remember. There's no way we could go after a marketing activity. You know, the company was all bootstrapped in those years. We never had investors until Google bought us, which was after 20 years. So we didn't have any resources to go after hospitals.

So at one day, Rob and I were looking at that and we said, we'd built another simulator for knee arthroscopy. And we said, this isn't going to work. And we killed it. And we moved on, and that was really a milestone in the company because we sort of understood who we were and what would work and what wouldn't, even though technically it was really a fascinating thing.

It just always felt right once we did it, you know?

Well, there was the AIBO Runner, but it wasn't even a whole robot. It was just legs that we, we took off the legs on AIBOs and attached the legs we'd made. We got that working and showed it to the Sony people. We worked pretty closely with Sony in those years. One of the interesting things is that it was before the internet and Zoom and anything like that.

So we had six ISDN lines installed, and we would have a telecon every week that worked at very low frame rates, something like 10 hertz. You know, English across the boundary with Japan was a challenge, trying to understand what each of us was saying and have meetings every week. for several years doing that. And it was a pleasure working with them. They were really supporters.

They seemed to like us and what we were doing. That was the real transition from us being a simulation company into being a robotics company again.

Yeah, no, four legs, yeah.

Mostly we learned that something that small doesn't look very exciting when it's running. It's like it's scampering. And you had to watch a slow-mo for it to look like it was interesting. If you watch it fast, it was just like a... That's funny. One of my things was to show stuff in video, even from the very early days of the hopping machines.

And so I was always focused on how is this going to look through the viewfinder. And running AIBO didn't look so cool through the viewfinder.

I mean, you got to say that big dog was, you know, sort of put us on the map and got our heads really pulled together. We scaled up the company. Big dog was the result of, uh, Alan Rudolph at DARPA, uh, starting a biodynotics program and he put out a, you know, a request for proposals and, uh, I think there were 42 proposals written and three got funded. One was Big Dog.

One was a climbing robot, Rise. That put things in motion. We hired Martin Buehler. He was a professor in Montreal at McGill. He was incredibly important for getting Big Dog started. Out of the lab and into the mud, which is a key step to really be willing to go out there and build it, break it, fix it, which is sort of one of our mottos at the company.

Well, it's the first thing that worked. So let's see, if we go back to the leg lab, we built a quadruped that could do many of the things that Big Dog did, but it had a hydraulic pump sitting in the room with hoses connected to the robot. Mm-hmm. It had a VAX computer in the next room. It needed its own room because it was this giant thing with air conditioning.

And it had this very complicated bus connected to the robot. And the robot itself just had the actuators. It had gyroscopes for sensing and some other sensors. But all the power and computing was off-board. Big Dog had all that stuff integrated on the platform. It had a gasoline engine for power, which was a very complicated thing to undertake.

It had to convert the rotation of the engine into hydraulic power, which is how we actuated

uh it so there was a lot of learning just on the uh you know building the physical robot and the system integration for that and then there was the controls uh of it so for big dog you brought it all together onto one platform right and then so you could you could take it out in the woods yeah and you did we did we spent a lot of time down at the uh marine corps base in quantico where there was a trail

called the guadalcanal trail and our uh milestone that darpa had specified was that we could go on this one particular trail that involved you know a lot of challenge and we spent a lot of time our team spent a lot of time down there those were fun days hiking with the robot what did you learn about like what it takes to balance a robot like that on a trail

Yeah. As challenging as the woods were, working inside of a home or in an office is really harder. Because when you're in the woods, you can actually take any path up the hill. All you have to do is avoid the obstacles. There's no such thing as damaging the woods, at least to first order. Whereas if you're in a house, you can't leave scuff marks. You can't bang into the walls.

The robots aren't very comfortable bumping into the walls, especially in the early days. So I think those were actually bigger challenges once we faced them. It was mostly getting the systems to work well enough together, the hardware systems to work, and the controls. In those days, we did have a human operator who did all the visual perception going up the Guadalcanal Trail.

There was an operator who was right there, who was very skilled at Even though the robot was balancing itself and placing its own feet, if the operator didn't do the right thing, it wouldn't go.

But years later, we went back with one of the electric, the precursor to Spot, and we had advanced the controls and everything so much that an amateur, complete amateur, could operate the robot the first time up and down and up and down, whereas it had taken us years to get there in the previous robots.

So Big Dog became LS3, which is the big load carrying one.

It was designed to carry 400, but we had it carrying about 1,000 pounds. Of course you did.

We had one carrying the other one. We had two of them. So we had one carrying the other one. There's a little clip of that. We should put that out somewhere. That's from like 20 years ago. Wow.

So, Big Dog and LS3 had engine power and hydraulic actuation. Then we made a robot that was Electric power, so there's a battery driving a motor, driving a pump, but still hydraulic actuation. Larry sort of asked us, could you make something that weighed 60 pounds that would not be so intimidating if you had it in a house where there were people?

And that was the inspiration behind the spot, pretty much as it exists today. We did a prototype the same size that was the first all-electric thing Non-hydraulic robot.

I mean, it was almost as simple as what I just said. You know, we were having a meeting. He said, yeah, geez, do you think you could make a smaller one that wouldn't be so intimidating, like a big dog, if it was in your house? And I said, yeah, we could do that. And we started and did.

I had been in love with hydraulics and still love hydraulics. It's a great technology. It's too bad that somehow the world out there looks at it like it's old-fashioned or that it's icky. It's true that you do. It is very hard to keep it from having some amount of dripping from time to time. But if you look at the performance, how strong you can get in a lightweight package.

And of course, we did a huge amount of innovation. Most of hydraulic control, that is the valve that controls the flow of oil, had been designed in the 50s for airplanes. It had been made robust enough, safe enough that you could count on it so that humans could fly in airplanes. And very little innovation had happened. You know, that might not be fair to the people who make the valves.

I'm sure that they did innovate. But the basic design had stayed the same. And there was so much more you could do. And so our engineers designed valves, the ones that are in Atlas, for instance, that had new kinds of circuits. They sort of did some of the computing that could get you much more efficient use. They were much smaller and lighter so that the whole robot could be smaller and lighter.

We made a hydraulic power supply that had a bunch of components integrated in this tiny package. It's about this big, the size of a football. It weighs five kilograms and it produces five kilowatts of power. Of course, it has to have a battery operating, but it's got a motor, a pump, filters, heat exchanger to keep it cool, some valves, all in this tiny little package. Hydraulics

you know, could still have a ways to go.

Well, I think having good hardware is part of the story, and people who think you don't need to innovate hardware anymore are wrong, in my opinion. So I think one of the things, certainly in the early years for me, taking a dynamic approach where you think about what's the evolution of the motion of the thing going to be,

in the future and having a prediction of that that's used at the time that you're giving signals to it, as opposed to it all being servoing, which is servoing is sort of backward looking. It says, okay, where am I now? I'm going to try and adjust for that. But you really need to think about what's coming.

It's interesting. I think that the number is only a couple of seconds for a spot. So there's a limited horizon type approach where you're recalculating, assuming what's going to happen in the next second or second and a half. And then you keep iterating. At the next, even though a tenth of a second later, you'll say, okay, let's do that again and see what's happening.

And you're looking at what the obstacles are, where the feet are going to be placed, and how to, you know, you have to coordinate a lot of things if you have obstacles and you're balancing at the same time. And it's that limited horizon type calculation that's doing a lot of that. But if you're doing something like a somersault, you're looking out a lot further, right?

If you want to stick the landing, you have to get the, you know, you have to, at the time of launch, have, you know, momentum and rotation, all those things coordinated so that a landing is within reach.

If you look at the first time we ever made a robot do a somersault, it was in a planar robot. It had a boom. It was restricted to the surface of a sphere. We call that planar. It could move fore and aft. It could go up and down, and it could rotate. The calculation of what you need to do to stick a landing isn't all that complicated.

You have to get time to make the rotation, so how high you jump gives you time. You look at how quickly you can rotate. And so if you get those two right, then when you land, you have the feet in the right place. And you have to get rid of all that rotational and linear momentum. But that's not too hard to figure out.

And we made, back in about 1985 or 6, I can't remember, we had a simple robot doing somersaults. To do it in 3D, really the calculation is the same. You just have to be balancing in the other degrees of freedom. If you're just doing a somersault, it's just a planar thing.

When Rob was my graduate student and we were at MIT, which is when we made a two-legged robot do a 3D somersault for the first time. There, in order to get enough rotation rate, you needed to do tucking also. You know, withdraw the legs in order to accelerate it. And he did some really fascinating work on how you stabilize more complicated maneuvers.

You remember he was a gymnast, a champion gymnast before he'd come to me. So he had the physical abilities. And he was an engineer, so he could translate some of that into the math and the algorithms that you need to do that.

Unfortunately, though, humans don't really know how they do it, right? We're coached. We have ways of learning, but do we really understand in a physics way what we're doing? Probably most gymnasts and athletes don't know.

Atlas still doesn't walk like a person, and it still doesn't walk quite as gracefully as a person, even though it's been getting closer and closer. The running might be close to a human, but the walking is still a challenge.

And something weird about the knee, that you can kind of do this folding and unfolding, And get it to work out just, a human can get it to work out just right. There's compliances. Compliance means springiness in the design that are important to how it all works. Well, we used to have a motto at Boston Dynamics in the early days, which was, you have to run before you can walk.

For a quadruped, probably. Of course, it was probably the loudest, too. So we had this little racing go-kart engine on it, and we would get people from three buildings away sending us, complaining about how loud it was.

Well, it's always a balance between wanting to get where you really want to get and what's practical to do based on your resources or what you know and all that. So, I mean, the whole idea of the pogo stick was to do simplification. Obviously, it didn't look like a human.

I think a technical scientist could appreciate that we were capturing some of the things that are important in human locomotion without it looking complicated. like it without having a knee and ankle. I'll tell you the first sketch that Ben Brown made when we were talking about building this thing was a very complicated thing with zillions of springs, lots of joints.

It looked much more like a kangaroo or an ostrich or something like that, things we were paying a lot of attention to at the time. Um, you know, so my job was to, uh, say, okay, well let's do something simpler to get started and maybe we'll get there at some point.

Oh yeah. We, we did, uh, we filmed and digitized, uh, data from horses. I did a dissection of a ostrich at one point, which has absolutely remarkable legs.

Most of it's up in the feathers, but there's a huge amount going on in the feathers, including a knee joint. The knee joints way up there. The thing that's halfway down the leg that looks like a backwards knee is actually the ankle. The thing on the ground, which looks like the foot, is actually the toes. It's an extended toe. Fascinating.

But, you know, the basic morphology is the same in all these animals.

You know, the slow-mos of cheetahs running are incredible. You know, they're they, there's so much back motion and, uh, you know, grace. And of course they're moving very fast. Uh, The animals running away from the cheetah are pretty exciting.

You know, the pronghorn, which, you know, they do this all four legs at once jump called the prong to kind of confuse the, especially if there's a group of them, to confuse whoever's chasing them. So they do like a misdirection type of thing? Yep, they do a misdirection thing.

The front on views of the cheetahs running fast where the tail is whipping around to help in the turns, to help us stabilize in the turns. That's pretty exciting.

But they also turn very fast.

Everything in the body is probably helping turn, because they're chasing something that's trying to get away that's also zigzagging around. But I would be remiss if I didn't say humans are pretty good, too. You watch gymnasts, especially these days, they're doing just incredible stuff.

In Big Dog. Big Dog. Yeah, Big Dog came first, and then Little Dog was later. And you know, there was a... There's a big compromise there. Human knees have multiple muscles, and you could argue that there's, I mean, it's a technical thing about negative work. When you're contracting a joint, but you're pushing out, that's negative work.

And if you don't have a place to store that, it can be very expensive to do negative work. And in Big Dog, there was no place to store negative work in the knees. But Big Dog also had pogo stick springs down below. So part of the action was to comply in a bouncing motion. Later on in Spot, we took that out. As we got further and further away from the leg lab, we had more energy-driven controls.

Sure. There's this idea called passive dynamics, which says that although you can use computers and actuators to make a motion, a mechanical system can make a motion just by itself if it gets stimulated the right way. Uh, so, uh, Tad McGeer in the, uh, I think in the mid eighties, uh, maybe it was in the late eighties starting to, started to work on that.

And he made this, uh, legged system that could walk down an inclined plane where the legs folded and unfolded and swung forward, you know, do the whole walk, walking motion where the only thing, there was no computer, uh, There were some adjustments to the mechanics so that there were dampers and springs in some places that helped the mechanical action happen.

It was essentially a mechanical computer. The interesting idea there is That it's not all about the brain dictating to the body what the body should do. The body is a participant in the motion.

I think that these days most robots aren't doing that. Most robots are... are basically using the computer to govern the motion. Now, the brain, though, is taking into account what the mechanical thing can do and how it's going to behave.

Otherwise, it would have to really forcefully move everything around all the time, which probably some solutions do, but I think you end up with a more efficient and more graceful thing if you're taking into account what the machine wants to do.

I like to talk about intelligence having two parts, an athletic part and a cognitive part. And I think Boston Dynamics, in my view, has sort of set the standard for what athletic intelligence can be. And it has to do with all the things we've been talking about, the The mechanical design, the real-time control, the energetics, and that kind of stuff.

But obviously, people have another kind of intelligence. And animals have another kind of intelligence. We can make a plan. Our meeting started at 9.30. I looked up on Google Maps how long it took to walk over here. It was 20 minutes. So I decided, okay, I'd leave my house at 9.00. which is what I did. You know, simple intelligence, but we use that kind of stuff all the time.

It's sort of what we think of as going on in our heads. And I think that's in short supply for robots. Most robots are pretty dumb. And as a result, it takes a lot of skilled people to program them to do everything they do. And it takes a long time. And if robots are gonna satisfy our dreams, they need to be smarter.

So the AI Institute is designed to combine that physicality of the athletic side with the cognitive side. So, for instance, we're trying to make robots that can watch a human do a task, understand what it's seeing, and then do the task itself. So sort of OJT, on-the-job training for robots, as a paradigm. Now, you know, that's pretty hard...

It's sort of science fiction, but our idea is to work on a longer timeframe and work on solving those kinds of problems. I have a whole list of things that are in that vein.

And using our hands, you know. Using your hands. The mechanics of interacting with all these things.

these two things you know you've never touched those things before right well i've touched ones like this okay look at all the things i can do right i can juggle and i'm rotating this way i can rotate it without looking i could fetch these things out of my pocket and figure out which one was which and all that kind of stuff yeah and uh i don't think we have much of a clue how all that works yet right and that's i really like putting that under the banner of athletic uh intelligence

I mean, one question you could ask, it isn't my question, but, you know, are they commercially viable? Will they increase productivity? And I think... You know, we're getting very close to that. I don't think we're quite there still. You know, most of the robotics companies, it's a struggle.

It's really the lack of the cognitive side that probably is the biggest barrier at the moment, even for the physically successful robots. But your question is, I mean, you can always do a thing that's more efficient, lighter, more reliable. I'd say reliability. I know that Spot, they've been working very hard on getting the the tail of the reliability curve up and they've made huge progress.

So the robots, you know, there's 1,500 of them out there now, many of them being used in practical applications day in and day out where, you know, where they have to work reliably. And, you know, it's very exciting that they've done that, but it takes a huge effort to get that kind of reliability in the robot. There's cost, too. You'd like to get the cost down.

Spots are still pretty expensive, and I don't think that they have to be, but it takes a different kind of activity to do that. Now that... I think that Boston Dynamics is owned primarily by Hyundai now, and I think that the skills of Hyundai in making cars can be brought to bear in making robots that are less expensive and more reliable and those kinds of things.

That's a good question. I think we're using the paradigm called stepping stones to moonshots. I don't believe, that was in my original proposal for the Institute, stepping stones to moonshots. I think if you go more than a year without seeing a tangible status report of where you are, which is the stepping stone, It could be a simplification.

You don't necessarily have to solve all the problems of your target goal, even though your target goal is going to take several years. Those stepping stone results give you feedback, give motivation, because usually there's some success in there. That's the mantra we've been working on. That's pretty much how I'd say Boston Dynamics has worked, where you make progress and show it as you go.

Show it to yourself, if not to the world.

Well, with Watch, Understand, Do, the project I mentioned before, we've broken that down into getting some progress with what is meaningfully watching something mean, breaking down an observation of a person doing something into the components, segmenting. You watch me do something. I'm going to pick up this thing and put it down here and stack this on it.

Well, it's not obvious if you just look at the raw data. what the sequence of acts are. It's really a creative, intelligent act for you to break that down into the pieces and understand them in a way so you could say, okay, what skill do I need to accomplish each of those things?

So we're working on the front end of that kind of a problem where we observe and translate the, if it may be video, it may be live, into

a description of what we think is going on and then try and map that into skills to accomplish that and we've been developing skills as well so you know we have kind of multiple stabs at the pieces of of doing that and this is usually video of humans manipulating objects with their hands kind of thing we're starting out with bicycle repair some simple bicycle repair oh no that seems complicated that seems really complicated it is but but there's some parts of it that aren't

Like putting the seat in, you know, into the, you know, you have a tube that goes inside of another tube and there's a latch. You know, that should be within range.

I think it is. And I think that's the kind of thing that people don't recognize. Let me translate it to navigation. Mm-hmm. I think the basic paradigm for navigating a space is to get some kind of sensor that tells you where an obstacle is and what's open, build a map, and then go through the space.

But if we were doing on-the-job training where I was giving you a task, I wouldn't have to say anything about the room. We came in here, all we did is adjust the chair, but we didn't say anything about the room, and we could navigate it. So I think there's opportunities to build that kind of navigation skill into robots. And we're hoping to be able to do that.

Yeah, and lack of specification.

I mean, that's what sort of intelligence is, right? Kind of dealing with, understanding a situation even though it wasn't explained.

You know, Since ChatGBT, which is a year ago, basically, there's a huge interest in that and a huge optimism about it. And I think that there's a lot of things that machine learning, that kind of machine learning. Now, of course, there's lots of different kinds of machine learning. I think there's a lot of interest and optimism about it.

I think the facts on the ground are that doing physical things with physical robots is a little bit different than language. The tokens sort of don't exist. Pixel values aren't like words. But I think that there's a lot that can be done there. We have... We have several people working on machine learning approaches.

I don't know if you know, but we opened an office in Zurich recently, and Marco Hutter, who's one of the real leaders in reinforcement learning for robots, is the director of that office. He's still half-time at ETH now.

the university there where he has an unbelievably fantastic lab and then he's half time leading will be leading off efforts in the Zurich office so we have a healthy learning component but there's part of me that still says if you look out in the world at what the most impressive performances are

They're still pretty much, I hate to use the word traditional, but that's what everybody's calling it, traditional controls, like model predictive control. The Atlas performances that you've seen are mostly model predictive control. They've started to do some learning stuff that's really incredible. I don't know if it's all been shown yet, but you'll see it over time.

And then Marco has done some great stuff and others.

I think we're going to find a mating of the two and we'll have the best of both worlds. And we're working on that at the Institute too.

Sure, technical fearlessness means being willing to take on a problem that you don't know how to solve. Study it, figure out an entry point, maybe a simplified version or a simplified solution or something. Learn from the stepping stone and go back and eventually make a solution that meets your goals. And I think that's really important.

Yeah, and you don't know how to do it. There's easier stuff to do in life. I mean, I don't know. Watch, understand, do. It's a mountain of a challenge.

Yeah. I mean, we have others like that. We have one called Inspect, Diagnose, Fix. You call up the Maytag repairman. Okay, he's the one who you don't have to call, but you call up the dishwasher repair person, and they come to your house, and they look at your machine,

It's already been actually figured out that something doesn't work, but they have to kind of examine it and figure out what's wrong and then fix it. And I think robots should be able to do that. Boston Dynamics already has spot robots collecting data on machines, things like thermal data, reading the gauges, listening to them, getting sounds.

And that data are used to determine whether they're healthy or not. But the interpretation isn't done by the robots yet. And certainly the fixing, the diagnosing and the fixing isn't done yet. But I think it could be. And that's bringing the AI and combining it with the physical skills to do it.

You mean convincing you it's okay that the dishwasher's broken?

Why is diligence important? Well, if you want a real robot solution, it can't be a very narrow solution that's going to break at the first variation in what the robot does or the environment if it wasn't exactly as you expected it. So how do you get there? I think having an approach that leaves you unsatisfied until you've embraced the bigger problem is the diligence I'm talking about.

Again, I'll point at Boss Dynamics. Some of the videos that we had showing the engineer making it hard for the robot to do its task. spot opening a door, and then the guy gets there and pushes on the door so it doesn't open the way it's supposed to, pulling on the rope that's attached to the robot so its navigation has been screwed up.

We have one where the robot's climbing stairs and an engineer is tugging on a rope that's pulling it back down the stairs. That's totally different than just the robot seeing the stairs, making a model, putting its feet carefully on each step. But that's what probably robotics needs to succeed, and having that broader

That broader idea that you want to come up with a robust solution is what I meant by diligence.

I learned something very early on with the first three-dimensional hopping machine. If you just show a video of it hopping It's a so what. If you show it falling over a couple of times and you can see how easily and fast it falls over, then you appreciate what the robot's doing when it's doing its thing.

So I think the reaction you just gave to the robot getting kind of interfered with or tested while it's going through the door, it's showing you the scope of the solution.

Well, I was the final edit for most of the videos up until until about three years ago or four years ago. And my theory of the video is no explanation. If they can't see it, then it's not the right thing. And if you do something worth showing, then let them see it. Don't interfere with a bunch of titles that slow you down or a bunch of distraction. Just do something worth showing and then show it.

That's brilliant. It's hard though for people to buy into that.

People have criticized, especially the big dog videos where there's a human driving the robot. And I understand the criticism now. At the time, we wanted to just show, look, this thing's using its legs to get up the hill. So we focused on showing that, which was, we thought, the story.

The fact that there was a human, so they were thinking about autonomy, whereas we were thinking about the mobility. And so, you know, we've adjusted to a lot of things that we see that people care about, trying to be honest. We've always tried to be honest.

I mean, it might be the most important ingredient, and that is robotics is hard. It's not gonna work right, right away. So don't be discouraged is all it really means. So usually when I talk about these things, I show videos and I show a long string of outtakes. And you have to have courage to be intrepid when you work so hard, you built your machine,

Then you're trying and it just doesn't do what you thought it would do, what you want it to do.

There's a video of Atlas climbing three big steps, and it's very dynamic, and it's really exciting, a real accomplishment. It took 109 tries, and we have video of every one of them. We shoot everything. Again, we, this is at Boston Dynamics. So it took 109 tries. But once it did it, it had a high percentage of success.

So it's not like we're cheating by just showing the best one, but we do show the evolved performance, not everything along the way. But everything along the way is informative, and it shows sort of there's stupid things that go wrong, like the robot just when you say go and it collapses right there on the start. That doesn't have to do with the steps.

Or the perception didn't work right, so you miss the target when you jump, or something breaks and there's oil flying everywhere. Yeah. But that's fun. Yeah.

Lots of control of evolution during that time. I think it took six weeks to get those 109 trials. Because there was programming going on. It was actually robot learning, but there were humans in the loop helping with the learning. So all data-driven.

It's remarkable. One of the accomplishments of Atlas is that the engineers have made a machine that's robust enough that it can take that kind of testing where it's falling and stuff, and it doesn't break every time. It still breaks. Part of the paradigm is to have people to repair stuff. You got to figure that in if you're going to do this kind of work.

I sometimes criticize the people who have their gold-plated thing and they keep it on the shelf and they're afraid to use it. I don't think you can make progress if you're working that way. You need to be ready to have it break and go in there and fix it. It's part of the thing. Plan your budget so you have spare parts and a crew and all that stuff.

I think it applies to everything anybody tries to do that's worth doing.

Fun. Technical fun, I usually say. I have technical fun. I think that life as an engineer... is really satisfying. To some degree, it can be like crafts work where you get to do things with your own hands or your own design or whatever your media is. It's very satisfying to be able to just do the work, unlike a lot of people who have to do something that they don't like doing.

I think engineers typically get to do something that they like, and there's a lot of satisfaction from that. In many cases, you can have impact on the world somehow because you've done something that other people admire, which is different from just the craft fun of building a thing. So that's a second thing. way that being an engineer is good.

I think the third thing is that if you're lucky to be working in a team where you're getting the benefit of other people's skills that are helping you do your thing, none of us has all the skills needed to do most of these projects. If you have a team where you're working well with the others, that can be very satisfying. And then if you're an engineer, you also usually get paid.

And so you kind of get paid four times in my view of the world. So what could be better than that?

I think it's both being a scientist or getting to use science at the same time as being kind of an artist or a creator. Scientists only get to study what's out there, and engineers get to make stuff that didn't exist before.

It's really, I think, a higher calling, even though I think most of the public out there think science is top and engineering is somehow secondary, but I think it's the other way around.

That is the art. Bringing metal to life or a machine to life is kind of... It was fun doing the dancing videos where it got a huge public response. We're going to do more. We're doing some at the Institute and we'll do more.

But I think sort of with it going back to the pieces of that, you design a linkage that turns out to be half the weight and just as strong. That's very satisfying. There are people who do that, and it's a creative act.

I think having the robots move in a way that's evocative of life is pretty exciting. So the elegance of movement. Or if it's a high-performance act where it's doing it faster, bigger than other robots. Usually we're not doing it bigger, faster than people, but we're getting there in a few narrow dimensions.

I mean, I'd like to do dancing that starts... You know, we're nowhere near the dancing capabilities of a human. We've been having a ballerina in who's kind of a well-known ballerina, and she's been programming the robot.

We've been working on the tools that can make it so that she can use her way of talking, you know, way of doing a choreography or something like that more accessible to get the robot to do things. And she's starting to produce some interesting stuff.

There is.