It’s probably for the best that human babies can’t run 9 miles per hour shortly after birth. It takes years of practice to crawl and then walk well, during which time mothers don’t have to worry about their children legging it out of the county. Roboticists don’t have that kind of time to spare, however, so they’re developing ways for machines to learn to move through trial and error—just like babies, only way, way faster.
Yeah, OK, what you’re looking at in the video above isn’t the most graceful locomotion. But MIT scientists announced last week that they got this research platform, a four-legged machine known as Mini Cheetah, to hit its fastest speed ever—nearly 13 feet per second, or 9 miles per hour—not by meticulously hand-coding its movements line by line, but by encouraging digital versions of the machine to experiment with running in a simulated world. What the system landed on is … unconventional. But the researchers were able to port what the virtual robot learned into this physical machine that could then bolt across all kinds of terrain without falling on its, um, face.
This technique is known as reinforcement learning. Think of it like dangling a toy in front of a baby to encourage it to crawl, only here the researchers simulated 4,000 versions of the robot and encouraged them to first learn to walk, then to run in multiple directions. The digital Mini Cheetahs took trial runs on unique simulated surfaces that had been programmed to have certain levels of characteristics, like friction and softness. This prepared the virtual robots for the range of surfaces they’d need to tackle in the real world, like grass, pavement, ice, and gravel.
The thousands of simulated robots could try all kinds of different ways of moving their limbs. Techniques that resulted in speediness were rewarded, while bad ones were tossed out. Over time, the virtual robots learned through trial and error, like a human does. But because this was happening digitally, the robots were able to learn way faster: Just three hours of practice time in the simulation equaled 100 hours in the real world.
Then the researchers ported what the digital robots had learned about running on different surfaces into the real-life Mini Cheetah. The robot doesn’t have a camera, so it can’t see its surroundings in order to adjust its gait. Instead, it calculates its balance and keeps track of how its footsteps are propelling it forward. For example, if it’s walking on grass, it can refer back to its digital training on a surface with the same friction and softness as the actual turf. “Rather than a human prescribing exactly how the robot should walk, the robot learns from a simulator and experience to essentially achieve the ability to run both forward and backward, and turn—very, very quickly,” says Gabriel Margolis, an AI researcher at MIT who codeveloped the system.
The result isn’t especially elegant, but it is stable and fast, and the robot largely did it on its own. Mini Cheetah can scramble down a hill as gravel shifts underfoot and keep its balance on patches of ice. It can recover from a stumble and even adapt to continue moving if one of its legs is disabled.
To be clear, this isn’t necessarily the safest or most energy efficient way for the robot to run—the team was only optimizing for speed. But it’s a radical departure from how cautiously other robots have to move through the world. “Most of these robots are really slow,” says Pulkit Agrawal, an AI researcher at MIT who codeveloped the system. “They don't walk fast, or they can't run. And even when they're walking, they're just walking straight. Or they can turn, but they can't do agile behaviors like spinning at fast speeds.”
This kind of reinforcement learning is an increasingly popular technique in robotics: It’s impossible for an engineer to hand-code behaviors for every conceivable situation a robot might find itself in, like slipping on frozen ground or tripping on a ledge or stepping on a rock of a particular shape. “What we're seeing here is one of the great features of machine learning—it just solves the specific problem it's been given,” says Tønnes Nygaard, who studies quadruped robots at Oslo Metropolitan University but wasn’t involved in the research. “In this case, the machine learning algorithm finds the quickest way this robot can run, however wonky that might end up looking.”
Roboticists can take cues from nature, to be sure, since evolution has already put biology through the same kind of trial and error process: What helped real four-legged species survive and reproduce has been passed down through generations and continuously improved upon. But robots don’t work exactly like animals. Yes, Mini Cheetah has four legs like a real cheetah, but it has motors instead of muscles and tendons. And while the brains of cheetahs and other big cats have evolved over millions of years to seamlessly control four-legged bodies, a robot’s software can evolve much faster to control its particular physiology.
That’s the power of this reinforcement learning technique, which will be increasingly critical as robots push into more “unstructured” environments. A robot arm on an automotive assembly line is bolted into place, so it’s not designed to anticipate unexpected terrain. Mini Cheetah, on the other hand, can explore the outside world, which is complex and chaotic, full of slippery surfaces and pedestrians. For that, it’ll need to draw on its previous experiences with similar environments in simulation.
Mini Cheetah is off to an impressive start, especially since it isn’t using a complex suite of sensors to understand its world. The next step, Agrawal says, is to give the robot vision, which will enable a more complex set of behaviors, like obstacle avoidance. The team also plans to publish a paper describing the research shown in the new video.
In the meantime, says Nygaard, the experiment shows that robot movement doesn’t have to be pretty, it just has to work. “Human researchers and engineers are limited by their own notions of what a good running gait could be,” says Nygaard. “Whether that is based on old design traditions, what others have done before on similar robots, inspiration from nature, or even a subconscious preference for symmetry or 'beauty,' it often limits our approach and ultimately gives worse solutions.”
- 📩 The latest on tech, science, and more: Get our newsletters!
- It’s like GPT-3 but for code—fun, fast, and full of flaws
- You (and the planet) really need a heat pump
- Can an online course help Big Tech find its soul?
- iPod modders give the music player new life
- NFTs don’t work the way you might think they do
- 👁️ Explore AI like never before with our new database
- 🏃🏽♀️ Want the best tools to get healthy? Check out our Gear team’s picks for the best fitness trackers, running gear (including shoes and socks), and best headphones
