These days any old schlub can pilot a drone without cratering it, what with good old autopilot tech, but there are drone pilots out there whose abilities push the limits of human cognition. Drone racing is a truly insane endeavor (now with its very own Drone Racing League!) with human pilots banking around corners and through obstacles at over 100 miles per hour, navigating it all through the craft’s onboard camera. It takes an almost unimaginable amount of coordination—but, alas, even this highly skilled job is in danger of automation.
Researchers have developed a system that allows drones to autonomously navigate an obstacle course of gates with 100 percent accuracy—that is, the robots don’t crash into something and explode. Not only that, because of the clever way the researchers trained the drones, the machines can adapt if a wily human moves a gate mid-run, completing a course that looks different than when they started. They run a bit slow at the moment compared to human pilot, sure, but they’ll only get faster from here.
When you think about robots roaming the world, you probably think first of self-driving cars. These can only work because engineers first use lidar to coat the world in beams of light, mapping it in fine detail. This helps the machines localize themselves in the static environment—trees and buildings and such.
But a new class of machines are beginning to sense their world more like we do. Boston Dynamics, for instance, makes the famous SpotMini robot dog. This machine doesn’t use lidar because lidar is computationally and energetically expensive. So instead, a handler remote-controls the machine through an environment as cameras capture its surroundings. Armed with this information, the robot can then walk the same route autonomously, using its cameras to eyeball a now-familiar world.
This new drone system works in much the same way. You can’t bolt a bulky lidar on a drone and expect it to get off the ground, so this system also runs on cameras. The researchers trained the drones by, well, holding them and “flying” them through the obstacle course first (comical mouthed airplane noises excluded), like SpotMini first walking a route. This allowed them to collect images, tens of thousands of them. The researchers used all this data to train a neural network on how to fly through the obstacle course, not with a detailed 3-D lidar map, but with sight.
When they let the drone loose, it could navigate autonomously using its onboard camera. “The drone receives an image from the camera and the neural network outputs, Hey drone, now you have to go two meters to the left,” says University of Zurich roboticist Antonio Loquercio, who helped develop the system. The drone is constantly taking in these images, processing them, and correcting its course, all based on its training on the neural network.
Because the drone isn’t just relying on a static map of its environment, it’s better equipped to react to the unexpected. “During data generation, we moved one or two gates on the track and adapted the trajectory,” says roboticist Elia Kaufmann, also of the University of Zurich. In other words, part of the robot’s training was to deal with changes in the environment.
Even when humans throw it challenges like this, the drone managed to complete 50 out of 50 laps without a collision. In fact, it bested a pro pilot the researchers brought in to fly the same course, who managed 45 out of 50, albeit at a greater speed—the human was an average of three times faster than the robot. “Drone pilots fly very, very aggressively,” says Loquercio. “They are more open to take risks, way less conservative than what our current approach is.”
These thinky machines are conservative both by design and by limitations of the technology. Drones are expensive, and the researchers preferred to not destroy them—plus the neural network isn’t powerful enough to match humans’ blazing speeds. We’re talking about a lot of data the drone has to crunch on the fly.
Impressively enough, though, the drone is already doing it all onboard, as opposed to tapping into a computer through Wi-Fi, thanks to more efficient neural network algorithms and a fairly burly processor—for a drone at least. And the machines will only get more powerful, and may soon match or best the speed of pro pilots, all while maintaining that coveted accuracy.
This approach could also find its way into other applications in the robotic future. Our skies are about to fill up with drones, delivery drones in particular, that will have to avoid not just one another, but the many obstacles of the big city. If they just relied on static maps, it’d be chaos (read: drones colliding and plummeting out of the sky and onto our heads). Drones will have to more dynamically adapt to their surroundings to be safe and effective.
For the nearer future, though, drone racers take note: the competition is about to get tougher. Just never make a mistake and you should be fine…