VR Drone Control

For years, I have had a dream of controlling a small squad (i.e., 3-10) of drones from VR with a combination of hand gestures and eye tracking. As an avid Starcraft player, I can't get the idea of shift-clicking (i.e., setting up a queue) of multiple actions for a drone to perform sequentially. 

As a first step, I needed to establish that it was possible to control a drone and view drone scan results from within VR, with sufficiently low latency for it to be usable. It took a few months and I learned the hard way how finnicky drones are, but I'm very happy with the end result.

The video shows the view of the VR user (left) and a camera view we had set up in the drone lab (right). The VR user gives the drone 3D navigation coordinates by placing the red sphere in various locations. You see the room scan building as the drone flies around the environment. The blue box shows where the drone has detected a human.

It's not yet the Ender's-Game-meets-Minority-Report interface I had in my head, but it's a critical first step. 

 Lessons Learned: Drones are hard to work with on multiple levels, from battery life to comms. Having a drone fly directly in front of your face is scary. Viewing 3D scan results live from VR is a powerful tool.

See Draper's post about this project here.