So what’s 3D navigation really good for?
After learning how to fly around the world, I stepped back to think about where the innovations might lie – what the ‘low hanging fruit’ and the ‘holy grail’ might be for making the most of Leap Motion and similar 3D interfaces.
Touch-free
In some environments, not touching a control device is a Good Thing. A few scenarios that quickly came to mind:
- wearing gloves (e.g. for insulation or protection) which could make it hard to use conventional devices (like mice and keyboards)
- not wanting dirty hands (or gloves) to crud up control devices, and
- not wanting clean hands (or gloves) to be contaminated by potentially dirty control devices.
To me, these exploits seem like the low-hanging fruit – quick wins. (I’m not saying they’ll be easy to accomplish – for instance, how much will a normal pair of winter gloves interfere with recognition of hand gestures by people working outdoors? But they’re somewhat obvious applications to target.)
Beyond touch-free
Thinking about ways to exploit touch-free user interfaces with existing applications is fun and cool, just like Tom Cruise’s gloved computer navigation in Minority Report. It’s like having a touchscreen in the air, and some useful things can be done with just that.
Yet it’s a far bigger, but more interesting, challenge to think of completely new ways to use 3D touch-free controllers to do things that maybe we don’t yet dream of being able to do. That’s where the real innovations to rock our world will come from. But how can we discover them?
Trying to ‘see the water’
We humans are capable of being “perfect filters”: our assumptions can be so deeply ingrained that often we don’t even realize we’re making them. As Alan Willett so nicely analogized at SEPGNA 2012:
“Most of us are like fish in the water – we don’t see the water.”
Just as ocean water is hard for the fish swimming in it to see, applications where 3D control (not just a 2D virtual touchscreen) is useful can be hard to see. Finding the killer apps for 3D interfaces will require questioning ourselves about assumptions we probably don’t even realize we’re making about how we interact with our computers.
3D spreadsheets FTW?
As an example, look at how we use spreadsheets to store and manipulate data. We are so accustomed to a 2D paradigm that thinking about navigating our row,column oriented spreadsheet tools with a 3D interface like Leap Motion seems silly. (At least, that’s what many of my colleagues said when I asked if or how a Leap controller could help them do their daily work on computers).
But what if 3D control became commonplace? What kinds of workbook tools could we have, and what new insights might we gain, if we could explore data cubes instead of sheets?
Look Ma, two hands
What sophisticated controls might be possible with two coordinated hands instead of one? In my experience this is very hard to do with devices, yet humans can learn to do things with two independent hands all the time, like playing the guitar. (Seriously, have you ever tried to use two joysticks or mice at once, one per hand? If you’ve ever gotten it to work, let me know! I’m not a hard-core gamer, but to me, it’s like patting your head and rubbing your tummy at the same time.)
The holy grail for gestural control
A perfect storm for gestural controls would seem to be be combining all three benefits: to have touch-free, two-handed, 3D navigation. Now that I want to see – and do – someday soon.
Leap Motion and the Leap Motion logo are the trademarks of Leap Motion, Inc. and are used here by permission.
Photo by DAVID ILIFF. License: CC-BY-SA 3.0
Agile Teams 
