Two privacy improvements

Tonight I finally tackled improving my online privacy by making two changes:

DuckDuckGo promises no ads and no tracking.

Zoho mail promises no email scanning, and says they are Safe Harbour compliant.

So far, so good with both. I only ran into one quirk getting Zoho IMAP configured on my phone (need to use syntax server:port). Liking DDG so much already that I got their iPhone search app, and am just loving the speed and relevance of the search results.

I’ll post an update after a few weeks to months of runtime with them. For now, I expect to sleep better tonight!

How applications for 3D user interfaces are like the ocean

Pristis pectinata - Georgia Aquarium Jan 2006

So what’s 3D navigation really good for?

After learning how to fly around the world, I stepped back to think about where the innovations might lie – what the ‘low hanging fruit’ and the ‘holy grail’ might be for making the most of Leap Motion and similar 3D interfaces.


In some environments, not touching a control device is a Good Thing. A few scenarios that quickly came to mind:

  1. wearing gloves (e.g. for insulation or protection) which could make it hard to use conventional devices (like mice and keyboards)
  2. not wanting dirty hands (or gloves) to crud up control devices, and
  3. not wanting clean hands (or gloves) to be contaminated by potentially dirty control devices.

To me, these exploits seem like the low-hanging fruit – quick wins. (I’m not saying they’ll be easy to accomplish – for instance, how much will a normal pair of winter gloves interfere with recognition of hand gestures by people working outdoors? But they’re somewhat obvious applications to target.)

Beyond touch-free

Thinking about ways to exploit touch-free user interfaces with existing applications is fun and cool, just like Tom Cruise’s gloved computer navigation in Minority Report. It’s like having a touchscreen in the air, and some useful things can be done with just that.

Yet it’s a far bigger, but more interesting, challenge to think of completely new ways to use 3D touch-free controllers to do things that maybe we don’t yet dream of being able to do. That’s where the real innovations to rock our world will come from. But how can we discover them?

Trying to ‘see the water’

We humans are capable of being “perfect filters”: our assumptions can be so deeply ingrained that often we don’t even realize we’re making them. As Alan Willett so nicely analogized at SEPGNA 2012:

“Most of us are like fish in the water – we don’t see the water.”

Just as ocean water is hard for the fish swimming in it to see, applications where 3D control (not just a 2D virtual touchscreen) is useful can be hard to see. Finding the killer apps for 3D interfaces will require questioning ourselves about assumptions we probably don’t even realize we’re making about how we interact with our computers.

3D spreadsheets FTW?

As an example, look at how we use spreadsheets to store and manipulate data. We are so accustomed to a 2D paradigm that thinking about navigating our row,column oriented spreadsheet tools with a 3D interface like Leap Motion seems silly. (At least, that’s what many of my colleagues said when I asked if or how a Leap controller could help them do their daily work on computers).

But what if 3D control became commonplace? What kinds of workbook tools could we have, and what new insights might we gain, if we could explore data cubes instead of sheets?

Look Ma, two hands

What sophisticated controls might be possible with two coordinated hands instead of one? In my experience this is very hard to do with devices, yet humans can learn to do things with two independent hands all the time, like playing the guitar. (Seriously, have you ever tried to use two joysticks or mice at once, one per hand? If you’ve ever gotten it to work, let me know! I’m not a hard-core gamer, but to me, it’s like patting your head and rubbing your tummy at the same time.)

The holy grail for gestural control

A perfect storm for gestural controls would seem to be be combining all three benefits: to have touch-free, two-handed, 3D navigation. Now that I want to see – and do – someday soon.

Leap Motion and the Leap Motion logo are the trademarks of Leap Motion, Inc. and are used here by permission.

Photo by DAVID ILIFF. License: CC-BY-SA 3.0

Leaping around the world

My next adventure after Touchless app for Windows was flying through Google Earth with Leap Motion.

I quickly got GE configured for 3D control and, like everyone else, promptly had the earth spinning wildly! I also kept seeing the ‘dots’ on the screen from Touchless, along with GE’s unexplained fat and thin up/down and left/right arrows. After disabling Touchless from the system tray, the dots went away, but my control didn’t improve.

The support forum FAQ article showed two distinct user populations: those that loved flying around the world via Leaping, and those who found it frustrating. Videos seemed to help many people, so I visited the company’s channel (it had 13,741 subscribers as of my visit).

I found lots of videos, mostly 2 weeks to 2 months old. The initial promo video provided by the company for leaping with Google Earth was cool-looking, but only 29 seconds long, and pretty useless for helping someone learn how to fly like that: it was impossible to judge correct hand position relative to the controller because the controller wasn’t visible in the video. More promising was a playlist of 40 videos on how to #LeapInto Google Earth, plus some direct video links from the support forum.

Fortunately, it became clear that some early adopters who tried it out had started documenting. Julia’s #LeapInto Sweden was short but helpful. By far the most useful link I found for guidance on flying without spinning was Leap Motion – Google Earth instructions. This video showed correct hand orientation and control, and explained what the fat and skinny arrows meant and how to navigate.

With that start, within an hour I was able to navigate comfortably around the buildings of NC State Centennial Campus and around Raleigh. Relaunching it an hour later required a bit of re-learning, but I was soon able to fly around San Francisco and under the Golden Gate Bridge, just like another demo video showed. I spent another 30 min or so flying to Vatican City, Sweden, the Statue of Liberty, and some places I used to live in the US.

Now I actually feel comfortable enough to consider showing it at an internal technology demo tomorrow. I can’t wait to hear what people think!

Leap Motion and the Leap Motion logo are the trademarks of Leap Motion, Inc. and are used here by permission.

All other trademarks and registered marks are the property of their respective owners.

Leaping into Windows 7

Next adventure after Painting by Leaping: the Touchless app for Windows – can I wave at my laptop to scroll and click instead of using a mouse?

I found a developer video (based on v0.7.3) illustrating Visualizer and other functions on Windows 8, and another video which seemed to show using Touchless instead of keyboard and mouse on Windows 8. Great, but I have Windows 7.

One thing that perplexed me after installing my Leap Motion software was the appearance of a new “Flicks” system tray item. Microsoft’s “What are flicks?” article says flicks are for a pen or finger only. It seemed logical though that the ‘flicks’ conventions for navigation (e.g. upward flick to scroll down) would apply to the Leap Motion controller.

Today (August 8) I launched the Control Panel to try out the Visualizer and was prompted to download another update. I did so, then tried launching Touchless from Airspace. It threw up the User Account Control dialog, I granted permissions, then – nothing. It just stopped. I saw one brief popup from the system tray about a smudge and needing to clean the window of the controller. I did that, but still no dice.

Restarted just for fun and relaunched Airspace and the Touchless app. No apparent change – nothing happened when I moved my hands over the sensor. This time I looked for hidden system tray icons, and found a new one. It didn’t actually say “Touchless” – it just said “Interaction”. “Open Help Guide” launched IE8 (my default browser for corporate reasons) with the tutorial. It doesn’t run well in IE though. Switching to Chrome let me see the tutorial. Back in the system tray icon, I looked under the Interaction submenu and saw that it was “Disabled”. I changed it to “Basic” and now I was able to see fingers on screen. Yay!

One immediate observation: the mapping of gestures to monitor only works on the primary monitor – no obvious way to navigate or control apps on the second monitor that I use when my laptop is in its dock.

Next I tried practicing scrolls and clicks and zooms on the tutorial page in Chrome. Clicking in the vertical scroll bar worked. The scroll and zoom gestures didn’t. Tried a Word document and had trouble scrolling in that, although a light overlay of a mouse kept showing on screen. I had some luck “clicking” the mouse buttons, but that’s not really what the sensor is good for! The keyboard icon popped up sporadically too – it wasn’t clear what I was doing with my gestures to trigger it. I opened a PDF (saved earlier from the PC Magazine article “10 Best Apps for the Leap Motion Controller“) and had similar results trying to navigate in Reader.

I revisited the Support Forum at this point to look for clues. I found one topic on what to do if the error “Touch Emulation Driver Corrupt or missing” appeared. Well, I wasn’t seeing that error, but touch emulation certainly wasn’t right. So I figured it wouldn’t hurt to try the advice – manually installing MultiTouch from the Leap Motion directory. After a User Account Control dialog, I got a dialog for about 15 seconds saying “Installing, this could take a few minutes…” When it finished, I switched back to Reader, and now two-finger zooming worked. Yay!

All in all, some progress and some frustrations. I tend to work with multiple apps and windows open at once, and I frequently made accidental clicks on other windows on the screen. I clearly need practice – and will look for more crowd-sourced tips before I try it again.

For fun, I tried turning ‘flicks’ off from the system tray. It didn’t seem to affect behavior at all.

Next stop: world travel with Leap Motion and Google Earth (the 11th item listed in the PC Magazine article).

Leap Motion and the Leap Motion logo are the trademarks of Leap Motion, Inc. and are used here by permission.

Painting by leaping

Adding my first Airspace app

After getting started with my Leap Motion controller, I proceeded to ‘got’ the free Corel® Painter® Freestyle™ app. The prerequisites listed on the page looked OK, although the page warned that it would need 79mb and ‘more space might be needed’ for .NET 4.0; so this app will make an interesting test of whether .NET 4.5 would meet the dependency.

When I clicked “Get App”, the Airspace store page for Corel Painter® Freestyle™ prompted me for my password, then quickly switched to ‘Purchased’, and my computer got a bit sluggish. For a few minutes, I assumed the app was being downloaded in the background to my laptop.

I did something else for about 15 minutes, but my Airspace home page hadn’t yet refreshed to include a new app button, so it wasn’t clear what was going on. [Based on later experiences with app ‘purchases’, no apps seem to actually download until Airspace is refreshed by exiting and re-launching.] I tried exiting the Airspace page by File|Exit, to no effect. Closing it by clicking the X at top right worked, though.

On restarting Airspace from my desktop shortcut, it detected that my LM sensor wasn’t attached, and offered me the option of continuing with mouse only. Again, this feature probably originated as a convenience for development and testing, but leaving it in for customers is a sign of thoughtful error handling in the design of the software. I selected that option and continued.

My Airspace page now loaded with the 5 previous apps plus my new Painter Freestyle app which began “Downloading …” About 15 minutes later, the download had finished. This seemed quite slow, even given that the app needed about 150 mb for its application files. Then I got a new Dependency page that didn’t tell me exactly what was needed, only asked me to install the “Painter® Freestyle™ Dependencies”. I always like to know what’s inside those boxes before I install them – so for now, I didn’t.

Painting in Airspace

Once I got the Orientation app working (see next story), I was able to launch this app without any difficulty, and without installing any additional prerequisites. It launched pretty fast and ran well (unlike Cut-The-Rope, which still doesn’t run).

Actually painting something with the app clearly takes practice. On my first tries, I found it hard to move to the toolbar above the paint area (to choose a new brush or color) without unintentionally painting. This needs more trial time before I can ‘draw’ any conclusions.

Saving a Painting

Saving my crude paintwork as .RIF (the default) yielded a 2644kb file (for a 1280×800 canvas). Windows, of course, didn’t know how to open it from Explorer. I was pleasantly surprised to be able to open the file from the app and save it as .PNG without using the controller (.JPEG, .TIFF, and .PSD formats are also supported). The PNG file wasn’t much smaller, but saving the JPG as ‘good’ quality or 65% yielded a compact 181kb file; at 80% or ‘high’ quality, the file was 265kb.

I’m not much of a non-digital artist, so I don’t expect to be creating any masterpieces with the app. 🙂 Strictly for the curious, here’s a half-size image of the example:

First try at using Corel® Painter® Freestyle™ with the Leap Motion controller

First try at using Corel® Painter® Freestyle™ with Leap Motion

More adventures to follow …

  1. Leap Motion and the Leap Motion logo are the trademarks of Leap Motion, Inc. and are used here by permission.
  2. Corel®, Freestyle™, and Painter® are trademarks or registered trademarks of Corel Corporation and/or its subsidiaries in Canada, the United States and/or other countries.

Orientation to leaping

Getting Orientation.exe to stop crashing

After getting started with my Leap Motion controller, I had stopped at the point where the Orientation.exe app was crashing. A few days later, I was ready to try again.

Checking for prerequisites

First step: reconfirm prerequisites. As of May 19, 2013, the minimum system requirements were:

  • Windows® 7 or 8 or Mac® OS X 10.7 – Windows 7 Enterprise (64-bit) with SP1
  • AMD Phenom™ II or Intel® Core™ i3, i5, i7 processor
  • 2 GB RAM – I have more than 8gb
  • USB 2.0 port – check
  • Internet connection – check

No problems there.

Next I looked in the Airspace Control Panel for software updates, and found one which said it fixed startup issues with Orientation.exe. But still no luck. I checked the support forum and troubleshooting guides and didn’t find anything useful. At this point (on July 29), I set my Leap Motion controller aside to work on other things.

First leap, take two

Absent any replies to my support email requests and twitter pings, I revived my investigation this past weekend (August 4) by considering possible unstated prerequisites. One that came to mind was the “.NET 4.0” that Cut The Rope apparently needs. Maybe Orientation.exe and other apps need it too?

Using Aaron Stebner’s .NET verifier, I confirmed that my prior . NET 4.5 installation did provide a functional version of .NET 4.0 ( both Client and Full).

Next I re-checked for updates (none found), and scanned the troubleshooting pages. Here I found a new Support entry from August 2 on “How to update your graphics driver“. Aha! (Should have thought of that.)

Rather than following their instructions to the letter and trying to figure out exactly which NVIDIA Quadro FX 880M driver I should manually download, I used the Windows “Update Driver” function. That worked beautifully (after about 20 min on a bandwidth-constrained connection). All I had to do after installation was reset the driver configuration to 1280×800 (listed as the minimum by many Leap Motion apps) and full width.

Getting Oriented!

Now Orientation.exe ran beautifully. I actually didn’t expect it to, so I still had my laptop on mute! I grabbed a pair of headphones to plug in and restarted it. It was way too loud at 50% headphone volume level – I reduced it to 15% and went through the app once. (Mute would have been fine. Orientation has no audio or aural instructions, and I found the music kind of annoying – more distracting than inspiring. YMMV.)

In ‘where it sees’ of Orientation, I noticed that the colors of the visualization changed towards orange or red when I made quicker moves – that was cool, and promising.

In ‘what it sees’, I quickly figured out how to get it to recognize both of my hands, and most of the time, all of my fingers. But some simple gestures that I expected to work well didn’t – e.g. gradually closing a thumb and finger into a pinch, or bringing a left finger and right finger together. In both cases, the two fingers would ‘disappear’ soon before the tips actually touched. I have not yet gotten this to work, but will keep experimenting with it.

‘How to draw’ was a good introduction to apps that use depth for ‘touching’ and ‘lifting’ from a surface. I definitely need more runtime on this too, but will probably just experiment with Corel® Painter® Freestyle™ app from now on. Trying to draw something of any complexity in Orientation, only to have the first parts erased, got to be a bit frustrating.

Who’s training who?

My techie husband saw me doing these Orientation experiments and got curious. One of his first questions was, “Are you training it, or is it training you?” It’s a great question. He shares my interest in understanding how well user interface technologies adapt to people, not vice versa. In the long run, technologies that require careful user training are not likely to ‘stick’ or be productive for sporadic use scenarios. (Key architectural quality attributes: learnability, memorability, and operability!) But for purposes of exploring a first version of a new UI technology, I don’t mind being trained a bit.

More adventures to follow …

  1. Leap Motion and the Leap Motion logo are the trademarks of Leap Motion, Inc. and are used here by permission.
  2. All other trademarks and registered marks are the property of their respective owners.