The U.S Patent and Trademark Office (PTO) today awarded Apple an incredibly broad patent for gesture-based controls.
The patent covers a "three dimensional user interface session control." In other words, you wave your hands and something happens on your device. When Apple bought PrimeSense last year, it was pretty clear the company wanted to own some—if not all—of the gesture-control space. Since then, all of PrimeSense's patents have been reassigned to Apple.
The best known, and most controversial, gesture control system is the Kinect. In fact, PrimeSense developed the Kinect for Microsoft, which made it a central part of its Xbox 360 and Xbox One consoles. The technology is pretty amazing, especially in its latest, high-definition incarnation. The Xbox One Kinect can detect up to 25 joints across six people. It can read a player's heartbeat, and with 1,400 points of articulation, it can tell if your mouth is open or closed. It is undeniably kick-ass technology, and no one cared.
Gamers, it turns out, prefer their controllers. Worse, including Kinect with every Xbox One allowed Sony to drastically undercut Microsoft's prices and steal market share. Last year, Microsoft de-bundled the Kinect from the Xbox One. There was never a killer app for the Kinect. At this point, it seems unlikely there ever will be.
Samsung didn't have much luck with gesture controls either. It tried out the technology with its 2012 8000 series of HDTVS. When we tested it in the Lab, it didn't quite seem ready for primetime. There was lag on the commands, and it didn't give you the ability to quickly drill down into menus. Clicking buttons on the remote was just faster. Samsung did include gesture controls in its new SUHD TVs at CES this year, but the technology was barely mentioned at the company's announcement. One assumes because no one wants it. And those who have it, don't use it.
Even the smaller, more nimble players have failed to make gesture controls appealing. The Leap Motion Controller made gesture control affordable. For less than $100, this fun, elegant device lets you add a gesture controls to any PC or Mac. I hounded the company to get a review unit and installed it as soon as it arrived. I lost interest almost immediately. It makes a great demo, but unfortunately the only practical uses were art projects and a really killer game of Cut the Rope.
Not encouraging, right?
Even so, Apple has a decent chance to succeed where others have failed. First, owning patents is always a good thing. And to my eyes, this patent is almost perilously broad. You can read it so that any gesture that can be detected by an optical or infrared controller that does anything to an interface is now owned by Apple. Kind of scary.
On a brighter note, Apple has a history of being smart with new interface technologies. It wasn't the first company to develop a touch-screen phone, but it built, arguably, the best one. The same went for tablets, a market that existed for a long time before Apple came in and transformed it with the iPad. More recently, when every laptop maker was rushing to add a touch screen to their laptops, Apple passed. It wasn't going to cram a touch experience into the laptop or desktop just because it could. It doesn't force interfaces or technologies where they don't belong. The MacBook Air and iMac are better for the restraint.
The patent actually dates back a while; Apple first filed for it in 2011. Indeed, if you look at the drawings submitted with the patent they seem positively dated—add-on Webcams and PC towers make it seem like this will be primarily a desktop computing interface. Don't be misled—this isn't about the desktop. It is about the living room.
A gesture-based control for the Apple TV makes a lot of sense, particularly if it is combined with a Siri-like voice control and the occasional iPhone or iPad-based control pad. Gestures make the most sense when you aren't in physical contact with a device, say, sitting on the couch opposite a TV. But there will always be multiple interfaces at work. Gesture controls won't replace the other interfaces, it will augment them.
Apple got it right with touch. Now it seems primed for a more hands-off approach.