While Apple is about to make a big move into augmented reality, Google's attempts so far have been impractical at best. But Google now has a major ally: chipmaker Qualcomm—currently at war with Apple over license fees—which says it will enable a much broader range of Android phones to do AR.
The keys will be Qualcomm's new Spectra Module Program and second-generation Spectra image processors, both announced this morning. The module program gives Android phone makers easy access to pre-configured, AR-compatible two- and three-camera setups, while the new image signal processor (ISP) includes the ability to track location through a 3D environment.
"This is the first of its kind and will be available to the entire Android phone ecosystem via our Camera Module Program and will be made available in our next-generation flagship SoC," Qualcomm said in a statement.
While virtual reality has gotten a lot of buzz so far thanks to gaming systems like the Oculus Rift and HTC Vive, most analysts predict that augmented reality will outpace VR starting in 2018. (Virtual reality blacks out the world around you; augmented reality enhances it.)
At Apple's developer conference in June, the company announced ARKit, a set of software tools for developing AR applications. ARKit will work with existing iPhones, but you really need at least two cameras for the depth-sensing in more advanced AR. While Apple hasn't publicly connected the dots, most observers expect this year's iPhones to include broad dual-camera AR capabilities, and for those abilities to filter back into the dual-camera iPhone 7 Plus. That will immediately give Apple a lead when it comes to installed, AR-capable devices.
Google has had an AR system called Tango for more than a year now, but it hasn't become popular because it requires a specific three-camera setup that's difficult and expensive for manufacturers to implement. So far, it has only appeared on two phones, the Lenovo Phab 2 Pro and the Asus Zenfone AR (pictured above), neither of which have been huge sellers.
Enter Spectra
Spectra is the (ISP) in Qualcomm's Snapdragon mobile platforms, which power most of the Android smartphones sold in the US.
The company is offering three new Spectra modules to phone makers. One has an iris scanner and doesn't have any AR capabilities. The second has two cameras for "passive" AR. The third has three cameras, one of which is an infrared camera that can map objects and spaces in 3D, for high-quality, active AR like you currently get with Microsoft's HoloLens. Of course, Android phone makers will be welcome to use their own camera modules with the new ISPs as well.
Qualcomm is very, very proud of "point cloud," which provides 3D mapping of objects and spaces 10-15 feet away in real time using the new module's infrared camera. It will allow for flexible, responsive augmented reality even in low light.
The point cloud also allows for some neat VR tricks. Because standalone VR headsets powered by Spectra will be able to see the position of your hands, they could work with virtual controllers rather than physical ones.
"Depth sensing will revolutionize standalone VR and AR headsets as tracking and object avoidance becomes more precise and hand-held controllers obsolete," the company said in a statement.
There are still a lot of missing pieces in this announcement. Google hasn't announced that Tango will work with two-camera setups. And the new Spectra ISP won't be a standalone product; it'll be part of next year's Snapdragon, which most people call the 845 although its official name hasn't been announced yet.
This shows one of the major challenges of the Android world. If Apple wants to do AR, Apple just does AR. But if, say, LG wants to do AR, it has to wait for both Qualcomm and Google to align properly. If all the pieces come together, this will put broad AR capabilities in flagship phones around the middle of next year: the Galaxy S9, LG V40 and Moto Z3 generations.