Facebook is betting VR and augmented reality is the future of computing, but a big obstacle is how people will control the technologies, without wearing a bunch of clunky hardware. On Thursday, the company debuted a possible solution: a wristband that can sense your finger and hand movements, and translate them into the digital world.
The wristband's technology comes from a startup that Facebook acquired in 2019. The device itself doesn’t read your mind; it detects the electrical signals moving through your muscles along your arm. The signals can then be analyzed to understand intent, like whether you’re trying to move an index finger or make a fist.
“The signals through the wrist are so clear that EMG (electromyography) can understand finger motion of just a millimeter. That means input can be effortless. Ultimately, it may even be possible to sense just the intention to move a finger,” Facebook says.
The company is currently developing the wristband to detect simple gestures. This includes pinch and release, and tapping your fingers together. “But that’s just the first step. EMG will eventually progress to richer controls. In AR, you’ll be able to actually touch and move virtual UIs and objects,” the company says. “You’ll also be able to control virtual objects at a distance. It’s sort of like having a superpower like the Force (a Star Wars reference).”
The same wristband also promises to act as a virtual keyboard. In a video, Facebook showed how this would work. When wearing two wristbands, the technology can detect the typing motions from your hands, and adapt to your typing methods.
“This will be faster than any mechanical typing interface, and it will be always available because you are the keyboard. And the beauty of virtual typing and controls like clicking is that people are already adept at using them," Facebook wrote in a separate blog post.
The wristband will also work alongside an AI, which will try and predict your requests. For example, over time, the AI will understand your daily routines, and how you like to play music in the morning, or read the news at night. Another feature includes adding haptic feedback to the wristbands, allowing them to vibrate as you touch and control objects in the digital world.
However, Facebook isn’t saying when any of this technology will arrive, just that “it’s still early days, but the future is promising.” The other challenge is working through the ethical implications around processing the wearer’s data. The wristband itself is but one piece of Facebook’s vision for the future of computing. The company is also working on a “mind-reading” headset capable of measuring brain activity to interpret your commands.
“As we invent new technologies, we are committed to sharing our learnings with the community and engaging in open discussion to address concerns,” wrote Facebook Reality Labs Research Science Director Sean Keller.