Design
Facebook Reality Labs, the social giant’s AR/VR wing, has created a wrist-worn device that translates your neural signals into digital commands inside of the AR world. It’s the closest thing to the Force or having a superpower.
Headsets will eventually shrink to the size of glasses (at least that’s the hope), but the most important question is input. How will we interface with the AR world? For AR to succeed as a platform, there needs to be a universal input that’s easy to understand like the mouse and keyboard and touchscreen and multi-touch. 🎥: Apple
We are very much still in the experimental phase. Facebook Reality Labs, the social giant’s AR/VR division, says it sees AR taking off as a platform when contextually-aware AI that understands what you’re trying to do and then provides the appropriate commands and actions converges with an “ultra-low-friction input.”
Facebook’s AI is some of the smartest there is when it comes to anticipating your needs. But the “ultra-low-friction input” still has a long way to go before we’re interfacing with AR naturally. For now, Facebook Reality Labs has a near-term input it’s calling the Intelligent Click.
The Intelligent Click is this AI + input inside of a wrist-worn prototype that Facebook has created to interface with augmented reality. Instead of tapping or swiping a piece of glass or using a hand controller (like with VR), the wrist gadget essentially gives you virtual hands to control AR objects.
This wrist-worn gadget uses EMG (electromyography) to “translate electrical motor nerve signals that travel through the wrist to the hand into digital commands.” The wearable isn’t detecting motion, it’s actually turning neuron signals inside of your muscles into actionable commands.
“What we’re trying to do with neural interfaces is to let you control the machine directly, using the output of the peripheral nervous system — specifically the nerves outside the brain that animate your hand and finger muscles.”
Thomas Reardon, Director of Neuromotor Interfaces at Facebook Reality Labs
Facebook tells me you could even add more fingers and control them with ease. Or, you could give a person with disabilities who may not have fingers virtual ones. In one demo, a person who was born without fingers was able to virtually grab objects using their own hand and the wrist-worn input device.
Another prototype Facebook involves a virtual QWERTY keyboard that works on any interface. Using machine learning and AI for autocorrection, you could potentially type on any surface without a physical keyboard. With practice, Facebook says it’s possible to touch type as fast as using a virtual keyboard that’s personalized to your typing style. Naturally, I’m skeptical. 🎥: Facebook
Then, there’s haptics. With a mouse, you feel a click, and with touchscreens, you might feel a buzz. Facebook says its wristband device can vibrate and pulse depending on different interactions. While it’s no PS5 Dual Sense controller, Facebook says it's able to “approximate the sensation of pulling back the string of a bow” for a virtual bow and arrow.
“The intelligent click gives you the ability to take these highly contextual actions in a very low-friction manner because the interface surfaces something that’s relevant based on your personal history and choices, and it allows you to do that with minimal input gestures.”
Tanya Jonker, Research Science Manager at Facebook Reality Labs
I know what you’re saying: Facebook care about privacy? Nonsense! While we don’t have any idea how AR and the Intelligent Click would be used or integrated with Facebook’s existing platforms, wrist-based gesture controls are more private than, say, using voice controls that everyone can hear.
As exciting as Facebook’s wristband and Intelligent Click sound, they’re both still prototypes. A working prototype — but one that’s still fluid and being worked on. It’s not a finished product and there’s no telling if it is the right form of input for AR. Facebook’s simply sharing its creation in an effort to create transparency and get feedback from the community for further development.
😴 Google’s Nest Hub 2 uses radar to track and improve your sleep
🔊 Sonos finally made the Bluetooth speaker everyone’s been asking for
🪁 DJI's new FPV racer drone can be flown with this sick motion controller
📸 These photographers still use the Game Boy Camera with incredible results