Facebook can now project your eyes onto a VR headset
A creepy new experiment from the people at Facebook Reality Labs has us thinking harder about CEO Mark Zuckerberg's new Metaverse idea. Reality Labs wants to devise Virtual Reality (VR) headsets that are less isolating for the wearer with methods including a way to project the obscured part of the wearer's face to the outer front side of the headset. Here are more details.
'Reverse passthrough VR' could help wearers transition to real-world conveniently
Earlier this week, Reality Labs released a paper about "reverse passthrough VR". This explained a feature using which cameras mounted on a VR headset helped the wearer transition from a virtual environment to the real one seamlessly, without taking the headset off. The Verge reported that Oculus Quest products already show users a passthrough feed when they step out of a VR space's limits.
Method helps people make eye contact with VR headset wearers
However, this system is far from being optimal since real-world people can't make eye contact with the VR headset wearer, even though the latter can see them clearly. Since 2019, Reality Labs scientist Nathan Matsuda has tried outfitting an Oculus Rift S with eye-tracking cameras on the inside and a screen on the outside so real-world people could make eye contact.
Reality Labs chief first thought the idea was 'goofy'
According to the blog post, the idea (thankfully?) wasn't well-received by Reality Labs chief scientist Michael Abrash given its impracticality. He explained that he doesn't tell researchers what to do since innovation is hampered by a lack of freedom to try new things. About Matsuda's project, he remarked, "My first reaction was that it was kind of a goofy idea, a novelty at best".
Facebook's video demonstrating the new VR headset concept
Now, wearer's face data is mapped to digital face model
Since then, Matsuda has developed the prototype and revealed it ahead of the SIGGRAPH conference next week. The current model adds lenses and stereo cameras to an ordinary VR headset that records the face and eyes inside the headset. The captured data is mapped onto a digital model of the face which is then shown to the outside world using externally-mounted light-field displays.
Prototype's light-field displays are the focus of the research paper
These displays (barely manage to) create an illusion that you are looking at the VR headset wearer's eyes, albeit through a thick pair of World War-1 era eyewear for pilots. Although the octagonal glasses with the plethora of wires and tubes running to them scream "prototype," this week's Reality Labs paper focuses on the potential of light-field displays and the prototype's less-isolating nature.
