The Future of VR Collaboration
How Immersive Technology Will Evolve Into an Essential Tool as Explained by Three Experts
Coming Improvements in Immersive Technology Will Make Collaboration Much Richer
Anthony Rowe, Professor of Electrical and Computer Engineering, Carnegie Mellon University
When I look at virtual reality, I think, wow, that’s the future of collaboration. But right now we’re far off from the sort of technology that we’ll want to use on a daily basis. The main place you see people using VR every day right now is in research labs like ours.
There are a lot of problems to solve. One is the “uncanny valley” problem with avatars, which is when an avatar looks more realistic than a cartoon, but isn’t completely realistic. That just looks creepy. Because of that, it doesn’t work well right now to use a live video feed of the user as the avatar. The feeds just aren’t good enough yet, and to avoid the creepy effect you have to use filters to make it cartoony. There’s a project here [at Carnegie Mellon University] called the Panoptic Studio, which can capture people in motion from different angles and create photorealistic avatars (Figure 1). I’m not sure you could say they’ve gotten all the way past the uncanny valley, but the avatars don’t look creepy.
People complain about “Zoom fatigue,” which comes from interacting with people on a flat screen. But VR brings on a different type of fatigue, and right now it’s worse than Zoom fatigue. It happens because of “motion-to-photon latency,” which is the delay between when you physically move and when that movement is visible in VR, and because of “the screen door effect,” which is due to the space between image pixels. Some people get motion sickness from the delay. It can all be fixed with higher resolution, variable-focus images, faster frame rates, and more powerful processing, but all that is expensive right now.
Another big improvement will be the creation of hi-fi VR environments, where it looks exactly like you’re walking through a real world. One way to do that is to embed high-quality sensors in the real world and link them to the images, so that you become a virtual presence in a reality that’s partly physical and partly digital.
A lot of these improvements will come the same way GPS did: From the defense industry. No one is more interested in new and better ways to collaborate and gain situational awareness than the military is. They have the funding to develop it, and then it will come to the rest of the world.
Figure 1: Carnegie Mellon University’s “Panoptic Studio” for generating photo-realistic VR avatars. (Image source: Spatial)
VR Will Eventually Be Compelling for Most People—Maybe Too Compelling
Jacob Loewenstein, vice president and head of business for immersive-reality software company Spatial Systems
VR collaboration was a highly niche market that ran only on hardware that few people could afford. Then the Oculus Quest (VR headset) made it affordable and accessible. Now the numbers are growing like crazy.
But we ended up realizing that most people aren’t ready yet to work in VR. Most people still don’t have any VR hardware yet. So instead of making the meetings all or nothing in terms of being in VR, which is how most VR applications are, we let people participate in a web browser if that’s all they have. Now more than half of the people who attend VR meetings with our software do it in a browser.
Eventually most people will have access to some sort of device that will let them participate in VR (Figure 2). But part of the problem is that today most people wouldn’t feel comfortable being in VR for a long period of time. For most of the people who use our software for VR meetings, they can stay creative and productive for maybe 30 minutes or so in VR. Even most hard-core VR gamers don’t stay in it for much more than an hour.
That will change as hardware gets better and people get more comfortable with it. And it will definitely change with a younger generation that will grow up with VR, just like the kids who grew up with PCs became the big users, and then the generation that grew up with smartphones. We’re already seeing a lot of use in education, with several universities hosting seminars in VR, including MIT, the University of Arizona, and the School of Social Work at Columbia University.
Longer term, the problem won’t be getting people to use VR; it will be people who become addicted to it. We already have superusers who stay in VR up to eight hours a day. That’s the exception right now, but I think we’ll see more and more of it. It will be incumbent both on users to develop good practices in using VR, and on VR companies to encourage reasonable use rather than addictive use. We already require users to be at least 18 years old, but we see VR apps from other companies that skew toward a younger audience, and that could become a problem.
When VR use starts to get heavier, we may want to start putting in design features that encourage users to get out of VR every so often and go outside for some physical activity. Although, using VR doesn’t mean you’re just sitting on a couch the whole time. We already see ways people can stay physically active even while they’re in VR. There are special shoes and platforms that let people walk through a VR space by physically using their legs in a walking motion, though they’re staying in the same place in real life. We’ll see a lot more of that.
The biggest request we get from the work world is to give people avatars that look like their real selves instead of cartoons. You can’t do that with streaming video right now, but it may be possible to do it with machine learning programs that generate a realistic avatar from photos. But there’s still a lot to figure out in terms of how avatars interact in VR.
Figure 2: People participating in a VR meeting through avatars, surrounded by shared content. (Image source: Spatial)
Augmented Reality Will Surpass Virtual Reality as a Collaboration Tool
Nikhil Balram, CEO of EyeWay Vision, an augmented-reality technology company, and former visiting professor of vision science at the University of California, Berkeley
The dream virtual meetings will be those that can combine organic humans with digital humans, all collaborating. The digital humans will be attending in virtual reality, and the organic humans will see live scans of them projected in 3D through AR glasses. The biggest challenge is letting the organic humans in the room see each other’s eyes, because AR glasses tend to obscure the eyes. Eye contact and following others’ gazes is a big part of communicating.
AR glasses good enough for collaboration aren’t available yet. They could be developed now, but the purchase and maintenance costs would be prohibitively high. The way to get around that is to use the same business model cable TV companies use. You don’t buy your cable TV equipment—you rent it with a monthly fee, and the cable TV company fixes any problems you have.
AR will be useful for anyone who wants to see some of their surroundings while they’re also looking at virtual images (Figure 3). In many cases, people will prefer full virtual reality and won’t need to see what’s around them. If you want to walk on the moon, you don’t want to see your real surroundings. But if you want to collaborate in virtual reality while you’re sitting in a coffee shop, you might want to keep an eye on what’s going on around you. You don’t want to emerge from virtual reality and discover your laptop is gone.
There will be real sweet spots for both VR and AR, where it will be very clear which one you prefer. They’re complimentary technologies, not competing ones.
Figure 3: In mixed reality, both people who are physically present and those who participate as avatars will be able to see each other and share content as if they were in the same room. (Image Source: Gorodenkoff/Shutterstock)