Living, Working, and Learning In Virtual Spaces
Immersive technology is already blurring the line between in-person and virtual collaboration
(Image Source: dotschock/Shutterstock.com)
Walking together for the first time through the top floors of what was soon to become Boston’s newest major residential tower, the building project’s top architects and engineers entered one of the spacious condominiums and made a horrifying discovery: Some of the unit’s large windows provided a clear, direct, unimpeded view—into neighboring apartments. That meant, of course, that the neighbors would just as easily be able to see in. It was an intrusion on privacy that would never fly with the fussy prospective buyers of these high-end dwellings. The windows on the top floors would have to be re-angled, which would entail changes to the very structure of the 32-floor building.
Having to make big changes to a building well along into construction is every builder’s nightmare. Such alterations can run millions of dollars and entail months of delays. But in this case, fixing the building was the work of a few hours, at relatively trivial cost. That’s because ground hadn’t even been broken on the project yet. The group walk-through of the top floors had taken place entirely in virtual reality, via a 3D computer-generated simulation of the planned structure, shared over headsets by key members of the design team, who were actually in their offices in different cities.
Virtual reality and other Immersive-reality technologies can make people feel as if they’re physically inside and interacting with computer-generated spaces and objects—a bit like stepping through your computer screen into a world behind it. These sometimes very real-seeming 3D simulations, usually experienced via special high-tech goggles and other gear, can provide highly compelling, even astonishing adventures in the sorts of artificial worlds that have long been the stuff of science fiction.
Increasingly, though, immersive reality is being harnessed for a more down-to-Earth purpose: Bringing people who are at a physical distance together into artificial rooms and other synthetic environments where they can collaboratively learn, share ideas, make discoveries, solve problems, and simply catch each other up, often in ways far more engaging and illuminating than email, phone conversations, or Zoom meetings. “You can communicate in a Zoom meeting,” says Jacob Loewenstein, vice president and head of business for immersive-reality software company Spatial Systems. “But in virtual reality you can connect in a way that you just can’t on a flat screen.”
It’s only in the past few years that immersive-reality hardware and software have become sophisticated enough to get beyond crude, cartoonish imitations of reality to achieve more life-like collaborative experiences. And as the technology becomes more affordable, it becomes increasingly available to almost any employee or even consumer. As the gear gets even better and less expensive, and more people and organizations become familiar with the technology, the results could be a huge change in how people work, learn, and relax together. Immersive reality is already impacting business, education, health, and many social spheres, and it may in the not-too-distant future even become a medium where many of us spend much of our day.
The Different Faces of Immersion
Immersive reality comes in two main forms: Augmented Reality (AR) and Virtual Reality (VR). AR users continue to see the real world, except with virtual objects and information superimposed on those real surroundings. It’s a technology that’s become familiar to hundreds of millions of people around the world through the Pokémon Go app, in which players peer at their surroundings through their camera-fed phone screens to see cartoon characters who appear to be lurking nearby.
AR has more practical uses, too. It can annotate and illustrate the world around us with helpful information, pointing out the locations of nearby restaurants and friends, the parts of a machine that need servicing, or the names and positions of fellow attendees or key products at a conference. Having to look at all that through a phone can be awkward, but in the coming years we’ll be able to experience AR through “smart glasses” that are already emerging in prototypes from Facebook and Lenovo, among other companies. “You can easily make any object you can imagine appear in front of you, and yet you can still see the room around you, so you can walk around the object, and take a sip from the can of Coke on your desk,” says Nikhil Balram, CEO of EyeWay Vision, which is developing an advanced AR system. “Eventually you’ll be able to have meetings where some of the people are there in the room with you, and others are digital versions of people who are across the world, and it will feel like you’re all together.”
VR, on the other hand, is a more encompassing experience that entirely replaces a user’s view of the surrounding real world with a simulated one, usually projected onto the user’s eyes via tiny display screens mounted in headset-like goggles. The headsets also carry sensors that track changes in the user’s head position, altering the view to match those movements and provide a realistic sense of looking around at a complete 360-degree environment. Adding to the sense of fully inhabiting a shared, simulated world are hand-held sensors that enable users to reach out in the scene to “grab” virtual objects, press buttons, or gesture to fellow visitors in the virtual environment; while micro-speakers built into the headset provide rich, lifelike, directional sound.
The pandemic drove home the potential value of virtual meetings, as tens of millions of people cut off from their offices turned to computer screens to collaborate. But it also introduced millions to the drawbacks of trying to keep people engaged during an awkward, panel-of-talking-heads get-together, where participants may struggle to stay focused on a small screen while trying to ignore what’s going on around them. “The pandemic was a catalyst for virtual technology, as well as for Zoom fatigue,” says Anthony Rowe, an electrical and computer engineering professor at Carnegie Mellon University who is at the forefront of VR research. “Now technology needs to step in and get us past virtual meetings that are just a flat grid of squares with faces. Interacting in two dimensions isn’t natural for humans.”
Collaborating in VR, in contrast, can provide a convincing sense of fully interacting in a shared space with other people. To be sure, the VR meeting environment can have a slightly cartoonish feeling to it, with people represented by “avatars” that may not seem realistic, but because the avatars are in 3D, can be made to resemble the people they represent, and can mimic their gestures and in some cases facial expressions, most participants get past those limitations quickly.
Organizations Go 3D
VR interactions can even feel richer than in-person meetings, thanks to the ability to share 3D views of settings, objects, and even abstract data that couldn’t be summoned up in a real-life room. “You can sit across from each other in a VR room and bring models, videos and white boards into it, sharing everything in a way that is super-collaborative compared to regular screen-sharing,” says Spatial’s Loewenstein. “Whether it’s a business meeting, a social hour, or a conference, it feels much more engaging.”
That’s why the business market for VR hardware and software, tiny compared to the video-gaming VR market just a few years ago, is on track to account for half the total VR market by 2024, according to industry research firm IDC. A survey by market research firm Forrester Consulting found that half of all companies plan to get into immersive reality within two years.
The architecture, engineering, and construction (AEC) industry has taken a strong lead in embracing VR collaboration. It’s not quite as big a jump for professionals in that world because the industry has long relied on 3D virtual models of planned construction projects. But now VR meetings enable a team to jointly inhabit the model and see it in true 3D, as well as to walk through it together, point out potential problems, and experiment with possible changes, all while speaking to one another—even if the team members are scattered around the world in their respective offices or homes. “VR dramatically reduces geography,” says EyeWay’s Balram.
That’s why major design and construction companies around the world, from Ennead in Shanghai to Mecanoo in Delft, the Netherlands, have been turning to immersive reality for technical and other collaboration. For example, Valiant TMS, an intelligent automation systems company with facilities around the world, conducts review sessions of system designs that bring together engineers in Austria, India, Canada, and Mexico, using VR software from Theorem Solutions in Tamworth in the U.K. “That proved especially useful during the pandemic, when travel was banned between many countries,” notes Katharine Edmonds, a Theorem executive.
VR is also how members of the team working on the Boston residential tower project, composed of design and engineering professionals from Odeh Engineers in Rhode Island and Stantec in Boston, found the privacy problem with the windows. One member of the team also discovered when “walking” into an electrical services room in the VR model that he hit his head—well, his avatar’s head—on a low beam, providing an invaluable heads-up that the room would violate building code.
Stories of AEC teams realizing enormous savings with VR abound. One team saved $32,000 in a Florida high-school construction project when VR meetings allowed quick resolution of a few dozen potential problems that would have otherwise required delays to research the problems, share the details, and get everyone on board with the solutions. The British public-utility firm Anglian Water likewise saved thousands collaborating in VR on an infrastructure project. And the supervisors of a new, $2 billion subway-line project in Norway were able to bring professionals together from four different companies across two countries into coordination and visual-inspection VR meetings that spotted multiple problems early on, preventing big cost-overruns and delays.
So productive are the VR meetings at the global architectural firm Gensler that now all 28 offices join in on a weekly get-together to review projects. The firm uses VR in its sales meetings, too, and enlists its VR models to enable designers and construction engineers to see how different lighting and materials can impact the look of a building, as well as to get a better sense of how the building and its elements appear at full scale—something that’s hard to do with a conventional virtual model or a miniature physical model. “There are lots of people now who use these tools as a core part of how they work, and they wouldn’t let you take it away from them,” says Gabe Paez, founder and CEO of architectural VR tool vendor The Wild. “Five years from now it will seem ridiculous to newcomers that the industry used to work on a 2D screen.”
Beyond Architecture
Product designers and engineers in industries outside of AEC aren’t far behind in embracing VR collaboration. Toy company Mattel regularly brings distant designers together into meeting spaces in virtual reality to work out the details of new toy designs with 3D models. And when the design is ready for manufacturing, the team is joined in virtual rooms by manufacturing engineers at the factories that will produce it, often located in China, so the engineers can suggest alterations that will reduce manufacturing costs and boost production rate and quality.
Aerospace company Lockheed Martin runs one of the largest virtual-reality design labs in the world in its Denver facility, where engineers and managers work out the designs of everything from satellites to spacecraft. They can even run simulated performance tests on the models to optimize the crafts’ aerodynamics and heat and stress resistance, all before physically building so much as a tiny model.
The advantages of VR collaboration aren’t limited to designers and engineers. Having discovered the power of virtual meetings during the pandemic, many organizations are now looking for ways to wring more applications and benefits out of virtual interactions—and increasingly VR fills the bill. Lloyds Banking Group and Ikea are among the global companies that are bringing job candidates and new hires into VR for interviews and on-boarding. Financial services company Societe Generale Group brings financial analysts and clients together in virtual spaces. One company, the San Francisco-based biodata-analysis firm Larvol, already encourages its entire work force of hundreds of employees across 14 countries to do any or all of their collaborative work in VR, maintaining only minimal office facilities.
Creating new types of office space in VR will be liberating for many organizations, insists Loewenstein. “The office spaces we go to every day contribute to our sense of identity, branding, and connection,” he says. “But it’s hard to change physical spaces into something really stimulating, like an art gallery, or a camping site, or a meadow. It’s easy to do in VR, and sharing those spaces reinforces our connections to the people who work there with us.”
Conference-holders are also turning to VR to remove physical distance as a barrier to attending while enabling virtual attendees to experience the same levels of engagement as in-person attendees. Fittingly, the VR-focused branch of the IEEE, the world’s largest organization of technical professionals, held its 2020 conference online, providing full VR access to all keynotes, sessions, and even social get-togethers.
Training, too, is already a big application for businesses. State Farm Insurance enlists VR to train assessors on how to do an insurance inspection of a flooded home, allowing trainees to open virtual cabinet doors and look under furniture in order to spot potential damage. Walmart lets its employees virtually face a surging crowd of shoppers during a big sale, so they won’t be overwhelmed when it happens in real life. And management consultancy PWC does high-level decision-making training for managers around the U.S., leaving those who graduate the VR program testing 35 percent higher on the learned material than those who took the course in an ordinary online form. The United Nations, UPS, and Tyson Foods are all experimenting with training applications of VR. “You can learn on a flat screen, but it’s boring,” says Balram. “Learning is as much about engaging interactions with other people as it is about information, and immersive technology is much better for that.”
Enabling the Transformative
The next big leap for the field will be more realistic avatars that enable those meeting up in VR to look more like their real selves, complete with their changing expressions. Rowe’s Carnegie Mellon lab is already working with headsets that include cameras that point back at the wearer’s face, so an image of the face can be projected onto the avatar.
Future headsets, Rowe adds, will include sensors that pick up facial muscle movement to further refine avatar expressions, and eye trackers to detect where the user is looking. “It feels much more real when you can tell the direction of someone’s gaze, especially when you’re looking at each other,” he says. Other big improvements in the realism of avatars will come from higher-speed online connections and more powerful processors to boost the resolution of the avatars, smooth out their motions, and reduce the time lag between a user’s movement and when that movement is reflected on the avatar’s actions.
As immersive technology improves, and more people embrace it, the ways in which it will transform our jobs and lives are nearly impossible to predict, says Balram. “Mostly what we’ve been doing is force-fitting the old ways of doing things into these new environments,” he explains. “Now we need to start completely rethinking how we do things to truly take advantage of these tools.”