Metaverse: Augmented reality pioneer warns it could be far worse than social media

At its core, augmented reality (AR) and the metaverse are media technologies that aim to present content in the most natural form possible — by seamlessly integrating simulated sights, sounds, and even feelings into our perception of the real world around us. This means AR, more than any form of media to date, has the potential to alter our sense of reality, distorting how we interpret our direct daily experiences. In an augmented world, simply walking down the street will become a wild amalgamation of the physical and the virtual, merged so convincingly that the boundaries will disappear in our minds. Our surroundings will become filled with persons, places, objects, and activities that don’t actually exist, and yet they will seem deeply authentic to us.

Early augmented reality (AR)

Personally, I find this terrifying. That is because augmented reality will fundamentally change all aspects of society and not necessarily in a good way. I say this as someone who has been a champion of AR for a long time. In fact, my enthusiasm began 30 years ago, before the phrase “augmented reality” had even been coined. Back then, I was the principal investigator on a pioneering effort conducted at Air Force Research Laboratory (AFRL) with support from Stanford University and NASA. Known as the Virtual Fixtures project, for the very first time, it enabled users to reach out and interact with a mixed reality of both real and virtual objects.

This early system employed a million dollars’ worth of equipment, requiring users to climb into a large motor-driven exoskeleton and peer into a makeshift vision system that hung from the ceiling, all while they performed manual tasks in the real world, such as inserting pegs into holes of different sizes. At the same time, virtual objects were merged into their perception of the real workspace, the goal being to assist users as they perform the complex task. The research was a success, showing that we could boost human performance by over 100 percent when combining the real and the virtual into a single reality.

But even more exciting was the reaction of the human subjects after they tried that very first version of AR. Everyone climbed out of the system with big smiles and told me without prompting how remarkable the experience was — not because it boosted their performance but because it was magical to interact with virtual objects that felt like genuine additions to the physical world. I was convinced that this technology would eventually be everywhere, splashing techno-magic onto the world around us, impacting every domain from business and commerce to gaming and entertainment.

Now, 30 years later, I am more convinced than ever that augmented reality will become central to all aspects of life, touching everything from how we work and play to how we communicate with each other. In fact, I am convinced that it will happen this decade — and yes, it will be magical. But at the same time, I am very concerned about the negative consequences, and it is not because I worry about bad actors hacking the technology or otherwise hijacking our good intentions. No, I am concerned about the legitimate uses of AR by the powerful platform providers that will control the infrastructure.

A dystopian walk in the neighborhood

Let’s face it: We find ourselves in a society where countless layers of technology exist between each of us and our daily lives, moderating our access to news and information, mediating our relationships with friends and family, filtering our impressions of products and services, and even influencing our acceptance of basic facts. We now live mediated lives, all of us depending more and more on the corporations that provide and maintain the intervening layers. And when those layers are used to manipulate us, the industry does not view it as misuse but as “marketing.” And this is not just being used to peddle products but to disseminate untruths and promote social division. The fact is, we now live in dangerous times, and AR has the potential to amplify the dangers to levels we have never seen.

Imagine walking down the street in your hometown, casually glancing at people you pass on the sidewalk. It is much like today, except floating over the heads of every person you see are big glowing bubbles of information. Maybe the intention is innocent, allowing people to share their hobbies and interests with everyone around them. Now imagine that third parties can inject their own content, possibly as a paid filter layer that only certain people can see. And they use that layer to tag individuals with bold flashing words like “Alcoholic” or “Immigrant” or “Atheist” or “Racist” or even less charged words like “Democrat” or “Republican.” Those who are tagged may not even know that others can see them that way. The virtual overlays could easily be designed to amplify political division, ostracize certain groups, even drive hatred and mistrust. Will this really make the world a better place? Or will it take the polarized and confrontational culture that has emerged online and spray it across the real world?

Now imagine you work behind a retail counter. AR will change how you size up your customers. That is because personal data will float all around them, showing you their tastes and interests, their spending habits, the type of car they drive, the size of their house, even their gross annual income. It would have been unthinkable decades ago to imagine corporations having access to such information, but these days, we accept it as the price of being consumers in a digital world. With AR, personal information will follow us everywhere, exposing our behaviors and reducing our privacy. Will this make the world a better place? I don’t think so, and yet this is where we are headed. 

The metaverse could make reality disappear

Over the last decade, the abuse of media technologies has made us all vulnerable to distortions and misinformation, from fake news and deepfakes to botnets and troll farms. These dangers are insidious, but at least we can turn off our phones or step away from our screens and have authentic real-world experiences, face-to-face, that aren’t filtered through corporate databases or manipulated by intelligent algorithms. With the rise of AR, this last bastion of reliable reality could completely disappear. And when that happens, it will only exacerbate the social divisions that threaten us.   

After all, the shared experience we call “civilized society” is quickly eroding, largely because we each live in our own data bubble, everyone being fed custom news and information (and even lies) tailored to their own personal beliefs. This reinforces our biases and entrenches our opinions. But today, we can at least enter a public space and have some level of shared experience in a common reality. With AR, that too will be lost. When you walk down a street in an augmented world, you will see a city filled with content that reinforces your personal views, deceiving you into believing that everyone thinks the way you do. When I walk down that same street, I could see vastly different content, promoting inverse views that make me believe opposite things about the very same citizens of the very same town. 

Consider the tragedy of homelessness. There will be those who choose not to see this problem for political reasons, their AR headsets generating virtual blinders, hiding soup kitchens and homeless shelters behind virtual walls, much like construction sites are hidden in today’s world. There will be others who choose not to see fertility clinics or gun stores or whatever else the prevailing political forces encourage them to “reality block.” At the same time, consider the impact on the poorest members of society. If a family cannot afford AR hardware, they will live in a world where critical content is completely invisible to them. Talk about disenfranchisement.

You can’t ever leave the metaverse

And no, you won’t just take off your AR glasses or pop out your contacts to avoid these problems. Why not? Because faster than any of us can imagine, we will become thoroughly dependent on the virtual layers of information projected all around us. It will feel no more optional than internet access feels optional today. You won’t unplug your AR system because doing so will make important aspects of your surroundings inaccessible to you, putting you at a disadvantage socially, economically, and intellectually.  The fact is, the technologies we adopt in the name of convenience rarely remain optional — not when they are integrated into our lives as broadly as AR will be.

Don’t get me wrong. AR has the power to enrich our lives in wonderful ways. I am confident that AR will enable surgeons to perform faster and better. Construction workers, engineers, scientists — everybody, young and old, will benefit. I am also confident that AR will revolutionize entertainment and education, unleashing experiences that are not just engaging and informative but thrilling and inspiring.

But AR also will make us even more dependent on the insidious layers of technology that mediate our lives and the powerbrokers that control those layers. This will leave us increasingly susceptible to manipulations and distortions by those who can afford to pull the strings. If we are not careful now, AR could easily be used to fracture society, pushing us from our own information bubbles into our own custom realities, further entrenching our views and cementing our divisions, even when we are standing face-to-face with others in what feels like the public sphere. 

Being an optimist, I still believe AR can be a force for good, making the world a magical place and expanding what it means to be human. But to protect against the potential dangers, we need to proceed carefully and thoughtfully, anticipating the problems that could corrupt what should be an uplifting technology. If we have learned anything from the unexpected evils of social media, it is that good intentions are not enough to prevent systems from being deployed with serious structural problems.  And once those structural problems are in place, it is extremely difficult to undo the damage. This means the proponents of AR need to get things right the first time.