How 5G could make augmented reality and mixed reality more real

You’d think that a technology that allows us to either escape from or enhance our view of reality would enjoy an enormous popularity surge in these crazy days.

But the truth is, although there have certainly been improvements in the sale of devices, applications, and services that offer virtual reality (VR), augmented reality (AR), or the combination called mixed reality (MR), they remain a niche market.

Part of the problem is that the technology powering these head-mounted display devices is still expensive. The products also need further refinements to achieve the kind of wow factor that many people expect. Finally, many of them are relatively large and get uncomfortable to wear for anything more than short bursts of time.

Thankfully, a number of important initiatives and new technologies are starting to come together to enable the kind of lightweight yet still impressively capable devices that most industry observers believe are necessary to make AR/VR/MR products go mainstream. Not surprisingly, one of them is 5G.

How displays are evolving

The fundamental challenge with the AR/VR/MR headset concept is that most people are only willing to wear something on their face that’s about the size and weight of sunglasses or regular prescription glasses. Unfortunately, fitting the computing, graphics, and display technology necessary to create a high-quality virtual or augmented reality experience into that size isn’t physically possible right now.

As a result, we’re starting to see a number of companies experiment with designs that separate the computing elements from the head-mounted glass display. Typically, these designs use a smartphone and a wired connection to the glasses to provide the complete experience.

Recently, Apple was rumored to be working on a device along these lines, potentially with a wireless connection between an iPhone and glasses-like display. Most Apple watchers think “Apple Glass” is more fiction than fact, though, and would still be several years away, even if it were real.

Dubbed XR Viewers, these head-mounted displays from companies like NReal, Panasonic, Pico, and Oppo connect via a USB-C cable to certain Qualcomm Snapdragon 855 or 865-powered 5G smartphones from vendors such as OnePlus, Oppo, Asus, and others.

Importantly, Qualcomm also discussed the details of its XR Optimized Program, which ensures that all the different XR viewers and compatible smartphones will follow the same critical technical specs and software standards to ensure that they can seamlessly work together.

A critical part of these devices’ operation is a 5G connection, which is where carriers, like Verizon here in the U.S., come into play. In order to get truly life-like image quality, as well as location-specific information, these XR viewers and companion smartphones also rely on a cloud computing connection.

By using the fast, low-latency (that is, extremely short lag time) 5G connection, applications running on the viewers can get even higher resolution images from the cloud. This can make the experience of using these head-mounted displays even more compelling and has the added benefit of integrating location-specific details.

Just last week, Microsoft announced that they will be adding 5G capabilities to their more business-focused HoloLens 2 MR headset by this fall via a plug-in module. Unlike the XR Viewers, the HoloLens 2 integrates all the computing elements inside the headset, but the result is a heavier, more expensive device.

Similar to the XR Viewers, the HoloLens 2 will leverage the 5G connection to provide access to cloud computing resources (in this case, from Microsoft Azure). Microsoft uses a concept called Spatial Anchors to do things like let multiple people wearing HoloLens’ view and interact with the same virtual objects. These and other interactive opportunities are why the 5G connections are so valuable.

Of course, not all AR/VR/MR headsets will use a 5G connection. Oculus’ popular Oculus Quest, for example, which is also an integrated, standalone device, only uses WiFi and is designed primarily for indoor usage. Similarly, HP’s new Reverb G2, which offers the highest resolution of any VR headset, plugs into PCs and only uses Wi-Fi connections.

As we look to the future, however, there is the expectation of being able to use augmented or mixed reality wherever we go. Besides some of the more widely touted examples of being able to learn more about objects or places in the real world by simply looking at them (or maybe getting reminded of someone’s name through a tiny speaker or via connected earbuds by glancing at them), AR and MR promise to give us new ways to learn how things work, play more interactive games, and much more.

In those and other types of scenarios all around us, the benefits of an always-connected 5G link start to become very clear.

By: Bob O’Donnell Special to USA TODAY