Turn Up the Volume: Sound AR and 3D Audio

Written BY

Emily Friedman

November 22, 2019

Why enterprises using augmented and virtual reality need to listen closely to noise in the workplace and the role of sound in immersive experiences

Noise. Or, in more technical terms, audio.

I was recently gifted a pair of the new Apple AirPods. While I’ve had noise cancelling headphones before, the AirPods Pro offer something new: Active noise cancelling and a transparency mode that uses built-in microphones to allow outside sounds in when I want or need them. Now, I’m not an audiophile and this isn’t an Apple ad, but I am very sensitive to sound when I’m working, particularly the sounds of other people in the office. I’m at my most productive while listening to music but as soon as a colleague takes a phone call, my concentration breaks. (Hell is other people, am I right?)

Source: Apple

I’ve been using the AirPods’ noise cancelling feature every day at work, which got me thinking about noise in the workplace and the role of sound in immersive experiences. This is a topic that came up at EWTS 2019, with multiple speakers touching on hearables as well as haptics and voice. As virtual reality headsets improve and major tech companies develop tools that make it easier to create 3D content, the absence of our other senses in XR becomes more and more obvious.

Big, bad noise

My research took me in a number of directions, but let’s start with Millennials in the office. The idea that Millennials have short attention spans is a stereotype; in fact, younger generations are getting more selective about what they pay attention to (1). Today, most people – not just Millennials – use headphones for their own ‘attention management.’ According to Bitkom Research, nearly 50% of people wear headphones to mute their surroundings while 20% do so to focus on work. Millennials, however, are especially concerned about rising decibels and more likely to wear headphones to drown everything out (2). While the government caps safe noise levels around 90 decibels, Cornell University reports that office workers are most comfortable between 48 and 52 dB. To put that in perspective, casual chatting is around 60 decibels.

Why does any of this matter? In the age of digital transformation, organizations are seeking to augment and upskill the workforce. There are now four generations in the workplace, but when it comes to recruiting and training the next generation of workers, the focus is naturally on Millennials and Gen Z. Traditional notions about how people best collaborate and what makes a productive work environment are proving false for younger age groups. Workstyles are changing: AR/VR training and collaboration are on the rise in industry, with numerous studies and use cases showing the benefits of immersive technologies over traditional teaching methods and work tools. Millennials want quiet spaces and employees are bringing their own noise-cancelling earbuds to the office because sound can, it turns out, significantly impact employee performance and satisfaction. And then there’s occupational hearing loss, one of the most common work-related illnesses resulting in over $240 million in workers’ compensation every year (3). So, where are the next-gen acoustic solutions for enterprise?

The Sound of Work

In many industries, as you can imagine, employees work in loud environments on a daily basis. OSHA set legal limits on noise exposure in the workplace in the early Eighties, yet the government acknowledges that these aren’t low enough to protect workers from hearing loss. It’s estimated that 22 million workers are exposed to hazardous levels of noise on the job every year. By industry, that’s 46% of manufacturing, 51% of construction, and 61% of mining, oil and gas workers (4). Though employers are required to take steps to protect workers’ hearing, including providing hearing protection and training, investing in low-noise equipment and adjusting work shifts, some employers don’t and 25% of workers just don’t like wearing earplugs or earmuffs (5). The thing about hearing loss is that it’s irreversible, and we now know a lot more about the health effects of noise than we did when exposure limits were first set to 90 dBA over an 8-hour workday. Experts agree that regular exposure to anything over 75 decibels is enough to cause long-term hearing damage. (For reference, most equipment operates above 100 dB and all of the following are above 75: Bulldozer, crane, hydraulic breaker, electric drill, jet takeoff on a tarmac, riveting machine, and newspaper press. Also, keep in mind that a small increase in decibels represents a significant rise in noise and potential damage.) As for health effects, noise has been linked to heart disease, high blood pressure, sleep disturbance, and even adverse birth outcomes (6).

The truth is we don’t fully understand the impact of noise on workers. Scientists are realizing that sounds much lower than 75 (and over shorter periods of time) contribute to long-term hearing loss and something called ‘hidden hearing loss’—a permanent reduction in neural response (loud noise kills ear nerve endings). I won’t go into the ecological devastation wreaked by human-created noises, but for work safety purposes, noise-induced hearing loss also reduces one’s ability to hear high frequency sounds like alarms and understand speech, impairing communication. If you’re wondering why a worker would neglect to wear hearing protection, current devices (foam and silicone earplugs, earmuffs, etc.) leave a lot to be desired. One of their major shortcomings is the need to remove them in order to communicate with others. That’s where active noise cancelling comes in.

The ear is where it’s at

Hearables are progressing rapidly thanks to the convergence of low-power components, smaller AI processors, advanced microphone arrays, and more. Touch and gesture controls are improving, and many hearables now have integrated voice assistants, can do real-time language translation, are controlled via smartphone, etc. Sensear, for one, makes intrinsically safe in-ear earplugs and headsets for industrial users. In addition to noise cancelling, the company’s SENS technology isolates and enhances speech, so the wearer remains aware of her surroundings while still being protected from harmful background noise. Every ear is unique and each person hears differently, so it makes sense that hearables are becoming more customizable, but there is plenty of room for innovation, increased connectivity and additional capabilities, bringing this familiar wearable form factor into offices and factories alike. Besides intelligent noise control, earbuds/plugs could be equipped with any number of sensors that track location, movement, biometrics and even emotion. (The ear is actually a good spot for monitoring pulse and electrical brain activity.) Imagine a hearable that cancels out certain sounds to reduce stress or an in-ear device allowing you to interact with artificial colleagues like AGVs and collaborative robots—the hearable category has great potential beyond noise cancelling, active or not.

Audio AR

The “last piece is audio,” as Raytheon’s Kendall Loomis said at EWTS 2019. At this year’s EWTS, Boeing, Con Edison and ExxonMobil all brought up hearing protection, but ‘audio wearables’ can also augment workers to help them work faster and smarter, with sound and/or auditory information changing based on the user’s context. At UPS, for example, audio wearables tell workers what to do, no training required. This is essentially audio AR, overlaying auditory information into the user’s environment, and it can be as simple as enabling a worker to hear a pre-recorded guide – one of your expert employees talking through a process – while on the job. AR isn’t purely a visual play; audio can be modified in real time based on tracked data, location or task and deliver safety, machine and other job-critical information from sensors or an ERP right into the worker’s ear.

Not surprisingly, Bose is taking a sound-first approach to AR with Bose AR-enabled products and apps that provide tailored audio content into users’ ears as opposed to a small screen. Bose Labs exhibited at EWTS 2019 to see how its sound technology might assist industrial workers and to meet enterprise AR hardware makers. See, immersive technologies are so effective for training because they’re experiential. Sound is a major part of the human experience, and the combination of visual and audio AR could be twice as powerful for the workforce.

3D Audio

Audio AR adds an extra layer of information, but what about sound in the virtual world? If VR is going to completely replace traditional enterprise training (and probably design) programs in the future, then it had better be as close to the real thing as possible. Since sound helps us orient ourselves in space (auditory spatial awareness), truly realistic VR must be able to simulate sound localization.*

*Occupational hearing loss negatively impacts spatial hearing, as well.

In case you’re wondering, surround sound is not the same as 3D audio because the listener and sound sources are, for the most part, in fixed positions. 3D audio, on the other hand, is full-sphere surround sound, moving along with the listener in the physical environment. In a 3D soundscape, the user would be able to understand where he is relative to the noises around him so he could, for instance, sense something happening behind him. For binaural recording, sound engineers typically use a dummy head, placing microphones in the dummy’s ears, but the listener is again in a fixed position. Another approach adds speakers at different heights. The complexity lies in the fact that your aural experience changes as you move through the world. Ambisonic mixing for virtual spaces requires enabling sound to adjust according to where the user and sound sources are in the VR world. The user(s)’ position relative to other sound sources and the sources themselves (objects) are in motion. Today’s VR audio tech uses specialized recording systems and algorithms to mimic lifelike sound. There are some ambisonic microphones on the market, as well as custom rigs of omni- and bi-directional mics, and a few audio companies have released encoding formats supporting 3D audio, but lifelike audio VR remains a few years away.

Beyond Sound

We’ve been using audio wearables for decades: There was Panasonic’s AM radio designed to be worn as a bracelet in the early Seventies and the Sony Walkman, which turned 40 in July. It’s now a habit to carry sound around with us, on our bodies, and yet sound is a secondary consideration in AR/VR. From augmenting one’s hearing against occupational hearing loss to enabling a technician in training to hear exactly how a healthy engine should sound and simulating all the sounds of assembly before a worker ever hits the plant floor, getting sound “right” in extended reality will only amplify the technology’s effectiveness in enterprise. And who knows, the addition of touch and smell might eliminate traditional job training forevermore.

  1. Prezi
  2. Oxford Economics
  3. CDC
  4. NIOSH
  5. Hearableworld
  6. The New Yorker

Main image source: Auditoryprotection.com

Enterprise Wearable Technology Summit 2020

The 7th Enterprise Wearable Technology Summit (EWTS) is October 20-22, 2020 in San Diego! Join hundreds of Fortune 1000s to try out the latest in wearable tech, including AR/VR/MR, body-worn devices, and even exoskeletons, and to learn how today’s biggest companies are profiting from and scaling the technology. More details, including program and the largest expo of industrial AR/VR/Wearable tech) to come on the EWTS 2020 website.

Further Reading