With the Microsoft Xbox Series X and Series S releasing today and the Sony PlayStation 5 joining it in less than 48 hours, gamers are preparing to sink hundreds — if not thousands — of hours into the new hardware over the next handful of years. But while most of the headlines will focus on the 4K visuals, significantly increased framerates, and effectively nonexistent loading times, one of the major upgrades for audiophiles out there is the improvement in spatial audio — or 3D sound, as most marketing teams are phrasing it.
In layman’s terms, what that means is effectively the next step up in tying sound sources to specific locations and directions. Although you may not get the full experience without a home theater “surround sound” kind of setup or a good pair of headphones (Astro’s new A20 Gen 2 headset emerged as an early leader for those looking to invest in the experience at launch, and Sony’s Pulse 3D is obviously designed to capture the PS5’s audio capabilities), there’s a good chance you’ll notice a difference in how precise the sounds are even on a regular TV.
Similarly to the upgrade from mono to stereo sound back in the day, the new focus on spatial audio means that while more complex dedicated hardware is needed to get the most out of it, the opportunities for those utilizing its power will venture into uncharted territory. To make life easier on the game developers, the audio hardware within the Series X|S enables audio teams to make use of hundreds of spatial sound sources, real-time convolution reverb, and renderers like Windows Sonic, Dolby Atmos and DTS:X. In other words, it gives game developers all the tools they need to be able to provide immersive spatial sound like players have never heard before
“One of the challenges faced by audio engineers today is the computational cost of delivering high-quality spatial audio on the CPU,” said Jason Ronald, Xbox’s Director of Program Management. “We designed a new custom audio hardware block more powerful than the entire 8-core CPU from the Xbox One X, which can be used to offload all audio processing from the CPU. Developers can offload their audio processing to the dedicated audio hardware enabling them to create dynamic, living worlds that not only look, but also sound as you would expect, further deepening player immersion into the game worlds and universes of the industry’s best creators. They don’t have to worry about reducing audio quality when delivering complex game scenes, such as seeing a flock of birds flying across the screen or entering a new environment with a number of in-motion NPCs, which would typically be considered a computationally complex task.“
Although it might not be noticed by some folks among all of the other improvements within the new consoles, the ability to craft huge quantities of high-quality and multilayered sounds without using up too much of the console’s power means the wide-ranging sounds of a lively city, countryside, or other setting are suddenly available to developers of all types. Considering how massive “open-world” games have become the default for many major titles — and that bringing authentic “life” to those worlds often separates the successful from the sub-par — it’s a shift that could help to shape several big releases in the coming years.
“With a new generation of audio capabilities, gaming soundscapes have been honed, shaped, and refined in every way,” Ronald said. “For example, with Project Acoustics — which generates an ‘acoustic lightmap’ — we are able to dynamically adjust the soundscape as the players navigate the play space. This results in more immersive, lifelike auditory experiences for the player, such as reverberation off concrete or sound dampening due to water or natural environments. In addition, spatial sound delivers deeply immersive audio, which allows the player to more accurately pinpoint objects in a 3D play space.”
Of course, these spatial audio improvements could also help improve the experience for those who prefer to use their consoles primarily for streaming TV, music, and other media. While the hardware improvements may not be noticeable immediately with every streaming service, the consoles will no longer be a roadblock for those who want theater-quality audio. On the other hand, those who want to play new games that rely heavily on audio cues ranging from gunfire to footsteps — such as Call of Duty: Black Ops Cold War — will likely feel the difference right off the bat.
“The next-gen consoles have finally allowed us to truly design a 3D soundscape,” said Brian Tuey, Director of Audio for Call of Duty at Treyarch Studios. “We are able to accurately render above and below, in addition to every space in-between. We are able to run more voices, more filters, and render the audio in a very dynamic way that we haven’t been able to do before, regardless of whether you are running a massive surround sound setup or just a good pair of headphones. With the new types of 3D audio that the next-generation of consoles provide, it’s much easier to provide better tactical information to players — and it sounds great, too.”
Upgrades in the actual delivery of the audio isn’t even the only improvement for major productions like Call of Duty. Travis Fox, the Senior Producer for Godfall by Counterplay Games, described the process as “no longer restricted by hardware” while Aldo Sampaio, Audio Director for Assassin’s Creed Valhalla noted how certain aspects of the new consoles — like their speedy loading times — have provided a huge quality of life increase in the development and creative process itself.
“The new improvements actually make life easier on our side in terms of approaching audio in games,” Sampaio said. “Since loading times are a lot quicker on the next-gen consoles, this means we spend more time actually in the game debugging and polishing during production. It made mixing the game a lot more enjoyable, as we could instantly teleport to different places on the map with next to no downtime.”
As for the folks actually composing the music for these games, each passing generation of consoles removes some of the limitations of the one before it. While those might be minor inconveniences or semantics at this point — seeing as the PlayStation 4 and Xbox One were already fully capable of playing anything you’d hear on a laptop or certainly a phone — the artists who have stuck around since the early days of the consoles can appreciate the changes that have happened along the way.
“The evolution of technology in the last 15 years has been astounding,” said Austin Wintory, the Grammy-nominated composer whose most recent project, The Pathless, is a PS5 launch title. “When I first began writing game music, we couldn’t even reliably line up MP3’s without some risk of desynchronization, and now we can be cavalier about stacking seemingly-infinite layers of uncompressed audio. On top of that, a game developed for PS5 can be simultaneously shipped on iOS. I never in a million years thought that a mobile device could serve as a vehicle for such deeply interactive music, so it’s beyond thrilling and it continues to make us reevaluate what games — and game audio — can truly be. The future is entirely unwritten and probably wildly beyond our experiments today.”
Although Wintory didn’t get to fully take advantage of the PlayStation 5’s spatial audio due to The Pathless being designed for PS4 and iOS as well, it still gave him a taste of what’s to come over the next handful of years. He, like many others in the industry, believes that the next big change within video game music will be in integrating the music as an interactive and changing part of the experience rather than simply serving as a soundtrack as if it were a movie.
“I think the parameters for interactivity are going to get more and more subtle and granular,” Wintory said. “The ways in which the scores can wrap about players’ experiences will get more refined and nuanced. It will be less and less like ‘Now I’m exploring,’ ‘Now I’m in a boss battle’ and more of a continuous evolution as if it was a lived experience and not moving through segments of walled-off game systems. We’ve already seen little glimpses of that, and I think it’s only going to get more interesting.”
For Dale North, the composer of Wizard of Legend as well as a number of other games, the primary focus for artists would be in finding new ways to make sure the music still captures the magic he felt listening to old “chiptune” tracks growing up. While the hardware may have moved far beyond the classic arcade sounds of the ‘70s, ‘80s, and ‘90s, the exploratory and adventurous spirit of video game soundtracks shouldn’t disappear as well — particularly when new technology could open up so many unexplored possibilities.
“I like to say that the heart of the music should continue that heritage that was established early on,” North said.”You can do wild stuff these days. I’m working on a lot of stuff that has dynamic music that changes per scene or based on player input. I can’t say too much now, but I’m working on a next-gen game where the music literally changes depending on where you’re at in the world. It’s not just how the music sounds that changes, but the actual music itself changes for each specific point in the 3D space. The technology is expanding all the musical possibilities for us, and it’s really up to our imaginations to make this new technology work and make musical sense. There are no rules yet, so we’re all just stepping our feet in it and trying to figure out what makes the most sense.”
But it’s important for everyone to remember that no matter how impressive the new upgrades are and whatever people define as “the future of music” for video games, it’ll eventually either become a passing fad or settle in as the norm for the industry. Nothing stays new and exciting forever, and when you’re someone like Tim Wright (who’s been writing game soundtracks for longer than some gaming composers have been alive), it’s always just fascinating to see the cycle continue into artificial intelligence or whatever else may come next.
“There’s always something new, whether it’s in psychoacoustic technology, new hardware, or software that makes developing soundscapes and in-game sound effects that much easier — with real-time generation of car engines, explosions and gunfire — and even auto-generated music,” Wright said. “Audio technological advancements might not be as exciting and obvious as developments in computer graphics, but without sound, games can be pretty dull.”
Leave a comment