Technical investigation into PS4 and XB1 audio solutions *spawn

[Shifty Geezer]

Binaural audio should be great for the positioning of effects like explosions. I'm not sure why there are 5.1 headphones, to be honest.

Because 5.1 is an established surround format whereas there are no binaural standards used in games, so without 5.1 you are limited to plain stereo. If a console came with binaural processing as standard, it could be supported, and would provide perfect audio positioning and resolution regardless of where the player/viewer is in relation to the screen.

 

[Love_In_Rio]

Will X1 exclusive games make binaural sound processing?. I have no heard anything about that. Wouldn´t the development cost be much higher than normal sound processing?. I must say i like the idea of having real 3D sound using headphones. I supposse the headphones for listening binaural sound mustn´t be anything special, right?.

Any good sample to test binaural on my phone/tablet?.

 

[novcze]

Because 5.1 is an established surround format whereas there are no binaural standards used in games, so without 5.1 you are limited to plain stereo. If a console came with binaural processing as standard, it could be supported, and would provide perfect audio positioning and resolution regardless of where the player/viewer is in relation to the screen.

Avarage gamer is always in front of the screen, no need to worry about his position in relation to the screen. I'm all for binaural audio in games, good headphones are cheaper than good loudspeakes (not to mention problems with room acoustics and neighbours) and headphone output is already on every TV set.

Best example of 3D recording I found on youtube https://www.youtube.com/watch?v=QbOmya3X4kw , imagine this kind of soud localization with proper in-game action on the screen, that would be real step forward for game audio.

 

[ERP]

On sound disign, which I know a litle (I am more into music production).

Mixing and processing fx on a few dozen voices (tracks if you prefer) can kill even a modern i7 cpu, specially if you are using good fx for essential stuff like reverbs.

In fact there is a feature in almost all modern DAW (digital audio workstation) that is freeze, that is pre render the track so it is possible to use more fx. More, there is even dsp cards to be bought so it doesnt it so much the cpu.


I really hope for a good audio dsp with good fx, specially reverb, audio in games still is quite primitive in many aspects (eg reverb).

As a curiosity, from pro audio developers I have talked to, they all say that gpgpu is good for audio (beside convulsion reverbs and still quite a hit). But dsp always have been a good option for audio, it is amazing how a little chip can do wonders.

Yep I've worked with audio engineers that would if they had their way expend the entire console CPU budget on 2 or 3 "high quality" reverbs.

I think the real value of a dedicated audio processing block is freeing up CPU/GPU cycles for everything else.

If devs were budgetting 1 CPU thread last gen, I'd expect them to be close to the same budget this time.

 

[Shifty Geezer]

Avarage gamer is always in front of the screen.

I tend to sit somewhat off to the side, and I know one friend who does likewise. Thus a bullet aiming straight out of the screen would sound to my left if the audio is based on speaker positions. Something like Kinect could track your position and adjust the audio accordingly, which would be cool.

Best example of 3D recording I found on youtube https://www.youtube.com/watch?v=QbOmya3X4kw , imagine this kind of soud localization with proper in-game action on the screen, that would be real step forward for game audio.

The up and down works very well in that for me, but there's zero forwards space on any such recordings I've heard. That's something I'd really like to experience, which needs a calibrated solution.

 

[Scott_Arm]

Because 5.1 is an established surround format whereas there are no binaural standards used in games, so without 5.1 you are limited to plain stereo. If a console came with binaural processing as standard, it could be supported, and would provide perfect audio positioning and resolution regardless of where the player/viewer is in relation to the screen.

That is the obvious answer I didn't think about, for whatever reason. Any 5.1 surround game can work on 5.1 headphones without any additional work.

 

[Deleted member 13524]

That's untrue. A speaker can only define a sound at a distance relative to that speaker from the listener. The only way to simulate larger spaces than those within the audio field of the speakers is to process the audio, and you can do that just as well if no better with binaural sound. The problem with binaural audio is it needs to be calibrated for each listener to be perfect, otheriwse it sounds more like wide stereo with a smaller enclosed volume.

I'll be convinced when a stereo headset makes my chest tremble from the sound of a blast.

You can do whatever you want to a couple of tiny speakers next to your ears, but it doesn't change the fact that you don't perceive sound waves with the eardrums alone, or that the outer ears have different shapes among all humans which in turn pre-equalize the sound that reaches the eardrums in a different way.

That was my point, and that's why there are 5.1 headsets.

 

[Davros]

Anyone heard the final cut by pink floyd ?
Its recorded using holophonics
on the album a car drives past and the first time I heard it on headphones i looked out of the window for the car

 

[Shifty Geezer]

I'll be convinced when a stereo headset makes my chest tremble from the sound of a blast.

That's true, but I don't know many folk who game that extreme. The households where the gamer can afford to run the audio loud with the woofer up are going to be far fewer in number than the households where the gamer has to keep somewhat low-key so as not to intrude on others, in my expectation. There's also nothing stopping both woofer and binaural sound for the best of both worlds. Given a choice though, if it has to be one or the other, I think 3D audio is better.

You can do whatever you want to a couple of tiny speakers next to your ears, but it doesn't change the fact that you don't perceive sound waves with the eardrums alone, or that the outer ears have different shapes among all humans which in turn pre-equalize the sound that reaches the eardrums in a different way.

That's what calibration and advanced modelling techniques are for. 5.1 audio processing has the same issues as the speakers try to present a sound-source at a location different to the speaker locations. In a console, every user account could have its own audio calibration file that the game uses for that user.

That was my point, and that's why there are 5.1 headsets.

5.1 headsets are the contrived solution to the problem of positional audio. They are far from perfect and are still dependent on very clever acoustic processing to generate realistic audio fields. And they are no better for the "chest tremble" scenario either. In the absence of binaural processing on consoles, they'll do to replace 5.1 audio setups, but it's disappointing to me that the industry is happy to settle with 'it'll do' when the alternative is that much better and easily accessible to all without them having to buy a £50+ pair of 5.1 cans and a £xxx 5.1 audio amplifier.

 

[Deleted member 13524]

That's what calibration and advanced modelling techniques are for. 5.1 audio processing has the same issues as the speakers try to present a sound-source at a location different to the speaker locations. In a console, every user account could have its own audio calibration file that the game uses for that user.

How would they perform such calibration using widely available consumer electronics?

5.1 headsets are the contrived solution to the problem of positional audio. They are far from perfect and are still dependent on very clever acoustic processing to generate realistic audio fields. And they are no better for the "chest tremble" scenario either.

It's not an ideal solution, but at least for me, I still find it better than using a pair of great earphones for games.
FWIW, some headsets (like mine) have a motor for rumble which adds some of the lost perception on the low-end.

In the absence of binaural processing on consoles, they'll do to replace 5.1 audio setups, but it's disappointing to me that the industry is happy to settle with 'it'll do' when the alternative is that much better and easily accessible to all without them having to buy a £50+ pair of 5.1 cans and a £xxx 5.1 audio amplifier.

Apart from some 5.1 headsets that are finally coming with larger drivers for the main channels, I think the audio in games has pretty much stagnated for over 10 years.. and I see no indication why this would change.
The transition from CRT to LCD TVs brought along the embedded crappy speakers, and thinner LEDs brought even crappier ones.
The low-end home cinema market is infested with sets that use panel-shaped satellites with 90% of all the sound coming from the small/neat subwoofer, because many prefer speakers to look good than to sound good.

The typical consumer just doesn't care about sound reproduction. As long as it's there, it's good enough.

 

[Shifty Geezer]

How would they perform such calibration using widely available consumer electronics?

User selection. You put the headphones on and are presented a series of 'select which sound best matches the picture' or whatever user-selected calibrations as various variations are tested, and maybe access some fine-tuning options. In the case of XB1, MS could provide head scanning as well and create an acoustic model perfectly matched.

The typical consumer just doesn't care about sound reproduction. As long as it's there, it's good enough.

That's because they haven't the opportunity to really experience better though. If better game audio is reliant on $500 of audio gear for an amp and speakers, that's not something most folks are going to commit to. But if it was present in every game just in a pair of earphones, which everyone has, I think they'll go a bundle. That's why I see binaural as so attractive. It's 3D for audio, but without needing any expensive gear to use unlike a 3D TV. It's the best solution for moving audio forwards, and you just know that if one of the consoles went that route, the internet would be abuzz. You could even demonstrate it perfectly online unlike headset VR or 3D. Record a YouTube clip of COD gameplay with the binaural audio track and people watching online will experience the benefit. Or some horror game like Alan Wake.

Given that audio is our primary spatial sense, the immersion will be even better in many cases than 3D or Illumiroom, and all for the price of a decent DSP and some good software. Strikes me as the best way to differentiate a console in a new field. The fact audio remains such low priority is just a lack of vision of the designers, who fail to appreciate what an impact it'll have.

 

[Arwin]

Will all due respect, the fact that studios choose to focus on high quality 7.1 surround sound instead doesn't necessarily mean it's a low priority or lack of vision, just because it doesn't match your ideal scenario.

 

[Nesh]

Not that I know of.

This was the demo of the effect for headphones that really illustrated it for me back in who-knows-when. the first recording was in 1996, according to the website.

Things like the scissors and whisper are incredibly authentic. You just can't do that with 5.1 (well, maybe you can with careful positioning and fancy processing). Binaural just involves sticking on a pair of headphones for the user. There are also loads of binaural recording on freesound.org. Often they just use a four-mic device and some software processing, so the results aren't always as good, but there are some very convincing ones. If the audio was computed and could be calibrated for each listener, it'd be perfect. For me, all these recordings lack a front dimension and I don't hear anything ahead of me.

Man that was fun. I closed my eyes so I could sense the effect as perfectly as possible.
I started laughing when I could hear the plastic bag wrapping around my head

edit: Btw is this the effect that the XB1 audio capabilities are supposed to provide?

 

[Arwin]

Listening to this again, and also to this new one:

http://www.youtube.com/watch?v=wT1XuB95qMk&list=PL818B5CCDAA3BC64F

I'm starting to think this works well with sounds behind you, because your ears can't really properly hear what's going on behind you and so your brain is wired to interpret that, whereas stuff in front of you can be located with less 'processing'?

Something like the thunderstorm defintely sounds more impressive still on a proper 5.1 surround, imho.

EDIT: by the way, I use a 5.0 surround sound, which I prefer - the house doesn't tremble, but the sound-balance is better - all 5 speakers I have are pretty good quality and have a pretty good range, so that the whole audio-spectrum is well balanced, and the surround stays more in-tact (rather than having certain low sounds come more from one central location)

 

[Shifty Geezer]

Will all due respect, the fact that studios choose to focus on high quality 7.1 surround sound instead doesn't necessarily mean it's a low priority or lack of vision, just because it doesn't match your ideal scenario.

How much have 3D graphics improved over the past decade? How much as audio processing and rendering? I rest my case that audio is low priority.

 

[BRiT]

How much have 3D graphics improved over the past decade? How much as audio processing and rendering? I rest my case that audio is low priority.

For that I blame Creative Labs, specifically their actions against Aureal then their lack of follow up with their technology.

 

[Shifty Geezer]

edit: Btw is this the effect that the XB1 audio capabilities are supposed to provide?

I don't think so.

Listening to this again, and also to this new one:

http://www.youtube.com/watch?v=wT1XuB95qMk&list=PL818B5CCDAA3BC64F

I'm starting to think this works well with sounds behind you, because your ears can't really properly hear what's going on behind you and so your brain is wired to interpret that, whereas stuff in front of you can be located with less 'processing'?

I just had an idea to try the headphones swapped over, and it made a huge difference. With that video at 30 seconds, matches shaking, with the headphones on correctly I hear the matches pass behind me twice, right to left to right. With the stereo reversed, it passes in front of me, and the space is far, far better. I'm gonna listen to a load more holophonics with the headphones reversed!!

 

[Shifty Geezer]

For that I blame Creative Labs, specifically their actions against Aureal then their lack of follow up with their technology.

Yep. We were making progress, and then it stopped and we've pretty much flatlined. Okay, 5.1 isn't quite flatlined, but there's nothing like the investment in audio as there has been in other areas of games.

 

[temesgen]

How much have 3D graphics improved over the past decade? How much as audio processing and rendering? I rest my case that audio is low priority.

completely agree, excellent point.

 

[Arwin]

How much have 3D graphics improved over the past decade? How much as audio processing and rendering? I rest my case that audio is low priority.

Most movies I watch and games I play (though notably not Gran Turismo) have a lot more realistic and high fidelity sound than graphics. I rest my case. It may seem that more effort is spent on graphics, but that's partly because it's much harder.

Of course there's still room for improvement. Ever since Surround Sound popped up, I've always thought that a proper sound setup should at least have a sound source in every corner of the room, meaning all eight. But 7.1 Master Audio can sound really rather incredibly good already. 3D graphics are still mostly limited to watching through a window, which is very blunt by comparison, but has recently at least seen a massive improvement, making it more common and affordable. Still, compare the number of movies that have a proper 3D version, versus how many movies have decent surround support. I was watching some LaserDisc movies in the nineties that practically blew my mind, sound wise (stuff like Terminator 2, or Outbreak, then Jurassic Park, which I admired almost more for the sound that the great looking dinos).

Oculus Rift is one of the better steps forward in that regard, but comes at the expense of resolution again. Uncharted 2, one of the better looking games of this gen, has high quality surround sound that far outstrips its graphics, and even uses path tracing for occlusion. Afaik Battlefield also spends a lot of time on good sound.