Technical investigation into PS4 and XB1 audio solutions *spawn

[Shifty Geezer]

Excellent post, my thinking exactly.

Yep. The only possible alternative to this IMO is if someone went with binaural audio and headphones. If one of the console reveals had all the audience wearing headsets and it placed then in the middle of an audio soundscape, with bees buzzing past their heads etc., everyone would be writing about it and raving about it. As it is, we just have more of the same with a lack of vision and audio is 'good enough'.

 

[DJ12]

Hasn't that already been done, raving about bee's buzzing around your head when surround sound was first revealed.

I think sound has had it's moment and while it's better for it to improve I doubt there will be another "wow" moment.

 

[Shifty Geezer]

Surround sound just isn't very accurate and is very constrained. No amount of 5.1 can place a bee right by your ear; the audio mechanics are all wrong. Binaural audio is in a world of its own and definitely provides a layer of immersion. If it were me, I'd provide everyone with headsets prior to the reveal. I'd start talking, and then play aggravating noises in everyone's headsets. They'd all hear a bee/mosquito and be panicked by it, and then you could explain the new audio of your console. Follow that with a delightful natural scenery that sounds just like being there, and then break it up with bullets whizzing past your ear and a battle breaking out, I think the press would be hooked.

I haven't a 5.1 system and don't care for a 5.1 system, but I'd pay extra for binaural in-game sound.

 

[Arwin]

5.1 isn't great for precise positioning for sound, but by goodness is it great at recreating the feeling you are in a certain space, and increasing the sound resolution (3x the channels after all). My father has great quality stereo sound, and I've had really great headphones in the past (until they were stolen) but for me 5.1 wins it every time.

That said, I did hear some excellent 3D simulations over a basic stereo headset.

 

[jonabbey]

I know this isn't the audio thread, but I'm looking forward to the audio improvements next gen that we might get purely due to the extra ram, hard drive installs, and cross platform blu-ray. I want a GTA game set in liberty city again that sounds something like what it actually sounds like to walk down the street in Manhattan, even if it were only in mono! It's the sheer amount of sound that we might get that's exciting to me.

 

[(((interference)))]

I'm fairly certain that PS4 (given what we know) won't have the same level of specialised audio processing hardware as XB1. The SHAPE block and audio processing cores can theoretically relieve the CPU from audio processing entirely and do it with far more effects/processing and concurrent audio streams than currently seen on even the most demanding games- audio wise - (like Forza which uses 3/6 Xenon threads).

There's a significant chunk of their 5 billion transistor budget budget going into that block (I mean the ESRAM only takes up ~1.6 bn transistors), and it's definitely not there merely for Kinect speech processing and MEC.

Depending on exactly how capable PS4's audio DSPs are, it could mean PS4 would have to trade off CPU/GPU time to match XB1 if devs want to have audio parity between the two. And if it turns out that Sony is also reserving 2 CPU cores then effectively XB1 has a slight CPU advantage as it doesn't need to dedicate one or more CPU cores to game audio (if SHAPE is properly utilised).

Using this as a comparison with the next gen consoles is flawed. 3rd parties will be presented with a system that has a 50% CU advantage, 100% ROP advantage and non-trivial bandwidth advantage compared to its competitor. In my opinion an argument can be made that this will be the difference between 720p and 1080p - at the same frame rate - by looking at the distribution of workload in both these cases you can see how the advantages fit surprisingly well.

Wow, Graham laying down the smack!
Rangers, since you're counting, here's one dev who doesn't think there won't be much difference between the two.

Yep. The only possible alternative to this IMO is if someone went with binaural audio and headphones. If one of the console reveals had all the audience wearing headsets and it placed then in the middle of an audio soundscape, with bees buzzing past their heads etc., everyone would be writing about it and raving about it. As it is, we just have more of the same with a lack of vision and audio is 'good enough'.

Interestingly enough, i've heard that they had (or still have) plans to support up to four wireless stereo headsets (simultaneously) with HRTF. Would be amazing for Halo split screen play.

 

[pc999]

On sound disign, which I know a litle (I am more into music production).

Mixing and processing fx on a few dozen voices (tracks if you prefer) can kill even a modern i7 cpu, specially if you are using good fx for essential stuff like reverbs.

In fact there is a feature in almost all modern DAW (digital audio workstation) that is freeze, that is pre render the track so it is possible to use more fx. More, there is even dsp cards to be bought so it doesnt it so much the cpu.


I really hope for a good audio dsp with good fx, specially reverb, audio in games still is quite primitive in many aspects (eg reverb).

As a curiosity, from pro audio developers I have talked to, they all say that gpgpu is good for audio (beside convulsion reverbs and still quite a hit). But dsp always have been a good option for audio, it is amazing how a little chip can do wonders.

 

[tuna]

I haven't a 5.1 system and don't care for a 5.1 system, but I'd pay extra for binaural in-game sound.

I got a really good 5.1 system a couple of months ago and I think it makes a great difference compared to stereo.

 

[DJ12]

I've got a 7.1 system thats too loud and big for the missus to have out in the living room :banghead:

 

[Arwin]

Incidentally, Shifty, isn't binaural a technology to simulate better bass-sound? That has basically already used in MP3 and CD recordings for a long time, to get better bass by giving the left and right ears slightly different frequencies, suggesting the brain to construct a third, lower frequency? If so, I don't see why we should pay extra for that in games.

 

[Cjail]

5.1 isn't great for precise positioning for sound, but by goodness is it great at recreating the feeling you are in a certain space, and increasing the sound resolution (3x the channels after all). My father has great quality stereo sound, and I've had really great headphones in the past (until they were stolen) but for me 5.1 wins it every time.

That said, I did hear some excellent 3D simulations over a basic stereo headset.

I am with you on this.


@Shifty Geezer

In the DS4 video presentation they hint a possible use of the DS4 for dynamic sounds.
They say they could make bullets whiz come form the DS4 speaker to create the sensation that bullets are flying by, which is similar to what you suggest with bees buzz.

 

[Davros]

Anyone got any idea how this shape chip compares to a EMU20K1 dsp (in the xfi)

Incidentally, Shifty, isn't binaural a technology to simulate better bass-sound?

no, to try to encode positional information over 2 speakers

ps: my holy grail of audio processing is realtime holophonic encoding, but it wont happen due to the poularity of multi speaker setups

 

[Shifty Geezer]

Incidentally, Shifty, isn't binaural a technology to simulate better bass-sound?

Not that I know of.

This was the demo of the effect for headphones that really illustrated it for me back in who-knows-when. the first recording was in 1996, according to the website.

Things like the scissors and whisper are incredibly authentic. You just can't do that with 5.1 (well, maybe you can with careful positioning and fancy processing). Binaural just involves sticking on a pair of headphones for the user. There are also loads of binaural recording on freesound.org. Often they just use a four-mic device and some software processing, so the results aren't always as good, but there are some very convincing ones. If the audio was computed and could be calibrated for each listener, it'd be perfect. For me, all these recordings lack a front dimension and I don't hear anything ahead of me.

 

[Shifty Geezer]

In the DS4 video presentation they hint a possible use of the DS4 for dynamic sounds.
They say they could make bullets whiz come form the DS4 speaker to create the sensation that bullets are flying by, which is similar to what you suggest with bees buzz.

That'll just be a sound in the palm of your hands. It'll be a cue, but it won't be immersive.

 

[Cjail]

I just think there's room for some interesting use of the DS4 speaker.
Audio cues help me to feel more immersed but surely they don't suffice, they alone don't make me feel immersed.
Anyway I surely won't design the audio of a game around DS4 speakers.

 

[Arwin]

Not that I know of.

This was the demo of the effect for headphones that really illustrated it for me back in who-knows-when. the first recording was in 1996, according to the website.

Things like the scissors and whisper are incredibly authentic. You just can't do that with 5.1 (well, maybe you can with careful positioning and fancy processing). Binaural just involves sticking on a pair of headphones for the user. There are also loads of binaural recording on freesound.org. Often they just use a four-mic device and some software processing, so the results aren't always as good, but there are some very convincing ones. If the audio was computed and could be calibrated for each listener, it'd be perfect. For me, all these recordings lack a front dimension and I don't hear anything ahead of me.

Yeah, that's the same tech I was talking about, but then I googled binaural and came to the explanation of the increased bass response.

I had that as an mp3, sounded better. But I would need to hear it applied a bit more broadly than just those examples.

Incidentally I see someone mentioning mono headset all the time for the PS4, but the DS4's audio out supports stereo, so I only think that mono bit refers to an included headset for chatting, if anything at all.

Surely the internal speaker can do some interesting things, but we've had the Wii controller support that for quite a while already, so it's not exactly new or exciting.

Kinect in theory could be a good help for sound positioning, as it knows where you are in the room, so sound levels could be manipulated to your position, which could be used for some interesting effects.

 

[Scott_Arm]

There is a lot of room for improvement in game audio. Hopefully we'll have more games doing positional audio with "HDR audio" like in Frostbite. If the SHAPE audio block makes that easier, then I'm all for it. I'm expecting PS4 to have some kind of audio processor this gen, because the CPUs aren't really a strong point. Vita has some kind of variation of an off-the-shelf audio processor. I would expect PS4 to be something along those lines.

 

[Deleted member 13524]

Anyone knows if any of the consoles will have a microphone-assisted, integrated room correction suite? I think this plays a crucial role in getting a decent 5.1 set to sound as expected in games and movies.
Both Windows 7 and Windows 8 have room correction, so I guess this should be a given for the xbone?
And Sony could use its proprietary DCAC from their line of A/V receivers.

As for the binaural audio, I wouldn't prefer it to a good 5.1 set in gaming.
Sure, a good set of headphones is much better than 5.1 speakers for emulating a mosquito buzzing around my head, someone cutting my hair or anything that happens near my ears.
But good 5.1 speakers will be much more convincing at playing the sound of an explosion happening 20 meters away from me, the sound of wind passing through the leaves in a forest or the engines of racing cars approaching 6 o'clock.
For ambient sounds, surround speakers will be best, and that matters more for games AFAIK.

I'm no strange to great quality stereo sound, but I largely prefer gaming with my 90€ Roccat Kone 5.1 headset than with my 250€ Shure SE425 earphones.
After that, playing in my living room with the 5.0 Jamo set is by far the best gaming experience.

 

[Scott_Arm]

Binaural audio should be great for the positioning of effects like explosions. I'm not sure why there are 5.1 headphones, to be honest. Sounds like the headset jack on the Xbox One controller might support 5.1. So I don't know if that means Binaural audio is not going to happen.

 

[Shifty Geezer]

But good 5.1 speakers will be much more convincing at playing the sound of an explosion happening 20 meters away from me, the sound of wind passing through the leaves in a forest or the engines of racing cars approaching 6 o'clock.

That's untrue. A speaker can only define a sound at a distance relative to that speaker from the listener. The only way to simulate larger spaces than those within the audio field of the speakers is to process the audio, and you can do that just as well if no better with binaural sound. The problem with binaural audio is it needs to be calibrated for each listener to be perfect, otheriwse it sounds more like wide stereo with a smaller enclosed volume.