Predict: The Next Generation Console Tech - AUDIO

[Shifty Geezer]

Holophonic audio uses plain stereo, it's just a brand name for a form of binaural recorded audio.

Yes, I know, but the announced audio capabilities of the headphone jack in the controller don't sound as if intended for binaural audio.

Finally, the PS4 will include a mono headset and microphone in every box that plugs directly into the DualShock 4. The system is capable of streaming 32Khz sound to the controllers' speakers for up to 2 players, but that reduces to 16Khz when 3 or more players are hooked up.

32 kHz would be okay, but it doesn't scream super audio experience to me.

I think its less the jack and more the ability of the console to synthesize multiple distinct holophonic audio 'experiences' for different controllers without the processing hardware up front. (Assuming the use case is 1-4 players, otherwise it would just for the one)

We have no idea what the audio capabilities of PS4's audio hardware is. Even then, the CPU has plenty of grunt to turn to audio work if the GPU is doing lots of typical heavy lifting. But with mono headset from the controllers, Sony don't appear to be aiming for a personal stereo experience out of the box for all users. If they were, wouldn't the current demos be showcased with binaural audio tracks on YT?

 

[expletive]

We have no idea what the audio capabilities of PS4's audio hardware is. Even then, the CPU has plenty of grunt to turn to audio work if the GPU is doing lots of typical heavy lifting. But with mono headset from the controllers, Sony don't appear to be aiming for a personal stereo experience out of the box for all users. If they were, wouldn't the current demos be showcased with binaural audio tracks on YT?

I didnt mean the PS4 couldn't do it, just that the stereo jack wouldn't be the reason why not. I'm sure they both can to some degree, its just a matter of at what cost.

 

[patsu]

3D audio processing on GPU (2010):
http://www.nvidia.com/content/GTC-2010/pdfs/2042_GTC2010.pdf

Some of the audio effects will depend on the geometry. Should be interesting to see if the GPU can process or preprocess visuals, audio and physics as it goes through the geometry systematically.

 

[Scott_Arm]

3D audio processing on GPU (2010):
http://www.nvidia.com/content/GTC-2010/pdfs/2042_GTC2010.pdf

Some of the audio effects will depend on the geometry. Should be interesting to see if the GPU can process or preprocess visuals, audio and physics as it goes through the geometry systematically.

Pretty interesting. I'd be curious to know if there are any games that have already used the GPU to assist with audio in any way.

 

[patsu]

Yes, I know, but the announced audio capabilities of the headphone jack in the controller don't sound as if intended for binaural audio.

32 kHz would be okay, but it doesn't scream super audio experience to me.

We have no idea what the audio capabilities of PS4's audio hardware is. Even then, the CPU has plenty of grunt to turn to audio work if the GPU is doing lots of typical heavy lifting. But with mono headset from the controllers, Sony don't appear to be aiming for a personal stereo experience out of the box for all users. If they were, wouldn't the current demos be showcased with binaural audio tracks on YT?

The controller has a stereo headset jack but a mono-speaker. The bundled headset is said to be mono too.

The PSP and Vita both use Wolfson audio processors (for mobile devices). Based on the interview with their audio engineer, there should be a programmable DSP in the Vita audio chip. Because of mobile phones, these audio chips can be equipped with echo and noise cancellation IPs (Wolfson sells them). However PSP and Vita probably don't need them.

For PS3, they went the SPU route for sheer power and full programmability.

For PS4, it sounds like they did the usual "interview the developers to find out what they need" rounds. They did the same for Vita. We don't know what technology they use this time. Developers may not have direct access to the audio hardware.

In Vita, they design something to fit into a mobile device's power and heat envelope. On PS3 and 4, they don't really have those concerns. Probably energy star rating, typical audio processing budget for games, and whatever the developers need.

Their end user vision will probably set the upperbound (e.g., the rumored Omnivisor with 3D audio, if true). I am guessing they will use the GPU for some advanced audio tasks. Cerny already mentioned audio raytracing in the latest interview.

 

[jonabbey]

Yes, I know, but the announced audio capabilities of the headphone jack in the controller don't sound as if intended for binaural audio.

32 kHz would be okay, but it doesn't scream super audio experience to me.

Eh? Isn't 32 kHz overkill for audio? Or are you thinking that 32 kHz would be divided in two for binaural?

 

[Scott_Arm]

Eh? Isn't 32 kHz overkill for audio? Or are you thinking that 32 kHz would be divided in two for binaural?

I think that's referring to sampling rate of the source.

 

[patsu]

tritosine5G dropped me a PM to say we are still far away from 3D audio (Takes too much processing for a big room). He complained that we still can’t get authentic sound right yet. So there, tritosine5G, you can post these stuff yourself next time. We won’t bite you. XXOO.

 

[Scott_Arm]

I hope the audio processor on Durango is the kick in the ass the industry needed for audio improvements next-gen. I'm sure PS4 will be able to do more on it's audio processor and maybe on the GPU as well. Overall, things should be better no matter which platform you pick. I've never seen much in the way of audio papers from GDC on this forum. Maybe I'll take a look when I get some time and post here if I find anything interesting. Sometimes you get those forward-looking talks, and I'd be curious to know if any of the devs have speculated about where audio will go.

 

[patsu]

We did discuss audio briefly:
http://beyond3d.com/showthread.php?t=45398

EDIT:
2 more links (advanced audio applications in various games) here:
http://beyond3d.com/showpost.php?p=1731591&postcount=324

Vita audio:
http://www.develop-online.net/features/1513/Heard-About-PS-Vita

360 and PS3 audio processing:
http://www.gamasutra.com/view/feature/132208/nextgen_audio_squareoff_.php?print=1

They were scattered in different posts on b3d.

I think that's referring to sampling rate of the source.

Which source are you referring to ?

 

[Blazkowicz]

32KHz is what the SNES's music chip would output, or what was used in NICAM (digital stereo sound broadcast over the air, to get stereo sound in television)
44.1KHz (as on Audio CD) and 48KHz are more commonly used.. It's not overkill, an old mathematical result in information theory (Shannon's theorem) says you need at least twice the sampling frequency to represent a signal of a given frequency, i.e. 32KHz allows up to 16KHz sound to be represented ; a bit more than that is ideal.

For human voice though it's overkill, 8KHz sampling can be considered good enough and 16KHz is the high quality standard.
If all you want to do is to beam up audio to a controller (and take mic input back) for in-game voice chat only then 16KHz is perfect (you need to be more concerned about the physical qualities of the headset and mike)

 

[Scott_Arm]

...

Which source are you referring to ?

The audio stream being sent to the controller for output through the headphone jack. I'm assuming Shifty is saying it is mixed at 32kHz rather than the 44.1kHz or 48 kHz that is common. I'm assuming that compromise is made to save bandwidth over whatever wireless connection they use for the controllers.

 

[patsu]

3DS is said to sport a 134MHz Ceva DSP dedicated for audio too (24 channels, 32kHz):
http://www.3dbrew.org/wiki/Hardware

 

[Shifty Geezer]

The controller has a stereo headset jack but a mono-speaker. The bundled headset is said to be mono too.

Sure, and PS4 may be capable of 3D audio. However, the like of stereo example to me suggests if it features on PS4 games, it'll be at the devs discretion, rather than a system-wide feature. I was hoping that there'd be a stereo headset included abd Sony would have told us about 3D stereophonic audio. The lack of noises in that regard makes me think Sony aren't interested, that and the lo-fi audio performance of the controllers (although 32 kHz might be good enough for all I know!).

tritosine5G dropped me a PM to say we are still far away from 3D audio (Takes too much processing for a big room). He complained that we still can’t get authentic sound right yet. So there, tritosine5G, you can post these stuff yourself next time. We won’t bite you. XXOO.

Not sure about that. I was checking out a holophonic audio plugin for Sonar not long ago. It's not outside the possibility of custom hardware I'm confident to say.

The audio stream being sent to the controller for output through the headphone jack. I'm assuming Shifty is saying it is mixed at 32kHz rather than the 44.1kHz or 48 kHz that is common. I'm assuming that compromise is made to save bandwidth over whatever wireless connection they use for the controllers.

Yep. The audio is 32 kHz, presumably MP3. That'll be fine for audio chat, possibly even overkill, but I think the budget is 4x 16 kHz streams and they upgrade to 32 kHz when the BW is available for fewer players. For in-game audio, the 44.1/48 kHz audio will need to be resampled at 32 kHz, losing the high end. That may actually make no perceptible difference though - the 44.1 kHz standard was selected on a supposed top frequency humans can hear as I understand it, but in real terms with the audio focus on the midrange, lower frequencies may be perfectly serviceable.

 

[patsu]

Sure, and PS4 may be capable of 3D audio. However, the like of stereo example to me suggests if it features on PS4 games, it'll be at the devs discretion, rather than a system-wide feature. I was hoping that there'd be a stereo headset included abd Sony would have told us about 3D stereophonic audio. The lack of noises in that regard makes me think Sony aren't interested, that and the lo-fi audio performance of the controllers (although 32 kHz might be good enough for all I know!).

Not sure about that. I was checking out a holophonic audio plugin for Sonar not long ago. It's not outside the possibility of custom hardware I'm confident to say.

How well does it work ? works with PC/Mac ?

Sony will want to upsell us the official surround sound wireless headset, like they did this gen. I expect the bundled mono-headset to be the throw-away type.

I have a "good" (read: expensive) pair of BT wireless stereo headset for my iOS devices. I can use it for the Playstations but for the theaterical game experiences, I would use my audiophile over-the-ear one.


I think the vendors should focus on fun, entertaining experiences. Something that makes my wife say, "that looks fun" immediately. Audio may be a part of the formula, but just like media sharing, they don't seem to be the main attraction.


As a gamer, I would love to see the next Rockband, SingStar, DanceCentral, SoundShape, Gran Turismo though.

I tried Biohock Infinite yesterday, the game has very rich audio experience to create that era feel. I think those type of audio experiences may be interesting too. Demon's Souls and the Ico/SotC games are like that too. Dark Souls kinda lost it a bit.

I guess stylized games like Journey, Flower; and stereoscopic 3D or visor games may be interesting from an audio perspective also.

I feel that something is still missing to reach out to more people.

 

[tritosine5G]

tritosine5G dropped me a PM to say we are still far away from 3D audio (Takes too much processing for a big room). He complained that we still can’t get authentic sound right yet. So there, tritosine5G, you can post these stuff yourself next time. We won’t bite you. XXOO.

well yeah that's close to the crux of it , there are worlds of difference between

'authentic sounding' vs. 'hyperrealistic sounding' , it will take a good long while for the latter to tone down .

 

[Xenio]

well yeah that's close to the crux of it , there are worlds of difference between

'authentic sounding' vs. 'hyperrealistic sounding' , it will take a good long while for the latter to tone down .

as we can't have authentic graphics.. we have to deal with approximations if I understood what you mean

 

[patsu]

WiiU is said to have a dedicated audio DSP as well:
http://gamerant.com/wii-u-tech-specs-case-152901/all/1/

Dedicated 120MHz audio DSP.
Support for 6 channel discrete uncompressed audio (via HDMI).
2 channel audio for the Cafe DRC controller.
Monaural audio for the Cafe Remote controller.

Can’t find any details on the DSP except one speculation.

 

[Davros]

The xfi dsp is rated at 10,000mips I did once find a chard of cpu's rated in mips and it put a 2.8 northwood at the same level, no idea if its true or not.

 

[patsu]

Probably can't compare MIPS across vastly different architecture and app instructions.