Technical investigation into PS4 and XB1 audio solutions *spawn
- Thread starter Brad Grenz
- Start date
Dominic Mallinson mentioned TrueAudio. Anandtech said PS4 audio DSP was based on TrueAudio. AMDGaming tweeted "The audio chip on PS4 has TrueAudio technology". No-one's talked about TrueAudio on PS4 in public. The only description we've had about how PS4's audio works is mention of mixing (VGLeaks confirmed by Sony) and talk of GPGPU for audio.
We have to the best of my knowledge zero PS4 games demonstrating TrueAudio technology (spatial 3D in headphones), and a slide from Killzone showing CPU processing 160 audio voices.
We have a VR headset described as featuring HRTF 3D audio yet Sony haven't described PS4 as featuring HRTF.
Does the evidence really weigh in that PS4 has exactly the same audio chip as AMD's TrueAudio GPUs? Or that it just uses the same decompress and mixing engine of TrueAudio but not the DSP side?
We have to the best of my knowledge zero PS4 games demonstrating TrueAudio technology (spatial 3D in headphones), and a slide from Killzone showing CPU processing 160 audio voices.
We have a VR headset described as featuring HRTF 3D audio yet Sony haven't described PS4 as featuring HRTF.
Does the evidence really weigh in that PS4 has exactly the same audio chip as AMD's TrueAudio GPUs? Or that it just uses the same decompress and mixing engine of TrueAudio but not the DSP side?
If the DSP side is in the PS4, it could be secret because it's reserved for Morpheus anyway, so no devs would ever know about it. But Morpheus is now public, and it has an external box to do the processing. It's hard to imagine why they'd still keep this DSP a secret, there's no reason.
Love_In_Rio
Veteran
And why wouldnt no morpheus games be allowed to use it?.For me is clear that PS4 only have the decompressing and mixing part of TrueAudio.
Last edited by a moderator:
TheWretched
Regular
Come on... how can it be clear to any of you what the DSP or audio hardware actually is or does? We have no info on it beside "it exists" and can do some stuff. But we don't know if it can ONLY do this and not the other.
You're right, no-one at the moment really knows. However, there's some evidence to look at. Surely the fact Sony haven't said, "our DSP can do HRTF," counts for a pretty great deal? That's not in keeping with their divulging of PS4's specs, nor their description of Morpheus.
As a technical investigation thread, the idea is to piece together the facts with the clues to come to an understanding of the console. At this point, don't you agree the weight of evidence is that PS4's 'DSP' is simply the mixing block as described?
TheWretched
Regular
I don't know, actually... Sony has been know to pronounce the prowess of their hardware in the past. With PS4, it has generally died down a little. So I am really not sure either way. Maybe it is there, but the software isn't ready... or it isn't and it's really just for compression tasks.
Rurouni
Veteran
The problem in supporting something for a specific usage, such as HRTF on a console is that unless you create the game with some computing power to spare for if someone enables HRTF, then you'll get a slightly difference experience between someone that enable/disable that (in this case, if you enable HRTF, you might get a slight reduction in fps or maybe something else). For VR, everyone will use head/earphone, thus HRTF can be used and games will be made with that in mind. It's similar to Kinect reservation in that games that don't use Kinect still have some reservation for it because the system might use it.
If Sony does reserve some computing power (either on CPU, GPU, or programmable DSP) for HRTF, that's basically computing power wasted because I think the majority of console players don't use headphone.
Maybe Sony is programming the DSP so that it can be used more instead of reserved for special case? Or maybe they did reserve it for special case, thus they don't want to talk about it because it will be processing power wasted for something that you don't use (Morpheus). Same thing as their insistence on keeping the light on Ds4.
Why nobody tried to confirm to Sony about it, especially after the AMD reveal? The only thing we've got is the DSP is based on TrueAudio and everyone that probably have closer access to Sony (journo) just assume that the same DSP on AMD chip is used on PS4.
If Sony does reserve some computing power (either on CPU, GPU, or programmable DSP) for HRTF, that's basically computing power wasted because I think the majority of console players don't use headphone.
Maybe Sony is programming the DSP so that it can be used more instead of reserved for special case? Or maybe they did reserve it for special case, thus they don't want to talk about it because it will be processing power wasted for something that you don't use (Morpheus). Same thing as their insistence on keeping the light on Ds4.
Why nobody tried to confirm to Sony about it, especially after the AMD reveal? The only thing we've got is the DSP is based on TrueAudio and everyone that probably have closer access to Sony (journo) just assume that the same DSP on AMD chip is used on PS4.
I know Digital Foundry were up to do a technical interview with Sony similar to the one they did with the XBox architects, but I guess that fell through. 
One of the strongest reasons against the DSP is the VGLeak documentation. We know this was valid, yet it doesn't mention a DSP beyond audio decoding and mixing. Unless the TrueAudio DSP was added after this leak as a last-minute addition (possible; maybe Sony couldn't be sure it'd be ready in time when they decided they'd definitely have audio mixing and released the development docs), it would surely have appeared in the tech specs.
One area of confusion though is how Morpheus can handle HRTF accurately. If it is positioning the sounds, it should have access to the all mono samples. Taking Killzone for example, 160 voices at CD quality would need 700 kbps * 160 = 112 MBps, which is too much for HDMI 1.4's Ethernet although possible number of voice could be capped to be compatible? If it hasn't got access to the mono samples, it can only be transforming 5.1 to binaural, which should be an inferior experience unless their engines are Uberclever. Once Morpheus is in the wild, we can determine if the binaural audio is from the PS4 or breakout box, and conclude the audio DSP situation.
One of the strongest reasons against the DSP is the VGLeak documentation. We know this was valid, yet it doesn't mention a DSP beyond audio decoding and mixing. Unless the TrueAudio DSP was added after this leak as a last-minute addition (possible; maybe Sony couldn't be sure it'd be ready in time when they decided they'd definitely have audio mixing and released the development docs), it would surely have appeared in the tech specs.One area of confusion though is how Morpheus can handle HRTF accurately. If it is positioning the sounds, it should have access to the all mono samples. Taking Killzone for example, 160 voices at CD quality would need 700 kbps * 160 = 112 MBps, which is too much for HDMI 1.4's Ethernet although possible number of voice could be capped to be compatible? If it hasn't got access to the mono samples, it can only be transforming 5.1 to binaural, which should be an inferior experience unless their engines are Uberclever. Once Morpheus is in the wild, we can determine if the binaural audio is from the PS4 or breakout box, and conclude the audio DSP situation.
D
Deleted member 11852
Guest
Sony have divulged PS4 specs when it suits them or when they felt they need too, such their clarification on the RAM available to games,
For whatever reason, which may or may not be to do with Morpheus, the audio hardware isn't available for developers to use. I believe Sony will remain silent on this until / if they chose to release access to the DSP.
D
Deleted member 11852
Guest
And the VG leaks documentation all came from early developer documentation, which you wouldn't expect to detail hardware that the developer has no access too. There's no detail on the custom chip/embedded ARM core either but we know it's there.
TheWretched
Regular
The best HRTF systems use all sorts of data to alter sounds before they reach the final mix, a lot of the info HRTF relies on is missing (object velocity, distance, etc) if you try and remix a 7.1 all of that is gone. There are several surround head sets that do that with 'Dolby Headphone' and it's not even as impressive as the old Aureal A3D tech to my ears.
TheWretched
Regular
It's been a while... but I actually had an Aureal3D card. Was quite interesting. But it was so long ago, when gaming hasn't really taken off, yet.
So... if not perfect, it is possible to reprocess a 7.1 stream? Then that's probably what they're doing, if it's easier.
So... if not perfect, it is possible to reprocess a 7.1 stream? Then that's probably what they're doing, if it's easier.
HRTF is a filtering process which can downmix from 4 channels and up without loosing too much localization information. The precision of the human hearing system is rather low when we talk about localization at the left or right side of the head because of attenuation. The precision is around 15 degrees at the left/right and around 1 degree at the front. If you change the downmix format you obviously need to recalculate the datasets.
HRTF can be performed either by convolution (time or frequency domain) or by filter network/banks.
The convolution process can be implemented via a GPU or CPU/DSP algorithm. The current x86 CPU's found in XBOX1 and PS4 can process 45-50 HRTF sound sources using no more than 12% of a single core. If you extract the frequency response and time information from the HRTF dataset you could use the filterbanks in SHAPE for the implementation instead of CPU/GPU.
HRTF is not hardware issue, but a software issue - many game engines doesn't have any proprietary audio engines. They distribute the DSP work to common platform APIs like XAudio2 (PC, XBOX1), Core Audio (Mac), ALAudio (Linux) etc. XAudio2 was released March 2008 without any special DSP updates since then. Core Audio even older. Since most games use platform specific audio engines, most games will sound different depending on the platform - XBOX1/PC use a reduced Princeton reverb, Mac uses AUMatrixReverb and PS4 a Sony implementation.
Last edited by a moderator:
I'm pretty sure
HRTF is not a filtering process which can downmix from 4 channels and up without loosing too much localization information. (it can do that but its not about downmixing really) its about adding positional cues to audio
convolution is about reverb not hrtf
what you do with convolution is you play a sound in an environment (something simple like a click or a sine wave) and record that sound then you do some fancy calculations (thats a technical term) possibly you remove the original sound from your recording and your left with a recording that contains the reverb characteristics of the environment (lets call it a reverb map). Then in game the dsp combines the game audio with that reverb map and your audio reverbs as if it was actually recorded in that environment.
How game devs are going to create their reverb maps I'm not sure? maybe wise/fmod et al has options for that.
HRTF is not a filtering process which can downmix from 4 channels and up without loosing too much localization information. (it can do that but its not about downmixing really) its about adding positional cues to audio
convolution is about reverb not hrtf
what you do with convolution is you play a sound in an environment (something simple like a click or a sine wave) and record that sound then you do some fancy calculations (thats a technical term) possibly you remove the original sound from your recording and your left with a recording that contains the reverb characteristics of the environment (lets call it a reverb map). Then in game the dsp combines the game audio with that reverb map and your audio reverbs as if it was actually recorded in that environment.
How game devs are going to create their reverb maps I'm not sure? maybe wise/fmod et al has options for that.
It's not "about" downmixing, but it would basically be a sort of downmixing in this case.
Convolution can be used in many kinds of filtering. The same train of thought that says you can record the response of a click in an echoing environment and convolve it with a sound signal to get the reverb result, also says that you can record the response of a click in the human ear and convolve it with a sound signal to get the "HRTF'd" result.
HRTF is a filtering process. Convolution is a mathematical operation (1st order Volterra) which can be used for numerious filtering operation, like Reverb, HRTF, Sinc interpolation etc etc. Convolution is used in statistics, DSP (audio and image), differential math etc etc.
Convolution Reverb is an extremely small part of the usage of convolution.
TheWretched
Regular
Well, my line of thought was, that if it's possible to do "good" HRTF from 7.1 channel audio, then Morpheus and any other HRTF headset can simply grab the regular audio from the HDMI out and reprocess it. That'd remove the need for "hundreds of audio streams" to be sent to the device beforehand.
Similar threads
- Locked
