Technical investigation into PS4 and XB1 audio solutions *spawn

[zupallinere]

It is somewhat interesting, however, that AMD specifically included an Audio DSP block to support audio in their GPUs rather than using compute for anything audio related in games. And from what Dave said, it was mainly due to synchronization issues rather than contention for GPU compute resources.

Meaning that it's likely far more complicated than one would assume to do properly synchronized game audio through GPU compute than otherwise (CPU for example). With audio manipulation programs that isn't gong to be an issue as there is nothing you have to synchronize it with. With video players, again, it isn't an issue as you can delay your video stream in order to synchronize with the audio. For games, you can't delay each frame of the game to synchronize with the audio.

As well, while much time and money has been invested to do audio manipulation through CUDA for Nvidia hardware, that doesn't apply to game audio. And it isn't as if Nvidia or developers are afraid to use GPU compute for non-graphics rendering purposes. Nvidia have been pushing hard for GPU compute to be used in games whenever possible, so them avoiding it for game audio is fairly conspicuous.

What isn't known, of course, is whether Sony have included any modifications to the the GPU block with an eye towards solving that problem.

Regards,
SB

Geez audio everywhere right now

There are a lot of problems to be solved and audio is one of them. Of course solving other problems leaves more resources for audio issues should there be any. I don't know the specifics of the synchronizations issues that Dave is mentioning but I have a hunch they have to do with access to the CPU. This would be an issue for all GPU cards, including Nvidia and their compute solutions, since they all hang out on the "wrong side of the tracks" on the PCIe bus :smile: This would be less of an issue for the consoles and maybe even less so for the PS4 ( assuming Sony is allowing a fair amount of fine grained control over caches and the like ). Meaning while HSA has benefits for GPU cards if they take advantage of it, there is more CPU/GPU collaboration possible with APUs as things now stand.

So while finding a GPGPU audio solution during the game might have fewer eyeballs on the problem than more generalized solutions to other gaming issues I would find it puzzling to think that audio will be the downfall of the PS4 performance wise. It's not like they wouldn't be aware of the issue which would bring us back to the point you make at the end. We just don't know

If there is enough weeping from the SDF with audio like with the memory allocation kerfuffle maybe Sony will let a few more beans spill

"Fine Grained Control" -> PS4 as "Balance (tm) -> XB1

 

[Scott_Arm]

I don't think there's any neat way you can tidily say SHAPE = X CUs or SHAPE = X CPU cores etc. We know some games, like Forza, took up a lot of CPU on the 360. You could ask the question, how much CPU time would have been saved if the 360 had a fixed-function audio processor like SHAPE. Can we actually answer that question? I don't know the answer. It really depends on what they're doing with the audio, to know how much of it the fixed-functionality of SHAPE covers. The CPUs on next-gen should be able to do a lot more, so SHAPE may offload a good chunk of the audio processing, but it may be a relatively small amount of processing when compared to what the Jaguar CPUs can do. Trying to say you'll save a core, or whatever, doesn't make a lot of sense, because you don't know if games would even use a full core for audio. SHAPE should free up a little bit more CPU time than the PS4 audio processor, but the significance of that is hard to define precisely.

 

[rokkerkory]

It sounds like it's extremely capable at what it does, but there are a lot of people who've inflated it's capabilities in their minds in an attempt to win console war arguments. Both in terms of how capable it is and in terms of how it can be used.

True and from what I read, others are downplaying it so they can also substantiate their next-gen purchase as well.

It's a good dedicated sound processor, let's give it that and see what results may come.

 

[SlimJim]

It sounds like it's extremely capable at what it does, but there are a lot of people who've inflated it's capabilities in their minds in an attempt to win console war arguments. Both in terms of how capable it is and in terms of how it can be used.

But one of it's the developers stated that it will severely outpower the most powerful current commercially available sound cards.
And this person is an authority on the subject as well, so take that as you will..

 

[pjbliverpool]

You mean those 8 year old sound cards?

 

[SlimJim]

You mean those 8 year old sound cards?

Unless he meant that the poster he was replying to can only afford an 8 year old sound card, I am pretty sure he meant available right now. Although "commercially" (which I used because I couldn't find the quote) suggests that it's consumer level, but the way I read it now it could also include the professional-grade sector.

...
I can't speak to what the PS4 has, but the X1 audio block would put the best sound card you can buy to shame. And that's _before_ you add in _any_ of the DSP cores.
...

edit: "put.. to shame" does not really equal "severely outpower" (again I couldn't find the quote when I made my post) but I don't think he meant that because of a design standpoint the other cards would be put to shame. Because the subject was the discussion of the processing power

 

[Cyan]

True and from what I read, others are downplaying it so they can also substantiate their next-gen purchase as well.

It's a good dedicated sound processor, let's give it that and see what results may come.

I wonder if bkilian thinks they could build a sound engine tailored to SHAPE and let it shine. Davros talked about the X/Fi and its tech, the Vector 2 processor, etc.

The thing is... SHAPE isn't compatible with those specific functions for the most part, but it could have its own solutions.

 

[mosen]

You mean those 8 year old sound cards?

No.

Sorry, I should have said "consumer sound cards". The X1 is not a professional mixing station, it's a game console.
And the mix buffers have 128 physical buffers, but can be used with over 4000 virtual buffers per audio frame. Think of them as registers that can hold an entire audio frame. The 21369 has 32, much smaller ones. The SRC can process 512 channels per audio frame, and the XMA decoder can decode 512 channels per audio frame.

The clock speed of the audio block is twice that of a 21369, and the fixed function blocks were calculated, per the hotchips presentation, to be 18 GOPS equivalent. The 21369 is 2.4 GFLOPS. If you assume the scalar tensilica cores are about the same power per clock of a 21369, and use the 15.4 GFLOPS value for the two vector cores, you're talking 23 21369s equivalent for the whole audio block. How much did that 12 core sound card cost again? I found an 8 core one for something like $1500. Let's change my statement to "the Xbox one audio block is far more powerful than any sound card you can buy for less than or equal to the price of an entire Xbox one."

I believe you heavily underestimate the power of the X1 audio block.

 

[liolio]

As side note yesterday during AMD presentation they made the point that dsp not only free CPU cycle but also has a positive effect on cache (and cache misses which translate in extra memory traffic).
Not that it changes much but as always the cycles freed doesn't tell the whole story .

 

[Cyan]

This video shows the first differences in audio between this generation and the next.

The Xbox One car in Forza 5 sounds much closer to the actual car.

 

[pjbliverpool]

I've no idea what a 21369 is but I doubt it has anything to do with accelerating game audio on the PC. The last "gaming" sound card on the PC to the best of my knowledge was the X-Fi about 8 years ago. If SHAPE didn't crush that I'd be amazed.

 

[XpiderMX]

I've no idea what a 21369 is but I doubt it has anything to do with accelerating game audio on the PC. The last "gaming" sound card on the PC to the best of my knowledge was the X-Fi about 8 years ago. If SHAPE didn't crush that I'd be amazed.

The SoundBlaster X-FI Platinum is from 2010, but you're right, It is hard that anything that come from Xbox One amaze you. xD (j/k).

 

[pjbliverpool]

The SoundBlaster X-FI Platinum is from 2010, but you're right, It is hard that anything that come from Xbox One amaze you. xD (j/k).

Did it (the platinum) come with a significantly enhanced dsp over that of the original X-Fi?

And for the record, I'm pretty amazed by Kinect 2 and the living room control interface in general. SHAPE also sounds pretty cool (yeah pun intended lol). Just because I like to try and keep people grounded in reality when it comes to performance claims/expectations doesn't mean I have some kind of hate for either console. XB1 is a day 1 purchase for me.

 

[Relab]

Just stop please.

This is becoming a joke (and a bad one). No benchmark or any evidence have been presented that show the actual performance of the audio system. The mentioned GFLOPS have no real meaning.

 

[XpiderMX]

Did it (the platinum) come with a significantly enhanced dsp over that of the original X-Fi?

And for the record, I'm pretty amazed by Kinect 2 and the living room control interface in general. SHAPE also sounds pretty cool (yeah pun intended lol). Just because I like to try and keep people grounded in reality when it comes to performance claims/expectations doesn't mean I have some kind of hate for either console. XB1 is a day 1 purchase for me.

I was just kidding, and yes, some people have expectations too high, but there are a lot of people with knowledges here in B3D.

 

[Cyan]

Except for some videos here and there, I don't know how the new games are going to sound on the PS4 and Xbox One, all I know this subtlety is going to be easy to emulate. I am astounded...

Now that I mention it, I can't wait for the first Formula E championship to begin. I wonder if they can outperform Formula 1 cars -most likely not, I'd say, we shall see-

 

[bkilian]

Just stop please.

This is becoming a joke (and a bad one). No benchmark or any evidence have been presented that show the actual performance of the audio system. The mentioned GFLOPS have no real meaning.

Because no benchmark can be given. You've already been told the chip can do 512 "high quality" polyphase SRC (resampling factor from 1/4 to 4 I think), 512 3 band EQ - comprised of 3 biquad filters, 512 Compressors with a hard knee response, 2500 state variable filters + volume changes, and 512 WMA like decodes per audio frame. Now you've already said just the state variable filters a would take 90% of one of your high end DSPs, which when included alone in a card costs about $250, you also estimated the EQ/CMP at 48% of a DSP, which seems a little low, but you're the expert. We're already at a minimum of a two DSP card (~$500) and we haven't even touched the SRC. Unfortunately, I don't know the number of taps on the FIR filter, but you could use the standard mid-quality 64 taps in your calculation if it makes you feel better.

The mix buffers will mix >4000 channels per frame, at 28 bit. The rest of the pipeline is 24 bit.

That's just Shape. One block out of 5 on the sound chip (and sadly, as far as I know, the only one easily accessible to developers)

For the non-audio folks here, mixing, filtering and SRC are a huge majority of the operations that an audio engine performs. Things that are missing here are Convolutions, FFTs and other custom DSP effects, but whether you use those or not, you will always be doing a lot of mixing, filtering and SRC.

The reason the team chose fixed function blocks to do those functions is because they could provide a way for the most common audio tasks to be accelerated to a degree not even an expensive DSP could easily match. But in doing so, they gave up the flexibility of the aforesaid DSP. So as long as you're only needing to do exactly what the chip can do, you get "more than a CPU core" of offload. In every other case, a general purpose DSP would be better, but a lot harder to convince developers to use, they hate writing DSP code.

 

[XpiderMX]

Except for some videos here and there, I don't know how the new games are going to sound on the PS4 and Xbox One, all I know this subtlety is going to be easy to emulate.

You can hear a F4 vs F5 here:

http://www.youtube.com/watch?v=mXSGAZ2eKkU#

 

[RudeCurve]

Because no benchmark can be given. You've already been told the chip can do 512 "high quality" polyphase SRC (resampling factor from 1/4 to 4 I think), 512 3 band EQ - comprised of 3 biquad filters, 512 Compressors with a hard knee response, 2500 state variable filters + volume changes, and 512 WMA like decodes per audio frame. Now you've already said just the state variable filters a would take 90% of one of your high end DSPs, which when included alone in a card costs about $250, you also estimated the EQ/CMP at 48% of a DSP, which seems a little low, but you're the expert. We're already at a minimum of a two DSP card (~$500) and we haven't even touched the SRC. Unfortunately, I don't know the number of taps on the FIR filter, but you could use the standard mid-quality 64 taps in your calculation if it makes you feel better.

The mix buffers will mix >4000 channels per frame, at 28 bit. The rest of the pipeline is 24 bit.

That's just Shape. One block out of 5 on the sound chip (and sadly, as far as I know, the only one easily accessible to developers)

For the non-audio folks here, mixing, filtering and SRC are a huge majority of the operations that an audio engine performs. Things that are missing here are Convolutions, FFTs and other custom DSP effects, but whether you use those or not, you will always be doing a lot of mixing, filtering and SRC.

The reason the team chose fixed function blocks to do those functions is because they could provide a way for the most common audio tasks to be accelerated to a degree not even an expensive DSP could easily match. But in doing so, they gave up the flexibility of the aforesaid DSP. So as long as you're only needing to do exactly what the chip can do, you get "more than a CPU core" of offload. In every other case, a general purpose DSP would be better, but a lot harder to convince developers to use, they hate writing DSP code.

I think that about sums it up...

 

[Cyan]

Because no benchmark can be given. You've already been told the chip can do 512 "high quality" polyphase SRC (resampling factor from 1/4 to 4 I think), 512 3 band EQ - comprised of 3 biquad filters, 512 Compressors with a hard knee response, 2500 state variable filters + volume changes, and 512 WMA like decodes per audio frame. Now you've already said just the state variable filters a would take 90% of one of your high end DSPs, which when included alone in a card costs about $250, you also estimated the EQ/CMP at 48% of a DSP, which seems a little low, but you're the expert. We're already at a minimum of a two DSP card (~$500) and we haven't even touched the SRC. Unfortunately, I don't know the number of taps on the FIR filter, but you could use the standard mid-quality 64 taps in your calculation if it makes you feel better.

The mix buffers will mix >4000 channels per frame, at 28 bit. The rest of the pipeline is 24 bit.

That's just Shape. One block out of 5 on the sound chip (and sadly, as far as I know, the only one easily accessible to developers)

For the non-audio folks here, mixing, filtering and SRC are a huge majority of the operations that an audio engine performs. Things that are missing here are Convolutions, FFTs and other custom DSP effects, but whether you use those or not, you will always be doing a lot of mixing, filtering and SRC.

The reason the team chose fixed function blocks to do those functions is because they could provide a way for the most common audio tasks to be accelerated to a degree not even an expensive DSP could easily match. But in doing so, they gave up the flexibility of the aforesaid DSP. So as long as you're only needing to do exactly what the chip can do, you get "more than a CPU core" of offload. In every other case, a general purpose DSP would be better, but a lot harder to convince developers to use, they hate writing DSP code.

Wow, just::.. it's astonishing. With all those capabilities at its disposal... Does it mean that they could create a 3D audio engine for Shape alone after the console is launched? -sometimes afterwards- Or just in my dreams, maybe?

XpiderMX, surely it sounds great and you piqued my curiosity. I have already watched the video you shared though and I even gotten further sharing it twice in two different threads.