Next Generation Hardware Speculation with a Technical Spin [post E3 2019, pre GDC 2020] [XBSX, PS5]

[JoeJ]

@JoeJ Maybe DXR came out because they wanted several years of work on RT in the game space before the new Xbox came out, so developers would be able to hit the ground running. If DXR came out on console launch, you'd have to wait years into the generation before devs figured it out. Maybe they'd been planning ray tracing on xbox since before DXR came out, or Nvidia announced RTX. We have no way of knowing when they started to plan it, or why.

Yes. The point might be they wanted to build up RT experience early for some reason.

Burden of Proof is on you for these claims, otherwise they are just completely baseless and sending everyone way off course

Yes, totally baseless speculations in the wrong thread - not intended to send people anywhere. (Which surely won't happen - everybody has a mind on his own )

Because without MS, RTX would never have the support it would need. Every game that would support it is completely custom and no developer would want to build separate RT solutions for each IHV. And MS will benefit from it because they want more developers to choose DirectX over Vulkan.

Sure. That's why we thought NV asked MS for supporting RT. Ofc. NV wants this, but again: They could have released their extension to DirectX without MS making a big thing about it. Technically there would be no difference, and i don't see a big contribution from MS side here at all, other than the publicity of presenting innovation.
The question remains: Why did MS 'make' an API for only one vendor? They never did this before. Intel / AMD still have no RT support, so why carve an API in stone years before there is a need for vendor abstraction at all? This is unusual, don't you agree?
Plus, DirectX is ruled by MS, not a committee like Khronos. AMD and Intel were informed, but this does not mean they were happy to know rival NV is the only one with supporting all DX features for some years, and they might not be happy with the API as is either.

To me, Sony planning RT for next gen is in deed the most plausible explanation for MSs unusual acting here, even if it sounds far fetched.
This does not mean i'm convinced it happened like this. To me it only matters Sony having custom RT becomes a bit more likely, which i did not consider to be probable until recently.

 

[mpg1]

I mean there are a lot of reasons why I don't think Sony is using some alternative RT solution other than AMD's. But the most obvious is Cerny's quote of "There is ray-tracing acceleration in the GPU hardware".

He isn't saying generally: "There is ray-tracing hardware in PS5". He isn't even saying: "There is ray-tracing hardware in the GPU" (not that would be a big difference). But his words go one step further in terms of specificity saying the acceleration is a feature of the GPU hardware....

 

[Scott_Arm]

...
Yet if it was readily doable on GPU, why hasn't it been done? I don't know how well GPU works for audio, but what if the 3D processing can be accelerated as well as video blocks handle video? No-one's going to arguing removing the HEVC decoders and doing video decode on compute. If there is 3D audio hardware in PS5, it'll be because it brings a better economy of silicon.

TrueAudio Next requires an AMD 400 series GPU. There is a low-latency queue in the GPU front-end for scheduling audio processing. Neither Xbox One or PS4 support this. They went with dedicated audio dsps and processors (in the case of PS4 it was AMD TrueAudio). Basically, why would anyone support TrueAudio Next right now, if it's only usable on AMD GPUs (the minority on PC) and not usable on PS4 and Xbox One? Games have to run on the shittiest and most common hardware available, which is the consoles and old gpus. Classic case of waiting for the lowest common denominator to catch up.

3D audio is going to require ray tracing, which is something the GPU will be good at. Why make a second ray-tracing device to do the job the GPU was already built to do? Whether the new consoles are RDNA1 or 2, the most likely case is the GPU will be relied on for most of the 3D part of the audio, which is occlusion, reflections and convolution.

 

[Scott_Arm]

And here's a good write-up on how it works and why it's good. That's from two years ago, pre-navi, so I imagine the solution has only improved since then on both a hardware and software level.

https://www.anandtech.com/show/12407/amd-valve-introduce-trueaudio-next-support-for-steam-audio

And here's another link from AMD explaining how audio is conventionally done vs how it'll need to be done for VR with respect to TrueAudio Next.

https://gpuopen.com/wp-content/uploads/2016/07/TAN_whitepaper_2016.pdf

 

[JoeJ]

I mean there are a lot of reasons why I don't think Sony is using some alternative RT solution other than AMD's. But the most obvious is Cerny's quote of "There is ray-tracing acceleration in the GPU hardware".

He isn't saying generally: "There is ray-tracing hardware in PS5". He isn't even saying: "There is ray-tracing hardware in the GPU" (not that would be a big difference). But his words go one step further in terms of specificity saying the acceleration is a feature of the GPU hardware....

Well, that's a point. Did not knew about this quote, do you remember the source?

To me, what speaks against custom mostly is the problem of latency. Likely they could not trace, shade, trace and even recurse within the same program.
On the other hand i would not have expected this to work well at all before RTX came up.

 

[mpg1]

Well, that's a point. Did not knew about this quote, do you remember the source?

To me, what speaks against custom mostly is the problem of latency. Likely they could not trace, shade, trace and even recurse within the same program.
On the other hand i would not have expected this to work well at all before RTX came up.

https://www.wired.com/story/exclusive-playstation-5/

Before they do, Cerny wants to clarify something. When we last discussed the forthcoming console, he spoke about its ability to support ray-tracing, a technique that can enable complex lighting and sound effects in 3D environments. Given the many questions he’s received since, he fears he may have been ambiguous about how the PS5 would accomplish this—and confirms that it’s not a software-level fix, which some had feared. “There is ray-tracing acceleration in the GPU hardware,” he says, “which I believe is the statement that people were looking for.” (A belief born out by my own Twitter mentions, which for a couple of weeks in April made a graphics-rendering technique seem like the only thing the internet had ever cared about.)

 

[Scott_Arm]

@mpg1 So Sony and Mark Cerny have used "Ray-tracing in hardware" and "ray-tracing acceleration" interchangeably. He also says, "in the GPU hardware," so I wonder where all the ray casting for the audio will be done?

 

[PSman1700]

@mpg1 So Sony and Mark Cerny have used "Ray-tracing in hardware" and "ray-tracing acceleration" interchangeably. He also says, "in the GPU hardware," so I wonder where all the ray casting for the audio will be done?

Secret hardware chip, maybe Cell

 

[Frenetic Pony]

Secret hardware chip, maybe Cell

Cell is back baby! Cell 2.0, Revenant

 

[mpg1]

@mpg1 So Sony and Mark Cerny have used "Ray-tracing in hardware" and "ray-tracing acceleration" interchangeably. He also says, "in the GPU hardware," so I wonder where all the ray casting for the audio will be done?

In the initial Wired article it's a bit confusing because they are generally talking about the befits of ray tracing including 3D audio while mentioning a separate "custom unit":

Ray tracing’s immediate benefits are largely visual. Because it mimics the way light bounces from object to object in a scene, reflective surfaces and refractions through glass or liquid can be rendered much more accurately, even in real time, leading to heightened realism. According to Cerny, the applications go beyond graphic implications. “If you wanted to run tests to see if the player can hear certain audio sources or if the enemies can hear the players’ footsteps, ray tracing is useful for that,” he says. “It's all the same thing as taking a ray through the environment.”

The AMD chip also includes a custom unit for 3D audio that Cerny thinks will redefine what sound can do in a videogame. “As a gamer,” he says, “it's been a little bit of a frustration that audio did not change too much between PlayStation 3 and PlayStation 4. With the next console the dream is to show how dramatically different the audio experience can be when we apply significant amounts of hardware horsepower to it.”

So maybe it's a audio processor that works in tandem with the GPU's RT hardware? I dunno...

 

[Globalisateur]

In the initial Wired article it's a bit confusing because they are generally talking about the befits of ray tracing including 3D audio while mentioning a separate "custom unit":



So maybe it's a audio processor that works in tandem with the GPU's RT hardware? I dunno...

The initial Wired article is not confusing at all. It's only confusing if you assume ray tracing support can only mean ray traced audio. If you don't assume anything (like PS5 absolutelty won't do Ray tracing ligthing, I don't believe it, OK maybe PS5 will do some kind of ray traced shadows because I read about it elsewhere), then everything is clearly explained. They are talking about ray traced lighting for the most part of the article, then a mention about possible ray traced audio, then another mention about a dedicated audio unit for 3D audio.

 

[Dictator]

The Wired article wasn't that long ago. The first time Sony came out with Hardware based language was Mar/April 2019.

DXR was announced and released as beta to devs at GDC 2018 running on Volta with the starwars demo. This is pre-Geforce 20xx RTX.
Hardware acceleration was the term used then and continues to be the term they use today.

I don't believe that this position makes sense from a timeline perspective.

@Dictator lol your commentary in your video here is in jeopardy!

I am so happy they are! haha

 

[3dilettante]

Yet if it was readily doable on GPU, why hasn't it been done? I don't know how well GPU works for audio, but what if the 3D processing can be accelerated as well as video blocks handle video? No-one's going to arguing removing the HEVC decoders and doing video decode on compute. If there is 3D audio hardware in PS5, it'll be because it brings a better economy of silicon.

The main objections Sony's audio engineer had back in 2013 were issues with latency and a lack of flexibility in terms of granularity.
Launch latency under load worsened due to contention with graphics demands spiking and potentially delaying the availability of CUs for potentially tens of milliseconds.
AMD has moved to improve things with the high-priority queues and reserved CUs to make sure that there's always resources available and a clear path to request them.
Whether that's improved things enough to meet Sony's desires is unclear.
Another objection was the desire for a fully flexible solution where sources and effects could be arbitrarily and freely combined in a programmable pipeline, which ran counter to the wide wavefronts needing the program to batch many work items doing the same thing to get decent utilization.
Navi's 32-wide wavefronts mildly improve this over GCN, but that might not be enough.

TrueAudio Next requires an AMD 400 series GPU. There is a low-latency queue in the GPU front-end for scheduling audio processing. Neither Xbox One or PS4 support this. They went with dedicated audio dsps and processors (in the case of PS4 it was AMD TrueAudio). Basically, why would anyone support TrueAudio Next right now, if it's only usable on AMD GPUs (the minority on PC) and not usable on PS4 and Xbox One? Games have to run on the shittiest and most common hardware available, which is the consoles and old gpus. Classic case of waiting for the lowest common denominator to catch up.

For the PS4, the DSP block was somewhat undesirable since it was often being used for system services, had limited computational resources, and using it injected additional latency.
Its latency was better than the GPU, and it didn't have the large batch size, but it seems Sony liked the block for its being a secure path for accessing and decoding assets rather than counting on it for heavy effects.

Even with the changes for Trueaudio Next, I wonder if Sony would want something more. We don't have head-to-head comparisons and latency figures for the solutions, and even with reserved CUs we don't know how quickly they can respond. Effects still need to go through the chain of command processors and launch wavefronts, which take non-zero time to spin up, and enough audio effects may lead to thrashing of the small pool of reserved CUs. The static partition of CUs for the highest response time is itself a source of inflexibility, and Wave32 or Wave64 aren't as flexible as Sony would have liked.

There's the patent concerning a CU modified to run continuous kernels that can almost take direct messages from the host, has narrower hardware, and has a more rapid task launch and flexible register file. Maybe that could show up for the most latency-demanding effects?

 

[mpg1]

The initial Wired article is not confusing at all. It's only confusing if you assume ray tracing support can only mean ray traced audio. If you don't assume anything (like PS5 absolutelty won't do Ray tracing ligthing, I don't believe it, OK maybe PS5 will do some kind of ray traced shadows because I read about it elsewhere), then everything is clearly explained. They are talking about ray traced lighting for the most part of the article, then a mention about possible ray traced audio, then another mention about a dedicated audio unit for 3D audio.

I was talking about how the 3D audio specifically works. The two brief paragraphs in the Wired article don't "clearly explain" how the 3D audio is working on a technical level...at least not for me.

 

[Scott_Arm]

@3dilettante The only latency data in the white paper is referring to the audio queue as the real-time queue and the following.

“Graphics is isolated from audio, and audio is isolated from graphics. Only memory bandwidth is shared, of which audio takes a very small share when compared with graphics, and for which DMA transfer latencies are far more than adequate. Glitch-free convolution filter latency as low as 1.33 ms (64 samples @ 48 kHz) has been achieved with over 2 second impulse responses. Typical audio game engines require 5 to 21 ms total buffer latency.”

I don’t have a frame of reference for comparison, but the entire paper is the suggestion that it is suitable for real-time 3D audio in vr. RDNA has come out since, and with RDNA2 likely supporting ray tracing in some capacity I’d expect changes in capability for audio.

 

[3dilettante]

@3dilettante
“Graphics is isolated from audio, and audio is isolated from graphics. Only memory bandwidth is shared, of which audio takes a very small share when compared with graphics, and for which DMA transfer latencies are far more than adequate. Glitch-free convolution filter latency as low as 1.33 ms (64 samples @ 48 kHz) has been achieved with over 2 second impulse responses. Typical audio game engines require 5 to 21 ms total buffer latency.”

I don’t have a frame of reference for comparison, but the entire paper is the suggestion that it is suitable for real-time 3D audio in vr. RDNA has come out since, and with RDNA2 likely supporting ray tracing in some capacity I’d expect changes in capability for audio.

Perhaps that is close enough. The remaining documentation from the Sony audio presentation doesn't have the audio where some figures were given. 1.33 ms seems close to the desired CPU-level range, though I think there was mention of sub-ms being possible on Jaguar. The fact that it's "as low as 1.33 ms" might leave some uncertainty for the most demanding scenarios.

 

[Scott_Arm]

Perhaps that is close enough. The remaining documentation from the Sony audio presentation doesn't have the audio where some figures were given. 1.33 ms seems close to the desired CPU-level range, though I think there was mention of sub-ms being possible on Jaguar. The fact that it's "as low as 1.33 ms" might leave some uncertainty for the most demanding scenarios.

There is some performance data in this steam audio blog post.

https://steamcommunity.com/games/596420/announcements/detail/1647624403070736393

The tldr is gpu convolution is necessary for a large number of audio sources, and steam audio offers higher audio options with TAN than on CPU. There is some detail and explanation of the benefits and was they chose to support it.


Edit: so further reading says TAN does not include ray tracing. Occlusion is reliant on Radeon rays. TAN is for intensive audio effects like real-time convolution reverb on the gpu.

 

[dobwal]

Yes. The point might be they wanted to build up RT experience early for some reason.

Yes, totally baseless speculations in the wrong thread - not intended to send people anywhere. (Which surely won't happen - everybody has a mind on his own )

Sure. That's why we thought NV asked MS for supporting RT. Ofc. NV wants this, but again: They could have released their extension to DirectX without MS making a big thing about it. Technically there would be no difference, and i don't see a big contribution from MS side here at all, other than the publicity of presenting innovation.
The question remains: Why did MS 'make' an API for only one vendor? They never did this before. Intel / AMD still have no RT support, so why carve an API in stone years before there is a need for vendor abstraction at all? This is unusual, don't you agree?
Plus, DirectX is ruled by MS, not a committee like Khronos. AMD and Intel were informed, but this does not mean they were happy to know rival NV is the only one with supporting all DX features for some years, and they might not be happy with the API as is either.

To me, Sony planning RT for next gen is in deed the most plausible explanation for MSs unusual acting here, even if it sounds far fetched.
This does not mean i'm convinced it happened like this. To me it only matters Sony having custom RT becomes a bit more likely, which i did not consider to be probable until recently.

Who says MS made DXR for only one vendor? If RT in the XSX is in the works in 2018 so is an API that supports it. And RT in both the PC and console space benefits everyone involved. Its basically all the same tech, so there are going to be solutions derived from one area that's applicable in the other. You want to get everyone on board as quickly as possible for the benefit of x86 gaming as a whole.

A ton of devs showing RTX's RT performance support console gaming so why would you want to wait on AMD's solution and deprive those devs getting their feet wet even though AMD's PC gpus and consoles aren't ready?

 

[JoeJ]

?

Nvidia describes non RTX general compute based ray tracing as “GPU accelerated software based raytracing”. Hardware based would imply the use of RT cores.

If MS rushed DXR then they must of did a better job than Nvidia because nvidia describes DXR as a low level api not replicated by OptiX until providing a low level version around sisgraph 2019.

I'm confused how your commend is related to my quoted one. My point was not about compute vs. fixed function, i only meant Sony used different terms because they may have worked on HW RT years before DXR/RTX announcement.
I don't know OptiX, but people said it's similar to DXR. Also DXR is pretty equal to the VK extension as far as i can tell. So my speculation is DXR is mostly the work of NV, with MS just adopting it and making it a standard. I don't see a big 'job' done by MS at all - DXR seems tailored to NV RTX.
The whole progression Volta SW -> Turing HW, API support for DX, VK and OptiX has happened very quickly, showing primarily NVs competence as a software company.
So i don't mean DXR was rushed in the sense it would be a poor or bad API. I mean Microsofts move to make it a generic standard, not a vendor extension which it still is, seems unusually rushed.

 

[JoeJ]

Who says MS made DXR for only one vendor?

Imagine other vendors would create different RT implementations, containing ray reordering and hitpoint - material sorting.
This would break DXRs assumptions a trace can be considered an atomic operation.
There would be a need to buffer ray launches and resulting intersections, and the user programs would need to process those as batches, not per single ray in a presistant control program.
To me this indicates DXR == current RTX, not more. Likely we will see a lot of changes...