Impact of nVidia Turing RayTracing enhanced GPUs on next-gen consoles *spawn

[Tkumpathenurpahl]

Is there anything that's sort of an in-between?

So, silicon that's maybe best used for ray tracing but can be used for rasterisation, for example. Or vice versa.

Dedicated ray tracing hardware makes sense for Nvidia: they're selling a ray tracing graphics card, which will absolutely see use in the likes of the film industry.

Spending a huge amount of silicon on ray tracing hardware is risky for a console because it's stuck with it forever, and ray tracing tech is so nascent.

If there's something nearly as flexible as CU's but that's also well suited to ray tracing, I can see that sort of a compromise being part of a console's design, giving it an opportunity to get its feelers out in the world of ray tracing.

 

[OCASM]

Which is at least 5 years away, if not more. RT needs to be fast enough compared to existing rasterization hacks for it to even make sense to re-build a big part of your engine for.

For full path tracing sure, for more moderate uses I'd say much sooner than that. Besides, RT isn't anywhere as complex as the algorithms it replaces. That's one of its selling points.

More than that, it economics. Why write a ray-tracing optimised engine for PC when 99% of your potential buyers don't have raytracing capabilities? Until it's ubiquitous, RT will be relegated to optional shadowing etc. If it becomes fast enough, RT could replace lighting hacks as a swap-in feature that'll look a lot better. But the market for RT optimised engines isn't there and won't be for years. If both next-gen consoles featured the same RT solution, perhaps it'd have a chance, but realistically we'll see hybrid rasterisers with RT adding some niceties for those who can afford it.

It won't be a one step switch. Not all games will use the technology to the same extent nor the same techniques.

In terms of engine development, EA has the SEED group, Epic is adding DXR support in version 4.22 of UE4 and Unity is working on it as well.

 

[iroboto]

Is there anything that's sort of an in-between?

So, silicon that's maybe best used for ray tracing but can be used for rasterisation, for example. Or vice versa.

Dedicated ray tracing hardware makes sense for Nvidia: they're selling a ray tracing graphics card, which will absolutely see use in the likes of the film industry.

Spending a huge amount of silicon on ray tracing hardware is risky for a console because it's stuck with it forever, and ray tracing tech is so nascent.

If there's something nearly as flexible as CU's but that's also well suited to ray tracing, I can see that sort of a compromise being part of a console's design, giving it an opportunity to get its feelers out in the world of ray tracing.

There's a lot more to Turing than just RT Cores and Tensor Cores. It's easy to look at benchmarks of games in which it's not performing well compared to older hardware and people will point at laugh.

But Turing comes with an abundance of features not found in any other card, which, despite having less brute force power (that could be gained by dedicating all the silicon to traditional rendering) they've pulled back and looked at how things could be done smarter.

I look at these features in Turing
link: https://www.nvidia.com/en-us/geforc...ented-new-technologies-in-rtx-graphics-cards/

And they are the type of features we need in the console space, because we don't have the luxury of power, so efficient and smart use of it is where we're going to get the biggest payoffs.

We talk a lot about Ray Tracing as this is the big one because MS is supporting it directly.

but missed from the conversation are usage for the tensor cores:

• Using AI to Up Res
• Using AI to perform anti aliasing DLSS

Both of these can be supported by Direct ML, and MS has the Azure ML that could totally support this type of DLSS functionality, and all console people want super high res screenshots.

Then when it comes purely to architecture:

• Variable rate shading
• motion adaptive shading
• content adaptive shading
• foveate rendering (for VR if that's still going to happen)
• mesh shading
• texture space shading

All of these would be super useful in console land where everything is so limited.
I'm not looking just at RT as being this thing alone that will define next gen, but looking at an overall package that would be able to define a new generation of console.

Imo a 2070 is really good for a 2020 console.

Certainly a console with less TF, but could do DLSS, AI-Up Res for screenshots, and have variable, motion, and adaptive, is likely to put out superior image quality and experience over using all that silicon the way we do today.

 

[see colon]

There's nothing to show for it yet. 2080 Ti class hardware at 1080p to barely reach 60fps won't cut it for consoles. The increase in die space alone means you have to cut corners where you know something works, for something that might work.

Back in the late 90s and early 2000s rasterization performance was damn near doubling every 6 to 12 months. Pixel shading followed a similar curve. If ray tracing does half that we would be looking at high end PC parts with twice (or more) the performance in two years. If that's true, and you could tune a game like you can on console, that barely 60 could be a solid 60 today or 120 in a few years, which is the same pixel rate as 4k30.

Which is at least 5 years away, if not more. RT needs to be fast enough compared to existing rasterization hacks for it to even make sense to re-build a big part of your engine for.

I don't know, there's been a lot of consoles that have made strange design choices that developers have used regardless of how much work it takes. We know the performance impact that RT has on RTX only based on graphics showcases designed only to show off the visual impact. We don't actually know if there could be a performance benefit in some cases. Take the real time reflections and shadows in the demos we've seen, not only are the shadows and reflections more accurate (light reacts the way it should), they are also higher quality. They are nice looking but what if there was a quality slider that produced more accurate than current shadows but at a lower quality than the ones showcased. Like, more accurate but at current quality (resolution, ect) settings. There's a possibility that there could be a performance benefit that isn't being shown because nVidia marketing needs to be able to show people the visual difference. You could also tune things on console to the performance of the chip. Only use RT for a few things within your render budget to maintain just enough of a visual edge over midrange PC parts that lack ray tracing. Looking back, RT is a lot like pixel shading was with the original Xbox. There wasn't the performance to use shaders on every surface in most cases. Many of those games used shaders for water and that's about it, but the water looked so good.

The only thing I would worry about RT being in next gen console is if it impacted rasterization performance. If it's an added bonus, that's fine and we'll get reflective puddles in every game like we got shaded water on XB.

 

[DavidGraham]

9.2 ms, over half the rendering budget, on the fastest RT GPU.

Those 9ms are used for a ton of RTX effects in Control. Shadowing, reflections and GI. Control is basically the game that uses the most RTX features out there, more than any other title. Considering the amount of RTX being rendered here, 9ms doesn't actually look too bad.

 

[Shifty Geezer]

It won't be a one step switch. Not all games will use the technology to the same extent nor the same techniques.

This doesn't make sense regards my point. You said games need to be designed for raytracing to showcase raytracing. I say there's not an economic argument to make games for RT GPUs when a tiny, tiny portion of the market will use them, and so Rt features added to games will be additions to rasterisers rather than games designed for raytracing.

So am I right or wrong? Are devs going to redesign entire engines around a base raytracing design for the benefit of the 0.1% of gamers who can use that RT? Or is the progression going to be gradual with changes here and there like addition of reflections and RT shadows until, when it makes economic sense, engines will be designed from the ground up to really showcase raytracing as you initially said was needed?

 

[Shifty Geezer]

Those 9ms are used for a ton of RTX effects in Control. Shadowing, reflections and GI. Control is basically the game that uses the most RTX features out there, more than any other title. Considering the amount of RTX being rendered here, 9ms doesn't actually look too bad.

Hmm, yeah. That report never mentioned that the RT time included the RT GI. What exactly is the 'fitting everything else in'?

Incidentally, I was quoting from the original post in the RT thread and didn't read the article myself before linking here.

Still, it's running 1080p on the fastest RT GPU out there, on a huge slab of silicon, and it doesn't look materially any more advanced than current games regards shading realism. Extrapolating from this example, are we to expect 1) A huge slab of silicon in the next-gen consoles, or 2) The same power is going to be reduced dramatically in the next year and fit a moderate sized slab of silicon, or 3) There's going to be a way more efficient system?

We don't know about option 3) as we've no idea what AMD might have cooking, if anything, but for 1 and 2 that seems quite the gamble. PS3 stuck in a huge slab of GPU silicon and that was one of the factor that made it a horrendous money sink. I can't see that happening again.

 

[Jupiter]

Posted in other RT thread...
9.2 ms, over half the rendering budget, on the fastest RT GPU. 2.3 ms for shadows and 2.5 ms for denoising, and apparently this is overall GPU time being eaten into and not with denoising happening parallel.independently of the shaders. If denoising is preventing you from shading, that's 30% of the rendering budget gone on 1080p60 shadows.

I think this speaks volumes about the cost/benefit ratio being completely off for next-gen consoles. If RT is going to happen in consoles, it needs to be something very different to RTX.

Why are final statements about the performance costs already made on the basis of this?

Softshadows (shadow maps) alone can cost up to 30% and therefore the Raytcing shadows would be even cheaper than shadow maps. In general shadows are one of the most expensive effects. Without something like softshadows shadows just look gamey. They also calculate the real-time GI and the reflections in this demo. Voxel GI needs a lot of performance and it limits more and it is less accurate than Raytracing GI.

Raytracing can also be optimized. This game is still far away and we dont know a lot about the raytracing resolution etc.

Dynamic lighting/environments is where RT really shines. Otherwise just stick to baking.

Baking can work well with GI but with shadows and reflections its problematic.

 

[DavidGraham]

Hmm, yeah. That report never mentioned that the RT time included the RT GI. What exactly is the 'fitting everything else in'?

In the original German report they mention each cost for each effect:
Shadows + denoising: 2.3ms
Global Lighting + denoising: 2.5ms
Reflections: 4.4ms
https://www.golem.de/news/geforce-r...e-an-leistung-kosten-koennen-1810-137115.html

 

[Deleted member 13524]

This is a debate about hardware choice. Putting in RT hardware presently means putting in less rasterising hardware. As a console engineer, you need to make the choice whether your owners are going to prefer 'RT on' or 'RT off' for the entire duration of the generation.

Which is why fixed-function hardware was supposed to be going away for gaming GPUs. Programmable units would be useful for any circumstance, regardless of the dev's approach.

So honestly I don't think dedicated fixed function RT units have much of a future in gaming GPUs. With programmable units they could find a way to implement it with e.g. lower precision, because "perfect precision" is apparently what kills performance from what I've read.

If AMD comes up with decent competition next year, using 7nm 300-400mm^2 GPUs that use all the extra transistors for increased rasterization performance, nvidia may not continue the RTX line at all.

Even the tensor cores may be questionable. Only practical use they seemed to find so far was resolution upscaling through extra dev efforts. For a rasterizer, maybe that extra die area could be better spent on more rasterizer hardware transistors that would provide the larger resolution without extra dev work and proprietary paths.

But they could/should keep the RT units for Quadros, as that will be a gamechanger for artists and industrial design engineers.

Local consoles = RT off
Local Consoles + xCloud Scarlet Streaming = RT on

Local consoles = RT off
Local Consoles + xCloud Scarlet Streaming = RT on + 300ms input lag

¯\_(ツ)_/¯

With typical game streaming all you're sending are tiny packets of data from the gamepad or KB+M. "Hybrid" renderization would require sending a data volume of a magnitude several times larger.
Might as well just stick to full streaming and deal with compression artifacts, AFAICT.

Even if 7nm comes into play and things change a little bit?

7nm won't change the fact that nvidia needs almost 19 Billion transistors just to get some pre-scripted demos running at 1080p60, just for reflections and shadows.
At best, TU102 shrinked to 7nm would be ~250mm^2 (probably closer to 300 TBH) , so it's not like you can jam 2/3x worth of a TU102 GPU inside a console SoC in that process. Not to mention the clocks..

 

[Tkumpathenurpahl]

There's a lot more to Turing than just RT Cores and Tensor Cores. It's easy to look at benchmarks of games in which it's not performing well compared to older hardware and people will point at laugh.

But Turing comes with an abundance of features not found in any other card, which, despite having less brute force power (that could be gained by dedicating all the silicon to traditional rendering) they've pulled back and looked at how things could be done smarter.

I look at these features in Turing
link: https://www.nvidia.com/en-us/geforc...ented-new-technologies-in-rtx-graphics-cards/

And they are the type of features we need in the console space, because we don't have the luxury of power, so efficient and smart use of it is where we're going to get the biggest payoffs.

We talk a lot about Ray Tracing as this is the big one because MS is supporting it directly.

but missed from the conversation are usage for the tensor cores:

• Using AI to Up Res
• Using AI to perform anti aliasing DLSS

Both of these can be supported by Direct ML, and MS has the Azure ML that could totally support this type of DLSS functionality, and all console people want super high res screenshots.

Then when it comes purely to architecture:

• Variable rate shading
• motion adaptive shading
• content adaptive shading
• foveate rendering (for VR if that's still going to happen)
• mesh shading
• texture space shading

All of these would be super useful in console land where everything is so limited.
I'm not looking just at RT as being this thing alone that will define next gen, but looking at an overall package that would be able to define a new generation of console.

Imo a 2070 is really good for a 2020 console.

Certainly a console with less TF, but could do DLSS, AI-Up Res for screenshots, and have variable, motion, and adaptive, is likely to put out superior image quality and experience over using all that silicon the way we do today.

Is there much known about the 2070's constituent parts? Namely, how much surface area's taken up by the tensor cores and RT cores?

And I agree that smarter use of silicon, such as the features you stated, are necessary. But most of those features are part of the normal cadence of graphics card improvements, and could exist without RT and tensor cores.

So I'm hesitant to agree fully, because I'm not yet convinced that RT and tensor cores are actually a smart use of silicon in the console realm, where die size is such a limiting factor.

Things like machine learning upscaling and AA are impressive, but could they just be done by equally modern upscaling hardware? If so, that silicon doesn't need to take up GPU die area.

Like I say, I don't know much about Nvidia's Turing architecture. But I do know that they've focused an awful lot on machine learning and self driving cars these last few years and I can't help but think that they've sort of retrofitted rendering features to a GPU that wasn't designed for rendering.

Again, that's based on no real knowledge of their technology, it's just a suspicion that hardware developed for self driving cars isn't the best solution for rendering and ray tracing.

 

[BRiT]

With typical game streaming all you're sending are tiny packets of data from the gamepad or KB+M. "Hybrid" renderization would require sending a data volume of a magnitude several times larger.
Might as well just stick to full streaming and deal with compression artifacts, AFAICT.

That is not and would not be typical streaming. We've talked a bit about that in the other threads.

 

[Jay]

Are devs going to redesign entire engines around a base raytracing design for the benefit of the 0.1% of gamers who can use that RT?

I'm only talking about this specific part.
I expect that games and engines are designed with it in mind, and then faking it for the rest added on after.
Asset design probably starts of clean, then manipulated and fakeness added after.
Same with most engines.
the rt version can be the ground truth, early part of development.
At the moment it's being retrofitted, i.e. Removing the faking as the games are already far in development.
It seems relatively straight forward, meaning its not a whole new paradigm that breaks current engines and would require ground up changes.
It just needs to be done at start not end of development.

Even more so if it's in a console, then they would likely add the faking on top of the rt version, as it may also be the only way to get good performance from the rt based console

 

[steveOrino]

Is there much known about the 2070's constituent parts? Namely, how much surface area's taken up by the tensor cores and RT cores?

And I agree that smarter use of silicon, such as the features you stated, are necessary. But most of those features are part of the normal cadence of graphics card improvements, and could exist without RT and tensor cores.

So I'm hesitant to agree fully, because I'm not yet convinced that RT and tensor cores are actually a smart use of silicon in the console realm, where die size is such a limiting factor.

Things like machine learning upscaling and AA are impressive, but could they just be done by equally modern upscaling hardware? If so, that silicon doesn't need to take up GPU die area.

Like I say, I don't know much about Nvidia's Turing architecture. But I do know that they've focused an awful lot on machine learning and self driving cars these last few years and I can't help but think that they've sort of retrofitted rendering features to a GPU that wasn't designed for rendering.

Again, that's based on no real knowledge of their technology, it's just a suspicion that hardware developed for self driving cars isn't the best solution for rendering and ray tracing.

There is of course some overlap with Nvidia's technology used in other high margin sectors. As to consolidation of that tech into their consumer oriented products, I think their lead in desktop and workstation graphics has no doubt influenced their strategy in trickling down the cores used in Tesla/quadro to avoid the cost of having 2 different designs. Could we live in a world (where AMD is performance competitive) where the 2000 series is just another Cuda core increase? Plausible, but I guess we will never know. As to what is "best" solution for realtime light transport I think its anyones guess. Obviously what Nvidia thinks is the best course can't be the only one but here we are because no one else is bothering to do it yet.

 

[iroboto]

So am I right or wrong? Are devs going to redesign entire engines around a base raytracing design for the benefit of the 0.1% of gamers who can use that RT?

The opposite. It’s likely all devs will be moving to upgrade their engines to RT in every fathomable way. Leveraging RT to see their effects, lighting and shadows in real time. Bounces and what not. And then for the not RT people, bake it.
It’s certainly a lot faster than how we bake, look, bake again. Compile and baking times are forever.

RT will dramatically increase the speed at which world artists and effect artists can iterate by having a real time ground truth to fake against.

And for those that actually have the hardware to support it, they are in luck in being able to see those effects running real-time dynamic.

 

[iroboto]

And I agree that smarter use of silicon, such as the features you stated, are necessary. But most of those features are part of the normal cadence of graphics card improvements, and could exist without RT and tensor cores.

So I'm hesitant to agree fully, because I'm not yet convinced that RT and tensor cores are actually a smart use of silicon in the console realm, where die size is such a limiting factor.

Things like machine learning upscaling and AA are impressive, but could they just be done by equally modern upscaling hardware? If so, that silicon doesn't need to take up GPU die area.

Depends on how much area they take.tensor cores aren’t large. They are 4x4 matrice multipliers. That’s why they can output a massive amount of output, cause they deal in strictly small sizes. That begs to ask the question if there are other things that require compute, but don’t require 32/64 bit accuracy, then tensor cores could be a fit.

End of the day developers will need to make use of the hardware, and at the PC level not everything is exposed, but at the console level, we are likely to see a lot more hardware exposed.

Inthink it’s important to note that I don’t expect a RT game. It’s very much hybrid as much as possible. But letting developers choose where and when they want to enable RT is going to be a dramatic difference in the way we visually interpret games.

All the demos you have seen so far has been an add on of RT. Not something designed with the artistic merits of RT in mind.

You don’t need every bell and whistle for RT to be considered good, which I think is the counter argument here.

It just needs to be used wisely to bring out the effects you need for that scene.

 

[Deleted member 13524]

Is there much known about the 2070's constituent parts? Namely, how much surface area's taken up by the tensor cores and RT cores?

We know the 2070's TU106 has ~95% the die size of 1080 Ti's GP102 (445 to 471mm^2). Its number of shader processors + TMUs + ROPs is close to 1080's GP104 while being 42% larger (314 to 445mm^3). Performance is some ~5-10% above the GTX1080 in most cases.
One could say the 288 tensor cores and 36 RT cores occupy most of those extra 130 mm^2 / 3.6B transistors, but we don't know how much bigger the Volta SMs are in comparison to Pascal SMs.
The GV100 has about 2x the shaders + TMUs + ROPs of a GP104, but the first has 21B transistors vs 7.2B for the second. 2x GP104 would be 14.4B so about 6B transistors is how much GV100 is spending on 640 tensor cores + upgraded SMs.

Pascal is the better rasterizer in perf/mm^2, though that was predictable. But Volta and Turing get considerable gains on compute-heavy workloads. nVidia could probably launch a GTX 2060 that is basically a TU106 without tensor and RT cores and 192bit / 6GB GDDR6, and it would get RTX 2070 performance at a substantially smaller die size than GTX 1080's GP104. Tensor + RT cores are a significant burden to the RTX family.




So going back to a console adoption, tensor + RT cores should cost at least some ~30% larger die-area / transistor count, if it was to assume the same proportions as nvidia uses in the RTX GPUs.
One could assume they could e.g. forego the tensor cores, but it seems those are used with RT to denoise so... they can't really take them off.

 

[Malo]

The opposite. It’s likely all devs will be moving to upgrade their engines to RT in every fathomable way. Leveraging RT to see their effects, lighting and shadows in real time. Bounces and what not. And then for the not RT people, bake it.
It’s certainly a lot faster than how we bake, look, bake again. Compile and baking times are forever.

RT will dramatically increase the speed at which world artists and effect artists can iterate by having a real time ground truth to fake against.

And for those that actually have the hardware to support it, they are in luck in being able to see those effects running real-time dynamic.

That's one aspect I'm wary about, the transition from traditional lighting/shadows to RT over the next 5-10 years. Since implementing these techniques are going to be so much easier going forward, how much dev effort is going to be put into lighting, shadowing and reflections 2 years from now? 5 years? Are we going to see an actual degradation of traditional rasterization quality in favor of RT simply due to time factor and costs? How much are the EA's and Ubisoft's going to spend on their annual iterations, like Assassin's Creed in Space come 2022, in getting the effects looking just as great which takes considerable development effort when they could possibly spend half as much time getting it good enough, with the gap in quality between traditional methods and RT growing much wider?

 

[OCASM]

This doesn't make sense regards my point. You said games need to be designed for raytracing to showcase raytracing. I say there's not an economic argument to make games for RT GPUs when a tiny, tiny portion of the market will use them, and so Rt features added to games will be additions to rasterisers rather than games designed for raytracing.

So am I right or wrong? Are devs going to redesign entire engines around a base raytracing design for the benefit of the 0.1% of gamers who can use that RT? Or is the progression going to be gradual with changes here and there like addition of reflections and RT shadows until, when it makes economic sense, engines will be designed from the ground up to really showcase raytracing as you initially said was needed?

Ray tracing is a very scalable technology. Just run it at a lower quality level for those with old/week hardware. Worse case scenario just use the Crysis approach, disable it and have the game look awful but run great.

 

[iroboto]

That's one aspect I'm wary about, the transition from traditional lighting/shadows to RT over the next 5-10 years. Since implementing these techniques are going to be so much easier going forward, how much dev effort is going to be put into lighting, shadowing and reflections 2 years from now? 5 years? Are we going to see an actual degradation of traditional rasterization quality in favor of RT simply due to time factor and costs? How much are the EA's and Ubisoft's going to spend on their annual iterations, like Assassin's Creed in Space come 2022, in getting the effects looking just as great which takes considerable development effort when they could possibly spend half as much time getting it good enough, with the gap in quality between traditional methods and RT growing much wider?

You would know the settings for the lighting and GI by using RT. Using those settings you bake the level. I would argue that rasterized games would look and be cheaper to build in this regard.

Whether a game used works real-time GI or baked lighting, if nothing changes they should be the same in which baked lighting is desirable. It’s the dynamic stuff that requires RT hardware to run in real-time.