Next Generation Hardware Speculation with a Technical Spin [2018]

[OCASM]

I'm talking about developers with experience to a greater or lesser degree with experience with ray tracing.

They all by and large want RT in games. But they mostly don't think fixed function black box RT is the way to do it.

Regards,
SB

In the long run.

Exactly. Which is corroborates the idea it is just too early to have that implementation in a $400 console releasing at 2020 (which with that release date would have started it's design years ago, and is are finishing touches ate the moment.)

Design to implementation takes time. Same for RTX. It's not like NVIDIA designed it a couple months ago. If RT acceleration makes its way into consoles it would be a different and more advanced designed than RTX.

I'm all for features that make the whole of the machine more capable and flexible. I'm not a fan of features that are single purpose gimmicks that can't contribute to other aspects outside of their original intended use case.

Ray tracing is not a gimmick and RTX is suitable for many different use cases like collision detection, audio simulation and global illumination. Has any other "fixed function" hardware ever been as versatile as this?

In an ideal world you're be right. The idea of a generalized, non-fixed function based RT sounds a lot better than what Nvidia is doing. But once again, I'm not exactly sure what that means entirely, I also don't know if that's a limitation of DXR. Ie: only until DX11 were compute shaders brought into play. Compute _really_ just happened even though DX11 was around since 2007/08.

Having said that, we do need to start somewhere, and I'd rather start with basic RT features (before the mature ones) now, then to not have it at all next gen, and just go yet another generation of rasterization. It would imo, still open up a slew of new experiences imo. The idea that the inside of buildings will actually be lit by the lights and not baked etc. So that if lights are shot out or lights are turned off, in rooms, the hallways go black... etc etc.

There is more dynamism to stealth game-play when we can talk about manipulating light itself.

Reflections help gameplay too, now you can see enemies behind you and even around corners.

Well, I can live with that

I'm sry if my comments before seem to have ignore some arguments.
But I can tell you, I'm old enough that I won't get hyped for every new piece of technology (there were way to many in the past). Currently, nvidia praises the hell out of RTX, like companies do if they have something new they can sell. Like Shifty already mentioned there are other "traditional" ways reflections etc. can work without this new thing. They are not like the new thing, but good enough (what's always a point of the computer graphics) to be convincing with lower hardware-costs.
Currently there is nothing like demos and the promise that games are coming soon for both new nvidia technologies. If it only works good on a 2080 it won't be in consoles anytime soon. If it works good on the 2070 maybe, if nvidia is bringing out a 2060 with working RTX (that is usable in games) than chances are way better. But console manifactures look at every penny they spend for the chips, so it will depend on how good it works and how scalable it is (especially the AMD solution).

Those other methods have proven to be insufficient and too limiting, that's why pretty much nobody uses them. Ray tracing works on everything. In terms of console hardware, quality will be inferior to high end PCs but that is always the case.

 

[Shifty Geezer]

Having said that, we do need to start somewhere, and I'd rather start with basic RT features (before the mature ones) now, then to not have it at all next gen, and just go yet another generation of rasterization. It would imo, still open up a slew of new experiences imo. The idea that the inside of buildings will actually be lit by the lights and not baked etc. So that if lights are shot out or lights are turned off, in rooms, the hallways go black... etc etc.

That's not dependent on RT though. You could manage that less prettily than RT by dynamic lighting and SVOGI or somesuch internal lighting. RT already has to compromise how many shadow cast lights are possible so you're going to have compromises one way or another.

The only game changer I can really think of is seeing someone behind you in a reflection.

 

[iroboto]

That's not dependent on RT though. You could manage that less prettily than RT by dynamic lighting and SVOGI or somesuch internal lighting. RT already has to compromise how many shadow cast lights are possible so you're going to have compromises one way or another.

The only game changer I can really think of is seeing someone behind you in a reflection.

I agree. I think if I were to take the rasterization angle, we could go deep into svogi, and for ray traced shadows we have to be discussing conservative rasterization. There are tons of features in DX12_1 + SM6 that this whole generation missed out on. But in a discussion of purely features, and features only, the next gen console whether you choose to adopt RT or not, would have the same features. Added RT and Tensor cores are just more features. I don't really struggle with the concept of what rasterization is capable of. I'm struggling to see why less features would be more optimal than more features - the differential in power has got to be pretty great for that to be true.

Tensor Processing Units aren't exactly new either. Google is on their 3rd generation of TPU at the moment, Nvidia on their first 1st I think with volta. Their 2nd generation should be coming soon. AMD is likely to bring out their TPU as well, they won't be left behind in the deep based learning as that is where the money is.

And as you said, RT accelerators have been in PowerVR a long time ago and now nvidia finally has something equivalent or better in that space. It's really just a question of AMD here.

But i mean, we need to be realistic here, there are things about voxel GI that also have draw backs. There's a thread about it here on B3D and Tim Sweeny writes about it here:
https://www.dsogaming.com/news/epic-games-tim-sweeney-explains-lack-of-svogi-in-unreal-engine-4/

Epic Games’ Global Illumination technique is called Sparse Voxel Octree Global Illumination, a technique that was developed by Andrew Scheidecker. And despite what Epic Games showcased at last year’s GDC, it seems that this technique is too heavy for actual games.
As Sweeney told GameTrailers, that technique was extremely expensive. Therefore, Epic Games decided create a series of graphical effects that achieve the same image fidelity as SVOGI with far better performance.

So far the only engine supporting SVOGI is Crytek:

And regarding Crytek’s engine, yes; CryEngine 3 is the only engine – at this point – with proper dynamic Global Illumination. However, CryEngine 3’s light propagation volume is far less advanced than SVOGI. Not only that, but Crysis 3 has only one bounce GI and the sun is the only GI source.

But that's interesting because Drive Club is very similar in that regard. The Sun is the main GI source.

SVOGI was invented in 2012 and it was showcased and dropped. If it was going to happen it should have happened this year with the middle of the pack GPUs moving well into the 6 TF range. But instead of pushing svogi for their infilitrator demo (also 2012) they ran it with RTX instead.
I mean, I believe in rasterization techniques and such, but I have to admit if it was so simple we would have seen a SVOGI solution by now (given it's lead of 6 years) vs. Ray Tracing.

edit more articles:
https://www.eurogamer.net/articles/digitalfoundry-gdc-2013-unreal-engine-4

The key differentiating factor between last year's demo and this newer iteration is that the Sparse Voxel Octree Global Illumination (SVOGI) lighting system hasn't made the cut. Instead, Epic is aiming for very high quality static global illumination with indirect GI sampling for all moving objects, including characters.

"[SVOGI] was our prototype GI system that we used for Elemental last year. And our targets, given that we've had announced hardware from Sony, that's where we're going to be using Lightmass as our global illumination solution instead of SVOGI," senior technical artist and level designer Alan Willard told Eurogamer, stressing that this new iteration of the technology has evolved significantly beyond the current-gen system used in titles like Mass Effect 3. Certainly, just the presence of so much more memory on next-gen platforms should improve lightmap quality on its own.

Let me put this another way. 1080Ti/980TI etc have been around for quite a few years now. If SVOGI could be implemented at least at world builder level (so that you could use it to prototype lighting) on very strong hardware, we should have seen it by now. But we see exactly that happening with RTX.

So I don't think 1080TI is enough for SVOGI. That means it may not be enough for next-gen either. At least it's something to consider.
Here is: SVOGI 2.0 from nvidia on UE4 just released earlier this year. It's still exclusive to nvidia hardware as well... but ignoring that, the frame rate looks fairly comparable or worse to the full RT hardware demos we saw, with less accuracy, no reflections, probably less bounce... and without the directML to approximate things in.

I'm reading reports of 110FPS dropping to 30 sub 30.

 

[anexanhume]

Isn't that the same patent we already talked about last week, where result was the patent was merely updated by the patent office?

Edit: maybe not in this thread or in this forum, but this is all deja-vu to me.

I'm not a 100% sure on anyone posting this already - I haven't seen it. But I did post this, that had more to do with emulation, than backwards compatibility.

I believe we covered this patent when it was an application. Now it appears to be a granted patent.

 

[DavidGraham]

I mean, I believe in rasterization techniques and such, but I have to admit if it was so simple we would have seen a SVOGI solution by now (given it's lead of 6 years) vs. Ray Tracing.

NVIDIA experimented with various rasterization techniques before to push the visual boundary, more than any other I believe. They did HFTS shadows (based on conservative rasterization), VXAO, and VXGI. All of them had a very high cost to performance (50%+ hit) with a rather limited visual upgrade compared to RTX.

 

[iroboto]

NVIDIA experimented with various rasterization techniques before to push the visual boundary, more than any other I believe. They did HFTS shadows (based on conservative rasterization), VXAO, and VXGI. All of them had a very high cost to performance (50%+ hit) with a rather limited visual upgrade compared to RTX.

Here are some remarks on HFTS+ in games. (The Division)

i got the GTX 1080 too ( Palit GameRock ) but even with that card HFTS can take a big hit on the performance - there are situations / places where enemies appear the framedrop is going from 60 -48 with all settings on 100% and Ultra in 1080p. Switching over to PCSS in the same location and situation with the same settings i get a rock solid 60 fps.

keep in mind: HFTS IS ONLY ACTIVATED BY THE GAME DURING DAY at night its switching over to PCSS.

The idea that we're going to get SVOGI + Hybrid Ray Traced Shadows at 4K... because of 30% more silicon seems... unlikely.

direct ML is looking hot right now.

 

[DavidGraham]

Here are some remarks on HFTS+ in games. (The Division)

Here is Watch_Dogs 2 struggling to maintain 60fps at max settings and HFTS shadows on a 8700K and a 2080Ti @1080p! It even drops to 45fps!

People often complain how RTX costs too much, but we've seen much worse from Rasterization with much less visual flair. This has been happening for years on PC, and not just in Watch_Dogs 2, but also in games like Quantum Break (@Ultra and Native resolution), Kingdom Come Deliverance (Ultra), Gears Of War 4 (with Insane Reflections and Insane DoF), Ark Survival Evolved (@Epic settings and DFAO), Agents of Mayhem (Ultra with HFTS shadows), and Final Fantasy 15 (Ultra with VXAO and HFTS). All of these games already struggle to maintain 60fps @1440p and often 1080p as well, and that on a TitanXP no less!

 

[OCASM]

From the very influential and now freely available book "Physically Based Rendering: From Theory To Implementation":

http://www.pbr-book.org/

"The author team of Matt Pharr, Greg Humphreys, and Pat Hanrahan garnered a 2014 Academy Award for Scientific and Technical Achievement from the Academy of Motion Picture Arts and Sciences based on the knowledge shared in the first and second editions of the book this book. The Academy called the book a “widely adopted practical roadmap for most physically based shading and lighting systems used in film production.”"

I found these interesting bits:

http://www.pbr-book.org/3ed-2018/Retrospective_and_The_Future/Design_Retrospective.html

17.1.1 Triangles Only

Another instance where the chosen abstractions in pbrt impact the overall system efficiency is the range of geometric primitives that the renderer supports. While ray tracing’s ability to handle a wide variety of shapes is elegant, this property is not as useful in practice as one might initially expect. Most real-world scenes are either modeled directly with polygons or with smooth surfaces like spline patches and subdivision surfaces that either have difficult-to-implement or relatively inefficient ray–shape intersection algorithms. As such, they are usually tessellated into triangles for ray intersection tests in practice. Not many shapes that are commonly encountered in real-world scenes can be represented accurately with spheres and cones!

There are some advantages to designing a ray tracer around a single low-level shape representation like triangles and only operating on this representation throughout much of the pipeline. Such a renderer could still support a variety of primitives in the scene description but would always tessellate them at some point before performing intersection tests. Advantages of this design include:

• The renderer can depend on the fact that the triangle vertices can be transformed into world or camera space in advance, so no transformations of rays into object space are necessary (except when object instancing is used).
• The acceleration structures can be specialized so that their nodes directly store the triangles that overlap them. This improves the locality of the geometry in memory and enables ray–primitive intersection tests to be performed directly in the traversal routine, without needing to pass through two levels of virtual function calls to do so, as is currently the case in pbrt.
• Displacement mapping, where geometry is subdivided into small triangles, which can then have their vertices perturbed procedurally or with texture maps, can be more easily implemented if all primitives are able to tessellate themselves.

These advantages are substantial, for both increased performance and the complexity that they remove from many parts of the system. For a production renderer, rather than one with pedagogical goals like pbrt, this alternative is worth considering carefully. (Alternatively, triangles alone could be given special treatment—stored directly in acceleration structures and so forth–while other shapes were handled with a less efficient general purpose code path.)

Like I said before, triangles are handled by RTX-like hardware and special cases by compute units.

Also:

http://www.pbr-book.org/3ed-2018/Re...uture/Alternative_Hardware_Architectures.html

17.2.4 The Future

Innovation in high-performance architectures for graphics seems likely to continue in coming years. As CPUs are gradually increasing their SIMD width and adding more processing cores, becoming more similar to throughput processors, throughput processors are adding support for task parallelism and improving their performance on more irregular workloads than purely data-parallel ones. Whether the computer system of the future is a heterogeneous collection of both types of processing cores or whether there is a middle ground with a single type of processor architecture that works well for a range of applications remains an open question.

The role of specialized fixed-function graphics hardware in future systems is likely to become increasingly important; fixed-function hardware is generally substantially more power-efficient than programmable hardware. As the critical computational kernels of future graphics systems become clear, fixed-function implementations of them may become widespread.

Fixed-function is not the devil!

 

[Xbat]

The big question being is there enough silicon available to have Raytracing hardware for next gen? What's potentially the size range for a next gen APU?

 

[cheapchips]

At 7nm and 350mm² they should be around 20bn transistors? RTX2080 is 18.6bn

 

[Rootax]

Do we know if RT need a lot of vram bandwidth ? If it's more that average rasterisation, It will be a problem on console with the memory pool shared between cpu and gpu... (if the architecture doesn't change).

 

[jlippo]

Fixed-function is not the devil!

Indeed.
Just like for rasterization and texturing fixed function can be very helpful, I do not see why we should try to have compute only solutions for everything if it is not fast or power efficient enough.
https://research.nvidia.com/publication/high-performance-software-rasterization-gpus

 

[anexanhume]

At 7nm and 350mm² they should be around 20bn transistors? RTX2080 is 18.6bn

Vega 64 is 12.5bn. I’d use that as a starting point.

 

[OCASM]

That's not dependent on RT though. You could manage that less prettily than RT by dynamic lighting and SVOGI or somesuch internal lighting. RT already has to compromise how many shadow cast lights are possible so you're going to have compromises one way or another.

The only game changer I can really think of is seeing someone behind you in a reflection.

If you do the shadows path tracing style you could make the cost pretty much constant. Quality might not be as good though.

 

[Xbat]

Vega 64 is 12.5bn. I’d use that as a starting point.

Yup, people expecting rtx 2080 size GPU part are being very ambitious.

 

[iroboto]

Yup, people expecting rtx 2080 size GPU part are being very ambitious.

Xbox One X is 359mm^2
Xbox One is 363 mm^2

Vega is 510mm^2

I wouldn't use vega as a baseline for consoles, not the type of architecture that would be successful.

 

[Graham]

... lots of stuff about SVOGI ...

I expect the problems RT will face will be very much the same but likely worse problems as SVOGI, or similar techniques. If you consider the memory access patterns of an SVO 'ray' compared to an RT ray, then very rarely would the RT ray win out in terms of the amount of data or computation it would have to read/perform to generate a result.
This is the natural disadvantage of an algorithm like ray tracing where you have to get a precise result.

In my mind voxelization / SVO / SDF / cone tracing etc is fundamentally not that different from RT in how the data is represented or iterated, it's just *much* a simpler representation at the leaf nodes - you don't need to load and iterate lists of triangles, reevaluate / read complex materials and lighting etc, you can often sample at higher mips to approximate cones that represent more diffused light, etc.

Having said all that, I'm expecting to see interesting things from hybrid approaches.

 

[iroboto]

I expect the problems RT will face will be very much the same but likely worse problems as SVOGI, or similar techniques. If you consider the memory access patterns of an SVO 'ray' compared to an RT ray, then very rarely would the RT ray win out in terms of the amount of data or computation it would have to read/perform to generate a result.
This is the natural disadvantage of an algorithm like ray tracing where you have to get a precise result.

In my mind voxelization / SVO / SDF / cone tracing etc is fundamentally not that different from RT in how the data is represented or iterated, it's just *much* a simpler representation at the leaf nodes - you don't need to load and iterate lists of triangles, reevaluate / read complex materials and lighting etc, you can often sample at higher mips to approximate cones that represent more diffused light, etc.

Having said all that, I'm expecting to see interesting things from hybrid approaches.

agreed, the BVH traversal hardware appears to be successful in speeding things up and it's doubtful we'd be able to get to 1080p30/60 without it.

I'm not the best study in the world, but the usage of voxels vs BVH seem very similar in function.

 

[manux]

Wouldn't it be possible to simulate voxelization with dxr if one used boxes instead of finely meshed geometry for scene? unoptimal of course, but it would lead to correct result/approximation?

 

[Shifty Geezer]

I expect the problems RT will face will be very much the same but likely worse problems as SVOGI, or similar techniques. If you consider the memory access patterns of an SVO 'ray' compared to an RT ray, then very rarely would the RT ray win out in terms of the amount of data or computation it would have to read/perform to generate a result.
This is the natural disadvantage of an algorithm like ray tracing where you have to get a precise result.

In my mind voxelization / SVO / SDF / cone tracing etc is fundamentally not that different from RT in how the data is represented or iterated, it's just *much* a simpler representation at the leaf nodes - you don't need to load and iterate lists of triangles, reevaluate / read complex materials and lighting etc, you can often sample at higher mips to approximate cones that represent more diffused light, etc.

Having said all that, I'm expecting to see interesting things from hybrid approaches.

Yeah, it will be interesting to see how compromised the RT solutions are if present on mid-level GPUs in next-gen consoles. Things like reflections can be rendered pretty blurry and still look great for the visual impact like BF's flames, but low sampling of shadows could easily generate issues.

An interesting point about why SVOGI hasn't been more widely used is because it's very demanding, but then so too is raytracing. Just read about nVidia's VXGI, integrated into Unreal Engine. Here's a video of realtime GI in Unreal Engine on a 970 GTX, which is apparently ~4TFs

How well does it perform with a 2080 class GPU? If RTX wasn't released, would we be looking at demos using volumetric tracing and how different would they be in terms of quality and performance?

Edit: another more traditional one:

Edit: And a character: