Unreal Engine 5, [UE5 Developer Availability 2022-04-05]

[DavidGraham]

I think GI use software raytracing but higher quality with mesh SDF tracing and reflections only used HW-RT. Settings for console is Epic for Matrix Demo.

No, Matrix demo uses HW-RT only, and it uses it for all, whether GI, AO, reflections, and area lights shadows, no SW-RT whatsoever.

In general, the graphics options for UE5 titles will be as follows: Either SW-RT or HW-RT.

SW-RT:
High: 1/16 rays per pixel for generic GI/diffuse SS reflections (this is the setting Epic intends for 60fps gaming on consoles)
Epic: 1/4 rays per pixel for generic GI/diffuse SS reflections

HW-RT:
High: 1/16 rays per pixel for accurate GI/accurate reflections/AO/Shadows
Epic: 1/4 rays per pixel for accurate GI/accurate reflections/AO/Shadows
Arch-Viz: 1 ray per pixel for accurate GI/accurate reflections/AO/Shadows

An interesting note: Arch-Viz quality setting costs the same on the 2080Ti, as Epic setting costs the PS5 (both cost ~11ms, despite the 2080Ti operating at 4 times the rays per pixel count).

 

[Dampf]

This is really confusing.

So the High scalability mode continues to exist with HW-RT? I thought it was going to be replaced with SW-RT.

Anyway, yes I did notice both modes perform roughly the same if RT is optimized correctly. And yes, HW-RT can look vastly better. So much so, that SW-RT definately qualifies being medium settings in a future game.

Why use software RT on the consoles then? In my testing, high scalability mode with HW-RT already comes pretty close to that 60 fps target and there's again no performance difference between SW and HW, and I am running weaker hardware than the big consoles.

Epic said SW-RT is running faster in their presentation, so I assumed with real games SW and HW-RT performing equal will no longer be the case (maybe because of overlapping meshes) which is why the consoles might be using SW-RT instead. I have no idea.

Also there's one other very confusing thing: How will games handle the differences in visual fidelity between SW and HW RT?

Imagine benchmarking a future game. Max settings on a 2070 Super, Max settings on a 5700XT. The 5700XT would have completely different graphics to render than the 2070 Super, yet performing similar (if HW-RT continues to perform similar like it is now). Basically, max settings would NOT be max settings here. Likewise, if a game has a ton of overlapping meshes, the 2070 Super would run far slower than a 5700XT.

How are game developers going to tackle this? IMO the most logical thing to do is to lock High and Epic settings to HW-RT, SW-Lumen being medium and Lumen off being low. That also makes sense to how the games actually look. The consoles in 60 FPS mode would actually run medium settings then (High scalability with Software RT)

 

[DavidGraham]

Why use software RT on the consoles then?

Who said they will do this? The Matrix demo already uses HW-RT alone.

So the High scalability mode continues to exist with HW-RT?

Yeah.

there's again no performance difference between SW and HW

There is no performance difference because the SW path completely disables many effects: like reflections, dynamic GI, huge range GI, off screen AO .. etc, it's simplified RT as @Shifty Geezer stated, if you enabled all of these effects, the SW path will crumble to death, and be infinitely slower. The fact that HW gives the same speed as SW while also enabling these complicated effects is a testament to it's speed already.

Also there's one other very confusing thing: How will games handle the differences in visual fidelity between SW and HW RT?

HW RT will be branded it's own options as "Ray Traced Reflections/GI/Shadows", while SW RT will be rolled into the usual rasterization settings for Lighting: Ultra, High, Medium .. etc.
When Days Gone deployed SSGI, it put it under the Ultra Lighting option, Gears 5 put their SSGI setting under the Insane AO/Reflections/GI options. Since UE5 titles will use the SW RT as the fallback, baseline model, it will be put under Low, High, Epic settings for Lighting, while HW RT effects will have their separate options for Ray Tracing, just like Cyberpunk, or any current RT game really.

Imagine benchmarking a future game. Max settings on a 2070 Super, Max settings on a 5700XT.

Both will have Low, High, Epic presets for SW RT, the 2070 will have extra Ray Tracing options for HW RT effects, those options will be greyed out on the 5700XT or be not available at all.

The consoles in 60 FPS mode would actually run medium settings then (High scalability with Software RT)

Indeed.

 

[chris1515]

No, Matrix demo uses HW-RT only, and it uses it for all, whether GI, AO, reflections, and area lights shadows, no SW-RT whatsoever.

In general, the graphics options for UE5 titles will be as follows: Either SW-RT or HW-RT.

SW-RT:
High: 1/16 rays per pixel for generic GI/diffuse SS reflections (this is the setting Epic intends for 60fps gaming on consoles)
Epic: 1/4 rays per pixel for generic GI/diffuse SS reflections

HW-RT:
High: 1/16 rays per pixel for accurate GI/accurate reflections/AO/Shadows
Epic: 1/4 rays per pixel for accurate GI/accurate reflections/AO/Shadows
Arch-Viz: 1 ray per pixel for accurate GI/accurate reflections/AO/Shadows

An interesting note: Arch-Viz quality setting costs the same on the 2080Ti, as Epic setting costs the PS5 (both cost ~11ms, despite the 2080Ti operating at 4 times the rays per pixel count).

From the notes of the page 7 of the Matrix Awaken advanced realtime rendering SIGGRAPH presentation

http://advances.realtimerendering.com/s2022/SIGGRAPH2022-Advances-RayTracingOpenWorlds-NetzelCosta.pptx

The second system is shadows.
We support using fully ray-traced based techniques to simulate area lights.
In the Matrix Awakens, they were only used during cutscenes where we required the highest shadow quality.

EDIT:

http://advances.realtimerendering.com/s2022/SIGGRAPH2022-Advances-Lumen-Wright%20et%20al.pptx

Page 106 note

The Matrix Awakens: An Unreal Engine Experience presented some difficult challenges for the ray-tracing model. The open world demanded operating with a large instance count, which could easily overwhelm our top-level acceleration structure rebuild time. The sheer number of active cars and pedestrians in the world required a large number of dynamic refits to our bottom-level acceleration structures. Car paint and glass materials would not be realistic without mirror reflections.
The demo’s target shipping platform was Xbox Series X/S and PS5. These machines have native ray tracing support, but with less overall computational power and slower traversal than what is available on high-end PC systems. However, console APIs offer greater flexibility in the ray-tracing pipeline, which we could leverage to our advantage. For instance, construction of acceleration structures for static meshes can be pre-built and streamed, significantly reducing frame-time dedicated to building and refitting bottom-level acceleration structures. For more details on console-specific optimizations, we refer again to Aleksander and Tiago's talk.

EDIT2: Page 192 notes

Epic settings have some Mesh SDF tracing

All of the measurements shown here were captured at 1080p resolution, with Temporal Super Resolution outputting at 4k. We find that this gives much better final image quality than if we had run Lumen natively at 4k, with really low quality settings.
Our ‘High’ settings are targeting 60 frames per second, and our ‘Epic’ settings are targeting 30 frames per second. The ‘High’ settings don’t have any Mesh SDF tracing, they’re only using Global SDF tracing, the fastest method, and the ‘Epic’ settings have four times more rays per pixel.

Page 195 doesn't mean they utilise HW-RT for everything just they use it for mirror reflections on near field and far field. This is a city they need it.

This is the Lumen pipeline



EDIT: Maybe HW-RT is used on the Matrix Awakens for GI.

 

[DavidGraham]

EDIT: Maybe HW-RT is used on the Matrix Awakens for GI.

Come on man, you are all over the place, Epic clearly stated on multiple occasions, that the Matrix demo on PCs and consoles uses HW-RT for GI, reflections, AO and shadows for area lights. People have switched off HW-RT GI on PC and used the SW-RT with far worse results. See DF's cover on that.

 

[chris1515]

Come on man, you are all over the place, Epic clearly stated on multiple occasions, that the Matrix demo on PCs and consoles uses HW-RT for GI, reflections, AO and shadows for area lights. People have switched off HW-RT GI on PC and used the SW-RT with far worse results. See DF's cover on that.

Again they said directly presenting to dev they don't use raytraced shadows out of cutscene. This is not me who write it this is Epic Games. We can ask to @Andrew Lauritzen

We support using fully ray-traced based techniques to simulate area lights.
In the Matrix Awakens, they were only used during cutscenes where we required the highest shadow quality.

 

[Nesh]

From the notes of the page 7 of the Matrix Awaken advanced realtime rendering SIGGRAPH presentation

http://advances.realtimerendering.com/s2022/SIGGRAPH2022-Advances-RayTracingOpenWorlds-NetzelCosta.pptx

View attachment 7081



EDIT:

http://advances.realtimerendering.com/s2022/SIGGRAPH2022-Advances-Lumen-Wright%20et%20al.pptx

Page 106 note

View attachment 7082



EDIT2: Page 192 notes

Epic settings have some Mesh SDF tracing





View attachment 7083

Page 195 doesn't mean they utilise HW-RT for everything just they use it for mirror reflections on near field and far field. This is a city they need it.

This is the Lumen pipeline
View attachment 7084


EDIT: Maybe HW-RT is used on the Matrix Awakens for GI.

Thats the beauty I love about consoles. Even though they are weaker compared to high end PCs, developers can squeeze out more from the given hardware than they can on a PC

 

[chris1515]

Come on man, you are all over the place, Epic clearly stated on multiple occasions, that the Matrix demo on PCs and consoles uses HW-RT for GI, reflections, AO and shadows for area lights. People have switched off HW-RT GI on PC and used the SW-RT with far worse results. See DF's cover on that.

Again not shadows and funny he said it in DG video diffuse GI and specular reflections.

 

[Dictator]

Again not shadows and funny he said it in DG video diffuse GI and specular reflections.

I really do not think they would be using software lumen for diffuse on console mixed with HWRT (for the Matrix demo specifically). For one, that means maintaining the SDF/Global SDF AND the BVH (what a waste of resources!). Secondly, they most definitely did not use SDF for Diffuse on console as the Night Time view in the city actually looked good. With Software RT, the Night Time view does not work really at all because the software solution is not good with emissive surfaces. See here in my video:

 

[chris1515]

I really do not think they would be using software lumen for diffuse on console mixed with HWRT (for the Matrix demo specifically). For one, that means maintaining the SDF/Global SDF AND the BVH (what a waste of resources!). Secondly, they most definitely did not use SDF for Diffuse on console as the Night Time view in the city actually looked good. With Software RT, the Night Time view does not work really at all because the software solution is not good with emissive surfaces. See here in my video:

I just speak about raytraced shadows. I wasn't sure for GI but I view your video again. Out of overlapping terrain assets or 60 fps, they will use HW-RT.

 

[DavidGraham]

Again not shadows and funny he said it in DG video diffuse GI and specular reflections.

OK, I get it, RT shadows only for cut scenes. Still the big mistake in your posts was the SW-RT for GI, which is just wrong, it's HW-RT for GI and Reflections.

 

[Shifty Geezer]

I really do not think they would be using software lumen for diffuse on console mixed with HWRT (for the Matrix demo specifically). For one, that means maintaining the SDF/Global SDF AND the BVH (what a waste of resources!). Secondly, they most definitely did not use SDF for Diffuse on console as the Night Time view in the city actually looked good. With Software RT, the Night Time view does not work really at all because the software solution is not good with emissive surfaces. See here in my video:

Just fired up the Matrix Demo on PS5. I can't find a night-time option.

Edit: googled it, it's an Easter Egg.

 

[Dictator]

Just fired up the Matrix Demo on PS5. I can't find a night-time option.

Edit: googled it, it's an Easter Egg.

Green signs in the Map I think!

 

[Andrew Lauritzen]

Again they said directly presenting to dev they don't use raytraced shadows out of cutscene. This is not me who write it this is Epic Games. We can ask to @Andrew Lauritzen

RT shadows as in the engine today are unrelated to Lumen. RT shadows were indeed only used for the large area lights in the cutscenes (as shadow map methods can't really handle that kind of thing), whereas virtual shadow maps were used for the main directional light.

And yes the Matrix demo on consoles used HW-RT for Lumen and targeted 30fps.

Each game is going to have the find the best tradeoffs for the content and target frame rate/quality level. For Matrix I think it made sense to aim for the really cinematic settings and there was a lot of content to show off mirror-like reflections and so on. For other projects, probably not so much. I don't imagine either will be clearly the better solution for all cases in the medium term as there are too many tradeoffs that vary in terms of costs maintaining the various data structures and so on.

I don't think it's fair to say that SW-RT "lacks features" either. It obviously also implements reflections and indirect visibility and all of the other things, it's just the data structure it samples to resolve the rays has different tradeoffs. It's generally very appropriate for blurry/diffuse type effects, and less appropriate for sharp reflections and so on. RT however has its own overheads both in terms of tracing and - often ignored but probably even more important - building and maintaining the BVH's. The latter is normally the actual limitation rather than the tracing performance in my experience. It's also totally fair to note that HW-RT path often "skips" many things in the name of performance as well, including significant amounts of deformation (vertex animation, skinning, etc.), alpha testing/translucency and far field geometry. There are many considerations to both methods beyond just a single toggle on/off.

Very little of this is unique to Unreal Engine as well. In cases where you find things with significantly higher performance/quality/whatever axis, they will be making tradeoffs in the other axes. That is the nature of the beast; at least for this generation games will have to make a series of content-specific tradeoffs in these areas.

[Edit] Also no one is "getting rid of any modes". The slides there are just talking about the default scalability options. Most developers can and do change those to suit their title and platforms. They are just general guidelines to start from. If you want to make a 60fps HW-RT game and are willing to make whatever other sacrifices are necessary to get to that performance level, nothing is stopping you.

 

[Andrew Lauritzen]

then your RT cores / Ray accelerators are basically idling and wasting die space which is not something such a modern engine should do

I want to push back against this point in general. During basically no period in GPU history has every unit on the GPU been active all the time, not even close. But even more than in the past, GPUs are designed to not have "all units" active at the same time - they literally cannot provide the power/heat dissipation to do that. Even in RT heavy games the RT acceleration hardware is probably only active for ~10-20% or so of the frame. Is it being "wasted" the rest of the time? Of course not, that's just not how computer hardware works in practice for a long while.

Similarly is using RT and compute "wasting" the rasterization and ROP hardware? Where's the indignation at the double precision and transcendental hardware that "goes to waste" on most cycles in games? What about all the media and display hardware that games never touch? What about the fact that every single shader/wavefront is going to be bottlenecked in various places on some piece of hardware and never get near 100% usage on most of the machine?

I don't mean to pick on this too much, but I highly disagree with the entire characterization that comes with this comment. I get that you're into HW-RT; most people in the industry are too. We would love if we could use it as broadly as possible as it makes a lot of things a lot easier. The reality is - like everything else - it's not a silver bullet. It's a tool that is useful in a variety of situations, but with its own tradeoffs and caveats that may not be immediately obvious to the end users if we do our jobs right, but have to be called out to developers using the engine, which is what the talk you are referencing is aimed at of course.

 

[Andrew Lauritzen]

I think the big question is why software RT is faster than hardware RT. That's extremely counterintuitive and defeats the purpose of implementing acceleration in hardware!

With respect, I think you guys need to read more details about how the various paths work if you want to engage in this discussion. It's not a software implementation of triangle RT, it uses completely different data structures with different tradeoffs. Broad statements about how this or that default or decision is because someone hates either of the paths sounds completely ridiculous TBH. Every game has to make tradeoffs based on content and targets and so on. Unreal provides all of these different paths to allow each game to do what best suits it. IMO Unreal's HW-RT and SW-RT paths are both top tier if not best in class across the field at what they do.

For HW-RT the tradeoffs are less to do with Unreal itself and more to do with the content, as BVH building and such is an area where games have no control (on PC at least) and all play by the same fundamental rules. As I noted in the previous reply, games that lean more heavily on HW-RT need to make other compromises, generally around having a more static world, relying even more heavily on reprojection or irradiance caching and avoiding sharp reflections and such in general. For purely diffuse GI the quality differences between distance field and triangle RT based solutions are actually not entirely obvious or better one way or another. If you are doing mostly diffuse visibility queries, distance fields are fundamentally more appropriate than stochastic triangle RT + denoising.

 

[Dampf]

...

I understand, Andrew, these are all good points. But to me as an enduser, I don't see any tradeoffs with the superb Raytraced Global Illumination used in Metro Exodus Enhanced Edition compared to Lumen, which also uses Triangle based Raytracing. In terms of quality they look(!) pretty much equal and both offer multi bounce GI. Yet, the RTGI in Metro seems to be way faster than Lumen, running easily at 60 FPS on consoles. I am aware the game has completely different performance targets as it has much lower quality geometry, however the RTGI is still fast even compared to the rasterized GI they used in the normal Metro Exodus. If I were Epic, I would really try to get in touch with 4A Games, maybe there is something you guys are missing (though again I'm only a layman here).

So that's why I am wondering why Epic cannot achieve similar stuff for the HW-RT path in Lumen and why Epic chooses to ditch Hardware RT for the mode that everyone will use on console ( 60 fps mode). You are right that no part of silicon is always used every time, but there's a difference between RT accelerators being active 5% of every frame or 0% of every frame. For example, Housemarque Studios in Returnal uses the additional Raytracing hardware in PS5 to speed up some GI calculations, which is also a clever way of making use of the hardware at hand.

 

[Shifty Geezer]

With respect, I think you guys need to read more details about how the various paths work if you want to engage in this discussion.

I disagree. The discussion educated me as to why. If I want to learn by going off and reading, I shouldn't/wouldn't be posting here, same for everyone else. For me, the discussion board exists to inform and educate in a sociable learning structure. That's why I ask questions, don't pass judgement, and respect all takes including wrong ones and biased ones as I fill in my understanding. If I were to be into RT for a living as opposed to a passing interest, I'd be very well read and posting here on the other side of the discussion! Except no-one would be asking me how it works, or presenting wrong ideas to be corrected in conversation, because they'd all be as well read as me!

 

[Karamazov]

I understand, Andrew, these are all good points. But to me as an enduser, I don't see any tradeoffs with the superb Raytraced Global Illumination used in Metro Exodus Enhanced Edition compared to Lumen, which also uses Triangle based Raytracing. In terms of quality they look(!) pretty much equal and both offer multi bounce GI. Yet, the RTGI in Metro seems to be way faster than Lumen, running easily at 60 FPS on consoles. I am aware the game has completely different performance targets as it has much lower quality geometry, however the RTGI is still fast even compared to the rasterized GI they used in the normal Metro Exodus. If I were Epic, I would really try to get in touch with 4A Games, maybe there is something you guys are missing (though again I'm only a layman here).

So that's why I am wondering why Epic cannot achieve similar stuff for the HW-RT path in Lumen and why Epic chooses to ditch Hardware RT for the mode that everyone will use on console ( 60 fps mode). You are right that no part of silicon is always used every time, but there's a difference between RT accelerators being active 5% of every frame or 0% of every frame. For example, Housemarque Studios in Returnal uses the additional Raytracing hardware in PS5 to speed up some GI calculations, which is also a clever way of making use of the hardware at hand.

Maybe because metro's GI is combined with last gen graphics and matrix demo has to deal with nanite at the same Time as lumen ?

 

[Dampf]

Maybe because metro's GI is combined with last gen graphics and matrix demo has to deal with nanite at the same Time as lumen ?

Yes, that'd be the most straightforward answer. But AFAIK Lumen traces not against the full polygon models of Nanite, but uses low poly meshes instead.

So IDK, the difference in polygon count between Metro and Nanite enabled games should not be that drastic with this technique.