Unreal Engine 5, [UE5 Developer Availability 2022-04-05]
- Thread starter mpg1
- Start date
Tips and Tricks: Ray Tracing Best Practices | NVIDIA Technical Blog
This post presents best practices for implementing ray tracing in games and other real-time graphics applications. We present these as briefly as possible to help you quickly find key ideas.
developer.nvidia.com
Well worth a gander, best practices doesn't necessarily mean must, but it's decent insight into how nvidia thinks it should be optimized.
And their updated blog post here 1 year after:
Best Practices: Using NVIDIA RTX Ray Tracing (Updated) | NVIDIA Technical Blog
This post gathers best practices based on our experiences so far on using NVIDIA RTX ray tracing in games.
developer.nvidia.com
I do appreciate that in the updated blog they went ahead and addressed the jargon, felt like I was reading a telecom document.
For clarity, here are common abbreviations and short terms used in this post:
- AS—Acceleration structure
- TLAS—Top-level acceleration structure
- BLAS—Bottom-level acceleration structure
- AABB—Axis-aligned bounding box
- Instance—An instance of a BLAS in a TLAS
- Geometry—A geometry in a BLAS
Last edited:
Afaik, 'continuous LOD' should mean a smooth LOD transition, e.g. like we have with tessellation shader with tessellation controlled by a floating point number, and results change gradually.
But that's not what i mean. Lacking a better term, i misuse the word for any technique which can change detail on local sections of a mesh, e.g. Nanite, or the traditional methods used for height map terrain.
'Discrete LOD' seems well defined, meaning to have multiple detail levels per object. It thus works only for geoemtry which we can divided into multiple objects (modular architecture) or ofc. single objects (e.g. characters).
If anybody can define better semantics, i'm all ears. I think misusing continuous LOD is fine becasue truly gradual changes really are not needed or even desired.
NVs stochastic LOD is very limited. I barely remember the reason, but i guess you'd need to select the level when generating the ray, although you do not know yet at what distance it hits what.
For a general solution, traversal shaders like Intel proposes are requiered.
But even then, the cost of increased data divergence from traversing different BVH by otherwise almost equal rays is probably just too high. So this only becoems really attractive once we have HW ray reordering during traversal.
And even then, modifying just geometry would always be the better and faster solution.
So there is no way around exposing BVH data structures to programmers, or even having a BVH API for compute shaders. Something like that. It may take a decade, but only then we have options to implement proper continuous LOD for ray tracing.
On consoles it should be possible even now, but Epics BVH is not really optimized for RT. They would eventually need to change their data, and spend a lot of effort on console only solutions on top. I guess it's not worth it, even their engine is used by so many games. For other companies, chances are even lower to invest here i think.
LOD was unsolved for so many decades. Seems we just have to accept one more, because the guys wich brought us HW RT did not really spend a lot of thought on it. Making trivial offline legacy methods realtime was their only idea. But at least we have 2000$ 2kg GPUs now. How great. : (
Huh? AMD has both already. If they wanted their matrix cores in consumer GPUs they would be there.
Seems they spend alot of thought on hw ray tracing though. Now your consoles are behind by a country mile because AMD is late to the game, but eventually they will get there, in-line with what NV and Intel are doing in the RT and ML game.
Frenetic Pony
Veteran
I wonder if the stacking problems of building and maintaining a BVH structure that does everything developers would want it to do has made anyone reconsider looking into point splatting. Brian's recent Nanite presentation on the research he did prior to settling on triangles build a solid case for triangles from a purely rasterization point of view, but for random access tracing relying on IHV's to "fix" all the things needed for BVH build/traversal in a manner that's developer friendly, fast, and widely available seems like a longshot; and trying to do it ones self seems like a headache and perhaps not doable for everything.
While the challenges remaining to employing point splatting universally are still there, the advantages of being relatively simple, programmable, amenable to tracing, and lightning fast all recommend themselves as a different solution to banging heads against BVHs, IHVs, and etc. and trying for breakthroughs. Here, for one example, is a nigh photoreal recreation of part of the T-Rex scene from Jurassic Park, running with traced shadow and ambient occlusion, all runnable (I believe, I guess this is on PS5 but the level might work on a PS4 too) on quite low end hardware:
While the challenges remaining to employing point splatting universally are still there, the advantages of being relatively simple, programmable, amenable to tracing, and lightning fast all recommend themselves as a different solution to banging heads against BVHs, IHVs, and etc. and trying for breakthroughs. Here, for one example, is a nigh photoreal recreation of part of the T-Rex scene from Jurassic Park, running with traced shadow and ambient occlusion, all runnable (I believe, I guess this is on PS5 but the level might work on a PS4 too) on quite low end hardware:
They only made it faster, that's all they did. And you know at which price: 2000$, 450W, and it needs physical crutches to hold its weight. Gravity of heatsink causes a mini black hole. So maybe, after mining is over, now a Dyson Sphere in PC case is the next hobby for PC gamers to afford this. With shiny LEDs on it, ofc.
At least it works for self running gfx demos of RC cars in a diorama and making some retro games more pretty, while the motordriver in CP 'psycho mode' still uses environment probes to illuminate his jacket, afaict.
My jaw dropped so hard i still have to open my mouth intentionally to get my teeth apart.
And i don't have a console, damn! Atari 2600 was my last. But you can send me yours, if you want to get rid of it.
I have tried HZD, GoW and maybe some other Sony exclusives meanwhile. This stuff looks nice if you watch someone else playing, but by doing myself i noticed it's just interactive movies. Refunded. So i can't promise i'd treat your PS5 well.
If you'd offer me both a 4090 and a PS5, that's like choosing between cancer or a wheelchair. But i'd take a dev kit.
Yeah, it's this reason why i still consider it. Maybe clusters of points would be easier to get the texture shading advantage as well. We can cache lighting simply on those points. And i found it can do analytical AA at negligible cost, so no need for TA in screenspace. Maybe we could even turn certain clusters into impostors, to make upscaling and frame interpolation crutches obsolete.
Sadly all those advantegs make a combination with traditional triangles harder. And if i make too exotic stuff, nobody would have serious interst to use this, i guess.
But the largest problem seems: Points only make sense if you have insane detail for *everything*.
It works for a small scene made for a single camera setting, like the dino scene. Environment only exists where needed, like in offline rendering.
Now Dreams shows nice large playable scenes too, using instancing. But i remain doubtful mainly becasue of this detail thing.
The ps5 isnt for me mostly, its for the kids.
I dont think most would need a 4090.
You do have your preferences right?
Cardassian Bob
Newcomer
I hate LOD pop in. Surprisingly one of my favourite UE5 features.
I'm playing Dark Souls 3 pc and sometimes there are 3 or 4 LOD changes within 10m.
You've overstated the accomplishment here. The results are great, but they are far from photoreal and in particular they don't scale well in quality, such as no skin meshing and geometry being largely static. Watch the vid at 6:45 where the T-Rex takes on a Toy-Story feel. Splatting also isn't the answer as here the results in Dreams are achieved by using SDFs instead of dense triangle geometry, the same representation used in Lumen's 'fast' (software) path. These SDFs are still rendered as triangles as well as splats because the original splatting-only strategy didn't work completely.
Regards working on PS4, AFAIK there is no PS5 enhanced Dreams and all Dreams levels run on all hardware, so I'd be surprised if this doesn't run on PS4. It's a fabulous creation, but it's not one in the eye against traditional HWRT. We explored a lot of these options in the debate before HWRT was introduced, with some promising compromises, but they haven't been able to be developed past the inherent problems of accuracy in spatial representation of the objects.
If you're sensual to this, i'd like to know an which kind of objects you notice it, beside foliage.
Personally i rarely notice, at least not anymore with newer games. Or my eyes became just too bad.
I have good reasons for them.
Frenetic Pony
Veteran
Spatial representation is a problem with point splatting. I've seen it used effectively as a means of applying visuals to a higher level geometry abstraction, such as with Unity's new hair system where particles are splatted to curves which track the underlying geometry. I've also seen it used fairly convincingly for distant impostors, especially of aggregate objects like trees. Thanks to the square/cube law the resolution and memory of distant objects works out pretty well. This seems like it has the potential to replace both rasterization and tracing of distant high complexity objects like foliage without worrying about extending a bvh to multiple in game "kilometers" or wondering how triangle collapse is going to ruin foliage representation. One even includes point based representation of distant animated characters
An underlying geometry representation that's able to encode skinned geometry without relying on triangles, and is otherwise able to represent arbitrary resolution, seems useful for tracing purposes thanks to the inherent, and now apparently baked in, difficulties of a bvh. But is also an open research question, needing a shipping product and having time to see if challenges can be efficiently overcome at all are two different asks. That being said I'd say the dreams video is quite true to the movie for being a game, other than being too dark (tried to adjust the movie example to fit but), and more importantly far more detailed than just about any modern RT representation of the same scene. Even if points have their limitations currently, they seem to have advantages for RT scene representation.
Attachments
D
Deleted member 2197
Guest
Did not see this UE 5 video posted on July 22.
Frenetic Pony
Veteran
Really though, while detail is neat the major downside of hardware RT for the next, 7+ years (given console cycle/how long switches take) isn't the RT scene representation lacking detail. The high frequency contribution, even from reflections, on any reasonable (not artists making windows look like mirrors to show off RT exists at all) is still such a low overall contribution to scene detail that low end representations aren't that noticeable.
What is noticeable is the need for extreme temporal re-use of one kind or another, which leads to extreme undersampling and/or ghosting of moving objects. Moving objects isn't a new challenge for temporal integration, TAA has trouble there as well. But it get worse when piling TAA on top of other re-use. Other than tracing distant objects the advantage of point splatting is that it moves at all. Which means it works with foliage. Multi type/hybrid representations are new, but experiments show they're doable and can even offer speedups. While that report experiments with SDFs, it seems like point splatted foliage could be references as an AABB, either in an RT BVH and maybe in some SDF fashion as well, wherein upon hit the ray query would switch to tracing a point splatting representation of foliage. But then how would you handle overlapping types of geometry, splitting between point splatting and going down the tree to some triangle mesh sitting in the, uhm, tree. Hmm...
If it could be done though this could provide foliage tracing as fast or faster than an SDF query while providing animation without the complication of alpha tested RT or rebuilding/refitting every frame because artists want the wind to be blowing. Even if point splatting has problems for "doing everything" right now, which it very much does, it might still prove useful in hybrid renderer thanks to its strengths. The other strength of both point splatting and sdf's is they can support cone tracing, while both have leaking issues these might be minimized in short ray/cone length traces of fast moving objects, potentially solving ghosting as well.
What is noticeable is the need for extreme temporal re-use of one kind or another, which leads to extreme undersampling and/or ghosting of moving objects. Moving objects isn't a new challenge for temporal integration, TAA has trouble there as well. But it get worse when piling TAA on top of other re-use. Other than tracing distant objects the advantage of point splatting is that it moves at all. Which means it works with foliage. Multi type/hybrid representations are new, but experiments show they're doable and can even offer speedups. While that report experiments with SDFs, it seems like point splatted foliage could be references as an AABB, either in an RT BVH and maybe in some SDF fashion as well, wherein upon hit the ray query would switch to tracing a point splatting representation of foliage. But then how would you handle overlapping types of geometry, splitting between point splatting and going down the tree to some triangle mesh sitting in the, uhm, tree. Hmm...
If it could be done though this could provide foliage tracing as fast or faster than an SDF query while providing animation without the complication of alpha tested RT or rebuilding/refitting every frame because artists want the wind to be blowing. Even if point splatting has problems for "doing everything" right now, which it very much does, it might still prove useful in hybrid renderer thanks to its strengths. The other strength of both point splatting and sdf's is they can support cone tracing, while both have leaking issues these might be minimized in short ray/cone length traces of fast moving objects, potentially solving ghosting as well.
Last edited:
I remember Quake 1 did just that. : )
Why do you think point sampels would not require something like a full scene bvh? If you want to get RT features, you need to organize them somehow, to have a way to query them spatially.
There's nothing wrong with BVH. It's the most flexible structure because it supports refitting animation, the tree hierarchy can represent LODs, and it works for large scens too.
The problems we now see in HW RT only come from having no access to its most important data sructure, ruling out the LOD advantage, and thus eventually also the support for large, arbitrary scenes.
But the concept otherwise is the same. The short sighted amateurs which designed DXR and RT HW were just unaware (or not interested) about the optional but essential idea to use internal nodes for LOD.
The damage has been done and is now hard, or maybe impossible to fix.
The real advantages of points over triangles are quite unrelated to all this: Rendering a point is constant time. And texture mapping isn't necessarily needed, removing a level of indirection *.
Another difference is points don't encode a connected surface. There are no edges to close the gap to adjaecent points. So there is no need to manage adjacency at all, thus complexity is eventually lower.
But this also becomes the main disadvantage vs. triangles. If we zoom in, it becomes difficult to render a closed surface without artifacts. Which is why it feels restricted to applications of high detail only. : (
The trivial solution would be to use triangles for magnification, but points for minification, which Dreams maybe did. (I still refuse to believe they use SDF for rendering. I think they use it only for modeling.)
But this amplifies complexity, which usually negates the wins.
To render the environment probes of my GI stuff, i came up with a trivial soultion to handle both magnification and minification with splatting, at the cost of image quality. Another reason why i'm still intersted to try it out...
*) Notice: getting rid of texturing is highly desired, for reasns which may not be obvious to anybody:
Texture mapping enforces static topology of the model, talkign about true topological changes like holes closing or openeing as LOD goes up or down.
If such topological changes happen, the mapping to texture space breaks, and we get bad artifacts like seams. Nanite for example can not solve this problem, thus it still isn't a general solution to LOD.
To solve it, we would need new UVs and texture images for sections of the model where topological changes happen.
That's very hard and complex, and the main reason why no automated general LOD solution exists to the current day.
Points, not requiring a bijective mapping to texture space, would compeltely avoid this problem.
Last edited:
Frenetic Pony
Veteran
Sorry, "RT triangle BVH" problems with distance. You'd still need some sort of structure, a BVH without triangles might be fine, because suddenly the problems of point splatting not being able to represent high res surfaces go away if you're representing something far in the distance, as it's representable at very low res and memory without any problems anyway.
Isn't that a BVH LOD? Have a high detail one for close and a distant one where very low representation is adequate? Once you get there, whether you rasterise triangle or point splat doesn't make a difference, seems to me. What I'm hearing is use a coarse SDF representation and trace against that at range, and use your detailed trimesh BVH for up-close details. That concept sounds reasonable, like 'mapmaps' for spatial representation.
Similar threads
- Locked
