RDNA 2 Ray Tracing

Annoyingly, page 222 (230 of the PDF) has an incomplete sentence/paragraph:

I'm going to guess this is merely a reference to "texture" functionality that is not supported such as sampler mode, see the section "Restrictions on image_bvh instructions" on page 81 (89 of the PDF):

• DMASK must be set to 0xf (instruction returns all four DWORDs)
  
• D16 must be set to 0 (16 bit return data is not supported)
  
• R128 must be set to 1 (256 bit T#s are not supported)
  
• UNRM must be set to 1 (only unnormalized coordinates are supported)
  
• DIM must be set to 0 (BVH textures are 1D)
  
• LWE must be set to 0 (LOD warn is not supported)
  
• TFE must be set to 0 (no support for writing out the extra DWORD for the PRT hit status)
  
• SSAMP must be set to 0 (just a placeholder, since samplers are not used by the instruction)

It's notable that the query instruction isn't specialised for BVH or triangle testing, the node provided as the starting point determines whether the result consists of multiple box nodes or a single triangle node. This might imply that a single "texture" contains nodes that are either boxes or triangles. Or it could imply that there's one texture for boxes and another for triangles.

There's support for BVH data that is larger than 32GB, but the count of nodes can only be described by a 42-bit number.

There's support for float16 ray and inverse ray direction vectors, "for performance and power optimization", saving 3 VGPRs. I can imagine console developers being all over that optimisation. Though until someone captures the ray cast shaders, we won't really know much more.

There are some interesting flags:

• Box sorting enable
• Triangle_return_mode
• big_page

I'm struggling to understand why a single ray is producing multiple box hit results. "For box nodes the results contain the 4 pointers of the children boxes in intersection time sorted order." That is the order that the machine produced the results, not necessarily "box sorted". So the zeroth result, when sorted, would be a "first hit". But a first box hit doesn't mean there was a triangle to hit.

So I think this is merely a way to "speed-up" box traversal, by allowing the ray to progress into multiple boxes per query and then to determine which of those boxes should be queried further. Or rather, start to evaluate the boxes, since a triangle hit could be in any of them. I'm struggling to come up with a fast algorithm here, so not sure if it's a speed-up technique or perhaps the 4-way result merely reflects how AMD has structured the boxes...

I can't find anything that describes what happens when there's less than 4 box results. If you start the query one level up from leaf nodes, then you might get less than 4 results?

Triangle return mode either returns a pair of triangle ID and hit status or i/j coordinates. I'm not sure how this relates to DXR's concept of instancing, but I presume the i/j pair is the barycentrics for the triangle. The first pair of floats seem to be "intersection time". I can't find any description of the use that intersection time might be put to.

It seems that if you want both the triangle ID and the barycentrics you have to run the query twice?

I can't find any explanation for the use of the big_page flag (either no override or pages that are >= 64KB in size), but I suppose this could be an optimisation that suits the total size of the BVH.

Box growing amount is an 8-bit description of "number of ULPs to be added during ray-box test, encoded as unsigned integer", so allowing some fuzziness and control of the fuzziness. I've been wondering whether fuzziness is a direct technique to speed-up ray queries, lots of fuzziness at the start of traversal, with reduced fuzziness as the depth increases. I'm not sure how traversal can evaluate depth though, except for a fuzzy depth derived from the count of queries issued for a ray direction. So the overall algorithm would use a single fuzzy ray instead of a group of, say, 32 rays and then increase the ray count with depth and reduced fuzziness.

I suppose it would also allow boxes to implicitly overlap each other. So far I've thought of overlapping boxes as being a design decision for the IHV, in terms of a fixed choice for how the hardware works. BVH data structures are way more complex than I first thought (irregular octree), so I'm lost.

I'm intrigued by the idea of a single traversal shader having access to multiple "BVH" textures concurrently. There's just a base address, so seemingly multiple BVHs could be available.

Along the way I've found this page:

DirectX Raytracing (DXR) Functional Spec | DirectX-Specs (microsoft.github.io)

which is quite detailed "AABB" is the best way to search the page to understand bounding boxes/volumes. BVH isn't really used.

There's a concept of a degenerate AABB, which is worrying!:

I've not heard of bounds being inspectable before. There's nothing in the RDNA 2 documentation which would appear to support the idea of inspecting bounds. It sounds to me like wishful thinking...

Overall, pretty interesting, but less conclusive than I was hoping to find.

 

Excellent stuff!

I had a good look at all of those. The fog in my brain relating to BVH has cleared somewhat.

The Psychopath blog entry has clarified for me that AMD has implemented a BVH4 structure. Earlier I didn't appreciate that it's required to query all four children (or however many are actually present, if less). You could only do less if you knew the bounds before doing the queries.

His article on Light Trees:

https://psychopath.io/post/2020_04_20_light_trees

is interesting to me since it's based upon augmented nodes:

Would it be possible to augment each DXR BVH node? Would that be a CPU process to query the BVH that was built and then create a texture containing the per-node light energies?

He links to techniques such as stochastic light cuts:



which scales very impressively

The paper that describes Coherent Large Packet Traversal and Ordered Ray Stream Traversal is interesting again because of the augmented node data that's required for both. This time the data is quite large and complex, the principle idea being to classify child AABBs into relative orientations with respect to each other and assign that to their parent node.

ORST is particularly interesting because it gathers together diffuse rays by the signs of their direction vectors, e.g. all rays that run from far-north-east to close-south-west are evaluated as a packet.

Can't really tell whether any of these algorithms are practical on RDNA 2.

 

Honestly, I'm wondering what console devs will do with the bare metal.

I'm still struggling to understand how BVH build and update work. I get the impression that it's a split effort, with some kind of "pre-process" on the CPU which then sends the data to the GPU where more work is done to finalise it. I need to spend more time on the functional spec... The way that animation interacts with the BVH is a concern. e.g. a low-flying plane that flies rapidly through what were "empty" volumes which simply aren't present in the BVH before the plane arrived.

I'm assuming that the BVH is linear, as you describe. Since a custom traversal shader can read memory that isn't the BVH, a node augmentation buffer can be created, keyed upon node ID, as you describe.

So the problem then is to augment all of the nodes, which implies reading all of them and extracting their AABBs. On console this really shouldn't be an opaque data structure, at least on PS5.

One thing I've realised is that the settings in the Texture Descriptor (image resource), such as box growing amount, are fixed when the descriptor is created. So you can't vary the box growing amount as traversal progresses, unless several descriptors are associated with a single BVH and traversal issues queries against them separately as needed.

 

It's interesting how that stuff will work with D3D and appears to provide some "close to the metal" functionality. I wasn't really expecting that, but then I've never looked at AGS before.

Lots of reverse engineering still required it seems, since there's no real documentation.

 

I disagree. The explicit return of 4 results per query is specified, and since BVH traversal requires all child nodes to be queried simultaneously and since there is no way for the hardware to request query results for a subset of BVH nodes, it's not possible with the documented instructions to get results for more than 4 children in a parent node.

Clearly, if you wanted to run with two parallel BVHs to get more than 4 children per node, you could, but that would be pretty strange since 4 children is the optimal "power of two" child count for a BVH: you get the smallest count of traversal steps on average. There's a nice graph in the blog post that was linked earlier that shows that 3 is the optimal count of children.

If you wanted to use BVH2, I suppose you could, but you'd be throwing away half of the AABB intersection throughput, since it has been explicitly designed to produce four results per parent node.

I don't understand what you're saying here, since the arguments are specified. The resource descriptor tells the fixed function hardware some other facts about how to process the query, such as whether to sort AABB results.

You could build a quadtree.

I don't know what you mean by BVH4 processing. AMD's been quite explicit that a shader that issues ray queries will make use of shared memory (LDS) when handling BVH traversal. So that could be a pixel shader or a compute shader etc. (if not using DXR 1.0's ray cast shader). The shader takes the results and decides whether to continue with traversal (e.g. ray length limit reached) or query each of the 4 children (or less, in theory, when they are leaves?).

I don't understand how triangles work though. It's almost as if there can only be one triangle per leaf AABB, because if there were multiple transparent triangles inside an AABB a ray could hit an arbitrary subset of them and I can't discern how a query would get past the first one. Unless the shader computes a new ray origin which is a delta-sized step along the ray direction past the intersection point.

In truth it would appear that there are tens if not hundreds of compute cycles available per query: we should now talk about the ALU:query ratio as we once did with ALU:TEX. So computing delta steps along the ray direction isn't so terrible and in theory after the first hit, the remaining triangles will come in clumps from memory for moderately fast delta-queries.

I wouldn't call it experience. Simply that the concept of an octree was getting in the way of my thinking about BVHs.

A refit still has to process all descendents of the top-most node affected by the refit. It doesn't sound trivial to me. How deep are these BVH trees in games? 10? 15?

We sort of have a datapoint for BVH animation cost: in Cyberpunk 2077 the character that you play as is never reflected in ray traced reflections specifically because "it's too expensive". Sounds like something that should have gone into the "Pscho" setting, but that's for another thread.

I don't know if the game shows the vehicle being driven by the player in ray traced reflections. That's theoretically a much lower-cost BVH animation update...

Yes it seems that animation highly favours BVH, though BVH presumably slows down progressively as updates for animation are repeatedly applied.

DXR's BVH is opaque supposedly because it enables the graphics card companies to "do their own thing".

What will Intel's Xe do?

 

My theory is that NVidia made hardware accelerated ray tracing for the professional market and then went to Microsoft and said "hey this is realtime". That's why you have to run at 540 to 720p to get playable results when all the ray tracing effects are turned on, in games that do more than one ray traced effect simultaneously.