Unreal Engine 5, [UE5 Developer Availability 2022-04-05]

I think the 'inspired by' part is lighting and SDFs. Dreams has nothing in common with the triangulated assets of Quixel which are the massive datasets being drawn here.

I think it goes way beyond that. One of the main issues Dreams was tackling on the rendering side was storing geometry as a high resolution volumetric representation and how to polygonize it with as little artifacting as possible, being essentially LOD-less, and being prefilterable.

I feel that Epic did the same thing, maybe just implemented differently.
 
Dreams represents the geometry as SDFs. Unreal doesn't and uses lots of triangles. If they used SDFs, they wouldn't need god-tier storage speeds, would they? ;) Epic have literally told us it's the transfer speed (and whole IO stack performance) that enables the data to be streamed, rather than saying that they are using clever representations to compress the geometry detail down into tiny amounts.
 
If they're actually virtualizing the 3D textures - i.e. analog to virtual texturing -, they'd still want storage to be as fast as possible to stream in the required slices (say, drilling further down an octree). As a bonus you get much more responsible use of memory :D
 
Yes, but it'll be clever triangle structures and not abstract mathematical representations. They're still dealing with vertex data and aren't streaming some non-triangle representations.

That said, would something like an SDF representation allow an alternative, more efficient grouping than a basic volumetric subdivide? That could allow a low-RAM geometry approximation for data selection, and it's already present for Lumen.
 
That said, would something like an SDF representation allow an alternative, more efficient grouping than a basic volumetric subdivide? That could allow a low-RAM geometry approximation for data selection, and it's already present for Lumen.

If you store the representation as a tree (like an octree), you could theoretically pull from storage just as deep of a representation as you'll need based on distance from the camera, effectively giving you geometry that has its density linked to object on-screen-size. That would be one way to guarantee you have only one triangle per pixel.

I was watching the DF analysis of the UE5 Demo and, at one point, they note what it seems to them as a normal popping in for one of the statues, IIRC, even though Epic said they used normals just for small detail surface scratches and stuff. If they are doing something like that, using a SDF-like representation as an intermediate and then doing compute to polygonize them at run time, it might very well be that the "normal pop in" was actually the game streaming in a deeper lever of geometry representation for the statue.
 
If they are doing something like that, using a SDF-like representation as an intermediate and then doing compute to polygonize them at run time
That's very unlikely.
SDF is volume data, and usually takes more memory than surface data like a mesh. It's also difficult to associate volume data with 2D texture, and a simple 3D texture would need even more storage.
Extracting iso surface from volume data is pretty fast, but it still has a cost. If you work with triangles, your data is very likely just that - aside some alternative for sinele pixel draws like a point hierarchy.

In the debug view of the video we see there are discrete pops of triangle clusters. Many of them. I assume they have clusters of triangles at multiple discrete lods.
Likely UV seams are preserved. Quixel models have few islands of UV patches. They waste a lot of texture space but they have a low count of UV boundary edges.
Preserving them avoids the problem of requiring new UVs and texture as the model changes. It works well for models of low genus, and quixel models are mostly genus 0, nature prefers this in general.
Some pages above i posted my theory each triangle cluster would require 4 modes of triangulation to avoid cracks at cluster boundaries. (I've already forgotten how it works, but if it does this is really simple and can be precomputed with little space for permutations to store the modes.)

Up to this point the system is compatible with raster HW, and it scales to either HW or production limits where micro polygons are not welcome. There are pop ups, but TAA will hide them very well.

After that the revolutionary stuff of 'single pixel triangles'. I assume that's a hierarchy of points and not much different to Lidar point cloud rendering, the splatting mentioned in the Dreams paper, the splatting of high poly model beating the HW rasterizer in the Many LoDs paper from 2010, etc.
Pretty simple in comparison to the above. No textures are necessary - just points with material, and streaming can ignore the lower levels of the hierarchy at some distance.

Remaining questions:
Do they also implement hidden surface removal? With compute rendering they could by coarse front to back draw. They could render small tiles of screen and terminate if they are filled and further data is guaranteed to be behind the current depth.
So it's likely they do it. But that's not compatible with HW rasterized triangle clusters. So, two options:
1. Raster HW triangles first, get z buffer and start the splatting pass. Eventually divide this process into multiple ranges of depth.
2. Raster triangle clusters in compute (eventually along with the points), but only for depth and HSR result. If they are potentially visible, append them to a later draw call to render them with HW and full texturing.

Second option is unlikely because deferred texturing method would make more sense. And they claimed they do still use the HW rasterizer, so i guess it's somewhat option one.
It's cool to see HSR becomes topic again. It was the holy grail of realtime rendering, then GPUs came up and we silently ignored it for most. ;)
If they do it, the rendering has to follow coarse front to back order. This eliminates simple unordered splatting, and we come close to something like Unlimited Detail. Too bad Bruce Dell left the forum. I guess he was quite shocked too after seeing this :)
 
It works well for models of low genus, and quixel models are mostly genus 0, nature prefers this in general.
Requires correction. A tree is also genus 0, but needs many UV boundaries.
So it would be interesting to see how far Epics triangle reduction can go. If UV boundaries are preserved, it could not limit a tree to a small number of polygons. Only the points representation could reduce it further.
This would be a minor flaw. A forest would require some level of base performance that would not scale down to current gen or mobile eventually. The only option would be to remove tree models at some distance.
 
hmmm... i just realize: They have SDF per model. They could use this to precalculate hidden surface from intersections, and store a bitvector per instance to get rid of overdraw cause by this.
E.g. if you place lots of brick and stone models over a wall (or terrain) to add details, you can erase all the invisible surface of those.
This takes some storage and memory, but reduces culling work a lot, and HSR becomes less necessary.

I do this too. For quixel models it's easy. But for a model like Sponza or Sibenik it's hard. The problem with such models is they have no robust information of inside / outside. E.g. if you place a column on ground, inside the column you see the frontside of the ground but the backsides of the column. Even robust solid voxelization based on solid angle integration creates some bubbles of empty space at least. I managed to voxelize both models robustly without a need for manual work using some heuristics, but in few cases user interaction might be necessary for models of very high complexity.
Solid voxelization is also very slow. I think my algorithm is super fast, but for 20 models in 2K^3 it's still half an hour. I can not claim 'import from ZBrush and display immediately' because of things like this.

But if all quixel models are manifold, you can use a very fast voxelization algorithm based on RT or rasterization. So i think that's an option for Epic. After they culled the intersections, they scale the SDF down for use in Lumen.
Would be interesting to know how their support is for complex models of architecture (if this makes sense for Nanite at all).
 
Here are some tidbits apparently from the latest Edge Magazine

The tech goes for beyond backface culling (which detects with polygons are facing away from a view, and doesn't draw them, saving on processing power).
"It's in the form of textures," Karis explains. It's actually like, what are the texels of the texture that are actually landing on pixels in your view? So, it's in the frustum......It's a very accurate algorithm, because when you're asking for it, it's requesting it. But because it's in that sort of paradigm, that means as soon as you request it, we need to get that data in very quickly."
- Brian Karis (Nanite Inventor)
 
Here are some tidbits apparently from the latest Edge Magazine

The tech goes for beyond backface culling (which detects with polygons are facing away from a view, and doesn't draw them, saving on processing power).
"It's in the form of textures," Karis explains. It's actually like, what are the texels of the texture that are actually landing on pixels in your view? So, it's in the frustum......It's a very accurate algorithm, because when you're asking for it, it's requesting it. But because it's in that sort of paradigm, that means as soon as you request it, we need to get that data in very quickly."
- Brian Karis (Nanite Inventor)

Another tidbit
"We don't even need high throughput for the source of this data, just low latency and sufficient bandwidth for the most part."
 
Yeah, that's what we'd expect from tiled/virtual assets.

@Slifer : Does the article mention how the data is represented?

That quote is the extent of any technical content. "It's in the form of textures" made me think they're using geometry images, but I thought that had already been ruled out?
 
Last edited:
That quote is the extent of any technical content. "It's in the form of textures" made me think they're using geometry images, but I thought that had already been ruled out?
Oh, I overlooked that. So yeah, geometry represented as texture data. Tileable and streamable and compact. Should be very usable on any low-latency storage.

This does mean the raw geometry is going to need to be baked into the texture format, so there'll be a baking process. I wonder how slow that is?
 
Oh, I overlooked that. So yeah, geometry represented as texture data. Tileable and streamable and compact. Should be very usable on any low-latency storage.

This does mean the raw geometry is going to need to be baked into the texture format, so there'll be a baking process. I wonder how slow that is?

The of couple geometry image papers I looked at (and mostly went over my non-techie head) didn't have any info about baking times, sadly. I'd assume it's not too much of a disruption to workflows, given how they're talking about it. They could in principle pre-nanitify* the Quixel library as well.

*pretty sure this a real word.
 
Last edited:
Read the issue not much detail than what was already posted above but we finally have a reason why the tech was demoed on PS5. Epic and Sony were in very early talks and they had early access to PS5 devkits. They had a far earlier collaboration with Sony than Microsoft and that was why they decided to show it on PS5. They also discussed with Microsoft about adding an SSD in XSX.

I'm also getting the impression that the first UE5 games that will use nanite won't appear until 2 to 3 years into next gen. This part is just my opinion.
 
Last edited:
Back
Top