Unreal Engine 5, [UE5 Developer Availability 2022-04-05]

[techuse]

No, but that's a pretty irrelevant point as games aren't pushing anywhere close to that level of geometry and likely won't be until next gen arrives.



But I would argue that's more to do with the asset development side, rather than Nanite.

We've got games that don't use Nanite presenting extreme close up detail, arguably more detailed up close than games using Nanite.

Such as Horizon Forbidden West in places.

View attachment 11958



I'm particularly sensitive to LOD transitions so to me, this is one of the biggest selling points of Nanite/Mesh Shaders.

Are games paying the nanite fee and leaving visual fidelity on the table though? Could they have used higher levels of geometry with little to no performance hit? Is it a performance bottleneck or the fact that the art pipelines are lagging? That part is still unclear to me.

HFW might compare well to available UE5 games, sure. But are these games really pushing geometry over the best we have seen from standard pipelines? I would say no.

 

[davis.anthony]

It's more like he's saying if you design your game around the limitations of the fixed-function units in hardware, you can get really good performance and nanite may be slower in a game designed around those limitations. That seems a bit odd, because nanite is designed to have different tradeoffs.

But games aren't geometry bound because of rendering power of GPU's, they're geometry bound because of development constraints.

If developers were given time to make every asset out of millions of triangles, then Nanite is a no brainer and makes complete sense.

But assets aren't made that geometrically dense, so the advantage of something like Nanite is none existent, and could actually be slower.

And that's the message the guy in the video is trying to put across.

 

[davis.anthony]

Are games paying the nanite fee and leaving visual fidelity on the table though?

Definitely.

Could they have used higher levels of geometry with little to no performance hit?

Given how Nanite works, I would say so.

Is it a performance bottleneck or the fact that the art pipelines are lagging?

Personally, I believe it's the art pipeline lagging behind.

Given rising development costs, will it ever catch up?

HFW might compare well to available UE5 games, sure. But are these games really pushing geometry over the best we have seen from standard pipelines? I would say no.

So then, why use Nanite?

 

[techuse]

Definitely.



Given how Nanite works, I would say so.



Personally, I believe it's the art pipeline lagging behind.

Given rising development costs, will it ever catch up?



So then, why use Nanite?

Assuming no type of hardware limitation, my only guess would be development starting on UE4. Cost wise, shouldn't it be cheaper with nanite? Aren’t assets initially created at a super high fidelity as a reference point and then scaled down to fit the target performance level? I would think that nanite would just allow devs to axe the work required to scale them down.

 

[Scott_Arm]

But games aren't geometry bound because of rendering power of GPU's, they're geometry bound because of development constraints.

If developers were given time to make every asset out of millions of triangles, then Nanite is a no brainer and makes complete sense.

But assets aren't made that geometrically dense, so the advantage of something like Nanite is none existent, and could actually be slower.

And that's the message the guy in the video is trying to put across.

This does not make any sense. Games can't run with per-pixel triangles because performance would be crushed by the fixed-function rasterizer. Triangle size impacts performance. A software rasterizer can beat the hardware units in the case where a triangle covers fewer than ~6 pixels. That's one of the things nanite was trying to do - beat the hardware units. Without Nanite LODs (whether fixed or continuous) basically get generated to keep triangles large enough most of the time so that you don't frequently cave in performance.

Saying games don't need Nanite because they're intentionally designed to work around the limitations of renderers that don't have the benefits of Nanite doesn't really make sense. if you made a UE5 game with Nanite and you didn't scale up your geometry to actually see the benefits, it might be an odd choice. I think more obvious drawbacks of nanite is memory. Even though it has built-in compression, they still can't just make every mesh have five billion vertices and call it a day. The digital foundry talk with Ninja Theory keyed in on this. They had memory limitations to work around.

I get what the guy in the video is trying to say, I just think it's dumb. He took a racing(?) game that's something like 5-10 years old and used that as a test case, and in a contrived way. Yah, you might not need nanite if you don't share its goals. That's the same with anything. Your game might not need GI, or physically-based materials. Your game might not even need colour or 3D graphics. I'm exaggerating, but essentially we're saying Nanite is not needed in games where it's not needed, but it could be needed in the games where it is.

Remedy can decide that they can do enough wish mesh shaders for their visuals and art style. Epic can decide something else and continually improve Nanite support for vegetation, terrain and animated meshes. They've done all kinds of stuff since the UE5 preview that Sebbbi looked at.

Spoiler: Nanite improvements so far

Quick cut and paste from their public roadmap
UE 5.1:
The primary focus of Nanite for 5.1 is the addition of a programmable rasterization framework, opening the door to features such as masked materials, two-sided foliage, pixel depth offset, and world position offset. Please note that the exact feature list and expected stability and performance characteristics are still TBD.

Other updates include:

• Nanite material switch in the Material Editor
• Additional diagnostic and debug modes
• Many quality and performance improvements

UE5.2
Nanite feature and performance updates include:

• Additional feature support: Custom Depth and Stencils, Lighting Channels, and Global Clip Plane
• Variable precision normals—for example, for high-quality reflections on cars
• "Max World Position Offset Displacement" setting, to mitigate artifacts that are common to objects that use World Position Offset with Nanite; use "Visualize -> Out of Bounds Pixels" to see where WPO is getting clamped by the max offset
• The Nanite Streamer, responsible for streaming clusters of geometry data from disk, has been updated for performance, stability, and improved statistics
• Precomputed Nanite displacement mapping via static texture map (Beta)

UE5.3
Explicit Tangents

Nanite now supports explicit tangents in the data format and runtime. Until now, Nanite has relied on a tangent space that is implicitly derived in the material, based on the screen-space positions and UV gradients. This is computationally convenient, makes tangents free in terms of memory and disk footprint and works well in practice for many types of meshes, especially highly tessellated ones. But there have always been cases, especially for low-poly models, where the implicit tangents are too imprecise, and custom per-vertex tangents are needed to reach target quality. To support these cases, users can now opt to store and use the original model tangents on a per-asset basis. Enabling this comes at a modest cost of an additional ~10% memory and disk footprint for the asset.
Nanite Spline Meshes (Experimental)

An initial implementation of SplineMeshComponent for Nanite meshes has been introduced. This feature can be enabled by setting r.Nanite.AllowSplineMeshes=1 in a settings .ini file. WARNING: Enabling this feature currently comes with a performance cost that affects culling performance of all Nanite, and there are currently known issues with incorrect culling under extreme deformation.
Performance improvements for masked materials and PDO

A new sliding window vertex cache for programmable raster results in faster masked materials and PDO. Initial tests show a 20% speed improvement for rasterization of masked foliage.
Nanite Selection Outline

Nanite object selections no longer flicker with TSR/TAA enabled or get occluded by other objects, and they render at final resolution.
Nanite ISM / Foliage Instance Selection

Several issues in the editor were fixed related to selecting/modifying/deleting instances of Instanced Static Mesh or Foliage Actors that had Nanite enabled.
Fallback Target Setting

We've added a 'Fallback Target' setting to Nanite Static Meshes. This is a menu option providing more explicit control over which target to reduce to for the fallback mesh: Relative Error or Percent Triangles.
Overdraw Visualization for Masked Materials

Nanite's overdraw visualization mode now properly accounts for masked materials. A masked material might appear to show more overdraw than in 5.2 due to the more accurate visualization.

UE5.4
Spline meshes are used for deforming static meshes along the shape of a spline, for example to model roads over landscape terrain. Spline meshes remain a significant missing piece as scenes in UE5 trend towards using Nanite for more and more scene content, especially for improved Lumen and Virtual Shadow Map performance.

Support for Nanite spline meshes was released as Experimental in UE 5.3. Remaining work includes performance and memory optimizations, preventing cracks, and fixing up areas such as level streaming and transform caching.

Nanite Compute-Based Shading is a long-term project focused on moving Nanite materials from traditional raster shading over to compute shaders for a number of optimization and new-feature opportunities, in addition to making it possible to clean up a lot of complex code required for the raster approach.

The ultimate goal is to fully replace the pixel shader path in its entirety, offering increased performance on both CPU and GPU, improve code maintainability, and also make it possible to implement advanced Nanite material functionality that would not be otherwise possible.

Dynamic programmable displacement allows Nanite meshes to be modified at runtime using a displacement map or procedural material. Unlike World Position Offset which can only operate on the original mesh vertices, Nanite displacement tessellates the mesh at runtime into additional triangles to conform to the detail of the displacement map. Only as much triangle detail required for the current pixel density is generated.

Benefits include:

• The ability to use lighter source meshes in the authoring pipeline.
• Material-driven and animated displacement
• Creating detailed Nanite landscapes

 

[Scott_Arm]

Definitely.



Given how Nanite works, I would say so.



Personally, I believe it's the art pipeline lagging behind.

Given rising development costs, will it ever catch up?



So then, why use Nanite?

So it's your belief that nanite is actually slowing geometry processing down? Basically the hw units are actually faster than a software rasterizer in all cases? And you can tell this by comparing different games and looking at what? Resolution, fps, perceived geoemetric detail?

How do you account for all of the other factors of different game engine renderers? Data streaming, material systems, shading, the render thread/threading/RHI/jobs/draw calls? How do you look at Horizon Forbidden West and say, Hellblade 2 and seperate out all of the other factors and make a judgement on Nanite vs the way Decima handles geometry?

 

[davis.anthony]

So it's your belief that nanite is actually slowing geometry processing down? Basically the hw units are actually faster than a software rasterizer in all cases? And you can tell this by comparing different games and looking at what? Resolution, fps, perceived geoemetric detail?

Where are you getting that nonsense take of what I've said from?

Current geometry counts in modern games arguably, aren't high enough to justify using Nanite and the overheads that comes with it, and a traditional geometry pipeline could/would be faster (a view also shared by Sebbi) with todays games.

And looking at the UE5 games that have been released and use Nanite, none of them are setting the world alight in terms of geometry.

There are games that don't use UE5 and nanite that arguably have more visible geometry. So what's nanite actually offering the games that use it in terms of geometry, that justifies its performance overhead?

 

[davis.anthony]

Games can't run with per-pixel triangles because performance would be crushed by the fixed-function rasterizer.

But why would they do that when their art assets don't contain enough geometry to require that?

A software rasterizer can beat the hardware units in the case where a triangle covers fewer than ~6 pixels.

But again, see my above point.

That's one of the things nanite was trying to do - beat the hardware units.

And that's one of the point raised in that YouTube video, with current geometry counts, the hardware units aren't a bottleneck if it's setup efficiently.

So if current assets don't require a system that can handle pixel size triangles, and the traditional hardware units aren't a bottleneck, why use Nanite?

Saying games don't need Nanite because they're intentionally designed to work around the limitations of renderers that don't have the benefits of Nanite doesn't really make sense.

I've not said that, games don't need Nanite because current games don't have assets that require a system that can handle pixel sizes triangles.

Do you think the assets in The Remnant 2 are so dense with geometry that they need Nanite?

if you made a UE5 game with Nanite and you didn't scale up your geometry to actually see the benefits, it might be an odd choice.

But that's what current games are doing.

Remedy can decide that they can do enough wish mesh shaders for their visuals and art style. Epic can decide something else and continually improve Nanite support for vegetation, terrain and animated meshes. They've done all kinds of stuff since the UE5 preview that Sebbbi looked at.

I would argue that Alan Wake 2 has assets that are more geometrically detailed and dense than anything in a UE5 game that's using Nanite.

 

[Scott_Arm]

@davis.anthony I'm curious. Do you think Alan Wake 2's geometry detail is a function of it having a mesh shader based geometry system?

 

[davis.anthony]

@davis.anthony I'm curious. Do you think Alan Wake 2's geometry detail is a function of it having a mesh shader based geometry system?

No, I think it's primarily driven by their art pipeline and what their development time and budget allowed them to create in terms of assets.

And they went from there.

 

[Frenetic Pony]

Ok, to explain what Nanite even is:

Nanite is combination of software mini triangle rendering, meaning as long as the hardware has I believe it's good 64bit atomic support(?) the geometry isn't even run on the traditional raster pipeline at all unless the triangles are particularly large. This is what fundamentally allows the triangles to be about 1 pixel in size, and thus ideally for the camera to have 0 visible LOD transitions. All this should happen quite quickly, the optimization for the software raster part of nanite is very well done and isn't much different from traditional raster pipeline with much bigger triangles under otherwise identical scenarios.

What nanite doesn't do is optimize anything else for the developer. Nanite, like all modern triangle geometry, runs on "clusters" of triangles all at once. And if the devs put in multiple models overlapping in one area you can still get multiple clusters being rendered in one small part of the screen, meaning you need to render the entire cluster* (not totally, there is individual tri culling but that still takes time) in a small area of the screen even if only a few triangles from that cluster are needed. The same cost goes up for shadow rendering as well.

On top of that it doesn't optimize (by itself) dev created shader cost, which is still a major performance concern. EG "Doom looks great at 60fps!" in part because there's an extremely limited material system for artists to keep shading costs as low as possible, or "That can't realtime" for Fable (reboot) but it is in part because there's a multi page list of guidelines for tech artists creating shaders in an optimal way. This is among a myriad of other concerns over optimization.

"GPU driven geometry" isn't new, Carmack was publicly theorizing about it before he went off to do other stuff and the first geometry cluster rendering appeared all the way back in AC Unity, which is how they got that amazing crowd density. Nanite is logical extension of that taken to the extreme, and makes geometry rendering and asset creation much less of a concern for devs than traditional pipelines. But it's not a magic button that makes all games run at 60fps.

Only a tiny handful of games coming out this year like Stalker 2 even started on UE5, and Stalker 2 started pre 1.0 public release. I.E. no nanite foliage so that still needs to be optimized and done in a traditional pipeline, same with characters which obviously isn't in UE5 till 5.5 anyway. On top of that it needs to optimize shadows, which is expensive in UE5 and such a large part is still under dev control that an entire talk at Digital Dragons (eastern european game dev conference) was about optimizing shadows in UE5. And devs need to optimize shaders, and level art (don't put in translucent stuff please, don't overlap models, etc.) etc. etc. etc. Nanite is just one aspect of rendering that devs need to worry less about than they did. It's great at what it does, but what it does is less of a games overall runtime cost than might be expected.

 

[troyan]

The problem with Nanite is not performance but the combination with Lumen. What is the point of a fast geometry processing system when you combine it with the most inefficient lighting system?! You can have a tree with 1 billiion triangles but a reflection has just two... And it cant be a solution to switch to Pathtracing like in Black Myth: Wukong...

 

[DavidGraham]

Now we see the amount of geometry in some basic world objects. I didn't line up distances well here, but basically the spherical object stays a sphere even up close, but in non-nanite it is never a smooth curve.

In my opinion, Nanite's only usefulness is in these situations, smoother curves compared to even highly dense non natite meshes and the pop in/pop out free experience.

We can do a quick comparison for the pros and cons for each technique.

Nanite:
Smoother curves
Continous LOD (except for shadows, HLODs, and skeletal meshes)
Not Ray Tracing friendly (although recent efforts point to a different picture)
Don't work well with physics and destruction
Need Virtual Shadows for optimal performance
Potential worse performance
Automatic (less work)


Dense Geometry/Meshlets:
Slightly less smooth curves
Slightly more pop in/pop out
Ray Tracing friendly
Works well with physics and destruction
Doesn't need Virtual Shadows
Potential more performance
Manual (more work setting up LODs)

So in the end, a developer will calculate which technique best suits his needs, and will stick with that. Is Nanite worth it for slightly better curves and less pop in, against all the potential draw backs? Is Meshlets worth it with it's less automatic and more manual work approach?

 

[jlippo]

Indeed it does, it's triangles larger than 6 pixels iirc.

Have they changed from the original where they used software rasterizer for polygons with edges smaller than 32 pixels?

 

[SumoSaki]

Alan Wake 2 is it from what I know of, and a side form the rare visible LOD transition in the first forest area of the game it's a pretty flawless showing in terms of geometry. The game is a very good demo of what mesh shaders can actually do.

I thought Starfield did a great job not sure if that game uses mesh shaders.

 

[Charlietus]

Does demon's souls use primitive shaders? Honestly, that game is still the best looking game on console, at 1440p 60 fps locked too

 

[Globalisateur]

Does demon's souls use primitive shaders? Honestly, that game is still the best looking game on console, at 1440p 60 fps locked too

Yep, and that's a launch exclusive game. Which is I think the reason it's still the best looking game on consoles. They could push that specific hardware to the limits (lighting, textures, I/O) without compromises because of others platforms. Resogun did a similar thing too on PS4 with async compute. This is why we still need consoles!

 

[pjbliverpool]

Does demon's souls use primitive shaders? Honestly, that game is still the best looking game on console, at 1440p 60 fps locked too

Resolution and frame rate aside, better looking than Hellblade 2, Wu Kong, Avatar, Alan Wake 2? I'm not sure I could agree with that.

 

[trinibwoy]

Forever Winter - early access co-op shooter built on UE5. They've taken an interesting approach to a launch trailer and people seem to be loving the populist message.

Hope it's not a scam and there's a real game there. The animations look pretty janky though which isn't a good sign.

 

[Charlietus]

Resolution and frame rate aside, better looking than Hellblade 2, Wu Kong, Avatar, Alan Wake 2? I'm not sure I could agree with that.

Hellblade 2 is even better, even if it it's not 60 fps, so it's not a fair comparison (it's also very, very linear compared to demon's).

Wukong is a bit blurry, the shadows in some spots are not great and it has big framerate dips.

Avatar is a mystery to me, subjectively it doesn't look that great . But objectively, 720p to 1440p with fsr 2 in the 60 fps mode is blurry on a 55" tv.

Lastly, Alan wake 2 is very aliased, even in the 30 fps mode. I finished the game and some spots were dropping the framerate HARD (the beach set piece near the end). I know that demon's souls barely has any facial animations, as that's not the focus in that game, but in Alan, sometimes, they are frustratingly bad.

Demon's is just a very polished visual experience. The graphics are so complete that you don't think much about them. The resolution is high enough that it looks sharp and the framerate is rock solid.