Game development presentations - a useful reference

[JoeJ]

I think it makes some really nice in-roads in those areas of the image at 1spp where the path trace + restir merely returns dark + salt and pepper. Or even DDGI for that matter.

Comparing to 1spp PT does not make much sense, but with DDGI does. (Both methods provide a cache to lookup lighting instead accumulating many paths to get it.)

Very promising: They can provide high frequency details in the cache. No other caching method can do this in realtime and with practical memory costs.
Though, the video shows heavy temporal fluctuations in motion, even on static scene. I expect further progress on ML denoising to address this, but i doubt we could use that in a action game (where stability is more important than realism) anytime soon.

I think the most promising approach would be to have another caching system which is temporally stable but less detailed, and world space to handle disocclusion (e.g. DDGI). If combining strengths of both is possible, temporal issues might become acceptable.
Likely that's not so easy, otherwise they might have shown this as well.

 

[Lurkmass]

I think it makes some really nice in-roads in those areas of the image at 1spp where the path trace + restir merely returns dark + salt and pepper. Or even DDGI for that matter.

*snip*

When we take a look at the error metrics (lower is better) in the interactive results, most of the time (PT+ReSTIR+DDGI) manages to come out on top of (PT+ReSTIR+NRC) in the 1spp case ...

 

[Frenetic Pony]

Comparing to 1spp PT does not make much sense, but with DDGI does. (Both methods provide a cache to lookup lighting instead accumulating many paths to get it.)

Very promising: They can provide high frequency details in the cache. No other caching method can do this in realtime and with practical memory costs.
Though, the video shows heavy temporal fluctuations in motion, even on static scene. I expect further progress on ML denoising to address this, but i doubt we could use that in a action game (where stability is more important than realism) anytime soon.

I think the most promising approach would be to have another caching system which is temporally stable but less detailed, and world space to handle disocclusion (e.g. DDGI). If combining strengths of both is possible, temporal issues might become acceptable.
Likely that's not so easy, otherwise they might have shown this as well.

You can cache a spherical basis function instead of just a normal texture to increase detail versus memory. Just take this example and flip from storing irradiance to radiance. Further, it supports glossy reflections and is incredibly stable. H-basis lightmaps have been popular for a while, since you're doing final gather from a cache, other than the tracing step the idea is pretty much the same, as is all the benefits. Example of what kind of detail you can get with using an h-basis lightmap for specular approximation, just flip that and you'll get the same detail for outgoing light:

 

[JoeJ]

You can cache a spherical basis function instead of just a normal texture to increase detail versus memory.

But this never gives you such high frequencies closely to what's shown.


(See the lights reflections which are pretty sharp.)

 

[Frenetic Pony]

But this never gives you such high frequencies closely to what's shown.
View attachment 5634

(See the lights reflections which are pretty sharp.)

Just lumen software alone, with it's low res flat cache, can get pretty close to that, but not perfect of course. Mostly I was thinking about how to best trade-off memory for detail for a surface cache.

For really smooth reflections you'd probably want shade on hit no matter what else you're doing, and all the divergence that entails.

 

[chris1515]

Dori Arazi is director of photography at Sony Santa Monica very funny why they use chromatic aberration, film grain.

We actually worked pretty hard to make our visuals look worse. The whole thing came from the problem of the "uncanny valley," and for those of you who are not familiar with the uncanny valley, the traditional thought would say that: if something looks better, than it looks more believable. Kind of like in a linear manner - the more believable, the more relatable.

What actually happens is... When something becomes very close to realistic, our psyche starts judging it in a very harsh manner - it looks for realism, and it finds all the defects. There is a giant dip in believability just before you hit photorealism. And because of the power of our game engine these days, and our amazing tech render teams, and our amazing art teams... we fell right in the middle of the uncanny valley.

So what we try to do is go back to the documentary style, and find things that we relate to as realistic footage, and slap on top of the image to try to at least, if not completely skip over the uncanny valley, kind of make it more shallow, less deep. So we added film grain. We added hand-held. We added crap hitting your lens all over the place. Lens flares. Chromatic aberration. We slipped on focus. All these things that, if you were a filmmaker shooting live action, you would try at all costs to avoid. But what it did is spike our brains and calibrate them toward a sense of realism, and help us skip over the uncanny valley, and relate to the footage without pushing back against it, and then having a giant dip in absorption.

 

[JoeJ]

For really smooth reflections you'd probably want shade on hit no matter what else you're doing, and all the divergence that entails.

Maybe we don't need to do this everywhere.
As a human, looking at the scifi image, i can immediately see the reflections of the lights are a bad guess at the boundary of the lights shape.
So ML approach should be able to detect such obvious error in a similar way, and spend expensive rays only at this boundary to fix it efficiently.
Other remaining errors which are not obvious to perception might not even need a fix.

That's kinda the stuff where i can imagine ML makes sense for realtime graphics.
Another example would be with TAA: A moving object over noisy background (a bush) leaves a trail of ghosting object color. Once it's in the history, we can't get rid of this visually obvious error.
I assume ML has better chances here than human engineering. But lacking understanding of ML, that's just a personal guess ofc.

 

[chris1515]

https://highperformancegraphics.org/slides20/monday_gruen.pdf

Open problem with current raytracing @JoeJ

https://twitter.com/i/web/status/1410339766571343883

Find it searching Brian Karis tweet explaining current raytracing is not ideal for Nanite.

 

[chris1515]

https://twitter.com/i/web/status/1411368484102148109

https://twitter.com/i/web/status/1411368488384540673

https://twitter.com/i/web/status/1411383263252430857

https://twitter.com/i/web/status/1411389918702718979

https://twitter.com/i/web/status/1411437650293235714

 

[JoeJ]

https://highperformancegraphics.org/slides20/monday_gruen.pdf

Open problem with current raytracing @JoeJ

Yay, nice numbers to give me ammo for my repetive BVH and LOD arguments

The stuff in the red box could go down to less than a millisecond if we had both.

 

[jlippo]

Visibility Buffer Rendering with Material Graphs by John Hable

 

[Jawed]

The most unique implementation detail about Nanite's renderer compared other visibility buffer implementations is that there are "3 splits" to it's shading pipeline ...

On a traditional deferred shading pipeline, we usually have the lighting pass separated from the geometry pass so we pay overdraw for attribute interpolation and material shading in this case.

With other visibility buffer pipelines, it's common to see a visibility pass with a coupled material and lighting pass which isn't all that conceptually far off from forward shading so we can still realistically apply MSAA and transparencies at a relatively cheap cost too since we have a thin G-buffer.

Nanite tries to evolve deferred rendering into it's natural direction with the visibility buffer so that we can defer the attribute interpolation/material shading pass along with the lighting pass. Generate the visibility buffer to avoid overdraw during material shading but keep the G-buffer so that we can create specialized shaders to control the register pressure which will further minimize our shading cost! The downside is that MSAA or transparencies become very expensive in this scenario compared to other visibility buffer pipelines because we need a fat G-buffer as well ...

Perhaps you've just read John Hable's paper referenced by jlippo?

There appears to be nothing in Hable's implementation that wouldn't run on a HD 6970 or GTX 580. Sure you'd have to code around the limitations of compute from back in the early D3D11 days, but nothing in the article looks impossible on that 10-ish year-old hardware.

Sure, the performance may be underwhelming. But 10-pixel triangles at 1080p back in 2010 would look better than pretty much every game that was around back then.

 

[Lurkmass]

Perhaps you've just read John Hable's paper referenced by jlippo?

There appears to be nothing in Hable's implementation that wouldn't run on a HD 6970 or GTX 580. Sure you'd have to code around the limitations of compute from back in the early D3D11 days, but nothing in the article looks impossible on that 10-ish year-old hardware.

Sure, the performance may be underwhelming. But 10-pixel triangles at 1080p back in 2010 would look better than pretty much every game that was around back then.

http://c0de517e.blogspot.com/2016/08/the-real-time-rendering-continuum.html

Yes but I think the idea of hybridizing deferred shading with visibility buffers was first theorized in one of Angelo Pesce's posts behind section "Cutting the pipeline "early"".

In theory, nobody prevents these techniques to work with more than one split: after the geometry pass a material g-buffer could be created replacing the tile data with the data after texturing.

I think of Hable's post as an extension to Pesce's hypothetical scenario raised at the time.

 

[Frenetic Pony]

Visibility Buffer Rendering with Material Graphs by John Hable

Was surprised at how fast the visibility buffer caught up, surely hardware fastpath has to be faster until the worst case scenarios right? Well, turns out that happens much faster than one might anticipate.

Maybe UE5 will switch to all visibility buffer after this test, as this implies both that this should cut down on overall render times even for non nanite geo, and that it should help artists worry less about their material cost regardless of the geometry type. Hypothetically one could keep a visibility only renderer for translucency as well, after all you're not using traditional deferred features like SSR for translucency anyway; at most you're keeping some sort of proxy card. Then you'd get complex materials on translucency as well, though one can imagine artists quickly trying to do some sort of refraction shader worst case scenario like he mentioned.

 

[chris1515]

 

[Jawed]

http://c0de517e.blogspot.com/2016/08/the-real-time-rendering-continuum.html

Yes but I think the idea of hybridizing deferred shading with visibility buffers was first theorized in one of Angelo Pesce's posts behind section "Cutting the pipeline "early"".

I think of Hable's post as an extension to Pesce's hypothetical scenario raised at the time.

Seems entirely reasonable.

And Pesce also mentions texture space lighting, which when temporally cached with ray-traced results for static scene surfaces seems viable as another performance win.

 

[chris1515]

https://www.4a-games.com.mt/4a-dna/...upgrade-for-playstation-5-and-xbox-series-x-s

 

[chris1515]

https://substance3d.adobe.com/magazine/bluepoint-games-the-masters-of-remasters/

Interview about texture and material creation of Demon's souls remake

 

[jlippo]

Raytracing tidbits.
https://interplayoflight.wordpress.com/2021/07/10/raytracing-tidbits/

 

[chris1515]

EDIT: