Game development presentations - a useful reference


Blurring the Line between Physics and Imagination: A Glimpse of Neural Rendering's Future

Marco Salvi, Principal Research Scientist, NVIDIA

Abstract

In this talk, I will discuss the transformative impact of machine learning techniques, which are breathing new life into real-time rendering. Advances in novel scene representations and image reconstruction approaches have emerged alongside sophisticated generative models that do not rely on the physics of light transport.

These developments are bringing about a major shift in how we generate images, challenging traditional rendering pipelines. I will explore some of the possibilities that lie ahead as these distinct approaches may converge, seamlessly blending physics and imagination to create realistic and accessible computer-generated imagery.

Bio

Marco Salvi is a Principal Research Scientist at NVIDIA, where he works on developing high-performance software and hardware rendering algorithms for the post-Moore's law era.

His research interests span a broad range of topics, from anti-aliasing and order-independent transparency to ray tracing hardware, texture compression and neural rendering. Marco has made numerous contributions to rendering APIs, including raster order views and micro-meshes, and his early work on real-time neural image reconstruction has led to the development of NVIDIA DLSS.

Prior to becoming a researcher, Marco worked at Ninja Theory and LucasArts architecting advanced graphics engines, performing low-level optimizations and developing new rendering techniques for games on two generations of Playstation and XBOX consoles.

Marco holds an M.S. in Physics from the University of Bologna, Italy.
Click to expand...
 
Remij said:
Also another great presentation in slide format from REAC. Ubisoft's Dunia Engine long-term shader pipeline vision. They go over how they evolved their shader pipeline over the course of Farcry 4-6. They also reference shader compilation issues in many games with images from Digital Foundry on one of the slides.


Click to expand...

They really did credit DF with motivation for the presentation. Alex's efforts are paying off.

This is really smart though probably won't work as well for games with lots of unique interiors / environments.
  • Render the world behind the loading screen
  • Spinning the character around itself
  • This primes the virtual textures and shader variations
 
trinibwoy said:
They really did credit DF with motivation for the presentation. Alex's efforts are paying off.
Click to expand...
I would give credit to the entire DF team, as they extensively covered the struggles with stutters in Far Cry 4, where it was caused by the paging of megatextures. Since then, it seems that they have made a significant effort to eliminate the stutter across the engine and ensure that it did not return with DX12. This initial criticism and extensive coverage of stuttering in press served as a kind of vaccination against stuttering for the Dunia teams. Wonder whether Epic will follow the suit, given that the responsibility on them is much higher. The sad part is that it still took Alex's intervention to prompt them to take action (and only in UE 5).
 
Last edited:
Nice. :D

Instead of rasterization, we decompress each selected cluster into a small bounding volume hierarchy (BVH) in the format expected by the ray tracing hardware.
Click to expand...

Unfortunately, the only way to do this is to apply for a job at Intel, NV, or AMD. Not so nice. :(

But at least there is growing awareness about static meshes not being enough...

EDIT: Still not done with reading the whole paper, but some comments anyway.
From the paper we now know Intel uses BVH6, while AMD uses BVH4 according to ISA.

The papers proposal differs from my Nanite example used in discussions.
I had proposed to convert the LOD hierarchy, which may be a BVH8 for example, directly into the HW format of BVH6/4/?.
Beside the branching factor, advanced HW may have some more constraints, e.g. speculated treelet compression for NV.

In the paper they avoid such conversion complexity. Instead they just take the current lodcut of visible clusters and build a BVH from scratch over them per frame.
That's an option i've had in mind, and we could do this already by making each cluster a 'mesh/model' with it's BLAS only needed to be build on LOD changes,
and one TLAS to conncet them all. That's really all we could currently do to make granular LOD traceable.

But the cost of building such a big TLAS over so many clusters feels way to high for me. In they paper they say it's practical for realtime, but imo that's a yes for research, but still a no for games on affordable HW.
People would still mostly turn RT off for a better experience, and it stays an optional feature.

Right now i somehow miss the point of the paper. I hoped they utilized native HW access to do things not possible using DXR.
But maybe they did it just to use existing libraries like Embree BVH builders more easily.
 
Last edited:
https://zheng95z.github.io/publications/roma23

Approach on software RT with interesting performance. Tracing a ray is O(1), and generating the 'acceleration structure' from scratch is linear to scene complexity.
They do a low res voxelization per frame, then generate 2D images from a set of random directions, by rotating the voxels to the direction and storing a slab of 32(or more) bit voxels per texel.
A ray is then snapped to the closest direction, and tracing intersection only requires to test the bits we get from a single fetch. To deal with snapping restrictions they choose different directions each frame and filter temporally.
Precise ray directions work too, but then they need some more texel fetches from the closest image to do a search.

Though, i guess for practical use in video games we would need nested mip levels to support large scenes, so costs would go up by the number of levels.
Due to low detail, it also fails on hard shadows or sharp reflections.
 
You must log in or register to reply here.

Similar threads

S
SDL_Gpu API, SDL 3.0 ... New cross-platform graphics API.
Replies
1
Views
1K
Lurkmass
L
G
Tech Differences between Switch and 7th Gen (PS3/360) Games
2
Replies
21
Views
9K
GuardHei
G
N
Hellslave (Saturn Homebrew)
Replies
6
Views
4K
Nesh
N
Rys
Reopening Architecture and Products + rules and guidelines changes
3 4 5
Replies
90
Views
16K
Shifty Geezer
Shifty Geezer
iroboto
Moving Gears to Tier 2 VRS Variable Rate Shading
Replies
1
Views
4K
trinibwoy
T
Back
Top