Next gen lighting technologies - voxelised, traced, and everything else *spawn*

[milk]

Cool variant of Imperfect SM / Many LoDs ideas:
https://worldoffries.wordpress.com/...dreds-of-lights-with-out-hundreds-of-shadows/

A similar techniche was proposed in 2014 for virtual shadow map buffers for many lights using traditional geometry and heavily tiled shadowmaps (a pointlight would consist of hundreds os small tiles paged to the virtual shadowmap memory space instead of just 6 faces at the same res) to provide best shadow-texel to screen pixel fitting. It could run at PS4 level hardware then, but I assume it's overhead was not practical for game engines doing a dozen other complex things within a frame. The closest we got was Trial's tiled sun shadows and IdTech's virtual shadow map buffer for doom (albeit without the clever)

That's the kind of thing I think many people who say traditional real time rendering does not have much room for evolving without RT are unaware of. Sure RT opens many possibilities, but we still have a thousand others without it anyway.

 

[fuboi]

Does it ever get sharp? Would it be possible to update hardware texture scaling with something better than bi/trilinear too, and maybe have some gaussian element? Using super low buffer still ends up with obvious texel squares. Presently we need to sample in the shaders, but as it's such a frequent feature and probably moreso going forwards, is there value in updating the texture sampling process to get better results at the atomic level?

IMG has a bi-cubic Vulkan extension. It's probably not expensive in terms of hardware.

 

[AlNom]

Cutscenes aren’t gameplay.

 

[MfA]

I'm not baiting mods

I never said you were off topic, yet when you replied you immediately implied I did and that it was me who was off topic ... if you're not intentionally doing it, then maybe be a little more careful then.

How else do I protest against that type of statement.

Say you don't think it's fair and move on.

If you don't want a discussion on how NVIDIA devrel influences the direction game developers go then you can't go stating that direction is some kind of proof positive of their honest belief in the relevance of ray tracing at the moment. Lets be real here, their devrel is the only reason there are DXR games at the moment. If all they had released was hardware and sample code I doubt Epic would even consider spending the time to put it in their engine for a couple of years ... look how Vulkan support dragged.

 

[Shifty Geezer]

It's not a fair statement to say RT based games still look like video games, no one ever said RT based games should look like real life.

But that's exactly what this thread is about. What are the limiting factors with compute game visuals and what technologies do we have to advance past them. Part of that discussion is observations. Saying presently RT games don't look any different from other games isn't in any way a bash on RT. It's an observation. It's one followed with a theory (in my case, area lighting is the difference) from which can follow discussion on what really is the difference between 'the CGI look' and the 'game look' and what techs are possibly going to get us past them.

A critique of the shortfalls of RTRT in the here-and-now is not in any way a judgement call on the technology or concept, any more than a critique of rasterisation is a judgement call. We just have observations, theories (including 'this has a future' and 'this is a dead end'), and discussion of solutions, with reference to whatever techs we can find.

If you don't want a discussion on how NVIDIA devrel influences the direction game developers go then you can't go stating that direction is some kind of proof positive of their honest belief in the relevance of ray tracing at the moment. Lets be real here, their devrel is the only reason there are DXR games at the moment. If all they had released was hardware and sample code I doubt Epic would even consider spending the time to put it in their engine for a couple of years ... look how Vulkan support dragged.

This isn't the thread for discussing the politics of different feature adoption. Whether we use raytracing because the aliens brainwashed the Senate or we use voxelised lighting because the Illumanti are funding it doesn't matter. We're just discussing rendering requirements and solutions and their pros and cons.

 

[iroboto]

I never said you were off topic, yet when you replied you immediately implied I did and that it was me who was off topic ... if you're not intentionally doing it, then maybe be a little more careful then.

Say you don't think it's fair and move on.

If you don't want a discussion on how NVIDIA devrel influences the direction game developers go then you can't go stating that direction is some kind of proof positive of their honest belief in the relevance of ray tracing at the moment. Lets be real here, their devrel is the only reason there are RTX games at the moment. If all they had released was hardware and sample code I doubt Epic would even consider spending the time to put it in their engine for a couple of years ... look how Vulkan support dragged.

Like any feature, support is going to be needed. This doesn't seem comparable to nvidia game works. The scale just seems completely different here. I mean we can talk about lighting and all this stuff. But at the end of the day, games tend not to look better without either more time, or that there was an evolution in game development that allowed them to build better stuff for significantly less labour.

A lot of these demos are very concentrated scenes with months of work put in a confined scenarios. We can't extrapolate that to a video game, it's gotta be way cheaper (or faster) to be able to deploy something like that for us to see it happening in a video game. And imo, either the developer is going to foot the bill to build those things (thus an incredible amount of time and money), or the consumer will in terms of hardware processing power for some very less efficient trades.

 

[Shifty Geezer]

IMG has a bi-cubic Vulkan extension. It's probably not expensive in terms of hardware.

Imagination have all the best techs.

 

[OCASM]

Adapt what? You could take any game scene, chuck it in Arnold, and get something that, at worst, looked like a photo of miniatures. It'd still look like a photo. There'll be a degree of material complexity that can't be matched, especially in organics, which is something games will look gamey in, but for metals and plastics and stonework, games should be able to match the look (not necessarily complexity or fidelity) of CGI using raytracing.

As I said though, I think it's just the lighting. Once we get area lights, we reach the next step of realism. This was very true in offline raytracing. The first package I know with raytraced area lights was Realsoft3D and it had a whole level of impressive over the other renderers. That and proper specular (reflected objects) thanks to an HDR-capable pipeline instead of just phong. Realsoft3D was overtaken in the realism stakes when other renderers went physically-based, and of course games have PBR materials so are all ready for improved lighting to look amazeballs.

Actually, at worst you can make it look worse than a current gen game. Game lighting practices are based on the constraints of rasterization. You have to give artists some time to adapt the way they think about lighting from now on (or replace them with offline CGI artists ).

 

[JoeJ]

Does it ever get sharp? Would it be possible to update hardware texture scaling with something better than bi/trilinear too, and maybe have some gaussian element? Using super low buffer still ends up with obvious texel squares. Presently we need to sample in the shaders, but as it's such a frequent feature and probably moreso going forwards, is there value in updating the texture sampling process to get better results at the atomic level?

To make it sharper he could increase SM resolution or use unique traditional SMs when required.
But ofc we always end up with quantization artifacts. (Just like in voxel based techniques)

To fix / improve this there exist some options (generally speaking - independent of shown demo):

Prefilter the texel contents, e.g. using MSAA which is just downsampling / averaging a finer quantization. Or doing it 'right' which looses the performance, similar to the problem of cone tracing which is better done by averaging many rays.

Hiding hard transitions, e.g. exponential shadow maps, blurring etc.

Hybrid tech like raytracing problematic texels.

Temporal accumulation - so jitter the SM and average over time like TAA. (I wonder why there is so little research about this, considering it's a similar problem as RT denoising)


Personally this problem was reason for me to use RT for visibility. I tried little framebuffers as in the ManyLoDs paper, but a light can go from 1 pixel to zero pixel, and neither can capture the intensity well enough for a robust radiosity solver.
But there are always cases where the problems are acceptable, if only it helps to reduce more expensive rays for example. The problem i face all the time is this: All tech / algorithms we have suck. Nothing works. But can we combine some things to get acceptable results with acceptable complexity?

A similar techniche was proposed in 2014 for virtual shadow map buffers for many lights using traditional geometry and heavily tiled shadowmaps

I guess you mean this, or other works of Olsson?: http://www.cse.chalmers.se/~d00sint/more_efficient/clustered_shadows_tvcg.pdf
That's a good example of my questioning of acceptable complexity. You really end up thinking: "All this, just for efficient shadows?" Ofc. it always depends on what you need.

The big problem i see with SMs is the need to update all of them every frame if they see some movement. We want stochastic updates everywhere to trade IQ/lag vs. constant frame times. RT shadows can have more options here, but they are not perfect either - nothing is... :|

IMG has a bi-cubic Vulkan extension.

I really want this. Could help a lot with making lower resolutions look much better. I guess we'll see this from NV / AMD as well sometime...

Cutscenes aren’t gameplay.

Yeah, death to cutscenes! Make games, not movies! Movie makers should be envious about what we can do, not the other way around!

Looking the fairy UE4 RT movie, there shows a frame time graph. They tailor every scene so they get the EXACT frametime they can use. Interesting - surely a big reason real games never look as good as those (almost pointless) demos.

 

[Shifty Geezer]

Again, what do artists need to change and why? I don't think it's the art at all defining the 'gamey look'. It's the lighting and shadowing. Games are probably too clinical, which is why CA and DOF and blur can actually improve authenticity. If you look at the video I linked to on area lights, the green car transitions quite nicely from 'game' to 'CGIish' just with the change in lighting.





If you want to make the case it's needing artists to do something differently rather than requiring tech, you'll need to present some examples of what's not working in the artwork.

 

[iroboto]

Again, what do artists need to change and why? I don't think it's the art at all defining the 'gamey look'. It's the lighting and shadowing. Games are probably too clinical, which is why CA and DOF and blur can actually improve authenticity. If you look at the video I linked to on area lights, the green car transitions quite nicely from 'game' to 'CGIish' just with the change in lighting.

View attachment 2979

View attachment 2980

If you want to make the case it's needing artists to do something differently rather than requiring tech, you'll need to present some examples of what's not working in the artwork.

I think this largely depends on the studio. I mean, lighting artists, world builders, lead artist etc, someone is overlooking the whole thing. I think the idea is that they want a particular mood/or feel per scene, and world builders build the world, and lighting artists put in the work to light the area. But when it comes to having access to dynamic lighting, you can change the way the scenes are built, and for one they wouldn't be lit incorrectly indoors or underwater as much as they are, and (b) you can have more dynamic gameplay events that play off dynamism in lighting. Baked lighting is great if nothing changes, and lately that's a lot of it, nothing changing. I think we're missing out on a lot of possible creative ways to setup a scene because we've baked a lot of stuff.

A lot of people still feel that Drive Club is the best looking game because it really nailed dynamic GI.

 

[JoeJ]

Game lighting practices are based on the constraints of rasterization. You have to give artists some time to adapt the way they think about lighting from now on (or replace them with offline CGI artists ).

Really not that fair to the artists - they manage to work within technical limitations and you thank it by replacing them

Actually the hybrid era makes it much harder for them: Setup traditional lighting for the masses, also setup some extra area lights for few RTX owners. Tweak and tune both, still place probes because RT isn't fast enough, select some glass for RT reflections... all manual extra work.
You are right this will take 'some time'. Probably the same time it takes to make those 'compute solutions' to avoid this problem all together

 

[milk]

IMG has a bi-cubic Vulkan extension. It's probably not expensive in terms of hardware.

While that is cool and all, we are way overdue the point where we could open up the texture units and raster engines to be way more programmable at the dev-level (I assume they are already more flexible than we imagine before microcode and drivers come in). Even if that had some considerable performance hit, the possibilities it would open up would make up for it. Historically, it always does.

 

[OCASM]

Again, what do artists need to change and why? I don't think it's the art at all defining the 'gamey look'. It's the lighting and shadowing. Games are probably too clinical, which is why CA and DOF and blur can actually improve authenticity. If you look at the video I linked to on area lights, the green car transitions quite nicely from 'game' to 'CGIish' just with the change in lighting.

View attachment 2979

View attachment 2980

If you want to make the case it's needing artists to do something differently rather than requiring tech, you'll need to present some examples of what's not working in the artwork.

Well the problem is that you haven't really defined what does "gamey" or "cgish" mean exactly. I guess a big part of it is the general softness of the image, not just the shadows. This of course is antithetical to the current trend of ultra sharp graphics.

Really not that fair to the artists - they manage to work within technical limitations and you thank it by replacing them

Actually the hybrid era makes it much harder for them: Setup traditional lighting for the masses, also setup some extra area lights for few RTX owners. Tweak and tune both, still place probes because RT isn't fast enough, select some glass for RT reflections... all manual extra work.
You are right this will take 'some time'. Probably the same time it takes to make those 'compute solutions' to avoid this problem all together

Yes, can't wait for next-gen when RT shadows is the default and they just have to fallback automatically to cheap shadowmaps.

 

[Scott_Arm]

Adapt what? You could take any game scene, chuck it in Arnold, and get something that, at worst, looked like a photo of miniatures. It'd still look like a photo. There'll be a degree of material complexity that can't be matched, especially in organics, which is something games will look gamey in, but for metals and plastics and stonework, games should be able to match the look (not necessarily complexity or fidelity) of CGI using raytracing.

As I said though, I think it's just the lighting. Once we get area lights, we reach the next step of realism. This was very true in offline raytracing. The first package I know with raytraced area lights was Realsoft3D and it had a whole level of impressive over the other renderers. That and proper specular (reflected objects) thanks to an HDR-capable pipeline instead of just phong. Realsoft3D was overtaken in the realism stakes when other renderers went physically-based, and of course games have PBR materials so are all ready for improved lighting to look amazeballs.

I think the lighting and material formats (tightly coupled) have a long way to go. I think Disney BRDF is still considered one of the best material models, but implementation is probably only an approximation on the game side. Right now you also need to make sure your games will look good with and without ray tracing. Not having worked in the game industry I can only guess that the content pipeline is also an issue. I think right now with "raster" pipelines they've hit a point where development of the renderer has slowed and now the rest of the game engine is getting more attention. It's probably long overdue, but making the lives of the artists and animators easier, and allowing them to do more complex things more easily is becoming a priority. I even find it weird that gpu profiling tools are just adding tools that I would have thought would be available ten years ago.

Unity just happened to post this today, which is kind of on-topic.
https://blogs.unity3d.com/2019/03/28/pixars-universal-scene-description-for-unity-out-in-preview/

 

[milk]

I guess a big part of it is the general softness of the image, not just the shadows. This of course is antithetical to the current trend of ultra sharp graphics.

Bingo. A reason I never really understood or liked this popular fetish for sharpness and precision in the world of gaming. But I guess we are finally getting beyond that dark phase. I mean, RDR2 and The Order did happen and mostly everybody considered them some of the most good looking games ever, and I don't even hear the catchphrase "Vaseline smeared" nearly as much as it was routinely yelled during the ps360 gen. I guess some of it is the native resolution and AA quality have gotten to a point at which some sub-pixel blur is not as offensive.

 

[Scott_Arm]

There's also this problem: image composition
https://ventspace.wordpress.com/2017/10/20/games-look-bad-part-1-hdr-and-tone-mapping/

 

[Ike Turner]

There's also this problem: image composition
https://ventspace.wordpress.com/2017/10/20/games-look-bad-part-1-hdr-and-tone-mapping/

This 200%. Game devs for the most part don't have a clue about good tone mapping.

 

[Shifty Geezer]

Well the problem is that you haven't really defined what does "gamey" or "cgish" mean exactly.

That's because I can't define it, which is why I asked what makes them different such that games are immediately recognisable as such. If you're aware of changes artists could make, that'd point me to an answer.

I guess a big part of it is the general softness of the image, not just the shadows. This of course is antithetical to the current trend of ultra sharp graphics.

Applying a low level blur to a game screenshot doesn't make it look more realistic though. I think by softness, you mean...separation? Like, everything's computed in passes - albedo, shadows, lighting - in perfect fidelity, and composited in a clinically clean fashion. Like the RT images of old, that weren't the slightest bit realistic because they were perfect mathematical creations.

I'm going to stand by my area-light theory though. I reckon the first game with a good area-light solution is going to be immediately discernible as itself and not like the usual, interchangeable 'AAA' style we have now.

 

[MfA]

In principle all GI is area lighting.