Firstly, GPUs aren't saturated yet. Secondly I thought the ML stuff on GPUs was largely being unused save for DLSS. Are nVidia Tensor cores laregly saturated during games without doing RT buffer upscaling? Thirdly, what's the other option?
1. Depends on engine and size of GPU. I assume modern engines already do enough async stuff so adding one more async task can't be free. (That's all i wanted to say with my post.)
2. Tensor cores do only math ops. To use them, regular shader cores must run a program and call those instructions. So you can't get ML async work for free just becasue it uses tensor cores. (But i fully agree upscaling should be applied to the expensive RT stuff, not the whole frame. Though that's what UE actually does. They have reflections at 1/16 and final image upscaling at something like 1/2, for example.)
3. Scratch Lumen and use a faster GI algorithm. I have the impression UE5 is mainly a 30fps engine, and Lumen seems the cause.
I mean, upscaling is completely welcomed to render less pixels and then do work to get that approximating the high-end target with less overall effort. Why is the same principle applied to RT reflections a bad idea? Obviously it'd be a choice for games to prioritise where some will need every spare cycle for other work, but it's also a point I think that smart RT buffer processing will help get more overall quality per cycle by working smarter, not harder, once again.
I agree in principle, but there may be a different cost / benefit ratio. DLSS for instance upscales the whole frame and gives a constant speedup pretty independent of the scene and game. So the cost is predictable and well spent.
But if you use another ML pass just for reflections, probably at similar cost, the speedup depends on how many sharp reflections there are currently visible. In most images, sharp reflections are not very noticable. And it's very scene dependent. Looking at a mirror at first, then some frames later, looking at a wall of concrete. So the cost of reflections isn't constant but fluctuates over time, which i think is the primary problem. And ML can't help this.
The best general option for now seems to put little focus on accurate reflections, but invest on shadows and GI, which is both visually more important and causes more constant workloads too, so optimization is worth it.
The options for specific cases then are RT, planar reflections, dynamic cubemaps, or eventually some new ideas like spending accuracy only on light sources, because on most materials that's all we actually see in reflections. On the red robot image, we could search for bright spots in the reflection of the 'High' setting, refine resolution for those and get higher quality only there. Looks like we could get the same quality from the 'Epic' setting by refining 10% of the image area.
I think there are many options left to try, including ML methods. We will see progress in some form. And that's what we want.
There already is a major uplift vs. prev gen reflections in UE5, so we can be happy with the current state as well, imo.
Maybe i missed the proper context of the discussion, but it sounded like many people complained about blurry reflections.
Though, isn't that nit picking? I mean, the image with the red robot is not really impressive or realistic at all. It's gamey, maybe by intent, maybe not. But it does not look like better reflections make a big difference about this impression as a whole.
