Nvidia DLSS antialiasing discussion *spawn*

I agree that we need more advanced AA method than TAA, but DLAA (as DLSS) is locked to nV.... but we need some IHV and platform agnostic method.

So I'm not excited at all because nV will use DLAA just for marketing purposes

 

Given how horrible some taa solutions are, I'll be happy to have an other solution if it's looking better....

 

And it wouldn’t help with adoption of RT.

 

Can we get a DLSS-style upscaler for video content please?

 

I think it should probably provide a nice benefit over normal TAA - as it will not over blur or ghost as much. It's cost will depend on the GPU in question - ranging from very cheap on a 3090, to not so cheap on a 2060.

 

Video compression is already a DLSS-style upscaler for video content.

 

I would not expect too much over DLSS on static scenery unless DLAA uses aggressive negative texture LOD bias.

 

Content creation with jitteted Fov and compressing motion vectors could be challenging.

There are other methods which do allow some nice super resolution.

 

DLAA is out today on ESO test servers.

https://www.nvidia.com/en-us/geforce/news/windows-11-game-ready-driver/

I hope this comes to UE soon.

 

No need for both. Video compression relies upon key frames rather than camera jittering and there is optical flow instead of motion vectors.
Jittering is a real time graphics thing because rendering every second, third or fourth frame at full resolution like in video compression would unavoidably cause stuttering and make a game unplayable.

Temporal incoherence will be a hard thing to overcome with GANs, though there is a progress in this field too - https://nvlabs.github.io/alias-free-gan/

 

I'm not sure i understand. I'm talking about low-res video like older DVD stuff or TV shows like Babylon 5.

 

Video codecs use temporal coherency to store only a small fraction of frames in high resolution, so temporal reconstruction is already in play for the video.
DLSS style reconstruction is all about combining pixels from multiple frames, so it would not work for the video type of content because the same principles are already used for compression, i.e. it's like applying DLSS on top of another DLSS)

Temporal reconstruction won't work without jittering (or higher res keyframes) and GANs (spatial upscaling via generators networks) have problems with temporal coherency and the details they hallucinate will never match reference higher resolution details.
That's a fundamentally ill-posed problem, though it might look ok with temporal coherence. Also, that's like coloring the old films, result will never match the original filmed content and will likely look highly unnatural.

 

Video compressors can use information from the past and also the future. We can’t do that with real-time image reconstruction in interactive applications.

 

I see now what you mean, I do not agree though.

Video compression relies in part on higly compressed B-Frames, less compressed P-frames and barely compressed I-Frames, the "ground truth" of video, so to speak. Higher quality video often uses I-Frames more often or uses less destructive compression on them. Highest quality video (apart from umcompressed ofc) would use all I-frames.

Video compression's main goal is to save storage space, while still being decompressable by dedicated logic. A lot of trade-offs made, sacrificing quality.

What I was thinking is: An neural-net based decompressor could, in theory, use the best material inside a compressed video stream (i-frames) and use those to extrapolate detail instead of P- and B-frames. P- and B-frames could still be used in order to check, if the guesses of the neural net are on the right track for this particular sequence.

And since we're talking about massively higher processing power than in smartphone chips for example, they could use much larger data sets (frame counts) to check and refine their findings.

Wouldn't it be much easier for reconstruction to know in advance what's coming, where the motion is heading to, what the next ground truth image inside a particular video stream looks like? Sure, it's not a 1:1 port of DLSS but for my layman's mind it sounds like a much easier target to achieve.

 

The ESO PTS is out, and I made a video compares between three different AA options: TAA, TAA+DSR (4x), and DLAA.
I uploaded the video on YouTube here:



Unfortunately the video quality doesn't seem to be very good (not sure if it's going to be better later), so I put the video file (~188MB) in my Google drive here:

Spoiler

Personally I think DSR 4x still produces the best result (DSR 4x+DLAA is probably a bit better than DSR 4x+TAA), but for people who don't want to lose performance to DSR 4X, DLAA is a bit better than TAA.

 

May I spread the word & file link?

 

Fairly certain a more in depth DLAA analysis will appear on review sites shortly.

 

Yes I think it’s ok.