Digital Foundry Article Technical Discussion [2022]

[Jay]

That depends what the input data is and where in the pipeline the upscaling occurs. If you're doing the upscale way down close to post processing, yes. If you can get the upscaler to do the heavy lifting on the cpu intensive items then perhaps you can still come out with a relatively good image. Not saying it's easy, but we're only a few years into realtime AI upscaling, and I've seen some crazy stuff for non-realtime.

I mean thats why I think AI upscaling will work better from a lower input resolution compared to TAAU.
So it's even more beneficial for XSS.

I actually think that there's some interesting things you could do with ML-U. We're very early on, like other inputs. Maybe RT input so could use less rays etc

 

[PSman1700]

I just don't think the difference will be big enough to matter to most people.
Hence other things are more important in that regards.

indeed if it mattered, these consoles wouldnt have released with the current hardware specs.

 

[iroboto]

I mean thats why I think AI upscaling will work better from a lower input resolution compared to TAAU.
So it's even more beneficial for XSS.

I actually think that there's some interesting things you could do with ML-U. We're very early on, like other inputs. Maybe RT input so could use less rays etc

yea. I think if we just go by today, you're right. 720p is the absolute minimum they can probably take on. But hopefully one day they can go further. XSS is definitely not made for a 4K set though, going from 720p to 4K is not reasonable.

 

[Jay]

yea. I think if we just go by today, you're right. 720p is the absolute minimum they can probably take on. But hopefully one day they can go further. XSS is definitely not made for a 4K set though, going from 720p to 4K is not reasonable.

10:15 seconds in, and I'm surprised that 540p can look like native 1080p.

So if XeSS comes in at roughly the quality of DLSS 2.0 then could have 540p -> 1080p & 720p -> 1440p modes etc.

 

[snc]

Using what we know of XeSS.
The general implication uses Dp4a INT 8 I believe.
So not taking into account things like data localisation, doesn't dp4a also have additional instructions.

So PS5 could easily be twice or maybe even four times slower. I'm just guessing 4, so let's just go with 2.5 - 3, someone with more knowledge about dp4a and inference will have to chime in.

The point is that just being able to convert it to RPM 16 it may still use a lot more compute than you would think.

I'm not saying PS5 can't use XeSS, just that right now I would put it bellow XSS.
Reason being if it works like DLSS it's based on output resolution.
So PS5 2160p, XSS 1080/1440p using dp4a.
And XSS may not have enough headroom to do it at 60fps?

Can't wait to start to get decent benchmarks from XeSS etc.

Edit:
Found this blurb from an intel pdf

Looks like marketing material but gives some indication that will likely be more than twiice as fast.

maybe my math is wrong but if ps5 can use fp16 we have 20.56 tops and xss in int8 16 tops

 

[Jay]

maybe my math is wrong but if ps5 can use fp16 we have 20.56 tops and xss in int8 16 tops

I'm going to assume your maths is correct.
But then that's my point, that's just raw iops.
Take into account the additional instructions, data locality, then dp4a could easily be well above twice as fast. Intel talking about it mentions 5x, but I'm not taking that at face value.

Then factor in that performance of the upscale is based on output resolution(DLSS), which for PS5 output is 2160p and XSS is 1080p.

That's why I say that PS5 could actually be worse off in comparison to XSS, for its target use.

 

[snc]

I'm going to assume your maths is correct.
But then that's my point, that's just raw iops.
Take into account the additional instructions, data locality, then dp4a could easily be well above twice as fast. Intel talking about it mentions 5x, but I'm not taking that at face value.

Then factor in that performance of the upscale is based on output resolution(DLSS), which for PS5 output is 2160p and XSS is 1080p.

That's why I say that PS5 could actually be worse off in comparison to XSS, for its target use.

based on this vague statement ?

DP4A is a GPU instruction set to accelerate
neural network inferencing with INT8 data type,
resulting in up to 5X performance gains.

Intel didnt even bother to tell 5x performance gains comparing to what ;d I could guess comparing to fp32 so should be 4x but for maybe reasons you mentioned is 5x but just guessing

 

[Jay]

based on this vague statement ?

Intel didnt even bother to tell 5x performance gains comparing to what ;d I could guess comparing to fp32 so should be 4x but for maybe reasons you mentioned is 5x but just guessing

Many reasons why I don't take that at face value so I say 2x. But even 1.5 would make big difference.

Also why I hoped someone would have some experience or knowledge of dp4a usage as that was the only figure I found.
Think I also came across Nvidia saying something like 4x but no idea where I saw that, so since couldn't find it again I never mentioned it.

Maybe your google-fu may find some data also?

 

[snc]

Many reasons why I don't take that at face value. But even 2 or even 1.5 would make big difference.

Also why I hoped someone would have some experience or knowledge of dp4a usage as that was the only figure I found.
Think I also came across Nvidia saying something like 4x but no idea where I saw that, so since couldn't find it again I never mentioned it.

Maybe your google-fu may find some data also?

they usualy just simple compare with fp32, so int8 is 4x comparing to fp32 but 2x comparing to int16, so my point stand, xss has using int8 16tops vs theoretical 20.5tops ps5 using fp16, so its not 4-5 faster but rather slower

 

[Jay]

they usualy just simple compare with fp32, so int8 is 4x comparing to fp32 but 2x comparing to int16, so my point stand, xss has using int8 16tops vs theoretical 20.5tops ps5 using fp16, so its not 4-5 faster but rather slower

I never used those figures. And i never said xss is faster than PS5.
But even at 16 v 20 not including new instruction and data locality, given that PS5 needs to output 2160p and XSS is 1080p/1440p you need less. So int8 brings the XSS lot closer to inference power to the PS5 even though needs to do less. So that's why I currently rate XSS higher.

I'm also on the record saying don't even know if XSS has enough overhead to do it at 60fps.

Doesn't mean PS5 can't do it, it might be to slow compared to using TAAU. But we won't know that until we see it benchmarked.

Just realised, when I say could be more than twice as fast, I mean per raw single flop. Not overall.
Just realised that's how it could be read.

 

[snc]

I never used those figures. And i never said xss is faster than PS5.
But even at 16 v 20 not including new instruction and data locality, given that PS5 needs to output 2160p and XSS is 1080p/1440p you need less. So int8 brings the XSS lot closer to inference power to the PS5 even though needs to do less. So that's why I currently rate XSS higher.

I'm also on the record saying don't even know if XSS has enough overhead to do it at 60fps.

Doesn't mean PS5 can't do it, it might be to slow compared to using TAAU. But we won't know that until we see it benchmarked.

Just realised, when I say could be more than twice as fast, I mean per raw single flop. Not overall.
Just realised that's how it could be read.

Cant see anything change as it has slower theoretical capabilities than ps5 in this aspects, xsx is more interesting with more than double perf in int8 vs fp16 ps5, that should be enough for some ai reconstruction but yet to be seen

 

[Globalisateur]

What's the point of rendering fur at native 4K on animals if you destroy the details with per-object motion blur? And that's on a still pic, I can't imagine how all the details from the forest must be at vaseline induced 30fps.

I have looked at the comparisons but 60fps seems the winner to me. Where is that big improvement from the quality mode? If you look at still pictures, sure, but in motion? Very odd.

 

[snc]

What's the point of rendering fur at native 4K on animals if you destroy the details with per-object motion blur? And that's on a still pic, I can't imagine how all the details from the forest must be at vaseline induced 30fps.

I have looked at the comparisons but 60fps seems the winner to me. Where is that big improvement from the quality mode? If you look at still pictures, sure, but in motion? Very odd.

I play the game and Quality mode looks miles ahead of Perf, sad to say but perf mode is one of the most noisy and with biggest impact I saw comparing to quality

 

[Allandor]

10:15 seconds in, and I'm surprised that 540p can look like native 1080p.

So if XeSS comes in at roughly the quality of DLSS 2.0 then could have 540p -> 1080p & 720p -> 1440p modes etc.

Nah.. it doesn't really look native. Even with 1440p + DLSS in that game, the game looks still blurry while playing. You really don't get this captured in videos, but as the resolution goes sub 1080p, the overall picture is very blurry. Even native 1080p (without DLSS) looks overall better in motion and the picture quality is much more consistent. But that might just be my taste.
I would really prefer if you just select a base resolution and the DLSS option "upscales" the image to resolution X.

 

[iroboto]

Nah.. it doesn't really look native. Even with 1440p + DLSS in that game, the game looks still blurry while playing. You really don't get this captured in videos, but as the resolution goes sub 1080p, the overall picture is very blurry. Even native 1080p (without DLSS) looks overall better in motion and the picture quality is much more consistent. But that might just be my taste.
I would really prefer if you just select a base resolution and the DLSS option "upscales" the image to resolution X.

If XeSS or equivalent ever arrives to XSS such that 540p upscaling is happening. It's likely to be ahead in term of quality of what you see here with control just when you consider the time of maturity. I would expect Nvidia to be much further ahead as well.

 

[cwjs]

What's the point of rendering fur at native 4K on animals if you destroy the details with per-object motion blur? And that's on a still pic, I can't imagine how all the details from the forest must be at vaseline induced 30fps.

I have looked at the comparisons but 60fps seems the winner to me. Where is that big improvement from the quality mode? If you look at still pictures, sure, but in motion? Very odd.

I think this is the wrong way to think on two levels.

1- The motion blur isn't "'destroying the image", the motion blur is the image that we're trying to see. Good graphics aren't about detail detail detail all the time in constant fine level noise, they're about realism, style, and accuracy. The cleaner, more precise, more accurate blur, which captures (and blurs!) more of the surface detail is clearly worth some lost performance.

2- In HFW in particular (but also in any other game using temporal reconstruction) the low res version actually looks much better in stills than it does in motion. The 4k version has a significantly more stable image, with less stairstepping and crawling along man made structures, and less noise in details like grass, hair, etc. It's not about any one shot looking "more detailed" as much as it's about the overall image looking more stable and more realistic.

(disclaimer: i'm playing on performance mode, but I've noticed enough artifacts to switch and compare in places and build a decent sense of what quality does better)

 

[Shifty Geezer]

Conceptually, the degree of fur detail, or indeed any shader detail, could be scaled back based on post effects being applied. Anyone know if any games are doing this?

 

[cwjs]

Conceptually, the degree of fur detail, or indeed any shader detail, could be scaled back based on post effects being applied. Anyone know if any games are doing this?

You can do this for the pixel shader with software or hardware VRS -- Gears 5 did this.

 

[Cyan]

10:15 seconds in, and I'm surprised that 540p can look like native 1080p.

So if XeSS comes in at roughly the quality of DLSS 2.0 then could have 540p -> 1080p & 720p -> 1440p modes etc.

one of the best DF videos back in the day. Glad to know that the mythical, great, the legendary creature Shifty is back btw

 

[PSman1700]

the legendary creature Shifty is back btw

He has been for awhile now