Digital Foundry Article Technical Discussion [2021]

[Allandor]

About 5% advantage for XSX in this game using exactly similar scenes shown by VGTech (when both are dropping). Most gameplay scenes shown by DF weren't exactly like for like.

Well, seems like the SDK is getting better
Question is, what is limiting in those scenes. And it would still be interesting what the difference is above the 60fps limit.

btw, has anybody tested the Series S with VRR and the RE demo? Could help, at least with RT enabled.

 

[Shortbread]

how would it be visible? You clamped it at 60.

What if I clamped the frame rate to 55 fps? You'll see a perfect straight line until the really bad dip; and when they dip both they're within 1fps of each other. at 54 and 53 respectively. Is their performance gap < 3%?

I'm not trying to say that XSX is performing better. I'm just trying to ensure there is a separation of arguments that
a) PS5 performs more or less like XSX in the game (true; respectably there's no difference imo, 5% is not enough to really matter)
b) PS5 performs more or less like XSX with respect to the settings they have provided and within range of 9% (true, from an experience perspective you're unlikely to notice without a metric counter)
c) That the additional CU that XSX only manifest to a 9% increase in performance over the clockspeed differential on PS5 for ray tracing (false, the metrics are not an indication of how the individual components are working towards their final output)

You can't prove C. Because you'd need to see the whole thing uncapped to really know what's going on beneath the hood.

Typically XSX has been a poor performer with a lot of alpha, how do you separate it dipping from having issues with alpha and PS5 not having issues with alpha. That may be a scenario where PS5 is making up time on RT by being better at rasterization since RT is a fixed cost. While I'm not saying it is, I'm just saying, you can't use this to describe how well the hardware performs on RT. Dips are not equivalent (see Hitman 3)

I mean; it's hardly RT enough to begin with to really put the RT units to test.

Take 'Control' as an example. The additional headroom that XBSX had in the uncapped photo mode ranged from 3%-32% advantage over PS5 depending on the area. However, this advantage regardless of how high it was, still didn't translate into maintaining a rock-solid 30fps gameplay experience when RT was enabled. If I’m not mistaken, PS5 actually holds a slight performance advantage during actual gameplay.

I agree raw numbers are good to have on providing a certain amount of context and clarity on how these game systems stack up in generic benchmarking, but if those raw numbers aren’t translating into real and meaningful actual gameplay performance, then it becomes pointless wanting an uncapped framerate to see the highest value reached. IMHO, if the consoles aren’t able to maintain a certain locked/capped framerate (30fps, 60fps, etc.) by the developer, then for me anyhow, its visible that any additional performance above those caps aren’t enough, or simply isn’t there.

 

[Shortbread]

Well, seems like the SDK is getting better

SDKs tend to get better. Look at the games that PS4/XBO offered at the beginning of their generation, versus the tail-end of that generation. Huge difference! I expect nothing less from this generation.

 

[see colon]

Could you be seeing fixed cost(overhead?) of ray tracing? Building BVH for example takes same amount of time irrespective of display resolution. Another thing could be some fixed hw/driver overhead that is more visible in lower resolutions.

Second thing that comes to mind is effect of caching. Lowering rendering resolution makes the rays more divergent as they still have to cover same area with less rays. These more divergent rays could be worse for cache and cause relatively worse performance. Maybe in higher res/higher ray count case hw/driver can group rays together in more cache friendly way. This could lead to lowered resolution not to give linear performance increase. The potential bottle neck due to cache misses can happen both in going through BVH and shading steps.

Yeah, that would be my guess also. Like, there is a fixed, or minimum frame time cost to adding RT, so it disproportionately affects scenes where you would otherwise not be GPU limited. The I'm not fully convinced it's the CPU bottleknecking the rendering, as some have assumed.

 

[higherARC]

Take 'Control' as an example. The additional headroom that XBSX had in the uncapped photo mode ranged from 3%-32% advantage over PS5 depending on the area. However, this advantage regardless of how high it was, still didn't translate into maintaining a rock-solid 30fps gameplay experience when RT was enabled. If I’m not mistaken, PS5 actually holds a slight performance advantage during actual gameplay.

I agree raw numbers are good to have on providing a certain amount of context and clarity on how these game systems stack up in generic benchmarking, but if those raw numbers aren’t translating into real and meaningful actual gameplay performance, then it becomes pointless wanting an uncapped framerate to see the highest value reached. IMHO, if the consoles aren’t able to maintain a certain locked/capped framerate (30fps, 60fps, etc.) by the developer, then for me anyhow, its visible that any additional performance above those caps aren’t enough, or simply isn’t there.

A patch was released that fixed most of the stuttering that happen on the XSX. It’s really smooth now which I’m sure makes it more stable now if it weren’t for the random stutters previous numbers showed

 

[iroboto]

Take 'Control' as an example. The additional headroom that XBSX had in the uncapped photo mode ranged from 3%-32% advantage over PS5 depending on the area. However, this advantage regardless of how high it was, still didn't translate into maintaining a rock-solid 30fps gameplay experience when RT was enabled. If I’m not mistaken, PS5 actually holds a slight performance advantage during actual gameplay.

I agree raw numbers are good to have on providing a certain amount of context and clarity on how these game systems stack up in generic benchmarking, but if those raw numbers aren’t translating into real and meaningful actual gameplay performance, then it becomes pointless wanting an uncapped framerate to see the highest value reached. IMHO, if the consoles aren’t able to maintain a certain locked/capped framerate (30fps, 60fps, etc.) by the developer, then for me anyhow, its visible that any additional performance above those caps aren’t enough, or simply isn’t there.

Sure and I agree on this point, from an experience perspective this makes sense; but it doesn’t actually tell you whether having more RT units vs less RT units and a higher clock results in only the differential in frame rate.

I find that to be highly unlikely. That would imply that they rasterize equally in all scenarios which we know at this point in the generation they don’t.

 

[Shortbread]

Sure and I agree on this point, from an experience perspective this makes sense; but it doesn’t actually tell you whether having more RT units vs less RT units and a higher clock results in only the differential in frame rate.

I find that to be highly unlikely. That would imply that they rasterize equally in all scenarios which we know at this point in the generation they don’t.

Don't you just wish UL Benchmarks could provide such a tool? 3DMark the console edition!

 

[zed]

A game such as Ori and the Will of the Wisps is a fairly complex game even if it isn't a fully 3D game. But since it's not fully 3D, I'm going to guess you wouldn't consider it a complex game.

Digital Foundry also made the same mistake with this game, when you have a limited camera like it is in that game (its much much easier to make it run well), my game is similar (diablo style isometric camera) which runs great but if you change to down low behind person viewpoint camera, performance can tank. Ori is a very good looking game due to its art style but its doing a lot less than say, ghosts of tsushima
Im sure there are better game choices, but I have no idea about what engines what game uses, is there a database?

Though I do agree with what you say elsewhere, the algorithms are the most important and not the engine. Where unreal is much better than unity are the tools for the content makers and this is very important for making a game, perhaps the most important thing (plus also unity's rendering pipeline is a joke). but in a lot of things unreal (which also has gc) is a lot worse than unity, much more unstable, much worse UI, not built on a FPS, lack of documentation, C++ vs C# (if you are getting GC spikes in c# then you're doing it wrong, manage memory the same way as c++, you should be doing this anyway) (Rust may be even better than both but I havent really looked into it)

 

[rabbit]

In fact, I do not see much point in arguing about the performance and quality of RT games in the first wave, at least without power consumption measurements.
We can argue to infinity why one console has higher resolution and the other one has less fps drops. And then it turns out that at least one of them is running at less than 75% of its power (as it was with Hitman 3 and xsx).

This is especially true for projects with identical graphics settings.
I bet that in RE8 the xsx runs far below ~200 watts.

 

[Allandor]

Btw, RE8 does not seem to be a CPU intensive title. E.g. PS4 Pro almost always delivers 40fps and at times 50. So from a CPU perspective, it is not a very demanding title for the current-gen consoles. So PS5 can assign all power to the GPU. So I really expect that the difference between the two consoles will grow over time, as soon as last-gen will no longer be supported.

 

[snc]

Btw, RE8 does not seem to be a CPU intensive title. E.g. PS4 Pro almost always delivers 40fps and at times 50. So from a CPU perspective, it is not a very demanding title for the current-gen consoles. So PS5 can assign all power to the GPU. So I really expect that the difference between the two consoles will grow over time, as soon as last-gen will no longer be supported.

Nah, probably will be very similar to end of the grnerarion as 20-25% teoretical compute/rt intersection/bandwidth advantage will not increase magically over time

 

[Globalisateur]

Btw, RE8 does not seem to be a CPU intensive title. E.g. PS4 Pro almost always delivers 40fps and at times 50. So from a CPU perspective, it is not a very demanding title for the current-gen consoles. So PS5 can assign all power to the GPU. So I really expect that the difference between the two consoles will grow over time, as soon as last-gen will no longer be supported.

It's going to be the contrary. The biggest differences are mostly seen in cross-gen or GPU limited games. It is what it is. Next gen games (or modes) pushing the CPU are performing very similarly between both machines like (DMC5 & COD 120hz or NBA 2K21 with next gen assets at 60fps).

 

[eloic]

anybody remember street fighter ex

That game is a hidden gem. I loved every bit of it! Yeah, it was different, but it was superfun and the OST rocked.

 

[snc]

That game is a hidden gem. I loved every bit of it! Yeah, it was different, but it was superfun and the OST rocked.

for me it was like good example how devs struggled to keep 2d pace/gameplay in 3d world but just checked review scores after your post and indeed not bad

 

[mr magoo]

we had great conversation about hw advantages in xbox thread, now its clear to me that you cannot expect system X to perform xx% better than system Y all the time. Both ps5 ans xsx have advantages during specifics stages of rendering and if this is rendering pipeline [A,B,C,D] => (frame) with abcd being imaginary steps xsx may perform AB faster than ps5 but ps5 will perform CD faster. What now? Will xx% more CUS be always better than 20% more clock speeds? Will 20% more clock speed be enough to make up for xx% less cu? Depends on load of specific steps and this may very a lot between applications heck this may vary a lot in different moments in the same application.
Having this is mind it is clear to me that there is no winner here, even if the future will be more CU heavy or will favour faster clock speeds it is not everything that matters. You will complete whole cycle only as fast as the slowest step.

 

[iroboto]

Btw, RE8 does not seem to be a CPU intensive title. E.g. PS4 Pro almost always delivers 40fps and at times 50. So from a CPU perspective, it is not a very demanding title for the current-gen consoles. So PS5 can assign all power to the GPU. So I really expect that the difference between the two consoles will grow over time, as soon as last-gen will no longer be supported.

But RT as a feature does dramatically increase CPU usage as we have seen in other titles.

I don’t know many other graphical related features that drive CPU up as much. As we move culling and other related functions to the GPU I suspect this will stay on the CPU.

 

[cwjs]

But RT as a feature does dramatically increase CPU usage as we have seen in other titles.

I don’t know many other graphical related features that drive CPU up as much. As we move culling and other related functions to the GPU I suspect this will stay on the CPU.

Why, though? First of all, I could just be missing something, so I'd appreciate if you fill me in if so. But from what I know, the big cpu costs I can imagine are building and maintaining the acceleration structures (bvh trees) and dispatching rays. Both can be done on the gpu in theory and in at least DXR, probably other APIs too.

I guess the case could be made that the console gpus are fairly weak and they want to save compute as much as possible for the rest of rendering, so they offload RT related costs to the cpu. But even if that's the trend, I don't expect we'd see it every single game.

 

[iroboto]

Why, though? First of all, I could just be missing something, so I'd appreciate if you fill me in if so. But from what I know, the big cpu costs I can imagine are building and maintaining the acceleration structures (bvh trees) and dispatching rays. Both can be done on the gpu in theory and in at least DXR, probably other APIs too.

I guess the case could be made that the console gpus are fairly weak and they want to save compute as much as possible for the rest of rendering, so they offload RT related costs to the cpu. But even if that's the trend, I don't expect we'd see it every single game.

I don't have an answer unfortunately. It's just something I've been noticing. @Dictator might be able to chime in on this. I think he states in his Metro video that the RT is significantly more taxing on the CPU with the latest version versus the older 2019 RT variant.

I mean, perhaps it's because they don't trust async compute on PC? I'm not sure. It is quite slow to go back and forth between CPU/GPU.

They could do everything GPU side, I mean, that would make sense, but perhaps it's not trivial to do -- more time to bake this type of feature as there are more challenges to overcome.

 

[cwjs]

I don't have an answer unfortunately. It's just something I've been noticing. @Dictator might be able to chime in on this. I think he states in his Metro video that the RT is significantly more taxing on the CPU with the latest version versus the older 2019 RT variant.

I mean, perhaps it's because they don't trust async compute on PC? I'm not sure. It is quite slow to go back and forth between CPU/GPU.

My counter-theory would be that these are all relatively early games using RT (as in, engine wasnt build from the ground up with rt in mind from the start) so maybe the cpu side is just easier and more sensible to implement for an engine thats already using the gpu 100% in the existing architecture, so we may see more of a variety in the future. But this is a wild guess (could just as well be that the gpu versions have huge drawbacks nobody else will ever want to incur.)

 

[iroboto]

That is confusing -- you mean longer in total, not longer proportionally? Something weird is up.

As far as fixed costs, you can shoot less rays per pixel, but you could also choose to shoot a fixed count I guess. Additionally, as far as fixed costs go, dealing with the bvh tree (on gpu or on cpu -- theoretically you could do either one, not sure what the rt apis permit) is going to be the same regardless of resolution.

You may want to reach out to OlegH to explain this one further.
GPU Ray Tracing Performance Comparisons [2021] *spawn* | Page 3 | Beyond3D Forum