Do we use the lowest performance (one frame) when we benchmark GPUs in order to compare them in a fair way?
No. Usually we use average for a good reason. Because comparing using only one frame is obviously not representative.
Digital Foundry Article Technical Discussion [2021]
- Thread starter BRiT
- Start date
- Status
No. Usually we use average for a good reason. Because comparing using only one frame is obviously not representative.
shrug =P would be curious to see an investigation, but I can only suspect these types of videos that Alex makes are fairly costly. You need the equipment and time and talent to make it all happen.
I don't expect them to come back to patch it either, and that's fine, that's rather unfortunate for XSX owners that the game can't climb as high. But I can't see it selling so well that they want to go back to improve things either.
No because, we can benchmark GPUs properly with all the same settings putting all the pressure and bottlenecks on the GPU.
We can't do that with PS5, so you' need to find points in time where everything goes down together to gain insight as to what is happening.
Alex is very clear about the results in the post here:
https://forum.beyond3d.com/posts/2189256/
To add to what iroboto said, we actually do measure the lowest performance when benchmarking PC GPU's. Or at least most decent reviews show minimum frame rates as well as averages.
And besides, he did show other scenes where the PS5 was faster and specifically called them out in the video so I really don't know what all the fuss is about.
D
Deleted member 86764
Guest
Got the timestamp? I can only see stuff from that same scene.
Last edited by a moderator:
12:59 shows a different scene but I didn't mean entirely different scenes/environments, just different things happening on screen that show different performance characteristics.
D
Deleted member 86764
Guest
Got it - same scene, but yeah different characteristics. Would you say that based on the two drops that the 2060S was performing ~5% better than the PlayStation?
Oh.
You should re-watch this part. May explain what we are seeing here. Possible bug on PC, high/medium is the same (medium is high setting). And low is below PS5.
In an area with lots of reflections, they may matter.
But anyway, as per the video. The graphic he's showing 5% better than PS5 is titled correctly.
Minimum FPS during Miami Stage Intro
Then during the scene being referenced he indicates that PS5 is performing better than the 2060s in rasterization performance.
Last edited:
D
Deleted member 86764
Guest
Explain to me what you're seeing and how that effects the conclusions elsewhere in the video?
So the 5% better than PS5, is just specifically in reference to the Miami intro, he does not use it as a conclusion to say it's performing 5% worse overall to 2060s. I think there seems to be a comprehension problem, the remaining conclusions here showcase that the lack of bandwidth may likely be the cause of the reason why PS5 is running a lower resolution than XSX when looking at the same GPUs in the same bandwidth heavy areas that drops are being pushing across the GPUs.
He says in non bandwidth limited scenes shown here PS5 is out rasterizing the 2060s.
With respect imo as to why the 2060s is getting hammered possibly by that sequence, some of it may be attributed to the fact that the settings are not the same as PS5, and it's just a little too much for it to continue at 60fps. That's is a side discussion separate from the videos end conclusions about PS5 wrt PC performance.
Transcript:
describing memory bandwidth limitations
14:09 on performance so
14:10 other areas of that benchmark show the
14:13 rtx 2070 super scaling below the series
14:16 x like we would expect it to based upon
14:18 theoretical numbers
14:20 still this information that we get from
14:21 this test is enlightening as even
14:23 without the same resolution and settings
14:25 we can see how the series x fares better
14:28 in comparison to pc gpus
14:30 than the playstation 5 does thus going
14:33 part of the way
14:34 explaining perhaps why the series x is
14:37 running this game at a higher resolution
14:39 than the playstation 5.
14:40 with these numbers we can also see how
14:42 gpus with less compute yet
14:44 similar bandwidth such as the 2060 super
14:47 in comparison to the playstation 5.
14:49 how the 2060 super is coming closer in
14:52 performance in this
14:53 instance than might be expected perhaps
14:55 pointing to the shared memory bandwidth
14:57 on consoles between
14:58 cpu and gpu depressing performance in
15:02 comparison
15:03 which might explain why a number of
15:05 console games we've seen this generation
15:06 at its start
15:07 are using lower resolution particle
15:10 effects than available on pc
15:12 but that's really all i can reasonably
15:13 say for now
Transcript 2:
12:41 compute wise the playstation 5 should
12:43 rather easily pass the rtx 2060 super
12:45 and rasterization
12:47 and it does and this benchmark here in
12:49 miami shows it as well
12:51 in other sections where the playstation
12:53 5 is doing just fine at 60 fps
12:55 while the rtx 2060 super at the same
12:57 settings we'll see some sputters down to
12:59 55 fps or so
13:01 sadly vsync is getting in the way in
13:02 such moments to see how much better the
13:04 ps5 is actually running than 60 fps here
Last edited:
D
Deleted member 86764
Guest
Based on the evidence provided in that video, I'd say that any conclusions can, at best, be described as inconclusive. No matter the platform.
Like the earlier one, I also wouldn't conclude that the 4Pro was the same as the XSS, as the 4Pro performance appears to be due to frame interpolation.
Like the earlier one, I also wouldn't conclude that the 4Pro was the same as the XSS, as the 4Pro performance appears to be due to frame interpolation.
snc
Veteran
yeah, 5700xt or 2070super droping frame in mendoza level using sniper or near folliage and we could create different narration
what's inconclusive exactly? That the PS5 cannot run at a higher resolution than the current setting?
There's no conclusions to be made really, 1800P is the setting, that is the conclusion. Alex offers some explanations really.
Next week's digital launch of Control Ultimate Edition for PS5/XBSX are going to be interesting to contrast and compare to each other and PC hardware.
D
Deleted member 86764
Guest
I'm not sure what you're suggesting here.
The video shows a major drop that I think we can all agree is due to bandwidth limitations. During that section of the video there is the following graphic:
While this heading refers directly to that one scene, it can lead one that's not so technically minded to assume that there's a 5% rendering deficit between the 2060S and the PlayStation. Only one other minor drop is reported on the PC that can easily be attributed it using a higher reflection level, as you've pointed out.
No other drops are reported.
We do not have enough data to make conclusions, but this has been described as the best demonstration of the difference between PC and consoles.
This is horrid, because I genuinely love the work that DigitalFoundry do, so really it's a minor criticism to an incredible body of work. They're the best at this.
Technical information is often best just 'written and read' as to not lose the audience on the actual graphics themselves. That I agree with. Can't do squat with a power point presentation imo.
But if people are getting mislead, that's a comprehension problem for them (rather for a lot of folks, most people would rather watch than read now a days). He's not purposefully hiding the title, it's the largest font on the screen, front and center, white against black and he speaks to it in the voice over.
It's described as one of the best console to PC comparisons because he was given the exact settings like for like. He can do nothing about the lack of DRS or the vsync 60.
Last edited:
D
Deleted member 86764
Guest
Based on the evidence provided, including that graphic, is a misnomer. Concluding the relative performance of any of the consoles based on the data provided in this video is incorrect.
hmm.. that graphic is not a misnomer by definition of the word.
That graphic is very specific. It would be a misnomer if that performance on that graphic carried through outside of 'During Miami Introduction Scene'
D
Deleted member 86764
Guest
How would *you* define the relative perfomance of the two consoles based on this video?
- Status
Similar threads
