No DX12 Software is Suitable for Benchmarking *spawn*

Razor1 said:
I wasn't saying you were, I'm saying we have to start thinking of other theories, cause it doesn't look to be a singular problem




There are three contrapostion arguments not just one, One is API another is Platform, the three is Drivers. All have to conclude the same thing for the one proposed to be true
Click to expand...
Agreed. But the original argument was that testing on Nv GPU in DX12 was showing Ryzen in worse light, relative to 7700K. This was raised by Adore and i see TechSpot's article as an attempt to dig into that. Of course he also did other test as well.
He couldn't make an assertion because not all the data fit into what he thought was the original case.
Click to expand...
True. But he made opposite assertion that is also not justified (as shown in my graphs) saying there is not evidence to support original claim. He even said he will exclude some of the titles in the future due to weird results he could not explain. So his conclusion is contradicting his own test results.
 
HOCP tested several DX12 games with the GTX 1080/Ti and RX 480.

It seems that for the most part DX11 is still the "better" option, in terms of overall performance. However, there are a few games finally catching up. The game with the most potential is actually the most recent game, Sniper Elite 4. You should definitely run this game in DX12 with Async Compute on every video card. In terms of the other games experiment, you may find The Division and Hitman do well in DX12 on your system.

What we’ve learned from this evaluation is that some games just haven’t improved in the DX12 department. However, newer games with more recent DX12 support are finally showing some measure of improvement. We hope to see this trend continue and perhaps games coming out later this year will be even better with DX12 performance. DX12 gaming finally feels like it is starting to become a reality, and boy has it taken forever to arrive.
Click to expand...

https://www.hardocp.com/article/2017/03/22/dx12_versus_dx11_gaming_performance_video_card_review/11

Hitman results were interesting, as perhaps for the first time, DX12 is now better than DX11 on NV hardware. In fact in several reviews the 1070 is actually faster than FuryX in DX12, which never happened before. I took care to present reviews which didn't rehash old results and used the final DX12 driver (378.78) from NV.

Hitman_Average_FPS.png


http://www.eurogamer.net/articles/digitalfoundry-2017-nvidia-geforce-gtx-1080-ti-review
http://techreport.com/review/31562/nvidia-geforce-gtx-1080-ti-graphics-card-reviewed/10
http://www.techspot.com/review/1352-nvidia-geforce-gtx-1080-ti/page3.html
http://www.tomshardware.com/reviews/nvidia-geforce-gtx-1080-ti,4972-3.html
https://www.purepc.pl/karty_graficz...ti_gaming_x_pascal_bardzo_wypasiony?page=0,12
 
Last edited:
mrcorbo said:
You can't test everything. The best you can hope for is to present a representative sample. 4 games from one publisher on a short list is not a representative sample. I tried to take your list and turn it into a representative sample which, yes, includes games that are already being used for benchmarking and I think my resulting list is better than yours.
Click to expand...

DavidGraham said:
But you did the same, Hitman and Tomb Raider are from Square Enix, add that to Deus Ex Mankind Divided (which is also widely tested), and you have 3 games from the same publisher, 3 is no different from 4, no?
Click to expand...

I didn't include Deus Ex and Tomb Raider could be replaced with something else. How about Gears of War 4? Hitman is a good title to benchmark, though, as they have included some DX12-enabled features in their engine that I think are interesting and relevant.

DavidGraham said:
Also it doesn't matter which games are from which publisher, the only thing that matters is that they are recent, popular, graphically intensive or all of those.
Click to expand...

I disagree. Publisher-owned studios often share tech. 4 benchmarks covering the same basic engine is redundant. Removing older games from your benchmark suite also removes historical context from your benchmark numbers. If you want to track how driver improvements and architecture improvements change performance over time having a constant set of games is vital.

DavidGraham said:
My argument is to expand your list (10+ games) and be versatile, I won't adhere to certain names or specify ones that should be tested, what matters is that you take a diverse enough sample to cover most workloads to truly represent the complete picture.
Click to expand...

That's not practical. There are time constraints. Finding a way to devise a repeatable benchmark for some games is a challenge and the execution on some can be challenging in a way that takes even more time. That's why games with built-in benchmarks tend to be favored by reviewers.

There are good and valid reasons that reviewers establish a benchmarking suite and stick with it over time. When they do change things up, they often make big sweeping changes, because the intention is for that process to be consistent for a period of time.
 
mrcorbo said:
I didn't include Deus Ex and Tomb Raider could be replaced with something else. How about Gears of War 4?
Click to expand...
Yeah Gears 4 have good DX12 tech as well. Sniper Elite 4 is another one. Forza 6 or Forza Horizon 3 is another fantastic candidate.
mrcorbo said:
I disagree. Publisher-owned studios often share tech. 4 benchmarks covering the same basic engine is redundant.
Click to expand...
It depends, Hitman, Deus Ex, Final Fantasy 15 and Tomb Raider all use completely different engines. So does Watch_Dogs 2, Ghost Recon Wildlands, Division, Assassin's Creed and For Honor. Fallout uses a different engine than Doom, etc. Every publisher has multiple studios that use their own technology and engines. Some like to unify these (EA likes to use FrostBite everywhere, and so does RockStar with their RAGE engine). But others use completely different tech.

mrcorbo said:
That's not practical. There are time constraints.
Click to expand...
It is practical, several sites do that already, TechSpot, ComputerBase, TechPowerUp, PCGamesHardware and others.
mrcorbo said:
There are good and valid reasons that reviewers establish a benchmarking suite and stick with it over time. When they do change things up, they often make big sweeping changes, because the intention is for that process to be consistent for a period of time.
Click to expand...
If they will not be adding new games, or changing the line up to reflect a more modern situation, their conclusion becomes invalid or worthless. It's a risk worth taking.
 
DavidGraham said:
It is practical, several sites do that already, TechSpot, ComputerBase, TechPowerUp, PCGamesHardware and others.
Click to expand...

Not if you prefer a deeper analysis of each result.

DavidGraham said:
If they will not be adding new games, or changing the line up to reflect a more modern situation, their conclusion becomes invalid or worthless. It's a risk worth taking.
Click to expand...

They do change them. But only when change is warranted.

Also, I found the contents of this graph really amusing in the context of this discussion.

GameComparison.png
 
Was a 580 always 30% ahead of a 1060? Even Vega64 only needs 10% to catch a 1080Ti with those numbers.

DX12 seems consistently worse across vendors, which isn't surprising as it's likely tacked on. Still wish they tested DX12 with lesser processors than a 6900. Leave somewhat of a bottleneck that typical gamers would encounter. Throw in a Ryzen 1600 for instance.
 
Clukos said:
RotTR latest patch, Dx11 vs Dx12 in CPU bound and GPU bound areas(1440p very high preset)
Click to expand...
Wow, these are mighty impressive gains right there, didn't know Tomb Raider could yield such great increases under CPU bound conditions in DX12, though the DX12 path is marred by being inferior to DX11 image quality wise, as VXAO only works in DX11 and does wonders for shadows quality.

FWIW, I couldn't replicate your gains in The Division on my 1070 and i7 3770, so I am left wondering if it's a Ryzen thing?
 
Last edited:
You must log in or register to reply here.

Similar threads

R
id Software releases Open-Source Wolf3D iPhone port
Replies
13
Views
6K
Ailuros
Ailuros
O
Unreal creator Tim Sweeney: "PCs are good for anything, just not games”
3 4 5
Replies
91
Views
18K
Mobius1aic
M
A
Full transcript of John Carmack's QuakeCon 2005 Keynote
3 4 5
Replies
98
Views
34K
nemisis
N
Back
Top