Value of Hardware Unboxed benchmarking

If the data you need is not in their video then you look at another video or online article.

This has become a common refrain recently for some reason. If you don’t like it don’t watch it. However product reviewers arent politicians and this isn’t CNN vs FOX. Reviews are not supposed to be propaganda pieces that are only consumed and enjoyed by people with similar biases. They’re meant to provide reasonably objective data and recommendations and inform the public who don’t have access to the same testing resources.

If that objectivity and scientific rigor is missing the community has every right to talk about it. It’s scary that people think it’s perfectly ok for there to be no standards in GPU reviews. Times certainly have changed.
 
Are you ready to base all your purchasing decisions on their performance numbers alone?

HUB’s numbers are perfectly fine. But they’re also drawing conclusions and making recommendations based on those numbers. It goes for a toss if you manipulate the testing to produce a desired outcome.

Take the conclusions from the article that spawned this thread.

The 7900 XTX is competing with a far worse value product in the RTX 4080, and while the Radeon GPU is cheaper, it's also generally inferior when it comes to ray tracing performance, it still lacks a true upscaling competitor to the widely supported DLSS, it sucks down quite a bit more power and driver support isn't as mature with a few strange niggles here and there.

How can the 4080 be both a “far worse value product” while providing superior RT, upscaling, driver support and power consumption for $200 more? HUB are contradicting themselves here.

We also think moving forward ray tracing performance might be less of an issue, at least if what we're seeing in Fortnite is anything to go by. Sure, there will always be RTX titles sponsored by Nvidia where Radeon GPUs get slaughtered, that's just how the game seems to be played, but overall we think AMD's ray tracing performance is now in more solid ground than before.

This one is really special. AMD’s raytracing performance is 100% on more solid ground now. What is that nonsense though about Fortnite and sponsored RTX titles? Anybody reading that would assume raytracing is some garbage being forced on developers instead of something they’ve wanted for decades and are super excited about. Nope every developer is going to forget all about that because of Fortnite. Sheesh.
 
It's not FUD.

Their performance numbers and frame rate data is solid and perfectly usable by people to gauge performance of a certain GPU in a specific game and then to make a choice based on that data.

I think if you look at each individual benchmark in isolation then this is a perfectly fair point. My issue with the article is quite similar to my issues with some of NXG's articles in that it's the conclusions being drawn from the data presented that are the issue rather than the raw data itself.

In this case that manifests as him drawing a more generalised conclusion of the overall average performance position of the two cards relative to each other which he essentially says is the same. He even further justifies the validity of that claim by pointing out how 'huge and thus reliable' his test set is and that he "of course" included some RT games in there.

The thing is, had he simply tested all games in his own test set once - at their maximum settings as you should be able to do on a $1000 GPU then the average performance position would have been wildly different and that would have necesarily had to impact his final conclusion as well that the XTX is the better choice.

For the record I'm not saying I necessarily disagree with him. I've made my feelings on the 4080's pricing situation perfectly clear (not that I hold the XTX in any higher esteem). My only issue here is how the general conclusion has been heavily skewed by the very particular mix of games that seems to have been specifically crafted to reach that conclusion. I'd absolutely be just as critical of a review that did the opposite, say in only testing heavy RT games at max settings and drawing a "4080 is x% faster than the XTX in our super representative and balanced test set" conclusion from that.
 
HUB’s numbers are perfectly fine.
I can argue that they are not tbh as their numbers are usually quite a bit different from those I see in other outlets. But okay.

But they’re also drawing conclusions and making recommendations based on those numbers. It goes for a toss if you manipulate the testing to produce a desired outcome.
Yes, this. Which is why using HUB numbers alone is completely misleading for any sort of your buying decision. When you look at other reviews they all tend to produce similar data with similar conclusions - and only HUB manages to produce something unique here on a regular basis.
Can it be seen as "FUD"? Well, yeah, because people see this and go "well, everyone is saying this but Steve says!.." In this case it would be "but Steve from HUB says that RT h/w doesn't matter anymore!"
 
That's how they make money. If they just being bland and objective, they would get less views, less engagement, less chance for the algo to push their videos.

Their genius is in the crafting the perfect selection of games and the conclusion they want to have the most views.

Not many youtubers have that kind of skill.


Edit: I'm not saying that there are no market for drama free objective videos. But IME as a small time youtuber that got demonetized by Google with zero warning (I'm still salty that they refused to provide specifics of what wrong), drama free objective videos get less views
 
I would be happy for you to provide some examples.
I don't see the point really. Examples are available to everyone on the internet.
Here's one from today's 4070Ti review:
halo-infinite-2560-1440.png
1672853726887.png vs
Halo_1440p-p.webp


I don't know why this happens with HUB results and it is very likely to be due to some settings being different or Steve using some completely unique benchmarking scenario. But I've been noticing these for a while now.
 
My natural instincts are to disagree with DegustatoR, but I unsubscribed to HUB a few months back because of my disappointment with them and I haven't regretted it.

I put Steve and Jayz on a whole other level than HUB, a much better level.
 
I don't know why this happens with HUB results and it is very likely to be due to some settings being different or Steve using some completely unique benchmarking scenario. But I've been noticing these for a while now.
Funnily enough, HUB has covered this exact game in a video explaining why different media outlets get different test results in their game benchmarks:


So from this one example you brought up, it looks to me like they have a valid explanation for differing results and your suspicious attitude is proven unfounded. But maybe you have many other examples.
 
My natural instincts are to disagree with DegustatoR, but I unsubscribed to HUB a few months back because of my disappointment with them and I haven't regretted it.

I put Steve and Jayz on a whole other level than HUB, a much better level.

Agreed on the first, but disagree on the second haha.
Gamer's Nexus (Steve) is the gold standard. Jayz is pretty terrible though.

This whole thread reeks of cringe. It's like the Apple fanboys who freaked out at HUB when they benchmarked the M1 Pro equipped MacBook Pro.
 
I don't see the point really. Examples are available to everyone on the internet.
Here's one from today's 4070Ti review:
halo-infinite-2560-1440.png
View attachment 8030 vs
Halo_1440p-p.webp


I don't know why this happens with HUB results and it is very likely to be due to some settings being different or Steve using some completely unique benchmarking scenario. But I've been noticing these for a while now.
Different areas of a game have different performance characteristics.

You can use the Doom Eternal Raytracing part, too. Having found another review in which a 7900XT gets over 500FPS in 1440p.
The 500 FPS is with no RT. They also test in the opening level of Doom Eternal where performance is much higher than in later levels. Sometimes up to 2x as fast on my 1080ti. It also has much fewer metallic surfaces lowering the amt of RT calculations.
 
Last edited:
Different areas of a game have different performance characteristics.
The problem is that most other review outlets provide close results while HUB is constantly giving out numbers with a completely different cards positioning. This can't be easily explained by the fact that different areas of games have different performance characteristics - especially since it's not really that often that different areas are moving the averages so much that cards exchange their places. The most common scenarios for such cases are areas which are GPU limited vs areas which are CPU limited - and it's weird to use the latter when benchmarking GPUs, no?
 
The problem is that most other review outlets provide close results while HUB is constantly giving out numbers with a completely different cards positioning. This can't be easily explained by the fact that different areas of games have different performance characteristics - especially since it's not really that often that different areas are moving the averages so much that cards exchange their places. The most common scenarios for such cases are areas which are GPU limited vs areas which are CPU limited - and it's weird to use the latter when benchmarking GPUs, no?
DF videos have shown plenty of times that performance advantages between competing GPUs swing back and forth depending on the scene. HUB even made an entire video showing unarguable evidence that their Halo Infinite numbers are legit. They also explained in a video years back why their Doom Eternal numbers were different. Many benchmarkers go and clear out all the enemies before doing a run through, HUB does not. With enemies present Nvidia GPUs performed better relative to AMD. With no enemies AMD cards took a noticeable lead.
 
DF videos have shown plenty of times that performance advantages between competing GPUs swing back and forth depending on the scene.
Plenty? I can't think of any examples over the last year? Could you point me to them?

HUB even made an entire video showing unarguable evidence that their Halo Infinite numbers are legit.
Which makes you wonder if they actually are even more tbh. When you need to make a video to explain why your benchmark results are different from most other outlets then this in itself is a problem.
 
Which makes you wonder if they actually are even more tbh. When you need to make a video to explain why your benchmark results are different from most other outlets then this in itself is a problem.
A problem which surprisingly comes down to viewers/readers not thinking about different test scenes making a difference in the results. Hardwareunboxed does show their test sequences during the reviews, and many other sites do too. What more are they supposed to do?
Not having a huge headline saying their results may differ because of using different scenes is something many others also could be criticized for.

And that is why saying what the tests are is one of the key aspects of testing. It's basic stuff in any kind of testing, and that's when we possibly can speak of a de facto standard, you describe/show what you did and what tools you used, so if anyone finds the results questionable, they can repeat it in order to verify the results.

And that's the big negative I constantly see in the rants against HUB...Sure, accuse them of having a test suite that favour AMD, that can be a real discussion, but the constant accusation of sloppy tests and wrong results, people should stop with that unless they actually can show it.

At least some of the new posts here discuss more how HUB possibly can come to this and that conclusion after saying/showing that earlier, rather than these attacks about the tests being faulty.

To be honest though, I have never prioritized the various authors' own conclusions. I focus on the actual tests and draw my own conclusions, using a mix of multiple different sites.
 
He admits his fanbase that always influences his decisions are mostly AMD fanboys, I just don't know why he would stoop so low to destroy his credibility by appealing to them. He doesn't have to. He can be semi neutral while also presenting facts without the need to twist the truth.


This was in response to:


I.m surprised he admitted it publicly.. amd in general has a big only presence
 
Agreed on the first, but disagree on the second haha.
Gamer's Nexus (Steve) is the gold standard. Jayz is pretty terrible though.

This whole thread reeks of cringe. It's like the Apple fanboys who freaked out at HUB when they benchmarked the M1 Pro equipped MacBook Pro.
Gamers nexus is bad too.. he tries to be gunny but is pretty bad very arrogant smug guy
 
Back
Top