*spin-off* Activision's Call of Review Hero: Controversy: Spin-Off

[Scott_Arm]

I'm not sure that there are a limited number of words. If we're talking about the 2 or 3 physical magazines on the market, I'll actually give them a pass. Big review sets don't use paper, though, there's no such limit.

As to where we stop, well, we could use a set of consistent criteria. Is AA important? Is AF? Framerate? Texture quality? General image-quality (a catch-all that would catch blur filters and resolution, since pixel counting may in fact be too much to ask). None of these seem particularly controversial. There could be more. And as to people actually bringing it up, this is where objectivity is important: fanboy gloss gets in the way of an accurate analysis. GTA4 was not panned for its framerate, which as grandmaster has found out, was very very janky (especially on PS3). People didn't see it because of that fanboy gloss.

If all reviews were written by completely reasonable people then okay, maybe objectivity wouldn't be such a big deal. But very often, especially for really hyped games, reviewers just overlook issues. And again, not because they're dishonest, but because their enthusiasm for the game makes them overlook these things. And so when people aren't behaving rationally, that's when a set of objective criteria to discuss around helps matters. It would help ground the reviewer (hopefully) and it would allow the buyer to make a more informed decision. Not everyone who cares deeply about graphics is on B3D -- a lot of people are actually thankful about Lens of Truth or Eurogamer's analysis, even though they show up well after the game's release.

For people who don't care about such things, well, they can gloss over the numbers and just get the conclusion. It's like saying that reviews shouldn't have text because most people just want the score.

I played through GTA4 and the first expansion DLC. I don't think the frame rate was panned because it didn't impact the game as far as I could tell. I have no idea what the frame rate is, and don't really care. Sure, if it was 60fps it would be really nice, but it's not and it doesn't really matter, at least to me. If the review had said it dropped to 10fps during big explosions, I would have bought it anyway. That number, in itself means very little.

How are you going to set a criteria of what is important, and how do you make any kind of statement about each of those things without it being entirely biased? This goes back to the idea that you can analyze any of those things in the context of a game without making a biased judgement. I think you'd be hard pressed to find anyone that could make meaningful conclusions about those things that would accurately express how good a game looks or plays, especially without introducing personal bias. So the problem of fanboyism wouldn't be solved.

And to say that people glossed over the problems in GTA4 because of fanboyism is a little bit conspiracy theory to me. Mostly I think the problem was the way the reviews were conducted, which wasn't disclosed to the readers. The real problems in the review industry and kickbacks to mags from publishers which puts pressure on writers from editors and owners, or other such conflicts of interest.

When you get into criteria for grading graphics, you still have the problem of overall impression vs check list. You can have or not have depth of field. Does depth of field inherently make a game look better? What approach to depth of field is best. Does a game score points just for having it, or is there some qualitative measure that determines whether it is a value add, or not? Is it better in one type of game, and not another? You're going to end up with a huge list of criteria, and what exactly is the end result going to tell us? I'm not even talking about delving into nitty gritty coding level discussion. Just the feature itself. It isn't cut and dry how a criteria about any of the new buzzwords would work. I think you're asking for something that is far more complicated that it would seem, and it wouldn't provide any value, or be any more unbiased than a general impression of "graphics."

 

[specwarGP2]

Graphics scores should be relative to games that are already out. Scores should also reflect a technical component. So a game that runs at 60fps or is an open world game wouldn't be penalized because their graphics naturally don't look as good as a 30fps game or one that is linear.

I think a 10 for graphics should only be awarded when that game is the best looking game at the the time of its review. So KZ2 at the time was the best looking game so it deserves a 10. Compared to KZ2 (10), MW2 could be a 7. since its 60fps, I'd give it a extra .5-1, for a total score of 7.5-8 for graphics.

 

[Scott_Arm]

Graphics scores should be relative to games that are already out. Scores should also reflect a technical component. So a game that runs at 60fps or is an open world game wouldn't be penalized because their graphics naturally don't look as good as a 30fps game or one that is linear.

I think a 10 for graphics should only be awarded when that game is the best looking game in its class at the the time of its review. So KZ2 at the time was the best looking game so it deserves a 10. Compared to KZ2 (10), MW2 could be a 7. since its 60fps, I'd give it a extra .5-1, for a total score of 7.5-8 for graphics.

You have to realize that what you're saying is completely unreasonable and unrealistic. I would score Little Big Planet higher than Killzone2 for graphics, so who's right and who's wrong? You have to give up on reading too much into the numbers and read the reviews.

 

[obonicus]

I'll defer on commenting about GTA4 and fanboyism; I don't think it's only GTA4, I think all big games suffer from it, but that's a way bigger rant.

As to criteria, I don't think specific technology should be considered negatively unless it's essentially a commodity. If depth of field next-gen is standard, then yeah, lack of that is a problem. Having it, this gen, though, is a bonus -- if you're scoring a game highly, that's what you'd use. The point is to have reviewers who are trained enough to be able to tell this sort of thing, who actually have the language to express these issues.

And it's not just about preventing bias. I've even pointed out that the reviewer can come to a conclusion completely opposite to their analysis. That's their prerogative. They just have to explain why they came to that conclusion above 'I just like it'. Overall impression is a very very flawed criteria, since as Cornsnake pointed out, it's often tied into the fun you're having. And these are distinct criteria. Again, if gameplay is inseparable from graphics, then they shouldn't be separate criteria.

 

[specwarGP2]

You have to realize that what you're saying is completely unreasonable and unrealistic. I would score Little Big Planet higher than Killzone2 for graphics, so who's right and who's wrong? You have to give up on reading too much into the numbers and read the reviews.

compare them to games that are similar in class then. LBP and KZ2 are very different have nothing in common visually... so it's like an apples to oranges comparison. I can see and accept how someone might think LBP looks better than KZ2.

KZ2 and MW2 on the other hand... that's an apples to apples comparison.

 

[Acert93]

It's not right that MW2 is getting better scores for graphics than KZ2. Sure it's subjective, but cmon... really? Maybe if there was a technical component for graphics scores like obonicus suggests, then MW2 wouldnt be getting 10s for graphics. KZ2 looks like its doing a lot more which enhances the visuals (better particles, animation etc).

And runs at 30fps.

Those who appreciate 60Hz, and believe it is beneficial to gameplay, could argue that 30Hz games like KZ2 have BAD graphics technically because they fail to achieve peak gameplay. So it goes back to reviewer and reader criteria.

No one has suggested an objective method of correlating a score to a technical check list.

Now I won't lament someone giving MW2 an "8" in graphics. But I also can see where someone like Phil could rate a 60Hz game a 10 and higher than a very good 30Hz game based on an objective metric of visual impact in relation to gameplay.

But it is no surprise that the cowboys at B3D continue to circle around the KZ2 wagon.

And for the record, from the little I have seen of MW2, I would say while I dislike KZ2's art direction (and some of their art assets) I like it visually better than MW2. A lot of KZ2's technical decisions resonate with me and a lot of their choices don't break the experience as quickly (although technically I think KZ2 will show its age quicker than more artistic games). My opinion may change when I play MW2 and get more than the first 20 minutes--but I am not even disagreeing that I would rate KZ2 higher than MW2.

Yet from a visual presentation angle in the context of gameplay I can see how someone could argue 60Hz is far superior in terms of technicals and argue the composition of the one screen images is better. It reminds me of Gears of War 1 when I believe IGN mentioned how if they looked around behind corners they shouldn't have gone they could find the rare low resolution texture, but in the context of the gameplay they thought [at the time] the visuals of the gameplay were amazing and had few complaints.

One thing I will consent to is this (never disagreed in fact): If a reviewer finds certain graphical issues hinder gameplay (or the visuals) they should be mentioned. Load times, tearing, frame drops, inconsistant framerate, etc. But this will be subjective--and will change from game-to-game. e.g. It is pretty common to have LOD for grass (e.g. poly foliage to sprite to none). Most times in the past it wasn't worth mentioning--industry standard practice (what, you want ALL games to compare to reality/CGI???) But in Battlefield:Vietnam this became relevant to gameplay because the grass could disappear and while you thought you were concealed someone at distance could spot you. Ditto if your shadows LOD out and allow the easy spotting of players in the distance. Or shadows passing through floors giving away your position (CS:S). There will be subjectivity of whether a visual issue impacts gameplay--or even the visual package.

A classic example of why pixel counting fails as the ends of the means (not means to an end) is aliasing. There are games with no-anti-aliasing that have cleaner edges visually in screen composition than games with MSAA enabled. There are rendering tricks and artistic choices that affect aliasing and it is NOT merely a technical issue. Restricting a score or how 'good' a games visuals are--if they are better or not--based on a rendering implimentation is pretty obtuse.

We should not mistake the screen composition (end result) and the visual impact (subjective to a degree) with the technicalities behind the scenes. These are only implimentation details, the same way movie studios use animatronics, CGI, live actors, and other tricks but the reviewer only cares, "Were the images on the screen convincing and aided the experience" and not "well if you freeze frame you could tell ILM's full GI rendering did X,Y,Z that Pixars Reyes approach didn't." Who cares? A good example is Shrek 1. In the Shrek 2 vid doc they discussed how Shrek 1 didn't do HDR.

Does that technicality matter to the viewer? Are we going to rate games with HDR as "technically superior" to Shrek 1's graphics based on this. Does a movie with HDR techncially better than one without--what if the one without reaches its desired visual target and didn't need it?

I bet Laa-Yosh can submit some low-tech CGI that is visually WAY better than what I can do with superior technology/rendering path. He would be the first to tell you it is about using the right technologies in the right places, cheat like crazy, and there is no substitute for a good artist. I bet he picks good artists over technology every day of the week and 2x on Sunday.

 

[Scott_Arm]

As to criteria, I don't think specific technology should be considered negatively unless it's essentially a commodity. If depth of field next-gen is standard, then yeah, lack of that is a problem.

Why?

 

[Acert93]

As to criteria, I don't think specific technology should be considered negatively unless it's essentially a commodity.

Subjective.

Why does a commodity effect add to the visuals? What if it was neglected not due to technical reasons but artistic. Again, check box mentality as if technical choices have any bearing on graphics.

You sound like reviewers who gave games without Lens Flairs bad points. Or those that marked down a game for not having HDR when it would detract from the visual presentation.

If depth of field next-gen is standard, then yeah, lack of that is a problem. Having it, this gen, though, is a bonus -- if you're scoring a game highly, that's what you'd use.

Subjective.

DOF can be counter intuitive to the visual design. It could not match the artistic direction and essentially be a "Lens Flair."

DOF doesn't work with every game design. I have read a lot of reviews and heard a LOT of gamers slag DOF because while "technically" a check-box it detracted from the gameplay experience.

So you have subjectively identified an effect as a "bonus" and "scoring highly" when it can be a visual and gameplay detriment.

Whether DOF adds to the visual impact (and gameplay) is a game-by-game call.

The point is to have reviewers who are trained enough to be able to tell this sort of thing, who actually have the language to express these issues.

As Scott_Arm noted most quality reviews do mention when graphical issues impact the experience (framerate issues, tearing, graphical affects that are annoying).

But your DOF example and "Commodity Effect" examples do exactly what I was saying: They show how your theory fails in practice.

Full reviews of a handful of games when compared side-by-side would only continue to reproduce examples of why these criteria are (a) Subjective (b) self selecting [as there are thousands of technical decisions to filter through] and (c) and inconsisten in regards to the final character of what we classify as graphics and visual impact.

Grading a game visually up for having DOF and another down for not having DOF is wrong.

And limiting it to the discussion of the text of the reviews, even if we could weed through the THOUSANDS of visual effects, the question becomes not if DOF is used/not used, but whether it even matters. There are clear examples where DOF is detrimental to (a) Gameplay and (b) Visuals.

There is a science to art, but art can never be confined by absolute technical checklists that are universally good in terms of composition.

Overall impression is a very very flawed criteria

The two examples you gave are very flawed as they are subjective and not directly related to the quality of the images.

since as Cornsnake pointed out, it's often tied into the fun you're having.

The complexity of this issue was issued in a recent study / survey I believe which indicated gamers had more fun with graphically GOOD games. The arguement was a lot of "good" games are inferior to less graphically great games. Take an issue like controls where some called it completely subjective (i.e. it could not be wrong).

And these are distinct criteria. Again, if gameplay is inseparable from graphics, then they shouldn't be separate criteria.

Now that may be a more fruitful line of discussion. Not that I agree or disagree.

I take the approach: There are many reviewers of games for the same reason there are many movie reviewers: Different Strokes, Different Folks. You can disagree with the facts, argue the details of why you think a decision was good or not, but you cannot disagree with whether someone liked/disliked something. You may even relegate certain reviewers as outliers and irrelevant (e.g. I don't really care what Famitsu says about games) but it is better to understand their market and their approach before saying it is wrong. They serve an audiance with various desires. Not every consumer wants to be educated when all they want to know is if they are going to like it.

 

[Acert93]

Why?

That is the subjective rabbit hole.

Lens Flares for all!

 

[dobwal]

This is alway a way to bring objectivity to a subjective matter and thats by standardizing the reviews.

The problem with game reviews is that there very open to bias. You don't want reviewers enjoying the gameplay of a title so much that they ignore a visual flaw of a game, only to point out the same type of visual flaw in other titles that they may not have found as enjoyable.

Yes, ultimately if you provide a score to a game its going to be subjective. It doesn't mean that you can't take a more objective stance in how you approach your reviews. Reviews are already broken down by general aspects of a game such as visuals, sound and gameplay. These aspects are readily covered in all reviews. Reviews would be greatly enhanced if those general aspects were broken down even further and those subsets of aspects of a title were routinely part of all reviews.

While how someone experiences a game is totally subjective. A game can readily be described in a very objective manner. Average frame rate, frame rate stability, level of AA, resolution of textures, overall resolution, animation and level of AF can be described in a non subjective manner. There is no excuse to why some game reviewers have a huge lack of objective analysis of the titles they review.

 

[obonicus]

Why?

Because, as you said, how do you judge the absence of something? It's not required, it's not a standard, how can you hold it against a game? If it's there, great. If not, well, move on.

 

[obonicus]

This is alway a way to bring objectivity to a subjective matter and thats by standardizing the reviews.

The problem with game reviews is that there very open to bias. You don't want reviewers enjoying the gameplay of a title so much that they ignore a visual flaw of a game, only to point out the same type of visual flaw in other titles that they may not have found as enjoyable.

Yes, ultimately if you provide a score to a game its going to be subjective. It doesn't mean that you can't take a more objective stance in how you approach your reviews. Reviews are already broken down by general aspects of a game such as visuals, sound and gameplay. These aspects are readily covered in all reviews. Reviews would be greatly enhanced if those general aspects were broken down even further and those subsets of aspects of a title were routinely part of all reviews.

While how someone experiences a game is totally subjective. A game can readily be described in a very objective manner. Average frame rate, frame rate stability, level of AA, resolution of textures, overall resolution, animation and level of AF can be described in a non subjective manner. There is no excuse to why some game reviewers have a huge lack of objective analysis of the titles they review.

Thank you. This is what I'm saying.

 

[mrcorbo]

I'm of the opinion that objective reviews do not, can not and should not exist.

If more people were willing to abandon the concepts of the objective review and the "perfect" 10 and the belief that it is possible to compare two different reviews of two different games (often done by two different people) a lot of the furor around specific reviews would disappear instantly.

To address another point brought up in the thread I would agree that reviewers, broadly, are susceptible to a game's hype and marketing. The reason I don't really have a problem with this is that so are consumers. Perception is reality. If a professional reviewer's opinion of a game after having played it is swayed by factors outside the content of the game itself is it not reasonable to believe that a consumer's opinion of a game would be similarly affected?

With the innumerable sources of reviews, previews and other coverage of games available it should be possible for one to find a source that has tastes and opinions that are consistently similar to one's own. I have found several. Collectively, they provide a pretty robust picture of most new releases and allow me to make informed purchasing decisions.

I still haven't found a site with consistenly good pre-release coverage of sports games, though.

 

[Scott_Arm]

Because, as you said, how do you judge the absence of something? It's not required, it's not a standard, how can you hold it against a game? If it's there, great. If not, well, move on.

Lack of depth of field is a problem if it becomes "standard." Why?

 

[obonicus]

Lack of depth of field is a problem if it becomes "standard." Why?

Because then it's standard. I don't think DoF will ever become standard, but you chose the example. Are sub-HD resolutions a problem? Sure. There's not a single situation in which, on its own, sub-HD resolutions are better than HD resolutions. And this is a valid complaint because HD resolutions were decided upon as a standard early this-gen.

Now, sub-HD may be part of a compromise to try and make the game look better otherwise, which, may or may not be totally worthwhile (this is why I said 'on its own'). Stable framerates are another standard -- 60 fps games are prized, but 30 fps games aren't penalized unless the framerate is unstable. Microsoft made AA standard, which is why everyone complains about jaggies.

 

[Acert93]

I still haven't found a site with consistenly good pre-release coverage of sports games, though.

Probably because (a) featuresets aren't announced until they are locked which can be late-ish in a game with a 12mo dev cycle and (b) sports games are played thoroughly--much longer than their preview time allocates--and many issues don't arise until thorough play testing and (c) game companies want review sites to FOCUS on the bullet-points. You need to play nice with the PR machine.

Publishers have a lot of control. They choose when, what, where, and notibly WHO gets access to games before release. Even pre-release reviews are subjugated to certain conditions even if not vocalized (e.g. a PR department could consider certain companies "unprofessional" or "not worth" an early review copy). This is big business and due to the lack of a single company rising to the top in terms of consumer relations there is no one company that can say, "Look, if you DON'T give us an early review copy and DON'T give us free-Q/A sessions and play tests before launch your sales WILL be hurt because we WILL give this EXPOSURE to other companies."

There is no mandate.

A new media site that is open (e.g. hours played, how directed their Q/A was, etc) with clearer criteria (e.g. reviewer bios with games reviews, favorite games, likes/dislikes, etc) and more open-panel reviews would be a nice change.

 

[obonicus]

If more people were willing to abandon the concepts of the objective review and the "perfect" 10 and the belief that it is possible to compare two different reviews of two different games (often done by two different people) a lot of the furor around specific reviews would disappear instantly.

Comparisons are impossible, but they're necessary, because unlike reviewers, we have limited budgets. We have to pick, among all games, which to consume.

To address another point brought up in the thread I would agree that reviewers, broadly, are susceptible to a game's hype and marketing. The reason I don't really have a problem with this is that so are consumers. Perception is reality. If a professional reviewer's opinion of a game after having played it is swayed by factors outside the content of the game itself is it not reasonable to believe that a consumer's opinion of a game would be similarly affected?

Not really. I might play the game 6 months later, after all the fanfare has died off and the backlash set in. Then the game has to stand on its own two feet without the support of whatever media frenzy helped buoy the game in the first place.

With the innumerable sources of reviews, previews and other coverage of games available it should be possible for one to find a source that has tastes and opinions that are consistently similar to one's own. I have found several. Collectively, they provide a pretty robust picture of most new releases and allow me to make informed purchasing decisions.

But this is essentially the problem; reviewer subjectivity has made it so a single review is just not sufficient on its own. You have to go and look for aggregates and multiple sources to coallesce the reviews together. You have a blob of text for which you have to and find a frame of reference. Caveat emptor and all that, but as consumers we could hope for a higher signal-to-noise ratio.

 

[Acert93]

Because then it's standard. I don't think DoF will ever become standard, but you chose the example. Are sub-HD resolutions a problem? Sure. There's not a single situation in which, on its own, sub-HD resolutions are better than HD resolutions. And this is a valid complaint because HD resolutions were decided upon as a standard early this-gen.

Now, sub-HD may be part of a compromise to try and make the game look better otherwise, which, may or may not be totally worthwhile (this is why I said 'on its own'). Stable framerates are another standard -- 60 fps games are prized, but 30 fps games aren't penalized unless the framerate is unstable. Microsoft made AA standard, which is why everyone complains about jaggies.

If depth of field next-gen is standard, then yeah, lack of that is a problem.

Why?

You failed to explain why, if "standard," having DOF is "good" and why not having DOF is "bad."

You are dancing around the obvious implications it has for the visual impact in the "big picture."

Unless you believe, "standard" will always be "good."

In which case see => Lens Flares.

 

[obonicus]

snip

Joshua, you understood the exact opposite of what I said about commodities.

 

[obonicus]

Why?

You failed to explain why, if "standard," having DOF is "good" and why not having DOF is "bad."

You are dancing around the obvious implications it has for the visual impact in the "big picture."

Unless you believe, "standard" will always be "good."

In which case see => Lens Flares.

I just explained, Joshua. I'm not dancing around anything. If a certain feature is expected[/ B] of games, then not having it is bad, yes. Maybe there's a compromise involved that will totally pay off. You seem to be under the idea that you're springing gotchas on me, when all along I've said that reviews can be subjective while being supported on objective arguments. Not having a standard feature isn't an immediate 'it sucks', but I do expect a reviewer to examine why this standard feature was not included.