Metacritic in 2014

[Rodéric]

I agree. The basics for www.techingames.net to support an experiment are in place - you can add reviews, and the review average is shown. Now it's a matter of adding a lot of reviews to a few good test games and then adding some nice filters. The question is of course how to best do that. Perhaps it would be best to start with adding your own reviews / scores for a few typical games, and then start matching those against existing review scores to see who you align with. But as we have game features in there, we could also try to predict what new games you may like that share features. Perhaps by also adding an option to rate features? And that could also help predict if a game becomes more valuable after certain features have been added in a new version, or the impact of certain issues that a game is experiencing to at least temporarily suggesting to steer clear from a game. For instance, if you're all about online, then MCC and DriveClub were pretty much no-go areas for a long while, but if that's a minor thing to you, then they were great titles for you at launch.

Being able to check or rate features is useful to sort the list that the website generated for you based on your reviews and other critics reviews matchs. (Also note that there's no reason you shouldn't match a user with another user...)
Trying to use features seems at least delicate, so I wouldn't try it from the get go. (Depending how you explain it, if it's just rating how much the feature mattered in your game review, it might just work out fine, the automatic matching system would take that into account. People should also be able to use features as search criterium when they are browsing the list, to allow them to only see, say couch coop game.)

 

[Arwin]

It could make for interesting standardization and future proofing reviews. Say that you knock a point off for the framerate, and a future patch solves the framerate issues, then the score for the game could be updated in a later version. It's a lot of work of course and some people would get bored, but it could end up really interesting. But it may be something for the long term. Right now I'll probably also need to focus on how to get more people interested in contributing content.

I'm definitely considering looking at making an odata2 entry to the site to allow for developing desktop and smartphone data entry and consumption features.

 

[joker454]

One thing I'll add to this in case anyone here does decide to make their own review website, is to add "Price Paid" to the review listing. That is one thing Steam is missing and to be honest is quite a critical piece of information. Because I suspect that if someone paid $60 for a game then they will be far more critical of it compared to someone that has paid $5 for it. I think that could be one of the reasons I tend to be less harsh on games and often enjoy some games that others hate, because I usually pay $2 to $10 for games now so it's easier for me to feel entertained compared to others that paid a ton for a game that are more likely to feel ripped off. Pricing also means I don't feel inclined to force my way through a game to justify what I paid for it, instead I just stop playing the minute I've lost interest which also means I'm less likely to feel frustrated by the game. Some guys that paid $60 for a game slog through it in it's entirely even if they aren't having fun, almost like they feel obligated to do so because of the high price they paid for it so it's not surprising that some of those people will be less favorable to a game by the time they are finally done with it. It's why I feel pricing along with hours played are both critical to any review. Steam really needs to add that info to their reviews.

 

[Rodéric]

Makes sense indeed.

 

[Arwin]

Not a bad idea. I am working on a user values section that allows users to enter values that should be tracked at the user level. So this could be independent of reviews. For instance I am also planning to add 'own, started, finished, 100%-ed', or preferably, make it customiseable like the features section (with which I may still integrate it) so that anything can be tracked.

 

[Billy Idol]

Interesting concept, but it'd require a lot of knowledge from the users to select their trusted sources. You'd effectively have to learn all the sources and even individual users. Perhaps a better way would be to put in your own ratings for games you've played and the system finds reviewers who scored similarly. Where there is a pattern between a reviewer's (site's) scores and a user's, that reviewer (or site) could be added to the 'trusted' list and used for future scores for unowned games.

Hmm, that's actually a pretty awesome idea. It'd work just as well with movies and other stuff. Effectively a 'find like minds' process. If applied to something like a social network (Facebook), you could produce a 'similarity index' between users, and use that to generate recommendations. That'd be a trillion times better than some of the current category based or simple popularity based recommendations we see.

This idea is indeed quite awesome imo! The more games you play, the more scores you enter yourself, the better this approach should predict your personal score for a new game. I like it! And this is also quite interesting scientifically: does this approach really converge, i.e. does the prediction fit better with your personal perception? Or is there even some influence: Shifty's scoring predicts a high rating for this new shining game XXX...what are my expectations and how does this change the score you put in after you played the game...cool!

 

[Billy Idol]

Not a bad idea. I am working on a user values section that allows users to enter values that should be tracked at the user level. So this could be independent of reviews. For instance I am also planning to add 'own, started, finished, 100%-ed', or preferably, make it customiseable like the features section (with which I may still integrate it) so that anything can be tracked.

I am really looking forward to this!

 

[Arwin]

At a basic level, it's now up and running. Very basic though, but it works. User data is treated differently (kept in a different table), but still shows up along with the rest of the features of a game (version) and can be tracked the exact same way. Except that here your values are only your values, and not the game's specific values, and one game version feature can have 1000s of different user values.

 

[Silent_Buddha]

One thing I'll add to this in case anyone here does decide to make their own review website, is to add "Price Paid" to the review listing. That is one thing Steam is missing and to be honest is quite a critical piece of information. Because I suspect that if someone paid $60 for a game then they will be far more critical of it compared to someone that has paid $5 for it. I think that could be one of the reasons I tend to be less harsh on games and often enjoy some games that others hate, because I usually pay $2 to $10 for games now so it's easier for me to feel entertained compared to others that paid a ton for a game that are more likely to feel ripped off. Pricing also means I don't feel inclined to force my way through a game to justify what I paid for it, instead I just stop playing the minute I've lost interest which also means I'm less likely to feel frustrated by the game. Some guys that paid $60 for a game slog through it in it's entirely even if they aren't having fun, almost like they feel obligated to do so because of the high price they paid for it so it's not surprising that some of those people will be less favorable to a game by the time they are finally done with it. It's why I feel pricing along with hours played are both critical to any review. Steam really needs to add that info to their reviews.

Indeed, there's a Youtuber I watch somewhat frequently that doesn't do traditional reviews of games. He mostly gives first impressions and always with a warning that his views of a game or genre may not match your views of a game or genre because you value different things.

But one thing nice that he does is give his opinion of a game based on it's price point compared to what he values in a game.

So he'll often go that at XX price that the game is currently at, he'd have a hard time recommending it unless you really value the things the game has to offer. But if the game was at a lower price point of YY, then he'd have no problems recommending it to people. And that if it ever drops to ZZ price during a Steam sale, it's a no brainer to grab it and give it a go.

That is, of course, when he doesn't give it his whole hearted recommendation (rare) or condemnation (less rare).

Regards,
SB

 

[msxyz]

When I visit Metacritic, I enjoy looking at the discrepancies between the average of the 'official ratings' and those given by users.

You can tell a good game from an over-hyped, publisher shoved piece of crap looking at this delta.

...and before somebody mention the fact that most user reviews are either 0 or 10, I consider them more a binary answer like/don't like so when the total user reviews are in the order of hundreds, or thousands the user score is still meaningful albeit a bit different in meaning from a true rating.

 

[Nesh]

When I visit Metacritic, I enjoy looking at the discrepancies between the average of the 'official ratings' and those given by users.

You can tell a good game from an over-hyped, publisher shoved piece of crap looking at this delta.

...and before somebody mention the fact that most user reviews are either 0 or 10, I consider them more a binary answer like/don't like so when the total user reviews are in the order of hundreds, or thousands the user score is still meaningful albeit a bit different in meaning from a true rating.

The difference though is that in truth it is not even a binary answer.
There are users who barely play (if at all) a game and give it an 0 or a 10.
For example some pissed off Nintendo guy might give a 0 to Dragon Age Inquisition because it won a GOTY award instead of Bayonetta 2
A GT guy might give Forza a 0 because its a competing franchise while the Forza guy might do that in reverse
A PS guy might give Killzone a 10 because its a "Halo Killer" and Halo a 0, while an XBOX guy might give Uncharted a 0 and Halo a perfect 10.
I believe Titanfall was partly sabotaged by zeros since it was supposed to be a flagship XB1 exclusive during a time when fanboys were "fighting" which console is the best. Its user score is 6.2 but I am pretty sure this could have been something closer to 7 or 7.5


There are ton such examples because unfortunately the user reviews are infested by "fanboys" who pretend that they have played a game.
The only way to get an idea is to actually read some of the user reviews and pick those that at least appear to have some basis. But then again that involves some guesstimates. Some people are looking for generic complaints in forums or negatives in reviews in order to add credibility to their unsubstantiated user score.

Of course there are examples where I agree with you that user reviews DO show a better idea about the game's appeal.
One such example is COD:Ghost which is multiplatform. It got a 78 in reviews but the user score is below 5/10.

But curiously since there are more people arguing about XBOX vs PS, the Wii U user scores are more in line with the metascore. Since the Wii U is considered a "failure" less fanboys from competing camps bother to "sabotage" reviews for the console so the user scores are left more untouched by fanboys and touched by people who actually own and play the games.
For similar reasons niche games have metascore and user review averages that are closer (see Guilty Gear Xrd) since those that bother to review are people who were interested genuinely and played the game while at the same time reviewers dont have pressure for publishers to make favorable reviews

 

[Deleted member 11852]

Yup. For the most part, user scores on sites like Metacritic are a complete nonsense and tell you nothing other than how many idiots there are playing games.

 

[Sigfried1977]

One thing I'll add to this in case anyone here does decide to make their own review website, is to add "Price Paid" to the review listing. That is one thing Steam is missing and to be honest is quite a critical piece of information. Because I suspect that if someone paid $60 for a game then they will be far more critical of it compared to someone that has paid $5 for it. I think that could be one of the reasons I tend to be less harsh on games and often enjoy some games that others hate, because I usually pay $2 to $10 for games now so it's easier for me to feel entertained compared to others that paid a ton for a game that are more likely to feel ripped off. Pricing also means I don't feel inclined to force my way through a game to justify what I paid for it, instead I just stop playing the minute I've lost interest which also means I'm less likely to feel frustrated by the game. Some guys that paid $60 for a game slog through it in it's entirely even if they aren't having fun, almost like they feel obligated to do so because of the high price they paid for it so it's not surprising that some of those people will be less favorable to a game by the time they are finally done with it. It's why I feel pricing along with hours played are both critical to any review. Steam really needs to add that info to their reviews.

For me it's exactly the opposite. If I pay full price I usually try and get as much out of any game as possible. This also means begrudgingly accepting certain flaws. If I never paid a dime, I'll gladly rip into any game as long as it's deserved. I think this is one of the primary reasons why user reviews tend to come in significantly higher than press reviews on average. The urge to defend something isn't quite there when no money is involved. Then there's of course all the fanboys who uprate or downrate games because they feel like the average isn't quite to their liking. Heck, they usually even admit to it in the text.

The gaming press, flawed and borderline corrupt as it may be, is still nowhere near as useless as your average user review.

 

[Arwin]

As mentioned in the techingames thread, I now have user filter / score modifier options running, allowing you to score correct individual publications as well as reviewers. Multiple filters can apply at once. I'll add user reviews next I think.

 

[Arwin]

- you can now add user reviews to individual game versions, so you can write an updated review for a later version. The current average ranking takes all reviews, so if you started with a bad version on release, but the game got better with newer versions, your aggregate game score will still be lower than when you released the game in that later state. I think I'll eventually change that to only take the score for the highest version number.

- you can now add user reviews to publications, so you can review a site

- you can now add user reviews to reviewers, so you can now review a reviewer

 

[Arwin]

I spent a fair bit more time polishing this. You can now review any feature for any game specifically, and I've integrated the aggregate scores for these features in the overview, both on the level of the feature itself and on the level of the category (e.g. Controls, Graphics).

Clicking on a user review for a game(version) will also show you all the feature reviews, as well as all user values for the game (like price paid, started, campaign finished, completed, etc.), so I got very close to most of the suggestions here.

Looking at this more and more, I think this is the best thing I have so far that comes close to being a 'game changer' in the way reviews work. Being able to track and then review individual features really opens up a lot.

 

[Prophecy2k]

I spent a fair bit more time polishing this. You can now review any feature for any game specifically, and I've integrated the aggregate scores for these features in the overview, both on the level of the feature itself and on the level of the category (e.g. Controls, Graphics).

Clicking on a user review for a game(version) will also show you all the feature reviews, as well as all user values for the game (like price paid, started, campaign finished, completed, etc.), so I got very close to most of the suggestions here.

Looking at this more and more, I think this is the best thing I have so far that comes close to being a 'game changer' in the way reviews work. Being able to track and then review individual features really opens up a lot.

Excellent work Arwin. Well done