Friday, August 16, 2013

Reviewing Reviewers - Explaining wide discrepancies in review scores

This week’s release of Duck Tales: Remastered has been met with polarizing reviews. On one end, some sites, such as, are gushing about pretty much every aspect of the game and offering up near perfect scores. On the other end, some sites, such as, are panning the game. I think that both of these reviews are valid and make solid case for their respective scores. The question is thus raised, what is causing such a rift in the reviews?

A brief synopsis

ScrewAttack’s review praised adherence to the original gameplay while providing slick visuals and a slew of extra bonuses. Specifically, the review mentioned the relatively high difficulty being true to ‘old-school gaming’ and praised the voice-acting featuring the original cartoon show cast. In general, the review came off as extremely excited about the release and contemplated that it delivered on everything it promised and more. Gamespot, on the other hand, agreed that the visuals were slick but panned almost everything else. The difficulty was criticized as unfriendly and not conducive to modern gamers, the story and voice-acting was dismissed as contrived and a barrier to gameplay, and the bonus content was basically described as dull fan service. Overall, the review came off as being unimpressed with the game and suggested that it was a nostalgia trip that has not withstood the test of time. I think what is highlighted in the discrepancies of these two reviews is how much expectations can influence our experience of a game.

Why the difference?

ScrewAttack’s expectations were one of bringing an older game up to modern visuals with a minimum of obtrusive alteration to the original experience. The game is unabashedly a rehash playing on our nostalgic need to relive our childhood. Playing a game with punishing difficulty and quirks in design are all part and parcel of the classic gaming experience. In this case, Duck Tales would be an utter failure if it was a slave to the tenants of the modern platformer. For ScrewAttack, the game far exceeded expectations by delivering the original gameplay in an appealing modern skin.

On the flip side, Gamespot went with the expectation that the game would be a re-imagining of the original for a modern audience. In this case, failure to provide concessions in difficulty and level design meant that Duck Tales had failed as a modern game. While the game, no doubt, should show nods to its roots, it should also serve as a way for newer generation gamers to experience a piece of gaming history. Providing a bit of hand-holding through an easier difficulty does not besmirch the legacy of the original and is not exclusive to putting a ‘Nintendo Hard’ difficulty option for those who want it. Further, tightening controls and improving level design is simply presenting the game the way it would be if it first developed today. All told, for Gamespot, by exclusively catering to the nostalgic gamer, Duck Tales failed to recognize that modern audiences want to play too and thus the game was unworthy of a positive review.

I think both of these perspectives have merit and, depending mood, most gamers will side with one or another. I don’t think that it is fair to criticize either review for “missing the point”; the stance each review takes is valid and the results stem naturally from this stance. In other words, I do not think the conclusion of either review was off-base given the starting point. The difficulty lies, for the reviewer, in choosing what starting point to take. In the case of an all new intellectual property being released in modern times, no reference point exists and, as such, a proper review will invariably compare the game to the modern standard. It doesn’t make sense to compare a new FPS release to Doom and praise all the crazy new innovations it has made. This is true even if the game is designed to emulate a classic game; fundamental aspects of design and controls are expected from modern games and failure to deliver is simply unacceptable to the modern games. Citing classic influence for clunky controls doesn’t make sense as not all old-school games suffered from awful controls. Duck Tales doesn’t necessarily fit the bill (pun unintended) here because it can also be directly compared to the original release. This reference point dichotomy (to compare the game to the original or to the modern standard) is where, I think, that the review score discrepancies are rooted.

Roger Ebert’s review philosophy applied to games

The positive/negative divide in reviews evokes memories of an interview I once saw with Roger Ebert discussing his review philosophy (I have desperately tried to find the source but have come up short). In the interview, Ebert describes how one of his jobs as a reviewer is to develop criteria for success for the movie he is reviewing. While some of these criteria apply almost universally (acting, cinematography, editing, etc.), often movies also need to be judged on their intended audience and within their own genre. For Ebert, a Barney movie aimed at three to five year old might be worthy of a ‘thumbs up’ because it is both engaging and educational to young viewers even if it does not appeal to adults. As such, a well made children’s movie shouldn’t be panned because adults won’t enjoy it (they aren’t the intended audience to begin with!). Although the Barney movie may not be an instant-classic bound for a spot in the Smithsonian, it may be a fantastic example of its genre and thus praise-worthy on those merits. The thumbs up from Ebert is thus considering that the audience will at least exercise some sort of autonomy in selecting what movie they will go see rather than a blanket recommendation for everyone. In short, if you are the sort who thinks the Barney movie will appeal to you, Ebert’s thumbs up says to go for it (and if you are an adult, perhaps seek help).

 I think many game reviewers, if not consciously, apply a similar approach to Roger Ebert. To very broadly make the point, you don’t see a good reviewer docking the latest Grand Theft Auto points because it is an awful real-time strategy (it isn’t a RTS!). More narrowly, I think reviewers recognise that it is unfair to compare certain elements of the new Saints Row with the new GTA; comparisons on shared mechanics such as driving and shooting make sense but a discussion on which plot is superior is mostly irrelevant as the intentions behind them are completely different (SR uses a plot as a vehicle for comedy where GTA typically moves to explore the gritty underside of human behaviour). In this case, stating that GTA has the better storyline than SR is like stating that apples taste better than oranges (which, in fairness, they totally do). One of the few exceptions that would make such a comparison valid would be in a situation where GTA’s plot is objectively measured to dramas and SR’s plot is measured to comedies and the statement is formulated such that GTA succeeds as a gritty exploration of humanity much better than SR succeeds as a comedy. Obviously, such comparisons don’t really provide much tangible information to potential purchaser; I, for one, want to know if the latest SR is funnier than the last, not some if it is, obtusely, funnier than GTA if all of the dramatic elements in GTA were morphed into comedy.

Conclusion and some quick side-points

To bring this back to the reviews for Ducktales, I think that the wide split in scores is chiefly due to envisioning the audience and scope differently. ScrewAttack judged the game on the merits of bringing old-school gaming in a modern skin whereas Gamespot reviewed on the merits of providing a modern take on old-school gaming. The difference is subtle but, based on the scores given, crucial.

  • I personally side with the Gamespot review. If you want old-school gaming, go hook up your Nintendo and blow out the dust from the bottom of the cart. Duck Tales is a classic that doesn't need fancy visuals to be enjoyed.
  • RIP Roger Ebert, I’m sure you would have come around on gaming eventually
  • I only briefly discuss the possibility of some sort of objective standard where games can be compared cross-genre. I think it is totally valid to say that GTA is a better game than some shovelware iPhone game even if they aren’t in the same genre. The full elaboration of the logic here would be enough to fill another post.