05 Jul 13

Game reviews and betas

This week the South African printing of Popular Mechanics carried an interview with us about Desktop Dungeons. Magazine production cycles being what they are, the interview that led to the article actually took place nearly two months ago in an awesome burger joint in the city center. We ended up having a thoroughly enjoyable conversation for a couple of hours, meandering from the realities of game development here in SA, to what it’s like being on-stage at the IGF, to marketing your game and getting people to notice it.

One of the topics that came up was how the press interacts with a game that’s in active beta, mostly because of this PC Gamer UK review which, while it said some awesome things about Desktop Dungeons, doesn’t mention that the game they reviewed was the closed beta and has gone on record on Metacritic as giving us an 81. Unfortunately, that Metacritic score is locked in now – even if PC Gamer UK were to review the game a second time (say, when it’s actually released) and give us a better score (which they might, seeing as the problems mentioned in that review are all things that come with the beta being web-only, which the full version won’t be) Metacritic won’t update that score at all. This is a problem with Metacritic, one of the many known issues with it as a platform.

But the question of how to handle a game that’s still being developed is a good one: Should things only be covered by previews, no scores given? What if people are essentially charging money for a beta? At what point is a game a product that needs to be compared to others? Are scores even useful anyway? Our interviewer, Rogan, decided to preview Desktop Dungeons along with the interview. That’s why I was rather surprised when he contacted me a few days ago to apologise for what was in the magazine…

Apparently someone on the editorial staff on the magazine had decided that “preview” actually meant “review” and changed the title accordingly. That done, they realised that all reviews need a score, yet this one didn’t have one, so a score was summoned from thin air without the game being played. Desktop Dungeons magically earned an 80. Rogan was upset about this, rightly so. After we picked up the magazine to see for ourselves, we could see that Rogan had managed to distill hours of conversation into an interview that not only made us sound smart, but provided lots of useful encouragement and information to hopeful game developers. His preview was descriptive and critical, mentioning that the game was a beta and how you could get hold of it (you’d be amazed how many local publications don’t do that). Having a phantom score applied to his writing wasn’t cool.

Thankfully this ends well: I emailed the editor of the magazine, explaining why we felt reviewing a beta was disingenuous, not to mention someone other than the (p)reviewer giving it a score… I got a reply within minutes in which the editor not only apologised for the mistake, took full responsibility for the slip-up and pledged to be more aware of this issue in future, Popular Mechanics will also be printing a retraction/explanation of the review in their next issue to print (not the August one, that’s already been finalised). That’s good journalism.

I’ve always liked Popular Mechanics, I’m glad they’re principled and willing to learn about a medium they might not otherwise know too much about. Thanks for covering Desktop Dungeons! Here’s a picture that the photographer they sent to our house took, enjoy!



Copyright © 2021 QCF Design
Powered by WordPress, theme based on one from Themelab.com