SimCity is out – sort of. The servers have been struggling, meaning that many people are unable to play, or even to access their cities, because you need to be able to access the servers to play. Which is all sorts of crap.
Polygon originally gave SimCity a 9.5 out of 10 score, based on their experience with the game in optimum conditions before its release. Then, the game launched for real and it was significantly broken. So Polygon lowered its score to an 8.
When the first reviews of SimCity were going up, I found myself wondering if the high scores would be warranted if it did indeed turn out to be borderline inaccessible. When that came true, and Polygon lowered its score, I then found myself wondering if the original score wasn't still appropriate, presumably because I'll even be contrary with myself if no-one else is around to be irritating at.
At what point do we separate the content from the delivery method? Ben Kuchera at Penny Arcade Report likened it to critics seeing a film at a cinema and giving it high marks, followed by the cinema burning down, so no one else could see it. I don't think that's quite the right analogy, though, because you can simply go to one of any number of other cinemas in most cases, or even wait for it to arrive in another format (DVD, TV, online…).
If there's a real-world analogy to be drawn, I see it more like a painting in a gallery. There's only one place you can go to see it, and a critic might label it as a true masterpiece when they see it at, say, an invitation-only event at the gallery. Oh, and tickets to the gallery can be bought in advance and any time, regardless of how many people can fit in the gallery.
Opening day arrives. Everybody who pre-booked a ticket turns up, and the queue dominates the whole street. Only a lucky few at a time can see it. There's no other option, of course. You can give up and go home, or you can wait – but bear in mind that you've already paid for the ticket.
Has that work of art become any worse? Should the critic revisit it in light of the impossibility to guarantee that you'll be able to get in to see it if you buy a ticket? Should the work stand on its own, but with a warning about the difficulties of seeing it noted separately?
Of course, art isn't given a rating out of 10. That's a key difference. You can praise one aspect while criticising another, without having to work how that averages into a score. I don't know why these server issues are worth a docking of 1.5 points exactly. Why not a 7? Polygon's review policy lists a 7 as: "Sevens are good games that may even have some great parts, but they also have some big 'buts'."
More dramatically, why not a 1? From Polygon's policy: "A score of one indicates that Polygon review staff believe said game doesn’t properly function. Most reasonable people will not be able to finish a game with a score of one due to massive technical, design, and execution problems." It seems to fit.
I don't actually mean to pick specifically on Polygon here, even if I sort of am. I happen to think that review scores do have a place as a way for readers to judge comparable games (though when you try to compare across genres, the usefulness dissipates somewhat). I also think that Polygon's policy of updating its review scores as games change over time is bold, and a smart way to deal with the increasing number of online games that will build on themselves, and likely improve.
But it raises a kind of existential problem for the idea of games reviews.
"Maybe it would be better if reviews got rid of the numbered score, at least for a bit, just to say 'not yet'," Kuchera noted, while wondering if SimCity was immune to the idea of a classical review because of all this.
I'm now wondering whether a review of a game is buying advice for customers, or whether it's a critique of form for enthusiasts. The former would surely tear SimCity a new sinkhole, but the latter could reasonably ignore its technical issues.
Of course, there's no 'right' way to review – a publication might do either of those, but I think this example raises the question of whether it's possible to do both consistently, and whether it will be in the future.
PS Perhaps the closest analogy for SimCity's release is House of Cards on Netflix. It got positive early reviews, but if Netflix had gone down and it had been unavailable, would those reviewers have revisited their score? Would we have been saying that they shouldn't have scored it highly until we knew if Netflix could cope with the demand for it?