Star ratings, challenged and explained
February 25th, 2008 by Besha Rodell in Food & Life, Food media, RestaurantsGot a letter from Lisa, who writes to share her frustration with our star rating system:
I love your reviews. I was worried you’d never measure up to Bill Addison. I initially prayed that when he left, Cliff Bostock would take his place, but I have put that panic to bed (though I adore Cliff Bostock). But I have one huge complaint. WHAT IS UP with giving three stars to every damn restaurant? It drives me insane. I mean, reread the opening paragraph of the AquaKnox review. Even the blurb on the TofC [table of contents] page (”all glitz and no soul”). And then tell me, is that how you begin a review for a restaurant rated “very good”? It is not. Why on earth does this get three stars? Why, why, why? How are your dear readers supposed to make decisions about when the splurge is worth it if everything is “very good”? The grade inflation at CL is crazy-making. And while I’m at it, what is up with ONE star out of FIVE being “fair”? Twenty per cent is fair? Two out of five is “good”. That’s a 40%, a very low F. If I gave a student a 40 on a paper and then wrote “good work!”, s/he would have a fit. I know that’s CL stuff and was already like that before you got here, but it’s just wrong.
Arg, the star ratings. They drive me crazy. But I thought Lisa had some good points. Here is what I wrote as a response:
Let me tell you, if there is one thing a restaurant critic hates, it’s star ratings. Seriously, we all get together and bitch about it! I hate assigning stars, because comparing restaurants is like comparing apples and oranges. There are so many shades of good and bad. But to answer your question, here are some of my thoughts:
The reason one star is “fair” is partly because we have no half stars, and because we DO have a “no stars” rating, which means “bad”. This is consistent with other publications, including the AJC. There’s also a practical reason, which is that I like to keep the five star rating for only the very best in Atlanta – those that I consider world class. In my opinion, there are only two restaurants in town which deserve that rating right now. That’s a tiny percentage of restaurants! So I see most places as operating on a four-star playing field. It should be hard to get four stars, and it should be rare that I give one star or less (why waste the space?). So yes, most restaurants I write about will be in the 2-3 star range. In the past five weeks I’ve written 1, 2 and 3 star reviews.
As for AquaKnox, you’re right, it’s a place with no soul. But the food is really good! If it wasn’t such a blatant whiz-bang soulless marketing exercise, I’d be tempted to give it four stars (the food’s that good). Part of why I hate star ratings is because I think it discourages people from reading the whole review – they think they can get a snapshot by looking at the rating. But restaurants are complex organisms, and judging them is a complex sport. What to do when the restaurant gives you the heebie-jeebies but the food is great? Three stars, I guess.
I think there is an argument to be made for what you seem to be proposing – the star rating system similar to a grading system, in which one star is bad, two is fair, three is average, four is good and five is excellent. But in grading, you have plus and minus. And really then, how would you distinguish the really great from the very good? The great neighborhood place from great fine dining? Under that system, you only have two ratings to assign places you like, four and five! And then, what if something is amazing? You’ve got nowhere to go! (Imagine if you could only give As to the true geniuses, the ones that only come along every five or ten years? Well, I have that prerogative and so I make use of it). This way, I have three and maybe four gradations for places that are good and worth reviewing in the first place. There are other ways to do it, but I think, for our purpose, this is the best system. It is not the same as giving grades – just as it is not like giving Olympic medals. It’s not that out of the ordinary – in New York, getting even one star from the New York Times is seen as a huge honor for most restaurants.
Ah, but it’s an ongoing debate, and I welcome your comments. I truly appreciate the feedback.
Anyway, thanks Lisa for bringing it up, and giving me the opportunity to vent my own frustrations about star ratings! What I’ve written above is the long answer — the short answer is: Read the review. It will give you a much better picture of the place (and whether you’d want to eat there) than any imperfect rating system could.
February 25th, 2008 at 9:51 pm
“Read the review” is the operative phrase. The stars don’t mean a whole lot unless you have a really good idea of what the review likes and dislikes, both of which should show in how he or she writes about the restaurant. Despite those two sentences, I do look at the stars and am usually struck by the disconnect between them and the words.
February 26th, 2008 at 12:11 pm
Besha-
Does the fact that stars mean different things in different parts of the paper also cause confusion or annoyance among readers and restaurant people?
When you give two stars, it means Good. When Curt and Felicia give two stars, it means Don’t Bother.
And then there’s the problem of grade inflation overall in the media.
Entertainment mags give CDs and movies two stars simply for existing. Rolling Stone gives Sheryl Crow’s new album 3 1/2 stars. The music they pipe into the kitchen department at Macy’s is more interesting than Sheryl Crow, but Sheryl Crow gets 3 1/2 stars!
Is there a downside to ditching the star system?
What would happen if, instead of stars, your reviews just said Poor, Fair, Good, Very Good, Excellent, Scrum-diddly-umptious at the bottom. Instead of having stars and their translation — they would just have the translation.
February 26th, 2008 at 12:32 pm
I think there was a time when we wanted to have all the star ratings across the paper mean the same thing. But (and maybe this is because I have food-colored lenses on) I find it more helpful to have ratings that compare to other food media. I like the comparison between what I give a restaurant, what Meredith gives it in the AJC, and what Christiane gives it in Knife & Fork. The fact that these ratings are all comparable is helpful I think, but again, maybe that’s just me.
So that’s one downside to ditching the star system, but the other is that readers and restaurants really do love them. I’d get rid of them in a heartbeat, I’d even forgo the “good” “fair” etc. I feel like I’ve got 900 words to make my point, so if you haven’t figured it out by then, I’m not doing a particularly good job. There are great critics out there who do not give ratings – the Pulitzer winning Jonathan Gold for instance. It frees him up to write about all kinds of things too, rather than confining him to the strict boundaries of a formal review (although much of what he does reads like formal review). But the thinking goes that readers like the ratings, they like to be able to have a context that goes across all food media in town, and that restaurants like to wear their star ratings as badges of honor.
February 27th, 2008 at 6:52 pm
Come on. The stars have one bottom-line function. They are for marketing. The inevitable ambiguity of their meaning leaves only a few restaurants threatened. I haven’t seen Christiane, for example, give an “unacceptable” ranking in many moons. Everyone else falls into basically the same category.
During the years I edited the paper, I refused to use them. We had a continual battle with the ad department over them.
They’re also another symptom of the ADHD-ing of the press. Why read, when you can get the gist in a single glance?