jamie goode's wine blog: The clustering of point scores

Monday, December 01, 2008

The clustering of point scores

Had a reader write in recently, making a very good point:

I'm wondering about points and the '100 point' scale and inflation and the tight range that wines score. What percentage of the wines you review score between 86 and 93 points? Not specifically referring to your use of this scale, but it seems to me that either there is inflation, or wines have got better in recent years (quite possible, I agree). In the old days, I thought that 80 was considered a good score. Sort of a Jacob's Creek Chardonnay and 90 or above was pretty damn awesome. These days it seems that for some people, anything below 85 is a bit of a failure. Though I'm sure you'll agree with me that scores are not the be all and end all, just a useful guide. By my worry is also not just about inflation, but clustering. To you, I'm sure there's a big difference between 93 and 89 points, yet in other walks of life, say your kid's result in a maths exam, that wouldn't be a big deal. [Andrew Halliwell]

The point is well taken. My scores cluster horridly - effectlively I'm using a 15 point scale, not 100 points. But then do Robert Parker's, and the reason I use this scale is because he has established it as the 'standard' scale for rating fine wines. What are your views?



At 11:43 AM, Anonymous Jim (of the shallow pockets) said...

Isn't one of the prime things to identify who/what the score is for?

If I had "bottomless pockets" high scoring expensive wines are novel.

Why not bracket wines a little according to ambition in the market place. ie. by the dreaded term "price point"

Then we might see "for 5 quid this is a 90 point wine" or conversely "for 25 quid this is appalling and well worth using the "low decimal" end of the famed 100 point scale"

At 3:43 PM, Blogger Beda said...

I've been discussing this in my own blog and with friends and tasters in the past few weeks. Tough subject, isn't it? Thing is, as useful as scores and points may be, what matters, really, is the feeling you have about the wine you just tasted and this is hardly "scoreable" in an objective way. I'd say people should declare either they like it or not, rather than how much did the wine score, even if they want to add notes on how technically good or bad the sample was...

At 3:49 PM, Blogger Rob Bralow said...

The important part of the rating system is "Did the taster like the wine?" When it gets down to how much is when the scale really comes into effect. For the most part people only need a 'yea' or 'nea' to decide if they should buy it or not. And what happens when a inexpensive wine tastes better than expensive wines? There needs to be some system to compare them, if it be stars, points, thumbs up, smiley faces, etc.

At 5:20 PM, Blogger Janik said...

I feel that a score alone isn't worth much without a wine description. I like it when people describe wines with enthusiasm and when they show if they like it or not. I can give wines a good rating because they are technically well made and tasty, even though I don't like their style (maybe too oaky or lacking acidity for me, etc.). But then I will tell people what I don't like and they can decide if that feature bothers them not.

At 5:22 PM, Blogger paswines said...

To a certain extent I think that the 100 point scale is a bit like grades when I was in school. Below 70 was a failing score. 70-80 wasn't very good but it was passable - just. 80-90 was quite good and, in fact, most people seems to score in that range. Over 90 was indeed very good.

When Parker's 100 point scale is looked at from that perspective, it makes a bit more sense.

Then there's the infamous "grade inflation". I think we've seen that with A level scores and the same thing may be happening with wine scores.

Overlying all of this is the dramatic improvement in winery and vineyard hygiene and general knowledge of wine making that has impacted the production of wine all over the world.

As little as 15-20 years ago hygiene was so bad in most cellars it was really hit and miss whether a wine would be good. The grapes might be decent but production was dirty. And, of course, there was much more of a "peasant" production mentality in the fields (more is better). Most of that has changed now. Today it is unusual to find a truly faulty wine (cork taint aside).

All of that said, I read many of the regular reviewers of wine (Jancis, Parker, Wine Spectator, Wine Enthusiast, Decanter and, of course, Jamie). After reading lots of those reviews and tasting a fair number of the wines they've reviewed, I have my own "filter" with which I read the reviews.

The number itself really isn't as important to me as the words said. For example, if Parker talks about a "big" wine, or lots of "extract", or "butteriness", I know that the wine is probably made in a style I won't like, even if the wine has a 100 point score.

At 6:12 PM, Anonymous doug said...

Hugh Johnson has a good system:

one sniff - the minimum score; emphatically no thanks
one sip - one step up
two sips - faint interest (or disbelief)
a half glass - slight hesitation
one glass - tolerance
two glasses - means you quite like it (or there's nothing else to drink)
three glasses - you find it more than acceptable
four glasses - it tickles your fancy
one bottle - means satisfaction
a second bottle - is the real thumbs up
a full dozen - means that you're not going to miss out on this one

As Kathleen Burk writes: Presumably the top score in the Johnson universe is the "whole vineyard"

At 7:37 PM, Anonymous Doug said...

The 20 point scale used by the World of Fine Wine is the most nuanced: eg
7.5-10 - sound but dull or boring wine of no character or appeal


19-20 - a great wine, of spellbinding beauty and resonance, leaving the drinker with a sense of wonder

I like the fact this scale is based as much on aesthetic criteria as correctness. The quality of greatness thus revolves around beauty and articulacy. Technical wines, even those of some dimension, can never inspire in the same way and would consequently be rated in a lower points band...

At 8:04 PM, Anonymous Kirk - Sandihurst Wine NZ said...

As a winemaker, we tend to dismiss scores/stickers as pointless but we recognise their ability to sell our wine. Most people need guidance or assurance when buying wine. The 100 point scale often feels like it is only a 1-point scale - 90 points + or not. You can live with 89 but anything less feels like a failure and can make a real difference to sales.

At 9:09 PM, Anonymous Laurence said...

There is also the psychological (not sure that is the right word) element. I notice that over time the amount I pay for wine increases - I would say, pay £6.99 then, I find its £8.39 and then £9.99 a bottle. I get used to paying that bit extra - I therefore subconscioulsy anticipate that the wine will be that much better. Whether it is or not, I expect it to be and therefore it is - (bit like my mum's 'well it is £7.99 reduce to £4 so it must be good' - I have tried to tell her sooo many times, that perhaps it wasnt worth £7.99 in the first place)

Now, with professional tasters and critics, they must have to dumb down their (taste) faculties to ge back down to my normal levels. How difficult is that?

Each time you try a wine you are putting a reference in your mental taste database. For example,nce you have tasted a 'wow' wine, then if something else hits that spot - that is also a 'wow' wine. THis must lead to inflation - unless you recalibrate every couple of months.

At 12:34 AM, Blogger Michael Pollard said...

Scoring wine on a 100 point scale has been around for quite some time - at least some 25 years before Robert Parker came on the scene.

Like it or not, giving a wine a score does provide quite a bit of information, in a simple to understand format, to those who buy wine without having a great interest in wine (i.. the vast majority of people who buy wine).

When you think about it, it’s not hard to see why most critics use a fairly narrow range within the 100 point score. First, most recommended wines score 85 or above with very few, if any, wines receiving below that score ever seeing their names mentioned in print. Second, most critics do not taste across the breadth and depth of wines even in a single region. Hence we don't see the poorest quality wines reviewed, even though the prices of such wines may make them attractive pricewise to the average consumer. Don't those consumers need an advocate? Last, I see score creep among many reviewers, particularly amateurs, and believe that this is indicative of the failure to taste the iconic wines and establish benchmarks within a style, region, and/or vintage. More importantly, even if a critic has been regularly exposed to the benchmark wines how well is that "taste memory" kept so it can be used to calibrate lesser wines? Are there critics who say they can do that? Are they willing to demonstrate the ability in public?

At 7:53 AM, Anonymous Rob Malcolm said...

The 100 point system is an intellectual fraud. At best it is a 50 point system. To those that say, "well, we all know that it really works like that": you're just perpetuating another wine myth. One day we're going to have the ultimate indignity: a wine that is scored 101! What's the intellectual difference between that and giving every wine the first 50 points? Jamie the 100 point system may be good commercial sense (copying RP), but you know it is bad science: shame on you.

Doug is right to say a mark should be about aesthetic qualities. I give a simple mark out of ten for the nose and another mark out of ten for palate. I'm not interested in colour/clarity as I don't get any intrinsic pleasure from the look of wine (I'd enjoy a glass of wine as much if I were blind). I keep the marks separate, that way I know that a wine that smelt great, but then disappointed on the palate, e.g. "8,6", is different from a wine that was the other way around e.g. "6,8". I find this scoring system very intuitive and easy, and also a lot more meaningful when I return to look at it down the track. A technically correct, but uninteresting wine is worth half marks. Dirty or ugly tastes/aromas score below 5. Beautiful tastes/aromas score above 5.

I do agree there is score creep happening, but mainly for the reasons given above, i.e. less bad wine being made.

And I totally agree that one needs words to describe a wine and why one likes or dislikes it. But a score is equally important to giving an honest and frank judgement. The 100 point system isn't honest.

At 4:37 PM, Anonymous Anonymous said...

Indeed a tough subject. I don't think exam comparisons are sound - either the student has got 90 out of 100 questions right or he/she hasn't. (OK, the argument holds less water in say an history exam!). I think far more subjectivity is at play in wine-tasting.

I look to the experts to tell me how good or bad a wine is technically - and that can be out of 10 for all I care, provided the score is supported with a good description, as others have suggested.

The score, the description and the price will then help me determine whether or not it is for me. Given the above, I don't find score ratings within a narrow band of say 85 and 95 to be too instructive, I must say.

John V. in Budapest

At 5:09 PM, Anonymous HM said...

One of the wine merchants here sent an offer based on the Wine Spectator's Top 100 Wines of the Year (I don't usually follow these lists). Two of the four offers were:

#5 Domaine du Vieux Télégraphe 2005 – 95 points

#19 Chateau Smith Haut Lafitte Blanc 2005 – 96 points

I couldn't but help raise an eyebrow. Isn't that an admission that either (a) their rank order is wrong, or (b) their point scoring is wrong??? Can't have it both ways surely?

At 7:54 PM, Anonymous Sklenicka said...

I agree with Janik, score without description is worthless. Evaluation in 100-point system starts at the top and penalises the faults, subtracts points. I think that average reader (drinker) evaluates wine in a different way. Starts at 0 and rewards beauty and pleasure, adds points :o)

At 12:07 AM, Anonymous Tish said...

If nothing else, the quite intelligent yet widely diverging comments here simply prove that the so-called 100-point-scale system is hopelessly broken. Too many numbers dished out by too many "critics" and then (ab)used selectively by too many marketers and retailers appealing to too many grade-minded consumers. And let's not get started on whether any given rating was done by a panel (hardly ever) or represents the "potential" quality or current taste impression. It is terribly ironic that the ratings (now of course almost totally relegated to 85-95 points)imply precision, and yet it is crystal clear that agreement on what they mean and how they can/should be used is impossible. The 100-point scale is a monster that has grown out of control!


Post a Comment

Links to this post:

Create a Link

<< Home