Is everything we know about wine wrong? asks the headline of an article that appeared earlier this week in UK national newspaper The Telegraph. ‘No’ is the answer, and I’ll explain why this article gets it wrong.
Over the last couple of years, there have been quite a few articles in the national press about wine competitions, and how professionals aren’t very good at tasting wine blind. This, the latest,
is a poor article that casts wine professionals in a bad light.
It’s yet another piece relying on the statistical work by Robert Hodgson from The California State Wine Fair. You can’t argue with statistics (when they are applied correctly, of course), and Hodgson’s study shows that the results from this wine show aren’t very reliable. I can’t argue with that: he has the data, and they appear sound. Hodgson also looked at the performance of different wines across a range of US-based competitions, and found that the same wine got different results at different competitions. His suggestion was that wines were performing randomly, according to chance.
Will Storr, the author of The Telegraph article, throws a lot of other rather unconnected material into his piece, including an extensive quote from Jilly Goolden where she claims to have invented the modern language of wine, among other things. But I’m bothered by his final conclusion that ‘despite the work of the scientists, it still appears as if there’s very little rational behaviour to be found among wine lovers and makers.’
It’s as if he’s bundled together as many interesting snippets about wine tasting that he could find, and then run out of space, so he’s reached a conclusion that certainly isn’t justified by what went before. Outsiders frequently want to believe that wine tasting is nonsense, and because it seems rather opaque and difficult to them (many find all wines taste the same), they are reassured when any evidence emerges that those of us in the wine trade are bluffing or making it up.
The problem is, we aren’t. Yes, tasting wine blind is difficult, but it is not impossible. If it were impossible – and experts really couldn’t tell the difference between cheap and expensive wine, for example – then no one would pass the blind tasting section of the Master Sommelier or Master of Wine exams.
The Australian Wine Research Institute regularly tests the blind tasting ability of winemakers and show judges. No one is perfect, of course, but some people are very good at it, and are consistent tasters. There are data on this, but journalists like Storr don’t bother to ask around properly – or simply ignore these relevant data.
And sensory scientists work with panels of tasters and come to statistically significant results. People are not perfect tasters, but some people are very good. Again, the data show it.
So how do I explain Hodgson’s results? Tasting blind in a competition setting is difficult. Many wine shows and competitions simply do not have the required calibre of judges. To get good, reproducible results in a competition, you need excellent tasters (and to get these, you’ll likely need to pay them), and you need a robust system that helps these tasters get the right results. It is likely that the shows Hodgson has looked at simply don’t have these required attributes.
Storr is right in as far as some wine professionals aren’t very good tasters. But many are, a fact which he’s ignoring on the basis of Hodgson’s sampling of individuals who clearly weren’t very good tasters, and that’s unfair.