Yesterday I tasted 62 different Brunellos from the 2010 vintage. I wrote short tasting notes, and gave each wine a score.
But it’s quite a task tasting 62 wines like this together in a short space of time. Do I think I got all the wines right? And how much confidence do I have in my scores? These are important questions for wine writers, because critics do this sort of thing all the time. Tasting notes and scores like these are their currency. It’s what they are selling. If you are an important enough critic, people will use these scores to sell bottles. Collectors will use them as the basis of dropping serious $$$ on wines that they haven’t tasted.
So, to answer my questions. No, I think I will have overrated some wines and underrated others. But tastings like this give me a chance to get a broad perspective. I’d be much more confident of my verdicts if I’d spent more time with each wine – say, opening two or three of them and spending the evening with them, or sitting down with flights of five or six at a time, and spending much longer critiquing each. So, as with many things in life, there’s a trade off between volume and quality. Interestingly, I’m much more confident of some of these instant verdicts than others.
This raises an interesting broader question: what would the perfect wine critic look like?
In many areas of professional endeavour we are used to the idea that there are objective measures of performance, and that those who perform to higher standards get rewarded and recognized over those who exhibit less competence. Is this also true of wine tasting?
There are two elements to the performance of a wine critic, which can be separated out, and in my opinion only one of these is measurable.
The first is in terms of raw tasting ability, and this would be fairly straightforward to measure by sensory scientists, although I can think of precisely zero critics who would allow their palates to be assessed like this. How well do a critic’s tasting faculties work? Are they sensitive or insensitive to smells and tastes? Faced with a large set of wines including duplicates, will they pick the duplicates out? Faced with repeated sets of wines, will they be consistent in their scoring? And let’s bring memory into this: how good are they at recognizing wines when they are tasting double blind?
It’s interesting that some of the leading critics have allowed stories to circulate that suggest that they, among all critics, are particularly gifted. They infer that nature has bestowed on them rare and unusual powers in the realms of taste and smell. But this can be measured.
The second element can’t be measured, but is perhaps even more important, and it’s because of this there can be no such thing as a perfect critic. It’s the exercising of the critical faculty: deciding which wines are better than others. It’s ‘taste’ as in aesthetic appraisal. The idea that there is one correct way to read or assess a wine, and that as critics get better at their job they converge on this correct assessment, is false. There’s a level at which wines can be thought of as good or bad, but this is a very basic level of assessment. Beyond this, critics make style choices, and even highly competent critics are likely to disagree on many wines. There’s room for a plurality of opinions, and if we are to use critics we need to choose which ones align more closely to our own tastes.
10 Comments on The perfect wine critic, and assigning confidence to scores
Some interesting points there. On the sensory aspect, what do wine critics do if they have a bad cold affecting their taste? Cry off a tasting like an opera singer?
We have to be fair to ourselves and other “taste” critics.
Reviews, whether of a movie, a play, a meal, a game, or wines, will always be subjective – when articles appear saying “wine critics tested in blind tasting mode will give a wine a different score depending when and how they taste it”, which happens about twice a year, they always try to imply it is somehow wrong, or even a scam/scandal.
But nobody would think twice when told “film critic rates movie differently depending if they had a good day or not, and what the last 3 films they have seen have been”. “restaurant critic rated visits at restaurant differently each time”. “games review scores vary depending on expectations of the reviewer prior to the test”.
People find it normal in many settings, and it is perhaps because wine reviewing tried hard to appear more systematic (with points and score scales etc) often without the nuances that will be used in other reviews
As you say, we all find ways to bring more confidence and less subjectivity to our tastings and notes – things like retasting a set of earlier wines halfway through and at the end, in a different order; things like knowing our biases and buttons and trying to compensate for them or at the very least making them very clear. And those biases aren’t any worse than the biases in any other part of life.
I would love to be tested by sensory scientists. My palate memory has allowed me to score well in competitions and exams, but how sensitive is my palate actually? Where are my strengths and weaknesses? The first tasting I conduct for my wine school is a component tasting, that to some degree allows me to make just such a map of my, or my students’ sensitivities. It is a tasting I recommend all pros do regularly.
I recently spent a month tasting wines “differently” – only a month because I no longer regularly critique wines – I tasted the wines double blind as usual, except that this time I drank the wines, usually with dinner and throughout the evening, instead of tasting a small amount of them against one another. It was much more in line with how wine is consumed by the public, and the reviews did not suffer either because of palate fatigue or from comparison to other wines.
Finally, I hate scoring wines. Wines are not numbers. Because there does need to be some easy to understand rating, I use the most universal of all. A scale of how likely I would be to buy the wine. After all, this is all that really counts.
I’ve been thinking about tasting capabilities a lot myself lately. ow good can a palate still be after tasting more than 20 wines in rapid succession? How deep can you know a wine that you have spent all of 2 minutes with? It’s a balancing exercise for a critic/winewriter between trying a sufficient amount of wines to properly assess the vintage in this case, and tasting too much to make a wellfounded recommandation on what to buy.
I tend to look at larger tastings as only a first step in discussing a wine. I went to Dive Bouteille a couple of weeks ago and out of all the wines I tasted, there are only a couple of wines that I can still recall now and I would much rather spend my time getting to know these in more detail than going to the next 100+ wine tasting.
Good post and related comments:
When I’m looking for wine to buy, I will generally go to the relevant fair, taste as much as I can and make very very brief notes or just yes/no/maybe. I then follow up the “winners” later, to see what other people think of them etc. If I liked a wine on the day and it’s well received, then I might buy it. At which case I can get another chance to get a better look at it and write tasting notes etc.
This “quick verdict” system works for me – usually the wines I liked on the day still seem good a few weeks / months later. But I would certainly take another look at a wine that I had missed / marked down this way, if the need arose.
As for exercising critical faculty, I agree. Today we had a fairly obviously vanilla oaky Rioja to try. Most people liked it, I quite liked it too. I could recognise that the obvious oak might put it a bit out of whack for some people, but equally I know there are many who would enjoy the style too. I suppose the “perfect wine critic” could both consistently describe/attribute points to a wine, plus have a big enough vault of experience to place this wine against that context and be aware where any personal bias or palate sensitivities/defects lay.
Your questions “Are they sensitive or insensitive to smells and taste” suggests that they might not have any specific anosmias or insensitivities.
The reality is they will most definitely have one or more aromas they can not detect as well as the wider population. It is suggested that any one odourant will have 1-2% of the populations not detecting it, while other odourants can impact 25+% of the populations
Some more common specific anosmia influencing wine people include
Pyridine (mouse) – wine spoilage compound from lactic acid bacteria
Β Ionone (turkish delight) – grape flavour
B Damascenone (fruity) – grape flavour
Cis- 3- hexen 1 ol (grass) – grape flavour
Isovaleric acid (sweat, vomit) – wine spoilage compound from Brettanomyces
1,8 cineole (euclyptus) – grape spoilage compound from gum trees
Diacetyl (buttery) – MLF falvour
I guess the other issue with tasting big brackets of tannic wines line Brunello is the cumulative impact of the tannins on the taster. You might be ‘right’ early in the piece, but tasting differently for the later wines.
If wine critics would be validated like scientific instruments in an analytical lab, no one would have the certification. That’s why no known critic has ever accepted to pass such a test. That would be the end of their career.
Claude, over time wine critics are “calibrated” by their readers.
Speaking as an ex-literary critic, one key thing a critic needs is a sense of context. So it’s not just a question of “is this book well written?” and “is it enjoyable?” but also: Where does it fit in the canon of genre/literature? Is it a cliche or something bold? Does this overturn or build on something that came before, or is it just an imitator? And, finally, is this complete crap, and if so, how can I express my contempt for it while still being fair and helping to advance the conversation?
This dimension is what’s often missing in wine criticism.