Who judges the wine judges? And how can we increase judge diversity?


Who judges the wine judges? And how can we increase judge diversity?

Wine judging is tough. This week and next it’s International Wine Challenge judging, which will keep me and quite a lot of other people busy. I guess if you are looking in from outside it looks like a whole lot of fun, tasting wine all day. Well it is – the judges are invariably nice people and it’s like being part of a big family – but it’s also very hard work. It takes concentration, stamina and expertise. Tasting lots of wines blind is hard, and if you want sensible results at the end of the process, then you need good judges. The proof is in the pudding: look at the results of a competition and you’ll see how well the judges did, if you know the wines. Now blind tasting is good because there’s a level playing field, and it can throw up surprises. But if you know the wines and the results are bizarre, then the judges didn’t do so well.

A tweet I saw this morning made me think about our judging panels. It pointed out that the list of judges for the recent Nederberg wine auction tasting were mostly older white males. That panel could do with some more diversity. The same could be said for the co-chairs at the International Wine Challenge (five male, one female, and new hires Sarah Abbott and I are the youngest, but we’re no spring chickens!).

How do we achieve this? Tokenism won’t work. If you care about your competition, you can’t have senior judges or panel chairs who aren’t very good. Just one person who’s not competent can really mess results up, either on a panel, or in a more senior position. No judge gets it right all the time, of course. That’s why we have panels, rather than relying on just one person. Panels on the whole give better results.

What makes a good judge? First of all, they need to have the physical apparatus needed to taste: good functioning olfactory and gustatory systems. Many people have this. After all, good wine tasters are made not born.

They need to be reliable and consistent judges. The hope is that if a judge is given the same wine on repeated occasions, they will score it similarly. Presentation order and external factors, plus the internal state of the taster – palate fatigue, mood and so on – make this tougher than it sounds. But if a taster is not pretty consistent, then they will not be a useful judge.

They need to have stamina. In an ideal world we’d taste no more than 40 wines a day, but in order to make competitions work, especially when tasters are busy people and they are being paid, you need to taste around 100 a day. It’s possible to do this and do it well, but you need to be able to cope with the physical demands of a long day of tasting.

They need to work well as part of a team. This is often underestimated, but if you’ve ever had anyone with a big ego who doesn’t respect others’ opinions on your panel, you realize what a vital skill this is.

Finally, you need experience. There’s no way round this one. Because good tasters are made not born, a good judge needs to have tasted a lot of wines, and a lot of different styles of wine. This is one of the reasons most senior judges are a bit older, because it takes time to get the sort of palate experience that’s needed to be a top judge. Judging wine isn’t just about how much you like the wine. You have to deal with lots of different styles of wine and almost take a step back from your personal preferences to be consistent in rewarding the best wines in a range of styles.

The International Wine Challenge (IWC) has a way of spotting talent and promoting people. There’s extensive feedback: the panel chair reports on the panellists, and the panellists give feedback on the panel chair. Then, we co-chairs have data on how well each panel, and panel chair, performed in terms of how often we have to adjust their scores. It’s not a perfect system, but it does result in people getting promoted, and in some cases demoted. There’s quite a bit of movement and there are new judges all the time coming through the system. And there’s a strong emphasis on training new associate judges, too. Anyone who has an interest can usually get a couple of days as an Associate Judge, which is where everyone starts, even well established trade figures.

“The IWC does not take into consideration reputations of judges either given by themselves or others. We mark them on their performance as judged by their peers. As an Associate, if two panel chairman report a good performance and high scores based on their assessment, they will be promoted to Judge. As a Judge this will also be the same process to be promoted to Senior Judges. Senior Judges could be asked to be Panel Chairs based on their abilities and their availability to taste for a full week. Panel Chairs can be promoted to Co-chair.” From the IWC website

The important thing here is that all the judges are being judged. When you’ve judged with someone else (assuming you are competent), you get a feeling for who is good and who isn’t. It’s not always the famous people or the people with letters after their name who turn out to be the best judges. [I know some MWs who have passed a difficult blind tasting paper, but who are weak, inconsistent judges.]

The Australian Wine Research Institute runs the Advanced Wine Assessment Course. This is aimed at spotting and training wine show judges, and is a thorough assessment of the palate strengths and weakness of the attendees. This is a great idea, and I would love to take part (I’d learn a lot about my palate). It would be good to put all judges through this, but unfortunately it is only available in Australia, and I don’t know of any other similarly rigorous course elsewhere.

If we are serious about increasing the diversity of senior wine judging panels, then what would be really useful is to have a way of spotting new talent, and also of training up potentially talented judges who just lack experience. And of course a commitment to change. It may take time, but something has to change.

1 Comment on Who judges the wine judges? And how can we increase judge diversity?
wine journalist and flavour obsessive

One thought on “Who judges the wine judges? And how can we increase judge diversity?

  1. I’m pleased to read of serious efforts to judge judges. In my experiences back before I threw my hands up and abandoned judgings and critics, I was often appalled at who got to pass judgement.
    I worked for a couple of years with the organizer of the Cenral Washington State Fair, the most difficult venue to win a medal. Judges were wine makers. Try and evaluate them!
    For years I’ve wanted to hold a Judge Judging. Some basic trio tastings and fault tastings. With published public scoring. Reproducibility tastings. Perhaps wisely my wife says nay. I already irritate most critics.
    Over fifty? Declining olfactory senses.
    Male? As a population inferior to female.
    Obese? Over 20% body fat has a high association with olfactory ability.
    Smoker? Definite impairment .
    Conclusion? Don’t trust old fat guys who smoke cigars.
    Yet, experience matters.

    Paul Vandenberg
    Paradisos del Sol

Leave a Reply

Back To Top