Back to square one
The WSA musty taint survey

With a fierce debate ranging in the wine trade over the validity of the Wine and Spirit Association’s research into cork taint, Jamie Goode raises doubts over the methodology employed and asks is it time to rip up the report and start again?
(Reproduced with permission from Harpers Wine and Spirit Weekly, 11 October 2002, p 36-38 )

Back in May 1999, John Corbet-Milward of The Wine and Spirit Association (WSA) was in the audience at a panel debate on cork taint, held as part of that year’s London Wine Trade Fair. ‘The mood in the audience was one of confusion and crossness’, he recalls. ‘People were coming up with all sorts of figures, and there was no scientific basis to what was being said’. Instead of this ‘internicine strife’, as Corbet-Milward puts it, he thought it would be much better for the trade as a whole to work together to help produce taint-free wine for the consumer. So, after discussions with several key companies from various parts of the supply chain, and a quick whip round, the ‘WSA Musty Flavour Defects in Wine in the UK’ survey was born.

This survey involved a consortium of 18 companies, including retailers, producers, wholesalers and stopper manufacturers. Over the course of 12 months, from January 2001 to January 2002, data were collected on over 13 000 wines tasted by assessors in the contributing companies during the course of their work. The goal was to establish a ‘benchmarking baseline’ to estimate the true level of musty defects in wines on the UK market. Quentin Rappoport, director of the WSA emphasizes that ‘these are not WSA results; we merely facilitated this study—we felt it was about time that everyone stopped fighting each other.’ 

Just 0.7%?
The need for solid data on the rate of cork taint is an acute one, so you’d think a study like this that promised to demarcate the extent of this problem would have been welcomed by the trade. But the publication of the final report, in June 2002, provoked a storm of controversy, principally because the final quoted figure of verified musty taint prevalence was almost bizarrely low, at 0.7%. (Table 1)  ‘We were astounded to see such a low figure’, says Warren Adamson, UK head of New Zealand’s Villa Maria. ‘Everything we’ve seen, from show results to specific tastings, suggests the real figure is 5–6%’. Helen McGinn, product development manager for wine at consortium member Tesco, concurs that the WSA results ‘don’t reflect our experience of TCA taint.’ She adds that ‘on the basis of our tasting experience, the level is nearer to 5%. We taste 100–200 wines on site every week as part of our regular quality control checks.’

What’s going on here? Is the real rate of cork taint very much lower than most of us had previously suspected, or is the WSA survey deeply flawed? Time to investigate.

TABLE 1 Results of the WSA Musty Flavour Defects in Wine in the UK survey

Notes:
^aThis number is reduced by 9 as some samples exhibited more than one defect.

Survey methodology
The raw material for the study consisted of wines that ‘approved assessors’ from nine of the participating companies tasted as part of their normal duties. These assessors had to attend two training days held by the Camden and Chorleywood Food Research Association (CCFRA), who later collated all the data from the survey and produced the report. For each wine tasted a form was filled in indicating the wine type, country of origin, price, closure type and the condition of the wine. So far so good.

The next stage involved ‘verification’ of suspected TCA taint. If wines were judged to have a ‘musty’ taint by the assessors, the ullaged bottles were then resealed with the original closure and sent to one of two independent companies for verification: Geoff Taylor’s Corkwise and David Bird’s DBQA. The idea behind this stage of the process was to check that the assessor’s hadn’t misattributed the source of the wine fault. The final report doesn’t mention the method of verification used by these labs, but when questioned they both confirmed that it was another round of sensory analysis: in each case the wines were re-tasted by company staff within a week of receipt.

Remarkably, of the 277 samples identified by company assessors as musty, only 34% were ‘verified’ as musty. The report concludes, ‘It can be assumed from this that there was a significant degree of misclassification in terms of false positives, either in the form of other defects being wrongly reported as musty or satisfactory samples being classified as musty.’ This is a staggering discrepancy. Indeed, it seems to have been the cause of some internal strife within the consortium itself. Intriguingly, the report tells us that ‘One participant withdrew from the trial in October 2001 due to their concerns over the disparity between tasters and verifiers.’ Initially, the WSA chose not to disclose the identity of this participant, but some digging around revealed that it was Oddbins who had opted out.

Oddbins’ concerns
Steve Daniel of Oddbins confirms ‘Our major objection was methodology, specifically the lack of scientific controls and how the wine was verified as musty.’ The major problem was indeed the ‘huge discrepancy’ between what was submitted as musty and what was found in the verification step. This led to a further disagreement centred around the WSA’s decision to focus solely on ‘commercially significant’ mustiness. Samples with low-level taint might not come across as overtly musty, but could still be out of condition. Things came to a head at a public meeting of consortium members in June 2001, where the WSA were openly questioned by many members about the verification procedure, and in particular what had happened to the budget supposed to be in place for chemical analysis of submitted samples. The Consortium was reassured that this was still in place. Subsequent to this meeting, the steps to rectify the verification problem ‘weren’t aggressive enough’, and Oddbins eventually withdrew in October. When questioned about the overall budget for the survey, the WSA’s Rappoport revealed that it was in the order of £30 000—so it’s hardly surprising that there wasn’t any chemical analysis.

Fudge factors
Aside from the rather opaque verification process, Oddbins’ criticism of the study highlights one of a couple of fudge factors that could be in part responsible for the low final rate of taint claimed by the survey. The emphasis of the project was on ‘defects considered to be at a level that is likely to be detected by a discerning consumer’. Rappoport confirms that ‘we were not measuring incidence [of cork taint] in terms of zero tolerance’. This leads us to an unanswerable question: what constitutes commercially significant musty taint? Who gets to decide this?  It is known that people differ in their sensitivity to TCA; what is not clear is that a discerning consumer is any less sensitive to TCA than the professional assessors in this survey. What about low-level cork taint that introduces a very faint musty taint and strips the wine of its fruit?

A second fudge factor is the fact that the final rate of musty taint quoted by the report includes all closure types, not just cork-based ones. Ironically, while the report studiously avoids using the term ‘cork taint’, none of the 1934 wines that were sealed with non-cork closures showed any mustiness—an important observation.

Faulty methodology
But the most damning criticism of the WSA’s methodology comes from the findings of the Australian Wine Research Institute (AWRI). A single 10 minute phone conversation with the AWRI’s Peter Godden was enough to expose the gaping holes into the scientific design of this survey.

There are two fundamental assumptions underlying the verification step in the WSA survey. The first is that TCA is stable enough that musty taint detected by assessors will still be detectable by the verifying laboratories up to a week later, when the wines are re-tasted. The second is that TCA is readily detectable against a background of oxidation, which will have occurred between the first tasting and the retasting of the ullaged bottles. Godden thinks that both of these assumptions are false.

When the AWRI began their first closure survey three years ago, they tested the stability of TCA in opened bottles of corked wine. ‘We tested ullaged bottles with a reasonably high level of TCA—15 ng/l. Ullaged bottles were recorked and left on a desk for 2 weeks. Just a trace of TCA was found: all the rest had been absorbed back into the cork.’ Godden thinks it is ‘quite probable’ that most of the TCA in the musty wines submitted for verification could have been absorbed back into the cork. For this reason, in their studies the AWRI insist that samples to be tested later for TCA should be transferred after opening to all-glass containers (with ground glass stoppers) or glass bottles with an aluminium foil barrier between the wine and the stopper. Plastic is no good because the TCA will all be absorbed by the plastic within a few days. 

Godden also strongly disagrees that musty off-flavours will be readily detectable over the background of oxidation. ‘In the last 6 months’, he says, ‘we have done a major investigation in an insurance case where there has been random bottle oxidation, in which we have investigated how oxidation affects the perception of TCA.’ The AWRI carried out sensory analysis of all the bottles, together with chemical analysis of the oxidised bottles. The conclusion? ‘Oxidation has a massive effect on the ability of experienced tasters to assess TCA’.

‘We expected the cork people to be touting these WSA results more’, adds Godden. He guesses that the reason the pro-cork lobby haven’t done this is that they may suspect there is a problem with the study.

Like many others, Godden is convinced that the real level of cork taint is substantially higher than the 0.7% claimed by the WSA study. Once a year the AWRI run a wine assessment course for potential wine show judges. In this event there are at least two bottles of the same wine open at the same time, and for a wine to be recorded as tainted with TCA, there as to be an overwhelming consensus. ‘We’ve got good stats that 62 out of 1062 bottles we have opened have been corked’, says Godden. ‘That’s 5.5%, and statistically, we can be 99% confident that the real level of taint is 4%–7.7%.’

Conclusions
Where does this leave the WSA survey? First, it provides an explanation for the controversial discrepancy between the number of submitted musty samples and those that were verified as musty by the independent labs, making a mockery of the survey methodology in the process. Second, it means that the bizarrely low quoted rate of musty taint or 0.7% is anything but a ‘benchmark baseline’. Instead, it can be dismissed as an artefact of a flawed methodology. While credit is due to the WSA for initiating this survey in the first place, it is a shame that the poor study design meant that this turned out to be a largely wasted opportunity. In addition, there are good grounds for suggesting that the final report— available free to journalists but otherwise £500 a pop—should be rewritten in light of the methodological problems exposed here.  

Back to top