I posted a few days ago on an article in a trade publication based on an MW dissertation by Alison Eisermann-Ctercteko that examined the extent of screwcap damage in the retail chain, and the implications for wine quality. This caused quite a storm, because it claimed that 26% of all caps showed some physical damage, and over 8% of screwcapped wines showed sufficient cap damage to cause significant chemical changes.
The IMW kindly sent me a copy of the dissertation so I could see the data, and after having spent some time examining it, I think this figure of 8.2% is misleading.
Eisermann-Ctercteko examined over 10 000 bottles for damage, and then purchased 600 bottles of a 2012 Sauvignon Blanc for chemical analysis. She did a formal trial of 444 bottles with increasing cap damage, which she classified from A-D in ascending severity. There was also a class E which was where the cap was undamaged but hadn’t been applied correctly.
There were 84 controls and 72 wines in each category, with the damage inflicted artificially on each damaged bottle to the appropriate level. The wines were stored for 10 weeks, and a number examined chemically at two week intervals, with the largest sample being at the end of the 10 weeks when the remaining bottles were tested. The chemical analysis was for absorbance at 420 nm (looking at browning, a marker of oxidation) and free and total sulphur dioxide (SO2, again a marker of oxidation). Sensory analysis was also carried out.
Now let’s look at the data. Damage categories B-D all result in significant chemical changes, which is where the figure of 8.2% comes from (8.2% of bottles in retail have damage levels in these categories). But while these changes are statistically significant by some statistical tests, they are not very large, with the exception of damage category D.
Looking at the error bars (and let’s focus on the results at week 10, the most important because of the large sample size and because the analysis was by the AWRI), the only striking result is at damage level D.
Let’s now look at free SO2.
Let’s concentrate on week 10. The free SO2 levels are 33.9 (control), 32.8 (A), 32.7 (B), 32.6 (C) and 33.4 (E). D is the only significantly different level at 29.65. However, the technique used to examine the statistics, REML, does show significance for A-D at 10 weeks, apparently. I find this hard to see from the graph. Irrespective of this, the differences in free SO2 in A-C and E are nothing at all troubling. Even the level for D is pretty healthy, and not a cause for alarm.
The conclusion I would draw from these data is that damage level D is the only one I’d be worried about, and even then, I wouldn’t be too upset. Screwcaps in this setting seem pretty robust. And even D isn’t as bad as cork taint. In the sensory analysis, no significant differences were found by the panellists for any of the damage categories.
Here’s a picture illustrating just how badly damaged level D is:
From Alison’s data on the incidence of level of damage found in retail stores, that figure of 8.2% now becomes 0.06% (the incidence of level D damage found). It’s a much less alarming figure.
It should be added that this work forms just part of the dissertation, and while I disagree with the interpretation on this point, the experimental method and the work involved is very impressive, and cost a good deal of money (she wasn’t funded in any way). The survey data on the number of damaged screwcaps in retail stores are important, and while this doesn’t seem to be resulting in many sensory changes in the wine over the short term, it points out a potential issue that the supply chain and retail trade needs to address.
16 Comments on More on the MW dissertation claims about screwcap damage
I can only go on what you’ve written, not having read the original. But the salient thing seems to be the level D damage results in some detectable laboratory differences, but no difference at 10 weeks for just purely sensory analysis.
Additionally to find her 72 bottles with level D damage she examined 10000 bottles. I would conclude this means that there is potential cellaring problem in 0.072% of Sauvignon blanc sealed with screwcap. This seems of marginal concern given that most consumers inspect bottles prior to purchase and presumably most retailers would be wary about selling wines with visibly defective seals.
This is certainly seems like a non issue. I wonder why there was such an attention seeking title and a misleadingly high figure of 8.2% attached to the original summary.
Excellent work Jamie, pointing out what we used to call when I was at Oxford “That’s marginal value epidemiology, also known as crap!”
So many stories, usually health scares, are based of this type of analysis of low probability edge-risk cases and no one seems to pick up that.it’s all a load of dingoes kidneys. So great reporting, Jamie, I’m pleased to see drivel being pointed out as such.
Thanks, I played with some numbers but couldn’t hit 8.2; 8.16 and 8.33 yes…
I was one of the first US retailers to actively promote screwcaps when the 2000 Clare Valley rieslings first arrived and then later with NZ wines. The two categories comprised 20% of our total wines sales for almost a decade which means we pulled a lot of screwcapped wines out of their boxes to be priced and then placed on our shelves. That’s a lot of data points.
I can recall less than a dozen bottles, if that, which were damaged enough to cause me any worry. And we had no one return their wines to us because they were faulty. We had WAY more bottles returned because of cork taint. I know which closure I would prefer and so have our customers. Except for those luddites who still prefer cork.
What next, research suggesting that wine doesn’t mature well in broken bottles?
At least the kind of damage that results in degradation of flavour is bloomin’ obvious. Unlike that with cork. The vast majority of bottles like that would get left on the shelf anyway.
Hi Jamie
Did it say on those graphs what were the error bars actually where ie 95% confidence intervals, standard deviation or standard error of mean?
Also did they do any statistical tests or correlation analysis?
EVERYTHING was between 29.5-37.5mg/ SO2?? Every bottle?? Even the worst category! I re-brand this a roaring success.
Now a dissertation worth reading would be arranging the exact same experiment with cork closure which have had good metrics done on them before bottling, and comparing results directly.
At least that’s what I’d do 😀
This post made me wish I had spent more time with math and less with history.
I always thought, even with the higher levels, that it was as Lew Newby pointed out, a supply chain issue. In addition, and this is the most intriguing part to me, how old were the 10,000 wines? How long had they been sitting in the store? How much did they cost? If we’re talking about oxidation, then we need to know if she tested cheaply made and older wines, which are more likely to oxidize.
Well Done Jamie
Hi Jamie,
I am sorry that there is still seems to be a misinterpretation of the data. The tables of data in the results for A420, TSO2, FSO2 show the mean changes and SD to levels of these components following analysis. The statistical analysis undertaken with the guidance of Senior lecturer in Statistics and Mathematics confirmed the significance of the data. While the oxidation changes are small they are present and after a very short time period (4weeks) which is of most concern. I think you have focused on one only part of the research, which I stand by with confidence, of which there were many more highly relevant to the trade such as the overall cap damage level of 26%, that wines in wooden and metal display racks have 56% damage level! that coloured and printed screw caps have 4-5 times damage level of standard black caps etc. The scepticism of your followers shows many have not read the initial journal article but happy to follow your opinion and interpretation. Many bloggers have attacked my research, the dissertation and the IMW. They may like to first view the article form their own opinion and take up the MW challenge!
Hi Alison, I think you are right, it will be interesting to read the original article. Is it a published article or the dissertation (which is, I understand, not available to the public)? Can you please tell me where I can find it?
Hi Alison, I would really like to read the original article as since 2001 in my business I have only come across 3 bottles where the screwcap failed to do its job, 2 of those were not sealed/crimped properly & only 1 was due to a damaged screwcap. In the same time I have opened 187 corked bottles of wine including a bottle of 1990 Chateau Lafite which I had cellared for my son’s 21st birthday – such a disappointment. I am an advocate for progress and technology- be it screwcap or glass stopper or what ever comes next- can’t wait until the middle ages technology of cork is no longer used in any wine that I buy.
Dan Traucki
Wine Assist P/L.
Hi Jamie,
We were interested to read the article on your blog and the different comments to date. The numbers quoted by Alison certainly don’t reflect the market experience of the members of the EAFA Closures Group (http://www.aluminium-closures.org) and we would certainly be amongst the first to know! The EAFA Closures Group is an association representing about 75% of the global aluminium closures production. Season’s Greetings from us all!
I think the issue with any sample data like this is that there are so many variables that controlling them all to then make meaningful conclusions on those things that you look to measure is very very hard indeed.
I haven’t read the paper – but nobody has said where Alison studied the 10,000 bottles to establish the number that contained some damage – because you’d have thought that knowing that would take the study further. Were they all in large retailers – a single large retailer, multiple types, cellar doors etc…?
Obviously the 8% figure for number of damaged caps is a worry – even if after 10 weeks there was no difference in the wine at all – because longer term storage could easily throw up further issues – 10 weeks really isn’t a very long time at all – particularly for finer wines.
Sauvignon is in some ways a good choice – but in others you could say it’s a variety picked to show this up the most – I mean pick Cabernet and you’d probably get naff all data to go on, an oaked Chardonnay – far less data I would guess.
As you rightly point out there are loads of different screw caps available – and I do wonder whether that is perhaps more significant – is there a type or brand that is more likely to be damaged, or more likely to fail?
I don’t think we should just ignore the warning signs in the data – but I do think we ought to urge caution about drawing conclusions based on one study with many variables unclear.
It’s also interesting when you artificially inflict damage on things – how was that done – and although the physical look of the damage may have been the same – was the unseen damage – to things like the final seal – actually the same – impossible to know for all sorts of reasons.
Hi Jamie, Alison et al.
Useful work that bears out the thinking that screw cap is the best closure for wines destined to be drunk young, when in fact most wines are drunk. It does not however prove that screw cap is a useful closure for wines designed to be aged. Would any of the wines with damaged closures have survived 20, 10 or even 5 years of ageing? 8% failure rate is way, way higher that the rate of cork taint these days which in my experience is well under 1%. 8% is about the same as the level of cork taint from the early 1990’s, that everyone agreed was completely unacceptable. On top of that you do have some potential problems with reduction and ‘weird direction’ ageing on the wines under undamaged screw caps over the long term.
Regards to all. Tim
we started in 1998 with the Watervale Richmond Grove Riesling with blessing of John Vickery & Phil Laffer. NO ONE was under screwcap then in Clare, Marlborough or anywhere else….. Those wines look so good today. So proud