The Wiseman ‘Study’ – cheap versus expensive wine

Much has been written about a scientific ‘study’ by psychologist Richard Wiseman which showed that people can’t tell the difference between cheap and expensive wine. (See, for example, this report this report in The Guardian and this report in the Telegaph.)

But, in this world of soundbyte news reporting, it’s just the headline message that is reported, with few of the important details. So I feel I need to make some comments, because the coverage has left everyone with the impression that there is no readily discernable difference between cheap and expensive wine.

This is a message many people would like to believe, but it’s not true.

Wiseman, a psychologist from the Hertfordshire University, is one of the seven-strong advisory group for the Edinburgh International Science Festival (www.sciencefestival.co.uk), and it seems that this ‘study’ was in essence a clever publicity stunt to boost the profile of the festival by generating column inches—one that worked very well. It was not designed as a proper scientific study, although this is how it has been reported.

Almost all the news articles on Wiseman’s stunt lack any comment on the methodology, beyond reporting that 578 festival-goers were given wine to taste ranging from cheap to more expensive, and that they couldn’t tell the difference.

There is a single, crucial detail that is absent from these reports. It was not a comparison between two wines, one cheap and one expensive. Instead, subjects were given just a single wine to taste, and then asked to say whether it was cheap or expensive. ‘To keep it as realistic as possible, we presented them with a single glass of wine and they had to say whether inexpensive or expensive,’ revealed Wiseman when I asked him about this.

This makes the results, which showed that people had a more-or-less random chance of getting it right, unsurprising.

Tasting wine blind is difficult. Being asked whether a wine in front of you is expensive or inexpensive is a difficult task indeed. It would still be difficult, but considerably less so, if the subjects had been offered a comparison of two wines to chose between.

There is also a confounding factor. For supermarket wines, such as those used in this study, quality and price are not always well aligned.

Wines purchased from the producer for the same cost can end up at very different prices. A supermarket may pay a grower 1 Euro per bottle and then list the wine at £4.99 (a standard mark-up, given tax and logistical costs). They may then buy another wine for 1 Euro and list it at £7.99 or even £9.99, with a view to discounting it later. This is common practice. Also, you could get a much better wine for £6.99 from a less fashionable region (e.g. the Languedoc) than from one more fashionable (e.g. Burgundy), simply because some wine regions offer much better value for money.

The relationship between price and quality is so tenuous in supermarkets that, in this sort of study, you could get pretty much whatever result you wanted if you were canny about which wines you chose. I’m not suggesting this is what went on here, but it’s certainly a confounding factor.

Then there is the more telling point about the value of expertise. There are many areas of human endeavour where I am unable to discern absolute quality because I lack expertise. Take visual art: I am not well enough versed in this area to tell you what is good and bad, but my lack of ability to discern quality doesn’t lead me to dismiss the entire enterprise.

Even if Wiseman’s study was rigorous and fair (which I don’t think it is), and came out with the same results, the lack of the ability of novice or inexperienced wine drinkers to discern quality in wine should not lead commentators to suggest that fine wine is all in the head, or is a load of nonsense.

Of course, knowing about a wine changes our perception of that wine. This is a really interesting and important observation. But aside from this, there are genuine differences in character and quality in wines that have their genesis in the chemical composition of the wine, itself influenced by the site characteristics of the vineyard, the growing season, and the winemaking process. It is these very real differences that make wine so fascinating. And it would be a shame if, through a publicity stunt dressed up as a piece of scientific research, people were put off exploring wine because they were led to believe that differences in wine quality are actually illusory.

30 comments to The Wiseman ‘Study’ – cheap versus expensive wine

  • Good point about art. Or indeed designer clothes or more topically topically wedding dresses.

    9 out of 10 people can’t tell a Royal wedding dress from a Primark Wedding dress.

  • Tom

    Like you, Jamie, I was troubled by this article and, without knowing the details you have unearthed, I felt on a “gut-level” that the findings were misleadingly presented.

    I draw a number of additional conclusions which I am currently writing up for my own blog:

    - the study showed people cannot taste the difference, not that the differences do not exist

    - this highlights the need for greater wine education, rather than providing damning proof that more expensive wine is no better than plonk

    - more expensive wines are often more challenging and, to a novice accustomed to crowd-pleasers, can seem less enjoyable than a cheaper wine

    - in the UK, we are naturally a nation of beer drinkers as we have no real culture of making wine, so wine is simply not as much a part of our everyday lives as it is in France, say, where people grow up understanding what makes a good wine

    Cheers, Tom

  • Kieren Johnson

    Great article that should be published in all newspapers that joyfully jumped on the original story.

  • keith prothero

    I held a blind tasting,a few years ago, with a dozen keen and knowledgeable wine enthusiasts,in which I produced about a dozen pair of wines,and the guests were asked to advise which of the wines they preferred and which they thought were the expensive and cheaper wines.
    Interesting to advise,that out of the dozen or so pairs,the cheaper wine was preferred in most cases, and also naturally, thought to be the most expensive.
    The flights were organised by style of wine from sparkling to full bodied red.

  • Louise Hurren

    The problem with this kind of sloppy research – and the consequent reporting of it by the media – is that it just snowballs. Last night as I skimmed through Facebook, there was the Wiseman “study” and its “findings” dutifully reproduced in Portuguese by a winery rep who no doubt felt they were doing the right thing by sharing this “information”. Two weeks ago, it was the French wine twitter peeps who were chortling over the same story. People don’t think hard enough about the so-called study, who carried it out and what their agenda might have been. They’re happy enough to just translate, reweet, share… and the message – no matter how lightweight – becomes gospel. Or transformed, like Chinese Whispers, into “hah! there’s no difference between cheap and expensive wine”.

  • Ralph

    @ Tom: Most French people I know, know zip about wine. They drink the local stuff and know next to nothing about the village on the other side of the hill.
    You’re right though about the cheap tasting expensive and vice versa.

  • Charles Mutter

    Thanks Jamie! Excellent article.

  • As a wine maker and an engineer for several years, I find the argument futile. Wine is like many other commodities that require fundamentally good ingredients to start with, good processes to manufacture, and good presentation skills to enhance the Image of the product. The rest is very subjective. In fact, personally, I find that there is a diminishing return in quality as as the price goes up (I have not tried wines that cost thousands $$$ each bottle) The analogy of electronic stereo systems is very valid here.
    Do not forget that the experience of drinking has so much value too, I do not like to drink wine while I am doing something else as it detracts from the pleasure.
    Next time you drink a glass of wine, sit in a pleasurable location, surround yourself with great friends, have something to accompany your wine, and I trust you will enjoy even the most modest wine you purchase. If not come and try my wines.

  • The best way for me to determine the quality of wine is to open three similar wines with dinner. At first taste, some wines are pleasing. But soon, the wines reveal their true selves. Residual sugar in some wines becomes overwhelming, low acid wines just don’t refresh, others are so intensely flavored to be tiresome. At the end of the meal, I always check for the empty bottle on the table, for it is the wine that stood a true taste test.

  • Alex Lake

    A good article and depressing. I think while some people will use it as continuing evidence that wine geeks are self-deluded and others that it’s yet another demonstration that “scientists” can have no shame in undermining their “profession” with such ill-considered publicity stunts…

    Having said that, it’s most probably true that there are so many confounding factors that’s fair to say that you don’t usually get what you pay for, especially if you’re not very experienced with wine.

    What Keith doesn’t say is that he still buys and drinks expensive wine (nor that his “cheap” and “expensive” wines were actually “expensive” and “ridiculously expensive”! Blind tasting is not an effective model for measuring enjoyability of normal drinking, much as one’s inner pragmatist would like it to be.

  • Great explanation. We all know that a survey can only be as accurate as the size of the surveyed population.
    In blind tastings there is always someone who will prefer the ‘supermarket’ wine, expensive and inexpensive are rather relative terms after all. I believe the scientific explanation to this phenomenon is that this particular individual may just really like that wine.

  • Alex Lake

    Oops, meant “cheaper” rather than “cheap” – sorry Keith

  • Isn’t there another subjective issue here?

    What’s “expensive”?

  • Spot on Jamie. If Wiseman really is a psychologist (and therefore part of the scientific community) he ought to be ashamed of himself. Surely someone up at the Edinburgh International Science Festival ought to have picked him up on his methodology? 578 scientists obviously didn’t!

    Is there a website for shaming psuedo-scientists like Wiseman?

    That said, I do (anecdotally) think that price is a poor proxy for Beauty in the world of wine. As you point out the “eye of the beholder” is as important as the intrinsic Beauty of the object. And Supermarket and wine trade pricing policies are tragic at best, fraud more often.

    Perhaps a better headline for the articles should read:
    Wiseman study exposes 578 scientists as fools with no taste!

  • Very good point, The Sediment blog. Maybe we should think about this issue more carefully. To be honest, I find that a good Bordeaux at about 25-30 Euros, to be relatively cheap…

  • I think the trial was markedly flawed. Our private customers often trade up after tasting a wine that is a pound or two more than their standard price and appreciate our recommendation.

  • Charmion

    The real loser is the legitimacy of university studies and the world of academics. Poorly designed, or publicity stunts, it makes people wonder if they should believe anything reported out from academic research, and if they are wasting their money sending the kids to college or subsidizing ivory tower eggheads via tax dollars.

  • Thanks for posting this article. The study had bothered me as well.

    I think the study should have addressed three separate questions, as follows.

    (1) Can people detect any difference at all between cheap and expensive wine?
    To determine this, present people with three glasses, two of which are identical (either two cheap and one expensive or two expensive and one cheap). People are are told only that one wine is different from the other two, and they are asked to identify the wine that is different. It seems to me that this is the proper way to see if people can tell any difference at all between cheap and expensive wine. (Of course, as pointed out in the post, canny selection of the wines could still be an issue.) If, as I expect, people can distinguish between cheap and expensive wines, then the following two questions become of interest.

    (2) Do people prefer expensive to cheap wine?
    Ask people which wine they prefer, the one they identified as different from the other two, or the two they believe to be the same. I would expect that people would disagree with one another, although the aggregate results could go either way because of the loose relation between price and quality that you mentioned, and because expensive wines may be more challenging, as Tom pointed out.

    (3) Can people tell which wine is more expensive?
    Simply ask people which wine they think is more expensive. People may not know how to detect features in wine that signal its likely price, but as a heuristic they may presume that whichever wine they prefer is the more expensive, which is why it would be important to align their answers with the preferences collected in (2).

    Thanks again for this post.

  • Philip

    This received front page coverage in the Daily Telegraph. I am entirely unable to tell the difference between the Daily Telegraph (price £1.00) and the Daily Mail (price 50p), so I guess the cheaper option is better.

  • Doug

    Between the cheap and expensive wine I prefer the middle one…

  • Alex Lake

    @rob – Ben Goldacre’s BadScience website http://www.badscience.net/ fulfils that purpose. I’m sure Wiseman has previous. He’s the Malcolm Gluck of Science…

  • Philip – interestingly, the Daily Mail reporting on this “experiment” was a lot better than the Telegraph’s!

    I think Wiseman, though a self publicist, is a respectible enough scientist, and he has an interesting and funn website. But, apart from issues with the methodology he used here, I fail to see what at all it has to do with science. What scientific theory was he trying to disprove or support?

    The conclusions drawn by the press have a lot more to do with economics and marketing of wine than science. You could say they raise more questions about how wine is priced than the taste of individuals. They are doubtless correct, but they have been supported a lot better by previous work done in “The Wine Trials”, which I discussed here: http://www.winenous.co.uk/wp/archives/691

  • Alex Lake

    Er, the “theory” that people can’t tell the difference between cheap and expensive wine?

  • But the price of wine is scientifically arbitrary, Alex. From day to day it is determined by discounting and special offers – in the longer terms it is dependent on changes in supply and demand. It is in the realm of ecomonics, not science. Because wine prices are so subject to variation “your” theory is not at all general and it has no predictive power – it is not scientific.

    That is not to say that the subject is not interesting. I certainly think it is. But it tells us more about economics than psychology.

  • Henk Bruinse

    Jamie,

    You have a point with the different price levels in combination with the lack of experience.

    Even with a comparison there is a possibility of a guess, due to the lack of experience. You can explore wine knowledge by using a comparison.

    More important is the lack of a control group and the composition of the participants in this study.

    The only conclusion is: 578 people at the festival can’t tell the difference between cheap and expensive wine.

    This is not true for the rest of the world (and not necessarily true for the participants of the next festival).

    The risk of reacting defensively creates a new (subjective) reality with negative consequences. It is more powerful to do something.

    The result of this research may lead to further scientific research.

  • Excellent column with good arguments. There is more material on the net, but most of it is just repeating the press release or other journalists. For some extra arguments and facts, see Wiseman on Wine: maybe the right conclusion, but invalid nonetheless.

  • Jem

    I see HRH has weighed in now and links to this article too

  • Alex Lake

    Henk, Dunno if you’re still reading, but surely the conclusion should be “578 people at the festival can’t tell if a wine served blind is deemed ‘cheap’ or ‘expensive’”

  • Emma B

    The expensive wines weren’t particularly expensive – IIRC they ran from a tenner to just under £30. At times the wines being “compared” were only separated by about £5. So even if the methodology wasn’t already clearly flawed, the choices of materials to be tested were absurd.

Leave a Reply

 

 

 

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

*