Top
Ten, Top One Hundred, Wine of the Year, wine ratings abound and
they are mostly baloney. The business of distributing and selling
wine is intensely competitive. No doubt, there are winners and
losers in the business of wine. However wine, the product, just
doesn’t lend itself to point scores and rankings. The universe
of wines is too large and too diverse for any single approach
to ranking. For example, the region of Bordeaux has thousands
of products reaching the market at any given time. Even the most
exhaustive survey tasting can’t handle a significant minority
of the Bordeaux wine available. There is no chance of being comprehensive.
Diversity
is another problem for those keeping score. The wines of each
region (and even each hamlet within a region) are different. In
fact, that is what makes wine drinking such a fascinating pursuit.
It is preposterous to compare ratings of markedly dissimilar things.
How can we say whether a particularly fine Port is better than
a crisp dry White Burgundy? People may be prepared to argue whether
Tiger Woods is greater than Bjorn Borg or Pete Rose, but the exercise
is silly on many counts. Thank goodness there is no way to quantify
excellence.
Even
when the field is limited to a single type of wine and a reasonable
number of entrants, ranking is problematic. The order of service
can greatly alter outcomes. When "blind" tastings are
repeated, the outcomes vary greatly when the order of presentation
is changed. In fact, in a field of ten, it is best to be either
first or near the end. Wine number two has almost no chance of
winning. Prevailing style can alter the fairness of a judging.
A light crisp Chardonnay (an admirable style for the dinner table)
may not have a chance among big, rich, buttery competition. A
wine that might be independently judged elegant and well balanced,
might seem too light among blockbuster high-alcohol reds. In such
cases the "loudest" wines win.
There
are other, more subtle problems when committees are involved.
Averaging the scores of several judges is a common technique,
which presents problems. I once judged Chardonnays at a national
event. There were hundreds of entries and tastings were done in
three sessions. None of the judges could work all of the sessions
so three judges were assigned to each group and their scores were
averaged to determine the winners. After the fact, I checked the
range of scores and found that some judges scored high and others
(especially myself) were conservative. I thought a score of 90
out of a possible 100 should indicate a great wine. Others seem
to offer many scores in the upper 90’s. I calculated that no wine
that was in the group I had rated had any chance of coming in
first because of my conservatism. The organizers seemed unable
to understand the unfairness of this. Incidentally, I have avoided
further anxiety about this by refusing to serve as a judge ever
again. On the whole, I would rather read the comments of a single
expert than review the work of a committee. Averaging a number
of opinions may seem scientific but it is not.
Style
prejudice is unavoidable. No matter how well intentioned, a judge
can’t shake his past. Those of us who came to love wine in an
era when most of the better wine available was from Europe can
hardly help judging "New World" wines by the standards
of their predecessors. By contrast, many recent wine enthusiasts,
are well versed in California wines and can’t fairly judge a French
wine of very different style. In fact, there is no taster free
of prejudice. Perhaps, its best to take advice from "judges"
expert in the particular wine being considered.
Consider
a few more aspects of ratings. Magazines publishing most wine
ratings derive considerable income from advertising the same products.
No rating system offers a money back guarantee for the product,
while most retailers do. Most great ratings reach the public after
the wine is sold out anyway. Shopping for "top rated"
wines virtually assures paying inflated prices (some reviewers
send their results to the trade ahead of publication). Buyer,
beware – of ratings.
Back to List of Articles