Whisky scores

I'm guilty of this myself - for me personally a 5 would be abominable, 6 would be poor, a 7 is an average whisky, 8 is a good whisky, 9 is excellent, and a 9.5 leaves me thinking: 'this is briliiant!' Of course, it is subjective, and sometimes that brilliance may be more about value for money (and the HP 12 is brilliant by that measure) than actual brilliance of whisky. Mind you, I've never agreed with 50% being a pass mark - how can you call it a pass when you get nearly 50% of something wrong? That sounds like the person being tested has barely any idea at all! 70% seems much more reasonable - at least you got far more right than you got wrong!

All that aside (and now I go into personal defensive mode cause my review of the Laphroaig 18 year old could be the only one you mentioned), I've only tasted one whisky I would rate higher than the current batch (and it would have to be the current batch as previous reviews speak of a completely different whisky than what I have in the bottle on my shelf) of Laphroaig 18 year olds, and that would be the Ardbeg Uigedail of a couple of years ago (the current one is still great, but is just not quite as inspiring). I know, your comment wasn't meant personally, but I was genuinely blown away by the Laphroaig 18 year old! So nerr! (He says stamping his feet). And the guys behind the counter where I purchase were impressed. That's the primary reason I purchased it - because it was quoted to me as the best whisky released in the last 18 months, by a person who is required to taste all the whiskies that come into the store (my dream job incidentally). My drinking buddies have also been impressed. See if you can get your hands on a new bottle, you might be surprised...

I think part of the problem is that some people are obviously using the 100 point Jim Murray style system and converting an 85 to 8.5. As you know in that system 75 is a pretty poor score for average whisky. If these reviewers gave something a 6 it would mean they are recommending you only use it to clean the toilet.

Its seems to me that the community might need to agree some guidelines.

How about something like this:

• 1-5: awful to average whisky
• 6: drinkable but hardly outstanding
• 7: good whisky (maybe taking into account price)
• 8: great whisky
• 9: outstanding whisky
• 9.5: if you taste one whisky make it this
• 10: you can die happy if you've drunk this

I think a 10 star system is too limited anyway. I don't convert 90/100 to 9 stars because the majority of whisky is in the 80-90 region. That would mean 75% of all whiskies get 8 or 8.5 stars which makes the whole system pretty useless - only the really good ones and the really bad ones stand out and the middle part is really vague. For example, 88/100 or 92/100 is a huge difference to me, and they would both get 9 stars.

On my blog, I use a 100 scale. When converting to a Connosr score, I subtract 50 and divide by 5 which stretches the whole scale. 50/100 = 0 stars, 60/100 = 2 stars, etc. I think this gives a better idea about how good / bad it really is, with enough margin to include absolutely terrific stuff.

@jdcook: simply giving 9.5 to whatever is the best whisky you've tasted until then, is not a system you can use for years, I'm afraid, and it makes it necessary to re-score all your previous stuff once you find a better whisky.

By the way, I understand that value/money seems to be a good measure for scoring whisky, but it becomes useless once you want to score whisky bottled in the 1960's or a old bottling that fetches 100x more than its initial cost.

@WhiskyNotes Ruben, if i absolutely LOVE the Corryvreckan and think it's awesome, and havent tasted an ardbeg 1974, or if i preder my malts to be young and mascular. how would i rate the Corry? 7?

@WhiskyNotes How do you break down the marks out of 100? For example, is it made up from 4 areas with each of the following getting a score of 25:

• Nose
• Taste
• Finish
• Complexity

Perhaps 'value for money' should be a separate mark completely? That way removing the need to include it in the overall score.

In my personal opinion value for money is not relevant in scoring a whisky. It is a factor that I personally use for determining if I want to buy a whisky though. But in the end it is all about the taste (smell, finish, ...) for me and price is something that should'nt have an influence on my experience of a whisky. It's just a factor in the decision to buy it.

WhiskyNotes makes a very good point here. As a novice, I have absolutely no idea how to score a whisky and hence use an overall score (nose 25, palate 25, finish 25 and balance 25). But as I learn to appreciate more rare whiskies (like at a recent Malts of Scotland tasting), I feel like I should deduct at least 3 stars out of all my entry malt scores. Having said that, though, I would be happier to merely review without scoring at all, but this site doesn't allow for reviews without scores.

@galg You're right, a personal preference will lead to a high score for certain types of whisky. If you happen to like young and heavily peated whisky, then Corryvreckan will get a very high score. In that case, you're in line with Jim Murray. But still you need to be consistent, if you score Corryvreckan 9.5 then you wouldn't score a standard blend 8.0 but something like 3.0 (I'm just giving an example here - I notice that lots of people seem to like everything which I find unlikely...).

@LeFrog Yes, it's the standard system of nose/mouth/finish/balance. And you're right, value for money is something you should calculate after giving a quality score, and not as part of the score itself. BTW I sometimes add one or two bonus points when a certain whisky is extremely cheap considering the quality, but that's peanuts...

@Connosr In future releases, would you consider allowing users to choose how they score their reviews?

Perhaps choosing between a 0-10 star rating scale OR a 0-100 system?

Or allow publication of tasting notes without scoring stars at all?

Thanks to everyone for the suggestions above - this is a very constructive thread. As you know we aim to be as reactive as possible to the needs of the community so we'll take all these suggestions into account.

Please keep your ideas coming if you have any further thoughts, they are heard.

Connosr

Good questions. The more I score whiskies, the more I take them to be subjective not just of myself, but at a particular point in time - which infers a particular mood, expectations, etc. Ever had an album that you bought, didn't really care for, then come back to and loved after not hearing it for a few years? "Scoring" is ephemeral.

Any system has its drawbacks. For me, it's nice to look at a score - any score, A-E, 0-10, 0-100%, etc - as it sums up the reviewer's overall impression at the time of review. That's what I think is important.

After all, this isn't a test, right?

The 10 star system is useless, mostly because it does not allow for much variety - for instance, I average a score of 88, that would be 9 stars in Connosr ratings. However, the difference from a 88 to a 95 is HUGE. 88 is a very fine decent dram, while 95 is a fantastic, mind blowing dram - however in the star rating system, there'd only be a difference of ½ star.

Obviously someone will feel that the aforementioned HP 12 is the best dram in the world, and score it thereafter, and that's fine - but it's really a shame that you loose a lot of flexibility with such a rigid scoring system as the 10 star system.

I use the 100 rating system myself, 80-85 are good drams, 86-90 are really good drams, 91-94 are terrific drams and 95 and above are just unbelievable. I don't hand out individual scores for nose, taste, finish and balance, although I do take note of these and which I like best - but I'd much prefer to try and describe what these are like, and hand out a combined rating.

I find myself squarely in the @markjedi1 camp of wanting the ability to post tasting notes without a score. As @WhiskyNotes sagaciously suggested, I had already decided not to begin scoring anything until I felt like my experience was broad enough to provide sufficient basis for comparison. It would be nice however, if only for my own record, to be able to post tasting notes here without trying to assign a numeric rating. And I would like to read the same as well. As has been mentioned several times in this thread (and I agree) most rating scales are kind of useless to someone who is looking for a new dram to try given that they're all subjective and use only a small sub-range of the available points of precision.

Having said that, I realize that this would not be of much use to the Popular Whiskies page.

I was going to call this a storm in a teacup, but it's irritated me enough that I'm going to write a bit of a rant.

In a nutshell, everyone should rate it their own way. If you don't read the rest of this - that;s the most important sentence!

You see, over time if everyone reviews their own way, with enough reviews, everything will end up being statistically normalised. In other words, if someone uses an equation to convert from the 'traditional' JM style of numbering to a 10 point system, then they will nearly always end up with a lower number than if someone converts directly. And if someone else consistently gives reviews a little too high, that will have a similar effect in the opposite direction. But if enough people review enough drams, then the average of each dram will be affected by a roughly consistent amount. You will start to get a bell curve with some statistical outliers, but the average will start to appear.

On the current system, I like the 10 stars system because it is simple, and anyone walking in can get a feel for it. If you love it, rate it high. If you are an experienced reviewer, you can come up with ways of differentiating between the 'wide-ranging' scores, hell, you could even give an exact rating out of 100 in the review itself. However, and this is the important bit, if you are new, it is fairly simple and non-threatening.

Whisky tasting, by its very nature, is subjective. It depends on what you like, how you feel, on what you ate most recently, how experienced you are, how old you are, where you live, your cultural and socio-economic background, as well as your current surroundings - and that's just off the top of my head. Trying to create a 'scientific' or even vaguely 'precise' setup is impossible. Anyone who has been drinking whisky for more than a few months will be aware that your own tastes change, and whiskies you used to love, are now merely good. And whiskies themselves change over time, a whisky that used to be rubbish can start releasing good products, and vice versa.

Let people review in their own way. Nearly every half good malt will get reviews it doesn't deserve, both good and bad. Take the HP 12 for example, sure it got a 9.5, but it also got a 5.5, a 3, a 6 and a 9. I personally am much more comfortable with a 9.5 for the HP 12 than I am with a 3, but I don't think it is a big deal. When I disagree I often comment on the review in a relatively positive way.

...and why shouldn't value for money not be considered? Why shouldn't any aspect of a whisky be considered? Not saying you have to take anything into consideration, but why not? A mate of mine rates taste as virtually unimportant - he thinks nose and finish are what makes a whisky to him. Is he wrong? Of course not! That's his take on it, so he would rate many whiskies significantly different to many people.

You don't have to agree with the numbers attached to a review, but I think they are a good idea, it lets people get a quick snapshot of how someone else felt about a dram. Key word is 'felt.' Fiddling with it would create confusion, and setting a deliberate scale would also make people wary of reviewing if they felt they didn't know enough.

...and yes @WhiskyNotes, looking back on my reviews, there are several I would like to re-review (and probably will do at some stage), and they will get different numbers, both higher and lower. I dare say that with your extensive history of reviews, you would say the same. That doesn't make the numbers anyone places on any review any less valid! If someone choose to give a whisky a 2.5, a 4, 7, or a 9.5 (or any other number), then that is their choice, and who has the right to tell anyone otherwise? Hell, I've disagreed with the numbers of just about anyone who has reviewed more than a couple of drams, including yours, but that certainly doesn't invalidate anyones point of view, because THAT'S WHAT A WHISKY REVIEW AND SCORE IS - AN OPINION.

Gah! /endrant

Actually, now that I've thought about it, I like the idea of going through and slowly re-reviewing everything in my cabinet, just to see how differently I feel about things. If nothing else, it will be interesting, and I'll have a reason to hit every bottle in my collection again!

Caveat: I haven't rated any whiskys here. Given that most of my consumption is beer and wine, I don't consider myself competent to review whisky. With that said...

I find the system used on a popular beer site (not sure about connosr's "name dropping policy" ... I'll just say the site in question 'advocates beer') to be very simple to use, not intimidating, mostly self-explanatory, and at the same time nuanced enough to provide for great overall ratings.

Beers are rated based on appearance, smell, taste, mouthfeel, and drinkability (a term which gets discussed and critiqued on the site). Each category is on a 9 point scale: 1-5 with .5 iterations from a drop down menu that includes words like 'awful' 'good' and 'exceptional' to guide the rater. The points are then weighted and averaged, resulting in beers rated from 1.00 - 5.00, which allows for plenty of nuance.

As I said, the system is easy to use and not scary, takes care of some of the subjectivity that comes from a simple overall 10pt scale, and allows for lots of nuance in ratings. How easy would it be to program into a web site? Hmmm.... what dram was I going to have tonight..?

@ocmpoma - I agree that system is excellent, but what happens when someone comes in who has tried precisely two whiskies in their life and has never tried to review anything before, and is suddenly asked to rate on things like appearance (admittedly easy-ish, but not when you have little to comapre it against), taste (this second whisky they've just had tastes fantastic, why else would they have gotten excited enough to look up a site like this!), mouthfeel (a new whisky drinker might not know what that even means - I'd struggle and I've probably had a couple of hundred) and drinkability (another word like mouthfeel that would be intimidating to new users).

As I see it, this place is either a social site for whisky lovers where anyone is free to come and review as they please, or it is a professionally run tasting association. If it is the first, the current system is fine. If it is the second, then we all seriously need to sort this out, and all reviews will need to be moderated, not just for inappropriate content, but for levels of professionalism as well!

And if it is decided to go down that path, count me out! I drink whisky for fun, and review it because I want to, and don't want to have to be stuck with anyones criteria but my own. I like that we have a simple system that different people can use in whatever way they want. If you want to use the four lots of 25 system, or some other system and then convert it to the star system, you can. If you want to have a more complex system, like @WhiskyNotes, then that too is fine. If you just want to pluck a number out of the air because of the gut feeling you get about a dram, then that works as well! It's not perfect, but it's so simple that anyone can use it without any training or experience at all.

@jdcook Thanks for your replies, I enjoy reading them. I'm not going to respond to everything you've said because it seems you're taking this even more seriously than I am, and I started this ;-)

Your last reply really made me think. I really love posting reviews here and discussing with you guys, but maybe the platform itself is not really my thing.

ps/ I'm still convinced value for money is not an important factor in scoring. I've lived in different countries so I know prices can differ greatly and not just as a whole, one specific malt can be relatively cheap while another one is unreasonably expensive when compared between countries. Most people don't realize this. You would at least need to mention the price you've paid when you consider it in your score, and even then prices change over time. What's the use of a score that's only valid today and in my own liquor store?

Sheesh, I can't believe I wrote all that! Sometimes you just start out with a pretty simple idea and then you explain it a little, and then a little more, and it just starts rolling, and rolling and rolling, and pretty soon you're getting RSI! Sorry to inflict such huge tracts of land... I mean text... upon people! And @WhiskyNotes, I certainly don't want you to stop reviewing, I love watching the professionals at work!

Gah! I'm going to stop before this gets any longer... :p

The funny thing is that we all score differently, but when you read the reviews (which you usually do before looking at the score) it's already clear whether it's a novice (like me) or a connoisseur (like WhiskyNotes). And that helps to interpret the score more correctly as well. On a side note, it's truly nice to see the community at work here. That's what this forum is all about (and yes, WN, I do think this is more of a 'popular' (no offense) platform than a professional one like your personal blog - which I also enjoy thoroughly).

@markjedi1 +1 for point on assessment of the reviewer.

FWIW, does anyone actually go out and buy whiskies based on a 10-point scale? While, say, Jim Murray's score is a nice indication of the "quality" of a whisky, for instance, it feels like most people buy whisky based more on recommendation, intrigue and (dare I say) marketing.

All good food/drink for thought. Will probably end up making my reviews more.. "contextual" ;-)

@scribe To be honest, I don't go out and buy whiskies based on seeing scores in any system. I buy either bottles that I think will be very different from anything else I've tried or bottles that sound like they will appeal to me based on comments and recommendations I've read on the Internet or heard from friends (textual ones, I mean, not numeric).

@JDCook For me the problem with the 10 star system is that it's too simple. Take the HP12, if someone scores it 3, then you'd see the score and think "Well that's a bad whisky then" - and that's not the case, however it might make sense seeing other scores and ratings that, 3 then, is a fair score if £200+ whiskys can only score 9.

But in reality the HP12 is a very solid dram, and in the 80s in the 100 rating, which would much better reflect it's quality level, you could have bad blends in the 60s, a shoddy single malt in the 70s and truly legendary malts in the 90s. But as it is right now, a malt will either get to high of a score, or a to low a score because all the scores are bunched so close together.

@WN I agree with you completely that value for money should NEVER influence your score. You might mention it, and say, "This is a really nice dram with great value for money" - but that does not change the fact that there's most likely, a ton of drams out there that are better. Sure they might be more expensive, but if they're better, they should fetch a higher score, and it should be up to the individual standing in the liqueur store how much, he's willing to pay for that little bit extra.

@Scribe I'd never buy anything based on Jim Murrays opinion, we all remember the famous Canadian bottling of Uigeadail. Basicly Mr. Murray lost pretty much all his credibility amongst connoisseurs some time ago. He's still a major force in spreading the word and getting novices involved in the world of single malts, but do take most things he says with a grain of salt. :)

I think there's an overall desire to change the scoring, but no real agreement on what'd work best. Seems to me like people should be able to choose score/no score, and people who feel the need to score, well - maybe they need to be moderated or split in a amateur/connoisseurs. There's plenty of options, and no reason why you shouldn't be able to find a compromise that'll satisfy everyone.

@Kaspergvalentin : I agree that people should be able to choose "score/no score". I have not posted any review, yet, but in my mind an average "harmless" malt should score 50/100, while a good one should be in the 70's, a few outstanding ones in the 80's, and the rare oddity in the 90's. I just don't understand the point of rating out of 100, when all the "leading" reviewers actually rate out of "20+75" (ie. the minimum seems to be 75 for a bad malt and 95 for an outstanding one).

I think most reviewers don't say that 75 is a bad malt, just a mediocre one. The thing is that (speaking for myself) I normally don't review bad malts because I get my samples from thrustworthy sources, trade them or buy them myself. That's already a big filter. If you would review everything that's in your supermarket, I'm sure we'd need to use the lower part of the scale...

I agree that it would be nice if people could post reviews without a score. For those who do want to score, an easy solution would be to let them enter a score on a 100 scale and still show 10 stars with the detailed score in an alt tag that shows up when you hover your mouse?

@Kaspergvalentin - but I have met people who genuinely dislike the HP 12, how should they score it?

Perhaps @WhiskyNotes is right, leave the star system there, but also include a text box, so the reviewer can either continue to use the star system for simplicity (and doing so will populate the text box with an 85 for 8.5 stars for example), or they can enter a number directly into the text box (which will calculate a value of stars and puts it in automagically. That way if someone wants more granularity they can have it, but if they want to stick to the simpler system they can.

...and then let people review and score whiskies any way they want based upon criteria of their own choosing.

@WN - Yeah, I'd say a 75 malt is drinkable, but not really recommendable. And I don't ever (Very rarely) review poor malts, why would I? If I get a bad malt at a mates house, at a fair, trade show, a night out - whatever it might be, why would I bring it home with me where I could review it? And the fact that I only stock whiskies I like, I'm not going to score them poorly, why would I own them then? Actually I have some really cheep blends here, I'm planning a blind tasting in a few weeks. But that's really the only reason I have for drinking poor drams.

Well, would you set the 10 star score on your own or would there be a conversion factor? I kinda like the idea of simply doing a mouse over and showing the 10 star score, but it doesn't really solve the fact that it's always going to show a score that's "wrong" especially compared to a 100 scale score. I really don't think you should mix the two.

And I'd say if you were going to adopt one scoring system, the 100 scale is the most widely spread and accepted. And that's the one Connosr should adopt as well, some might have to change their scoring system, but I truly find the 100 scale the best scoring system, so it's all for the best.