Scoring woes?

Interesting topic. I'm pretty sure my baseline moves over time, when I look at the different whiskies that I've given the same score.

I'm sure this is true with Ralfy as well. His first video review was of Glen Breton Rare (I call it "not rare enough") which he scored 87. Yet he has scored much more solid whiskies the same or lower over the years.

I think this fluidity makes it difficult for other people to use scores as a guide, as @Victor has much more eloquently expressed in the past.

I used to think that reviews were primarily for providing information for others. I still think there is that role, but more and more I think we write reviews for other reasons. It could be narcissism, financial, or personal.

I find my reviews have become more detailed (read: longer) over time, but my tasting notes may not be as specific as they used to be. I'm ok with that because we rarely taste things the same way as others. I personally find that when I take the time to review a whisky, I get to know it better than if I'm simply having a dram of it. Once reviewed, I see it differently when I come back to it.

My desire to drink whisky has dropped in general since the pandemic started, and my motivation to review has as well. I wonder if that will have an effect in shifting the baseline again.

As @Nozinan says, a good topic and one I've also been thinking about for a while now. I think ralfy, as it happens, has talked about this quite a lot in relation to the law of diminishing returns. I think he's spot on and this hit me quite starkly recently when I reviewed a bottle of Benromach 10.

OK, it may not have been quite as good as previous ones (it was still very decent) but my overall enjoyment of that profile had shifted somewhat and wasn't giving me the 'orgasm' factor other bottles had. In truth, very little whisky wise gives me the 'buzz' that, say, my first Ardbeg 10 did and I think my marks have reflected this over the last couple of years. I tended to mark primarily on terms of enjoyment, with more technical aspects being a secondary concern, but now I think my focus has shifted a little more to the latter.

I guess this is eventually the same with any hobby ( it's why Rod Stewart keeps buying bungalows to fuel his crack like addiction to building model railways, allegedly) but it does leave me a little sad to know that those initial highs probably won't ever be repeated, and that I'm more likely to see flaws and awkward corners than I would have done, say, five years ago.

All of this reminds me that it should be about enjoyment, fun and not for taking too seriously! And I again agree with @Nozinan in that reviews are all a tad narcissistic but, hey, tell me what on line activity isnt?! Marks are always taken with a pinch of salt anyway and as long as there is consistency they are still useful guides - especially from folk on here whose tastes and ratings I have some knowledge (trust) of.

@RianC What I find works well to re-ignite that "high" is taking a long break from whisky altogether. Or leaving a particular category for awhile. I haven't had any Laphroaig in awhile and I'm sure that when I go back, my bottle of Laphroaig 10 will taste like heaven (my current open bottle seems to have come from a very good batch).

@OdysseusUnbound I think you might be on to something. I had none for almost 2 weeks this month and when I did it was the wee beastie and I enjoyed the experience (and the whisky) more than I had in quite some time.

Ralfy says "a change is as good as a rest", but when alcohol is involved, it may be more prudent to rest...

Interesting topic. Looking at my scores, my baseline looks like an 83. I've scored 9 whiskies an 83 which is the most by a fairly wide margin. That seems a bit high to me relative to other people's scores, so I would say if I gave something an 83 other people might give it an 80. Just a guess though.

@Nozinan Do you have any thoughts as to the reason why your desire for whiskey has dropped?

@OdysseusUnbound expansion of experience in tasting a large number and a wide variety of whiskies gives a perspective which does not exist before one has obtained that experience. Because I had tasted a lot of whiskies before I ever sat down to do a Connosr review I had that perspective from the earliest days of my review-writing. I think that my scoring has remained almost exactly the same as it was in the beginning, 9.5 years ago.

With respect to numbers I score according to the inner dialogue "How close does this (parameter) come to 'as good as it can possibly be' ?" If I see no way to increase the quality, that would rate 24/25 points with respect to a given parameter. The 25th point would be possible only with a "wow" characteristic of extreme potency, clarity, "brightness" etc. not usually found. 23/25 is excellent. 22/25 very good. 20/25 to 21/25 good. 17/25 to 19/25 fair. Lower than 17 points poor to very poor.

As for reviewers, Ralfy, Serge, etc draw no interest from me whatsoever. The reviewers who significantly matter to me are those whose tastes have extensively tested out to be closest to my own, thereby making the probability of my responding to a given whisky very similar to the way that they have responded. This is practical and useful. This eliminates a boatload of guesswork trying to figure out how the reviewer's taste corresponds with my own.

So for me, while I have many whisky friends, the Connosr ones whose taste makes my life easiest are: 1) @Nock, 2) @MaltActivist, and 3) @Benancio. Having read many of their reviews and comments, and having in-person tasted with # 1 and # 3, I know their tastes well, and know that there is a very high correlation between what they like and what I will like.

@Victor Thank you for the detailed response. I offered Ralfy and Serge as examples simply because they're well-known. Serge has become a bit more useful to me in the past few months, not because of his overall scores or even his notes, but because of his SGP (sweet, grain, peat) "scores". I also appreciate that he seems less "emotional" in his assessments than many others. His seems to be an approach of "the whisky, the whole whisky, and nothing but the whisky".

As for Ralfy, I just have a soft spot for the guy. He's like a well-informed, well-intentioned but slightly curmudgeonly uncle. I can't totally rely on his scores either since he seems to automatically award points for natural colour and NCF, even though these aren't guarantees of high quality flavour (though there does seem to be a correlation). He also seems to "punish" or take away marks for anything owned by Diageo (though again, this is sometimes justified).

My tasting "experience" wasn't that wide before I came to Connosr so I think my "average whisky score" has probably gone down since then..

How has your scoring evolved in the last 5, 10, 15 years?

I'm don't rely on scores when buying a whisky. I do assess multiple reviews and look for similarities and divergent view points in a reviewers' description of nose and palate. I'm not a fan of scoring whiskies even though it seems to the most "objective" way of grading a whisky. I am taking a liking to @JasonHambrey review scoring method. He maybe onto a more vague but, more reflective of how people view a whisky.

A good whisky should move the taster in an emotional way. I've witnessed how in a group tasting, a good whisky can bring a collective smile of satisfaction or evoke that stupefying silence among tasters. A divisive whisky tends to bring audible critical comments of "unbalanced", "unpleasant in a bad way, and "is this a whisk(e)y?. I've become more aware over time of a "clean spirit", notions of "complexity", a "wet cask", "about a whiskies form" and "cuts". There are those who are "peat heads", "sherry fans", and "bourbon aficionados". Personally, I enjoy the variety of flavors whiskies offered from different regions and distilleries, and countries. It is also good to have a few not so good whiskies on hand. They provide good reference points.

Subject to subjective taster E&O

Do you give more weight to your favorite reviewer's (Ralfy, Serge, etc.) more recent scores than you give to their older scores?

If this is a reference to Ralfy's "re-reviews" or to Serge's "batch reviews"(ie. Lagavulin 12 CS)? Then no, core ranges will vary, and profiles change to suit "modern" tastes. Their older scores are historical markers.

Biases:

Ralfy seems to leans towards those "balanced and harmonious flavor profiles". Serge hates wine finished whiskies and is a fan of "spirity" whiskies.

I do find it interest when a whisky recreates a split between high and low scores amongst reviewers. That whisky reveals more about a reviewer's flavor blind spot, and flavor preferences.

For me, the only point of scores is to measure an "enjoyment factor."

In my early days, I liked pretty much everything well enough, and I didn't yet know what my favorite styles were. Possibly to my credit, I bet if I looked through my earliest whisky journal, I'd find a bunch of 80s and high 70s.

These days, I know my tastes much better, I buy better, and I'm more critical of boring whisky. I'd wager the scores in my more recent journal pages skew both higher (because I'm confident pinning a 90+ on something like a zippy Springbank or a ballsy Ardbeg) and lower (because if I taste something "meh" like Scapa NAS Whatever, I'm confident throwing a 62 its way).

@JayRain I have a few theories about it.

I think the Pandemic is probably directly or indirectly responsible in a few ways.

Having a regular group of friends to come together a few times a year helps to keep up the enthusiasm. While zoom tastings have been fun, it isn't quite the same.

While I have few non-Connosr drinking companions, the pandemic has meant that they are not around either.

I think in general my stress level has been up. Not much time off, new challenges, and, of course, risks at work. Now with school back I worry about the kids. My productivity has gone down, so I'm not done my work stuff early enough to start a dram.

I'm not the type to have a drink to relax. I prefer more to be relaxed to enjoy whichever whisky calls to me that day. So I just haven't been in the mood as much.

It's funny, because studies have found people, especially those with kids, are drinking more during the pandemic, and I've noticed about a 40% drop in volume.

On the bright side, my kids will be able to make a fortune on all the bottles I leave behind...

@Nozinan Totally understandable and am many (whether Connosr's or not) feel the same way.

How has the change in consumption impacted the ratio goals that you set for yourself at the beginning of the year (have purchases also gone down subsequently)?

@JayRain

Good question. I think the decrease in consumption may have delayed me finishing a few bottles (though for bottles I like it’s hard to pour the last dram sometimes).

Other than that I don’t think it has curbed my enthusiasm to purchase. What has decreased my purchasing is lack of space, and a real desire not to expand my collection further (unless absolutely necessary).

It’s still early, because I find the bulk of my purchases occur in the last quarter of the year, but I have no plans to travel to Calgary this winter and there is currently nothing on my radar.

This year I have purchased only 8.5 (one is shared) bottles so far. But I have sold or gifted 4 bottles and opened a couple so my net increase to my collection is 3.5. Less than 2%. This year will likely see the lowest number of purchases since I started keeping track (in 2016), and the smallest growth in my collection since I kept track (2015).

This is a good time for people to take advantage of my generous nature and get me to part with some bottles....

@Nozinan Most importantly, may the good times w friends and your passion for whiskey rebound in 2021 (without the increased purchases)

Good post @OdysseusUnbound . Something I've wondered about at times.

I've found that using the 100 points system my scores tend to get clustered together. I actually prefer the system MALT use.

malt-review.com/about/scoring-bands/

They score out of 10 and use the whole range. I've seen them score a bad whisky 2 or even 1. 3-4 are ok whiskies lacking a little panache. With 5-6 being a solid dram that is worth a look. 7+ you should definitely consider buying. It seems to give a better idea of how good a whisky is.

@Wierdo I also like Malt's approach. I've considered borrowing that approach for my blog, but then I'd have to translate it when I post the reviews here. It would be SUCH a chore...

Perhaps, @OdysseusUnbound, you could score traditionally and also apply a MALT score. I think the former is more technical and the latter more based on enjoyment factor. It's like some of my reviews where I modified the score "based on enjoyment".

Chuck cowdery has a great chapter on rating whisky (or not) in his book Bourbon, Straight.

My opinion on this continues to change. Originally (~2012) i started using a Jim Murray type scoring system evenly divided between nose/palate/finish/balance but then i shifted from “balance” to “integration” and then “intrigue” because i found while balance is important, it doesn’t address things like the inappropriateness of breadth or depth, or uniqueness - at least the way i defined it. Then, after more reflection i realized i don’t value finish as much as the nose or palate, indeed i don’t even really care about it for mediocre whiskies (unless it is bad) other than getting a bit of tannic grip. For me, it detracts significantly from bad whiskies and adds significantly to great whiskies but there is a fairly large region where it doesn’t really alter my influence my assessment of a whisky.

It’s very challenging trying to objectively quantify a very subjective experience. I certainly have realized we don’t all value or taste the same way. I find tasting notes these days to be effectively useless unless i am able to get to know the reviewer, similar to what @Victor referenced. It really helps you put in context what the whisky is like. It’s also made my written tasting notes shorter and less detailed, even though my tasting process is more developed and involved.

I’ve gone through multiple more changes- changing weighting to better reflect my references, taking away my division of scores since i found both that i gamed the balance/integration mark to get an overall score that made sense. Then I also realized that dividing up the scores as i did didn’t actually reflect how i drank whisky when i wasn’t reviewing it. So i went back to an overall score.

Then, i became disillusioned with scoring since it is pushed so hard by marketers and probably is the reason it is so prevalent. It’s certainly the reason why you see more 90s than anything else in many rating organizations. I also became disillusioned with the 100 point scale since people rarely rate below 70, and in many cases below 80. I know many people in whisky journalism who have told me they don’t have real control over their scores, too.

I was also diligent to compare all my scores against other whiskies i had rated that way, to keep consistent - but, in doing so, i settled in a narrower band of scores than i would currently choose.

So i moved to a recommendation basis. Recommended/highly recommended/very highly recommended/exceptional with the “recommended” or higher mark representing around 80 per cent my reviews (i am fairly generous) and exceptional representing 2-3 per cent of my evaluations. This also gives me flexibility to recommend whiskies because they are special or unique rather than simply because they are critically excellent.

But, it’s still not perfect. If folks are rating, i now wish everyone would work on a percentile basis so you actually understand how “relatively” good the whisky is and prevents against score inflation. Or provide a score out of “100” and a percentile mark also. But the system is nearly unintelligible because of the masses of 90s in the market. In that system, a 51 would be quite a good whisky. You’d have to explain it every time. Of course, you also need to be roughly familiar with the range of whiskies a person would taste then.

I still doubt whether rating whiskies has increased my appreciation of whisky. On my own, maybe, but the prevalence of ratings everywhere - maybe not. Can you imagine what a weird world it would be if we rated all our meals out of 100? I don’t think that would increase my enjoyment of food.

But, I do think if you know a reviewer outside of the review content that enables you to actually interpret ratings.

@JasonHambrey A lot of wisdom in what you've written.

I would add that our perception of a whisky can be different each time we taste it. I certainly found that in my most recent review.

Without reiterating everything you've written, I would say that you have to get to know your favourite reviewers to interpret what you think you will like. In addition, getting to know your friends' tastes is helpful in predicting whether you will like a whisky that they like, and they can predict what you will like as well. This has been useful recently and has saved me a lot of money and grief.

Finally, I firmly believe that the person who most benefits from the review is the person who writes it.

I'm not very generous when scoring whether it be whisky or beer. Specifically the very high marks. To score a whisky in the 95+ range it has to be almost life-changing. Same with beer. On the other hand this lends itself to a more "academic" experience, for good or worse. One of the best whiskies I've ever had, subjectively, is bowmore 12. What made it taste so extraordinary was the company, the setting and the mood. Id probably score it in the mid 80s though if I'd write it up. I guess the scoring is based heavily on your approach to whisky in general. Are you a whisky academic ( @Nozinan :) ) or are you after the pure enjoyment.

@JasonHambrey that's a great post and sums up the pitfalls of rating a subjective experience.

@Alexsweden do I have to choose one or the other?

I am to some extent an whisky academic, as I am with many of my hobbies. I like to learn things and delve deeply into things that interest me. I enjoy picking apart whiskies, even though I’m not necessarily that great at it. Literally, those who can, do, and those who can’t, teach.

But I also like to sit and enjoy a dram without dissecting it. Preferably with company. It’s possible my academic preference for have a welcoming palate all the time decreases the frequency of my framing.

I must admit, I have given up on scoring. I like or dislike it, and I have my reasons. When I watch reviewers on Youtube end their review with a score, I think to myself, "How does this score relate to what I have just listened to?" Some reviewers admit their range doesn't really go below 80, and I think "Why have 100pts then, why not 0-20?" So, in the end the reviewer whom I enjoy the most, swooshing aside, is Horst. No scoring at all. HA. Swoosh swoosh. Maybe that is his scoring. The more loud his swooshing is, the more he likes it. : P

@ajjarrett His son swooshes twice as long. The stuff must be so diluted with saliva by the time he swallows it that the finish is essentially dead. And really, do you taste with your gums?

@BlueNote

I am not sure, but I did enjoy Juicy Fruit and Double Mint when I was a kid. ^_^ Oh wait, that's enjoying the taste of gum.

@ajjarrett I like Horst's whisper reviews as well, though I have to say his tastes and mine don't line up that often. He doesn't seem to like high proof offerings or heavily peated whiskies. His quiet, very German enthusiasm is fun to watch though.

@OdysseusUnbound

I agree. I actually enjoy his reviews. Granted they are in lines of helping sell the stuff at their store; however, it is better than most reviewers who think they are actually making content and simply not being a free ad for the distilleries. HA.

My favorite Horst moments are usually found in his facial expressions and sounds (not the swooshing) when he doesn't like something. There are times he doesn't hold back, or maybe that is how I am interpreting them.

In short, these people are fun for entertainment. I have stopped thinking online reviews video or print is something to really take much content from.

"Just an opinion." HA!

@OdysseusUnbound The Horst Whisperer? I think I saw the movie.

@BlueNote HAHA! "Horst whisperer", that's an instant classic!! I had forgot about the rocket-man, off to youtube!