menu 2
menu 4
menu 6
menu 8
menu 10
menu 12

Welcome Guest ( Log In | Register )

9 Pages V « < 2 3 4 5 6 > »   
Reply to this topicStart new topic
> Scoring system and personal preference, ...how to do it?
Ari
post Oct 24 2006, 01:38 AM
Post #46


Informed Imbiber
***

Group: Members
Posts: 924
Joined: 22-July 04
From: California, USA
Member No.: 892



I like all the ideas so far.

I think it best to have both this and Drinkboy's system used by many. Since they take two different approaches. A small group of those who know what they are talking about could be singled out as "asspert reviewers" and included along with everyone else's review. The idea that anyone can review lends some weight to the system as sometimes snobby reviewers seem to be speaking out of their flowery ass. So other opinions would serve to support or add other thoughts to those reviews. I also don't think old reviews should be deleted. Everyones tastes change, from the expert to the newbie. I think to see the change might make newbies understand why they don't tasting X in their absinthe or why they like Y when the site says it's crap.

(If all of that wasn't too disjointed).


--------------------
"you mean we're supposed to read 'em? i thought signatures were there to be ignored..."
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
bob_chong
post Oct 24 2006, 01:45 AM
Post #47


Paging Dr. CHONGER!!!
***

Group: Absinthe Mafia
Posts: 600
Joined: 21-July 04
From: The Queen City
Member No.: 891



Good thoughts by all, and I have thoroughly enjoyed this discussion. Kind of smacks of the good old days.

Anyway, I think Bill's review should NOT be deleted. It should absolutely be added to the mix. And hopefully more will be added. This whole discussion is not to beat up Bill, MM, or his review.

Trainer, I haven't seen the WS scoring system you speak of. I poked around there and couldn't find it. Link, anyone?

If we were to develop a rubric, what would it look like? Starting with the first category:

COLOR BEFORE WATER
10-9 pts = Color should be CLEAR, BRIGHT and NATURAL LOOKING. BLANCHE should be completely clear, and a VERTE should be green. ["but other colors such as orange or red are also acceptable"]. VINTAGE ABSINTHES may have faded to an amber brown. Free of sediment. Free of artificial coloring
8-7 = Mostly clear, bright, and natural looking. Mostly green but may show some signs of yellow. A very light haze may be present. No artificial coloring used.
6-5 = Somewhat clear and bright but exhibits some haze or plant material. Should be predominantly green but may have some yellow. No artificial coloring used.
4-3 = Hazy but still natural. May have some sediment. More yellow than green, but some green still present. No artificial coloring.
2-1 = Yellow, or any use of artificial color.
0 = Artificial color which is "off." May have haze or sediment.

--------
OK, I'm not saying this is perfect, but maybe it's a start? Comments? Improvements? For example, where would Hill's fall, with its bluish tinged, industrial solvent look? 0? If nothing else, a rubric could spell out, in points, how much a "penalty" should be. (BTW, why are orange and red perfectly acceptable?) In regards to this category, I tried to throw in the yellow aspect, but I am not happy with how it came out. I mean, should a yellowish green still get a 6?

Would something like this help? Someone care to try another category?

I sympathize with those who care not to write reviews. But as a service, or educational undertaking, the reviews on this site should really be the best out there. This place planted its flag first and has nutured hundreds of enthusiasts. I think the BG can really be something.





--------------------
To Sir Piss, With Love
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
G&C
post Oct 24 2006, 02:39 AM
Post #48


Still Master
*****

Group: Absinthe Mafia
Posts: 6464
Joined: 27-November 03
From: North & West of Valhalla
Member No.: 282



Slightly yellow with some sediment could get a 10.

Peridot, as I've read Absinthe described, is not lime green.

Natural coloring will always leave some sediment.
Perhaps too much should be a penalty, but how much is too much?


I still go with drink it or sink it.


--------------------
"If I knew I was going to live this long I'da taken better care of myself." - Kirk

No threat is Friendly.

Just drink it or sink it.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
traineraz
post Oct 24 2006, 03:18 AM
Post #49


Absinthe Mafia
*****

Group: Absinthe Mafia
Posts: 5749
Joined: 24-September 03
From: Salem, Oregon
Member No.: 38



Sea Monkeys™ would be too much. More than 50 chews to the swallow, or anything stuck in one's teeth, too much.



Chong, I discovered there's a reason a lot of folks can't see DrinkBoy's scoring system suggestions. Will ask Hiram how best to remedy the situation. He may suggest just cutting and pasting, but I'll czeque first.


--------------------
"Now that we've defined what marriage is, we need to take that further and say children deserve to be in that relationship." - Greg Quinlan of Ohio's Pro-Family Network, a conservative Christian group which apparently promotes pedophilia.

. . . and don't forget to read the FAQ and check out the Absinthe Buyer's Guide for brand reviews and distributor links!
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Absinthesizer
post Oct 24 2006, 04:02 AM
Post #50


Absinthusiast
**

Group: Members
Posts: 316
Joined: 21-April 06
From: The trunk of your car
Member No.: 2082



My $.02:

I have lamentably little experience with absinthe but lots of experience filling out judging forms, as a juror for art exhibits, play competitions, and music competitions. My strong preference in a judging form is the opposite of Bob Chong's - have very few specified point categories, and leave as much as possible up to the reviewer's experience and taste.

The main reason for this is that you don't want the judging form making decisions for you. For instance, in the current FV form, aroma is worth 50 percent more than taste - 30 points to 20. This may indeed be some people's experience of absinthe, but should it be enshrined in the judging form? Color before water counts half as much as taste, even though, with most blanches, color before water is a virtual non-part of the absinthe experience.

There are other distortions built into the form. With the exception of Blanchette, it inherently favors blanches over vertes, because blanches tend to be clear and hence score high. The specified 3/1 dilution penalizes extremely rich-tasting absinthes that are intended to be drunk at higher dilutions. Absinthes should be scored at whatever dilution the reviewer thought was best.

There are some excellent aspects of the current form: It helps guide one's absinthe experience, and thereby helps educate people. I doubt I would ever have taken the time to evaluate mouth-feel without it. But its over-parsing results in a misleading impression of precision - the doomed quantitative approach to objectivity that Hegel called the "schlecht-abstrakt," and I promise never again to mention Hegel on this forum. It's entirely possible that a reviewer could end up giving the same score to both a good-smelling, average-tasting blanche and a great-tasting, extremely complex verte, even though he or she actually greatly preferred the verte. Yes, there are 10 points for "overall impression," but that's not nearly enough leeway to correct this sort of error - it's supposed to be an overall impression, and not a place to dole out ones or tens to counteract previous distortions.

If I were allotting points at all, it would be along the lines of 40 points for “Before you drink,” covering color before louche, louche, color after louche, and aroma (but no set number of points for color before louche etc.); 40 points for “While you drink,“ covering mouth feel, taste, and aftertaste; and 20 points for “Overall Impression.” That would allow for more meaningful descriptions - and more representative point totals.


--------------------
Redundancy is alive, and well, and thriving.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
bob_chong
post Oct 24 2006, 04:27 AM
Post #51


Paging Dr. CHONGER!!!
***

Group: Absinthe Mafia
Posts: 600
Joined: 21-July 04
From: The Queen City
Member No.: 891



Good point: as it stands now, a full 60% of the score is before one even takes a sip. Seems high.


--------------------
To Sir Piss, With Love
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
hartsmar
post Oct 24 2006, 05:06 AM
Post #52


Clint Wormwood
*****

Group: Swedish Mafia
Posts: 7621
Joined: 25-November 03
From: Malmo, Sweden
Member No.: 278



Many good points are brought up. I've got to go to work here so not much time now...

I don't think Wild Bill's review should be deleted. I think it's a perfectly fine review.
Artificial coloring does deserve to be scored above 0 even if it's artificial.
Louche does deserve to be scored even if it's enhanced by star anise.

Those are things accounted for in the score sheet. Good points on how to "set rules" for it have been brought up though.

Regarding that fact that the Forum part of the Buyer's Guide and the "official" Buyer's Guide not being quite in tune, this is being worked on and you'll notice why this has been the case - shortly. I'm sure you will love the result.

If I can only find the time tonight, I'll do a review of Mari Mayans myself.


--------------------
Efficient but deadly

Voodoo Magic™ - Boveresse 2011
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Hiram
post Oct 24 2006, 05:17 AM
Post #53


Étouffeur de Perroquets
****

Group: Absinthe Mafia
Posts: 2598
Joined: 12-November 03
From: Seattle, WA
Member No.: 249



QUOTE(bob_chong @ Oct 23 2006, 06:45 PM) *
... I haven't seen the WS scoring system you speak of. I poked around there and couldn't find it. Link, anyone?

First thread and follow-up thread.


--------------------
"Fairy with sea green eyes and plum-colored coat, hair dressed with opals and emerald rings, your jealous sisters from villains your banishment secured, but none shall ever rise to your green throne, its velvet still preserving for our devotion the bilobate impression of your soft behind."
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
hartsmar
post Oct 24 2006, 06:27 AM
Post #54


Clint Wormwood
*****

Group: Swedish Mafia
Posts: 7621
Joined: 25-November 03
From: Malmo, Sweden
Member No.: 278



Well. That is all nice but...

For instance, the original "UCDavis" sets the score for "Color is correct for it's type and age" and "Color is nearly correct" etc. This still means that and artificially colored oil mix would be able to scoe equally high as a naturally colored distilled absinthe.

In the follow-up this is changed to:

QUOTE
E = Artificial
D = Weak or broken color
C = Correct, but thin
B = Blanche
A = Fully developed natural color


Nice intent but not fair and just won't work. As they themself indicated after testing it.

I think if changes are to be made the approach that bob_chong was talking about in the beginning, of more or less changing the phrasing and setting up just a little more rules on HOW to deduct or add points, would be the best way.

I am VERY happy with the scoring system as a whole. The 100 point scale may seem hard and scary to some but the problem isn't gone just by lowering the scale. To me this limits me a lot more.
If I was to give from 0-2 points for color, with 0.5 point increments, there would be A LOT of absinthes that would get the same score but wouldn't be equally good or bad in reality.



--------------------
Efficient but deadly

Voodoo Magic™ - Boveresse 2011
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
hartsmar
post Oct 24 2006, 06:52 AM
Post #55


Clint Wormwood
*****

Group: Swedish Mafia
Posts: 7621
Joined: 25-November 03
From: Malmo, Sweden
Member No.: 278



Come to think of it, the scoring system used in the Absinthiades is a go-between of the two but holds a certain amount of flaws...

The one thing I don't like about it is just that - to narrow scale to score 12 completely different absinthes fairly. The result, a lot of absinthes get the same score for color (which in their case accounts for color both before and after louche) and taste - simply beacuase the scale is too narrow.

With a larger scale, there would be room for adding or deducting that one single point that would actually make the difference.

We should also remember that an absinthe that scores 50 out of 100 on the Fee Verte scoring system is NOT a middle of road, semi-good absinthe. It's below average and maybe not something we would recommend. I'll find the post about that in here somewhere...



--------------------
Efficient but deadly

Voodoo Magic™ - Boveresse 2011
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
brucer
post Oct 24 2006, 08:17 AM
Post #56


Absinthusiast
**

Group: Members
Posts: 326
Joined: 31-December 04
Member No.: 1260



QUOTE
More than 50 chews to the swallow, or anything stuck in one's teeth, too much.


Does that include hairs ? I don't mind a few hairs stuck in my teeth when diving.

Bruce
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Oxygenee
post Oct 24 2006, 09:10 AM
Post #57


Pre-ban minutiae guy
********

Group: Admin
Posts: 40276
Joined: 24-September 03
Member No.: 41



I based the scoring system on the industry standards used in the professional tasting and evaluation of wine and spirits. Two broad systems are in use: one marked out of 20 (commonly used in the UK and Europe) and one marked out of 100 (commonly used in the US). Since most members here are US-based, I chose the latter. The two most important US based wine rating organisations (The Wine Spectator, and The Parker guides) use the 100 point system.

In very general terms, the way to use the system is to start with a theoretical perfect score in each evaluation category, and then deduct points for flaws, or less than ideal characteristics. There is no reason not to give an absinthe a perfect score in a particular category, and in some cases (eg the colour rating for a blanche) one might expect this to be the case 95% of the time.

Broadly speaking you'd expect a really excellent absinthe to score 90+, a very good one 80+, a fair one 70+, a drinkable but imperfect one 60+, a very poor one 50+ and something almost undrinkable 40+. Scores of below 40 are almost unheard of in professional tasting, even for dramatically faulty products

Within the framework of the tasting sheet, the scores are indeed entirely subjective, as individual opinions must inevitably be. The "objectivity" of the guide arises from two things:
1. The reviews here are disinterested. We don't sell absinthe, we don't reward members for posting specifically positive reviews, we don't edit reviews (except on narrow technical grounds). We let the chips fall where they may.
2. Over time, as we accumulate more and more reviews, the collective opinion expressed becomes more and more representive of the "objective" ideal. The same principle applies to other sites that solicit consumer reviews: if you look at say a camera for sale at Amazon, and it has one 4-star review, you'll likely (and correctly) take the rating with a pinch of salt. But if it has 60 reviews, averaging 4-stars. you'll correctly assume that that is a fair reflection of the quality of the product.

I'm open to the idea of tweaking the existing system, but I'm not convinced any of the suggestions made so far would be an improvement. I've extensive experience in wine and sprits judging at regional, national and international level, and am confident that the system is broadly speaking, as fair as it can be.

I do feel that it's a pity that more of the of intellectual energy devoted to criticizing the scoring system, isn't devoted to actually contributing reviews, because over time, what matters most if the number of reviews per product, not the specific characteristics of one single review. It's the wisdom of crowds thing. I also worry that criticising individual reviews, and demanding the reviewer justifies his decisions, can act as a disincentive for less confident and less experienced members to post reviews in the first place. The most powerful critique of a review you disagree with, is surely to publish your own review of the product.

We'll very shortly be introducing a new version of the Buyers Guide, which will vastly improve the ease of navigation and searching, as well as making it far quicker to upload new reviews. Kallisti and Hartsmar have done a tremendous amount of work on this, and it's now close to completion.

We're also looking into a way of publicly recognising and rewarding those who contribute significant numbers of reviews. This will apply retrospectively as well, so get reviewing!


--------------------
...et c’est l’absinthe enfin, la grande absinthe ou la petite, parure chaste des montagnes et des rivages marins, fille des grand vents purs, blé des espaces vierges, emblème de la liberté farouche.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Oxygenee
post Oct 24 2006, 10:47 AM
Post #58


Pre-ban minutiae guy
********

Group: Admin
Posts: 40276
Joined: 24-September 03
Member No.: 41



QUOTE(Wild Bill Turkey @ Oct 24 2006, 12:03 AM) *

I didn't review this absinthe with a final score in mind. I meant to give it low-to-middle numbers in the categories where it deserved them, and use the language of the review to convey my basic approval. The final score was higher than I expected. It's too late for me to change it, but if a lot of people feel it's an unbalanced review that skews the number in the index, I'd have no objection to having it removed.


There's absolutely nothing wrong with this review, and I don't think 63 is an untenable score for MM either. As you quite correctly say, it's an absinthe that often impresses newcomers, even in the face of competition from brands which the conventional wisdom rates higher.

A review is meant to be one reviewers personal, subjective opinion of a particular glass of absinthe, poured at a particular time and place. Such objectivity as the Buyers Guide has (or will have), comes from the accumulation of many reviews, from many reviewers, over many months or years. Each review is ultimately a piece of the bigger picture.

Striving for "objectivity" in an individual review is a fools errand. The only objectivity required is to try and rate each absinthe as far as possible on its individual merits, without regard for its method of production, price or reputation here on the Forum or elsewhere.

I can't stress this enough: EVERYONE IS ENCOURAGED TO SUBMIT REVIEWS. You don't need to be an "experienced" taster. If your review is rational, and follows the form specified in the Buyers Guide, we will include it.

If you'd like to have a review included anonymously, just email or PM it to Hartsmar or myself, and we'll post it on your behalf.

If you have a review under your name in the Buyers Guide which, for whatever reason, you'd now like to disown, we won't remove it for you, but we will happily remove your name from it.




--------------------
...et c’est l’absinthe enfin, la grande absinthe ou la petite, parure chaste des montagnes et des rivages marins, fille des grand vents purs, blé des espaces vierges, emblème de la liberté farouche.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Oxygenee
post Oct 24 2006, 10:53 AM
Post #59


Pre-ban minutiae guy
********

Group: Admin
Posts: 40276
Joined: 24-September 03
Member No.: 41



QUOTE(bob_chong @ Oct 24 2006, 08:27 AM) *

Good point: as it stands now, a full 60% of the score is before one even takes a sip. Seems high.


Not at all, this is absolutely in line with professional norms. In wine tasting, 50% of the score is usually devoted to non-taste factors - colour and aroma - and 50% to taste. With absinthe, one has additionally to evaluate the louche, so a 60-40 split seems about right.


--------------------
...et c’est l’absinthe enfin, la grande absinthe ou la petite, parure chaste des montagnes et des rivages marins, fille des grand vents purs, blé des espaces vierges, emblème de la liberté farouche.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post
Oxygenee
post Oct 24 2006, 11:35 AM
Post #60


Pre-ban minutiae guy
********

Group: Admin
Posts: 40276
Joined: 24-September 03
Member No.: 41



QUOTE(Absinthesizer @ Oct 24 2006, 08:02 AM) *

There are other distortions built into the form. With the exception of Blanchette, it inherently favors blanches over vertes, because blanches tend to be clear and hence score high.


This point has been made repeatedly before, and is absolutely, completely, utterly false.

1. The difference in points likely to be awarded to roughly equivalent quality blanches and vertes is very small. Yes, it's absolutely true that most blanches (the Combier being a notable exception) would automatically rate 9 or 10 out of a possible ten points. Almost all blanches are naturally distilled products. Following the scoresheet guidelines, most naturally distilled vertes are likely to rate between 6 and 10 as far as colour is concerned. So one is talking about an "advantage" of at most 4 points out of 100, and more likely 2 or 3.

2. Those vertes that score lower than 6 in the colour category, especially those that score ultra low markes like 2 or 3, are overwhelmingly likely to be poor quality, artificially coloured products that deserve to be marked down.

3. The best vertes are - on average - more likely to be rated highly for taste, complexity and overall impression than blanches are. Over a broad range, this more than wipes out any slight advantage blanches have as far as colour scoring is concerned. There is an exact analogy here with wine tasting: certain varieties - for example, young sauvignon blancs - are so pale and clear that they almost automatically score full marks for colour. Other varieties - pinot noir is an example - often full short of the ideal colour, even if they are superb in every other respect. But pinot noir's are not consistently outperformed by sauvignon blancs, because they more than make up any shortfall in the other tasting categories.

4. All tasting scoresheets, by their nature, must generalize. Otherwise there's no basis for comparing different products to each other. I'm not aware of one single professional wine tasting panel anywhere that uses different scoresheets for white and red wines, even though it's generally far easier for a white wine to achieve a perfect colour score than it is for a red wine. I'm not aware of a single shred of evidence anywhere that indicates that white wines therefore do disproportionately well at tasting competions - on the contrary, anecdotally, the overall medal winners are more likely to be red wines. There are no grounds for believing that the situation with blanche and verte absinthes should be any different.


--------------------
...et c’est l’absinthe enfin, la grande absinthe ou la petite, parure chaste des montagnes et des rivages marins, fille des grand vents purs, blé des espaces vierges, emblème de la liberté farouche.
User is offlineProfile CardPM
Go to the top of the page
+Quote Post

9 Pages V « < 2 3 4 5 6 > » 
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



Lo-Fi Version Time is now: 16th July 2018 - 04:44 PM