Quote Originally Posted by RGA
I have started a discussion with my fellow reviewers at dagogo about a similar problem that pervades the review industry.

To me the stereophile class rating system is highly problematic but most such systems are compromised - probably why Constantine has so far avoided them.

No one single person can review every product so how do we come up with a best of listing. A reviewer can vote on it and submit it to the editor but in the end it is the editor's choice.

Take Art Dudley of Stereophile - he reviewed the Audio Note AN E/SPe HE - and he bought them as his reference speakers. Stereophile rates it as a class B loudspeaker even though Dudley says they should be an A. (I believe his words were should be an A+++)

So who made the decision? Did other reviewers agree? Well Wes Philips recently said (Jan 2009) that a room with an AN E loudspeaker was not only the best sound at CES but the best sound he has EVER auditioned. That's a second Stereophile writer - and these two guys are some of the top reviewers at the mag. A third writer there owns an AN E (Peter Van Wellinswaard who is a technical adviser) and John Marks called the AN E one of the most memorable experiences of his life.
I guess the AN E must have been reviewed before Stephen Mejias started compiling the recommended components list... Since his policy that is stated in all the current rec listings is that Class A is based on one reviewer regarding the product as Class A... So the AN E should be Class A if Art Dudley thinks so...

Quote Originally Posted by RGA
What's more important - a reviewer who puts his own money out and buys them to use in his own house or something that is given a class A rating that not a single reviewer would touch with a ten foot pole? Audio Note does not advertise in Stereophile and John Atkinson and Peter Qvortrup don't see eye to eye on much. So I suppose it is amazing it got a class B - though PQ showed up at CES this year and you will notice they didn't cover the room - coincidence?

I think Arthur has a point about the rating systems with his analogy to the olympics. Even if stuff has improved there should be a league table where there is not more class A gear than class C - it makes no sense. Unless of course they are saying that it all sounds the same - in which case there are serious problems.

Notice that recently Stereophile gave very negative reviews to a Bryston and a Totem - the next issue did not have the usual advertising from either company.

To me there are so many problems with recommended component listings because they're highly arbitrary. Not everyone is on the same page at the same magazines so how on earth do they come up with such lists?

My list would be very different from Doug Schroeder's or Jack Robert's, or Fred Crowder's for example. I suppose the ones where we all agree that ABC speaker is very good but even then there are degrees of preferences.

The same problem pervades magazines that give out yearly awards - so if XYZ speaker wins best loudspeaker in 2010 how often do you see it win again - or is the award given every year just to have something new in the limelight. If I were giving out reviews I may have the same product win year in and year out until such time as something proves to me that it is better. That may happen but I never see a product win over and over and over. Stereophile though dumps them from the list after a few years or if they're no longer sold - that is a problem too. The assumption is if it is new it is better - it ain't so.

And of course as massive a problem as all that - what about the stuff that Stereophile misses? For yeard they were recommending amps in the Class B such as my old Arcam Delta 290p but no one there reviewed the classic Sugden A21a which is a much better amplifier. Finally, after it's 30th+ year or so selling them they bother to get around to it and Marks felt it was the best sounding integrated for the money. So while they're out recommending good but not star budget integrateds to the thousands of people buying the magazine - the entire time they WOULD HAVE recommended the Sugden instead.

A smaller magazine has an excuse to miss a staple product - but the longest selling integrated amplifier on the planet is missed by Stereophile?

None of this would be problem if most of the readers understood these things and didn't put much stock in such ratings but I get the feeling that a great many people do put a LOT of stock in them. They believe if they shop off the class A list they will have a great sounding stereo. Some of the stuff that I have heard on the class A list is truly shockingly bad IMO. I think to myself - something is really odd - nobody working there bought them.

In the end it comes down to either entertainment or finding a reviewer you agree with - has a similar ear and values certain things you value. that is why so much stuff gets raves - they will simply give the panel to the panel guy, the SS to the SS guy the tube to the tube guy. Everything gets a rave. What you want is if you are the tube guy is for the reviewer to review two tube amps at around the same price and tell you the pro's and cons of each. But manufacturers often don't like that.
A major part of the problem with recommended components lists (or any kind of product of the year awards or even reviews in general) is that consumers don't understand how to use them...

People just assume that if you combine 3 products of the year or 3 Class A components, you will get a great sounding stereo... Which can be a very costly mistake...

Despite the fact that many mags openly state how to use (and not use) their recommended products listings, so many persons either don't read or just ignore those explanations... It's a tough call for the mags....