AR Reviews of Interconnects and Speakers

[mystic]

Members who submitt reviews of audio components on audioreview.com are asked to rate the performance of the product on a scale of 1 to 5. These ratings are then averaged for the particular product(e.g., 4.5 for a 4 and a 5). I recently compared the reviews of main speakers and interconnects on this site expecting to find more variation in ratings for the former than the latter. I thought those who had submitted reviews likely would give a wider range of ratings to speakers because different models do perform differently, whereas different interconnects would not be expected to sound as different as the speakers. I did not find the pattern in the data, however, that I had expected.

In order to avoid the problems that can be encountered when models have a very small number of reviews, I ranked the models by the number of reviews, and to make the research more manageable, analyzed only the top 10 of each category. The range of average ratings for the 10 most reviewed speakers was 2.19 to 4.78 and the range for the 10 most reviewed interconnects was 2.51 to 4.93. The spread in the rankings reflected: low averages for some Bose models ( 4 of the 10 speakers in the group were Bose) and low averages for two models of speakers sometimes bought off vans in parking lots; and a low average for a Monster Cable interconnect.

Rather than deal with issues of Bose bashing and Monster bashing, I decide to exclude these items (and the speakers sometimes sold in parking lots) from the analysis, and expand to the 15 other most reviewed main speakers and interconnects. The average ratings for the 15 speakers ranged from 4.37 for the B&W DM 602(166 reviews) to 4.78 for the Klipsch Reference RF-3(202 reviews). In comparison, the averages for the 15 interconnects ranged from 3.78 for the Kimber Kable PBJ(98 reviews) to 4.93 for the Pure Silver Sound Quartet (41 reviews) and AudioQuest Anaconda(15 reviews). If the averages for each category are averaged, the results are 4.55 for the 15 speakers and 4.53 for the 15 interconnects, hardly any difference at all.

What do Forum members think about the comparison?

[shokhead]

I dont put to much into these reviews. I go and buy a cheap reciever and speakers{i dont know any better} and to me as i dont have friends to go over and visit and listen to there real setup,mine is the bomb. Sounds great. Well what kind of review am i going to write? I'm giving glowing review. I cant belive this Emerson sounds so good on these ford speakers. If you want a kick a$$ setup for a good price. Well along comes another not so up on this stuff and you see what happens. Your going to give your stuff a little higher rating because its your stuff.

[Kaboom]

Bingo! people give good review to what they review because hardly anyone downgrades. You listen to a boombox and u upgrade to a 300$ system and u think it rules. then u go to a propper audiophile shop and u bust 2k on a system. and it kicks ass compared to the 300$ system. Then you go all out on some Nautilus powered by 8 independent monoblocks. that really IS the bomb.
I'm guessing that if you suddenly decided to buy a rotel with a pair of B&Ws 601s you wouldnt be so impressed. You compare against what you are used to hearing.
havent heard of anyone voluntarily downgrading yet.

[RobotCzar]

I find your attempt interesting. You do not go far enough. You definitely need to look at standard deviation and increase the sample size. I agree with omitting the controversal ones, but I do not agree with looking only at the highest rated items (I suggest a random selection).

As speakers almost totally determined what we hear (the recordings does too) and, at the very most, cables should have a very small effect, we might logically expect that speakers have a greater standard deviation. On the other hand, if cable differences are "all in the listener's head" then perhaps a wide SD for cables is a reasonable expectation. While speaker preference might be subjective, cable preference might be (probably is) totally subjective. This situation might lead to a larger variation in the raking of cables.

[E-Stat]

Members who submitt reviews of audio components on audioreview.com are asked to rate the performance of the product on a scale of 1 to 5...What do Forum members think about the comparison?
While I applaud your statistical approach, I think there are two overriding challenges with such reviews:

1. They are from product owners.
2. Their point of reference is unknown.

We really don't know if the reviewer in question compared a number of products before arriving at their choice. Did you find any "well I bought these and think they are horrible" results?

Everything is relative. What I rate as a "4" may be completely different from someone else's judgement based upon exposure and significance. Aside from the best reference, live unamplified music, my point of reference is a staggeringly good reviewer's system I've heard many times that costs more than my house. Consequently, everything that offers less resolution (most certainly including my own system) would be judged in that light.

rw

[mystic]

I find your attempt interesting. You do not go far enough. You definitely need to look at standard deviation and increase the sample size. I agree with omitting the controversal ones, but I do not agree with looking only at the highest rated items (I suggest a random selection).

As speakers almost totally determined what we hear (the recordings does too) and, at the very most, cables should have a very small effect, we might logically expect that speakers have a greater standard deviation. On the other hand, if cable differences are "all in the listener's head" then perhaps a wide SD for cables is a reasonable expectation. While speaker preference might be subjective, cable preference might be (probably is) totally subjective. This situation might lead to a larger variation in the raking of cables.

You may have read the description a little too fast. I didn't look "only at the highest rated items." I first ranked the main speakers and the interconnects by numbers of reviews, omitted the controversial ones, and then compared the ratings of the remaining 15 most reviewed products in each category. I concentrated on the most reviewed items because I am wary of ratings for products with only a few reviews. If a product only has one or two reviews, for example,how do we know the manufacturer or an employee were not the submitters? I should add that the data on interconnects were a bit thin. The 11th through 15th most reviewed interconnects had only 12 to 10 reviews each, whereas the 15th most reviewed speaker had 136 reviews.

Here are the rating averages(means) again with the addition of standard deviations: main speaker 4.58 + or - 0.12 ; interconnects 4.53 + or - 0.30.( I originally misreported the speaker mean as 4.55.) The greater variability in the ratings for the interconnects isn't consistent with my notion that reviewers would notice greater differences in speakers than interconnects. I still believe they would hear a greater difference with speakers, but as you suggested, perhaps the mind produces the variability more than the product does.

About the only conclusion I can draw is that these reviewers generally rated their main speakers and interconnects about the same. I haven't studied reviewers ratings for other components, but I doubt there would be much difference between most categories. An exception might be CD recorders -- at a glance the ratings seemed relatively low.

Thanks for your interest.

[shokhead]

You may have read the description a little too fast. I didn't look "only at the highest rated items." I first ranked the main speakers and the interconnects by numbers of reviews, omitted the controversial ones, and then compared the ratings of the remaining 15 most reviewed products in each cattegory. I concentrated on the most reviewed items because I am wary of ratings for products with only a few reviews. If a product only has one or two reviews, for example,how do we know the manufacturer or an employee were not the submitters? As it was the data on interconnects was a bit thin. The 11th through 15th most reviewed interconnects had only 12 to 10 reviews each, whereas the 15th most reviewed speaker had 136 reviews.

Here are the rating averages(means) again with the addition of standard deviations: main speaker 4.58 + or - 0.12 ; interconnects 4.53 + or - 0.30.( I originally misreported the speaker mean as 4.55.) The greater variability in the ratings for the interconnects isn't consistent with my notion that reviewers would notice greater differences in speakers than interconnects. I still believe they would hear a greater difference with speakers, but these data don't lend support.

About the only conclusion I can draw is that the comparison suggests these reviewers generally rated their main speakers and interconnects about the same. I haven't studied reviewers ratings for other components, but I doubt there would be much difference between most categories. An exception might be CD recorders -- at a glance the ratings seemed relatively low.

Thanks for your interest.

If there is only a couple of reviews,i'm thinking it must be crap because nobody has it. Also,i dont look at the good reviews,i look at the bad ones to see if its the same problem.

[mystic]

If there is only a couple of reviews,i'm thinking it must be crap because nobody has it. Also,i dont look at the good reviews,i look at the bad ones to see if its the same problem.

Yes, I too would avoid products that have a lot of negative reviews.

[mystic]

While I applaud your statistical approach, I think there are two overriding challenges with such reviews:

1. They are from product owners.
2. Their point of reference is unknown.

We really don't know if the reviewer in question compared a number of products before arriving at their choice. Did you find any "well I bought these and think they are horrible" results?

Everything is relative. What I rate as a "4" may be completely different from someone else's judgement based upon exposure and significance. Aside from the best reference, live unamplified music, my point of reference is a staggeringly good reviewer's system I've heard many times that costs more than my house. Consequently, everything that offers less resolution (most certainly including my own system) would be judged in that light.

rw

I agree that "everything is relative." I see the rating as an indicator of the reviewing owner's satisfaction with the product. Comparisons, however, may need to be price specific to be meaningful. A $300 speaker with a 4.9 average rating, for example, should not be expected to perform as well as a $3,000 speaker with a 4.6 average. Each speaker should be compared with speakers in its price range. Even among price comparable items, close differences may not be meaningful. Obviously, the number of reviews also is important. A product with a 4.7 average rating from 100 reviews would impress me more than a perfect 5 rating for a product with 5 reviews.

[tillerman]

I'm in complete agreement on the reviews being made within comparable prices. I recently reported to AR a review that I thought objectionable. The reviewer was not an AR member. The review was of a $500 Jolida amp(I have same model). Instead of just reviewing it, he put it up against two other amps(one an overseas make with double the power at$2000+) and the other A US made with no specs, price, or products databased at Audiogon or anywhere else. The Audio Review inspector
(whatever) felt that there was no reason to remove it.
The other makers are out of business or restructured. Reviews should be made by AR members(accountability) on the product under review: not to rip other makers, give plugs for other products, or show off your own stuff thats so much better. My 10 cents worth!

[E-Stat]

A $300 speaker with a 4.9 average rating, for example, should not be expected to perform as well as a $3,000 speaker with a 4.6 average. Each speaker should be compared with speakers in its price range. Even among price comparable items, close differences may not be meaningful.
Interesting. Maybe it's just me, but I would rate products completely independent of their cost.

I'm quite happy, for example, with my 2.4 Double Advents.

rw

[Geoffcin]

I'm in complete agreement on the reviews being made within comparable prices. I recently reported to AR a review that I thought objectionable. The reviewer was not an AR member. The review was of a $500 Jolida amp(I have same model). Instead of just reviewing it, he put it up against two other amps(one an overseas make with double the power at$2000+) and the other A US made with no specs, price, or products databased at Audiogon or anywhere else. The Audio Review inspector
(whatever) felt that there was no reason to remove it.
The other makers are out of business or restructured. Reviews should be made by AR members(accountability) on the product under review: not to rip other makers, give plugs for other products, or show off your own stuff thats so much better. My 10 cents worth!

Perhaps you should add a review of your Jolida amp to AR. Then, you can reference the fact that the other reviewer is comparing this amp to amps of 4X the price, and double the power.

[tillerman]

Geoffcin & E-Stat,
I thought it was against AR review rules to discuss or disagree with another review, right in the guidelines? I haven't submitted my review yet because
I've only had the 1701 for six weeks, and am experimenting with different tubes and speaker setups. I want to get it right, so far I'm extremely pleased and anticipate giving it a very positive rating.
My feeling is that a review of a product should be just that, not a comparison test. A good review =
"The Craptonite 74EX is an excellent product; solidly built, dead silent operation blah blah blah crisp highs blah blah , I'm giving it a 5 for the following reasons." A Bad Review = "The only good thing about the Craptonite 74EX is its price, it sounds like crap compared to the ***** and*******(no mention of prices{4X}, specifications,number built), always lots of Craptonites for sale on Agon so they must be junk, I rate them a 2." In the Bad Review the reviewer fails to mention that only twenty *******'s were ever made and 500 *****'s were made, compared to tens of thousands of Craptonites. It is therefore likely more EX's are going to be for sale. The bad review also fails to mention the 4ohm impedence of his speakers a power hungry load, and the fact that the other makes are twice as powerful.
There are plenty of product recomendations and comparisons, asked for and given, on the AR Discussion Forum. My apologies to all Craptonite 74EX owners, its
a very fine disfronigator; if my frons get any higher I'll definitely audition one. Tillerman

[A-Audiophile]

Bingo! people give good review to what they review because hardly anyone downgrades. You listen to a boombox and u upgrade to a 300$ system and u think it rules. then u go to a propper audiophile shop and u bust 2k on a system. and it kicks ass compared to the 300$ system. Then you go all out on some Nautilus powered by 8 independent monoblocks. that really IS the bomb.
I'm guessing that if you suddenly decided to buy a rotel with a pair of B&Ws 601s you wouldnt be so impressed. You compare against what you are used to hearing.
havent heard of anyone voluntarily downgrading yet.

I "voluntarily downgraded" my IC's to Radio Shack Fusion cables because they beat the Kimber PBJ and HERO's I had in my system.

Paying more does not automatically mean it sounds better....but then I updgraded again to AQ Diamonds and King Cobra IC's....oh well... :D

[A-Audiophile]

I haven't studied reviewers ratings for other components, but I doubt there would be much difference between most categories. An exception might be CD recorders -- at a glance the ratings seemed relatively low.

Thanks for your interest.

I have looked at the reviews on this site too....IMO many or at least some are baited...when ever I see 75 out of 75 perfect reviews, my eyebrow goes up...company rep's do what ever they can to drum up sales.