Blind listening comparisons

[tube fan]

Yes, I am a huge advocate of blind tastings in wine, and an equally big advocate of blind listening of "high end" stereo systems. I still remember hearing a blind comparison of speaker systems (about 50 years ago). When they got to the AR3a, I cried: "stop"! I bought my Fulton Js after comparing them to the latest Infinity speakers blind. Still have them 35 years later!

Trust me, there is very little actual correlation between price and perceived quality, either in wines or in "high end" equipment.

[YBArcam]

I'm not sure about that last line. Perhaps. But I guess my point is, compare a speaker that is $300, then $800, then $1,500, then $2,500, etc. It might be a Paradigm Atom, then a B&W 685, then a Tannoy DC6, then finally a PMC TB2i. I'd say there's improvement as you go up.

Or amps. NAD 326, Exposure 2010s2, Naim Nait XS. And so on. IMO, more money definitely buys better gear. But that's not to say that a cheaper component can't sound better than a more expensive one. I wouldn't be surprised if the 2010s2 could smoke a few of the more expensive amps that are out there. To me that price (about $1,300) buys a lot of performance.

[JohnMichael]

I am totaly against blind testing when it comes to audio. I have to listen to the equipment for a period of time before I decide if it is right for me. In quick blind comparisons I can hear differences but not what is best over the long haul. I think blind listening tests are bull****.

[YBArcam]

I am totaly against blind testing when it comes to audio. I have to listen to the equipment for a period of time before I decide if it is right for me. In quick blind comparisons I can hear differences but not what is best over the long haul. I think blind listening tests are bull****.

I agree. I think it's tougher to pick up on meaningful differences if you switch quickly between components. I prefer to listen to a system for a couple of months (at least) and get used to the sound. Listen to at least a few songs many times over. You become used to the presentation, and eventually you know what a song is going to sound like at every point within it. You'll know which parts of the song connect most with you. Then when you finally change up the gear, all differences, be they major or minor, becomes much more noticeable.

It's kind of like test driving new cars. Unless one is a perfect fit, they are likely to all be good. Picking out meaningful differences when you drive one right after the other is tough. You may think you like this, and don't like that. But drive one for a few weeks and get used to it, and then make a change, and the differences will be much more clear.

[Ajani]

I am totaly against blind testing when it comes to audio. I have to listen to the equipment for a period of time before I decide if it is right for me. In quick blind comparisons I can hear differences but not what is best over the long haul. I think blind listening tests are bull****.

Blind tests are fun, but I don't think they're that useful longterm... I remember a criticism of the Pepsi Challenge (DBT); which is that in the small samples given, persons preferred the sweeter taste of Pepsi in hoards... But when it came to drinking an entire glass, persons found the Pepsi too sweet and preferred Coke... Think of how many sweet or salty foods are great as a small snack, but would make you feel sick if you ate a plateful of them... I feel that sums up a lot of problems with DBT... You need to be careful not to draw the wrong conclusions from a quick switch back and forth... I've found that to be very true of components I've auditioned at home and at dealers... Initial impressions can be very positive, but after a month of continuous use, your opinion can go in the complete opposite direction...

Also, even though there is no time limit on a DBT... Who is really able to get someone to help you setup one to last a month or so?

[Sir Terrence the Terrible]

Blind testing is completely relevant to audio. But I would not use it to choose a component for purchase, as that is the wrong way to use DBT.

DBT is used to define differences without introducing prejudices that sighted testing can do. Like testing the differences between Dolby Digital and DTS, and interconnects against interconnects(or speaker wire). It can be used to test the audible differences between tube and SS amps, or a metal dome against a soft dome within the same speaker.

Listening to equipment for purchase is quite a different thing than listening for differences. One has to reach you emotionally, the other more critically without emotion.

[woofersus]

Blind testing is completely relevant to audio. But I would not use it to choose a component for purchase, as that is the wrong way to use DBT.

DBT is used to define differences without introducing prejudices that sighted testing can do. Like testing the differences between Dolby Digital and DTS, and interconnects against interconnects(or speaker wire). It can be used to test the audible differences between tube and SS amps, or a metal dome against a soft dome within the same speaker.

Listening to equipment for purchase is quite a different thing than listening for differences. One has to reach you emotionally, the other more critically without emotion.

Agree completely. DBT is a great way to see if a difference can be heard and/or pointed out consistently. However, among gear in which there is an audible difference (sometimes it's obvious without DBT too) a blind test, or "sample" isn't going to help you decide which you prefer. It's not always a case of which is objectively better.

I will also say that both the DBT-is-king crowd and the always-trust-your-ears crowd can be a bit over the top at times. Some people are so dogmatic about DBT that they can never admit anybody hears a difference in anything unless they've followed a 50 step process and the stars align properly and it's supervised by one of the DBT high council members to make sure nobody could possibly have cared beforehand what they would hear. Others are so stunningly un-objective that they never allow for the validity of anything to ever be questioned. There a few forums out there that are so rife with fights over this topic that they become irritating to visit.

[Ajani]

Agree completely. DBT is a great way to see if a difference can be heard and/or pointed out consistently. However, among gear in which there is an audible difference (sometimes it's obvious without DBT too) a blind test, or "sample" isn't going to help you decide which you prefer. It's not always a case of which is objectively better.

I will also say that both the DBT-is-king crowd and the always-trust-your-ears crowd can be a bit over the top at times. Some people are so dogmatic about DBT that they can never admit anybody hears a difference in anything unless they've followed a 50 step process and the stars align properly and it's supervised by one of the DBT high council members to make sure nobody could possibly have cared beforehand what they would hear. Others are so stunningly un-objective that they never allow for the validity of anything to ever be questioned. There a few forums out there that are so rife with fights over this topic that they become irritating to visit.

There are forums that straight out ban discussion of DBT as it is generally not a productive discussion... It too often descends into all out war between DBT diehards and the ears only crowd...

IMO, there is some truth to both the DBT and the ears only crowd's positions, so there is no need to be black and white about the issue...

I prefer to base my purchase decisions on a combination of measurements and my own long term listening (rather than DBT).... While I understand that something can measure poorly and yet sound good and that something can measure well yet sound really bad, I like products that both measure well and sound good... I don't believe the 2 are mutually exclusive...

[GregLee]

But I would not use it to choose a component for purchase, as that is the wrong way to use DBT.

DBT is used to define differences without introducing prejudices that sighted testing can do.

So you want to base your purchase decisions on prejudices, rather than on the sound differences revealed through DBT. Well, hmmm, that's just you.

[JohnMichael]

So you want to base your purchase decisions on prejudices, rather than on the sound differences revealed through DBT. Well, hmmm, that's just you.

Actually I would say it is many of us. Yes you might hear a difference but will that difference please long term. How many times has a component that stood out in blind testing become annoying when used long term. Oh this IC had such extended highs compared to the other. Yet in weeks that extended high was bright and unnatural. Certainly not the way I choose a component for years of listening pleasure.

[3LB]

Blind testing is completely relevant to audio. But I would not use it to choose a component for purchase, as that is the wrong way to use DBT.

DBT is used to define differences without introducing prejudices that sighted testing can do.

+1

no need to fear the DBT

[tube fan]

I am totaly against blind testing when it comes to audio. I have to listen to the equipment for a period of time before I decide if it is right for me. In quick blind comparisons I can hear differences but not what is best over the long haul. I think blind listening tests are bull****.

I agree, but only that longterm blind listening tests are needed. Of course, the same is true for wines. Of the last ten blind wine tastings I have attended, the most inexpensive wine won 4 times (prices from $10 to 250). I suspect that the same thing would occur in blind listening tests, both short and long durations. Yes, I bet most "golden ear" experts would not rate their favorite, very expensive components at the top. I bet many cheaper, especially tube driven systems, would be preferred over the unbelievably expensive systems, especially digital and ss based ones.

[poppachubby]

I agree, but only that longterm blind listening tests are needed. Of course, the same is true for wines. Of the last ten blind wine tastings I have attended, the most inexpensive wine won 4 times (prices from $10 to 250). I suspect that the same thing would occur in blind listening tests, both short and long durations. Yes, I bet most "golden ear" experts would not rate their favorite, very expensive components at the top. I bet many cheaper, especially tube driven systems, would be preferred over the unbelievably expensive systems, especially digital and ss based ones.

tube fan, I like you but I will play devil's advocate for a second.

I have heard you say on more than one occasion that you have blindly picked out your favorite (AR?) speakers from a line up of others.

If you are able to do that, then why can't a "golden ear" (and they do exist) pick out a fine piece of tube kit from horrid state?

[tube fan]

tube fan, I like you but I will play devil's advocate for a second.

I have heard you say on more than one occasion that you have blindly picked out your favorite (AR?) speakers from a line up of others.

If you are able to do that, then why can't a "golden ear" (and they do exist) pick out a fine piece of tube kit from horrid state?

Poppa, I am NOT claiming that there are not significant differences in equipment, especially between ss and tube, and analogue and digital. I am NOT contending that no one can consistently rate and identify equipment. I just happen to be quite good at consistently rating and identifying wines
tasted blind. I AM contending that many "golden ears" could not rate consistently their favorite expensive equipment vs more modest ones.

[poppachubby]

Poppa, I am NOT claiming that there are not significant differences in equipment, especially between ss and tube, and analogue and digital. I am NOT contending that no one can consistently rate and identify equipment. I just happen to be quite good at consistently rating and identifying wines
tasted blind. I AM contending that many "golden ears" could not rate consistently their favorite expensive equipment vs more modest ones.

Don't get defensive, I'm not trying to make you look foolish.

I don't see how your reply answers my question. What you are talking about is apples and oranges. If someone can't pick out expensive gear vs. inexpensive, how are you able to pick out a pair of speakers vs. others?

My point is not expensive vs inexpensive, simply A vs B. I think some people probably can pick out a piece of gear from others, just as you can pick out the AR speakers.

[RGA]

I agree, but only that longterm blind listening tests are needed. Of course, the same is true for wines. Of the last ten blind wine tastings I have attended, the most inexpensive wine won 4 times (prices from $10 to 250). I suspect that the same thing would occur in blind listening tests, both short and long durations. Yes, I bet most "golden ear" experts would not rate their favorite, very expensive components at the top. I bet many cheaper, especially tube driven systems, would be preferred over the unbelievably expensive systems, especially digital and ss based ones.

Martin Colloms of Stereophile already conducted such a test where an old no feedback tube amp beat expensive SS amps that were the best out at the times. But get this - the blind test was done with all the TOP SS designers and they chose the tube amp as sounding better than their own amps. You don't have to BET that the experts prefer no feedback - they do. here's the link http://www.stereophile.com/content/f...thout-feedback

[TheHills44060]

I happen to like blind testing just to keep me honest. But at the same time if I am going to lay down some major cash for a set of gear (especially loudspeakers) i like to see them. Not only do I want my gear to sound good i think it should look good too. Once you get to a certain price point there is no reason a manufacturer cant make the gear sound good and look pleasing to the eye too.

[Ajani]

.................................................. ..
if I am going to lay down some major cash for a set of gear (especially loudspeakers) i like to see them. Not only do I want my gear to sound good i think it should look good too. Once you get to a certain price point there is no reason a manufacturer cant make the gear sound good and look pleasing to the eye too.

I agree... While I'm sure most persons would disagree with each other on what that "certain price point" is, I think looks matter to many of us... Also, I don't see why a component can't both look and sound good... At $200 a speaker can be excused for not having a nice finish, but at $10,000 there is no excuse IMO...

[poppachubby]

And here I always thought John Micheal liked blindfolds.

[JohnMichael]

And here I always thought John Micheal liked blindfolds.

When the complete zippered hood is not available.

[RGA]

When the complete zippered hood is not available.

bring out the gimp!

Blindfolds are very helpful in many instances when you have not had enough beer.

[Woochifer]

I think the issue here is simple bias control. There's definitely a tendency, whether consciously or otherwise, to hear what we want to hear. A blind comparison is a very useful tool for assessing differences or non-differences between components, formats, et al. I've caught myself before believing that I heard a difference when actually nothing had been changed in my setup.

I also recall similar tricks done by some high end audio manufacturers. A few years ago, Wilson Audio demoed a new speaker system using an iPod as the source. But, they'd first put up this elaborate looking test rig, so the audience didn't know they were listening to those high end speakers through an iPod.

At high end audio shows, Dunlavy and McIntosh have also pretended to swap out speaker cables to make a point about exotic cabling. People would swoon about the "night and day" difference that the cabling made, only to find out that nothing in the system setup had changed between listenings.

With wine tasting, I've definitely seen how preferences and expectations will often boil down to the price point. When you put the blinders on, then it really tests just how good someone's purportedly well-trained taste buds are.

As T mentioned, this is not necessarily an assessment of preference, but rather of reliable comparison. And whether we admit it or not, I think we all have some biases that need to be controlled to some extent.

[Ajani]

I think the issue here is simple bias control. There's definitely a tendency, whether consciously or otherwise, to hear what we want to hear. A blind comparison is a very useful tool for assessing differences or non-differences between components, formats, et al. I've caught myself before believing that I heard a difference when actually nothing had been changed in my setup.

I also recall similar tricks done by some high end audio manufacturers. A few years ago, Wilson Audio demoed a new speaker system using an iPod as the source. But, they'd first put up this elaborate looking test rig, so the audience didn't know they were listening to those high end speakers through an iPod.

At high end audio shows, Dunlavy and McIntosh have also pretended to swap out speaker cables to make a point about exotic cabling. People would swoon about the "night and day" difference that the cabling made, only to find out that nothing in the system setup had changed between listenings.

With wine tasting, I've definitely seen how preferences and expectations will often boil down to the price point. When you put the blinders on, then it really tests just how good someone's purportedly well-trained taste buds are.

As T mentioned, this is not necessarily an assessment of preference, but rather of reliable comparison. And whether we admit it or not, I think we all have some biases that need to be controlled to some extent.

I agree... As I mentioned earlier, both the DBT crowd and the anti-DBT crowd have points... But too often it turns into a hardcore idealogical debate (and shortly thereafter a pointless quarrel), rather than a productive discussion of how best to use sighted versus blind listening...

[Woochifer]

Trust me, there is very little actual correlation between price and perceived quality, either in wines or in "high end" equipment.

This line however I do take issue with. I think that there's a very strong correlation in the lower price points. When you move up from $20 speakers to $100 speakers, there is definitely a marked improvement. And likewise when you move from $100 to $500.

I agree though that there is a point of diminishing returns, and that's where you get into the stickiness of whether any perceived improvement in performance is worth the often exorbitant price increases.

[tube fan]

This line however I do take issue with. I think that there's a very strong correlation in the lower price points. When you move up from $20 speakers to $100 speakers, there is definitely a marked improvement. And likewise when you move from $100 to $500.

I agree though that there is a point of diminishing returns, and that's where you get into the stickiness of whether any perceived improvement in performance is worth the often exorbitant price increases.

I am NOT talking about $20 speakers. I AM talking about under $10,000 speakers compared to $150,000 ones. I AM talking about something like my system: AR SP8, AR D-70, Fosgate Phono, VPI Scoutmaster, Benz Ruby 3, and Fulton J or Dunlavy SC-IV speakers compared to ones costing $400,000 (like ones I heard at the California Audio Show).

Bias is an all too human trait. When we KNOW we are tasting a Screaming Eagle $1,500 bottle of wine, we tend to look for positives. When we are comparing a Benzinger $15 bottle to the Screaming Eagle BLIND, the resulting evaluation will often be quite different. I recall a series of wine tastings conducted by the NYT. All were Pinot Noir based, from France, Oregon, and California. Tasted and rated knowing the bottle name, the French wines were rated first, then Oregon's, followed by California's. Tasting EXACTLY the same wines blind, California wines came in first, followed by Oregon's and French wines. I've seen it hundreds of times in blind tastings. I bet a similar thing would occur in blind listening tests, short or longterm.