Sighted listening unreliable?

[okiemax]

In respose to posters who claim to hear differences when using different cables, some forum members reply with the counter claim that sighted listening comparisons aren't reliable, because seeing the cables can bias the listener. The "heard a difference" claims usually are cable specific(e.g, the Kimber model X sounded different than the Monster model Y.) On the other hand, the "sighted listening isn't reliable" claims are general and seem to imply broad application. But surely, those who hold this view don't mean sighted listening always leads to the wrong conclusion. How would they know if a particular conclusion is right or wrong?

[Pat D]

In respose to posters who claim to hear differences when using different cables, some forum members reply with the counter claim that sighted listening comparisons aren't reliable, because seeing the cables can bias the listener. The "heard a difference" claims usually are cable specific(e.g, the Kimber model X sounded different than the Monster model Y.) On the other hand, the "sighted listening isn't reliable claims" are general and seem to imply broad application. But surely, those who hold this view don't mean sighted listening always leads to the wrong conclusion. How would they know if a particular conclusion is right or wrong?

That's a pretty good definition of unreliability: you don't know if it's right or wrong.

http://dictionary.reference.com/search?q=unreliable

Next question!

[mtrycraft]

In respose to posters who claim to hear differences when using different cables, some forum members reply with the counter claim that sighted listening comparisons aren't reliable, because seeing the cables can bias the listener. The "heard a difference" claims usually are cable specific(e.g, the Kimber model X sounded different than the Monster model Y.) On the other hand, the "sighted listening isn't reliable claims" are general and seem to imply broad application. But surely, those who hold this view don't mean sighted listening always leads to the wrong conclusion. How would they know if a particular conclusion is right or wrong?

How could that general reply be anything more? A sighted listening protocol is biased and unreliable. Nothing further needs to be expaned on. It applies to all such listeing regardless of the unit being tested. You need a protocol that minimizes or eliminates bias.

[Tony_Montana]

You need a protocol that minimizes or eliminates bias.

....and lets not forget the memory factor also

You said that a sighted listening protocol is biased and unreliable.

Would that statement also apply to instantaneous switching listening protocol?
IMO instantaneous switching method (whether sighted or not) is more reliable than any other method since it does take away the memory factor

[okiemax]

That's a pretty good definition of unreliability: you don't know if it's right or wrong.

http://dictionary.reference.com/search?q=unreliable

Next question!

Forum members sometimes give their own special meanings to words. I recall being criticized for using the word "mid-fi" here on the Cable Forum. I used the term to indicate a level of equipment performance, and a Google search confirmed it usually is used that way, but I was informed that it had a derogatory meaning on this forum.

So I was wondering if "unreliable" also had some special meaning here. Perhaps some see this word as synonymous with "incorrect." The meaning to you and me, however, seems to be the same.

[mtrycraft]

....and lets not forget the memory factor also

You said that a sighted listening protocol is biased and unreliable.

Would that statement also apply to instantaneous switching listening protocol?
IMO instantaneous switching method (whether sighted or not) is more reliable than any other method since it does take away the memory factor

Swithing method has no bearing on the issue of bias or bias control.

It does have a bearing on memory as you allude to. That is why the so called claims for long term listeing is just another silly claim by audiophiles.

Rapid swithing doesn't take away memory, it just reduces it. You still process, you still judge what you remember. It certainly is the most sensitive and discriminating, less time to forget.

[okiemax]

How could that general reply be anything more? A sighted listening protocol is biased and unreliable. Nothing further needs to be expaned on. It applies to all such listeing regardless of the unit being tested. You need a protocol that minimizes or eliminates bias.

If you describe sighted listening for audible differences in cables as unreliable, I think it is fair to ask what you mean. If a listener claims cable X and cable Y sound different, and you reply that sighted listening is unreliable, does this mean you don't know whether the difference was real, or does it mean you don't believe the difference was real? If you don't believe it was real, and apply this to all listeners, aren't you saying sighted listening is reliable(i.e., anyone hearing a difference is wrong, and anyone not hearing a difference is right)?

[skeptic]

I'm the one who criticized the term mid-fi. Mid fi IMO is a derogatory term "audiophiles" use to describe equipment they don't like, usually equipment much less expensive than they own themselves. They place audio equipment in a kind of insane heirarchy which usually has no logical rationale to it. It is based on the same type of mentality that Audiophile magazine uses to rate equipment A, B, C etc. Like Audiophile Magazine's view of the world of electronics, it is not based on any objective set of standards or meaningful kind of measurement. If Audiophile Magazine wanted to make a real contribution to the ability of its readers to understand, evaluate, and select audio equipment based on the value of performance rather than personal preference, its "technical staff" would devise measurements which correlate to actual performance criteria based on objective double blind testing and publish them in professional journals like JAES for critical peer review.

Even the term high fidelity has all but become obsolete. Fidelity to what? By whose standards? By what criteria? The one time practice of double blind live versus recorded testing and demonstrations pioneered by the likes of Acoustic Research is virtually unknown any more. How many people who post here have actually ever been to one in their lifetimes? How do audiophiles evaluate audio equipment? They take their discs or vinyl records they like to their local high end audio salon and whichever speaker or amplifier or phono cartridge sounds best to them is the one they proclaim the winner. Small wonder every one of five thousand different models claims to be the best. What a joke. Most of the recordings people use for these evaluations are so gimmicked in the recording studio, they never bear even the slightest resemblance to what actually happened when the recording was made. And every recording engineer has his own bag of tricks to commercialize his clients' sound. Meanwhile, real progress towards accurate reproduction of unamplified musical instruments and voices of performers who are actual artists at their craft having devoted a lifetime to it, makes practically no real headway. This is an industry which long ago found out that it was both fruitless and unnecessary to struggle with real technoligical progress. Instead it produces endless variants of the same old paradymes it has had available for the last 40, 50, even 60 years and makes financial hay by packaging and repackaging it and hyping it to the moon. Audiophile cables with their allure and illusion are just one example of this.

For anyone who doubts any of this, just go to a few live performances of a symphony orchestra playing at your local concert hall or to a church to hear an organ recital or a choir and then go home and listen to you recordings through your sound system. It will sound dead and pathetic by comparison no matter what equipment you own. That's the current state of the art.

[Monstrous Mike]

In respose to posters who claim to hear differences when using different cables, some forum members reply with the counter claim that sighted listening comparisons aren't reliable, because seeing the cables can bias the listener. The "heard a difference" claims usually are cable specific(e.g, the Kimber model X sounded different than the Monster model Y.) On the other hand, the "sighted listening isn't reliable" claims are general and seem to imply broad application. But surely, those who hold this view don't mean sighted listening always leads to the wrong conclusion. How would they know if a particular conclusion is right or wrong?

Unreliable most certainly does not mean incorrect. It means that is unlikely that you are correct all of the time. Sighted listening is fraught with the possibility of errors in judgement, test method, biases from previous knowledge or expectations, etc. Therefore, it is appropriate to call sighted listening unreliable. The same could be said for DBTs which do not follow a stringent protocol. As a matter of fact, even a DBT with the most stringent of protocols and statistical validity will still have some degree of unreliability. What is being argued is that sighted listening has a much larger degree of unreliability due to the above factors and others.

I am confident most people here could reliably conduct sighted listening tests on certain speakers due to the large differences in sound. It should be generally accepted that cable differences would be much smaller and this would increase the unreliability of sighted listening.

While 100% reliability can really never be achieved, the level of reliability can be increased from a basic home sighted listening test by controlling factors which bias the results of that test. And knowing what you are listening to during such a test is a large factor in biasing the results, especially when differences are subtle.

So what does all this mean to us average Joe home audio enthusiasts. Well it shouldn't stop us from buying and testing home audio equipement to find a system we are happy with that's for sure. Nobody here is going to do DBTs using stringent scientific protocols to see if their new cables or CD player sounds any better. It's really more just a cautionary note that you can get carried away in the moment and spend lots of money for only an apparent or perceived improvement that may or may not be real.

For me personally, the really questionable statements regarding cables are the flowery descriptive characteristics given to certain brands and usually the more expensive translate to the more descriptive. There is no wrong or right regarding these informal reports of cable euphoria but the value and reliability of this sort of testing and test reporting can't be considered high or even average because they usually ignore factors which should be taken into consideration for proper comparisons.

[Monstrous Mike]

If a listener claims cable X and cable Y sound different, and you reply that sighted listening is unreliable, does this mean you don't know whether the difference was real, or does it mean you don't believe the difference was real?

If a listener reports cable X sounds different than cable Y, the first question I would ask would be what testing method and protocol was used. At this point, the difference may or may not have been real.

If the difference was real, it may have been the result of some other factor like the speakers were slightly moved when changing speaker wires or the listener wasn't in the exact same listening position. Even something like closing some drapes can affect the sound in a room. So if the listener did detect a real difference then we would need to find out if the difference is solely the result of the different cables.

If the difference was not real then we would have to determine how the listener came to this wrong conclusion. Perhaps it was because he expected cable X to be better. Perhaps he simply has bad audio judgement. Perhaps he has bad hearing.

And don't get me wrong. I never imply that the listener is dishonest or deceptive. There are many reasons for truthfully reporting cable differences when such differences could be the result of something entirely unrelated to the cables themselves.

Unfortunately, us scientists bash people over the head when they report a conclusion but have not done a proper test. That is a no-no in science land. Every conclusion has a degree of strength. The assessment of this strength is really a scientific and logical process. However, skewing this process happens all the time in real life. My uncle put up a bug zapper and concluded that the mosquitos in his back yard were reduced by 50%. This sounds good right off the bat (especially if you hate mosquitors), but if we look a little closer there are questions which need to be answered. In a simplistic scenario, he might have put the zapper up when a cold front moved in or a breeze picked up. Thus, the reduction in mosquitos would have been the result of natural weather rather than the zapper. (As a side note, bug zappers have been proven ineffective due to the fact that they only kill about 0.01% of the bug population at the best of times which is rather insignificant).

When I hear stuff like the bug zapper story I take it under advisement. This means I do not deny the claim but I hold it in question until more evidence in presented. This is the current situation I hold right now regarding cable sonics.

[mtrycraft]

If you describe sighted listening for audible differences in cables as unreliable, I think it is fair to ask what you mean. If a listener claims cable X and cable Y sound different, and you reply that sighted listening is unreliable, does this mean you don't know whether the difference was real, or does it mean you don't believe the difference was real? If you don't believe it was real, and apply this to all listeners, aren't you saying sighted listening is reliable(i.e., anyone hearing a difference is wrong, and anyone not hearing a difference is right)?

His report and his perception is unreliable based on the protocol being unreliable.

[okiemax]

His report and his perception is unreliable based on the protocol being unreliable.

I think you are saying you don't know if he is right or wrong.

[okiemax]

Unreliable most certainly does not mean incorrect. It means that is unlikely that you are correct all of the time. Sighted listening is fraught with the possibility of errors in judgement, test method, biases from previous knowledge or expectations, etc. Therefore, it is appropriate to call sighted listening unreliable. The same could be said for DBTs which do not follow a stringent protocol. As a matter of fact, even a DBT with the most stringent of protocols and statistical validity will still have some degree of unreliability. What is being argued is that sighted listening has a much larger degree of unreliability due to the above factors and others.

I am confident most people here could reliably conduct sighted listening tests on certain speakers due to the large differences in sound. It should be generally accepted that cable differences would be much smaller and this would increase the unreliability of sighted listening.

While 100% reliability can really never be achieved, the level of reliability can be increased from a basic home sighted listening test by controlling factors which bias the results of that test. And knowing what you are listening to during such a test is a large factor in biasing the results, especially when differences are subtle.

So what does all this mean to us average Joe home audio enthusiasts. Well it shouldn't stop us from buying and testing home audio equipement to find a system we are happy with that's for sure. Nobody here is going to do DBTs using stringent scientific protocols to see if their new cables or CD player sounds any better. It's really more just a cautionary note that you can get carried away in the moment and spend lots of money for only an apparent or perceived improvement that may or may not be real.

For me personally, the really questionable statements regarding cables are the flowery descriptive characteristics given to certain brands and usually the more expensive translate to the more descriptive. There is no wrong or right regarding these informal reports of cable euphoria but the value and reliability of this sort of testing and test reporting can't be considered high or even average because they usually ignore factors which should be taken into consideration for proper comparisons.

Thank you for an excellent response. You have made it clear what you mean by sighted listening being "unreliable." I agree with you that the results of any listening test, be it sighted or blinded, can be unreliable. I also agree there may be value in making people aware that biases can affect their perception and judgement, as long as we recognize biases travel a two-way street. If suggestion has power, either yeasayer bias or naysayer bias can influence a newbie's perception and judgement in a detrimental way, reducing either his funds or his enjoyment.

[mtrycraft]

I think you are saying you don't know if he is right or wrong.

That is one way of saying it. No one can be, not even the person in the test.

[okiemax]

That is one way of saying it. No one can be, not even the person in the test.

Regarding the second sentence, if I say "I know I'm right," and then prove it in a blinded test, wouldn't it be illogical for you to say "you didn't know you were right until you proved it?"

[okiemax]

I'm the one who criticized the term mid-fi. Mid fi IMO is a derogatory term "audiophiles" use to describe equipment they don't like, usually equipment much less expensive than they own themselves. They place audio equipment in a kind of insane heirarchy which usually has no logical rationale to it. It is based on the same type of mentality that Audiophile magazine uses to rate equipment A, B, C etc. Like Audiophile Magazine's view of the world of electronics, it is not based on any objective set of standards or meaningful kind of measurement. If Audiophile Magazine wanted to make a real contribution to the ability of its readers to understand, evaluate, and select audio equipment based on the value of performance rather than personal preference, its "technical staff" would devise measurements which correlate to actual performance criteria based on objective double blind testing and publish them in professional journals like JAES for critical peer review.

Even the term high fidelity has all but become obsolete. Fidelity to what? By whose standards? By what criteria? The one time practice of double blind live versus recorded testing and demonstrations pioneered by the likes of Acoustic Research is virtually unknown any more. How many people who post here have actually ever been to one in their lifetimes? How do audiophiles evaluate audio equipment? They take their discs or vinyl records they like to their local high end audio salon and whichever speaker or amplifier or phono cartridge sounds best to them is the one they proclaim the winner. Small wonder every one of five thousand different models claims to be the best. What a joke. Most of the recordings people use for these evaluations are so gimmicked in the recording studio, they never bear even the slightest resemblance to what actually happened when the recording was made. And every recording engineer has his own bag of tricks to commercialize his clients' sound. Meanwhile, real progress towards accurate reproduction of unamplified musical instruments and voices of performers who are actual artists at their craft having devoted a lifetime to it, makes practically no real headway. This is an industry which long ago found out that it was both fruitless and unnecessary to struggle with real technoligical progress. Instead it produces endless variants of the same old paradymes it has had available for the last 40, 50, even 60 years and makes financial hay by packaging and repackaging it and hyping it to the moon. Audiophile cables with their allure and illusion are just one example of this.

For anyone who doubts any of this, just go to a few live performances of a symphony orchestra playing at your local concert hall or to a church to hear an organ recital or a choir and then go home and listen to you recordings through your sound system. It will sound dead and pathetic by comparison no matter what equipment you own. That's the current state of the art.

I agree that the term "mid-fi" could be used in a derogatory way. But I don't think it's derogatory to say there are different levels of performance in audio equipment. Nor do I see anything wrong with ratings based on subjective reviews. These reviews are what they are -- OPINIONS. What is wrong with a reviewer saying that In his opinion this amp is an A, that one a B, and the other a C? Is it unethical to voice an opinion?

I am assuming you meant Stereophile Magazine when you referred to "Audiophile Magazine." While I sometimes get annoyed at Stereophile, I think their reviews can be helpful to readers who are looking for models to consider. A few years ago I was in the market for inexpensive speakers and found their reviews of the Polk RT25i and Radio Shack Optimus LX-4 to be interesting and informative. I am providing a link to the RT25i review for your comments.

http://www.stereophile.com/loudspeak...417/index.html