Interconnects double-blind [Archive] - Audio & Video Forums

PDA

View Full Version : Interconnects double-blind



okiemax
04-01-2004, 12:28 AM
It's hard to find good double-blind studies on audible differences in interconnect cables on the web. The best I have seen so far is the TAG McLaren experiment which got null results comparing two expensive cables, the Nordost Solar Wind and the TAG Mclaren F3-10-ANA. The problems with this study and its conclusion have already been discussed on the Forum. Another study, which I don't recall being discussed, got null results comparing an inexpensive interconnect with 5 specialty models:

http://www.pcavtech.com/abx/abx_wire.htm

Five interconnect from AudioQuest, MIT, Monster Cable, H.E.A.R., plus Belden cable with Vampire connectors were compared to a $2.50 blister pack RCA phono interconnect. The 7 listeners participating in the study used Etymotic Research ER4 in-ear phones driven by the headphone jack of a Bryston 2B power amplifier. There were 139 total trials, and the participants correctly identified interconnects 70 times. This is all the information that was given, and it is insufficient for a review of the study. However, I do have a few comments:

I could not verify that the Bryston 2B power amplifier has a headphone jack. Does anyone know about this?

The Etymotic Research ER4 in-ear phones probably are very good for portable use, such as in an airplane, but I doubt many audiophiles use them at home. Shouldn't interconnects be tested in a system with speakers rather than phones?

Aggregation can inadvertently conceal differences. We would not know from the total correct score(70 out of 139) if a few individuals identified a few interconnects correctly ever time they tried. It looks like each participant listened to each cable only 4 times(139/5/7= about 4). A few "4 correct out of 4 tested" scores on individual cables could be hidden in the total score. Whether this happened or not, I don't know, but it's certainly possible.

The possibilty that aggregation concealed differences, however, is not the only problem. If a listener only had 4 tries per interconnect, he would not have the opportunity to prove that his performance was not just random, since more tries would be necessary to dismiss that possibility. Therefore, the study may have had a bias against recognizing individual performances from the start.

Does anyone know of other double-blind studies of interconnects available on the web?

E-Stat
04-01-2004, 05:27 AM
The Etymotic Research ER4 in-ear phones probably are very good for portable use, such as in an airplane, but I doubt many audiophiles use them at home. Shouldn't interconnects be tested in a system with speakers rather than phones?
I think it depends upon what you are trying to test. I have yet to hear any phones (and I've heard some good ones from Jecklin and Stax) that are able to float an orchestral image in space like l've heard with the best speakers. Given that I find that part of what the best ICs do is to preserve spatial cues, I would agree.

rw

Thomas_A
04-01-2004, 07:59 AM
Interconnects can display audible differences in certain test situations, e.g. shielded vs. non-shielded cables. In my rig, some very cheap cables are introducing hum if they're close to some other cables. By moving them around, they will "sound different". ;)

T

rb122
04-01-2004, 10:17 AM
Thank you for the links.

One area pertaining to the possible fallibility of double blind testing for small differences that has come up on this board in the past has never been properly refuted. My apologies to those who are tiring of the cable debate but I'd appreciate anyone's thoughts on this. As a cable "dontcaresayer", I'm interested out of curiousity, not out of hidden agenda.

If we were to view two pictures of, say, a blue filled in circle and the two were very slightly different in hue, we may not be able to tell them apart without looking at them both at the same time - in a quickie test, such as the ones for DBT audible differences. But to look at them both at once, we could fairly immediately tell that one blue was subtly different than another. On the other hand, if we looked at them both individually over a longer period of time, we may come to recognize them as different even if we couldn't hold them up next to one another for differentiation.

Does DBT irrefutably unmask small audible differences as is claimed? Is there anyway to conduct a longer term DBT rather than quick 30 second swatches of music? Many audiophiles claim that it takes awhile to get to know and perceive the differences of one cable to another and quick tests are a flawed means of distinguishing them. And I don't believe that any audiophile would truly term cable differences as anything but subtle.

Any thoughts are appreciated.

mtrycraft
04-01-2004, 03:55 PM
It's hard to find good double-blind studies on audible differences in interconnect cables on the web. The best I have seen so far is the TAG McLaren experiment which got null results comparing two expensive cables, the Nordost Solar Wind and the TAG Mclaren F3-10-ANA. The problems with this study and its conclusion have already been discussed on the Forum. Another study, which I don't recall being discussed, got null results comparing an inexpensive interconnect with 5 specialty models:

http://www.pcavtech.com/abx/abx_wire.htm

Five interconnect from AudioQuest, MIT, Monster Cable, H.E.A.R., plus Belden cable with Vampire connectors were compared to a $2.50 blister pack RCA phono interconnect. The 7 listeners participating in the study used Etymotic Research ER4 in-ear phones driven by the headphone jack of a Bryston 2B power amplifier. There were 139 total trials, and the participants correctly identified interconnects 70 times. This is all the information that was given, and it is insufficient for a review of the study. However, I do have a few comments:

I could not verify that the Bryston 2B power amplifier has a headphone jack. Does anyone know about this?

The Etymotic Research ER4 in-ear phones probably are very good for portable use, such as in an airplane, but I doubt many audiophiles use them at home. Shouldn't interconnects be tested in a system with speakers rather than phones?

Aggregation can inadvertently conceal differences. We would not know from the total correct score(70 out of 139) if a few individuals identified a few interconnects correctly ever time they tried. It looks like each participant listened to each cable only 4 times(139/5/7= about 4). A few "4 correct out of 4 tested" scores on individual cables could be hidden in the total score. Whether this happened or not, I don't know, but it's certainly possible.

The possibilty that aggregation concealed differences, however, is not the only problem. If a listener only had 4 tries per interconnect, he would not have the opportunity to prove that his performance was not just random, since more tries would be necessary to dismiss that possibility. Therefore, the study may have had a bias against recognizing individual performances from the start.

Does anyone know of other double-blind studies of interconnects available on the web?

If you check the pcavtech link you will see that each listener had 20 trials. What is not clear is how many listened to each of the 5 cables tested. Some listened to 1 some to more than one.
If one of the listener had a statistically significant outcome, that would have been so reported. The author knows how to test and report.]

There are no audible differences in ic that is not broken. There is no such evidence.

okiemax
04-01-2004, 09:58 PM
If you check the pcavtech link you will see that each listener had 20 trials. What is not clear is how many listened to each of the 5 cables tested. Some listened to 1 some to more than one.
If one of the listener had a statistically significant outcome, that would have been so reported. The author knows how to test and report.]

There are no audible differences in ic that is not broken. There is no such evidence.

My point was aggregation can conceal differences. The study could have avoided questions about the possibility of such a problem simply by presenting all relevant data. Doing otherwise undermines crediblity. Rather than rely on your confidence in the author's ability to test and report, I would prefer to see the facts.

You have, however, called my attention to another problem with the study.
If each listener completed 20 trials and some as you say "only tested one interconnect," then all interconnects did not get an equal number of trials. I doubt any interconnect got only one trial from one listener, but that would not be inconsistent with the totals, and only the totals were reported. Again, aggregation conceals the differences.

I am not a statistician, but I suspect what the study did in summing this mixture of cables and trials for hypothesis testing might be questionable statistical practice.

Thomas_A
04-02-2004, 03:32 AM
A way to investigate individual are repeat tests, whithout knowing the score from the first trial.

I have seen fewer repeat tests, but they exist. None has shown audible difference as far as I know. In the Tag McLaren test, the best scorers with cables did poorly with the amp test.

The Tag link is gone now, or does anyone know where it is?

T

okiemax
04-04-2004, 02:29 PM
A way to investigate individual are repeat tests, whithout knowing the score from the first trial.

I have seen fewer repeat tests, but they exist. None has shown audible difference as far as I know. In the Tag McLaren test, the best scorers with cables did poorly with the amp test.

The Tag link is gone now, or does anyone know where it is?

T

IAG acquired TAG audio and recently temorarily removed the TAG website. Maybe the ABX study will still be there when the site is returned. If not, perhaps someone has saved a copy. It always puzzled me why the study was on their site to begin with, since the results seemed to be more a reason to not buy the TAG interconnect and amp than a reason to buy them.

mtrycraft
04-04-2004, 08:38 PM
You have, however, called my attention to another problem with the study.
If each listener completed 20 trials and some as you say "only tested one interconnect," then all interconnects did not get an equal number of trials. I doubt any interconnect got only one trial from one listener, but that would not be inconsistent with the totals, and only the totals were reported. Again, aggregation conceals the differences.

I am not a statistician, but I suspect what the study did in summing this mixture of cables and trials for hypothesis testing might be questionable statistical practice.


You don't need equal number of trials per cable. You need a sufficient number of trials by one listener which seems to be 20 trials per person for a cable, not one trial. 7 listeners, 139 trials.
Yes, they could have made more people happier by listing each cable by itself.
As Thomas A indicates, repeating it is what is needed.

okiemax
04-04-2004, 11:45 PM
You don't need equal number of trials per cable. You need a sufficient number of trials by one listener which seems to be 20 trials per person for a cable, not one trial. 7 listeners, 139 trials.
Yes, they could have made more people happier by listing each cable by itself.
As Thomas A indicates, repeating it is what is needed.

I don't know why the results of listening trials for each interconnect were not tested for statistical significance separately in the first place. Lumping results for all five interconnects together, without showing results for each cable and participant, and then testing the totals for significance doesn't seem like a good idea to me. It can raise questions about the possibility that something significant is buried in the totals.

Unequal trials raise additional questions because more weight is given to some interconnects than others in the significance test. In the interconnect study the ratio of correct scores to number of trials could have been a perfect 10/10 for one cable and a combined 60/129 for the remaining four with no change in the overall total of 70/139, and of course, the same null result. I doubt this was the case, but it shows how unequal trials can be misleading.

Thomas_A
04-05-2004, 04:33 AM
IAG acquired TAG audio and recently temorarily removed the TAG website. Maybe the ABX study will still be there when the site is returned. If not, perhaps someone has saved a copy. It always puzzled me why the study was on their site to begin with, since the results seemed to be more a reason to not buy the TAG interconnect and amp than a reason to buy them.

Well there can be many reasons to buy a certain amp. Features, build quality etc. Some amps are also not well-built, with higher hiss/noise, and would easily be detected in a blind test (e.g. a Marantz reciever which I have tested). In extreme situations amps can be scored differently in blind tests also for e.g. the bass response. Many amps are filtered in the low end to protect from DC, and this may be felt (not heard) in the body when playing signals in the 10 Hz region. Such tests are made by the Swedish Audio-Technical Society using a system able to give >139 dB at 20 Hz in-room response at one metre (measured, not calculated). Some amps have been passing this before/after test, e.g. NAD208 and Rotel RB1090, with no or very small remarks, while other amps have shown significant deviation (e.g. 7/7 correct answers, two independent tests, one person = 14/14 correct ansvwers). CD player tests are also made, with significant blind test scores for certain players. No one has ever been able to score any difference in cables though.

T

okiemax
04-05-2004, 10:35 AM
Well there can be many reasons to buy a certain amp. Features, build quality etc. Some amps are also not well-built, with higher hiss/noise, and would easily be detected in a blind test (e.g. a Marantz reciever which I have tested). In extreme situations amps can be scored differently in blind tests also for e.g. the bass response. Many amps are filtered in the low end to protect from DC, and this may be felt (not heard) in the body when playing signals in the 10 Hz region. Such tests are made by the Swedish Audio-Technical Society using a system able to give >139 dB at 20 Hz in-room response at one metre (measured, not calculated). Some amps have been passing this before/after test, e.g. NAD208 and Rotel RB1090, with no or very small remarks, while other amps have shown significant deviation (e.g. 7/7 correct answers, two independent tests, one person = 14/14 correct ansvwers). CD player tests are also made, with significant blind test scores for certain players. No one has ever been able to score any difference in cables though.

T

I would agree there should be more difference in amps than interconnects. However, in the TAG study the participant's scores were better on the interconnects than on the amps. One participant had a close to statistically significant score on the interconnects. There were no follow-up tests by the best scoring participants.

All the items tested were expensive, but the TAG interconnect was more expensive than the one from Nordost, and the TAG amp cost more than the Bryston. Since the study concluded that participants didn't hear differences in these cables and amps, why would anyone want to pay more for the TAG products? And why did TAG put information favoring their competitor's products over their own on the TAG websits ? It makes no sense to me.

I don't know whether anyone has ever been able to hear difference between cables in a controlled test. If they have, I haven't seen it reported. Does that mean it has never been done and can't be done? Nah!

okiemax
04-05-2004, 08:13 PM
Thank you for the links.

One area pertaining to the possible fallibility of double blind testing for small differences that has come up on this board in the past has never been properly refuted. My apologies to those who are tiring of the cable debate but I'd appreciate anyone's thoughts on this. As a cable "dontcaresayer", I'm interested out of curiousity, not out of hidden agenda.

If we were to view two pictures of, say, a blue filled in circle and the two were very slightly different in hue, we may not be able to tell them apart without looking at them both at the same time - in a quickie test, such as the ones for DBT audible differences. But to look at them both at once, we could fairly immediately tell that one blue was subtly different than another. On the other hand, if we looked at them both individually over a longer period of time, we may come to recognize them as different even if we couldn't hold them up next to one another for differentiation.

Does DBT irrefutably unmask small audible differences as is claimed? Is there anyway to conduct a longer term DBT rather than quick 30 second swatches of music? Many audiophiles claim that it takes awhile to get to know and perceive the differences of one cable to another and quick tests are a flawed means of distinguishing them. And I don't believe that any audiophile would truly term cable differences as anything but subtle.

Any thoughts are appreciated.

Although I probably could tell two different hues of blue apart if they were next to each other, I don't know whether I could do it they were isolated, even if I had been seeing them every day for several months. It would be interesting to try.

If you are curious, you could do a blinded test of cables at home over as long of a period as you wanted. However, if you are trying a new cable you might want to finish testing before the 30-day or 60-day money back guarantee expires. I am not sure about the frequency of switching between the items being compared, but a single-blind test would be practical if the frequency is in days rather than minutes, and not having to use a comparator box avoids expense and the issue of whether the box is screwing up the test. The experiment probably is best done with inteconnects, which are easier to conceal than speaker cables.

The results you get may not change the views of anyone here on the Forum.

mtrycraft
04-05-2004, 09:25 PM
.One area pertaining to the possible fallibility of double blind testing for small differences that has come up on this board in the past has never been properly refuted.

What problems? There are no problems. The raised questions are no proof that it is a problem in th efirst place, just a guess.



If we were to view two pictures of, say, a blue filled in circle and the two were very slightly different in hue, we may not be able to tell them apart without looking at them both at the same time - in a quickie test, such as the ones for DBT audible differences. But to look at them both at once, we could fairly immediately tell that one blue was subtly different than another. On the other hand, if we looked at them both individually over a longer period of time, we may come to recognize them as different even if we couldn't hold them up next to one another for differentiation.

Why don't you try this? See if you can match your rooms color which you should know by hart, at th epaint store. Or, as was suggested, by looking at one chip for a while and another, remotely, and see if you kan tell?
But you are testing your memory and it isn't even close. It doesn't work.

Does DBT irrefutably unmask small audible differences as is claimed?

Yes, or it would not be the gold standard in audio.

Is there anyway to conduct a longer term DBT rather than quick 30 second swatches of music?

Yes. I have cited an amp test lasting months. Again, your memory fades rather quickly. Talk to people in the audialogy field. Acoustic memory for small differences is very short, seconds, really. Even that 30 seconds is way too long.

In real research, at times only one note is compared rapidly for differences.



Many audiophiles claim that it takes awhile to get to know and perceive the differences of one cable to another and quick tests are a flawed means of distinguishing them.

Why have they not demonstrated these claims then? Anyone? Just because some audiophile makes some empty claim doesn't mean it is true.
The corollary I like to use is about alien abductees, psychic subject, etc. They swear by what they experience as fact.

And I don't believe that any audiophile would truly term cable differences as anything but subtle

You need to read more:) unless you dismiss outright those who do as non audiophiles.

mtrycraft
04-05-2004, 09:33 PM
I would agree there should be more difference in amps than interconnects. However, in the TAG study the participant's scores were better on the interconnects than on the amps. One participant had a close to statistically significant score on the interconnects. There were no follow-up tests by the best scoring participants.

Why could this not be just plain chance event?
That is what it was.


All the items tested were expensive, but the TAG interconnect was more expensive than the one from Nordost, and the TAG amp cost more than the Bryston. Since the study concluded that participants didn't hear differences in these cables and amps, why would anyone want to pay more for the TAG products? And why did TAG put information favoring their competitor's products over their own on the TAG websits ? It makes no sense to me.

A number of years ago Quad did a DBT, 576 trials with their amps and others:)

I don't know whether anyone has ever been able to hear difference between cables in a controlled test.

Not between comparable cables. Greenhill was able to demonstrate differences between 24 ga and 16 ga by some participants. :)

If they have, I haven't seen it reported. Does that mean it has never been done and can't be done? Nah!

You are right. But without it, it is really moot untill it happens.

mtrycraft
04-05-2004, 09:38 PM
I don't know why the results of listening trials for each interconnect were not tested for statistical significance separately in the first place. Lumping results for all five interconnects together, without showing results for each cable and participant, and then testing the totals for significance doesn't seem like a good idea to me. It can raise questions about the possibility that something significant is buried in the totals.

Unequal trials raise additional questions because more weight is given to some interconnects than others in the significance test. In the interconnect study the ratio of correct scores to number of trials could have been a perfect 10/10 for one cable and a combined 60/129 for the remaining four with no change in the overall total of 70/139, and of course, the same null result. I doubt this was the case, but it shows how unequal trials can be misleading.


You know what? Why not send an email to David Carlstrom at the bottom of the page. I am sure he can get the answer for you.

Unequal trials if in fact they didn't keep separate scores, I am sure they did, can be hidden as you state.
Send that email:) Let us know what happened:)

okiemax
04-05-2004, 11:23 PM
You know what? Why not send an email to David Carlstrom at the bottom of the page. I am sure he can get the answer for you.

Unequal trials if in fact they didn't keep separate scores, I am sure they did, can be hidden as you state.
Send that email:) Let us know what happened:)

I followed the instructions on the ABX Web Page and sent a request to Tom Nousaine at nousaine@aol.com. The body of that e-mail is shown below:

The ABX Web Page gives the results of a listening test with 7 participants comparing 5 specialty interconnects to a $2.50 blister pack RCA phono interconnect. I have been discussing this test with members of the Audio Review Cable Forum, and have raised the question about whether anything of interest might be in the details. For the purpose of the discussion, I would like to request additional information on the test. Specifically, I would like to obtain each participant's score for each interconnect.

mtrycraft
04-06-2004, 04:40 PM
I followed the instructions on the ABX Web Page and sent a request to Tom Nousaine at nousaine@aol.com. The body of that e-mail is shown below:

The ABX Web Page gives the results of a listening test with 7 participants comparing 5 specialty interconnects to a $2.50 blister pack RCA phono interconnect. I have been discussing this test with members of the Audio Review Cable Forum, and have raised the question about whether anything of interest might be in the details. For the purpose of the discussion, I would like to request additional information on the test. Specifically, I would like to obtain each participant's score for each interconnect.


Yes, that should do it:) I think Tom has all the data someplace:)

Norm Strong
04-06-2004, 09:18 PM
I've seen this argument a few times lately: Maybe the aggregate results are concealing a couple of golden ears that really could spot the difference. In that case, how did the aggregate results get back down to average? Other panel members would have had to do worse than average. How would that be possible? What mechanism is there that can produce worse than average results? Think about it.

okiemax
04-07-2004, 10:03 AM
I've seen this argument a few times lately: Maybe the aggregate results are concealing a couple of golden ears that really could spot the difference. In that case, how did the aggregate results get back down to average? Other panel members would have had to do worse than average. How would that be possible? What mechanism is there that can produce worse than average results? Think about it.

Random selection does not guarantee the numbers of correct and incorrect answers in a test will be exactly the same. If you don't believe my 60/129 example is possible, please explain why?

okiemax
04-09-2004, 09:43 PM
It's hard to find good double-blind studies on audible differences in interconnect cables on the web. The best I have seen so far is the TAG McLaren experiment which got null results comparing two expensive cables, the Nordost Solar Wind and the TAG Mclaren F3-10-ANA. The problems with this study and its conclusion have already been discussed on the Forum. Another study, which I don't recall being discussed, got null results comparing an inexpensive interconnect with 5 specialty models:

http://www.pcavtech.com/abx/abx_wire.htm

Five interconnect from AudioQuest, MIT, Monster Cable, H.E.A.R., plus Belden cable with Vampire connectors were compared to a $2.50 blister pack RCA phono interconnect. The 7 listeners participating in the study used Etymotic Research ER4 in-ear phones driven by the headphone jack of a Bryston 2B power amplifier. There were 139 total trials, and the participants correctly identified interconnects 70 times. This is all the information that was given, and it is insufficient for a review of the study. However, I do have a few comments:

I could not verify that the Bryston 2B power amplifier has a headphone jack. Does anyone know about this?

The Etymotic Research ER4 in-ear phones probably are very good for portable use, such as in an airplane, but I doubt many audiophiles use them at home. Shouldn't interconnects be tested in a system with speakers rather than phones?

Aggregation can inadvertently conceal differences. We would not know from the total correct score(70 out of 139) if a few individuals identified a few interconnects correctly ever time they tried. It looks like each participant listened to each cable only 4 times(139/5/7= about 4). A few "4 correct out of 4 tested" scores on individual cables could be hidden in the total score. Whether this happened or not, I don't know, but it's certainly possible.

The possibilty that aggregation concealed differences, however, is not the only problem. If a listener only had 4 tries per interconnect, he would not have the opportunity to prove that his performance was not just random, since more tries would be necessary to dismiss that possibility. Therefore, the study may have had a bias against recognizing individual performances from the start.

Does anyone know of other double-blind studies of interconnects available on the web?


I have a follow-up on the base post -----

Following instructions on the ABX Web Page, I sent an e-mail to Mr.Tom Nousaine on 4/6/04 requesting additional information on the interconnect test. Specifically, I asked for each participant's score on each interconnect, since I was interested in finding out whether anything of interest might be buried in the test totals, which were the only data reported. He replied on 4/7/04 with the following message:

"The test used an ABX comparator with interconnects directly connecting a cd player to a Bryston 2B power amplifier. Subjects listened singly in a separate room using headphones and often supplied their own program material and brought their own interconnects. Subjects were asked to agree to a minimum of 10 trials before starting.

I'd have to dig through the archives to find individual score sheets but suffice it to say there were no individually significant scores buried in the averaged results. IOW no single subject scored 9/10 or better. Nor were there reversely significant results. ( 0 or 1 of 10 trials.)"

Mr. Nousaine's reply answers a question I raised in another post about whether any 10/10 scores were buried in the test totals. Not having the requested scores for each participant for each interconnect, however, precludes any further attempts to analyyze the scores and leaves some questions unanswered. If one interconnect was tested only by two participants with scores of 7/10 and 8/10, for example, the combined 15/20 score would be statistically significant (p less than 0.021), and the remaining 55/119 would still be a random result.This probably didn't happen, but it is not inconsistent with the totals reported. Trying to guess everything the totals might possibly conceal, however, is needless if the detailed data become available.

I thanked Mr. Nousaine for his prompt reply, and added that If he finds the scores of each participant on each interconnect, I would appreciate receiving this information