Interesting Thesis on IC's

[Ajani]

Then you should step up and demonstrate this. I believe there are still some cash prizes being offered to those who can do this without peeking.



Sure, there have been many people over the past 30 some odd years who have made such claims. But so far, no one has ever demonstrated this ability under controlled conditions.



But as yet, no one has demonstrated actual audible differences under controlled conditions except when the differences were trivially measurable and within known thresholds of audibility.

If someone has, please point me to it so I can check it out.

se

The idea that no one has ever passed a blind test is a myth. In fact here's a test published in the Wall Street Journal. Both John Atkinson and Michael Fremer of Stereophile were able to identify the more expensive speaker cable in a blind test:

http://online.wsj.com/article/SB1200...p_us_inside_to

Using two identical CD players, I tested a $2,000, eight-foot pair of Sigma Retro Gold cables from Monster Cable, which are as thick as your thumb, against 14-gauge, hardware-store speaker cable. Many audiophiles say they are equally good. I couldn't hear a difference and was a wee bit suspicious that anyone else could. But of the 39 people who took this test, 61% said they preferred the expensive cable.

That may not be much of a margin for two products with such drastically different prices, but I was struck by how the best-informed people at the show -- like John Atkinson and Michael Fremer of Stereophile Magazine -- easily picked the expensive cable.
Its sound was described as "richer," "crisper" and "more coherent." Like some wines, come to think of it.

In absolute terms, though, the differences weren't great. Mr. Atkinson guesstimated the expensive cables sounded roughly 5% better. Remember, by definition, an audiophile is one who will bear any burden, pay any price, to get even a tiny improvement in sound.

More info on the testing is provided by John Atkinson in post #7 here:
http://www.stereophile.com/content/a...comment-334228

[Steve Eddy]

The idea that no one has ever passed a blind test is a myth. In fact here's a test published in the Wall Street Journal. Both John Atkinson and Michael Fremer of Stereophile were able to identify the more expensive speaker cable in a blind test:

Yes, I'm familiar with that.

However it wasn't a test so much as a dog and pony show and did absolutely nothing to establish whether there were any audible differences between the cables.

Each listener simply switched between A and B and then stated their preference if any. So even if there was no real difference at all, each listener had a 50/50 chance of saying they preferred the Monster cable.

That's it. One trial, 50/50 chance of picking the Monster. That JA and MF both happened to say they preferred what ended up being the Monster cable is ultimately meaningless.

Even JA admits it lacked any sort of scientific rigor.

Now, if JA or MF had gone through say 10 or 20 trials, with the cables being randomly assigned to A or B for each trial, and they consistently preferred the Monster cable, or at least enough to be statistically significant, then you might have something.

But that's not what happened here.

se

[Ajani]

Yes, I'm familiar with that.

However it wasn't a test so much as a dog and pony show and did absolutely nothing to establish whether there were any audible differences between the cables.

Each listener simply switched between A and B and then stated their preference if any. So even if there was no real difference at all, each listener had a 50/50 chance of saying they preferred the Monster cable.

That's it. One trial, 50/50 chance of picking the Monster. That JA and MF both happened to say they preferred what ended up being the Monster cable is ultimately meaningless.

Even JA admits it lacked any sort of scientific rigor.

Now, if JA or MF had gone through say 10 or 20 trials, with the cables being randomly assigned to A or B for each trial, and they consistently preferred the Monster cable, or at least enough to be statistically significant, then you might have something.

But that's not what happened here.

se

So it's mere coincidence that the experts had no trouble then?

Also JA has mentioned on the Stereophile forums, taking another blind test with MF, where he got 4 out 5 and Fremer got 5 out 5 correct... yet their results were seen as statistically insignificant because the 'average person' in the test didn't do well.... (I'll post the link later, if I can find it again)... So the combined results of 2 experts was 9 out of 10, but I'm sure that is also coincidence, until they both submit to around 20 trials each, right? Then their results would likely still be thrown out if the 'average man' doesn't score that well....

DBT relies way too much on statistics for me to take it that seriously...

[Steve Eddy]

So it's mere coincidence that the experts had no trouble then?

What exactly do you mean by "no trouble"?

Apparently none of the 39 who participated had any "trouble" stating their preference for A or B.

And what exactly makes them "experts"? Just because they write for an audio magazine?

Consider this and then tell me if you truly think MF at least qualifies as an "expert."

The common stated purpose behind high end audio is to preserve the signal and not alter or otherwise damage it in any way. There is endless marketing literature out there about all the pains taken to do this, using the purest conductors, the finest dielectrics, etc.

Some years ago Harmonic Technology introduced their "CyberLight" cables. While intended to be used between analogue components, they were in actually an optical cable. Electro-optical converters built into each end converted the source's electrical signal into an optical signal and at the other end, from optical back to electrical.

However these converters ultimately perform worse than the cheapest opamps you're likely to find in mass market gear. They had a huge hump in the low frequency response and massive amounts of harmonic and intermodulation distortion.

They mangled the signal so badly that when JA ran measurements on them, he wrote "If this were a review of a conventional product, I would dismiss it as being broken."

He further wrote "I am puzzled that Harmonic Technology, which makes good-sounding, reasonably priced conventional cables, would risk their reputation with something as technically flawed as the CyberLight."

And the real nail in the coffin, "I really don't see how the CyberLight P2A and Wave cables can be recommended."

However this is how the "expert," MF, summed them up in his review:

Harmonic Technology's Light Analog Module Photon Transducer is the most significant single technological breakthrough I have experienced in my career as an audio reviewer. It is immediately superior in every way.

So again, what exactly qualifies him as an "expert" if he finds huge frequency response aberrations and gross amounts of distortion to be "superior in every way"?

Also JA has mentioned on the Stereophile forums, taking another blind test with MF, where he got 4 out 5 and Fremer got 5 out 5 correct... yet their results were seen as statistically insignificant because the 'average person' in the test didn't do well.... (I'll post the link later, if I can find it again)...

Yes, I'm familiar with that test as well.

And I stated at the time that just because the average of everyone who took the test wasn't statistically significant, then JA's and MF's results shouldn't have been dismissed out of hand.

So the combined results of 2 experts was 9 out of 10...

No, you can't combine them like that. That's just as flawed as dismissing them because the average of all participants was no better than chance. You can only rightly consider them individually.

...but I'm sure that is also coincidence, until they both submit to around 20 trials each, right?

While 5 out of 5 may be statistically significant, the confidence level isn't very high.

So while I don't think that result should have been dismissed, neither do I think it provides any sort of conclusive evidence that there were actual audible differences between the cables.

More trials should have been done in order to improve the confidence level in the event there actually were audible differences.

Then their results would likely still be thrown out if the 'average man' doesn't score that well....

Again, I don't think their results should have been dismissed the way they were. Demonstrating actual audible differences doesn't require some group of individuals all score high. All it takes is one person.

DBT relies way too much on statistics for me to take it that seriously...

But it's only through adequate controls and statistics that we can establish actual audible differences with any confidence.

Don't be critical of those who ran the test for dismissing JA's and MF's results out of hand and then turn around and be just as dismissive yourself.

se

[Ajani]

What exactly do you mean by "no trouble"?

Apparently none of the 39 who participated had any "trouble" stating their preference for A or B.

And what exactly makes them "experts"? Just because they write for an audio magazine?

Consider this and then tell me if you truly think MF at least qualifies as an "expert."

The common stated purpose behind high end audio is to preserve the signal and not alter or otherwise damage it in any way. There is endless marketing literature out there about all the pains taken to do this, using the purest conductors, the finest dielectrics, etc.

Some years ago Harmonic Technology introduced their "CyberLight" cables. While intended to be used between analogue components, they were in actually an optical cable. Electro-optical converters built into each end converted the source's electrical signal into an optical signal and at the other end, from optical back to electrical.

However these converters ultimately perform worse than the cheapest opamps you're likely to find in mass market gear. They had a huge hump in the low frequency response and massive amounts of harmonic and intermodulation distortion.

They mangled the signal so badly that when JA ran measurements on them, he wrote "If this were a review of a conventional product, I would dismiss it as being broken."

He further wrote "I am puzzled that Harmonic Technology, which makes good-sounding, reasonably priced conventional cables, would risk their reputation with something as technically flawed as the CyberLight."

And the real nail in the coffin, "I really don't see how the CyberLight P2A and Wave cables can be recommended."

However this is how the "expert," MF, summed them up in his review:

Harmonic Technology's Light Analog Module Photon Transducer is the most significant single technological breakthrough I have experienced in my career as an audio reviewer. It is immediately superior in every way.

So again, what exactly qualifies him as an "expert" if he finds huge frequency response aberrations and gross amounts of distortion to be "superior in every way"?



Yes, I'm familiar with that test as well.

And I stated at the time that just because the average of everyone who took the test wasn't statistically significant, then JA's and MF's results shouldn't have been dismissed out of hand.



No, you can't combine them like that. That's just as flawed as dismissing them because the average of all participants was no better than chance. You can only rightly consider them individually.



While 5 out of 5 may be statistically significant, the confidence level isn't very high.

So while I don't think that result should have been dismissed, neither do I think it provides any sort of conclusive evidence that there were actual audible differences between the cables.

More trials should have been done in order to improve the confidence level in the event there actually were audible differences.



Again, I don't think their results should have been dismissed the way they were. Demonstrating actual audible differences doesn't require some group of individuals all score high. All it takes is one person.



But it's only through adequate controls and statistics that we can establish actual audible differences with any confidence.

Don't be critical of those who ran the test for dismissing JA's and MF's results out of hand and then turn around and be just as dismissive yourself.

se

So let me get this straight, you don't think Fremer and JA's results of 5/5 and 4/5 respectively should have been dismissed, yet you say things like this:

Sure, there have been many people over the past 30 some odd years who have made such claims. But so far, no one has ever demonstrated this ability under controlled conditions.



But as yet, no one has demonstrated actual audible differences under controlled conditions except when the differences were trivially measurable and within known thresholds of audibility.

If someone has, please point me to it so I can check it out.

It seems to me that you are dismissing the possibility that audible differences exist, yet you acknowledge that with (smaller than your ideal sample size of 10 to 20 trials) Fremer and Atkinson have shown that they can determine differences...

Fremer and Atkinson both claim to hear differences in cables, and the limited tests they've taken so far have done nothing to contradict that claim... Actually the results so far have only strengthened their claims...

It's theoretically possible that Fremer or Atkinson would pass tests of 1 and 5 and then turn around and fail a test of 10 or 20. However, I see the limited tests as providing at least enough evidence to make a reasonable person question whether the "accepted science" that there is no audible differences between cables is correct...

IMO. there are 2 unreasonable stances in audio: 1) Every tweak, mod or dollar thrown at a system makes an audible difference & 2) Because we don't know how to measure something or it hasn't been statistically proven, means it doesn't exist.... I find both positions to be equally ridiculous...

Finally let me ask you this: If Atkinson and Fremer agreed to do tests of 20, under scientific conditions you approved of, what would the results have to be for you to believe that they can hear differences in cables? 15/20? 20/20? What if Atkinson got 20/20 and Fremer got 10/20?

[Steve Eddy]

So let me get this straight, you don't think Fremer and JA's results of 5/5 and 4/5 respectively should have been dismissed, yet you say things like this:

Sure, there have been many people over the past 30 some odd years who have made such claims. But so far, no one has ever demonstrated this ability under controlled conditions.

But as yet, no one has demonstrated actual audible differences under controlled conditions except when the differences were trivially measurable and within known thresholds of audibility.

If someone has, please point me to it so I can check it out.

It seems to me that you are dismissing the possibility that audible differences exist...

Not at all.

I'm simply saying that to date, no one has demonstrated that there are. At least not when the measured differences are below known audible thresholds.

...yet you acknowledge that with (smaller than your ideal sample size of 10 to 20 trials) Fremer and Atkinson have shown that they can determine differences...

I acknowledged no such thing.

Neither MF nor JA showed that they could determine differences.

What I said was, their results shouldn't have been dismissed out of hand. By that I mean that they could very well have been due to pure guessing. But five trials is too few to rule that out and that more trials were warranted to find out.

Fremer and Atkinson both claim to hear differences in cables, and the limited tests they've taken so far have done nothing to contradict that claim... Actually the results so far have only strengthened their claims...

The results so far are too ambiguous to strengthen anything.

It's theoretically possible that Fremer or Atkinson would pass tests of 1 and 5 and then turn around and fail a test of 10 or 20. However, I see the limited tests as providing at least enough evidence to make a reasonable person question whether the "accepted science" that there is no audible differences between cables is correct...

The "accepted science" is simply that no one to date has demonstrated to any reasonable level of confidence that there are. That's not the same as saying there are none.

IMO. there are 2 unreasonable stances in audio: 1) Every tweak, mod or dollar thrown at a system makes an audible difference & 2) Because we don't know how to measure something or it hasn't been statistically proven, means it doesn't exist.... I find both positions to be equally ridiculous...

As do I.

Finally let me ask you this: If Atkinson and Fremer agreed to do tests of 20, under scientific conditions you approved of, what would the results have to be for you to believe that they can hear differences in cables? 15/20? 20/20?

I would like to see something on the order of 90%. And preferably I'd like to have a bit more than 20 trials.

What if Atkinson got 20/20 and Fremer got 10/20?

As I said earlier, it only takes one person to demonstrate audible differences. So it wouldn't matter that MF got 20/20.

se

[Ajani]

I would like to see something on the order of 90%. And preferably I'd like to have a bit more than 20 trials.



As I said earlier, it only takes one person to demonstrate audible differences.

So back to the question: how many trials to satisfy you then? 30? 100? What if the person only got 85% in 100 trials?