Testing and the Scientific Method

[pctower]

I am familiar with the Wireworld devices. One must be careful with it as it is manual and hence if the operator is with the listeners, it becomes single blind.
Interesting that when Tom Nousaine asked for data from them on DBT wire results, Wireworld disappears into the emptyness of space, no data to be found. I wonder why that is.
I suppose one could use their switching, somewhat expensive.

Anstending didn't publish any references that I saw, just assumes a/b comparison is only for visual senses. No memory in visual senses? That in itself needs a journal reference that is missing. It is all relative, after all, isn't it? If you compare the color on your wall to the chip in th epaint store, it certainly requires memory. But, if you bring it to the room, the time is reduced, not eliminated. I think you need to scrutinize some of the links you post for the ansurdity factor, no?

I am sure the whole of psychology, acoustics world would be interested in their new discovery. Sure would make things easy and less expensive in developing anything audio.

I guess JJ wasted his tenure at AT&T all that A/B testing for not or Toole's 25 years of research for not and the ongoing research in Canada,
http://www.crc.ca/en/html/aas/home/e...n#recent_tests
or Sandia Labs
http://www.sandia.gov/LabNews/LN04-2...tor_story.html

As to your last link, it is interesting, especially since he didn't read your link to Astending A/B is not valid

If he published a journal paper, He may be right about what he needs to consider. Certainly not at home. And certainly, the simple comparisons that audiophiles use have no place in discussions hereafter.

Boy do you have a norrow view of science. Is this the mytrcrafts rule: after 25 years of research on a subject, nothing could be left to learn, question or discover - no new protocols can be suggested or developed beyond that 25-year cutoff.

With respect to the sources your mention, none dealt with audio cables that I can see, nor, unless I missed it, do they describe their test protocols, raw data or statistical analysis in sufficient detail to allow for anyone to attempt to recreate their tests or subject them to independent peer review. But, these sources are, according to you, unassailable and not subject to question - I assume.

I guess it's about time to trot out for the 1,000,003th time Sagan's tools for critical thanking (with continuing acknowledgement to MM for first turning me on to this reference):

Tools for skeptical thinking by Carl Sagan

What skeptical thinking boils down to is the means to construct, and to understand, a reasoned argument and -- especially important -- to recognize a fallacious or fraudulent argument. The question is not whether we like the conclusion that emerges out of a train of reasoning, but whether the conclusion follows from the premise or starting point and whether that premise is true.

Among the tools:

1. Wherever possible there must be independent confirmation of the "facts."

2. Encourage substantive debate on the evidence by knowledgeable proponents of all points of view.

3. Arguments from authority carry little weight -- "authorities" have made mistakes in the past. They will do so again in the future. Perhaps a better way to say it is that in science there are no authorities; at most, there are experts.

4. Spin more than one hypothesis. If there's something to be explained, think of all the different ways in which it could be explained. Then think of tests by which you might systematically disprove each of the alternatives. What survives, the hypothesis that resists disproof in this Darwinian selection among "multiple working hypotheses," has a much better chance of being the right answer than if you had simply run with the first idea that caught your fancy.*

* NOTE: This is a problem that affects jury trials. Retrospective studies show that some jurors make up their minds very early -- perhaps during opening arguments -- and then retain the evidence that seems to support their initial impressions and reject the contrary evidence. The method of alternative working hypotheses is not running in their heads.

5. Try not to get overly attached to a hypothesis just because it's yours. It's only a way station in the pursuit of knowledge. Ask yourself why you like the idea. Compare it fairly with the alternatives. See if you can find reasons for rejecting it. If you don't, others will.

6. Quantify. If whatever it is you're explaining has some measure, some numerical quantity attached to it, you'll be much better able to discriminate among competing hypotheses. What is vague and qualitative is open to many explanations. Of course there are truths to be sought in the many qualitative issues we are obliged to confront, but finding them is more challenging.

7. If there's a chain of argument, every link in the chain must work (including the premise) -- not just most of them.

8. Occam's Razor. This convenient rule-of-thumb urges us when faced with two hypotheses that explain the data equally well to choose the simpler.

9. Always ask whether the hypothesis can be, at least in principle, falsified. Propositions that are untestable, unfalsifiable are not worth much. Consider the grand idea that our Universe and everything in it is just an elementary particle -- an electron, say -- in a much bigger Cosmos. But if we can never acquire information from outside our Universe, is not the idea incapable of disproof?

10. You must be able to check assertions out. Inveterate skeptics must be given the chance to follow your reasoning, to duplicate your experiments and see if they get the same result.

11. The reliance on carefully designed and controlled experiments is key, as I tried to stress earlier. We will not learn much from mere contemplation. It is tempting to rest content with the first candidate explanation we can think of. One is much better than none. But what happens if we can invent several? How do we decide among them? We don't. We let experiment do it.

Note in particular number 3.

The following is a post that I think raises questions that I see people like you simply gloss over and ignore and that I see seldom if ever discussed by those who choose to throw scipticism to the four winds and rely upon antecdotal reports of audio cable "DBTs" that have appeared in Stereo Review and elsewhere.

I would caution you about any short term blind A-B comparisons. They will not tell you what you have to live with in the long term. Listening fatigue can be a problem that is not identifiable in these tests because it will not be known which, or both, of the components under test is causing it. If a component causes you listening fatigue over a couple of hours and another does not, then these A-B tests are not useful to discover it unless your A-B trials are hours or even days long with hours or days of rest in between. Only long term audition will tell about this. People will not in general perform A-B tests for two hours per trial to try to determine the most fatiguing component.

As an example, I have been spending time on a new amp and I found out that it was too fatiguing to my ears and I wanted it off after only about an hour. I have spent several days and several iterations trying to tame this. I am still optimizing the amp. There are no quickie A-B tests I can rely on that gives me reliable information about long term satisfaction. I have to make changes and then live with it for a couple of days and then see if it's better or worse as a result.

One of the things not well written about is the half-life of the placebo effect. It is not infinite, but how long exactly is it? Not a perfect measure there, for sure. This means you can't be fooled forever by something that is not there. Eventually you realize the pill is not really doing any long term good when it's for a never-ending chronic condition. It may be great for a short acute condition, but not forever in a chronic condition. That's why long term listening will eventually shake out the proud papa syndrome and other such causes of a short term placebo effect. At least it works well for me, and I've had lots of practice. In both medications that I take long term and in long term audio tests, the placebo effect is way overstated in its power. Definitely the placebo effect works on me like anybody else, but its effect has a finite life span. Eventually it fades away and I have a much more "objective" subjective evaluation. In the short span of the quick A-B test, sighted tests can have a big placebo effect going, and blind tests can cause you to overlook the long-term "feel" or "wear-and-tear" of the sound, this "feel" being the very subtle aspects you normally can't hear in the short term.

If a component cannot pass a difference in a blind A-B test AND it has no long term effect, then I would make the conclusion at that point that the change just does not matter. And there have been many of those I have come across. The difference in short lengths of internal wiring between different solid copper gauges has shown little effect for me, and hardly matters, for one example. That's for my ears, at least. Others may disagree.

See: http://www.audioasylum.com/scripts/t...rophead&m=5545

This was part of a broader discussion in which many of the usual suspects paricipated:

http://www.audioasylum.com/scripts/t...rophead&m=5268

[mtrycraft]

Boy do you have a norrow view of science. Is this the mytrcrafts rule: after 25 years of research on a subject, nothing could be left to learn, question or discover - no new protocols can be suggested or developed beyond that 25-year cutoff.

With respect to the sources your mention, none dealt with audio cables that I can see, nor, unless I missed it, do they describe their test protocols, raw data or statistical analysis in sufficient detail to allow for anyone to attempt to recreate their tests or subject them to independent peer review. But, these sources are, according to you, unassailable and not subject to question - I assume.


See: http://www.audioasylum.com/scripts/t...rophead&m=5545

This was part of a broader discussion in which many of the usual suspects paricipated:

http://www.audioasylum.com/scripts/t...rophead&m=5268

You haven't shown anything new, just some old roads traveled again.

Where are the real papers supporting those speculations of that web site about only sight can be A/B ed? Interwesting discussion at AA. I guess that since he is a stat professor he cannot possibly conducted a flawed test? 6 of 6 correct?
Even the decimal series of pi has two runs of 10 odd and even digits in a row withing the 1st 1000 digits. Chance event? Oh, no, he couldn't possible have introduced administrator biasing to his wife?
Oh, yes, the othe about short term and long term listeing. Just another discussion, no evidence.
But hey, that is all we do here, discuss. No evidence for difference as ever surface.
Just show me some evidence that can be banked.

[maxg]

"The concept of randomness is a non scientific human invention. If you believe in cause and effect, in other words in a rational universe, then no event is random. Randomness expresses the human need to generalize especially a large number of events or a large number of samples, or something that occurs over a very long time. For example, we say that atoms of gas are randomly dispersed in a container but each atom or molecule was the subject of a specific history of collisions which set them on their exact course and it is in a specific location with a specific velocity at any given time because it cannot be anywhere else."

I dont think I agree with your assertion that randomness is a "non scientific human invention." A rational universe, however, is defnitely a "non scientific human invention."

From the little I think I know, of anything, the Heisenberg uncertainty principle effectively applies to exactly this issue, namely, that you cannot predict specifics relating (in his case) to Electrons and their movements. This defines randomness as the inability to predict the outcome.

[skeptic]

The highly esoteric notion of the Heisenberg uncertainty principle does not negate what I have said. For those who forgot it, or never knew it, Heisenberg created a mathematical formula to express the notion that the more you know about the position of an electron at any given time, the less you know about its velocity and visa versa. This is to say that on the one hand, if you could freeze time and say an electron is in a certain place at that time, you woudn't know anything about how fast it was going or which direction it is heading in and conversely, if you know how fast it is going and its direction, you wouldn't know where it is. This does not change the fact that it has a specific trajectory and location determined by the history of every collision it experienced since the universe was created. It is where it is and it is going where it is going because that is the only possibility for it.

The notion of randomness is that when we see a quadrillion electrons all at once and they all look exactly alike to us, and we can't tag any of them, and they all seem to be traveling in different directions and are in different places, we throw up our hands in desparation and characterize the lot of them as "random." Then we make statistical predictions about how they will behave as a group. And these generalizations often turn out to be right but it can't tell us anything about any one particular electron.

[skeptic]

ABX tests are invalid. When you test wire "A" against wire "B", you are comparing two inaccurate devices and only making a judgement about whether the difference in inaccuracies are inaudible, or what you prefer if you hear a difference at all. The only valid tests are those where you DBT wire "A" against nothing or in other words a shunt. To the degree that Wire "A" sounds different from the shunt, that is the degree of its audible inaccuracy. That is the degree to which the wire distorts the sound and fails to perform its function perfectly. You have to take into account that you are assuming that the shunt performs perfectly. In a perfect test, the test circuit for wire "A" would incorporate a duplicate identical shunt as well.

[pctower]

You haven't shown anything new, just some old roads traveled again.

Where are the real papers supporting those speculations of that web site about only sight can be A/B ed? Interwesting discussion at AA. I guess that since he is a stat professor he cannot possibly conducted a flawed test? 6 of 6 correct?
Even the decimal series of pi has two runs of 10 odd and even digits in a row withing the 1st 1000 digits. Chance event? Oh, no, he couldn't possible have introduced administrator biasing to his wife?
Oh, yes, the othe about short term and long term listeing. Just another discussion, no evidence.
But hey, that is all we do here, discuss. No evidence for difference as ever surface.
Just show me some evidence that can be banked.

"Where are the real papers supporting those speculations of that web site about only sight can be A/B ed?"

There are no "real papers" on audio cables, as far as I can tell. Thus we are all reduced to speculation.

The safe scientific position is "not proven". Beyond that we are all just discussing, as you say.

"Just show me some evidence that can be banked."

I'll leave that up to the people that are making the claims that need to be proven. I've made no such claims and I have not volunteered to serve as this board's resident geek.

[WmAx]

ABX tests are invalid. When you test wire "A" against wire "B", you are comparing two inaccurate devices and only making a judgement about whether the difference in inaccuracies are inaudible, or what you prefer if you hear a difference at all. .

IN an ABX test you do not compare A to B or vice versa. You compare A to X or B to X and decide which(A or B) sounds closest to X. A, B and X are randomly chosen in a valid ABX test. You can use a shunt in this test instead of two wires, that is, one wire and a shunt. No need to actually compare wire A to wire B.

The only valid tests are those where you DBT wire "A" against nothing or in other words a shunt. To the degree that Wire "A" sounds different from the shunt, that is the degree of its audible inaccuracy. That is the degree to which the wire distorts the sound and fails to perform its function perfectly.

If a shunt is to be used, you can use the shunt as one of the test samples in an ABX trial.

-Chris

[mtrycraft]

ABX tests are invalid. When you test wire "A" against wire "B", you are comparing two inaccurate devices and only making a judgement about whether the difference in inaccuracies are inaudible, or what you prefer if you hear a difference at all. The only valid tests are those where you DBT wire "A" against nothing or in other words a shunt. To the degree that Wire "A" sounds different from the shunt, that is the degree of its audible inaccuracy. That is the degree to which the wire distorts the sound and fails to perform its function perfectly. You have to take into account that you are assuming that the shunt performs perfectly. In a perfect test, the test circuit for wire "A" would incorporate a duplicate identical shunt as well.

Hardly invalid. Since you need to use wires except in self powered speakers. It is useless to compare one wire to no wire as you have no choice but to use a set of wires. After that, it is a matter of differences or no differences. If there are no differences, then it really matters not which wire is used.

[Pat D]

Hardly invalid. Since you need to use wires except in self powered speakers. It is useless to compare one wire to no wire as you have no choice but to use a set of wires. After that, it is a matter of differences or no differences. If there are no differences, then it really matters not which wire is used.

People are certainly throwing the terms "valid" and "invalid" around here freely, without specifying valid or invalid for what purpose!

Actually, there are amplifiers that can be mounted on a speaker and thus used with very short leads, thus almost eliminating what little effect speaker cables can have. Bryston makes their PowerPac monoblocks and the descriptions can be selected from the following list:

http://www.bryston.ca/ampsel.html

[Pat D]

Here is a quote from: http://home.xnet.com/%7Eblatura/skep_1.html#1.1

"On the other hand the theory that "there is an invisible snorg reading this over your shoulder" is not falsifiable. There is no experiment or possible evidence that could prove that invisible snorgs do not exist. So the Snorg Hypothesis is not scientific. On the other hand, the "Negative Snorg Hypothesis" (that they do not exist) is scientific. You can disprove it by catching one. Similar arguments apply to yetis, UFOs and the Loch Ness Monster. See also question 5.2 on the age of the Universe."

I would have to say that audio cables fit the Snorg Hypothesis quite well. It is interesting that it appears the negative of this hypothesis is actually the scientific one, i.e. cables do not sound different. So it looks like you have to catch one (i.e. show that cables can sound different) to disprove this Negative Snorg Hypothesis that I for one hold onto as the front-running candidate for the Truth.

I was thinking more of the fact that it was long known that the planet Uranus did not quite follow the orbit predicted by the law of gravity and the information at hand. Did the scientists conclude that the law of gravity was invalid? Did this falsify the law of gravity? No, it worked quite well in so many areas that the scientists were loath to come to such a conclusion. They decided that there must be some factor affecting the orbit of Uranus of which they were not aware. So, they started looking for another planet and eventually found Pluto. This was an ad hoc hypothesis, and as it turns out, it was a correct one.

One trouble with Snorgs are that they are undefined, and so how could one know whether one had actually found one.

[pctower]

It' not my opinion, it's a fact. Both Snorgs and audio cable differences have been claimed but not proven. Myself, being a fairly rational scientific type and the fact that I would probably discount the existance of Snorgs a lot more quickly that cable differences is my opinion but it doesn't change the fact that something doesn't exist until it is proven to exist. Snorgs and audio cable differences have not been proven to exist. That's a fact.



Ironically, I feel I am the one being "looked down" upon for my conclusions. And if I really want a good dose of that I just have to ask a simple question over at the Cable Asylum.

I think the main point I am always trying to make is that given what we know today, it would be impossible for somebody to decisively conclude that similar cables can sound different. Yet, that is precisely what some people do. That puts them open to challenges and that is why we are here today.

Thank you, you've been a very kind audience. I'll be here all week....

And if I really want a good dose of that I just have to ask a simple question over at the Cable Asylum.

I haven't signed on as an apologist for the Cable Asylum, or for AA.

[skeptic]

I feel like Diogenes, the Greek philosopher who went out into the forest with a lantern in daylight looking for an honest man. Had he come to New York City, in five minutes they would have stolen his lantern.

Looking for an honest answer from people who make and sell audiophile cables with a lantern of knowledge, going into an audio store makes me feel like they are trying to steal my lantern too. It won't work. You can't forget what you actually know, especially when the lying comes from people who do not earn your respect.

[Chuck]

I honestly don't understand your reply.
As of today, cable differences are like the invisible Snorg. Some people claim to have seen him or felt him or are sure he has been in the room with them, but nobody can prove it.

Hi Mike,

This is my first visit to AR in six months or more; What I can't understand is why so many people are wasting time arguing with other people who think they have invisible Snorg's reading over their shoulder. Rational men don't believe in Snorg's, so you know going in that you will be arguing with people who are irrational, and apparently delusional. Why not just treat them the same way you do the other groups who imagin things (like having been abducted by UFO's). I think it is highly discriminatory to spend all your time picking on the people who believe in the Snorg. The world is full of people who have delusions, so why waste months and years picking on the Snorgophiles? That just doesn't seem fair. What makes Snorgs so special?

Guess I’ll stop back in another six months or so, and see if anyone has learned anything. Seems that the last six months passed without any progress, since absolutely nothing has changed. What a complete waste of time:

[Monstrous Mike]

One trouble with Snorgs are that they are undefined, and so how could one know whether one had actually found one.

You wouldn't. You would first have to find one and then go from there to see what you have. But I still hold this analogy is true with cable sonic differences. You would first have to find a difference, and then explore the nature and cause of the difference. But right now, Snorgs and cable differences are both claimed to exist but we really don't have proof which makes it impossible to speculate on further information about either.

[okiemax]

You wouldn't. You would first have to find one and then go from there to see what you have. But I still hold this analogy is true with cable sonic differences. You would first have to find a difference, and then explore the nature and cause of the difference. But right now, Snorgs and cable differences are both claimed to exist but we really don't have proof which makes it impossible to speculate on further information about either.

It looks to me like the primary purpose of the Snorg analogy is to disparage people who claim to hear a difference in cables rather than show how you can't prove a negative. You could substitute "God" for "Snorg," and the analogy would be as accurate, but that wouldn't belittle cable yeasayers, and your purpose wouldn't be served. Why beat around the bush? If you believe anyone claiming to hear a difference in cables is imagining the difference, why not just say so.

Why does the absence of positive results "make it impossible to speculate on further information? I can understand questioning whether such an effort holds promise, but I don't see it as being "impossible." And wouldn't the possiblity of insufficient testing and/or invalid testing methodology make further investigation worthwhile?

[Monstrous Mike]

Hi Mike,

This is my first visit to AR in six months or more; What I can't understand is why so many people are wasting time arguing with other people who think they have invisible Snorg's reading over their shoulder. Rational men don't believe in Snorg's, so you know going in that you will be arguing with people who are irrational, and apparently delusional. Why not just treat them the same way you do the other groups who imagin things (like having been abducted by UFO's). I think it is highly discriminatory to spend all your time picking on the people who believe in the Snorg. The world is full of people who have delusions, so why waste months and years picking on the Snorgophiles? That just doesn't seem fair. What makes Snorgs so special?

Guess I’ll stop back in another six months or so, and see if anyone has learned anything. Seems that the last six months passed without any progress, since absolutely nothing has changed. What a complete waste of time:

This is the best I can come up with.

Personally, I have always had a scientific mind. I like to find out how things work, why they work, build better ones, etc. This involves the pursuit of logic and rationality as well as the laws of science. It is about structure and discipline.

I do have an artistic and philosophical side as well, although I would not consider it nearly as developed as my analytical skills. I find these types of discussions representative of the artistic and romantic versus the scientific and rational. It may appear that we get nowhere and that is likely true when the same arguements come up over and over, yet there is usually a new twist or angle that somebody presents every once and a while. Every point of view can only enhance the discussion.

So one of my purposes here is to experience a side of life that I feel may be lacking for me. The points of view of non-scientifically inclined people are valuable to me. It helps me break away from structure and get "outside the box" for a while

It is not worthless to me. As an example, from what I have learned, it is not worthless to buy an expensive power cord or speaker cable either. Even if the sound reaching the sensor known as the ear is very close to what it was before, the experience of the sound may be different and for the better and that is not worthless.

The mood of the discussions lately here has been very positive. Disagreement does not equal a negative but rather fosters a greater likelyhood of understanding each other. Further, this sort of philosophical concept can apply to a lot more than audio cables. And perhaps it is better to garner this mutual understanding and appreciation under the auspices of audio cables which is a generally benign topic unlikely to drive people to their gun cabinets.

Have a great day...and keep in mind it's about more than audio cables.

P.S. One used soapbox for sale.