Proof of placebo effect ?

[okiemax]

Naysayers argue there is no scientific basis for claims of audible differences in cables, and that listeners who make such claims are experiencing a placebo effect rather than hearing real differences. However, there may be no scientific basis for naysayer's claims about the placebo effect. Can anyone offer proof?

[Rockwell]

Naysayers argue there is no scientific basis for claims of audible differences in cables, and that listeners who make such claims are experiencing a placebo effect rather than hearing real differences. However, there may be no scientific basis for naysayer's claims about the placebo effect. Can anyone offer proof?

\

Proof of the placebo effect? If they aren't hearing real differences, then what are they hearing? People haven't proven that they can hear differences between brands of cables, so one might assume that these percieved audible effects are the result of psychological factors. To some people this is very difficult to assume, and try to attribute this lack of proof to any number of other factors like testing methodolgy, hardware inadequecy, hearing, etc. Most scientific types, however, must assume that people can't hear differences between most cables until its demonstrated. That is the proof you should be focusing on.

Lets say that you devise a listenning test where you tell participants that fancy cable A will play first, and low-grade cable B will play next, and note what they are hearing. But, instead of switching you just leave the same cable in place. You will certainly get positive results from this test with few of the participants stating that the cables are the same. If this isn't "proof," I don't know what you would be willing to accept.


Why is it so difficult to accept the placebo effect? If you look a the overally accuracy of human perception, you will find that it is very easy to fool the senses and for people to fool themselves. Video and audio compression schemes are based heaviliy on fooling your ears and eyes. In some cases 90% of the information is removed, but you can hardly notice. Placebo in medicine can actually produce physical changes in people. Eyewitnesses notoriously inacurrate. People can actually convince themselves that something happened, when it really didn't(false memory syndrome). Your eyes have blindspots, but you don't see them because you brain fills in that missing information without you even knowing it. So, it's not that far fetched to me that people can hear all these different attributes in cables where none exist.

[okiemax]

Rockwell, you are offering a theory as proof. It is an interesting theory, but we also might consider other theories, such as those that question the testing. I'm not a big fan of the placebo effect in general, believing that far too much is attributed to it both in medicine and audio.

[Rockwell]

Rockwell, you are offering a theory as proof. It is an interesting theory, but we also might consider other theories, such as those that question the testing. I'm not a big fan of the placebo effect in general, believing that far too much is attributed to it both in medicine and audio.

I am not so sure that placebo effect really needs to be proven and I don't think I was trying to really offer proof either but, that example test seems pretty clear cut. If you change none of the variables except what wire you tell the listeners is connected, then whatever positive results are offered must be attributed to what the listener expects from the respective cable, not the sound. This would establish that the placebo effect is real and how easy it is to hear results when none exist. Do you agree?

[okiemax]

I am not so sure that placebo effect really needs to be proven and I don't think I was trying to really offer proof either but, that example test seems pretty clear cut. If you change none of the variables except what wire you tell the listeners is connected, then whatever positive results are offered must be attributed to what the listener expects from the respective cable, not the sound. This would establish that the placebo effect is real and how easy it is to hear results when none exist. Do you agree?

I don't know of a test where no variable except the two wires has changed, but suppose for the sake of argument you could do such a test, and the listener who previously claimed he could hear a difference could not do so in the test. The result would suggest the placebo effect is a possibility.

[woodman]

[QUOTE=okiemax]Rockwell, you are offering a theory as proof. It is an interesting theory, but we also might consider other theories, such as those that question the testing.[/QUOTE

What Rockwell offers is as much "proof" as should be necessary for anyone that's not been terminally infected with the audiophile virus!

I'm not a big fan of the placebo effect in general, believing that far too much is attributed to it both in medicine and audio.

Tell me (if you can) how you would explain the mind-boggling demonstrations performed by stage-hypnotists who tell their subjects how delicious the chocolate cake they've given them tastes, when in fact what they've actually given them is a piece of lemon? Or, when the subject is told that everyone in the audience is very sad and crying, the subject joins in and cries (real tears) right along with them, when in fact everyone in the audience is actuall laughing their asses off? Need I go on? Doesn't the hypnotized subject HEAR the laughter? Really?

If you aren't willing to consider the "placebo" effects (which I call ABEs)as they relate to audio, you have my deepest sympathy, for you are the willing victim of all of the scam artists that this beloved hobby is chock-full of, and it's costing you dearly for your stubbornness!

[okiemax]

Tell me (if you can) how you would explain the mind-boggling demonstrations performed by stage-hypnotists who tell their subjects how delicious the chocolate cake they've given them tastes, when in fact what they've actually given them is a piece of lemon? Or, when the subject is told that everyone in the audience is very sad and crying, the subject joins in and cries (real tears) right along with them, when in fact everyone in the audience is actuall laughing their asses off? Need I go on? Doesn't the hypnotized subject HEAR the laughter? Really?

If you aren't willing to consider the "placebo" effects (which I call ABEs)as they relate to audio, you have my deepest sympathy, for you are the willing victim of all of the scam artists that this beloved hobby is chock-full of, and it's costing you dearly for your stubbornness!

I never said I wasn't willing to consider that there is such a thing as the placebo effect in audio. It is theory that lacks proof. Despite all the research that has been done in medicine on the subject of the placebo, there is still controversy over it. Are you saying there shouldn't be controversy over it in audio.

I guess there is some relationship between hypnosis and placebo in that both have to do with suggestion, but I don't think you help your case by the examples you use. Demonstrations of hypnosis are easy to fake.

Mmm...this lemon taste like chocolate cake. Can you say S-H-I-L-L ?

On second thought, perhaps there is more to this hypnosis thing than I want to admit. The pretty pictures in those Stereophile mag ads may present subliminal messages: BUY OUR EXPENSIVE CABLES. BUY OUR EXPENSIVE CABLES. BUY OUR EXPENSIVE CABLES.

But I'm a chepskate, so I bought a $42 interconnect instead of one of the $500 advertised models. I didn't like this cable, so I returned it. Now, I think I see the problem. Buy a cheap cable, get a cheap placebo. If you want a REAL PLACEBO, you got to shell out big.

Just kidding ! Hope you have happy holidays!

[DrJeff]

Naysayers argue there is no scientific basis for claims of audible differences in cables, and that listeners who make such claims are experiencing a placebo effect rather than hearing real differences. However, there may be no scientific basis for naysayer's claims about the placebo effect. Can anyone offer proof?

There is a huge volume of information available concerning the placebo effect, dating back to the 1950's (Beecher). A search of any medical or psychiatric database will cite numerous studies concerning the placebo effect, as used in medicine. Doctors employ placebo effect when they prescribe antibiotics to treat a cold or flu, which is a viral infection and therefore will not repond to antibiotics. Recently, there has been much debate about whether placebo effect really works in a medical setting. Holistic and mind-body healers say that 1/3 of those treated with a placebo respond to treatment, while some recent studies show a correllation that is much lower.

As an example, see: http://www.annals.org/cgi/content/full/136/6/471

From the Annals of Internal Medicine, "Deconstructing the Placebo Effect and Finding the Meaning Response" by Daniel E. Moerman, PhD and Wayne B. Jonas, MD , 19 March 2002, Volume 136, Issue 6, Pages 471-476.

HOWEVER, "placebo effect" as it applies to audio is a bit of a misnomer. In placebo effect, a patient is given what he believes to be a treatment that will work, and if his condition improves, it is attributed to placebo, and is often cited as the mind's power to heal. There is no doubt that the mind plays a large role in the health of the body, although the direct placebo effect is hotly debated.

In audio, what is mistakenly called placebo effect is more appropriately described as a prejudice or predetermination of expected results. In anything but a DBT, you allow your preconceived notions, prejudices, and assumptions to cloud your judgement. If you expect a $1000 cable to sound better than a RatShack Gold, then it most likely will, WHEN YOU KNOW IT IS CONNECTED. Conversely, when you don't know which cable is connected and must decide which you like better, you are relying ONLY on your ears, and therefore your jujdgement is not biased. If there is truly an AUDIBLE difference in the quality of the sound due to the interconnect, and with all other factors being equal (system, db level, positioning, etc.) then you should be able to consistently identify the better-sounding interconnect.

As in any debate, the burden of proof that something exists lies with the person making the claim. In this case, you say, "prove the placebo effect exists" and I say to you there are many scientific studies that show it exists in a medical capacity, although the extent to which it exists is questionable. But I do not have to prove it exists because I am not asserting that it plays a role in your audio experience. What I am saying is that I question your ability to percieve audible differences in interconnects (as long as they are properly functioning) and so you must prove to me that you can, without knowing which one is connected when you are listening. I am not asking you to measure anything or use a scope, just listen.

In an audio setting, if you claim interconnect A sounds "better" than B, I will say to you, prove it. The only way you can do this is by telling me when it is connected on the basis of your auditory experience only. I hook them up, you listen, you tell me which sounds better, and do it 7 out of 10 times on average, and you have proven that there is an audible difference in the IC's, and that you can identify one as sounding "better" than the other (although your idea of better and mine might be different). Otherwise, any claims you make when you know it is connected are invalid.

[okiemax]

I see similarities in placebo theory whether we are discussing it in audio or in medicine, but agree there are differences. But the term is convenient to use and there seems to be general agreement on what it means, so we probably are stuck with it.

I guess I'm just tired of hearing "placebo effect" every time I claim to hear a difference in two cables, two amps, or two of anything else. If I question this knee-jerk explanation, naysayers counter with the medical science card. So it was refreshing for me to see that some researchers have found the placebo effect may have no scientific basis (see "Is the Placebo Powerless? -- An Analysis of Clinical Trials Comparing Placebo With No Treatment," New England Journal of Medicine, May 24, 2001). The study found that subjects receiving placebo treatment fared about the same as subjects receiving no treatment. Apparently, a proportion of ill people will just get better on their own, sugar pill or not.

http://content.nejm.org/cgi/content/...ct/344/21/1594

I agree you can question any claim I make about hearing differences in cables without mentioning even the possibility of the placebo effect. But naysayers on this forum frequently do talk about as if it were fact. When I have asked for details on the placebo effect, I have been told it can be short-term or long-term, and maybe intermittent. But the effect can't be cummulative(additional pleasure with each new cable). Why? Because this would make the whole idea seem absurd.

In your comments on burden of proof, you seem to be implying that a claim of detecting a difference with the senses incurrs an obligation to prove the difference. If you prefer Coke to Pepsi, should I ask you to prove you can tell the drinks apart in a blinded test? Well, regardless of your answer, I can claim to hear differences in cables without feeling obligated to offer proof with blinded testing. You can believe the differences are real or imaginary. If you tell another person that what I claim is imaginary, however, you are going beyond what you know. And that frequently is what I witness naysayers doing in this forum.

[DrJeff]

....If you prefer Coke to Pepsi, should I ask you to prove you can tell the drinks apart in a blinded test? Well, regardless of your answer, I can claim to hear differences in cables without feeling obligated to offer proof with blinded testing. You can believe the differences are real or imaginary. If you tell another person that what I claim is imaginary, however, you are going beyond what you know. And that frequently is what I witness naysayers doing in this forum.

If you say you prefer Coke to Pepsi, then absolutely you should be able to identify them in a DBT, IF you prefer it because of the taste. Since Coke is one of my clients, I ALWAYS prefer Coke, even if it's because they pay part of my salary. In that case, it may not be the taste of the product, but a larger issue that steers me toward Coke (loyalty to my client).

The Coke analogy is actually very applicable to the cable issue. If you claim to be able to taste a difference in Coke vs. Pepsi, but cannot identify a difference in a DBT (i.e. you don't have to say "this is Coke" or "this is Pepsi" but you do have to say if both cups contain Coke, or if they are different) then I say you can not taste a difference, and your percieved ability to tell a difference is based on more than just taste.

So if you say you can hear a difference in cables, but cannot prove by listening alone in a DBT that you hear a difference, then your claim is based on things other than the audio signal, just as in the Coke analogy.

[skeptic]

The Coke analogy is actually very applicable to the cable issue.

I can't agree.

The purpose of Coke and Pepsi is almost purely for sensory stimulation in preference to far more effective thirst quenchers such as water.

Cables perform a physical function which is entirely objective and can be precisely measured if anyone cared to. That is the transmission of an electrical signal from one point to another in an electrical system or network with the least distortion. Whether the results of one individual's ability to distinguish the effectiveness of one cable versus another in performing this function unless he can identify which is the less distorted and if he prefers one over another because of subjective reaction is not relevant. Unless you can argue that the shortcomings of one element in the network will predictably offset or negate the shortcomings of another such as frequency response distortion where a cable of high shunt capacitance for example would offset the high frequency peak of an overly bright loudspeaker or a phonograph cartridge with a resonant peak. But how could this be predicted by anyone but the most extraordinarily trained and informed user and what other remedies are there that would be far more effective, predictable in their actions and cost effective? Reducing the arguement over cables to a matter of personal preference disregarding all measurements and objective tests is to turn your back on all of the science that has been used to advance audio technology in the first place.

[DrJeff]

I can't agree.

The purpose of Coke and Pepsi is almost purely for sensory stimulation in preference to far more effective thirst quenchers such as water.

Cables perform a physical function which is entirely objective and can be precisely measured if anyone cared to. That is the transmission of an electrical signal from one point to another in an electrical system or network with the least distortion. Whether the results of one individual's ability to distinguish the effectiveness of one cable versus another in performing this function unless he can identify which is the less distorted and if he prefers one over another because of subjective reaction is not relevant. Unless you can argue that the shortcomings of one element in the network will predictably offset or negate the shortcomings of another such as frequency response distortion where a cable of high shunt capacitance for example would offset the high frequency peak of an overly bright loudspeaker or a phonograph cartridge with a resonant peak. But how could this be predicted by anyone but the most extraordinarily trained and informed user and what other remedies are there that would be far more effective, predictable in their actions and cost effective? Reducing the arguement over cables to a matter of personal preference disregarding all measurements and objective tests is to turn your back on all of the science that has been used to advance audio technology in the first place.

See, now you're opening a can of worms, I had purposely avoided discussing the physical and electrical properties of the cables as they relate to the sound quality.

I agree that you should be able to look at inductance, capacitance, run square and sine waves through the cable and look at them when they come out the other side, and evaluate cables on the basis of their ability to transfer a signal without changing its nature.

However, many cable enthusiasts will claim that some differences are not measurable, and they claim that scientific analysis gets in the way of the listening experience, and that scopes and multimeters won't see the whole picture. For example, I have ohms, ACV, DCV, current, etc. settings on my scope, but there's no "open soundstage" scale, so I can't measure that.

So if I avoid the technical side, and just ask someone to prove (using their ears) in a DBT that they can distinguish between cables, components, etc., there's no need to have the argument of science vs. humanity in the cable debate. Had I talked about that side, someone would have said, "But your scope doesn't feel the music" or something like that.

Personally, I look for an IC with a nice big plug that will stay put, good insulation, and a jacket that will resist chafing. I think I spent about $15 a pair at most on my IC's, and they've worked for years with no complaints.

[mtrycraft]

Why would there be differences between no treatment and a placebo pill in the first place? a placebo pill has nothing in it to start with. In essence it is also a no treatment pill?

I am not sure what you are asserting here?

Perhaps you didn't read the research right or misunderstanding it and drawing the wrong conclusion from it.

A placebo has no real value other than a control in a DBT.
After all, these protocols are only valid for DBt protocol to account for bias, a placebo in effect.
That is what is the case in audio, the need for DBT to account for bias.

Perhaps you need to read some of these:

"Listening Tests, Turning Opinions Into Facts", Toole, F. E., Journal of the Audio Engineering Society, Vol 30, No.6, Jun 1982, pg 431-445.

"Subjective Measurements of Loudspeaker Sound Quality and Listening Preference", Toole, F. E., Journal of the Audio Engineering Society, Vol 33, No 1/2, Jan/Feb 1985, pg 2-32.

"Loudspeaker Measurements and Their Relationship to Listening Preferences", Toole, F. E., Part 1, Journal of the Audio Engineering Society, Vol 34, No.4, Apr 1986, pg 227-235; Part two, JAES Vol 34, No.5, May 1986, pg 323-348.

"Listening Tests-Identifying and Controlling the Variables", Toole, F. E., Proceedings of the 8th International Conference, AES, May 1990.

"Hearing is Believeing vs Believing is Hearing: Blind vs Sighted Listening Tests ond Other Interesting Things", Toole, F. E. and Olive, S. E., 97th AES Convention, Nov 1994, Print #3894.

[okiemax]

Why would there be differences between no treatment and a placebo pill in the first place? a placebo pill has nothing in it to start with. In essence it is also a no treatment pill?

I am not sure what you are asserting here?

Perhaps you didn't read the research right or misunderstanding it and drawing the wrong conclusion from it.

A placebo has no real value other than a control in a DBT.
After all, these protocols are only valid for DBt protocol to account for bias, a placebo in effect.
That is what is the case in audio, the need for DBT to account for bias.

Perhaps you need to read some of these:

"Listening Tests, Turning Opinions Into Facts", Toole, F. E., Journal of the Audio Engineering Society, Vol 30, No.6, Jun 1982, pg 431-445.

"Subjective Measurements of Loudspeaker Sound Quality and Listening Preference", Toole, F. E., Journal of the Audio Engineering Society, Vol 33, No 1/2, Jan/Feb 1985, pg 2-32.

"Loudspeaker Measurements and Their Relationship to Listening Preferences", Toole, F. E., Part 1, Journal of the Audio Engineering Society, Vol 34, No.4, Apr 1986, pg 227-235; Part two, JAES Vol 34, No.5, May 1986, pg 323-348.

"Listening Tests-Identifying and Controlling the Variables", Toole, F. E., Proceedings of the 8th International Conference, AES, May 1990.

"Hearing is Believeing vs Believing is Hearing: Blind vs Sighted Listening Tests ond Other Interesting Things", Toole, F. E. and Olive, S. E., 97th AES Convention, Nov 1994, Print #3894.

Mtycraft, you seem to be arguing against the placebo theory instead of for it. Referring to the NEJM study, you said," Why would there be a difference between no treatment and a placebo pill in the first place?" I thought you believed a placebo could be a powerful influence on the mind, so powerful as to make an ill person feel healed or a listener hear things that aren't real.

In the study, if the placebo had power, improvement would have been greater in subjects who received the placebo than in those who recieved no treatment. But the study showed improvement was about the same for the two groups. So if the placebo (or power of suggestion) in a medical setting really isn't so powerful, why not question theories about it in other settings?

I don't have any of the referenced papers by Dr. Floyd Toole so I can't comment on them. My guess is they say something to support your believe that audiophile cables are no better than lamp cord and that people can't trust their ears. If that is the case, I wonder why Dr. Toole's employer (assuming he is still with Harman International) has this to say in the owner's manual for their JBL Tik Series speakers:

"Careful selection of of cables and interconnects can have quite a dramatic impact on the dynamic contrasts experienced by listeners."

If you are interested in the manual, do a Google search on the following term: jbl home support. Then choose the Ti10K from the product list -- it's almost at the bottom.

There also is an interesting interview of Dr. Floyd Toole by Melanie Garrett of of HOME CINEMA online. The following quote is from it: "Floyd is of the opinion that technical measurements are not much use unless they can reliably predict what we as listeners will subjectively experience as good or bad sound quality. In a nutshell, he not only has faith in his own ears, but he is also interested in yours and mine as well ..."

You can get to the interview through Google by entering the following term: home cinema online garrett toole.

[Swerd]

Thanks to the two replies above, I have little to add other than provide these weblinks:

http://www.tagmclaren.com/members/news/news77.asp

http://www.bostonaudiosociety.org/ba...l_thinking.htm

Both describe listening tests where listeners blindly choose whether they hear a difference between two conditions. Are conditions A and B the same or different? Sometimes A and B were genuinely different and sometimes they were identical. The first link explains the statistics needed to analyze the results.

Both make the point that when the group of listeners get it right about 50% of the time, it is no different than if they had been guessing randomly. Both conclude that the listeners could not hear any difference between the conditions being tested.

There are those who claim that they do hear a difference with sighted listening tests, or when they listen over extended periods of time (days, weeks or longer) in their homes. The only reasonable explanation available, is that they THINK they hear a difference. This is the placebo effect at work.

Listening tests performed using valid scientific controls that eliminate, minimize, or account for conscious or unconscious listener bias cannot reproduce those positive results.

[pctower]

Question:

In the Tag McLaren tests, two participants scored 11 out of 15 correct on the cable tests. As they point out, the binomial distribution indicates a 5.9% (about 1 in 17) probability of scoring 11 or better by chance.

Why would one not conclude that in all liklihood those two individuals heard true differences in the cables?

[DrJeff]

Question:

In the Tag McLaren tests, two participants scored 11 out of 15 correct on the cable tests. As they point out, the binomial distribution indicates a 5.9% (about 1 in 17) probability of scoring 11 or better by chance.

Why would one not conclude that in all liklihood those two individuals heard true differences in the cables?

You need to understand the science of statistics to understand this. Two people scored 11 out of 15, and there is a 1 in 17 probability they would do this by chance. Therefore, you might say that if there were only 34 people involved in the test, there would be 2 that would guess 11 of 15 correct. But there are only 12 people in this test, so shouldn't there have been only been 0.71 people scoring 11 of 15?

Look at it this way, if I give you a quarter and tell you to flip it 4 times, statistics would seem to dictate that you'll get 2 heads, and 2 tails. In reality though, you are *almost* just as likely to get heads 4 times or tails 2 times. But if I tell you to flip it 10,000 times, you will get alsmost the same number of heads and tails, because your sample group is much larger, and therefore more valid. The Tag McLaren test group is not large enough to be statistically valid.

You have to look at the distribution of the results with respect to statistical analysis. However, I would have taken these 2 people and repeated the test to see if they got similar results the second time around. Then I might attribute that to them being able to hear an audible difference. Otherwise, I wouold have to discount it as an anomoly due to the small number of subjects.

I would also question whether you could gain an accurate representative group from 12 people...

[Swerd]

I would also question whether you could gain an accurate representative group from 12 people...

As a follow-up to this point, clinical trials of new drugs are done in stages involving progressively larger numbers of patients. For example, if laboratory studies suggest that a new drug may work against leukemia, a so-called phase 2 clinical trial is done with a small number (20 to 30) of leukemia patients. If the trial has positive results, this number of patients is large enough only to provide statistical data that SUGGEST the drug may be active. No data from numbers of patients this low can demonstrate any positive result in a statistically significant manner.

With positive phase 2 results, several phase 3 trials would then be started involving many hundreds of patients, in order to generate statistical analysis with real power. It has happened more than once where a drug had some positive phase 2 results, and failed completely when the larger phase 3 trials were done.

To go back to the McLaren study, there was no statistical trend or suggestion in the data from 12 listeners that would warrant a larger study which could provide numbers large enough for definitive conclusions.

[pctower]

You need to understand the science of statistics to understand this. Two people scored 11 out of 15, and there is a 1 in 17 probability they would do this by chance. Therefore, you might say that if there were only 34 people involved in the test, there would be 2 that would guess 11 of 15 correct. But there are only 12 people in this test, so shouldn't there have been only been 0.71 people scoring 11 of 15?

Look at it this way, if I give you a quarter and tell you to flip it 4 times, statistics would seem to dictate that you'll get 2 heads, and 2 tails. In reality though, you are *almost* just as likely to get heads 4 times or tails 2 times. But if I tell you to flip it 10,000 times, you will get alsmost the same number of heads and tails, because your sample group is much larger, and therefore more valid. The Tag McLaren test group is not large enough to be statistically valid.

You have to look at the distribution of the results with respect to statistical analysis. However, I would have taken these 2 people and repeated the test to see if they got similar results the second time around. Then I might attribute that to them being able to hear an audible difference. Otherwise, I wouold have to discount it as an anomoly due to the small number of subjects.

I would also question whether you could gain an accurate representative group from 12 people...

Seems to me that by adopting the coin toss analogy to this situation you are assuming as a given that the hearing ability, training and experience of all the participants are equal.

I don't know statistics, but I know enough to recognize the difference.

As for medical trials with large number of participants and trials, they are attempting to determine the effectiveness of a particular medicine on a large enough segment of the populace as to justify its use. Whether it works or not (it either does or doesn't) on a single individual within the broad group is meaningless, as they are concerned with percentage of effectiveness as to the entire group. They have to do it this way, as they can never be sure that a test subject who received real medicine and appears to have been cured was cured as a result of the medicine or a result of the placebo effect. They can only compare the overall results of the non-placebo group with that of the placebo group

I submit that in audio DBTs you don't have that situation. If the test is conducted properly, the pacebo effect, or the power of suggestion, or the ABE's or whatever you want to call influences that are not actual audible differences, should all be controlled out of the test, with the only thing left to test being actual audible differences. If one person can identify correctly 11 out of 15 times, there's no possible chance that placebo or ABEs could be at work, again assuming the test is properly conducted, so his correct guesses either have to be the result of chance (a very low possibility) or due to actual sonic differences he was able to detect; whereas, as I said, if the medicine seems to work on a single individual there still is no way of determinining if it was due to placebo or the real thing. This is a significant difference between audio DBTs and medical DBTs.

In the McLaren test, it doesn't make sense to me that they would want to know how a large group in general would perform, as each person is different with different capabilities. Seems to me that the important question is whether there are people who hear and can detect under blind conditions actual differences. In this case, there apparently were two.

Let's approach it this way. Let's say that out of 100 test subjects, only two hit the 11 out of 15, and the overall correct results of the group was only 50%.
Seems to me that could be significant data to the cable company whose cables are under test. If the test had been set up so that the distribution of hearing, experience and training adequately represented the general population, then such results might suggest to them that only 2% of the population could hear actual differences. If they were intent on marketing their product only to people who could actually hear the differences (remember, this is just a hypothetical for purpose of illustration and I'm trying to anaolgize to the medical situation), then they would have to decide if manufacturing the product that had been tested would be worthwhile if it could only be marketed to 2% of the general population.

If a medical test with 100 subject (I know - the numbers used are far larger in actual use and that improves the dependability of the results significantly, but I'm trying to keep this as simple as I can) showed that the medicine only worked on 2% of the general population (by comparing the non-placebo group to the placebo group and extrapolating that percentage number - which is the way I assume they do it, which is much more involved that the simple audio DBT test where the placebo can be controlled out at to every test subject), the company and FDA would face the same general question as the cable company. However, their decision might be quite different than the cable company, because the cable company might well decide that they can identify and focus their advertising to that 2% sufficiently to make it worthwhile to market their products. I assume drug companies can do that, although I'll ask my daughter-in-law who is a rapidly rising star with one of the biggies.

[DrJeff]

Seems to me that by adopting the coin toss analogy to this situation you are assuming as a given that the hearing ability, training and experience of all the participants are equal.

Perhaps so, but let's assume for the sake of your argument that we will use only those who describe themselves as "audiophiles" in our next hypothetical test.

I don't know statistics, but I know enough to recognize the difference.

As for medical trials with large number of participants and trials, they are attempting to determine the effectiveness of a particular medicine on a large enough segment of the populace as to justify its use. Whether it works or not (it either does or doesn't) on a single individual within the broad group is meaningless, as they are concerned with percentage of effectiveness as to the entire group. They have to do it this way, as they can never be sure that a test subject who received real medicine and appears to have been cured was cured as a result of the medicine or a result of the placebo effect. They can only compare the overall results of the non-placebo group with that of the placebo group.

I agree with that statement entirely.

I submit that in audio DBTs you don't have that situation. If the test is conducted properly, the pacebo effect, or the power of suggestion, or the ABE's or whatever you want to call influences that are not actual audible differences, should all be controlled out of the test, with the only thing left to test being actual audible differences. If one person can identify correctly 11 out of 15 times, there's no possible chance that placebo or ABEs could be at work, again assuming the test is properly conducted, so his correct guesses either have to be the result of chance (a very low possibility) or due to actual sonic differences he was able to detect; whereas, as I said, if the medicine seems to work on a single individual there still is no way of determinining if it was due to placebo or the real thing. This is a significant difference between audio DBTs and medical DBTs.

I agree that you can have a DBT test that very effectively minimizes placebo effect and other factors, leaving only one variable for the subject to judge. However, I think that a person guessing 11 out of 15 ONCE is just as likely as any other scenario. In irder to make the test valid, you would want to perform multiple runs of the same test. For example, can this person get it right 11 out of 15 times per run, for 5 consecutive runs. Of course the cable selection would be random so no pattern existed in switching. If his correct guesses for all 5 runs were in the 10 to 15 range, then I am convinced he can perceive a diffeence. However, if his guesses came up something like 11, 4, 10, 7, 5 then I would have to say he cannot hear a difference.

It can get very confusing, so I would leave it to people with more time and desire, but I believe a proper test could be conducted, I just haven't seen it yet.....

[Swerd]

pctower

Your reasoning that medical trials and listening tests are different is faulty. You are correct that medical trials do attempt to determine the effectiveness of a particular medicine on a large enough segment of the populace because there is significant variation among individuals in the population. The same problem exits for audio listening tests, precisely because hearing ability, training and experience of all the participants are NOT equal. It is not possible to perform a test where all non-audible differences are, as you describe, “controlled out of the test, with the only thing left to test being actual audible differences.” If it were, we wouldn’t need to use statistics and we wouldn’t be having this discussion.

As you point out from the McLaren test, 2 out of 12 people scored as if they were able to detect differences between cables. Twelve is such a small sample number that making any positive conclusion from those numbers would be wrong. To verify that finding, it would be necessary to test large numbers of people to eliminate a statistically defined problem called sampling error. Does the original test of 12 people truly represent the larger population as a whole? Does their hearing ability, prior training, and experience resemble that of the general population? That larger number people, off the top of my head, would be closer to 1000 than it would be to 100 or less. To put it simply, if we tested 1200 people, would 200 of them be able to detect the differences between cables? If this were the case, then I would support the conclusion that some people, about 1 out of 6, could hear differences in cables. It would then be up to the scientists, both biological and electrical to search for how this happens. At present, no listening test has provided such evidence.

[Norm Strong]

There is also a 5.9% chance that a person could be WRONG 11 out of 15 times. What would you say if that were the case? (For all I know it is the case. :-)

[pctower]

There is also a 5.9% chance that a person could be WRONG 11 out of 15 times. What would you say if that were the case? (For all I know it is the case. :-)

I would say what I said in my other responses in this sub-thread.

[Monstrous Mike]

Question:

In the Tag McLaren tests, two participants scored 11 out of 15 correct on the cable tests. As they point out, the binomial distribution indicates a 5.9% (about 1 in 17) probability of scoring 11 or better by chance.

Why would one not conclude that in all liklihood those two individuals heard true differences in the cables?

Umm, because they hit that 1 in 17 chance thingy?

Surely you know that if you flip a coin long enough you are going to get ten heads in a row. Does this mean the coin is imbalanced? Hey, I hit 20 free throws in a row once. Does this mean I have a good shot? No, it means that some days, the sun even shines on a dog's ass.

You can't isolate test results to support your theories. You wouldn't let me do it and I won't let you do it.

[pctower]

Umm, because they hit that 1 in 17 chance thingy?

Surely you know that if you flip a coin long enough you are going to get ten heads in a row. Does this mean the coin is imbalanced? Hey, I hit 20 free throws in a row once. Does this mean I have a good shot? No, it means that some days, the sun even shines on a dog's ass.

You can't isolate test results to support your theories. You wouldn't let me do it and I won't let you do it.

I've already addressed the coin flip analogy, which is not appropriate. I'm surprised you don't see the difference. Or did you just think I would be too stupid not to see it.

That analogy assumes all listeners are identical as to hearing ability, experience and training (remember, your god, Dr. Toole, say training of the listeners is very important). Every flip of the coin is equivalent. But each listener is a unique "packet" and different from all the other "packets". In a coin toss I can pick any interval of 15 tosses I want in order to come up with the particular ratio out of 15 tosses I'm looking for. Can't do that with the individual "packets".

I don't claim the McLaren tests show anything reliable because of the two subjects that hit 11 or 15. However, if they came back 4 or 5 times and repeated that performance, only a few people such as yourself would find some weird way of ignoring those results.