Proof of placebo effect ?

[pctower]

You need to understand the science of statistics to understand this. Two people scored 11 out of 15, and there is a 1 in 17 probability they would do this by chance. Therefore, you might say that if there were only 34 people involved in the test, there would be 2 that would guess 11 of 15 correct. But there are only 12 people in this test, so shouldn't there have been only been 0.71 people scoring 11 of 15?

Look at it this way, if I give you a quarter and tell you to flip it 4 times, statistics would seem to dictate that you'll get 2 heads, and 2 tails. In reality though, you are *almost* just as likely to get heads 4 times or tails 2 times. But if I tell you to flip it 10,000 times, you will get alsmost the same number of heads and tails, because your sample group is much larger, and therefore more valid. The Tag McLaren test group is not large enough to be statistically valid.

You have to look at the distribution of the results with respect to statistical analysis. However, I would have taken these 2 people and repeated the test to see if they got similar results the second time around. Then I might attribute that to them being able to hear an audible difference. Otherwise, I wouold have to discount it as an anomoly due to the small number of subjects.

I would also question whether you could gain an accurate representative group from 12 people...

Seems to me that by adopting the coin toss analogy to this situation you are assuming as a given that the hearing ability, training and experience of all the participants are equal.

I don't know statistics, but I know enough to recognize the difference.

As for medical trials with large number of participants and trials, they are attempting to determine the effectiveness of a particular medicine on a large enough segment of the populace as to justify its use. Whether it works or not (it either does or doesn't) on a single individual within the broad group is meaningless, as they are concerned with percentage of effectiveness as to the entire group. They have to do it this way, as they can never be sure that a test subject who received real medicine and appears to have been cured was cured as a result of the medicine or a result of the placebo effect. They can only compare the overall results of the non-placebo group with that of the placebo group

I submit that in audio DBTs you don't have that situation. If the test is conducted properly, the pacebo effect, or the power of suggestion, or the ABE's or whatever you want to call influences that are not actual audible differences, should all be controlled out of the test, with the only thing left to test being actual audible differences. If one person can identify correctly 11 out of 15 times, there's no possible chance that placebo or ABEs could be at work, again assuming the test is properly conducted, so his correct guesses either have to be the result of chance (a very low possibility) or due to actual sonic differences he was able to detect; whereas, as I said, if the medicine seems to work on a single individual there still is no way of determinining if it was due to placebo or the real thing. This is a significant difference between audio DBTs and medical DBTs.

In the McLaren test, it doesn't make sense to me that they would want to know how a large group in general would perform, as each person is different with different capabilities. Seems to me that the important question is whether there are people who hear and can detect under blind conditions actual differences. In this case, there apparently were two.

Let's approach it this way. Let's say that out of 100 test subjects, only two hit the 11 out of 15, and the overall correct results of the group was only 50%.
Seems to me that could be significant data to the cable company whose cables are under test. If the test had been set up so that the distribution of hearing, experience and training adequately represented the general population, then such results might suggest to them that only 2% of the population could hear actual differences. If they were intent on marketing their product only to people who could actually hear the differences (remember, this is just a hypothetical for purpose of illustration and I'm trying to anaolgize to the medical situation), then they would have to decide if manufacturing the product that had been tested would be worthwhile if it could only be marketed to 2% of the general population.

If a medical test with 100 subject (I know - the numbers used are far larger in actual use and that improves the dependability of the results significantly, but I'm trying to keep this as simple as I can) showed that the medicine only worked on 2% of the general population (by comparing the non-placebo group to the placebo group and extrapolating that percentage number - which is the way I assume they do it, which is much more involved that the simple audio DBT test where the placebo can be controlled out at to every test subject), the company and FDA would face the same general question as the cable company. However, their decision might be quite different than the cable company, because the cable company might well decide that they can identify and focus their advertising to that 2% sufficiently to make it worthwhile to market their products. I assume drug companies can do that, although I'll ask my daughter-in-law who is a rapidly rising star with one of the biggies.

[DrJeff]

Seems to me that by adopting the coin toss analogy to this situation you are assuming as a given that the hearing ability, training and experience of all the participants are equal.

Perhaps so, but let's assume for the sake of your argument that we will use only those who describe themselves as "audiophiles" in our next hypothetical test.

I don't know statistics, but I know enough to recognize the difference.

As for medical trials with large number of participants and trials, they are attempting to determine the effectiveness of a particular medicine on a large enough segment of the populace as to justify its use. Whether it works or not (it either does or doesn't) on a single individual within the broad group is meaningless, as they are concerned with percentage of effectiveness as to the entire group. They have to do it this way, as they can never be sure that a test subject who received real medicine and appears to have been cured was cured as a result of the medicine or a result of the placebo effect. They can only compare the overall results of the non-placebo group with that of the placebo group.

I agree with that statement entirely.

I submit that in audio DBTs you don't have that situation. If the test is conducted properly, the pacebo effect, or the power of suggestion, or the ABE's or whatever you want to call influences that are not actual audible differences, should all be controlled out of the test, with the only thing left to test being actual audible differences. If one person can identify correctly 11 out of 15 times, there's no possible chance that placebo or ABEs could be at work, again assuming the test is properly conducted, so his correct guesses either have to be the result of chance (a very low possibility) or due to actual sonic differences he was able to detect; whereas, as I said, if the medicine seems to work on a single individual there still is no way of determinining if it was due to placebo or the real thing. This is a significant difference between audio DBTs and medical DBTs.

I agree that you can have a DBT test that very effectively minimizes placebo effect and other factors, leaving only one variable for the subject to judge. However, I think that a person guessing 11 out of 15 ONCE is just as likely as any other scenario. In irder to make the test valid, you would want to perform multiple runs of the same test. For example, can this person get it right 11 out of 15 times per run, for 5 consecutive runs. Of course the cable selection would be random so no pattern existed in switching. If his correct guesses for all 5 runs were in the 10 to 15 range, then I am convinced he can perceive a diffeence. However, if his guesses came up something like 11, 4, 10, 7, 5 then I would have to say he cannot hear a difference.

It can get very confusing, so I would leave it to people with more time and desire, but I believe a proper test could be conducted, I just haven't seen it yet.....

[pctower]

I agree that you can have a DBT test that very effectively minimizes placebo effect and other factors, leaving only one variable for the subject to judge. However, I think that a person guessing 11 out of 15 ONCE is just as likely as any other scenario. In irder to make the test valid, you would want to perform multiple runs of the same test. For example, can this person get it right 11 out of 15 times per run, for 5 consecutive runs. Of course the cable selection would be random so no pattern existed in switching. If his correct guesses for all 5 runs were in the 10 to 15 range, then I am convinced he can perceive a diffeence. However, if his guesses came up something like 11, 4, 10, 7, 5 then I would have to say he cannot hear a difference.

It can get very confusing, so I would leave it to people with more time and desire, but I believe a proper test could be conducted, I just haven't seen it yet.....

I agree and me too.

[Swerd]

pctower

Your reasoning that medical trials and listening tests are different is faulty. You are correct that medical trials do attempt to determine the effectiveness of a particular medicine on a large enough segment of the populace because there is significant variation among individuals in the population. The same problem exits for audio listening tests, precisely because hearing ability, training and experience of all the participants are NOT equal. It is not possible to perform a test where all non-audible differences are, as you describe, “controlled out of the test, with the only thing left to test being actual audible differences.” If it were, we wouldn’t need to use statistics and we wouldn’t be having this discussion.

As you point out from the McLaren test, 2 out of 12 people scored as if they were able to detect differences between cables. Twelve is such a small sample number that making any positive conclusion from those numbers would be wrong. To verify that finding, it would be necessary to test large numbers of people to eliminate a statistically defined problem called sampling error. Does the original test of 12 people truly represent the larger population as a whole? Does their hearing ability, prior training, and experience resemble that of the general population? That larger number people, off the top of my head, would be closer to 1000 than it would be to 100 or less. To put it simply, if we tested 1200 people, would 200 of them be able to detect the differences between cables? If this were the case, then I would support the conclusion that some people, about 1 out of 6, could hear differences in cables. It would then be up to the scientists, both biological and electrical to search for how this happens. At present, no listening test has provided such evidence.

[skeptic]

The cable proponents have not even demonstrated that there is statistically significant evidence that there are audible differences when the testers are prescreened to include only individuals of high auditory accuity. The small sample in this case is not significant and if that is the best they can come up with, they still haven't done anything IMO.

I'm not going to relate the story again about the NY Audio Labs demo at the WQXR auditorium in 1983, but IMO, most audiophiles and audio engineers have far less auditory accuity than they would like to think. This is especially true for those who have been exposed to very loud sound for any period at sometime in their lives. Like loud rock music played at deafening levels.

[Swerd]

The cable proponents have not even demonstrated that there is statistically significant evidence that there are audible differences when the testers are prescreened to include only individuals of high auditory accuity. The small sample in this case is not significant and if that is the best they can come up with, they still haven't done anything IMO.

I'm not going to relate the story again about the NY Audio Labs demo at the WQXR auditorium in 1983, but IMO, most audiophiles and audio engineers have far less auditory accuity than they would like to think. This is especially true for those who have been exposed to very loud sound for any period at sometime in their lives. Like loud rock music played at deafening levels.

Two thoughts:

It has been my opinion that differences in human audio perception are not so much due to differences in auditory acuity as it is to the less well understood processes that take place between the ears. Its kind of like saying that one's visual acuity determines how one interprets poetry or reacts to paintings. Your eyesight only has to be good enough to read the text or see the painting. Much happens after that initial process take place.

Instead of prescreening listeners to include only individuals of high auditory accuity, or only cable believers, it would be interesting to test all kinds of listeners, both golden- and tin-eared, identify them in a questionaire, and compare the results of the listening tests to see if being a believer or non-believer or a golden- or tin-ear makes any difference.

[FLZapped]

I'm not going to relate the story again about the NY Audio Labs demo at the WQXR auditorium in 1983, but IMO, most audiophiles and audio engineers have far less auditory accuity than they would like to think. This is especially true for those who have been exposed to very loud sound for any period at sometime in their lives. Like loud rock music played at deafening levels.

Musicians are as well. Think about Violin and Viola players with years of playing with the instrument under their left ear. They all will have a hearing imbalance from left to right as a result.

Drummers are notorious for being deaf.

What about the poor guys who sit in front of the brass section....

I've had my hearing tested and I sure wish I had taken better care of my hearing earlier in life.

-Bruce

[pctower]

pctower

Your reasoning that medical trials and listening tests are different is faulty. You are correct that medical trials do attempt to determine the effectiveness of a particular medicine on a large enough segment of the populace because there is significant variation among individuals in the population. The same problem exits for audio listening tests, precisely because hearing ability, training and experience of all the participants are NOT equal. It is not possible to perform a test where all non-audible differences are, as you describe, “controlled out of the test, with the only thing left to test being actual audible differences.” If it were, we wouldn’t need to use statistics and we wouldn’t be having this discussion.

As you point out from the McLaren test, 2 out of 12 people scored as if they were able to detect differences between cables. Twelve is such a small sample number that making any positive conclusion from those numbers would be wrong. To verify that finding, it would be necessary to test large numbers of people to eliminate a statistically defined problem called sampling error. Does the original test of 12 people truly represent the larger population as a whole? Does their hearing ability, prior training, and experience resemble that of the general population? That larger number people, off the top of my head, would be closer to 1000 than it would be to 100 or less. To put it simply, if we tested 1200 people, would 200 of them be able to detect the differences between cables? If this were the case, then I would support the conclusion that some people, about 1 out of 6, could hear differences in cables. It would then be up to the scientists, both biological and electrical to search for how this happens. At present, no listening test has provided such evidence.

All good and valid points. I don't feel my reasoning was faulty. I just didn't go far enough (trying to keep it simple) by stating that many more tests and much broader samples would be required to start drawing any reasonably conclusions. As just one example, I would certainly want the two individuals who scored 11 out of 15 to come back at least 4 or 5 more times to seek if they could duplicate their success. I would also want to test a much broader group of people to see both how the group as a whole does as well as whether any more "golden ears" start to appear.

At this point it's all conjecture other than the fact that we certainly know that the cable proponents have never produced any scientific evidence of actual sonic diferences between cables of similar gauge and length. Reasonable inferences might be drawn from that fact alone.