Page 5 of 6 FirstFirst ... 3 4 5 6 LastLast
Results 101 to 125 of 128

Thread: Null Hypothesis

  1. #101
    Forum Regular
    Join Date
    Nov 2003
    Posts
    884

    Do you have some point to make?

    Quote Originally Posted by pctower
    I understand his view fine and I understand the null hypothesis. But the null hypothesis can be mis-applied just like anything else. He is mis-applying it for the reasons I gave, which were very specific in nature.

    But then again, you don't like to deal in specifics. Sweeping generalizations couched in scientific terms designed to make you feel "important" are what you are all about - at least as you exhibit yourself on this board.
    No, I have shown you and everyone else draws conclusions from partial evidence, which makes you a hypocrite. The sun will rise tomorrow, PC, whether you like it or not. But I can't absolutely prove it. However, such conclusions can be upset by contrary evidence. Robot isn't saying anything else.You are nitpicking about trivialities and seem to have no real point to make.
    "Opposition brings concord. Out of discord comes the fairest harmony."
    ------Heraclitus of Ephesis (fl. 504-500 BC), trans. Wheelwright.

  2. #102
    Forum Regular
    Join Date
    Mar 2002
    Posts
    365
    Quote Originally Posted by Pat D
    No, I have shown you and everyone else draws conclusions from partial evidence, which makes you a hypocrite. The sun will rise tomorrow, PC, whether you like it or not. But I can't absolutely prove it. However, such conclusions can be upset by contrary evidence. Robot isn't saying anything else.You are nitpicking about trivialities and seem to have no real point to make.
    He has drawn a conclusion from test results, none of which can be demonstrated to have resulted from proper protocol or statistical analysis. Moreover, these tests involve a miniscule sample of the set for which he has drawn conclusions.

    If you cannot see the fault in that and believe it is comparable to the sun example, then you are beyond hope. However, I have suspected as much for quite a long time.

  3. #103
    Forum Regular
    Join Date
    Nov 2003
    Posts
    884

    Manufacturing disagreement.

    Quote Originally Posted by pctower
    He has drawn a conclusion from test results, none of which can be demonstrated to have resulted from proper protocol or statistical analysis. Moreover, these tests involve a miniscule sample of the set for which he has drawn conclusions.

    If you cannot see the fault in that and believe it is comparable to the sun example, then you are beyond hope. However, I have suspected as much for quite a long time.
    Did I say it was comparable? So far, all you do is point out that his conclusions cannot be absolute, which no one disagrees with, but that nevertheless we draw the conclusion, and that was the purpose of the example, as I in fact think you know. You are manufacturing a disagreement with Robot for your own purposes, and God knows what they are. If anyone thinks Robot's conclusions can be upset, they are free to do so. We've been waiting for a long time.

    You also forget what Robot said about comparing what cables can do with what people can hear, PC. I told you haven't grasped his argument, and you haven't! You simply ignore those parts inconvenient for your exposition.

    You seem to have no idea how science works, PC. According to you, Newton was wrong to come up with a universal Law of Gravitation. After all, he couldn't possibly have established it worked reasonably well for more than a miniscule part of the universe, a tiny, tiny proportion. Indeed, I understand some are now trying to find out how accurate it really is. Does gravity really obey the Inverse Square Law? Newton couldn't measure nearly as accurately as we can today. Nevertheless, the Law of Gravity has stood the test of time pretty well.
    "Opposition brings concord. Out of discord comes the fairest harmony."
    ------Heraclitus of Ephesis (fl. 504-500 BC), trans. Wheelwright.

  4. #104
    Forum Regular
    Join Date
    Sep 2003
    Posts
    259
    Quote Originally Posted by RobotCzar
    You need a course (or book) in logic and scientific method. You are simply not understanding the difference between certainty and very very likely. This difference AND your explanation as to why negative results are not sufficient to establish certainty is why we say "science can't prove negatives". You are also confused as to the meaning and purpose of the null hypothesis, which is what I tried to explain in the base note.

    The bottom line is that good scientific tests test a postitive assertion, like "I (or people) can hear difference in cables." When tested, nobody tested (and therefore people in general) has been able to demonstrate that they can consistently distinguish cables. A reasonable view at this point is that nobody can (and it will stay reasonable until somebody does).

    Needless to say, a logical analysis of what people can hear vs. what cables can do, leads one to logically conclude that people can't hear differences in typical cables, independent of a direct test.
    OK, Mr. Statistician, how about you telling me the meaning and purpose of the null hypothesis so I won't be confused anymore. Just pretend you are taking a stat course exam, and fill in the blanks. (1)The meaning of the null hypothesis is ......................... . (2)The purpose of the null hypothesis is ....................... . For extra credit, point out how I have misinterpreted the true meaning and purpose of the null. If all this seems like too much trouble, just remember you claimed that I'm a dummy, so the burden of proof is on you.

    Your last reply took 2 months, so answering these questions might take 2 years. Why not save time by just admiting you don't know which end is up when it comes to statistics. I'm not sure I do, and I suspect we have a lot of company here on the Forum. Anyway, I don't think you need to be an expert on hypothesis testing to present your argument.

    I understand your argument, but I don't agree with your conclusion. You believe the results of listening tests together with what is known about cables mean it is "very very likely" all cables sound the same to all listeners. I view the same evidence as inconclusive. I don't understand why you feel it is necessary to reach a verdict even if supporting evidence is lacking. Do you think scientific method requires you to reach conclusions with incomplete information?

    .

  5. #105
    Forum Regular Monstrous Mike's Avatar
    Join Date
    Dec 2001
    Posts
    335
    Quote Originally Posted by okiemax
    (1)The meaning of the null hypothesis is ......................... . (2)The purpose of the null hypothesis is .......................
    (1) A null hypothesis is a statement that two things are not different. The null hypothesis can be qualified by a number of variables (simply because in nature no two things are identical) but that is the essence of the meaning.

    (2) The purpose of the null hypothesis is to reject it and and therefore examine an alternative hypothesis.

    If you find that one guy that really has golden ears and can consistently hear a difference between two cables, then I do believe we can reject the general null hypothesis that all cables sound alike for cables of comparable gauge and length (see these are some of the qualifiers I was talking about). Now you can start investigating alternative hypothesis like what types of cables sound different and what kind of hearing is needed to detect those differences as well as how significant those differences are.

    However, since we still don't even have one guy who can consistently show he can hear differences, that null hypothesis is still a pretty big matzah ball. We can't explore any alternative hypotheses because we haven't even rejected the null hyptothesis.

    You and Robot seem to disagreeing about agreeing, if that even makes sense. The people who really can't grasp statistics or logic are to ones who demand that somebody needs to prove that the null hypothesis is true.
    Friends help friends move,
    Good friends help friends move bodies....

  6. #106
    Forum Regular
    Join Date
    Mar 2002
    Posts
    365
    Quote Originally Posted by Pat D
    Did I say it was comparable? So far, all you do is point out that his conclusions cannot be absolute, which no one disagrees with, but that nevertheless we draw the conclusion, and that was the purpose of the example, as I in fact think you know. You are manufacturing a disagreement with Robot for your own purposes, and God knows what they are. If anyone thinks Robot's conclusions can be upset, they are free to do so. We've been waiting for a long time.

    You also forget what Robot said about comparing what cables can do with what people can hear, PC. I told you haven't grasped his argument, and you haven't! You simply ignore those parts inconvenient for your exposition.


    You seem to have no idea how science works, PC. According to you, Newton was wrong to come up with a universal Law of Gravitation. After all, he couldn't possibly have established it worked reasonably well for more than a miniscule part of the universe, a tiny, tiny proportion. Indeed, I understand some are now trying to find out how accurate it really is. Does gravity really obey the Inverse Square Law? Newton couldn't measure nearly as accurately as we can today. Nevertheless, the Law of Gravity has stood the test of time pretty well.
    Pat, Pat, Pat

    You are really far afield here. Science does not counteanance the use of flawed, unreliable tests for any purpose - period - end of story.

    The control of variables that enabled the development of such things as the Law of Gravity within what was possible in Newton's day given measurement techniques and the development of theoretical knowledge then is no way comparable to the virtual lack of any valid or indept research whatsoever on cable sonics to date.

    I suspect my knowledge of science far exceeds yours.

  7. #107
    Forum Regular Monstrous Mike's Avatar
    Join Date
    Dec 2001
    Posts
    335
    Quote Originally Posted by pctower
    the virtual lack of any valid or indept research whatsoever on cable sonics to date.
    There's been plenty of indepth research into cable sonics. Many people have written very descriptive technical analysis on concepts like the Skin Effect, biwiring, frequency transformations, LCR parameters, etc. Some people seem to think they can invalidate this research by doing an in home test on their cables and claiming to hear a difference when there shouldn't be one, according to this research.

    Therefore, while nobody can conclude that cable sonics don't exist at all, for anybody, or for all cables, the valid research so far indicates this is the likely conclusion. And the lack of any proper test to reject this null hypothesis only adds to this likelyhood.

    Like I and many others have stated, science is not going to prove that cable sonics do not exist. The next steps in investigating cables sonics are to find a proper listening test which shows sonic differences in some cables and then to discover the reason for those differences. And as a matter of fact, if that can be done, then we have a foundation for actually making better cables.

    As it stands right now, audio cables are jewelry to adorn an expensive home audio system and nothing more. Of course, this jewelry can lead to better enjoyment by the listener and that is usually his goal so nothing more needs to be said about that.
    Friends help friends move,
    Good friends help friends move bodies....

  8. #108
    Forum Regular
    Join Date
    Sep 2003
    Posts
    259
    Quote Originally Posted by Monstrous Mike
    (1) A null hypothesis is a statement that two things are not different. The null hypothesis can be qualified by a number of variables (simply because in nature no two things are identical) but that is the essence of the meaning.

    (2) The purpose of the null hypothesis is to reject it and and therefore examine an alternative hypothesis.

    If you find that one guy that really has golden ears and can consistently hear a difference between two cables, then I do believe we can reject the general null hypothesis that all cables sound alike for cables of comparable gauge and length (see these are some of the qualifiers I was talking about). Now you can start investigating alternative hypothesis like what types of cables sound different and what kind of hearing is needed to detect those differences as well as how significant those differences are.

    However, since we still don't even have one guy who can consistently show he can hear differences, that null hypothesis is still a pretty big matzah ball. We can't explore any alternative hypotheses because we haven't even rejected the null hyptothesis.

    You and Robot seem to disagreeing about agreeing, if that even makes sense. The people who really can't grasp statistics or logic are to ones who demand that somebody needs to prove that the null hypothesis is true.
    There are plenty of people who claim to hear a difference in cables. The problem is no one, as far as we know, has done it in the only way naysayers or doubters will accept(i.e., double-blind listening tests). The cable believers could be wrong, but would it be unreasonable to consider the followng two other possibilities? (1) The listening tests done so far are flawed or insufficient in number. (2) The listening tests by their nature do not accurately represent actual listening for pleasure.

  9. #109
    Forum Regular Monstrous Mike's Avatar
    Join Date
    Dec 2001
    Posts
    335
    Quote Originally Posted by okiemax
    (1) The listening tests done so far are flawed or insufficient in number. (2) The listening tests by their nature do not accurately represent actual listening for pleasure.
    (1) I would have to agree. For example, any listening tests done so far, even if accurate, that show a null result really don't prove anything other than those specific cables for those specific listeners didn't sound different. The most interesting test I have read about was the one where the guy told the listeners that he was swapping between cheapo A and expensive B and sure enough B won the test. But there was actually no swap done. If that is true, then it sure highlights the fact that prior knowledge can indeed affect our assessment.

    (2) I'm not sure what you mean by this. The DBT tests we quote are only designed to detect audible differences in cables. Preference and listening pleasure are another matter.

    I guess the basic problem is that there is usually a difference between two things when you state a preference. Therefore, when somebody says they bought a new cable and they prefer it over their old cable, that would imply that something is different. Usually the person reports that the sound is better. That's what needs to be investigated. And I don't mean to imply that everybody needs to have this investigation done before they can enjoy their new cables.

    But in reality, that new cable may sound better because it looks better, it's more expensive or the audio signal travelling through it is affected differently in an audible manner to the that particular person with that particular setup.

    And we won't know for sure until we can prove it.
    Friends help friends move,
    Good friends help friends move bodies....

  10. #110
    Forum Regular
    Join Date
    Mar 2002
    Posts
    365
    Quote Originally Posted by Monstrous Mike
    (1) I would have to agree. For example, any listening tests done so far, even if accurate, that show a null result really don't prove anything other than those specific cables for those specific listeners didn't sound different. The most interesting test I have read about was the one where the guy told the listeners that he was swapping between cheapo A and expensive B and sure enough B won the test. But there was actually no swap done. If that is true, then it sure highlights the fact that prior knowledge can indeed affect our assessment.

    (2) I'm not sure what you mean by this. The DBT tests we quote are only designed to detect audible differences in cables. Preference and listening pleasure are another matter.

    I guess the basic problem is that there is usually a difference between two things when you state a preference. Therefore, when somebody says they bought a new cable and they prefer it over their old cable, that would imply that something is different. Usually the person reports that the sound is better. That's what needs to be investigated. And I don't mean to imply that everybody needs to have this investigation done before they can enjoy their new cables.

    But in reality, that new cable may sound better because it looks better, it's more expensive or the audio signal travelling through it is affected differently in an audible manner to the that particular person with that particular setup.

    And we won't know for sure until we can prove it.
    Perhaps what he meant by #2 is that the tests haven't been conducted in an atmoshpere that is similar to the ones to which people are accustomed in their own homes.

    As I understand it, there is a movement in psychological testing to get the testing "out of the lab" into "real world" environments so that the manner in which the tests are conducted resembles as closely as possible the environment where the activity being tested usually occurs.

    I would certainly agree that intangibles such as preference and listening pleasure are other matters. The only thing that matters for purposes of "claim verification" is the actual ability of people to differentiate cables based solely and exclusively on auditory stimulus produced under absolute identical circumstances but for the DUT (the cables being tested).

  11. #111
    Forum Regular
    Join Date
    Sep 2003
    Posts
    259
    Quote Originally Posted by Monstrous Mike
    (1) I would have to agree. For example, any listening tests done so far, even if accurate, that show a null result really don't prove anything other than those specific cables for those specific listeners didn't sound different. The most interesting test I have read about was the one where the guy told the listeners that he was swapping between cheapo A and expensive B and sure enough B won the test. But there was actually no swap done. If that is true, then it sure highlights the fact that prior knowledge can indeed affect our assessment.

    (2) I'm not sure what you mean by this. The DBT tests we quote are only designed to detect audible differences in cables. Preference and listening pleasure are another matter.

    I guess the basic problem is that there is usually a difference between two things when you state a preference. Therefore, when somebody says they bought a new cable and they prefer it over their old cable, that would imply that something is different. Usually the person reports that the sound is better. That's what needs to be investigated. And I don't mean to imply that everybody needs to have this investigation done before they can enjoy their new cables.

    But in reality, that new cable may sound better because it looks better, it's more expensive or the audio signal travelling through it is affected differently in an audible manner to the that particular person with that particular setup.

    And we won't know for sure until we can prove it.
    The problem with the "switch that wasn't a switch" is we don't know whether the subjects really thought the attractive cable sounded better or were just trying to please the tester. If you show somone a plain cable and a fancy looking cable, demonstrate them, and then ask which sounds best, that person may think you already believe the fancy cable sounds best, and may tell you what he thinks you want to hear. Or he may think there is a correct answer, and that he is being tested for his ability to get it right, so he may choose the fancy cable for that reason. The tester may think he is fooling the subject, but he may be just fooling himself.

    Regarding the other question, I'm not sure whether blinded testing accurately represents listening for pleasure. It seems like it should. However, although it's hard to explain, I don't think listening for differences in equipment involves me in the same way as listening for pleasure. I have also found that I sometimes hear things better when I'm not trying than when I am trying.

    I recall an experience many years ago that may be relevant to this discussion. After hearing a big improvement in an inexpensive Onkyo turntable after moving it from a floor stand to a wall mount, I got the turntable bug, and took a Linn LP-12 home for a 30-day trial. I thought the Linn sounded a little better than the Onkyo, but it wasn't as big as the improvement from the wall mount, so I returned the Linn because I decided it wasn't worth the much greater cost.. But after going back to the Onkyo, I missed the Linn. It was much better than I thought. I don't know how to explain why it took being without the Linn to realize how much I had grown to like it. So I bought one.

  12. #112
    Forum Regular Rockwell's Avatar
    Join Date
    Nov 2003
    Posts
    156
    Quote Originally Posted by okiemax
    The problem with the "switch that wasn't a switch" is we don't know whether the subjects really thought the attractive cable sounded better or were just trying to please the tester. If you show somone a plain cable and a fancy looking cable, demonstrate them, and then ask which sounds best, that person may think you already believe the fancy cable sounds best, and may tell you what he thinks you want to hear. Or he may think there is a correct answer, and that he is being tested for his ability to get it right, so he may choose the fancy cable for that reason. The tester may think he is fooling the subject, but he may be just fooling himself.
    Bias is bias.
    "You two are a regular ol' Three Musketeers."

  13. #113
    Forum Regular
    Join Date
    Sep 2003
    Posts
    259
    Quote Originally Posted by Rockwell
    Bias is bias.
    What a great idea for a new an imaginative use of the word "biased." Next time I fib to please someone, and get caught, my explanation will be "I was just biasing you." Or if stumped for an answer on a multiple choice test, I'll just resort to my bias. And what about the guy who rigged that phony cable switch -- was he guilty of deception? Nope, he was just doing a little biasing .

  14. #114
    Forum Regular
    Join Date
    Nov 2003
    Posts
    884

    So, I have won my point, PC!

    Quote Originally Posted by pctower
    Pat, Pat, Pat

    You are really far afield here. Science does not counteanance the use of flawed, unreliable tests for any purpose - period - end of story.

    The control of variables that enabled the development of such things as the Law of Gravity within what was possible in Newton's day given measurement techniques and the development of theoretical knowledge then is no way comparable to the virtual lack of any valid or indept research whatsoever on cable sonics to date.

    I suspect my knowledge of science far exceeds yours.
    We have only your word that the tests are flawed and unreliable, PC. But then you evidently haven't looked into the matter and you seem to have no standards for reliability. But it's a red herring. We don't have to prove they sound the same. Those who claim they sound different have the burden of proof. So, where is the evidence that reasonably constructed 3 foot interconnects and 10 foot speaker cables of appropriate gauge make an audible difference, PC?

    I take it that I have won my point that we all make generalizations from incomplete evidence, including scientists like Newton, as you don't bother to dispute it.

    Resident Loser raised one point in a different way: " I take it then you feel any generalization of a class is invalid...even if it walks like a duck and quacks like a duck and..."

    Well, those do think like that (and so far, that is about all your objections amount to) can't very well do science. Q.E.D.

    Anyway, you don't deny that Newton lacked complete evidence for his generalizations, and we still don't. Never will, either. We simply can't measure everything in the universe to see if it behaves according to those generalizations. But we still find many of them reliable for many purposes.

    So, your objection falls completely flat. People have been trying to justify high priced cables with claims of superior performance for quite a number of years, PC. It is easy to make speaker cables that sound different and it ain't rocket science. The capacitance of phono cables can make an audible (and easily measureable) difference, and that ain't rocket science, either.
    "Opposition brings concord. Out of discord comes the fairest harmony."
    ------Heraclitus of Ephesis (fl. 504-500 BC), trans. Wheelwright.

  15. #115
    Forum Regular
    Join Date
    Mar 2002
    Posts
    365
    Quote Originally Posted by Pat D
    We have only your word that the tests are flawed and unreliable, PC. But then you evidently haven't looked into the matter and you seem to have no standards for reliability. But it's a red herring. We don't have to prove they sound the same. Those who claim they sound different have the burden of proof. So, where is the evidence that reasonably constructed 3 foot interconnects and 10 foot speaker cables of appropriate gauge make an audible difference, PC?

    I take it that I have won my point that we all make generalizations from incomplete evidence, including scientists like Newton, as you don't bother to dispute it.

    Resident Loser raised one point in a different way: " I take it then you feel any generalization of a class is invalid...even if it walks like a duck and quacks like a duck and..."

    Well, those do think like that (and so far, that is about all your objections amount to) can't very well do science. Q.E.D.

    Anyway, you don't deny that Newton lacked complete evidence for his generalizations, and we still don't. Never will, either. We simply can't measure everything in the universe to see if it behaves according to those generalizations. But we still find many of them reliable for many purposes.

    So, your objection falls completely flat. People have been trying to justify high priced cables with claims of superior performance for quite a number of years, PC. It is easy to make speaker cables that sound different and it ain't rocket science. The capacitance of phono cables can make an audible (and easily measureable) difference, and that ain't rocket science, either.
    "We have only your word that the tests are flawed and unreliable, PC."

    My word has nothing to do with it. This board is the naysayer's holy temple. Yet, no one has ever referenced test results that carry the necessary information to satisfy basic scientific requirements for test reliability.

    The people that claim such test results are reliable have the burden of proof of demonstrating reliability.

    "Those who claim they sound different have the burden of proof."

    YAWN! I can't even imagine what it must be like to have a mind so empty that it could tolerate writing such a basic truism over and over and over.

    "But then you evidently haven't looked into the matter and you seem to have no standards for reliability."

    Wrong Bozo Man! I have looked into it and I have repeated stated my standard for reliability (and I am the only person as far as I know who has ever done so on this board).

    My standard is that for a test to be scientifically valid it should be conducted according to protocol that would be acceptable in any college level or above class (it would help is such class was a psycology class or the like, rather than an engineering classe where I doubt tests similar to cable DBTs are ever conducted) or would be considered acceptable according to standards set by any recognized national research lab, AND

    statistically would avoid unacceptable Type II errors:

    The synopsis of Leventhal’s article that appeared in J.AudioEng.Soc.,Vol.34,No.6, 1986, June follows:

    “When the conventional 0.05 significance level is used to analyze listening test data, employing a small number of trials or listeners can produce an unexpectedly high risk of concluding that audible differences are inaudible (type 2 error). The risk can be both large absolutely and large relative to the risk of concluding that inaudible differences are audible (type 1 error). This constitutes systematic bias against those who believe that differences are audible between well-designed electronic components that are spectrally equated and not overdriven. A statistical table is introduced that enables readers to look up type 1 and type 2 error risks without calculation. Ways to manipulate the risks are discussed, a quantitative measure of a listening test's fairness is introduced, and implications for reviewers of the listening test literature are discussed. “


    Shanefield responded at J. Audio Eng. Soc., Vol. 35, No. 7/8, 1987 July/August, page 567, part of which follows:

    “While the Leventhal treatment enhances our theoretical understanding of A-B tests, the engineering usefulness is another matter. A value judgment needs to be made as to whether a low number of p is at all useful. My own judgment is that low numbers for p are of little practical use, even though it is satisfying to have a mathematical understanding of them. In other words, if two amplifiers can only be distinguished audibly a small percentage of the time (with a low p), but the experiment can be repeated fairly precisely (with a high "confidence"), then the audible difference is "useless" to me, even though it might be "statistically real." This low p is just as useless as a high p with a low confidence, and the reason is that, either way, there would be no audible difference on which we can depend. In fact, since the same data could usually lead to either conclusion, one type of uncertainty is simply being exchanged for another.”

    In that same issue, Leventhal responded to Shanefield at page 569, part of which follows:


    “Regarding old conclusions, published listening studies employing a small number of trials or listeners (small N) typically fail to produce statistical significance at the 0.05 level. Many readers reach the conclusion: "there were no audible differences." My paper suggests a more accurate conclusion: "there were no large audible differences, but judgment should be withheld about small or moderate audible differences because the studies were not sufficiently powerful (sensitive) to find them when they occur." Professor Shanefield and others interested only in large audible differences will find no practical difference between the conclusions. But those interested in small or moderate audible differences will find the conclusions to be different.

    Regarding engineering usefulness, even engineers interested only in large audible differences should find the paper useful. First, it discusses statistical assumptions and design considerations for listening studies, Published listening studies often founder on one or both of these requirements. Second, an engineer de- signing a listening study can use Table 3 to find the minimum N to employ before the risk of overlooking large differences becomes unacceptably large. For ex- ample, assume it important to keep type 1 and type 2 error risks small and approximately equal. Table 3 shows that one interested only in large differences (p _> 0.9) can employ merely 10 trials or listeners (N = 10) and require eight correct for significance at the 0.06 level (actual type 1 error risk, that is, exact significance level = 0.0547).]° Here the risk of over- looking an audible difference (type 2 error) when the difference is large is 0.0702. Thus type 1 and type 2 error risks are reasonably small and approximately equal (FC0.0 = 0.0547/0.0702 = 0.78). So Table 3 shows that an equal-error listening study looking only for large differences can employ an N of 10 and that most published studies, because they use N greater than 10, are needlessly long for this purpose. The third way the paper is useful is that an engineer reading a study with nonsignificant results can use Table 3 to find whether N was sufficiently large to uncover large audible differences when they occur. Since most studies employ an N greater than 10, they can uncover large differences and, with Table 3, the engineer will know why.

    See also: http://www.stereophile.com/showarchives.cgi?141


    "I take it that I have won my point that we all make generalizations from incomplete evidence, including scientists like Newton, as you don't bother to dispute it."

    You can take whatever your closed mind wants to take. You apparently have no understanding that simply because one particular generalization may be reliabile doesn't mean ALL generalizations are reliable.

    "People have been trying to justify high priced cables with claims of superior performance for quite a number of years, PC. It is easy to make speaker cables that sound different and it ain't rocket science. The capacitance of phono cables can make an audible (and easily measureable) difference, and that ain't rocket science, either."

    And so? Are we now on to an entirely different subject?

  16. #116
    Forum Regular
    Join Date
    Nov 2003
    Posts
    884

    More nitpicking, no proof of differences.

    Quote Originally Posted by pctower
    "We have only your word that the tests are flawed and unreliable, PC."

    My word has nothing to do with it. This board is the naysayer's holy temple. Yet, no one has ever referenced test results that carry the necessary information to satisfy basic scientific requirements for test reliability.

    The people that claim such test results are reliable have the burden of proof of demonstrating reliability.

    "Those who claim they sound different have the burden of proof."

    YAWN! I can't even imagine what it must be like to have a mind so empty that it could tolerate writing such a basic truism over and over and over.

    "But then you evidently haven't looked into the matter and you seem to have no standards for reliability."

    Wrong Bozo Man! I have looked into it and I have repeated stated my standard for reliability (and I am the only person as far as I know who has ever done so on this board).

    My standard is that for a test to be scientifically valid it should be conducted according to protocol that would be acceptable in any college level or above class (it would help is such class was a psycology class or the like, rather than an engineering classe where I doubt tests similar to cable DBTs are ever conducted) or would be considered acceptable according to standards set by any recognized national research lab, AND

    statistically would avoid unacceptable Type II errors:

    The synopsis of Leventhal’s article that appeared in J.AudioEng.Soc.,Vol.34,No.6, 1986, June follows:

    “When the conventional 0.05 significance level is used to analyze listening test data, employing a small number of trials or listeners can produce an unexpectedly high risk of concluding that audible differences are inaudible (type 2 error). The risk can be both large absolutely and large relative to the risk of concluding that inaudible differences are audible (type 1 error). This constitutes systematic bias against those who believe that differences are audible between well-designed electronic components that are spectrally equated and not overdriven. A statistical table is introduced that enables readers to look up type 1 and type 2 error risks without calculation. Ways to manipulate the risks are discussed, a quantitative measure of a listening test's fairness is introduced, and implications for reviewers of the listening test literature are discussed. “


    Shanefield responded at J. Audio Eng. Soc., Vol. 35, No. 7/8, 1987 July/August, page 567, part of which follows:

    “While the Leventhal treatment enhances our theoretical understanding of A-B tests, the engineering usefulness is another matter. A value judgment needs to be made as to whether a low number of p is at all useful. My own judgment is that low numbers for p are of little practical use, even though it is satisfying to have a mathematical understanding of them. In other words, if two amplifiers can only be distinguished audibly a small percentage of the time (with a low p), but the experiment can be repeated fairly precisely (with a high "confidence"), then the audible difference is "useless" to me, even though it might be "statistically real." This low p is just as useless as a high p with a low confidence, and the reason is that, either way, there would be no audible difference on which we can depend. In fact, since the same data could usually lead to either conclusion, one type of uncertainty is simply being exchanged for another.”

    In that same issue, Leventhal responded to Shanefield at page 569, part of which follows:


    “Regarding old conclusions, published listening studies employing a small number of trials or listeners (small N) typically fail to produce statistical significance at the 0.05 level. Many readers reach the conclusion: "there were no audible differences." My paper suggests a more accurate conclusion: "there were no large audible differences, but judgment should be withheld about small or moderate audible differences because the studies were not sufficiently powerful (sensitive) to find them when they occur." Professor Shanefield and others interested only in large audible differences will find no practical difference between the conclusions. But those interested in small or moderate audible differences will find the conclusions to be different.

    Regarding engineering usefulness, even engineers interested only in large audible differences should find the paper useful. First, it discusses statistical assumptions and design considerations for listening studies, Published listening studies often founder on one or both of these requirements. Second, an engineer de- signing a listening study can use Table 3 to find the minimum N to employ before the risk of overlooking large differences becomes unacceptably large. For ex- ample, assume it important to keep type 1 and type 2 error risks small and approximately equal. Table 3 shows that one interested only in large differences (p _> 0.9) can employ merely 10 trials or listeners (N = 10) and require eight correct for significance at the 0.06 level (actual type 1 error risk, that is, exact significance level = 0.0547).]° Here the risk of over- looking an audible difference (type 2 error) when the difference is large is 0.0702. Thus type 1 and type 2 error risks are reasonably small and approximately equal (FC0.0 = 0.0547/0.0702 = 0.78). So Table 3 shows that an equal-error listening study looking only for large differences can employ an N of 10 and that most published studies, because they use N greater than 10, are needlessly long for this purpose. The third way the paper is useful is that an engineer reading a study with nonsignificant results can use Table 3 to find whether N was sufficiently large to uncover large audible differences when they occur. Since most studies employ an N greater than 10, they can uncover large differences and, with Table 3, the engineer will know why.

    See also: http://www.stereophile.com/showarchives.cgi?141


    "I take it that I have won my point that we all make generalizations from incomplete evidence, including scientists like Newton, as you don't bother to dispute it."

    You can take whatever your closed mind wants to take. You apparently have no understanding that simply because one particular generalization may be reliabile doesn't mean ALL generalizations are reliable.

    "People have been trying to justify high priced cables with claims of superior performance for quite a number of years, PC. It is easy to make speaker cables that sound different and it ain't rocket science. The capacitance of phono cables can make an audible (and easily measureable) difference, and that ain't rocket science, either."

    And so? Are we now on to an entirely different subject?
    And this is what you call looking into it? Do I have to remind you that Cheever's master's thesis only used SBTs? Or isn't that college level enough for you! And he also referred to subjective reviews! Standards ha!

    No, it's not a different subject. There are pretty reliable standards for level matching. If those are exceeded, then one could presume the difference would be audible under some circumstances, at least. And, MM brought that up, but you just keep on ignoring it. This is, to say the least, disingenuous. Indeed, I would say that to limit your consideration to particular cables rather than performance parameters is basically unscientific.

    You may not like the null hypothesis in regards to cables but it has stood up for quite a number of years. Should be easy to disprove, PC, and nitpicking won't do it! For practical purposes, I'll just keep accepting it under normal circumstances until the data comes along to disprove it. This in no way excludes research PC, and it would be misleading to suggest otherwise. Of course, just finding Cable A sounds different from Cable B is fairly trivial, though necessary. Then begins more the interesting work of determining what it is in the performance that makes the audible difference. And it ain't rocket science, PC, not so far. Are you able to grasp science on that level, PC?

    Not only do you foist views on us that we don't hold, you foist them on Professor Leventhal. Leventhal did not say the DBTs are not scientific. How useful the results are depends on the purpose for which they are used.
    "Opposition brings concord. Out of discord comes the fairest harmony."
    ------Heraclitus of Ephesis (fl. 504-500 BC), trans. Wheelwright.

  17. #117
    Forum Regular
    Join Date
    Nov 2003
    Posts
    1,720
    Quote Originally Posted by Pat D
    And this is what you call looking into it? Do I have to remind you that Cheever's master's thesis only used SBTs? Or isn't that college level enough for you! And he also referred to subjective reviews! Standards ha!

    No, it's not a different subject. There are pretty reliable standards for level matching. If those are exceeded, then one could presume the difference would be audible under some circumstances, at least. And, MM brought that up, but you just keep on ignoring it. This is, to say the least, disingenuous. Indeed, I would say that to limit your consideration to particular cables rather than performance parameters is basically unscientific.

    You may not like the null hypothesis in regards to cables but it has stood up for quite a number of years. Should be easy to disprove, PC, and nitpicking won't do it! For practical purposes, I'll just keep accepting it under normal circumstances until the data comes along to disprove it. This in no way excludes research PC, and it would be misleading to suggest otherwise. Of course, just finding Cable A sounds different from Cable B is fairly trivial, though necessary. Then begins more the interesting work of determining what it is in the performance that makes the audible difference. And it ain't rocket science, PC, not so far. Are you able to grasp science on that level, PC?

    Not only do you foist views on us that we don't hold, you foist them on Professor Leventhal. Leventhal did not say the DBTs are not scientific. How useful the results are depends on the purpose for which they are used.
    And, he only used 70% confidence level to meet Leventhols thinking. Totally unacceptable as it is not science. 70% is not accepted anywhere in science.
    mtrycrafts

  18. #118
    Forum Regular
    Join Date
    Nov 2003
    Posts
    884
    Quote Originally Posted by mtrycraft
    And, he only used 70% confidence level to meet Leventhols thinking. Totally unacceptable as it is not science. 70% is not accepted anywhere in science.
    Yes, that is truly bizarre.
    "Opposition brings concord. Out of discord comes the fairest harmony."
    ------Heraclitus of Ephesis (fl. 504-500 BC), trans. Wheelwright.

  19. #119
    Forum Regular
    Join Date
    Mar 2002
    Posts
    365
    Quote Originally Posted by Pat D
    And this is what you call looking into it? Do I have to remind you that Cheever's master's thesis only used SBTs? Or isn't that college level enough for you! And he also referred to subjective reviews! Standards ha!

    No, it's not a different subject. There are pretty reliable standards for level matching. If those are exceeded, then one could presume the difference would be audible under some circumstances, at least. And, MM brought that up, but you just keep on ignoring it. This is, to say the least, disingenuous. Indeed, I would say that to limit your consideration to particular cables rather than performance parameters is basically unscientific.

    You may not like the null hypothesis in regards to cables but it has stood up for quite a number of years. Should be easy to disprove, PC, and nitpicking won't do it! For practical purposes, I'll just keep accepting it under normal circumstances until the data comes along to disprove it. This in no way excludes research PC, and it would be misleading to suggest otherwise. Of course, just finding Cable A sounds different from Cable B is fairly trivial, though necessary. Then begins more the interesting work of determining what it is in the performance that makes the audible difference. And it ain't rocket science, PC, not so far. Are you able to grasp science on that level, PC?

    Not only do you foist views on us that we don't hold, you foist them on Professor Leventhal. Leventhal did not say the DBTs are not scientific. How useful the results are depends on the purpose for which they are used.
    I have no more time for responding to the blatant intellectual dishonesty you are foisting on me and others who read this thread. I did not come close to claiming Leventhal thought DBTs were not scientific. You claimed I didn't have standards of reliability. I cited his discussion of Type II errors as one of the issues a DBT would have to satisfy in order for me to consider it reliable.

    You are playing games. I have no interest in joining your sand box.

    Bye Bye

  20. #120
    Forum Regular Rockwell's Avatar
    Join Date
    Nov 2003
    Posts
    156
    Quote Originally Posted by okiemax
    What a great idea for a new an imaginative use of the word "biased." Next time I fib to please someone, and get caught, my explanation will be "I was just biasing you." Or if stumped for an answer on a multiple choice test, I'll just resort to my bias. And what about the guy who rigged that phony cable switch -- was he guilty of deception? Nope, he was just doing a little biasing .
    Sarcasm noted.

    Here's a definition for you: Bias (2): systematic error introduced into sampling or testing by selecting or encouraging one outcome or answer over others

    Seems like that is exactly what happened.
    "You two are a regular ol' Three Musketeers."

  21. #121
    Forum Regular
    Join Date
    Sep 2003
    Posts
    259
    Quote Originally Posted by Rockwell
    Sarcasm noted.

    Here's a definition for you: Bias (2): systematic error introduced into sampling or testing by selecting or encouraging one outcome or answer over others

    Seems like that is exactly what happened.
    So the tester introduces the bias, and then attributes the bias to the test subjects. Hilarious! Lewis Carroll would have loved it!

  22. #122
    Forum Regular Rockwell's Avatar
    Join Date
    Nov 2003
    Posts
    156
    Quote Originally Posted by okiemax
    So the tester introduces the bias, and then attributes the bias to the test subjects. Hilarious! Lewis Carroll would have loved it!
    I don't know the details of the tests, but I think that was the point of it; to show how easy it is to bias the testers with the look of the cable.
    "You two are a regular ol' Three Musketeers."

  23. #123
    Forum Regular
    Join Date
    Sep 2003
    Posts
    259
    Quote Originally Posted by Rockwell
    I don't know the details of the tests, but I think that was the point of it; to show how easy it is to bias the testers with the look of the cable.
    Yes, but the bias may be a result of more that just the looks of the cables. Consider the following possibilty where we have a plain looking cable and a fancing looking cable, which actually are identical disguised zip cords. The tester asks the subject to listen to the fancy cable and the plain cable, and choose the one that sounds best. The subject thinks the tester believes one cable sounds better than the other, and figures it's probably the fancy one. So even though the subject can't hear a difference during the test, he tells the tester the fancy cable sounds better because he thinks that is what the tester believes. The tester then erronously concludes that the subject believes the fancy cable sounds better. The tester thinks he has fooled the subject, but he has just fooled himself.

  24. #124
    Forum Regular
    Join Date
    Nov 2003
    Posts
    884
    Quote Originally Posted by okiemax
    Yes, but the bias may be a result of more that just the looks of the cables. Consider the following possibilty where we have a plain looking cable and a fancing looking cable, which actually are identical disguised zip cords. The tester asks the subject to listen to the fancy cable and the plain cable, and choose the one that sounds best. The subject thinks the tester believes one cable sounds better than the other, and figures it's probably the fancy one. So even though the subject can't hear a difference during the test, he tells the tester the fancy cable sounds better because he thinks that is what the tester believes. The tester then erronously concludes that the subject believes the fancy cable sounds better. The tester thinks he has fooled the subject, but he has just fooled himself.
    Likely also that the subject has fooled him/herself, too! Some elements of this are not conscious processes.
    "Opposition brings concord. Out of discord comes the fairest harmony."
    ------Heraclitus of Ephesis (fl. 504-500 BC), trans. Wheelwright.

  25. #125
    Color me gone... Resident Loser's Avatar
    Join Date
    Mar 2004
    Location
    Nueva Jork
    Posts
    2,148

    HUH!!!...okie-dokie Max...a few things...

    "...What a great idea for a new an imaginative use of the word "biased." Next time I fib to please someone, and get caught, my explanation will be "I was just biasing you..."

    How can you equate "bias" with lying? Or "fibbing" if that's more to your liking...Have YOU never responded to a question with the answer you THINK the other person wants to hear...You are not necessarily trying to mislead, but we all seem to have this inborn trait of wanting to give the "right" answer...even if there is no such thing.

    "...Or if stumped for an answer on a multiple choice test, I'll just resort to my bias..."

    Well, perhaps in a way, yes...you'll take your best guess. What might that be based on? Education? Practical experience? Anything that prompts you do do something can be said to be the result of a "bias".

    "...And what about the guy who rigged that phony cable switch -- was he guilty of deception? Nope, he was just doing a little biasing..."

    Double HUH!!! This "phony switch" was, in essence, one of the items under test. If you are testing "x" and "y" really doesn't exist(except through inference induced by said switch), you are still testing "x"...If people say they hear a "y" how should those results be viewed?

    "...So the tester introduces the bias, and then attributes the bias to the test subjects. Hilarious! Lewis Carroll would have loved it!...

    A tester CAN introduce bias...Have you ever seen a magician do tricks...card tricks specifically...Magicians consciously use a technique called, in the art, "forcing"...a quite sucessful tactic..."mentalists" use a similar device, when questioning subjects..."Gee! Howdy do dat?"...Testers may do the same "type" of thing on a subconscious level, hence the need for DBTs where neither the subject, nor the tester, are privy to the information...

    jimHJJ(...then of course we have the "treble-blind test' wherein no one knows anything...)

Page 5 of 6 FirstFirst ... 3 4 5 6 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Proof of placebo effect ?
    By okiemax in forum Cables
    Replies: 132
    Last Post: 01-15-2004, 06:06 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •