Quote Originally Posted by RGA
The problem is that most studied tests in controlled environment have some correlation in real world listening environments...and we can go on ad nauseum about the tests forever, but the test environment is not the same(identical) to a non test environment...and there is no coreelation between the two but a lot of assumptions and innuendo as to what the result of a test says and what the real world says. Floyd Toole's also notes that these are results for the test environment not a real world environment. DBT's have shown that within the testing environment and the controls set-up - people have failed to distinguish differences to a statistically significant level better than chance.

That is ALL there is on the subject...Innuendo by the uninformed beyond this is why Americans got fat eating low fat diets for 30 years instead of following the once maligned now considered food God Dr. Atkins. The body of sicence was wrong because they took short cuts and made ASSUMPTIONS with having ALL the facts. Audio may not be the same...but there are certainly ASSUMPTIONS. There are two terms about testing Reliability which reproduces the same results over and over so we can reliably predict what is going to happen in a test involving trials. Then there is validitiy...how does what is being tested directly relate to that of reality. If a stereo is designed to provide long term musical enjoyment in one's home - then how valid is a test not set-up to that goal? Vague yes...but lots of bad tests have reliability, validity is the most important and of the two MORE important than reliability. You'd need both. Problem is that the direct problem is that normal listening is sighted, which is contradictory to what a DBT demands...it is this that causes "some" of the confusion and bickering. Nothing wrong with Double Blind tests - The complete story not according to psychologists or statisticians - the complete story to engineers? pick your field.
"The problem is that most studied tests in controlled environment have some correlation in real world listening environments..."

If so, the question becomes how much, what aspects, etc of the real world. Without concrete answers, the testing is again basically worthless. Nothing more than guessing again. Even how the "stereo" is setup could make a difference in the results.

It is obvious crafts doesn't know much about electronics as he didn't understand how DA and DF manifest themselves, thinking it would be measured by a harmonic distortion analyzer.

With this backround, he is obviously in the phycho camp and claims EEs aren't experts in DBT testing. It is also evident that phychos don't understand electronics like EEs do, and therefore do not understand that simply setting up a stereo for a DBT test could easily invalidate the testing.