SACD vs. CD test synopsis

[Feanor]

In case anyone missed the original (as did I), I'm passing on this MIXonline synopsis (http://mixonline.com/recording/mixing/audio_emperors_new_sampling/) of the Amerian Engineering Society's journal article, Audibility of a CD-Standard A/D/A Loop Inserted Into High-Resolution Audio Playback. (My thanks to Duilawyer at AA.)

Apparently the test was a rigorous, ABX DBT. An interesting aspect is the comparision was between stereo SACD layers and the same layer down-sampled to 16/44.1, (not an SACD layer to CD layer comparison as you or I might do).

Read on for the conclusions. As with any DBT, the results can't prove that differences don't exist, only that they couldn't be detected by the actual participants under the conditions of the test. Basically correct identification of SACD vs. CD was equivalent to coin tosses.

After the synopsis itself the authors proceed to talk about a few things including why hi-rez might sound better for spacial clues and how the "comb effect" might created perceived differences because of listener postion that aren't really attributable to the equipment or sources.

As to the spacial clues, it is quoted that humans can detect 5 microsecond difference in sound reaching the right as opposed to left ear. 5 microseconds is not within the 16/44.1 resolution. Of course this would pertain to left vs. right source channel, not left vs. right speaker. And obviously it would explain why difference were heard and not why they weren't heard.

The authors mentioned the "comb effect" which is frequency resonse variations on account of differences in listening position (in non-anechoic rooms such as the test rooms used). This is offered as an explanation why changes are heard, say, when the listener gets up to swap cables and doesn't return to exactly the same listening position. So in the case of these particular tests (non-)differences weren't heard because the listeners didn't have to get up to swap anything, they only had to press a button.

What wasn't discussed at any length is the placebo effect, viz. perceived differences are created by the listeners expectations rather than by actual differences.

[RGA]

"As with any DBT, the results can't prove that differences don't exist, only that they couldn't be detected by the actual participants under the conditions of the test."

And that is all that really needs to be said about this or any other DBT. Incidentally, no one can tell the difference between CD and Tape, Vinyl and CD, Vinyl and Tape, Coke and Pepsi, bottled water versus tap water, your monarchy and a $5.00 clock radio, and a whole host of other things and SACD versus CD - did anyone seriously even doubt for even a second that there would be a difference detected in a DBT. If nobody can tell tape from CD they sure as hell ain't going to hear the difference between cd and SACD. At least with vinyl the odd pop or tick helped.

A test is a test - life is something else. The fact that someone failed a test doesn't mean they failed life. Just as though they could not detect the difference in the test does not mean they can't detect differences. Unfortunately the science that actually created the test know this - the people at the AES don't. But then engineers are glorified repairmen and hardly in any way shape or form scientists.

All that said - hey it's a free country buy your SACD or your CD and for whatever reason you wish to...WHO CARES?

[Feanor]

"As with any DBT, the results can't prove that differences don't exist, only that they couldn't be detected by the actual participants under the conditions of the test."

And that is all that really needs to be said about this or any other DBT. Incidentally, no one can tell the difference between CD and Tape, Vinyl and CD, Vinyl and Tape, Coke and Pepsi, bottled water versus tap water, your monarchy and a $5.00 clock radio, and a whole host of other things and SACD versus CD - did anyone seriously even doubt for even a second that there would be a difference detected in a DBT. If nobody can tell tape from CD they sure as hell ain't going to hear the difference between cd and SACD. At least with vinyl the odd pop or tick helped.

A test is a test - life is something else. The fact that someone failed a test doesn't mean they failed life. Just as though they could not detect the difference in the test does not mean they can't detect differences. Unfortunately the science that actually created the test know this - the people at the AES don't. But then engineers are glorified repairmen and hardly in any way shape or form scientists.

All that said - hey it's a free country buy your SACD or your CD and for whatever reason you wish to...WHO CARES?

That I can't taste the difference between Coke and Pepsi?? :confused5:

I agree the DBT is the be-all and end-all. But in a well-designed test if no difference is found, then actual differences are extremely small. Audiophiles aside, this is what 97% of the population is interested in.

[RGA]

The problems with the DBT supporters is so numerous that frankly I don't have the interest or time to go down that route again. A little knowledge is destructive and most of them have little knowledge. Typically they have a 16 trial test (where they came up with that is strange as in real science with subjects such a test would be laughed out of the room). Some are even worse where they add the results of several listeners together like the lame Oakland University ABX site people love to produce as some sort of proof that A and B sound the same to people when it can't prove that. Basically it's science to support a case of innuendo and probability math.

If I say A is better than B much of the time in my opinion then someone will say I'm deluded - then they will suggest a DBT. Now if I select A 6/10 they will deem that I FAILED the test according to the .05 level of significance and that I could no better tell A from B than flipping a coin. Umm they're subtle differences so of course I or anyone else could get it wrong 4/10 times - that HARDLY means that A is in fact not better than B.

What is interesting is that in real science with human interaction high trials are critical - especially with sense perception choices. What the AES people NEVER tell people is that 6/10 ten times for a total score of 59/100 (one miss) is the EXACT same mathematically as a score of 9/10 to reach statisitical signifcance.

Umm that means that 6/10 is not a fail - it SHOULD mean they keep testing but they don't. That is the difference between someone who actually understands what a DBT is how it should be run and more importantly evaluating the numbers. Those jokers don't get it. More trial (not lumping different people's scores together) is critical.

I have met so many psychologists and doctoral student in these fields and when I bring up the words Double blind tests they're eyes roll - it is an interesting tool and nothing more than that.

Now having said that I agree with you about small difference. The audiophile community is asking for trouble when you have quacks running around telling people that they can detect the difference of every cable going and that the differences are VAST. No if they were VAST differences then you should get 9/10 or 14/16. The fact that they can't tells you that the differences are subtle and very possibly indistinguishable.

CD and SACD differences likely come down to the recording quality. And not all systems distinguish differences very well. It would be nice to see the details of the systems.

Hi Fi Choice listens blind and level matched but they are also using Audio Note AN E loudspeakers most of the time - some of the DBT I've read with speakers that I doubt I could tell a Rotel from a Krell or a Jolida tube amp. If a speaker makes everything sound like dead dreck then nobody will hear any differences.

Florescent lights after a time grow fatiguing for SOME people but not usually in quick duration tests - ditto for a lot of amps and cd players.

PS Most people can not score 9/10 comparing Coke to Pepsi to the .05 significance level. And nobody has yet to differentiate bottled water from tap water - but clearly when you drink it NOT in a test environment you can taste the difference.

[Feanor]

...
CD and SACD differences likely come down to the recording quality. And not all systems distinguish differences very well. It would be nice to see the details of the systems.
...

Your right about CD / SACD differences coming down to quality, as I've said many times. If there are purely sonic differences they the pale in comparison to those of recording. (The real advantage to SACD in multi-channel.)


The problems with the DBT supporters is so numerous that frankly I don't have the interest or time to go down that route again. A little knowledge is destructive and most of them have little knowledge. Typically they have a 16 trial test (where they came up with that is strange as in real science with subjects such a test would be laughed out of the room). ...

What is interesting is that in real science with human interaction high trials are critical - especially with sense perception choices. What the AES people NEVER tell people is that 6/10 ten times for a total score of 59/100 (one miss) is the EXACT same mathematically as a score of 9/10 to reach statisitical signifcance.
....

Have you read the AES journal article in question? I'll confess I have not. The "usual" critisisms of DBT might not necessarily apply in the case. I dare say pychoacoustic DBTs can be well-designed.



...
Now having said that I agree with you about small difference. The audiophile community is asking for trouble when you have quacks running around telling people that they can detect the difference of every cable going and that the differences are VAST. No if they were VAST differences then you should get 9/10 or 14/16. The fact that they can't tells you that the differences are subtle and very possibly indistinguishable.
...

Indeed! :smilewinkgrin:



....
PS Most people can not score 9/10 comparing Coke to Pepsi to the .05 significance level. And nobody has yet to differentiate bottled water from tap water - but clearly when you drink it NOT in a test environment you can taste the difference.
I'lll bet I could do at least 8/10. And when I come to beer I'll bet I could telll Sleeman Honey Brown for Coors Lite 1000/1000. :thumbsup:

[Smokey]

And nobody has yet to differentiate bottled water from tap water - but clearly when you drink it NOT in a test environment you can taste the difference.

Not in my area. One can taste/smell chlorine in tap water a mile away :)

[RGA]

Smokey

Leave your water out for a day or in the fridge and the Chlorine disappears.

Feaner - You should try the Philips Chocolate Porter - it tastes like chocolate looks like guiness. It's the best beer I've had. But you have to like Chocolate. I hate guiness - but this is very nice and no after taste. I'm not a big beer lover though so it might depend on your taste. I like the Honey Brown - it's my second favorite. Well third there was this Hoegarden one in Korea that was really nice.

Actually I believe it was JJ who said that a perfectly constructed test would require only one trial - he is correct. The fact that there are trials in the first place indicates a less than reliable result. JJ is the big AES acoustics guy the DBT guys support.

I agree with them when it comes to knocking down the claims of huge differences but they go way overboard on their conclusions.

[Smokey]

Smokey

Leave your water out for a day or in the fridge and the Chlorine disappears.



Did better than that, I put a faucet filter on my tap water. It really clean up the taste and odor :)