double-blind CDP or DAC testing?

[Mr Peabody]

Well for me I sometimes don't clarify "difference" from "improvement" in sound of components. I agree that nothing is as dramatically different than brands or types of speakers. However, I feel the largest overall improvement in sound can come from a step up in source with all components being about equal at the time. Obviously, if you have a decent mid level CD player and using a budget speaker, then the speakers need to be addressed. The "weakest link" thing.

There wouldn't be a noticeable difference between players using an optical connection. I am a bit surprised that the Yamaha's internal DAC sounds better than the Arcam's analog output/DAC. I forget which model of Arcam you have though.

[Kevio]

Well, either way, within statistical bounds. But (as a scientist ) I trust you agree that DBTs cannot prove differences do not exist in any absolute sense regardless of method and rigor of testing.

In the absolute sense, we live under the uncertainty of quantum mechanics and so I can assure that there is variation in everything around us. The question is whether the differences are significant. The audiophile definition of significant is vanishingly minute.

Simple ABX tests, IMO, are ... uhm ... too simple to reveal anything less than gross differences.

I have to disagree. The limitation of ABX is that it only reveals audible differences. Price, brand, fit and finish and marketing factors are removed from the equation. Listeners enjoy these aspects of their hobby and are rightfully upset when these factors are excluded from an evaluation.

[Mr Peabody]

I have my doubts that anyone who is shallow enough to buy audio equipment for name recognition or status has the ability to have the kind of listening it would take to pass a DBT. We all when looking for good equipment will gravitate toward gear we know has a good rep, I'm not talking about that. I mean some one who buys a Mac just so their friends will see the faceplate like so much designer clothes or other things bought for show. It's purpose becomes something other than what it was designed to do.

[Feanor]

... The question is whether the differences are significant. The audiophile definition of significant is vanishingly minute.
...

The the hard-core audiophile any real, (and for that matter, imagined), difference is significant. Value is relative, and to the very well-heeled audiophile, it can be worth tens of thousands to achieve that vanishingly minute difference.

To further emphasize my point, you can have a scientifically rigorous double-blind ABX test that won't reveal minute but real differences. The A versus B pre-familiarization I discussed does not invalidate the test if the test itself is double blind; however, as the 'philes assert, it might help them distingush finer real differences. The significance of any real difference is entirely subjective.

[Ajani]

I have my doubts that anyone who is shallow enough to buy audio equipment for name recognition or status has the ability to have the kind of listening it would take to pass a DBT. We all when looking for good equipment will gravitate toward gear we know has a good rep, I'm not talking about that. I mean some one who buys a Mac just so their friends will see the faceplate like so much designer clothes or other things bought for show. It's purpose becomes something other than what it was designed to do.

Part of the problem is that just like with other luxury gear... some people really just buy for the prestige (how many Rolex or BMW owners really understand and appreciate the differences between those products and Citizen or Honda?)... those guys tend to make it more difficult for non-audiophiles to accept that there are any differences... Then on the other hand, the guys who actually can tell the differences have trained for years to pick up small details and cues, that most of us would not be able to pick up in an audition...

I remember a few months ago when E-Stat started a thread about a website that tests your ability to hear distortion (using brief excerpts from a Tracy Chapman song)... I also remember the fantastic score he got versus the relatively pitiful one I got... So I'm sure he could probably listen to my setup and even hear differences from swapping out cables, while I wouldn't even notice a difference... So for me to argue with him and say no differences in cables exist because I can't hear them would be silly... but it would also be silly for me to plunk down a thousand dollars on cable upgrades that I won't be able to hear (without years/decades of training)...

[Feanor]

I have my doubts that anyone who is shallow enough to buy audio equipment for name recognition or status has the ability to have the kind of listening it would take to pass a DBT. We all when looking for good equipment will gravitate toward gear we know has a good rep, I'm not talking about that. I mean some one who buys a Mac just so their friends will see the faceplate like so much designer clothes or other things bought for show. It's purpose becomes something other than what it was designed to do.

That first statement is pretty judgemental. I'm sure there are people who hear well but are willing to trade of very best sound for "bling". If such a person makes a conscious decission, it's a perfectly rational choice.

After all, most of his buddies will be more impressed by the bling than the sound, let's face it.

[Kevio]

...you can have a scientifically rigorous double-blind ABX test that won't reveal minute but real differences. The A versus B pre-familiarization I discussed does not invalidate the test if the test itself is double blind; however, as the 'philes assert, it might help them distingush finer real differences. The significance of any real difference is entirely subjective.

Maybe some people consider it cheating but I certainly do not believe that non-blind familiarization/training prior to a double-blind test invalidates the results. Training is an important and integral part of critical listening.

[Ajani]

Maybe some people consider it cheating but I certainly do not believe that non-blind familiarization/training prior to a double-blind test invalidates the results. Training is an important and integral part of critical listening.

Why would that be cheating? If the purpose of the blind test is to prove/disprove that differences perceived during a sighted audition are not real, then familiarizing yourself sighted should have no effect on the outcome of the blind test... You should be allowed as much time as you need to become familiar with the components being tested...

[Kevio]

Because people have different definitions of "significant". Some believe that significantly different means obviously better. As in, "You spent $10,000 on cables. Now it takes you three weeks to determine whether they actually sound any better. Wake up! You've been taken."

[audio amateur]

I remember a few months ago when E-Stat started a thread about a website that tests your ability to hear distortion (using brief excerpts from a Tracy Chapman song)... I also remember the fantastic score he got versus the relatively pitiful one I got... So I'm sure he could probably listen to my setup and even hear differences from swapping out cables, while I wouldn't even notice a difference... So for me to argue with him and say no differences in cables exist because I can't hear them would be silly... but it would also be silly for me to plunk down a thousand dollars on cable upgrades that I won't be able to hear (without years/decades of training)...

Would it be possible to have access to that test, could you or Stat direct me it to me?

[Ajani]

Because people have different definitions of "significant". Some believe that significantly different means obviously better. As in, "You spent $10,000 on cables. Now it takes you three weeks to determine whether they actually sound any better. Wake up! You've been taken."

I see...

For me, three weeks of auditioning to detect a difference can't be considered the significant... frankly, if I need to directly compare 2 items to determine that there is a difference, then it's unlikely I'll think the difference is significant...

[Ajani]

Would it be possible to have access to that test, could you or Stat direct me it to me?

Here ya go:

Audiophile Test!

[audio amateur]

AA, I did not say you were like my friend. I tend to ramble from one thing to the other without spacing the lines. I sincerely wondered what you listen for. I listen for detail, such as can I hear something in one that isn't in the other or less noticeable. I think we all listen for low and high end response. Is it tight and punchy, or does it boom, can I hear the bass line clearly etc. Do the cymbals hang in air and fade natural or not, are the highs piercing and will drive me out of the room soon. How the performers are presented in the sound stage.

...

As far as me being able to hear which CD player is playing when I walk into the room, I'm sure it would be easier on a disc I've heard on those machines before opposed to something I have not. In a DBT where I can compare one to the other side by side, I'm certain I could. I'd love to participate in one of those. Maybe I'm over confident in my ability, or maybe not.

Nice write-up.

[E-Stat]

Simple ABX tests, IMO, are ... uhm ... too simple to reveal anything less than gross differences.

The other fallacy regards tests where the devices are never used in the manner in which they were designed and used. The best example is E. Brad Meyer's test regarding the audibility of Redbook vs. DSD where players were never directly compared. Instead, he inserted and compared an "A/D/A" loop through three players. The idealized world is frequently different from the real world. The audible effects of jitter, for example, would never have been isolated using this *equivalent* test methodology. So much for controls.

This reminds me of Soundmind/Skeptic's "shunt" test where he proved (to himself) that $1.00 Radio Shack ICs are inaudible when he switches them through a buffered tape monitor loop used in conjunction with at least two more pair in the system. Or when he proclaimed that his H-K Citation 11 and Marantz 3800 preamps are audibly perfect because he could not tell any difference when he piped one through the other.

rw

[E-Stat]

Would it be possible to have access to that test, could you or Stat direct me it to me?

The story behind my involvement in that test mirror's Ajani's sig line:

Staying alive in this forum is just like prison, you have to either beat someone up or become someone's b!tch...

I've participated in audio forums as a sharing of ideas and benefited greatly from that exchange. Opinions will vary, but for some sadly it has become a contact sport. I just love posters who begin a post with a personal taunt. Like another one I've mentioned here, that's the way this one started:

Audiophile Repellent Test

Unfortunately for him, it backfired. I've since asked a couple of other ankle biters (two post here as well) to take the test and post their results. Naturally, they have not. I used a Dell laptop using its basic built in sound card and Shure E3c earbuds. You'll note that others have done substantially better than I with this basic test.

rw

[audio amateur]

The story behind my involvement in that test mirror's Ajani's sig line:

Staying alive in this forum is just like prison, you have to either beat someone up or become someone's b!tch...

I've participated in audio forums as a sharing of ideas and benefited greatly from that exchange. Opinions will vary, but for some sadly it has become a contact sport. I just love posters who begin a post with a personal taunt. Like another one I've mentioned here, that's the way this one started:

Audiophile Repellent Test

Unfortunately for him, it backfired. I've since asked a couple of other ankle biters (two post here as well) to take the test and post their results. Naturally, they have not. I used a Dell laptop using its basic built in sound card and Shure E3c earbuds. You'll note that others have done substantially better than I with this basic test.

rw

I'm not quite sure about the sig, but thanks for attempting an explanation. I'll have a look at the test and see what I come up with. I'll be at a slight advantage as I'll be using my DAC and the headphone amp on the cambridge unit. In-ears are Sennheiser CX-300s.
I also have a Dell, using it stock the noise floor using the buds nis nasty (I'm guessing because the buds have such a high sensitivity)