Finally Finished My A/B Trials...Here're the Results

[magictooth]

Well, I finally got around to doing the blind A/B trials that I had been planning to do for a while. I won't bore you with the details of the methodology which can be found here: Please Judge My A/B Methodology, Mtry Others.

I did add another wrinkle into the trials. My first A/B was a true David vs. Goliath match up with the Toshiba SD-1600 going up against the Classe DVD/CD-1. The first is valued probably around $0-20 while the second on the second hand market is around $1200-1500.

My results with this match up were likely a bit surprising to the naysayers. I could tell the difference each and every time. There was no contest between the two. In fact, you could say it was "a night and day difference." Now you'd think that I'd just stop here and proclaim victory for the yeasayers, but I'm not sure how well a 5 year old 5 pound DVD player is going to work after such a long time. I'd be willing to wager that the normal useful lifespan of this player has long been exceeded.

Hence, the second A/B trial that I did was with a new Toshiba SD-3950 that I picked up at Walmart for $89Cdn. This unit purports to have 192/24 DACs and to have the newest Crystal decoding chip. It would be interesting to see how this new unit stacked up against the slightly older Classe model.

I did the A/B and I was a bit surprised at the results. In 6 separate trials with different types of music, I found that the Toshiba and the Classe were virtually indistinguishable from each other. I say virtually because there was one selection where a difference could be detected. This selection is the Mozart Flute and Harp concerto. The Harp when played on the Toshiba had a digital type of edge to it; when played on the Classe, it sounded more like a harp. I could tell the difference between the two units in this one case.

In all the other cases, such as with Gershwin, Bruch violin concerto, Dvorak cello concerto, Norah Jones, and Stone Temple Pilots, I could tell no difference at all between the two units. I had what you could consider a very poor result if you are a yeasayer. I got 14/25 tries correct and I'm sure that this means nothing at all given the small statistcal sample. For the harp selection, I scored 5/5, but I also realize that the small sample size may not mean a whole lot.

In any case, I'm not here to argue statistical significance, but to let people know that the results that I got from my A/B trials showed that buying a very expensive CD or DVD player may not result in as good an improvement in sound as you'd like to see - especially considering how many times more you've spent on the unit compared to a mass market model.

My final conclusion is that there is a difference between different CD/DVD players, but, for me, the difference is not sufficient to justify the multiplication of cost.

[Thomas_A]

Well, I finally got around to doing the blind A/B trials that I had been planning to do for a while. I won't bore you with the details of the methodology which can be found here: Please Judge My A/B Methodology, Mtry Others.

I did add another wrinkle into the trials. My first A/B was a true David vs. Goliath match up with the Toshiba SD-1600 going up against the Classe DVD/CD-1. The first is valued probably around $0-20 while the second on the second hand market is around $1200-1500.

My results with this match up were likely a bit surprising to the naysayers. I could tell the difference each and every time. There was no contest between the two. In fact, you could say it was "a night and day difference." Now you'd think that I'd just stop here and proclaim victory for the yeasayers, but I'm not sure how well a 5 year old 5 pound DVD player is going to work after such a long time. I'd be willing to wager that the normal useful lifespan of this player has long been exceeded.

Hence, the second A/B trial that I did was with a new Toshiba SD-3950 that I picked up at Walmart for $89Cdn. This unit purports to have 192/24 DACs and to have the newest Crystal decoding chip. It would be interesting to see how this new unit stacked up against the slightly older Classe model.

I did the A/B and I was a bit surprised at the results. In 6 separate trials with different types of music, I found that the Toshiba and the Classe were virtually indistinguishable from each other. I say virtually because there was one selection where a difference could be detected. This selection is the Mozart Flute and Harp concerto. The Harp when played on the Toshiba had a digital type of edge to it; when played on the Classe, it sounded more like a harp. I could tell the difference between the two units in this one case.

In all the other cases, such as with Gershwin, Bruch violin concerto, Dvorak cello concerto, Norah Jones, and Stone Temple Pilots, I could tell no difference at all between the two units. I had what you could consider a very poor result if you are a yeasayer. I got 14/25 tries correct and I'm sure that this means nothing at all given the small statistcal sample. For the harp selection, I scored 5/5, but I also realize that the small sample size may not mean a whole lot.

In any case, I'm not here to argue statistical significance, but to let people know that the results that I got from my A/B trials showed that buying a very expensive CD or DVD player may not result in as good an improvement in sound as you'd like to see - especially considering how many times more you've spent on the unit compared to a mass market model.

My final conclusion is that there is a difference between different CD/DVD players, but, for me, the difference is not sufficient to justify the multiplication of cost.

Thank you for making the effort to do these tests. I agree with you that it is difficult to hear any differences between players. Sometimes, between certain pairs of CD players it is possible, but during normal music listening at home, very difficult.

Thomas

[okiemax]

Well, I finally got around to doing the blind A/B trials that I had been planning to do for a while. I won't bore you with the details of the methodology which can be found here: Please Judge My A/B Methodology, Mtry Others.

I did add another wrinkle into the trials. My first A/B was a true David vs. Goliath match up with the Toshiba SD-1600 going up against the Classe DVD/CD-1. The first is valued probably around $0-20 while the second on the second hand market is around $1200-1500.

My results with this match up were likely a bit surprising to the naysayers. I could tell the difference each and every time. There was no contest between the two. In fact, you could say it was "a night and day difference." Now you'd think that I'd just stop here and proclaim victory for the yeasayers, but I'm not sure how well a 5 year old 5 pound DVD player is going to work after such a long time. I'd be willing to wager that the normal useful lifespan of this player has long been exceeded.

Hence, the second A/B trial that I did was with a new Toshiba SD-3950 that I picked up at Walmart for $89Cdn. This unit purports to have 192/24 DACs and to have the newest Crystal decoding chip. It would be interesting to see how this new unit stacked up against the slightly older Classe model.

I did the A/B and I was a bit surprised at the results. In 6 separate trials with different types of music, I found that the Toshiba and the Classe were virtually indistinguishable from each other. I say virtually because there was one selection where a difference could be detected. This selection is the Mozart Flute and Harp concerto. The Harp when played on the Toshiba had a digital type of edge to it; when played on the Classe, it sounded more like a harp. I could tell the difference between the two units in this one case.

In all the other cases, such as with Gershwin, Bruch violin concerto, Dvorak cello concerto, Norah Jones, and Stone Temple Pilots, I could tell no difference at all between the two units. I had what you could consider a very poor result if you are a yeasayer. I got 14/25 tries correct and I'm sure that this means nothing at all given the small statistcal sample. For the harp selection, I scored 5/5, but I also realize that the small sample size may not mean a whole lot.

In any case, I'm not here to argue statistical significance, but to let people know that the results that I got from my A/B trials showed that buying a very expensive CD or DVD player may not result in as good an improvement in sound as you'd like to see - especially considering how many times more you've spent on the unit compared to a mass market model.

My final conclusion is that there is a difference between different CD/DVD players, but, for me, the difference is not sufficient to justify the multiplication of cost.

Thanks for doing the test and sharing the results. Most forum members, including myself, talk and argue about testing, but rarely do any. It's good to see someone making an effort. I don't know whether the test results suggest the Toshiba SD-3950 is an over achiever or the Classe DVD/CD-1 isn't much better than low-priced players. Loving a bargin, I would hope for the fomer, but my guess would be the latter. Is the Toshiba something special?

[E-Stat]

I don't know whether the test results suggest the Toshiba SD-3950 is an over achiever...

The Tosh is an overachiever. I use one in the HT system. By comparion, my GamuT CD-1 does not sound forty time better. Naturally, part of the price differential is due to the disposable-plastic build quality of the Tosh vs. the GamuT. The thick anodized aluminum faceplate of the CD-1 alone costs more than the entire Toshiba unit.

As an aside, Toshiba also has a universal player, the 4960, for just a few bucks more. That may prove to be among the best player bargains.

rw

[mtrycraft]

Yes, thanks for the report. Interesting

[Geoffcin]

Well, I finally got around to doing the blind A/B trials that I had been planning to do for a while. I won't bore you with the details of the methodology which can be found here: Please Judge My A/B Methodology, Mtry Others.

I did add another wrinkle into the trials. My first A/B was a true David vs. Goliath match up with the Toshiba SD-1600 going up against the Classe DVD/CD-1. The first is valued probably around $0-20 while the second on the second hand market is around $1200-1500.

My results with this match up were likely a bit surprising to the naysayers. I could tell the difference each and every time. There was no contest between the two. In fact, you could say it was "a night and day difference." Now you'd think that I'd just stop here and proclaim victory for the yeasayers, but I'm not sure how well a 5 year old 5 pound DVD player is going to work after such a long time. I'd be willing to wager that the normal useful lifespan of this player has long been exceeded.

Hence, the second A/B trial that I did was with a new Toshiba SD-3950 that I picked up at Walmart for $89Cdn. This unit purports to have 192/24 DACs and to have the newest Crystal decoding chip. It would be interesting to see how this new unit stacked up against the slightly older Classe model.

I did the A/B and I was a bit surprised at the results. In 6 separate trials with different types of music, I found that the Toshiba and the Classe were virtually indistinguishable from each other. I say virtually because there was one selection where a difference could be detected. This selection is the Mozart Flute and Harp concerto. The Harp when played on the Toshiba had a digital type of edge to it; when played on the Classe, it sounded more like a harp. I could tell the difference between the two units in this one case.

In all the other cases, such as with Gershwin, Bruch violin concerto, Dvorak cello concerto, Norah Jones, and Stone Temple Pilots, I could tell no difference at all between the two units. I had what you could consider a very poor result if you are a yeasayer. I got 14/25 tries correct and I'm sure that this means nothing at all given the small statistcal sample. For the harp selection, I scored 5/5, but I also realize that the small sample size may not mean a whole lot.

In any case, I'm not here to argue statistical significance, but to let people know that the results that I got from my A/B trials showed that buying a very expensive CD or DVD player may not result in as good an improvement in sound as you'd like to see - especially considering how many times more you've spent on the unit compared to a mass market model.

My final conclusion is that there is a difference between different CD/DVD players, but, for me, the difference is not sufficient to justify the multiplication of cost.

You busted your butt and found out that good players are easy to spot with good recordings, nearly all of the time.You also found out the nasty other thing; That 90%-95% of all CD recorded music will not be served up any better between most CD players.

Is a Z06 Corvette worth 5x a Hundai? Not if your only going to drive on pot-holed roads to the supermarket. Or even just around town. On the few roads that will let you do what it's capable of, then it's magic. Is it worth it? To some people, the answer is obviously yes.

[Thomas_A]

You busted your butt and found out that good players are easy to spot with good recordings, nearly all of the time.You also found out the nasty other thing; That 90%-95% of all CD recorded music will not be served up any better between most CD players.

Easy to spot with good recordings? I don't see where that is written in this test.

[NickWH]

Unless I'm blind, I didn't see a list of associated equipment used for these "tests." I only see references to the actual players. Just curious...

[mtrycraft]

Unless I'm blind, I didn't see a list of associated equipment used for these "tests." I only see references to the actual players. Just curious...

That would have just satisfied some and dissapointed others. Besides, it is not that relevant.

[magictooth]

Unless I'm blind, I didn't see a list of associated equipment used for these "tests." I only see references to the actual players. Just curious...

I didn't think that it was that important, but the equipment used was as follows:

Audiomat Solfege Reference
Tannoy Studio 12 DMT II monitors
Some type of customs ICs ($15/pair)
Silver something or other speaker wire ($2.50/ft)

[Resident Loser]

...and it really isn't...there is that contingent, however, who will pooh-pooh any results that seem to erode their belief system...your speakers, your electronics...he!! the paint color in your listening room...all of these things can be grasped at to cause discredit.

Is their any guess as to why mtry barely lays claims to listening to a boombox...

jimHJJ(...note, I didn't say owns one...)

[NickWH]

didn't think that it was that important

Actually, it is very important. Without resolving and detailed equipment, how could one determine one component from another? I'm not saying this is the case here, just in general.

I have found in the past few years in my pursuit of happines that I have found much more joy out of helping others with no expectation of anything in return than I have out of spending large sums of money on stereo equipment, but that is just me.

Great, you deserve a medal.

[Beckman]

Hence, the second A/B trial that I did was with a new Toshiba SD-3950 that I picked up at Walmart for $89Cdn. This unit purports to have 192/24 DACs and to have the newest Crystal decoding chip. It would be interesting to see how this new unit stacked up against the slightly older Classe model....My final conclusion is that there is a difference between different CD/DVD players, but, for me, the difference is not sufficient to justify the multiplication of cost.

Thank you for taking the time to run a test. I am not suprised by the results, kind of wished I would have known this before purchasing my $400 Cambridge D500SE. Oh well, at least I helped out the economy.
Hmmm,
$89 vs. $1200.

[mtrycraft]

Oh well, at least I helped out the economy.
Hmmm,
$89 vs. $1200.

Yes, that is a worthwhile endevor when it is other people's money