HiFi Choice DAC blind listening test!

[Ajani]

So this month HiFi Choice had a blind listening test of DACs done by a panel of reviewers...

The contenders were:

Benchmark DAC1
PS Audio DLIII,
Stello DA100 Signature
Lavry DA10
Chordette Peach
Arcam rDAC


And the clear winner was....


Drum-roll please...


The Benchmark

[Ajani]

Note that the Lavry and PS Audio both also scored 5 out of 5...

[Feanor]

Can somebody briefly explain Hi-fi Choice's blinding testing procedure? Though it's blind, I wonder whether it's strict, scientific "double blind" testing. I wonder to how extensive their test are, viz. number of participants, number of trials, etc. -- they test a lot of equipment: how thoroughly do they do it?

[poppachubby]

When you say winner, how so? I guess I am wondering the same as Feanor.

[Ajani]

When you say winner, how so? I guess I am wondering the same as Feanor.

Essentially the best sounding DACs in the test were the Benchmark, Lavry and PS Audio. They found the Benchmark and Lavry to sound fairly similar with the edge to the Benchmark and the PS Audio had a smooth sound (though less detailed) that everyone liked... They were also in love with the Benchmark's headphone amp and felt the Benchmark had the clear edge on features, despite lacking a USB input...

[3LB]

it needs to be quadruple blind in order for it to be valid:

1) the evaluators can' see what they're listening to
2) the facilitators can't know what the evaluators are listening to
3) neither the facilitator nor evaluators can know why their listening
4) neither the facilitator nor the evaluators can know the results of the test

[frahengeo]

And the clear winner was....


Drum-roll please...


The Benchmark

Ajani,

Who manufactures the DAC for Benchmark (ESS, Burr-Brown(TI), Analog Devices)?? Or do they design their own chip?

[frenchmon]

Well that was HiFi' choice' and their pro reviews which sometimes we cant trust. I would rather trust consumers who know a bit about this hobby...how about a consumer shoot out. The benchmark which was not apart of the shootout but was owned by the reviewer seems to rank the benchmark at the bottom of the list. checkout the cumulative ranking at the end.

http://www.head-fi.org/forum/thread/226919/review-apogee-mini-dac-vs-aqvox-usb-2-d-a-mkii-vs-stello-da100-hp100-vs-corda-opera

[Ajani]

Well that was HiFi' choice' and their pro reviews which sometimes we cant trust. I would rather trust consumers who know a bit about this hobby...how about a consumer shoot out. The benchmark which was not apart of the shootout but was owned by the reviewer seems to rank the benchmark at the bottom of the list. checkout the cumulative ranking at the end.

http://www.head-fi.org/forum/thread/226919/review-apogee-mini-dac-vs-aqvox-usb-2-d-a-mkii-vs-stello-da100-hp100-vs-corda-opera

I trust user reviews even less than pro ones... I really put this thread up just for the fun of it...

At the end of the day, all that matters is which one sounds good to you...

[phileserver39]

You know, I will save $200 and buy the PS unit in a month when I get my $ back from the US government. I have been swayed to the "DAC side" after buying the Audio Alchemy unit from a dude on Craigslist on Friday. Does an outboard DAC really make a difference?

No.

Just kidding.

I can't wait to hear all the detail and "air" that sits waiting on my hard drive after getting the top-of-the-line PS unit.

I will post my take on the upgrade once it happens.

Rock on,

JC

[Mr Peabody]

Was that the new Arcam DAC that is only $399. or $499.00? I wonder if they thought the Benchmark was twice as good as the Arcam.

[RGA]

The good thing about the Hi-Fi Choice blind sessions is that it is NOT a test which is hugely problematic on so many psychological fronts it's not even funny.

Hi-Fi Choice has a panel of listeners who listen to the units. They write evaluations as they listen to each piece. The levels are matched. No one sees the products.

The listening session panel includes reviewers manufacturers. There is no contact between the listeners and the person running the tests. It is not as strict as a DBT because they are not listening for whether a difference exists but what the differences are between several units (differences are therefore assumed to be there). Still a panel is good in the sense that if most of the panel is choosing one or two as best and one or two as worst then it is pretyt clear that there was a difference. Since the likelyhood would not happen if they all sounded the same - then there would be little to no agreement on the best or the worst of the lot. Results would usually wind up being completely random results with all of the units getting top and bottom scores by someone.

I would suggest though that such tests are always limited by time, what they're up against in a particular issue, and the rest of the system.

The Benchmark Studio Electric system at CES was very enjoyable. benchmark took a chance choosing a virtual unknown speaker maker (and they've done it a few years in a row)- so kudos to them.

I am surprised the Arcam didn't win - the Brits usually seem like they like to protect their own. Probably why a Blind session is useful. PS Audio is certainly interesting. The guys at my dealer brought in one of their power bar/regulator doohickeys and I have not heard them rave so much for a long while. http://www.psaudio.com/ps/products/detail/power-plant-premier_copy?cat=power

[02audionoob]

You know, I will save $200 and buy the PS unit in a month...

These sometimes turn up on Audiogon. Weinhart Design and Underwood Hi-Fi sell demo, return and trade-in units there. You could probably save at least another $200.

[Pat D]

The good thing about the Hi-Fi Choice blind sessions is that it is NOT a test which is hugely problematic on so many psychological fronts it's not even funny.

Hi-Fi Choice has a panel of listeners who listen to the units. They write evaluations as they listen to each piece. The levels are matched. No one sees the products.

The listening session panel includes reviewers manufacturers. There is no contact between the listeners and the person running the tests. It is not as strict as a DBT because they are not listening for whether a difference exists but what the differences are between several units (differences are therefore assumed to be there). Still a panel is good in the sense that if most of the panel is choosing one or two as best and one or two as worst then it is pretyt clear that there was a difference. Since the likelyhood would not happen if they all sounded the same - then there would be little to no agreement on the best or the worst of the lot. Results would usually wind up being completely random results with all of the units getting top and bottom scores by someone.

I would suggest though that such tests are always limited by time, what they're up against in a particular issue, and the rest of the system.

The Benchmark Studio Electric system at CES was very enjoyable. benchmark took a chance choosing a virtual unknown speaker maker (and they've done it a few years in a row)- so kudos to them.

I am surprised the Arcam didn't win - the Brits usually seem like they like to protect their own. Probably why a Blind session is useful. PS Audio is certainly interesting. The guys at my dealer brought in one of their power bar/regulator doohickeys and I have not heard them rave so much for a long while. http://www.psaudio.com/ps/products/detail/power-plant-premier_copy?cat=power

Since, as you say, they don't really establish that the equipment actually sounds different, the tests aren't worth much.

Do they actually run trials and obtain statistical results? If not, it is essentially a single trial and totally unreliable.

Do they do group tests? From your description, it seems so. If they don't do individual tests, then there is always the possibility of influence by other participants.

[RGA]

Since, as you say, they don't really establish that the equipment actually sounds different, the tests aren't worth much.

Do they actually run trials and obtain statistical results? If not, it is essentially a single trial and totally unreliable.

Do they do group tests? From your description, it seems so. If they don't do individual tests, then there is always the possibility of influence by other participants.

It's more of a double blind level match review. But on the other hand it takes the test out of it. And unfortunately engineers don't have enough psych background to know that with the T - the results they are going for is worthless. It doesn't prove A=B and it ALSO doesn't prove that the listener can or cannot distinguish between A and B - only that they can't do it under a stress test environment. A proper test incidentally would require only 1 trial - the reason there is more is because there is a problem with the test. If you took some psych courses you'd know that.

[phileserver39]

These sometimes turn up on Audiogon. Weinhart Design and Underwood Hi-Fi sell demo, return and trade-in units there. You could probably save at least another $200.
Thanks Noob, will look into that option.

[Poultrygeist]

I recently ordered the Audio GD NFB-12 with dual Wolfson 8741 chips which is one heck of a deal at $200 plus shipping.

http://www.audiocircle.com/index.php?topic=89165.0

[Feanor]

I recently ordered the Audio GD NFB-12 with dual Wolfson 8741 chips which is one heck of a deal at $200 plus shipping.

http://www.audiocircle.com/index.php?topic=89165.0
Damn! That is a good deal. I'm really tempted.

I see that Pacific Valve has the NFB-12 for $237. Since I've had a good experience with PV, I'll probably pay them the small premium.

[jrhymeammo]

I recently ordered the Audio GD NFB-12 with dual Wolfson 8741 chips which is one heck of a deal at $200 plus shipping.

http://www.audiocircle.com/index.php?topic=89165.0

I, second, what Feanor had said.
That does look like a great deal. Not sure why they marketed NFB=Non Feedback.

I'm not afraid to purchase gear from China anymore, but it is certainly easier to purchase from PV. I might just grab one and see how it goes.

Thanks

[phileserver39]

I just this morning picked up a brand-spanking new PS Audio Digital Link III unit for $620 through Underwood Hi Fi. I picked it up from their warehouse in Marrietta. My, my, my.....I am very glad to have purchased this DAC. From what I have read it gets really good after 60 hours of burn-in.

I had been using my ASUS STX sound card with upgraded opamps as well as a recently acquired Audio Alchemy V 1.0 outboard DAC that I found through Craigslist.

The PS Audio unit hands them both their walking papers and throws bricks at them just to really rub it in.

Me likey....

[Mr Peabody]

Sounds like a good deal, enjoy!

[phileserver39]

These sometimes turn up on Audiogon. Weinhart Design and Underwood Hi-Fi sell demo, return and trade-in units there. You could probably save at least another $200.

Yup- I just bought one from Walter for $620 brand new and was able to pick it up in Marrietta which is close to where I live.

I love this PS unit! Thanks for the suggestion!

[jrhymeammo]

Yup- I just bought one from Walter for $620 brand new and was able to pick it up in Marrietta which is close to where I live.

I love this PS unit! Thanks for the suggestion!

Great to hear you had great experience with Wally.
I had purchased from him 2wice and had not so positive experience.
Since he lives in Hawaii now, and his warehouse is in GA, I thought he was starting to lose control of his inventory, especially with conditions of his used gear.
I think he means well and tries hard to be one of the good guys, but post transactions had kept me from making a 3rd purchase.

JRA