Trying out a subjective Rating System - LONG [Archive] - Audio & Video Forums

PDA

View Full Version : Trying out a subjective Rating System - LONG



RGA
09-01-2004, 09:40 PM
Okay I need a life I realize but I have been sick and indoors and figured what the hell. I have done several reviews but I'm going to stop get some feedback perhaps. I realize much of this is arbitrary so far - but hey so are most all rating systems.

Speaker rating system

Speakers are rated into several categories of certain aspects of the speaker’s sound which I have tried to account for in my listening sessions. For example, the “Bass depth” rating is not just bass depth, but also how well the speaker performs with what bass it does offer. Stand-mount speakers are usually at a disadvantage in this regard but this is a fact of life and no compensation is made for those wishing to add subwoofers. If a speaker has no deep bass it has none and will score a zero. By all means if this is unimportant disregard the bass depth aspect and purchase a sub-woofer.

Further, I have my own personal tastes in what I look for in loudspeakers. I prize the overall musical event more so than trying to rank individual aspects of treble or bass or imaging. As such in the areas most important to me I have added modifiers. So each aspect of sound such as imaging or tonality I have scored the product out of ten. However, because the Midrange is more important to me than Deep Bass, I have added an X3 modifier to the midrange so that a score of 8/10 with the X3 modifier will now be weighted 24/30.

There are eleven aspects of sound that I have used for my ratings with a total out of 200. Each aspect is scored out of ten, with the aforementioned weighted modifiers. This is to be as objective as I can be. This also allows the reader who may value Imaging and Sound-staging more than I do to weight that aspect more and perhaps high frequencies less.

An overall percentage is given of the speaker's performance. Percentages above or below the middle of the range expected will affect the overall grade given to speakers. I have developed a base line of where I expect a speaker's performance percentage to be given relative to the price of the speaker using the following table.

Based in Canadian Dollars(Subtract roughly 30% for US figure)

Up to $200 (30-40%)
$200.00-$500.00 (40-50%)
$500.00 - $1,000.00 (50-60%)
$1000.00 $2,000.00 (60 -? 70%)
$2,000 - $5,000.00 (70-80%)
$5,000.00 - $10,000 (80%- 85%)
$10,000.00 and up (85%-100%)

Note: Speakers receiving 90%-100% are given a Reference Recommended Tag.

For example, a speaker at $600.00 that scores 65% is performing well above expectations for its price range and will receive a Best Buy Tag.

A speaker performing in the upper part of its expected percentile will receive a Recommended Rating. A speaker performing within its expected percentile price range but at the lower end will receive no tag. Lastly, an expensive speaker performing below the expected performance percentile range will receive a Question Mark. This is my question to the manufacturer as to why the speaker is so highly priced given the performance offered up. It may very well be the case that some of these speakers outperform speakers receiving a Best Buy or Recommended tag at lower levels; however, it is not, to put it bluntly, performing to the level of similarly priced speakers or my expectations. It should be stressed that a question mark does not mean it’s necessarily a BAD speaker.

Note: Any speaker ranking 80% or better will be given a FREE from being given a Question Mark. While I may deem it overpriced, high end speakers to me have attained remarkable musicality which trumps the details.

A Speaker with a Best Buy rating is not necessarily a better product than one which receives a Recommended tag. However, a Best Buy tag in the same price range may be superior – if in doubt check the overall percentage.

Do not be put off by relatively low percentage scores. Nothing is likely to score a 100% and very few will rate higher than about 85%. I consider speakers rating 75% or better to be high end, 70% or better for bookshelf speakers. These are rough – obviously if a Speaker has a lot of 8s but did poorly in bass response and you like subwoofers anyway it should still be highly considered. Also if build and finish are critical to you then certainly taking a 63 over a 67 makes sense if the latter is a butt ugly beast.

Note: All speakers are judged on their sound when I have auditioned them. I have no interest in the look, finish, price or name recognition or even build quality of loudspeakers. Unless noted expect the build quality to be good for almost all loudspeakers built today at pretty much all price points. I am sure there are the odd speakers with gross defects, but that I suspect would be quite rare. What is attractive for your home is up to you just like the sound of the speaker.

My ratings are meant for me as a hobbyist and for your entertainment. My reviews like all reviews are only any good if you agree with them. Perhaps you will be inspired to listen to something you have not heard before – and that is the best thing any audiophile can do. Spend days and days listening and comparing against other speakers. Listen in different locations with different gear in good rooms and in bad. Listen with some known recordings knowing you will have expectation bias so try listening to genres you know but unfamiliar recordings and find the speaker that can delineate differences better.

PS. Before you ask -? Yes! I have modified and integrated the enjoythemusic.com system of rating along with Hi-Fi Choice value rating tagging system. Why re-invent the wheel?

PSS: All Prices are estimates as they are sold in Canada. Some speakers sold in the US may be sold at par with the Canadian Dollar. If a speaker is sold in your area at a different price this will heavily impact which TAG the speaker receivers as the value will change.

B&W DM 604S3 ~$2,000.00Cdn - $1500.00US
3-way vented-box system -- 39Hz - 22kHz ± 3dB on reference axis.

The B&W 600 line has been around for years and has improved a fair bit since the original models. This 3 way four driver system clocks in at roughly $2,000.00Cdn and offers much for the money. While the treble is a tad unruly and the bass sounds a bit thumpy the overall presentation in the mid-band is very good with an open less boxy sound than some more expensive speakers I have auditioned with similar looking cabinets. This B&W 600 series while less refined in the treble to the 700 line and does other certain technical things less well like imaging, manages to sound more macro-dynamically superior in presenting larger scale music or rock music in a more satisfying manor.

Listening to Sarah McLachlan's Afterglow album synthesizer and drums are more vibrant though more sloppy than the 703, nevertheless the sound is bigger and more rotund and is faster on the beat. The 602S3 is a wonderful stand-mount speaker and if you like sub sat matching may be a cost saving advantage and could yield better results for you. However the 604S3 and 603S3 both offer much of that 602S3 sound in a nice package but with added bass depth and the ability to play a fair bit louder. Given the price of entry into the 700 line, the 604S3 offers a much superior value for money with its superior drive.

Ratings are in absolute terms rated against all speakers of all price ranges and ratings are not inflated because they are deemed budget speakers.

Deep Bass (10hz - 30hz) ____________________________4/10
Bass (30hz - 80hz)_________________________________ 5/10
Midbass (80hz - 200hz)______________________________6.5/10 X2
Midrange (200hz - 3khz)_____________________________ 7.5 /10 X3
High Frequency (3khz - up)___________________________7.5/10 X2
Attack (including macro-dynamics)_____________________7.5/10
Decay____________________________________________6/10
Resolution (including micro-dynamics)__________________6/10 X2
Imaging and Soundstage ____________________________ 8/10
Tonality__________________________________________ _7/10 X2
Cohesiveness (integrity of the musical event)_____________6/10 X4

Overall__________________________________________ 131/200 = 65.5%
Rating = Recommended

http://www.bwspeakers.com/index.cfm/fuseaction/products.models/label/Model%20DM604%20S3

B&W Model Nautilus $60,000.00CDN - $40,000.00US
25Hz - 20kHz ± 0.5dB on reference axis (-6dB at 10Hz and 25kHz)

When I first saw these B&W speakers my first thought was holy Bat-Speakers Batman as they certainly would be right at home in the Bat cave. And you will certainly need a Cavernous room to house the required 8 power amplifiers to drive these Statement products from Bowers and Wilkens. There is little to say about speakers performing at this kind of level but to be transported by its ability to render the complete musical event while also being a work of art. Some have hailed it the best speaker currently available and there is reason for making such a statement that goes beyond the sticker shock. These speakers offer up incredible low end bass response and probably require a step-up transformer to your home in order to run them but it's only money right? There is of course no way to justify perhaps statement products from the likes of B&W and Dynaudio but to tip your cap to the cost no object engineering and quality of craftsmanship. These speakers manage to be tuneful, and offer the complete musical event in a breathtaking manor and technically immaculate from Piano to today's pop music. You will need a lot of power and a lot of room. I regret only listening to these speakers twice as they're no longer in my area. Yes, a metal tweeter I like. There is no sense of a disconnection between drivers nor is the treble leaning to a ringing effect when pushed. These speakers thunder, as they should.

Ratings are in absolute terms rated against all speakers of all price ranges and ratings are not inflated because they are deemed budget speakers.
Deep Bass (10hz - 30hz)____________________________ 10/10
Bass (30hz - 80hz)_________________________________ 9/10
Midbass (80hz - 200hz)_____________________________ 10/10 X2
Midrange (200hz - 3khz)_____________________________10/10 X3
High Frequency (3khz - up)___________________________9.5/10 X2
Attack (including macro-dynamics)____________________ 9/10
Decay____________________________________________8/10
Resolution (including micro-dynamics)__________________9/10 X2
Imaging and Soundstage_____________________________10/10
Tonality__________________________________________ _9/10 X2
Cohesiveness (integrity of the musical event)_____________9/10 X4

Overall___________________________________________ _187/200 = 93.5%
Rating = Reference Recommended.

http://www.bwspeakers.com/index.cfm/fuseaction/products.models/label/Model%20Nautilus

So does any of this make sense. Yes I had WAAAAAY too much time on my hands - I made up 23 so far though I have not done a write up for most of them just the ratings part. - LOL. :confused: :) :confused:

Woochifer
09-01-2004, 09:50 PM
Good gawd!? You must have a pretty nasty summer flu if you got the time to come up with THIS. Rather than coming up with a rating system, seems to me like you're trying to start up an audio magazine and pitch for some venture capital! If you were into American college football, I might think that you were also in the process of coming up with a new BCS formula.

Lot of interesting points raised, though I shudder to think what you might do if you start bringing up standard deviations and normalization curves to modify your scoring system!

RGA
09-02-2004, 08:08 PM
Yeah I know it seems a bit overblown. And I don't have the baseline to formulate this curve rating system.

But I guess I'm tired of just reading a review that basically says I liked it and so might you depending on positioning. I also don't like magazines that use measurements but in the end the subjective review is ALWAYS I liked it go check it out. So even if the measurement sucked the reviewer liked it - well that ure tells me how useful the measurement was in real world listening.

I like what enjoythemusic does but it's too involved. I like Hi-fi choice but they're way bigger than me. It would be fun to start a magazine. But personally I would rather takeover the Playboy Mansion than do Audio Reviews.

92135011
09-02-2004, 09:39 PM
what would you rate your J/spe based on your scale

RGA
09-02-2004, 11:21 PM
They're coming in stages.

I spent the day at Soundhounds listening to tunes and setting up some good systems(and lesser ones) with Terry.

Mike - the guy that wrote the review of the E/Kit in the AA speaker forum was in and I met him. It was quite startling to start with the E/L with a Rotel Integrated amp and Teac CD player. Then Terry changed over to the Oto Push Pull and it was startling. Much deeper bass a smoother top end but MORE extended not less. He then changed to the CD3.1 and they say CD players don;t make a difference - well whoever They is They need new speakers. Then we went to the Single ended version of the of the Oto and all those They's need to hear real bass response that amplifiers like Bryston can't get to. The measly 9 watts was more than enough on my Deodata and Pat Metheny Jazz tracks with Dynamics. There was a forward to back stage that the Rotel/Teac didn't present.

Then we changed cable from LX to the SPE and as you may know I'm not a cable supporter. I dunno but room consensus seemed to agree that bass had deepened(I'd want more time to be sure). Then we went to the 8 watt Meishu and the bass deepended once more. You can't run garbage in front of these speakers.

The E Sec silver rates higher for me at considerably less money - and the E/Spe and J/Spe don't quite make it. However both I rate higher than the N801.

While we were listening and making comments - Terry just kept saying "I can't say anything about my other brands - Must stay in business." LOL. He's been selling there for 30 years and his two ex-partners own Hi-Fi Center and Sound Plus. He is happy to have purchased the TT3 used for $12,000.00 and is saving for an AN amplifier. I had to laugh - they were playing a guitar piece in the main room and you know how they have that couch and short wall there - someone thought that it was a live guitar player until they realized it was the E.

I listened to the B&W604S3 with my great little Jazz album and I had on the Deodato 2001: Space Oddysey Jazzed up version along with some others. Anyway I was listening to this with Mike and the dynamic scale made me laugh - I felt embarrassed for the B&W. SO I brought Mike in and played it. Now the B&W had about double the price front end hooked up with a Linn Pre Power amp and CD Player. The sound was boxy and lifeless with almost no bass microdynamics whatsoever. In other words completey void of presenting the musical event in relation to the E/Spe. Basically you can't go back. Though the 604 is quite good for the money.

Their repair person will be sent with his family to the Audio Note plant to learn how their computer system works and how to match the speaker cabinets and drivers. Then when he returns he will be building J and K's. The E's can't be done because of the fitting process.

I found a lot of interesting stuff today. The prototype turtable is made in the Ferrari plant using their equipment - can you say tight tolerances.