Please Judge My A/B Methodology, Mtry Others

**magictooth** · 05-11-2004, 10:30 AM

Well, I convinced the wife to help me out one more time to do some A/B comparisons between CD players. She's not going to do 45 trials this time, so I can't say how many tests I'm going to do, but I hope it'll be at least 10-15.

Here's the methodology. Please critique and add any ideas that you might think would help.

I've got two identical CDs. One is going to be loaded into the Toshiba SD-1600, the other into the Classe DVD/CD-1. The prices are $200 (probably worth $20 now) and $3500 (worth $1300-1500 now) respectively. They're both hooked up to my Audiomat Integrated amp with exactly the same type of IC. One is going into line 4, the other line 5.

Both CDs will be playing at the same time, but the input selector will be changed (or maybe not), and I'll try to identify which DVD player is playing. In between the changing input, the mute switch is going to be clicked, and she's going to change the volume knob at random. This way there's no volume differences to be able to key off of. When she's changing the input, I'll have my eyes closed with my hands over my ears so I can't hear any click - although there's virtually no noise if she turns the properly anyways.

I'm going to cover the displays of each unit so I can't see if there's a discrepancy between the playing times of each CD.

Is there anything else you'd like to add to this method, mtry?

**996turbo** · 05-12-2004, 08:43 AM

The volume changes will make it kinda hard.
We usually try to level match that way you are not being confused by the volume changes. I am interested to hear what you think as some others think there will be no difference. Post your findings for us.

**magictooth** · 05-12-2004, 11:50 AM

Originally Posted by 996turbo

The volume changes will make it kinda hard.
We usually try to level match that way you are not being confused by the volume changes. I am interested to hear what you think as some others think there will be no difference. Post your findings for us.

I'm trying to make it harder to A/B.

**996turbo** · 05-12-2004, 12:09 PM

I understand. You want to show that regardless of the volume you will be able to tell the players apart.
I personally do not think a test is even needed for that. I have challenged Mytrcraft and the others to a challenge but they are obviously afraid to respond.

Good luck with your test and please inform us of the results.

**jeskibuff** · 05-12-2004, 12:13 PM

Originally Posted by magictooth

I'm trying to make it harder to A/B.

You actually want to make it EASIER to A/B. Throwing the volume level changes into the mix will just make it more confusing. Make sure you've got the input gains matched so the volumes are as close to each other as possible, as the players most certainly will have differing output levels. Forget the muting, too. Just have your wife cough when she makes the switch (or PRETENDS to make the switch).

The timing will be your toughest trick, but quick jabs at the pause button on the unit that's slightly ahead in time will help to get both units synchronized.

**magictooth** · 05-12-2004, 12:45 PM

Originally Posted by jeskibuff

You actually want to make it EASIER to A/B. Throwing the volume level changes into the mix will just make it more confusing. Make sure you've got the input gains matched so the volumes are as close to each other as possible, as the players most certainly will have differing output levels. Forget the muting, too. Just have your wife cough when she makes the switch (or PRETENDS to make the switch).

The timing will be your toughest trick, but quick jabs at the pause button on the unit that's slightly ahead in time will help to get both units synchronized.

I don't have any voltage adjustments available to me. We can't really match the volumes because the switch on the amp is just a knob - no numbers or graduation markers. I think there is about a 4-5 dB difference when you play a tone and measure with the SPL. I don't want to have any volume difference bias in this experiment hence, the variable volume.

About the mute switch, both discs are identical discs with a variety of music. They should be fairly easy to synchronize. Just hit play and then pause, and then play once we're set to go. The mute will allow us to have an interval to allow for the seamless switching or pretend switching to go off without a hitch.

**mtrycraft** · 05-12-2004, 09:51 PM

I don't have any voltage adjustments available to me. We can't really match the volumes because the switch on the amp is just a knob - no numbers or graduation markers. I think there is about a 4-5 dB difference when you play a tone and measure with the SPL. I don't want to have any volume difference bias in this experiment hence, the variable volume.

STOP right there.!!!!
If you cannot match levels to 0.1dB spl at several frequency bands, your test is flawed before strating.

If you cannot synchronize the two CD palers to withing a few milli seconds of time which you cannot, it is flawed.

Best if you play one song on one player and the same song on the next player as that is all you can do. But, without level matching, forget it.

About the mute switch, both discs are identical discs with a variety of music. They should be fairly easy to synchronize.

To a few milli seconds? If not, you can tell one from th eother just from the time delay.
Back to square one.

**996turbo** · 05-13-2004, 03:32 AM

Miliseconds?

.1 dB

What the hell

**FLZapped** · 05-13-2004, 05:08 AM

Originally Posted by 996turbo

Miliseconds?

.1 dB

What the hell

Yep, those nasty JNDs again. . . . -Bruce

**magictooth** · 05-13-2004, 05:52 AM

Originally Posted by mtrycraft

STOP right there.!!!!
If you cannot match levels to 0.1dB spl at several frequency bands, your test is flawed before strating.

If you cannot synchronize the two CD palers to withing a few milli seconds of time which you cannot, it is flawed.

Best if you play one song on one player and the same song on the next player as that is all you can do. But, without level matching, forget it.

To a few milli seconds? If not, you can tell one from th eother just from the time delay.
Back to square one.

I want some help designing a good A/B test with the equipment that I've got. I think that I've got a good methodology. Please indicate to me why a random volume change would make it easy to identify each CDP. If the master volume changes by up to +/- 15dB after each sample how will that be a key? I stress that the volume change will be random or possibly none at all. I won't have any idea about it.

About the synchronization to milliseconds, that is going a bit overboard. The mute switch will be clicked for between 5-10 seconds so any differences in synchronization of milliseconds, centiseconds, or deciseconds will be lost in the silent times. I'm going to have the displays covered so I can't see the times on the units either. For the start of the discs, I'm going to hit play, then back, then I'll pause both units, hopefully at right around the same time. I'm sure that I'll be able to get less than a 0.25 second discrepancy in the units. With the displays covered and a very long interval between samples, will that be sufficient to nullify any time discrepancy issues?

As an aside, I remember you writing that acoustic memory is just seconds at most. What exactly do you mean by that?

**996turbo** · 05-13-2004, 06:17 AM

The problem magictooth is that any test you devise if it comes out contrary to Mytrs beliefs will be criticized. The whole millisecond thing is proof. I have repetedly asked these people to participate in a test. They refuse because they have no experience with the gear I will be testing and do not know if their beliefs will stand up against a test. The only thing they can do is sit in their blind and shoot down things they are not willing to step up to the plate and participate in. They have proven that the only thing they can do is spit out things they have read. You are wasting your time asking him how to do the test. You could do it exactly as he states and he will find some unarguable bull to discredit your work.

**996turbo** · 05-13-2004, 06:36 AM

Mytr is somewhat correct in his length of Auditory memory as it is a subset of sensory memory. The problem is that things can and do move from sensory to short term (stm) to Long term memory (ltm).

**Resident Loser** · 05-13-2004, 06:55 AM

...you MUST remove all variables from the equation to even things out...

With re: to volume, the louder unit will always seem to sound better. This would skew the results of a one-on-one," which is better?" scenario...adding volume shifts for tests of your intent, which, if I read you correctly, is simply identifying units, it would seem to cloud the issue entirely...

As is true with "sync"ing, however, while one would think muting should mask the some potential identifiers, it has problems of it's own. Short term memory, especially regarding audio, is easily confused, thwarted, misled(I can't quite choose a simple term, so pardon me and I hope you get my drift)...long intervals between samples will only exacerbate this.

Everything should be seamless and strict protocols must be followed for any results to be meaningful...and of course, to stand up to the strictest of scrutinies, they should be repeatable.

jimHJJ(...BTW, ignore the noises you may hear from the duck-hunting, ball player...)

**magictooth** · 05-13-2004, 07:32 AM

Originally Posted by Resident Loser

...you MUST remove all variables from the equation to even things out...

With re: to volume, the louder unit will always seem to sound better. This would skew the results of a one-on-one," which is better?" scenario...adding volume shifts for tests of your intent, which, if I read you correctly, is simply identifying units, it would seem to cloud the issue entirely...

As is true with "sync"ing, however, while one would think muting should mask the some potential identifiers, it has problems of it's own. Short term memory, especially regarding audio, is easily confused, thwarted, misled(I can't quite choose a simple term, so pardon me and I hope you get my drift)...long intervals between samples will only exacerbate this.

Everything should be seamless and strict protocols must be followed for any results to be meaningful...and of course, to stand up to the strictest of scrutinies, they should be repeatable.

jimHJJ(...BTW, ignore the noises you may hear from the duck-hunting, ball player...)

I did an A/B where the difference between units was about 2dB. The louder unit being the the above mentioned Toshiba SD 1600. I scored 15/15 in telling which one I like better even though the cheapo one was louder. IMO, that debunks all the sayings that louder=better.

With regards to the sync issue, I can press the pause button at exactly the same time, and play once again at the same time, but there may be internal player issues such as how long the delay is for the commands to be executed, etc. That's why the delay in between.

Basically what I'm trying to do is to make it MORE DIFFICULT to tell the differences between players. If I'm reading you correctly, you are also saying that it will become more difficult to tell between players given my above listed protocols. Please correct me if I'm wrong.

Also let me clarify the hypothesis: Many people on this board believe that there are NO sonic differences AT ALL between any DVD player. I want to prove or disprove this idea with an A/B comparison with the equipment and manpower that I have available. I know that I can't accurately level match the 2 units, hence the possible variation of master volume. I realize that I may not be able to sync the units to a couple milliseconds (although I should get very close certainly less than .0.25 sec), hence the longer delay.

The music on both discs comes from a variety of sources. Thank goodness for CDRs. There are selections from 6 different discs, and the music from each disc plays at different volume levels so that even if I had a voltage attenuator, I wouldn't be able to match the output voltage in both machines so that the volume in dBs is exactly equal.

**Resident Loser** · 05-13-2004, 07:38 AM

"...Mytr is somewhat correct in his length of Auditory memory as it is a subset of sensory memory. The problem is that things can and do move from sensory to short term (stm) to Long term memory (ltm)..."

...that seem to lend credence to all the "ex-spurts" regardless of what they "spurt" about...

Whether iconic, echoic or hapatic, sensory memory IS passed to short term memory which then uses it for temporary recall of the information under process. However, short term memory decays in roughly 200ms. and has a finite capacity. Information is transferred to long term memory only by constant exposure to a stimulus or information. This is a process that happens over time and would seem render it incompatible, in fact being at odds, with the type of testing we are speaking of.

You may recall the words, the melody and possibly salient characteristics of a specific performance(perhaps a misplaced note or the "sound" of an instrumental solo) but these are memories we aquire via repetition, over time...

jimHJJ(...now please be quiet, the adults are engaged in civil conversation...)

**Resident Loser** · 05-13-2004, 08:22 AM

...hypothesis is flawed at the outset...CDPs, DVDPs, wires and cables, in fact, any component you care to test, can sound "different". The trick(and the source of all the unrest)is to consistently identify the units being compared under DBT conditions and result in an outcome that is statistically relevant.

"...That's why the delay in between..."

Here's where the auditory memory comes in...see my response to Porsche-boy...

"...Basically what I'm trying to do is to make it MORE DIFFICULT to tell the differences between players..."

Beginning with sources of varying qualities(loudness, etc.) giving you no baseline reference whatsoever, "more difficult" seems an understatement...

jimHJJ(...I'm sorry, it just seems to make no sense...)

**996turbo** · 05-13-2004, 12:14 PM

I have attempted to engage in meaningful dialogue with you but you insist on making little snide comments. If you are the adult you are I would suggest that you stop being such an a$$hole. You people criticize people on spelling or navigating this site. That is childish. Here is a bit of psychology for you. Which defense mechanism is used when a person belittles another?

**Resident Loser** · 05-13-2004, 12:18 PM

..are you referring to?

jimHJJ(...if ya can't stand the heat...)

**996turbo** · 05-13-2004, 12:56 PM

jimHJJ(...now please be quiet, the adults are engaged in civil conversation...)

jimHJJ(...if ya can't stand the heat...)

I guess you stm is as good as your hearing

**Resident Loser** · 05-14-2004, 05:16 AM

...that your misuse of vocabulary is indicative of your perceptive abilities...

Where in those two statements was I:

False, deceptive, dishonest or insinuating?

Your entire demeanor and attempt at insult is sophomoric, hence the "adult" reference...merely stating the obvious...

The "heat" phrase is quite simple...you seem to have no problem with dishing it out, but when the shoe is on the other foot...? You think nothing of casting aspersions on myself, Skeptic, woodman and of course, Mtry...and further you applaud posts by those whose only claim to fame seems to have been the reading of "101 Ways To Use the Word @$$"...your support indicates tacit agreement and thus complimentary induction to the "Birds of a Feather" club with all of it's perks...

If anything I've been quite blatant and overt! I reserve snide and sarcastic remarks for those whom I think can appreciate such subtleties...

jimHJJ(...and that ain't you!...hmmm...is that snide?...nah! Just an insult...)

**WmAx** · 05-14-2004, 06:18 AM

Originally Posted by 996turbo

I have attempted to engage in meaningful dialogue with you but you insist on making little snide comments. If you are the adult you are I would suggest that you stop being such an a$$hole. You people criticize people on spelling or navigating this site. That is childish. Here is a bit of psychology for you. Which defense mechanism is used when a person belittles another?

Is this the same 996turbo who started the 'mtrycrafts is an idiot thread', which has the sole purpose of belittling someone in a childish manner? Is the irony not apparent?

LMFAO

-Chris

**996turbo** · 05-14-2004, 07:23 AM

Originally Posted by WmAx

Is this the same 996turbo who started the 'mtrycrafts is an idiot thread', which has the sole purpose of belittling someone in a childish manner? Is the irony not apparent?

LMFAO

-Chris

The purpose was to point out that alot of us think there are differences in equipment. I suggest you read the post and not get cought up in the title. You will notice that in the meat of the thread I did say that I did not think he was an idiot I just disagreed with his position.

**WmAx** · 05-14-2004, 07:45 AM

The purpose was to point out that alot of us think there are differences in equipment. I suggest you read the post and not get cought up in the title.

I suggest you take the time to think about what you doing. Your point is made very poorly.

You will notice that in the meat of the thread I did say that I did not think he was an idiot

Hmmm. Ok.

996turbo, you know what you are? Let me give you a clue........

YOU STUPID MORON. You REALLLLLLLLY STUPID MORON. Your really a STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID STUPID MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON MORON.

STUPID MORON!

Oh, sorry. I did not mean anything at the beginning. Just look in the meat of the post. In case you can not see the picture I posted, in the center of the meat it says: "996turbo is NOT really a stupid moron, you just have to look in the meat of the post. :-P"

-Chris

**996turbo** · 05-14-2004, 07:51 AM

What did I do to you personally for you to be such an ass. It is it in your nature? If so I will understand. I do agree that this thread and others has gotten out of control. I think if you look at all the post yours are just as vitriolic as others.When you say that my demeanor is sophmoric I suggest you look at yourself and judge others only after you have looked inward.

I apolagize if I have offended anybody. It honestly was not my intention. I just completely disagree with Mtry and some of the others. He has opened himself to criticism as have others here (including myself) by openly displaying our opinions. We are all in the hobby called audio. I would think that we all have a common passion. There are so few of us that I would think this childish banter could end. I for one am ending it here. I will no longer respond to nor post any defamatory personal post.

I would think that if you are the adult that you portend to be that you can do the same.

**Resident Loser** · 05-14-2004, 07:53 AM

jimHJJ(...subtle it ain't...)

Thread: Please Judge My A/B Methodology, Mtry Others

Share

Thread Tools

Display

Please Judge My A/B Methodology, Mtry Others

Volume

AsI see it...

It's those little bits of jargon...

It seems as though the premise for your...

What "little snide comment"...

And I would guess...

That was not the pupose

Rotflmao...

Thread Information

Users Browsing this Thread

Similar Threads

Hey Mtry....

Don't judge no-name brands...and a thank you to those who helped me.

Bye-bye judge Roy Moore.

Posting Permissions

Latest AudioReview Articles