CHANGSTAR: Audiophile Headphone Reviews and Early 90s Style BBS

Lobby => Soapbox => Topic started by: Marvey on December 04, 2011, 10:21:23 PM

Title: Repeatability and Consistency of Measurements
Post by: Marvey on December 04, 2011, 10:21:23 PM
I've heard from two people now saying that Audeze addressed somewhere (on their blog?) the inexact repeatability of measuring their headphones on the dummy head they use; and that their graphs were more a general indicator of meeting a performance goal rather than providing an accurate measurement.
 
There is no problem with this at all. People should not be concerned of the minor variations of each graph that Audeze provides with each headphone. Slight deformations of the ear cushions and partial millimeter differences in placement on the headphone on the dummy head will give slightly different results.
 
The problem is when people make the wrong conclusion: that headphone measurement systems inherently lack precision or the ability to produce consistent results. This is bullshit.
 
If the measurer takes the time to precisely align the headphone (using guides, making marks, or even eye-balling), the results should very consistent assuming similar environmental conditions (temperature, air pressure, environmental noise, etc.) Proper measurement technique has more to do with repeatable results than with using Brand X measurement system or Brand Y dummy head.
 
The following graphs show the results of a Grado SR80 driver measured almost three months apart on 9/13/2011 and 12/4/2011 on the same rig and at the same spot. The measurements below are not exactly the same, but certainly very close. If you look carefully, there is a tiny shift in the frequency of the peaks, probably the result of air pressure differences when the measurements were performed.

While not exactly rigorous or meeting certain statistical standards, these informal "certifications" are run periodically using a small set of headphones on my measurement rig to make sure everything is "calibrated." I'll throw some additional  measurements of headphones measured at different times when I get chance. People who have dropped by in-person to get their headphones measured can certainly attest to how consistent my results are.

We'll see as winter sets in.
Title: Re: Repeatability and Consistency of Measurements
Post by: LFF on December 05, 2011, 05:40:28 PM
Excellent post dude!

If I am not mistaken, they use a Neumann KU-100 dummy head with CLIO software. Those heads are routinely used for measurements since they are essentially a poor mans Aachen head. Even though the KU-100 might not be the most accurate of the dummy heads, it can certainly provide accurate and repeatable results provided you know a little bit about making such measurements.

The idea that their graphs are more a general indicator of meeting performance goals is utter foobar. If their graphs can't provide an accurate measurement then why use them as general indicators?  ???

This sounds more like 1) a BS explanation, 2) a lack of attention to detail or 3) it could be they simply have no idea how to properly measure something. Going by the sound of the LCD-3, it could be all 3.  :-\
Title: Re: Repeatability and Consistency of Measurements
Post by: Marvey on December 05, 2011, 06:49:07 PM
Hehe. Personally, if I were Audeze, I wouldn't provide any measurements if they were indeed performed as a "general indicator". Seems kind of gimmicky and the cause of some customer consternation over there at HF. The lack of separate left and right measurements is troubling.
Title: Re: Repeatability and Consistency of Measurements
Post by: LFF on December 05, 2011, 07:27:42 PM
The lack of left and right might be troubling but it's easily fixed.

More troubling to me is hat their graphs are more a general indicator of meeting performance goals rather than being accurate. Shouldn't your graphs be accurate first and foremost before attempting to meet performance goals? Some flawed logic there.... ???
Title: Re: Repeatability and Consistency of Measurements
Post by: Marvey on February 15, 2012, 03:29:58 AM
Updated with a measurement rig check. I'll be doing these every once in a while.
Title: Re: Repeatability and Consistency of Measurements
Post by: Marvey on June 27, 2012, 03:29:47 AM
This is a tough test for various reasons:
Fostex TH900 left and right channels.

[1] Green June 16, 2012
[2] Red June 26, 2012
Title: Re: Repeatability and Consistency of Measurements
Post by: maverickronin on June 27, 2012, 04:37:11 AM
Fostex FR900 left and right channels.

Nobody's gonna believe you if you make a typo in the model number.   ;)
Title: Re: Repeatability and Consistency of Measurements
Post by: Marvey on June 27, 2012, 04:39:33 AM
Bah was thinking FR too much.  I should probably run them again since the pads may have conformed to my head since when I first measured them.
Title: Re: Repeatability and Consistency of Measurements
Post by: ihasmario on August 09, 2012, 09:05:26 AM
Can you post contrasts of "classic" errors that people like to target?

For example, a proper seal vs improper seal for both headphones and canalphones (if you have the means to do so for the latter)?