CHANGSTAR: Audiophile Headphone Reviews and Early 90s Style BBS

  • December 31, 2015, 09:35:46 AM
  • Welcome, Guest
Please login or register.

Login with username, password and session length
Advanced search  

Author Topic: Repeatability and Consistency of Measurements  (Read 2393 times)

0 Members and 1 Guest are viewing this topic.

Marvey

  • The Man For His Time And Place
  • Master
  • Pirate
  • *****
  • Brownie Points: +555/-33
  • Offline Offline
  • Posts: 6698
  • Captain Plankton and MOT: Eddie Current
Repeatability and Consistency of Measurements
« on: December 04, 2011, 10:21:23 PM »

I've heard from two people now saying that Audeze addressed somewhere (on their blog?) the inexact repeatability of measuring their headphones on the dummy head they use; and that their graphs were more a general indicator of meeting a performance goal rather than providing an accurate measurement.
 
There is no problem with this at all. People should not be concerned of the minor variations of each graph that Audeze provides with each headphone. Slight deformations of the ear cushions and partial millimeter differences in placement on the headphone on the dummy head will give slightly different results.
 
The problem is when people make the wrong conclusion: that headphone measurement systems inherently lack precision or the ability to produce consistent results. This is bullshit.
 
If the measurer takes the time to precisely align the headphone (using guides, making marks, or even eye-balling), the results should very consistent assuming similar environmental conditions (temperature, air pressure, environmental noise, etc.) Proper measurement technique has more to do with repeatable results than with using Brand X measurement system or Brand Y dummy head.
 
The following graphs show the results of a Grado SR80 driver measured almost three months apart on 9/13/2011 and 12/4/2011 on the same rig and at the same spot. The measurements below are not exactly the same, but certainly very close. If you look carefully, there is a tiny shift in the frequency of the peaks, probably the result of air pressure differences when the measurements were performed.

While not exactly rigorous or meeting certain statistical standards, these informal "certifications" are run periodically using a small set of headphones on my measurement rig to make sure everything is "calibrated." I'll throw some additional  measurements of headphones measured at different times when I get chance. People who have dropped by in-person to get their headphones measured can certainly attest to how consistent my results are.

We'll see as winter sets in.
« Last Edit: December 05, 2011, 06:50:22 PM by purrin »
Logged

LFF

  • Mastering Wizard & Restoration Guru
  • Mate
  • Pirate
  • ****
  • Brownie Points: +761/-0
  • Offline Offline
  • Posts: 1425
Re: Repeatability and Consistency of Measurements
« Reply #1 on: December 05, 2011, 05:40:28 PM »

Excellent post dude!

If I am not mistaken, they use a Neumann KU-100 dummy head with CLIO software. Those heads are routinely used for measurements since they are essentially a poor mans Aachen head. Even though the KU-100 might not be the most accurate of the dummy heads, it can certainly provide accurate and repeatable results provided you know a little bit about making such measurements.

The idea that their graphs are more a general indicator of meeting performance goals is utter foobar. If their graphs can't provide an accurate measurement then why use them as general indicators?  ???

This sounds more like 1) a BS explanation, 2) a lack of attention to detail or 3) it could be they simply have no idea how to properly measure something. Going by the sound of the LCD-3, it could be all 3.  :-\
Logged
These statements are false.
I rule with an iron fist and ears of gold!
The preceding statements were true.

The way to a man's heart is through her stomach.

Marvey

  • The Man For His Time And Place
  • Master
  • Pirate
  • *****
  • Brownie Points: +555/-33
  • Offline Offline
  • Posts: 6698
  • Captain Plankton and MOT: Eddie Current
Re: Repeatability and Consistency of Measurements
« Reply #2 on: December 05, 2011, 06:49:07 PM »

Hehe. Personally, if I were Audeze, I wouldn't provide any measurements if they were indeed performed as a "general indicator". Seems kind of gimmicky and the cause of some customer consternation over there at HF. The lack of separate left and right measurements is troubling.
Logged

LFF

  • Mastering Wizard & Restoration Guru
  • Mate
  • Pirate
  • ****
  • Brownie Points: +761/-0
  • Offline Offline
  • Posts: 1425
Re: Repeatability and Consistency of Measurements
« Reply #3 on: December 05, 2011, 07:27:42 PM »

The lack of left and right might be troubling but it's easily fixed.

More troubling to me is hat their graphs are more a general indicator of meeting performance goals rather than being accurate. Shouldn't your graphs be accurate first and foremost before attempting to meet performance goals? Some flawed logic there.... ???
Logged
These statements are false.
I rule with an iron fist and ears of gold!
The preceding statements were true.

The way to a man's heart is through her stomach.

Marvey

  • The Man For His Time And Place
  • Master
  • Pirate
  • *****
  • Brownie Points: +555/-33
  • Offline Offline
  • Posts: 6698
  • Captain Plankton and MOT: Eddie Current
Re: Repeatability and Consistency of Measurements
« Reply #4 on: February 15, 2012, 03:29:58 AM »

Updated with a measurement rig check. I'll be doing these every once in a while.
  • Black line = taken on 1/24/2012. Green/Red line = taken on November 2011.
  • The 12k null is a measurement artifact. It should be ignored. This was confirmed by listening to sine sweeps covering that area (not fun.)
  • The graphs above are raw uncompensated data. They should not be compared to anything else.
  • The earpad material maintains a remarkably consistent shape. Differences in earpad placement and compression can result in significant measurement differences.
Logged

Marvey

  • The Man For His Time And Place
  • Master
  • Pirate
  • *****
  • Brownie Points: +555/-33
  • Offline Offline
  • Posts: 6698
  • Captain Plankton and MOT: Eddie Current
Re: Repeatability and Consistency of Measurements
« Reply #5 on: June 27, 2012, 03:29:47 AM »

This is a tough test for various reasons:
  • Sealed headphones
  • Squishy pads
  • Angled drivers
Fostex TH900 left and right channels.

[1] Green June 16, 2012
[2] Red June 26, 2012
« Last Edit: June 27, 2012, 04:39:18 AM by purrin »
Logged

maverickronin

  • Objectively Sound
  • Able Bodied Sailor
  • Pirate
  • ***
  • Brownie Points: +58/-2
  • Offline Offline
  • Posts: 670
  • Your friendly neighborhood audio skeptic
Re: Repeatability and Consistency of Measurements
« Reply #6 on: June 27, 2012, 04:37:11 AM »

Fostex FR900 left and right channels.

Nobody's gonna believe you if you make a typo in the model number.   ;)
Logged
Heaven's closed - Hell's sold out - So I walk on Earth.

Marvey

  • The Man For His Time And Place
  • Master
  • Pirate
  • *****
  • Brownie Points: +555/-33
  • Offline Offline
  • Posts: 6698
  • Captain Plankton and MOT: Eddie Current
Re: Repeatability and Consistency of Measurements
« Reply #7 on: June 27, 2012, 04:39:33 AM »

Bah was thinking FR too much.  I should probably run them again since the pads may have conformed to my head since when I first measured them.
« Last Edit: June 27, 2012, 04:45:59 AM by purrin »
Logged

ihasmario

  • Powder Monkey
  • *
  • Brownie Points: +17/-27
  • Offline Offline
  • Posts: 63
  • Follow Your Dreams - Arif
Re: Repeatability and Consistency of Measurements
« Reply #8 on: August 09, 2012, 09:05:26 AM »

Can you post contrasts of "classic" errors that people like to target?

For example, a proper seal vs improper seal for both headphones and canalphones (if you have the means to do so for the latter)?
Logged
Audiofire 12 -> Stax T1 -> Stax Lambda Signature

Interested in recording and making music