Topic: DAC comparison methodologies (Read 5925 times)

thegunner100 · « **on:** January 08, 2014, 08:24:00 PM »

Since theres been a lot of talk here and on HF about Gary's recent dac comparisons, I'd like to know how you guys compare dacs.

Namely, there are these two by Purrin and co. :
https://docs.google.com/spreadsheet/ccc?key=0Ao9CyUbvb2nFdHZ4RlZaQ2JwX1JsZG9vQlAtTmhvM2c&richtext=true#gid=0
http://www.head-fi.org/t/693798/ranking-of-21-dacs-and-dac-configurations

How did you guys go about comparing the dacs? Did you guys level match, use a RCA switchbox, etc? I'd like some more insight as to how I can be better at comparing dacs.

As some of you may know, I bought myself a Emotiva Stealth DC-1 so that I could do some of my own testing. I mean, how could the DC-1 sound indistinguishable from a more aggressive dac like the Gungnir?

During my A/B test between the nfb-10es2 and Gungnir, I used ASIO4ALL to stream data to both dacs at the same time. The RCA out voltage on the a-gd was 2.5V so I lowered the volume on windows to 80% (under the assumption that 80% of 2.5v would be 2.0v). I then connected the two dacs to a radioshack RCA switchbox, which then fed my Vali and HD800s. After some extensive quick A/B switching between them, I did find that the Gungnir had a little bit more treble energy (a friend of mine noted that the mid-range was more forward), but it did have better bass texture and rendering. The gungnir did sound a bit louder, but I'm not sure if I should attribute it to the energy treble or the overall dynamics. Now I know there are a bunch of catagories in the dac-off comparison sheet, but I really don't think I am experienced enough to tell the difference between dacs using all of the catagories. I guess the easiest for me would be Treble rendering, tonal balance, macro-dynamics, and bass rendering.

So when I do my comparison between the Gungnir and DC-1 (same setup) this weekend, how can I improve my comparison? What can I look for? I also have a friend who's coming next Tuesday to do the comparison himself, using my hd800s and his pair of emotiva airmotiv 4s.

Marvey · « **Reply #1 on:** January 08, 2014, 10:22:50 PM »

Use of a set of recordings which we are familiar with down to the particular master. Some are good quality. Some are marginal. The marginal recordings are really good for pointing out trouble areas.
Use the same downstream equipment.
No level matching. You would be amazed how closely the volumes are matched when done by ear during a listening session, especially with familiar recordings. This is why I hate steppers.
No switch boxes. This is guaranteed to make all DACs sound the same. It's like drinking a Bordeaux from St. Julien followed by a Pauillac without first cleansing the palate.
Relax. Have lots of rest before the comparison. Trying too hard will result in fatigue.
Play. Stop. Change DACs. Play. Stop Go back to first DAC to confirm. Stop Return to second DAC to confirm.
Listen to extended time periods using any DAC X as the sole DAC.

Also, sometimes it's hard to level match. For example, in order to get the NAD M51 to have the same dynamics as the PWD2, I almost had to turn the volume knob up and down in real time along with the music.

Stapsy · « **Reply #2 on:** January 08, 2014, 11:48:26 PM »

I find I only hear non-specific impressions if I quickly A/B. Things like, I like x better than y, or x is really dark and y is really bright. I also find quick A/B is more susceptible to expectation bias than prolonged listening. Over a longer period of time I think it is easier to identify differences. More listening means more information gathered, which should theoretically lead to more information to make an opinion (obviously this doesn't mean all at the same time).

Over time I have gathered up about 10 songs that I like to use as a reference point. They cover the normal things, treble stridency, dynamics, background noise, imaging, separation, bass, tone, etc. Other than that I don't actively listen for any one aspect. After a while things kind of jump out at you.

Unlike a lot of folks here I don't have very much experience doing critical listening comparisons/reviews. I have a pretty good ear for music and a basic grasp of treble/mids/bass, but nothing spectacular. Like anything, critical listening is a skill and practicing is the only way to get better. If objectivity is your main goal try not to read other reviews before you listen. I doubt that it would be a big deal for Purrin, but for me it is hard not to think about what other people have said when I am trying to listen critically. Do your own tests and then go back and see what other people have to say. This way you are practicing your own listening skills and then learning from the experience of other reviewers after you have drawn your own conclusions.

thegunner100 · « **Reply #3 on:** January 09, 2014, 12:07:51 AM »

Thanks for the tips purrin! I'll start warming up the gungnir tomorrow in preparation for the comparison. I'll keep using the dc-1 as my sole dac til Friday and then switch to Gungnir for a day to see if there are any immediate differences.

Quote from: purrin on January 08, 2014, 10:22:50 PM

No switch boxes. This is guaranteed to make all DACs sound the same. It's like drinking a Bordeaux from St. Julien followed by a Pauillac without first cleansing the palate.
Play. Stop. Change DACs. Play. Stop Go back to first DAC to confirm. Stop Return to second DAC to confirm.

I think these two points will be particularly important. Lets say you listen to one track on dac X. Then you listen with dac Y; when you notice a difference, do you stop there and go back to dac X to confirm or do you finish the track first?

Stapsy, you have a good point there. Unfortunately I've read a ton of reviews on the gungnir before I even purchased it. The DC-1 however I don't really know much, other than it was in Gary's group of indistinguishables. So I guess after I do some comparing, I will go back and read what he has to say about it. I've only been in this hobby for about 2 years and it has only been a couple months since I made my first high-end purchase in the hd800s.

Marvey · « **Reply #4 on:** January 09, 2014, 12:09:17 AM »

Also comparing notes from others (at the same session) is key. Sometimes someone may catch something which might warrant going back.

I don't think we've ever had to repeat anything. This biggest issue with language and how to communication what is difficult to communicate. What I've found doing this comparison with three four others here is that we all hear the same things, just that we have different preferences.

Marvey · « **Reply #5 on:** January 09, 2014, 12:12:04 AM »

Quote from: thegunner100 on January 09, 2014, 12:07:51 AM

I think these two points will be particularly important. Lets say you listen to one track on dac X. Then you listen with dac Y; when you notice a difference, do you stop there and go back to dac X to confirm or do you finish the track first?

We'll go back if we have to - if it's close. But really, we've listened to the same tracks over and over and over so much that we usually stop at 45 seconds or 90 seconds (to cover all the necessary ground) and stop.

I know my methods would not stand scientific scrutiny, but this was never intended to be scientific. It's more like judging dishes at various restaurants. SABRE is my brussels sprout.

Original_Ken · « **Reply #6 on:** January 09, 2014, 12:42:29 AM »

Quote from: purrin on January 08, 2014, 10:22:50 PM

Use of a set of recordings which we are familiar with down to the particular master. Some are good quality. Some are marginal. The marginal recordings are really good for pointing out trouble areas.

Please give more details about your use of marginal recordings, thanks.

OJneg · « **Reply #7 on:** January 09, 2014, 01:48:12 AM »

Quote from: Original_Ken on January 09, 2014, 12:42:29 AM

Quote from: purrin on January 08, 2014, 10:22:50 PM
Use of a set of recordings which we are familiar with down to the particular master. Some are good quality. Some are marginal. The marginal recordings are really good for pointing out trouble areas.
Please give more details about your use of marginal recordings, thanks.

Yes please, I'd be interesting to hear which recordings you guys find to be best a judging the differences between DACs.

thegunner100 · « **Reply #8 on:** January 09, 2014, 02:05:34 AM »

There's a list of tracks on the bottom of the spreadsheet. I just copy and pasted it for reference.

Peter Gabriel So (original CD) - Mercy Street
Peter Gabriel So (original CD) - Don't Give Up
Peter Gabriel So (original CD) - In Your Eyes
Talking Heads Speaking in Tongues (remastered) - This Must Be The Place
Jennifer Warnes - Rock You Gently
Pixies Surfer Rosa (MFSL) - Brick is Red
Nirvana Nevermind (MFSL) - Lithium
Depeche Mode Strangelove (SACD) - Behind the Wheel
Depeche Mode Violator (SACD) - Halo
Eric Clapton Unplugged - Tears in Heaven
Eva Cassidy American Tune (Hires) - Drowning in the Sea of Love
Eva Cassidy American Tune (Hires) - True Colors
Rostropovich Cello Suites - Suite 1 - Prelude
Taylor Swift Fearless - Breathe
The Essential Darryl Hall and John Oates - Sara Smile
The Essential Darryl Hall and John Oates - I Can't Go For That
Michael Jackson Thriller (original CD) - Billie Jean
Cincinnati Pops Orchestra - Copland: The Music of America - Billy the Kid
Cincinnati Pops Orchestra - Copland: The Music of America - Fanfair for the Common Man
Nirvana In Utero (MFSL) - All Apologies
Eagles California Hell Freezes Over - Hotel California
Alison Krauss New Favorite (MFSL) - Let Me Touch You For a While
Alison Krauss New Favorite (MFSL) - The Boy Who Couldn't Hoe Corn
Alison Krauss New Favorite (MFSL) - The Lucky One
Sheffield Labs - Drum Improvisation
Scorpions-We'll Burn the Sky/Your Light
Ultrasone Demo Disc
Bobby McFerrin-Bang!Zoom/Friends
Bread-It Don't Matter to Me
Hall n Oates-One on One/You Make My Dreams
Elton John-Tiny Dancer
Gipsy Kings-Mosaique
Godsmack-Voodoo/Whatever
Shakti-La Dance Du Bonheur-hi-rez
Limp Bizkit-Take a Look Around
Madonna-Holiday
Rush-Tom Sawyer
Spyro Gyra-Goovin' for Grover-hi-rez
System of a Down-Atwa
Eagles-Hotel California; Hell Freezes Over
Anne Murray-Danny's Song
Chakachas-Jungle Fever
Vivaldi-Gloria (Missa Sancti Spiritus)
Vivaldi-Concerto for 2 Violins in D major, RV513 Aston Magna

Original_Ken · « **Reply #9 on:** January 09, 2014, 02:33:08 AM »

Quote from: OJneg on January 09, 2014, 01:48:12 AM

Quote from: Original_Ken on January 09, 2014, 12:42:29 AM
Quote from: purrin on January 08, 2014, 10:22:50 PM
Use of a set of recordings which we are familiar with down to the particular master. Some are good quality. Some are marginal. The marginal recordings are really good for pointing out trouble areas.
Please give more details about your use of marginal recordings, thanks.

Yes please, I'd be interesting to hear which recordings you guys find to be best a judging the differences between DACs.

No not which recordings, but how marginal recordings are used, as opposed to great recordings.

Author Topic: DAC comparison methodologies (Read 5925 times)

thegunner100

DAC comparison methodologies

Marvey

Re: DAC comparison methodologies

Stapsy

Re: DAC comparison methodologies

thegunner100

Re: DAC comparison methodologies

Marvey

Re: DAC comparison methodologies

Marvey

Re: DAC comparison methodologies

Original_Ken

Re: DAC comparison methodologies

OJneg

Re: DAC comparison methodologies

thegunner100

Re: DAC comparison methodologies

Original_Ken

Re: DAC comparison methodologies

(adsbygoogle = window.adsbygoogle || []).push({});

Author Topic: DAC comparison methodologies (Read 5925 times)