CHANGSTAR: Audiophile Headphone Reviews and Early 90s Style BBS
Lobby => Amp and DAC Measurements => Topic started by: Hands on January 30, 2014, 12:53:11 PM
-
(http://i.imgur.com/WB0Zfzel.jpg)
Test Hardware and Setup
-DIY Gaming Desktop - i5 2500K locked at 4.5GHz, Biostar TZ77XE3 motherboard, GTX 780, Antec EarthWatts 650W Green PSU, Auzentech Bravura sound card, 4 140mm fans, goofy UV cathode lights in case, Fractal Design Define R4 case...overall, most likely not optimal for audio. Plugged into APC H10 power conditioner. I did run Fidelizer on it for the tests. Using USB 3.0 ports, tested for which ones produce best results and use those two.
-JKSPDIF MK3 - Input with generic USB cable, output with 75ohm RF attenuator (-10dB) and 75ohm BNC to BNC coaxial cable to DAC. Battery powered.
-Metrum Acoutics Hex DAC - Takes JK BNC out to BNC in, RCA out to SB1240. Using the most recommended cables listed on Blue Jeans Cables for both. Plugged into wall socket.
-Creative X-FI HD USB (SB1240) - Takes line-out signal from DAC to line-in for recording purposes, powered by USB bus, plugged into desktop. Unit is limited to 48/96KHz input/output.
Software
RightMark RMAA 6.3.0, ARTA, JRiver MC
Misc. Notes
Creative SB1240 has a slight roll-off in the bass.
I do not guarantee 100% accuracy with these measurements.
Links
Metrum Acoustics Hex DAC - http://www.metrum-acoustics.nl/Hex_English.html
JKSPDIF MK3 - http://www.johnkenny.biz/home-1/mk3-hiface
Creative SB1240 - http://us.creative.com/p/sound-blaster/sound-blaster-digital-music-premium-hd
RightMark Audio Analyzer - http://audio.rightmark.org/index_new.shtml
ARTA: http://www.artalabs.hr/
-
RightMark RMAA measurements. 16/48, ASIO:
Frequency response (from 40 Hz to 15 kHz), dB | +0.11, -1.26 | Average
Noise level, dB (A) | -96.2 | Excellent
Dynamic range, dB (A) | 96.4 | Excellent
THD, % | 0.014 | Good
THD + Noise, dB (A) | -75.9 | Average
IMD + Noise, % | 0.024 | Good
Stereo crosstalk, dB | -95.9 | Excellent
IMD at 10 kHz, % | 0.099 | Good
General performance | Good
RMAA pictures will always be attached in the following order:
1. Frequency Response
2. Noise
3. Dynamic Range
4. THD
5. IMD
6. IMD Swept
7. Crosstalk
-
You'll notice in the 24/96 RMAA tests, the DR measurement looks a bit weird. It looks fine in 16/96.
Frequency response (from 40 Hz to 15 kHz), dB | +0.00, -0.29 | Very good
Noise level, dB (A) | -103.1 | Excellent
Dynamic range, dB (A) | 101.7 | Excellent
THD, % | 0.013 | Good
THD + Noise, dB (A) | -76.7 | Average
IMD + Noise, % | 0.022 | Good
Stereo crosstalk, dB | -96.2 | Excellent
IMD at 10 kHz, % | 0.058 | Good
General performance | Very good
-
Noting the odd 24-bit DR measurements, here's a 24/96 RMAA comparison between the Metrum Quad and Hex. The most interesting part is the frequency response difference in the treble. I can't remember the exact measurement setup I used for these Quad measurements, but it should be close enough to highlight the main differences.
Hex on left, Quad on right
Frequency response (from 40 Hz to 15 kHz), dB: +0.00, -0.29 | +0.06, -0.52
Noise level, dB (A): -103.1 | -102.4
Dynamic range, dB (A): 101.7 | 102.6
THD, %: 0.013 | 0.036
IMD + Noise, %: 0.022 | 0.041
Stereo crosstalk, dB: -96.2 | -97.2
No IMD Swept measurement.
-
After a good bit of discussion with firev1, he determined that there were some incorrect JTest generators floating around, and I had used those previously. It would appear the ARTA-generated JTest files are correct, so I'll be using those for now. If anyone knows otherwise or sees flaws in my method, please let me know.
Using JRiver MC to play the ARTA-generated 24/48 JTest file. Tested without (1st image) and with (2nd image) JRMC upsampling to 24/96. Generated file at 0dB, set to 0dB dBFS in ARTA settings.
Results could potentially be improved with a better hardware and software chain.
-
Got this test idea from firev1. -90dB 1KHz undithered sine wave test, both spectrum and over time (sine wave). I hope I'm doing it right! I ran these tests at both 16/24-bit and 48/96KHz, but I can only attach 7 of the 8 pictures I have. No worries, the 16/96 spectrum graph looked almost identical to the 24/96 graph, so I'll leave that one out.
1. 16/48 Spectrum
2. 16/48 Time
3. 16/96 Time
4. 24/48 Spectrum
5. 24/48 Time
6. 24/96 Spectrum
7. 24/96 Time
firev1 posted this link about these tests: http://www.stereophile.com/content/quality-lies-details-page-6
-
Here are some ARTA measurements that were generated from a sweep impulse. I believe this was done in 16/96. I did many of these and picked the one that gave the most consistent looking results. I'm not sure if some of these really matter for DAC measurements, and I could have set some of these up incorrectly. Pictures attached:
1. Sweep Impulse Response
2. Settings used for the test
3. Frequency response and distortion generated from the sweep impulse
4. Step response
5. Burst decay
6. Sweep impulse response envelope
-
Some noise and distortion measurements from ARTA. Done in 24/96. Most of these are dBFS, matched as close to 0dB as possible.
1. Noise - log graph
2. Noise - linear graph
3. 1KHz sine wave distortion - log
4. " - linear
5. 100Hz distortion - log
6. " - linear
-
Here are some ARTA 24/96 IMD-based measurements. I believe I matched these up with dBV as close to 0dB as I could, A-weighted.
1. 60Hz and 7KHz, 4:1 ratio
2. 250Hz and 8KHz, 4:1
3. 25Hz and 66Hz, 1:1
4. 13KHz and 14KHz, 1:1
5. 19KHz and 20KHz, 1:1
-
Some quick thoughts on the Hex. I have not had a chance to do a direct 1:1 test comparing the Quad and Hex, but I believe the Hex is noticeably less laid-back than the Quad. In the grand scheme of things, it's still probably laid-back. The measured elevated treble response compared to the Quad is probably a key contributor to the difference in sound (almost +0.5dB at 20Hz with 96KHz sampling rate vs Quad). Harmonic and inter-modulation distortion are noticeably improved, though still not absolutely stellar from a purely objective standpoint. It might still be outside what most can hear or low enough to be acceptable.
Other than that, it still retains the common Metrum traits, such as being smooth and fairly relaxed. If you weren't sold on the Quad, it's unlikely the Hex will dramatically change your opinion. If you're on the fence, there's a decent chance you could find a way to work the Hex into your system (may require software upsampling) that you'd quite like, but it might not be worth the price. If you already like the Metrum sound, the Hex is a noticeable step up from the Quad. I have heard neither version of the Octave DAC, so I can't compare with those.
I really like the minimalist look and overall feel the unit. It's a fairly large and hefty piece of equipment, but it feels very solid and doesn't draw attention to itself.
-
Hanns, check your bit depth settings on ARTA or something, your 24 bit test should not show 16 bit quantization steps. It should be a smooth sine wave.
For such test, I usually have generator laptop with the ARTA generated files and a recording side with another laptop or 2 windows of ARTA in order to keep the ADC in 24/96 mode while the DAC puts out the 16 bit tests.
Or is it because its a NOS which is why there is some quantisation error???
-
Bit depth settings should be good, but ASIO does not let me set those in ARTA. Perhaps I could try WDM, which does let me select, to see if there's any difference. I could try setting the ADC at 24-bit while the DAC is at 16-bit. I matched everything in those particular tests.
If I had to guess, it could be that it's a NOS DAC (they output a stepped sine wave), and the DAC chips themselves might not be true 24-bit. (IIRC, I read somewhere the Metrums really only resolve to 18 or maybe 20 bits?) Either way, I can always go back and re-do them.
-
Oh I see, most interesting, yes for these test, set the ADC always at 24 bit as I find that it adds its own noise into the measurements as well.
-
Great work hans :)p4
Is the performance increase from 16/48 to 24/96 significant to your ears?
-
I had a chance to hear the Hex not too long ago. It's definitely laid back and stereotypically analog-sounding. I really liked it FWIW.
Really curious to hear it (and see how it measures) compared to the Lampizator stuff.
-
What did you think of the soundstage Muppet? When I had the Hex for a couple of days I actually found it quite imemorable. There was little of the 3D soundstage of the Lampi and it couldn't match the heft and solidity of the Lampi either. Detail wise it resolved about as well as the PWD + bridge, which was impressive. The treble was also less smooth and grainier than I expected for all the talk of an "analog like" presentation. I just felt it was kind of meh for the price, though a pleasant enough unit. I chose both my Lampi and Master 7 over it without much heartache. Also. the Audiophileo didn't do much for it, which I would chalk up to a pretty good USB section.
-
Is the performance increase from 16/48 to 24/96 significant to your ears?
The difference can be noticeable, but perhaps not as much as you'd think. I'd say I do about 50/50 listening with and without upsampling in JRMC. The Hex will do up to 192KHz for the sampling rate, so I run 176/192 for software upsampling. I'd measure for that if my ADC supported it, but I'm guessing it would give me a fairly well-extended treble response with just a tiny bit of roll-off. Picking your own software upsampling method can be fun or interesting as well.
The way I hear it (and I could be wrong), software upsampling slightly alleviates some of the common complaints you get with NOS DACs while still preserving its characteristic traits. If you don't like the NOS sound with red book, the difference is likely not large enough to change your mind. In that regard, I would find it hard to label the differences as "significant," but I could see it being just enough to sway the opinion of a select few on the fence (not all). Frequency response and other measurements do allow us to at least somewhat quantify the benefits upsampling can offer.
BTW, I've attached a comparison shot of the Hex vs Quad at 16/48. I'm leaving out the other measurements because they were done on different setups (FR should still be good to compare, though). Differences look smaller than at 96KHz sampling rate, but there's still around a 0.4dB difference at 20KHz.
Even with the Hex's improvements over the Quad, and even if you use software upsampling, I still think it's a fairly niche product. For the minimum price of $2500 for the base Hex model (no USB input, no upgrade output transformers), which is what I'm using, I'm guessing most others can probably find options that better suit their tastes and needs (perhaps something like the M7?). I have to admit that I'm happier with the Hex than I had originally anticipated, and I think many others would find a lot to like about it. It works really well for me, but it's not a purchase you should jump into blindly. I just happened to find something particularly pleasing about the Metrum sound, but I do have particular tastes that others might not share.
-
These are inspired by what Ultrabike tested with the Quad. I took a 1-minute sample of a track I regularly listen to (16/44.1, WAV copy of Opeth's "Ghost of Perdition") and did an exp. average in the spectrum analyzer over the entire sample. These three attached pictures show playback through JRiver Media Center without any software upsampling, with upsampling at 96KHz, and with upsampling at 192KHz. This is more just to show that software upsampling does have at least some sort of effect on music playback, and you can gather some info about the type of upsampling JRMC uses based on these shots.
DAC was at 24/192, ADC at 24/96. Verified that setting DAC itself to 16/44.1 did not change the look of the spectrum graph.
-
Here are the results of a test where I generated a 16/48 test file in RightMark and used JRiver Media Center for playback. I also tried with 96KHz and 192KHz software upsampling in JRMC, and you can see how that compares below. The idea was to simulate what software upsampling might do for music playback (close to red book) on a NOS DAC. Note that the DAC was set to 16/192 and the ADC at 24/96 for all of these, RightMark set to 16/48 for evaluation.
1. Frequency response comparison
2. Noise
3. Dynamic Range
4. THD
5. IMD
6. Crosstalk
7. Swept IMD and a numerical overview of the results (this was not auto-generated with the HTML report)
-
What did you think of the soundstage Muppet?
First off I should say that [with a few exceptions] the differences between high-end DACs are usually subtle to me.
That being said, I felt the Hex actually did a good job conveying a sense of space. What stood out in particular was how recording dependent it was compared to the other DACs; the strength of the Hex seemed to lie in quite faithfully reproducing the recording environment. Which can be a blessing or curse.
It's definitely NOT a DAC with a "wow factor" from what I can tell. It makes absolutely no effort to try and win you over.
The Reimyo was much prettier-sounding---artificially so, but executed with finesse---almost like the DAC equivalent of the Sennheiser Orpheus. It was like a courtesan who wanted to make sure you were enjoying yourself.
-
Going back to that -90dB 1KHz sine wave test (low-level resolution), I'm more sure of what we're looking at now that I've tried the test on some other DACs (haven't posted anything for that). According to the Stereophile link, "This test signal produces three quantization steps: 0, +1, and -1. The three levels should be of equal amplitude, and the signal should be symmetrical around the center horizontal division." (My results might not be to proper scale to show this exactly.)
If your DAC upsamples and has good low-level resolution abilities, this test should display a smooth, clean sine wave. A good NOS DAC should show those 3 quantization steps, like the Hex shows in most of its measurements. In fact, the Hex does quite well in that test from a NOS perspective.
I tried this on the NOS1704, which has a noticeably higher noise floor. This directly impacted the DACs ability to resolve that sine wave even at 24-bits, 96KHz sampling rate. I'm not even sure you could call the results recognizable.
The SB1240, on the other hand, displayed a clean, smooth sine wave when I tested it.
-
I got the Gungnir loaner in today and did some comparisons with the Hex. Standard 16/44.1 playback from JRMC without software upsampling on either DAC. Used the BNC connectors on both with the JKSPDIF thingy. I'll be posting measurements of the Gungnir soon. Take these impressions with a grain of salt, as there's a decent chance I'm off or that you wouldn't agree with me.
What I immediately noticed was that the Hex did not seem to have any issues with bass or treble extension relative to the Gungnir, nor did it necessarily sound more laid-back than the Gungnir. In fact, I thought the Gungnir overall had a softer, maybe even more laid-back sound than the Hex. This held even when considering that the upper treble is a bit more attenuated on the Hex than the Gungnir (at least in this playback situation and setup).
The Hex seemed to have a blacker background, and it did a better job conveying micro-dynamics and "movement" within music (my terminology might be off). In other words, the Hex seemed to do a better job capturing quick attacks and properly presenting what I want to call small pockets of blackness in between the movements and notes. For one example, rapid attacks and fast chord progressions or complex melodies on a guitar coupled with palm muting and hand sliding came through with more rhythm, life, and power on the Hex. As mentioned, the Gungnir sounded a bit softer and less defined in this regard (more boring?). Both were pretty smooth and easy to listen to. The Hex may have sounded cleaner overall and perhaps a bit more dynamic/powerful.
The Hex did sound more condensed or collapsed than the Gungnir, though it didn't necessarily have less layering (I could be wrong). This seemed more dependent on the music. Occasionally the Hex seemed to convey a more natural sense of space than the Gungnir. The Gungnir would generally sound a bit more diffused overall. In a sense, this would sometimes give me the impression that the Hex was better focused and less laid-back. Vocals in particular sounded more centered and focused on the Hex with a bit more body to them. I often found myself preferring the more intimate, yet still layered and detailed quality of the Hex, but it really depended on the music.
Also, in particular, I thought the Hex did a better job showing off the rich harmonics in the chords of Akerfeldt's/Opeth's guitar work ("Ghost of Perdition," "Blackwater Park," etc.). The Gungnir sounded a bit mushier in that regard.
Perhaps tomorrow I will compare with the Gungnir on USB, and/or maybe throw software upsampling into the mix. Again, take all this with a grain of salt. And remember that these differences were often subtle.
-
interesting observations. compared to the other DACs i had here, the Gung was tuneful and dynamic. it wasn't the least bit soft or boring via USB. it reminds me of the Naim gears i owned but with better tonal balance and timbre.
have you compared USB to the JKenny USB-SPDIF converter? not that the JKenny is identical to my MF V-Link USB-SPDIF converter, but the V-Link made every DAC sound more subdued/relaxed and rounded.
maybe we should break this off into the Gungnir Loaner thread.
-
Waiting for more hans!
-
I'll get the Gungir measurements up tonight/very early tomorrow morning and will be trying out USB then. It measured a bit better with jitter than BNC with JKSPDIF, so I wouldn't be surprised if it sounds better.
Also keep in mind I'm speaking relatively here. Not saying the Gungir isn't those things. Also not saying I have great hearing. :)p12
-
Bah. Stop apologizing for yourself. You have good ears. If you didn't, you'd be walk the plank2 by now.
I actually know what you are talking about when you cite some of the Gungnir's shortcomings.
-
Haha, just trying to cover my ass for anyone that thinks otherwise!
I did some USB/BNC comparisons on the Gungnir. I'm pretty sure USB sounds better. The Gungnir overall sounded more "with it" and less soft. So, an improvement in all areas mentioned. Of course, I'd like to hear how USB compares to BNC from a better converter like the OR5. (The same applies to the Hex, which I believe is on the sensitive side in terms of the chain of gear behind it...it's probably limited by the JKSPDIF.)
The differences between the Hex and Gungir became more subtle after I switched to USB. Really, most of my thoughts from before still stand, just to a lesser degree. The Gungir still sounds a bit soft and slow in comparison, maybe slightly thicker sounding. But it does have a smooth, sweet quality to it. The Hex seems cleaner and more focused, but more condensed, as mentioned earlier. I like the more intimate nature of the Hex and think it helps flesh out vocals in any situation and helps with instrument separation/details in complex, fast, messy passages in genres like metal. The Gungnir might be better for, say, orchestral, as the more compact nature of the Hex lends itself better to 3-6 people bands with music that isn't particularly wide or spacious sounding (or whatever number...you get the idea). Just a guess, though, because I don't listen to much music that necessitates a wide, extremely layered soundstage.
In terms of dynamics, I found them comparable with the Hex being slightly more engaging. And in regards to the Hex sounding "cleaner," I thought it did a slightly better job pulling out details in the bass. It's hard to describe, but in some ways I found the DACs to be both more aggressive and more laid-back than the other in different ways. But if you consider the Hex has more of a roll-off in the very upper treble and that I find the Gungnir to be a bit softer sounding, it sort of makes sense. Keep in mind this was 16/44.1 tests without any software upsampling. I was actually surprised how not laid-back the Hex sounded compared to the Gungnir with this playback setup.
I touched on this in my other post, but vs. the Gungnir, I think the Hex does a better job conveying a sense of life, rhythm, and attack...I almost want to equate this to the underlying, emotional swings, movements, and beats that come from the culmination of the musical technicalities and the musician's unique "feel" for what they're playing and how they play it. Moments when there's a miniscule amount of blackness before everyone strikes heavily at the exact same moment, I felt the Hex handled that a bit better. Subtle differences in how a particular guitarists moves their hand and fingers across the fretboard or their picking techniques, the micro-nuances that bring out the underlying feeling and motion behind the chords, melodies, and overall music, I found the Hex did better.
Overall, I might consider the Hex to be more transparent. YMMV. This might also change if I ran the DACs in balanced mode. But, given the price of the Hex (especially of you want the USB input add-on or need to get a SPDIF converter), the Gungnir is a much saner purchase.
I'm becoming increasingly convinced that while NOS DACs do tend to have a particular sound to them and have known limitations (see discussions on NOS DACs floating around these forums), these limitations really don't play a huge factor during music playback. You'd think my ears would fall off based on how some rail against NOS DACs or that I'd hiss and howl every time I swapped from the Gungnir to the Hex, but I'd like to see how many could correctly identify NOS/OS DACs in blind testing (assuming good DACs overall). At worst, I think the NOS limitations impart qualities that could easily be found as pleasing to some, much like what you'd see with a well-designed tube amp that doesn't necessarily measure "excellent" in all areas. I'm guessing some or many of these limitations and poor measurements are below our hearing capabilities during music playback. Of course, certain hardware will probably handle a NOS output differently (in terms of ultrasonic stuff).
-
With all of the various artifacts NOS can create, some of which make their way into the audible spectrum, it's important and interesting to consider how much that actually factors into music playback and enjoyment. Creating ugly looking measurements on a NOS DAC is super easy. Here's a simple example: Turn on a 15KHz sine wave and take a look, even with a basic measurement setup. Believe it or not, I never actually tested the audibility of this or similar tests specifically until recently...and by accident, really. Until then, I had only listened to music to see if I heard anything amiss.
I was running a headphone through SineGen to see if I might be able to identify where certain peaks and valleys existed. Once I got beyond the 4-5KHz point, I started to hear something...odd. It got more powerful the higher I went. It wasn't the frequencies I was playing either. It sounded like digital artifacts and noise, or something not too dissimilar from a dial-up tone (think less severe and softer, though). I wondered if this might be the NOS DAC's artifacts, as I had never encountered this before. I tested out a couple of my OS DACs, they clearly did not have this issue, and this was unmistakable.
Surely if I was hearing these artifacts, I should be able to measure them. So, I got out ARTA for some tests and to create some 15KHz test files. The test files sounded pretty nasty, as expected. I took some measurements, and...I think they explain it entirely.
I had done music-based measurements before, and you can see my results earlier in this thread. Software oversampling clearly had an effect in the ultrasonic region, and you could see how the NOS DAC behaved without it. I wanted to see if software upsampling would audibly and measurably improve the 15KHz tone response. With a 16/44.1 file, I tried 88.2KHz software upsampling in JRMC. The artifact noise lessened and changed its primary pitch. I went to 176KHz upsampling, and the primary pitch again jumped, I believe by an octave. Most of that extraneous noise and artifacting disappeared as well. Once again, measurements tell the story.
I still believe that these NOS artifacts have little to no audible effect during most music playback, but that would entirely depend on what you're listening to. I have not yet heard anything red book, without software upsampling, on the Hex where I thought something was clearly messed up in the background (or foreground) like I could in these tests. Switching back and forth with software upsampling did not appear to sound dramatically different with music like these tests would suggest.
The measurements are rough, but they do what I need them to. Even if they aren't accurate, the audible changes were dramatic and clear. Without software upsampling, it was almost difficult to make out the 15KHz tone with some test files. It was much clearer after high upsampling. Take this all as you will. I bet the Hex w/ software upsampling (or even without, often) would make a lot of people quite happy.
1. 15KHz sine wave, 16/48, 0dB
2. Same, but -1dB
3. Same, but -3dB
4. 15KHz, 16/192, 0dB
5. Same, but -3dB
6. 15KHz sine wave, 16/48 file, upsampled to 192KHz in JRMC, 0dB (with this and the nest test, you can see how much of a difference this makes)
7. Same, but -3dB
Again, results aren't perfect. ARTA playback put in weird spurs and other artifacts/noise that otherwise weren't there, likely due to using WDM instead of ASIO (JRMC uses kernel streaming). Ignore that, as I confirmed JRMC playback will always measure slightly cleaner than WDM through ARTA. For example, the 192, 0dB playback from ARTA looks odd/noisy compared to the JRMC 16/48 upsampled file at 0dB, so just focus on the spikes and such. The ADC was at 24/96 for these tests.
-
Wow, thanks for taking the time and effort. Pretty interesting results - upsampling to push noise into ultrasonic bands seems to have tangible sonic benefits. Definitely going to be keeping an eye out for this functionality as I continue DAC shopping.
-
Have you listened to a 15kHz tone at lower output levels ?
The side-band/distortion products seem to lower significantly in level at lower amplitudes.
Everything below -80dB should be difficult to detect.
Ofcourse if multiple frequencies are present (as in music) there will be a big dense 'wall' of spikes present and as a lot of spikes, close to each other, all add in voltage levels (not in dB's) the average noise floor of that recording worsens considerably and may even become higher than that of the actual recording.
Low level noise that is NOT present when there is no signal, but is present with the music might not be that audible (masked by the music)
It may even give the impression there is more presence/treble when in fact it is rolled off in reality (see XNor's explanation) + has added side frequencies (which is what you heard)
There isn't a single piece of music around with high frequencies reaching 0dB or even -6dB it is MUCH lower.
At least it shows (as XNOR has argued before) that a NOS DAC without software upsampling playing redbook and having no reconstruction filter isn't the wisest thing to do.
-
Indeed I have. The difference between 15KHz at 0dB vs -3dB is like night and day. At 0dB, the 3KHz byproduct seems more audible than the 15KHz tone (given human hearing, headphone capabilities, makes sense), and it just generally sounds weird/nasty. At -3dB, the 15KHz tone comes through cleaner and stronger, though you can still hear the byproducts.
After software upsampling, the 0dB test sounds much better...about on par with the -3dB test (not upsampled), though slightly different sounding. I think measurements show how they'd sound different (both better and worse than the other in some regards). After upsampling the -3dB tone, it became much more difficult, but still possible, to pick out the byproducts. I didn't create a -6dB file to test (a bit lazy right now as well), but I'd imagine with that and software upsampling, it might be nearly impossible to detect (I'm estimating nearly everything weird would be below -90dB). I'll whip up that test and further more if anyone is interested. ;)
Considering that music doesn't generally contain tons of information that high in the treble and, as you stated, is likely not even going to reach -6dB, you could use that to potentially argue this isn't going to negatively impact music playback and listening even without software upsampling. But, as you mentioned with what can happen with the music's noise floor, you could use that to potentially argue the other way.
The potential for a perception of more presence and treble due to added side frequencies is a particularly interesting point, I think, and would be something worth testing for if possible. I also would like to see how well people would do in a blind test (or not, your choice), comparing NOS/OS DACs, w/without software upsampling on the NOS DAC (perhaps harder to detect than you think), etc., and I'm sure I've mentioned that before. :)
-
Those last tests I ran inspired me to do another music-based test. This is the same 16/44.1 music file: 58 second sample from Opeth's "Ghost of Perdition," WAV ripped from a CD I own. It starts right after that few-second intro and continues through that heavy, satan-music passage. Sure, it's not a reference-quality track, but it represents what I regularly listen to...it's actually the one track I primarily use to judge equipment (more for overall tone and performance than pure technicalities), just because I'm extremely familiar with it. In other words, if I get something that sounds perfect on this track, it's almost guaranteed I'll like it for everything else. I'm open to tests specific tracks or other things if you suggest them.
What I wanted to do for this test was zoom in on a relatively small range of frequencies to get a better view of the subtle changes brought from software upsampling. And, given that a NOS DAC has more and more "difficulties" with higher frequencies, I figured I'd focus in that area. I chose to look at the 10.5-11KHz region, which is an area a NOS DAC will struggle with but still contains useful musical information. I adjusted the dB range once I figured out where the average result would sit.
This test is also done "by hand," so it's not perfectly accurate or precise all the time. I start the playback in JRMC and then record in ARTA as quickly as I can. I manually stop recording shortly after the sample hits the 58 second mark. The ADC can also produce odd results at times when fed a signal with a sample rate that is or a multiple of 44.1KHz, so there's the potential these results have garbage in them. I'll eventually pick up a device that has more options and wider compatibility in this regard, but it does OK for now.
This test is also done as an exp. average over the sample and not a snapshot of any one moment. A potentially better way to look at the results would be to see the output over time (not spectrum), but given I start/stop everything by hand, it's very difficult to get that matched up. (I'm sure there are ways I could set this up for high precision and accuracy, but I'd have to look into that and am frankly not concerned enough to.)
To help make up for these "by hand" tests, I did 3 runs through JRMC without any software upsampling, and then I did 3 runs with 176.4KHz upsampling. (First three pics are without, last 3 are with.)
What is particularly interesting is how consistent the upsampled results look compared to the non-upsampled. And while there are some clear differences between w/without other than overall amplitude, they do still hold many similarities (at least, if you compare the w/without results that look the most similar...obvious, haha). Inaccuracies aside, I was looking to see if the results would be in any way different and still repeatable, and that was indeed the case to some extent.
Eventually I will measure this on something objectively better so that these tests can be compared against a known reference.