Saturday 15 September 2018

MUSINGS / MEASUREMENTS: About THD(+N) and standardizing testing here... (With a taste of the RME ADI-2 Pro FS as DAC...)

Last week, when I published my look into the RME ADI-2 Pro FS ADC performance, I started using SpectraPLUS for measurements in high resolution for THD and THD+N. I know, it's an old measurement and everyone does it. I realized shortly after publication of the article that while I was using the same technique to measure the RME vs. Focusrite ADCs, I had the ADCs running at 192kHz and so issued an addendum that the results included the rising ultrasonic noise from the ADC and not just a reflection of the DAC. Not unreasonable as a comparison between the two ADCs I think, but this would not be fair to the DAC or other component measurements since much of the noise would be arising from the ADC stretching out to a 96kHz bandwidth. It's worth taking some time to think, going forward, how I could improve the usefulness of this test and in a standardized way here when focusing on whatever device is being tested...

If we look around, we see that the THD+N spec is probably the most used objective "number" for audio equipment as a quick snapshot of fidelity. The THD (page on calculation, how it's done) component tells us whether harmonics are being added to the sine wave at the integer multiples of the fundamental frequency (these are the results of "nonlinearities" in the equipment), and the +N piece adds the noise component found in the signal being tested which of course is also subject to noise limitations of the measurement device and the computational limits of the FFT technique used. In essence, the THD+N ratio is a representation of everything that's being added to a simple single-tone test which of course has its limitations as a test paradigm as well when real music is far from static.

Like most measurements, one does need to know the context in which the result is produced. This is especially important when this single specification is being used for the purpose of making comparisons between devices. To state that I found "DAC X has a THD+N of -110dB" sounds great, but just stating this value itself is not enough obviously unless I explicitly tell you how that measurement was derived and with what general procedure.

A good reference I have seen for how measurements should be expressed is by Rane Audio here. For the THD+N value, it should at least be expressed like: "-110dB THD+N, +4dBu input level, 20-20kHz, with a 22kHz bandwidth".

There are recommended standards out there as well. The AES17 is one that originated back in 1991 (you can have a look here for the 2004 printing, here's a page on implementation). It specifies a 997Hz sine wave to be used. There is however no clear standard of all the variables that then go into the measurement itself. When using FFT techniques, how many FFT points? What FFT windowing function to use? What bandwidth?

Furthermore, when we're thinking about DACs, is the best test level actually a 0dBFS signal or should we be concentrating on lower amplitudes? After all, unless we're always listening to primarily modern pop and rock with crushed dynamic ranges and maximized amplitudes, even if a device throws up beautiful values at 0dBFS, most of the musical content "lives" lower down... Therefore, for the purpose of "real-world" testing, isn't it much more interesting to make sure the THD+N remains low at output levels significantly less than 0dBFS? (Note that the RightMark test suite I also use does include a THD measure with a -3dBFS sine wave.)

Considering those factors, here's a suggested "standardized" scheme I'll probably use going forward as 2 sets of measurements/graphs.

Set 1. The stereo 0dBFS THD+N: Yes, I think it is good to keep a 0dBFS measure. For better or worse, this is what others commonly use and is shown in specifications. Looking around, almost all the published measurements focus on a limited bandwidth close to the 20Hz-20kHz audible spectrum - fair enough.

So let's do this. With SpectraPLUS, let me start off with using these parameters for a 0dBFS measurement with the ADC set to the "Sharp" linear phase filter, running at 48kHz; therefore bandwidth up to 24kHz, slightly higher than the typical 20 or 22kHz but this should not change the result by much:

The test signal will be a 0dBFS 24/48 997Hz (as per AES17) sine wave created in Adobe Audition with spectrally "pure" characteristics. The measurement above will perform the test with an FFT size of around 250k (262144 to be precise), using Hann (Hanning) window function (windowing choice has its own complexities to consider). And I'll set averaging to 10 to get a more stable reading. Using these parameters, this is what the Oppo UDP-205 XLR output at +13dBu sent to the RME ADI-2 Pro FS looks like with the THD(+N) and SINAD results shown for both stereo channel:

Notice that I measured the output at around -3dBFS on the ADC so that there is some overhead but it is the full 0dBFS level from the Oppo XLR being fed in. As you can see, the measured THD is excellent at ~0.000120% or below -118dB and the THD+N result is very good at less than 0.00013% which is close to -118dB (reflected in the SINAD value). Oppo's specifications indicate <0.00018% THD+N, so what I'm seeing here is certainly to be expected.

Set 2. The "THD+N Matrix": Why don't we also try to capture a range of THD(+N) results at different amplitudes to see how the harmonics change and whether THD(+N) fluctuates in any unusual way below 0dBFS? Furthermore, we're in the era of high-resolution music - why not run the test then at 96kHz? This way we will also capture distortions and noise up to a 48kHz bandwidth, ensuring a measurement well beyond human audibility.

I decided to pick 4 amplitude values - -5dBFS, -15dBFS, -30dBFS, and -50dBFS. Unless automated, it would be onerous to run too many measurements so the 4 here captures a reasonable range for musical signals I think.

The SpectraPLUS settings would look like this for the 96kHz measurements:

Notice that I've doubled the FFT size to ~520k which corresponds to the doubling of sample rate so there will be no change to the computed noise floor compared to the 0dBFS reading above. Let's just focus on a single channel for each graph for space, and again we set averaging to 10 for more stable readings.

Here's what this would look like in a 2x2 matrix, again, with the Oppo UDP-205 XLR output as "guinea pig" fed into the RME ADI-2 Pro FS - notice that since I'm starting at -5dBFS, I'll set the RME ADC to reflect the actual levels (as usual, click on image to enlarge):

The "THD+N Matrix"
Interesting to see that the system was able to compute a remarkably low THD+N value of -122dB at -5dBFS off the Oppo. In fact, notice that this is even lower than the 0dBFS measurement above (-118dB). Of course, I'm not suggesting that anyone would be able to hear a difference between THD+N of -118dB vs. -122dB; they're both really low numbers :-). But it does suggest that at least with the Oppo and its ES9038Pro DAC, the signal remains extremely clean across the top levels. If we peek at those graphs, we see that lowering the volume to -5dBFS does reduced the 3rd, 4th, and 5th harmonics plus reduced sideband levels seen around the fundamental (we'll talk more about this later).

As expected, by the time we measure -30dBFS most harmonic peaks should be very low and by the -50dBFS signal almost all should be fully buried within the noise floor of the ADC; to see otherwise would speak poorly of a high fidelity DAC. One interesting thing we see with the Oppo is that with the -30dBFS measurement, notice how the 3rd harmonic increased unexpectedly for some reason though still low.

Here's a comparison showing the 0dBFS FFT at 24kHz bandwidth and the -5dBFS FFT measured across 48kHz with the latter constrained to just the 20Hz-24kHz portion of the spectrum so we can compare the harmonic levels and see why the computed result is lower for the -5dBFS signal:

Nice reduction of the harmonic levels in the -5dBFS signal. I know... Totally geeky pixel peeping to show off the microscopic but impressive THD(+N) that the Oppo DAC + RME ADC system can produce. :-)

Notice also in the "THD+N Matrix" image above, as the fundamental changes amplitude, how we see the appearance of "clusters" of various harmonics from the Oppo DAC (for example, at -15dB, there's a little group around 10kHz-25kHz) which is different from the -5dBFS clustering that tends toward higher up in frequency (>20kHz). I doubt this is audible, but in lesser DAC devices with higher distortion levels, perhaps there is an audible effect that a detailed test like this could inform us of.

Remember last time I also compared the result from the Focusrite Forte and showed that it did not fare as well compared to the RME? Since 0dBFS direct from the Oppo into the Forte causes overloading, here's the -5dBFS signal with a 24kHz bandwidth and then with the 48kHz bandwidth:

At 48kHz bandwidth (ADC sampling at 24/96), we see the obvious noise out around 37kHz which I've documented over the years originating from the Focusrite Forte. It is now being calculated into the measurement with the 48kHz bandwidth. As you can see, this along with other ultrasonic content significantly impact the THD+N score dropping from -110dB to around -100dB without as much of a change to THD itself.

For completeness, here's a look at the RME ADI-2 Pro FS going from 24kHz to 48kHz bandwidth (zoomed into just 20Hz to 24kHz for both for a more comparative look, we already know from the "THD+N Matrix" that the RME noise floor remains low all the way out to 48kHz):

Notice that there is a small difference between the 48kHz and 96kHz samplerates with a few noise/harmonics above 10kHz added when the RME ADC is running at 96kHz. It's tiny, each -140dB or less on the graph, and as you can see, the doubling of sample rate does increase the THD(+N) value slightly.

Okay, with that little exploration and experimentation this week, using the better quality RME ADI-2 Pro FS ADC, I'm going to start "standardizing" my procedure for THD(+N). It will include both the stereo measurement of 0dBFS output level using 24kHz bandwidth so the value can be comparable to published results and manufacturer specs elsewhere using typical similar methodology, as well as the 2x2 "THD+N Matrix" captured at 24/96 to show harmonics and noise out to 48kHz for -5 / -15 / -30 / -50dBFS signals. With all the tests, the RME ADC will be set to its "Sharp" linear phase lowpass filter.

FFT parameters are ~260k points in 48kHz, and ~520k in 96kHz so that the calculated noise floor remains the same for both sample rates, and Hann(ing) windowing will be used. I have 16/44 variant of the test signals that I can use in the event that I thought there's something wrong with the way the device handles the 44.1kHz clock base (I expect this to be rather unusual).

Apologies for being a bit pedantic, geeky and OCD in this post :-). But I figure it's worth spending time documenting the exact settings and procedures... As I've said before, my hope as a "more objective" hobbyist is to have others be able to measure and produce comparable results themselves. Sometimes, even if we can't hear a difference, it is nice to be able to understand and know about the quality of our devices. For example, even if we cannot directly hear high noise levels in the ultrasonic spectrum from a component, I think it would be nice to know that it exists since the signal needs to be fed into one's amps and speakers which could have negative consequences.


I did promise last week that I would show the RME ADI-2 Pro FS's DAC performance. While I obviously am not able to show much this time, let me offer you this consolation teaser from the THD(+N) measurements with the RME as DAC, compared to the Oppo UDP-205:

Isn't that interesting!? The above is the 0dBFS 997Hz pure sine comparing the RME itself (AKM AK4490 DAC chip, output level at +19dBu) with the Oppo (ESS ES9038Pro DAC chip, output level at +13dBu), both measured at -3dBFS on the RME ADC for a bit of overhead, with 24kHz bandwidth and showing both stereo channels to ensure good levels in each and concordant results.

Notice that the RME ADI-2 Pro FS DAC's THD(+N) measured slightly higher than the Oppo due to the harmonics (stronger even order harmonics like 2nd, 4th, 6th). But there is more relative noise in the Oppo's output hence the increase from THD to THD+N is larger than the RME. I see no reason to doubt ESS's claim that the ES9038Pro is currently the "king" of low THD+N DAC chips.

But of course it's never as simple as a single number even though that number does have meaning. Notice those low-level "sidebands" seen with the Oppo and its ESS DAC around the fundamental 997Hz signal; some of which appear stronger than the harmonic distortions. In comparison, the RME's AK4490-based DAC performance has a more consistent pattern with stepwise decline in levels with each higher harmonic even though overall THD is higher. Of course, the Oppo sidebands being close to the fundamental would mask audibility (hard to imagine humans hearing those extremely low levels down at -130dB!). These sidebands have been commented on elsewhere also and likely represent modulation of the DAC's voltage level. I do wonder whether this pattern is a typical finding of ES9038Pro DACs in general or is this just with the Oppo UDP-205? Unfortunately I did not have this level of measurement resolution when I examined the Oppo Sonica DAC (also based on ES9038Pro) back in mid-2017 to see if these were present.

Ultimately, remember that compared to the amount of distortion levels of amplifiers and speakers, or room effects / interactions during playback, the Oppo sidebands are like pimples on an elephant's back. That in fact is the main message audiophiles need to keep in mind! The level of DAC performance these days with well-engineered digital devices have I believe handily surpassed auditory perception such that the vast majority of subjective opinions and claims are a function of the placebo effect and human imagination.

Furthermore, I think it is impressive the amount of resolution even the home hobbyist can have at his/her disposal these days using a device like the RME ADI-2 Pro FS when it comes to seeking and understanding the performance of high fidelity devices. By all means, do try this at home!


To end off, in the "let's-talk-about-MQA-for-no-other-reason-than-we're-supposed-to-support-it" news bin, I see that Rafe Arnott in AudioStream appears to believe that:
"High-res downloads seemed to be the next big thing, but now streaming has become the undisputed heavyweight of the music industry’s music-delivery method, with MQA seemingly set to rise brightest on the format horizon."
As a subjective review, I think the article is fine, but why embed this IMO baseless statement while posing a question to McIntosh's CEO? MQA and Bob Stuart have been silent for the last year in the presence of significant criticism and evidence pointing to the limitations of the algorithm (instead, preferring to speak through their audiophile magazine mouthpieces). It has been almost 4 years since announcement with realistically only a trickle of encoded music primarily supplied by one streaming service. More recent high-resolution streaming services like Primephonic for classical and Qobuz are IMO wisely using an open, almost universally supported compression format (FLAC), not the lossy, essentially 16-bit, proprietary MQA codec. How is MQA the "brightest on the format horizon"!?

As per the article, apparently there is no comment about MQA from the CEO in his response. Perhaps Mr. Arnott missed the notice a few months back (June 2018) that McIntosh engineers "prefer to take a wait-and-see position, finding the format too lossy, with distortion that doesn’t meet the company’s high standards". Well, good to see that even The Absolute Sound was able to pass this along to the readership. Bizarre how the audiophile press seems to be the only place these days even bothering to mention MQA and out-of-the-blue bringing it up.

Have a great week ahead everyone! I've been listening to Paul McCartney's new one - Egypt Station - in the last week while working on this post. Not bad if you're into "Macca" and enjoy a more traditional sounding pop although I cringe a bit listening to some of the young-adult-oriented lyrics which would have been more appropriate from a much younger Beatle rather than from a man of my father's generation. I'm not sure how memorable this album will be for me. As usual, I will need to spend a little more time in replay.

Hope you're all enjoying the music...

Addendum - September 19, 2018
Hmmm, just a quick note. Looks like there's some issues with the SpectraPLUS measurement for the "THD+N Matrix" especially notable with the lower amplitude signals and abnormally low THD+N results. Will look into this more...


  1. Regarding Rafe's 'MQA is the brightest...,' it really is worse than that [though you have to give him credit for coming up with a new, seemingly innocuous way of spinning it; he's clearly a good writer].

    What I'm referring to is something he's written in the body of the article-- ''For my gear support I used a 48-inch George Nelson wood bench after getting rid of my previous steel and glass racks as I overwhelmingly preferred the sound of all my gear on wooden support structures whenever possible.''

    There's really no possible way to refute or correcct stuff like this. I would've been laughed out of engineering school for saying something like this.

    This kinda stuff, along with the MQA, makes me wonder what it'd be like if we had a forum for brain surgery aficionados, and there was a thread on trepanning, and there were a bunch of guys swearing up and down that that was the one, true way to perform successfully brain surgery, because that's how they did it in the [very] old [analog] days. [Here maybe Dr. Oz takes the place of BL as the off-stage high priest.]

    And then a famous surgeon with an actual medical degree weighs in. And the forum beats the heck out of him because not only does he tell them they're wrong, but he points out that use of anesthesia might be a good idea. Anesthesia, the ancient Egyptians did't need no stinkin anesthesia, comes the reply. Well, you get my drift.

    [And then the moderator weighs in and says, we'll I personally have heard some things -- not saying if they're good or bad, but just things -- so go ahead and say anything you want just as long as it's not about anyone's mother.]

    Personally, I no longer refer to myself as an audiophile, because the term now connotes a no-nothing tweaker. I really don't know term to use instead, but I won't group myself in with those low-information audio fans.

    1. Thanks for the note aonetech,
      Sigh... Yeah, so many things one could be concerned about with articles like these not just among a number of sites but also over the decades. If it's not wood vs. glass/metal audio racks, it's Cat 6... er... make that Cat 8 ethernet cables. Heck, digital cables in general and all the claims of audibility between otherwise competent wires.

      Then there's of course all the exaggerated claims whether it's MQA, tweaks, or even just "hi-res" in general (yes, there could be objective differences, but let's not make a mountain out of gopher hills).

      I think all isn't lost. In time all things change and I believe even in the world of audiophilia, the pendulum will turn at some point. I think the "rational audiophile" has a role in this process.

      BTW: Love the trephining vs. neurosurgery analogy. So true. So sad as well.

  2. Quick note...

    I've received a few E-mails that the SpectraPLUS graphs were hard to read due to poor contrast and the fineness of the lines. It looked better when I was editing but Blogger resizes images and I think the rescaling worsened things. Going forward will be changing the color scheme!

    1. Just replaced the last graph comparing the RME ADI-2 Pro FS and Oppo UDP-205 to an easier to read, more contrasty color scheme... As well as the "THD+N Matrix" at the 4 signal levels.

  3. Egypt Station is good enough, a number of nice songs on it (I don't know, Come on to me, Caesar rock, Confidante, People want peace)

    1. Yeah, I agree, it's "good enough".

      Not his best but there's good variety here and he rocks on track 16 - "Hunt You Down/Naked/C-Link".

    2. Yes he does and also he rocks on Caesar rock :-)

  4. Some links for inspiration:

  5. RE: Music Streaming

    I was recently recommended this site, and thought folks here would appreciate too, if you're not already familiar with it.

    The author explains loudness and dynamic range and the multiple ways to measure it in fabulous detail. What I found most interesting is that he covers all the major streaming services, and how they have automatic loudness limiting. If a music producer pushes the loudness above the streaming service's reference level, the tracks are simply brought down in volume to the reference. It doesn't restore DR, but it takes away the one reason producers had to sacrifice it in the first place, at least on the streaming services. Here's one one on a change Tidal made last year.

    And another one where he reports that Spotify actually lowered their loudness normalization level!

    I really liked this quote which sums up the situation:
    Old question: “Why does Song X stand out on the CD changer?”
    Old answer: “Because it’s louder.”

    New question: “Why does Song Y stand out online?”
    New answer: “Because it has great dynamics.”

    And that is the start of a really interesting conversation.

    It doesn't solve it for CD's or Hi-Res downloads, but it's definitely a step in the right direction. If consumers start complaining that the music they purchase doesn't sound as good as what they can stream, hopefully producers will be quick to adjust... just as I imagine the HiRes guys will be quick to adjust when word leaks out that their Redbook CD's sound better than their 24bit/96KHz FLAC's.


    (PS. Yes, this a duplicate comment that I made on the RPi post. I'm not sure everyone checks the comments on old posts, so I added it here too. In my defense, here is was in RE streaming services, whereas there it was in RE loudness wars. ;) So, on-topic both times.)

    1. Very cool Allan!

      I hope this is the beginning of a trend... Not holding my breath of course but encouraging :-).

  6. One more comment RE streaming services and enjoying the music.

    I recently subscribed to Tidal to give it a test run on a new-to-me Oppo BDP-105. I gotta say, Tidal the service is pretty nice. I've been enjoying it immensely. Their catalog is really deep and I have no complaints over the sound quality. It's kind of on the expensive side, so I see myself turning it on and off as my time and interest level waxes and wanes, but for now it's decidedly on. Anyway...

    The first 30 days is a free trial at their premium, lossless tier. The premium tier includes "Master" ie. MQA-formatted tracks. You can only access the lossless tracks through the desktop client, so I had to connect my laptop to the BDP-105 via HDMI port or USB connector. For the record, I could not hear a difference between the HDMI and USB transport links. And yeah, requiring the desktop client completely defeats having the built-in Tidal app if you're a lossless snob. Anyway, the desktop client also lets you choose which stream quality is sent. Guess what. The "Masters" streams were very noticeably louder than the regular lossless streams. Color me suspicious AF.

    As for enjoying the music, I happened upon Fine Young Cannibals self-titled debut album. Wow! How had I never heard that before? Everyone's familiar with their second album with their two-hit wonders "Good Thing" and "She Drives Me Crazy", which were cool and all, but in that over-produced, grab-your-attention on the FM dial way. You kind of have to be in the mood, preferably doing something else, to want to listen to that style. Their self-title album is an entirely different beast. It has the lead singer's compelling vocals, but with a much, much more interesting instrumental arrangement backing him up. Great lyrics too. And an Elvis cover! Check it out. I do endorse.

    Best Regards.

    1. One more thing on the FYC album. If you get a chance to listen to it, be sure to listen all the way to the end. The last track has a nice little surprise. I put the album on before dinner a few weeks back and asked my wife if she recognized the singer/band. She couldn't quite put her finger on it, and kept making all these wild, wrong guesses. As soon as the last track hit, she laughed and was like, "Ahhhh, now I know!" :) Good times.

    2. Hey Allan,
      That's great that you're enjoying Tidal. FYC - awesome man.. Yeah, everyone remembers The Raw And The Cooked but the first album is excellent and Roland Gift has an interesting voice. I remember getting a kick out of seeing him in the early-90's on the Highlander TV series.

  7. Paul McCartney's new one - Egypt Station - is "more appropriate from a much younger Beatle rather than from a man of my father's generation".
    Oh, my God! I belong to your father's generation (I'm 67)!

    1. Nothing wrong with that man!

      I was referring to some of the lyrics that seemed a bit... Well... See what you think, check out the track "Fuh You".

  8. In the "let's-talk-about-MQA-for-no-other-reason-than-we're-supposed-to-support-it" news bin...

    I just noticed that Spotify has started serving up MQA! Or at least the term "MQA".

    The 2L album, MOZART Violin Concertos by Marianne Thorsen & Trondheim Soloists, is now offered in 2 versions. One is the "original 2006 edition" and the other is the "MQA remix 2016". In a quick comparison I could not hear a difference. But back when I downloaded the very-well-produced 96/24 and MQA versions from the 2L website, I could not hear a difference either (although when I performed a "blind" test on a friend he picked the 96/24 version as better sounding).

    I then did a general search in Spotify for "MQA" and found one other album labeled as such - "Fairytales" by Radka Toneff and Steve Dobrogosz. In this case only the "Original Master Edition (MQA)" is available. I have not listened to it yet.

    Being this is Spotify, I have to believe these are all data-compressed 320Kbps streams. Can a 320 stream carry MQA "fold" info? Doesn't seem likely, but with the magic of "audio origami" who knows what might be possible?.

    1. Hey Mark,
      You're right, Spotify at 320kbps at best cannot deliver the "benefits" of MQA. Neither sound quality nor "authentication" :-).

      I remember trying to listen to a difference between the original 24/96 and the MQA of that Mozart track and didn't think there was anything all that obvious either.

      I can certainly understand that MQA wants their name out there. Unless it's in the public consciousness, it will fade from memory especially given the obvious lack of "uptake" in the last year. Of course no surprise that the name "MQA" would be attached to the 2L album and the Toneff Fairytales with MQA remastering was spoken of in Stereophile a number of months back. BTW the original CD from the 1980's sounded great already. Anyone know if the "Original Master Edition" brought anything new to the sound other than a drop in average DR from 13 on the CD to DR10 here:

      Wonder if the above is what the MQA version was based off of!?

    2. A friend emailed me earlier today to tell me he had purchased The Doors "Waiting For The Sun" 50th Anniversary set including both LP and CD, and that the CD is MQA encoded. I looked on Spotify and sure enough, that album was labeled as "MQA remix".

      The thing is, tonight I went to listen to it and found it is now labeled as just "remastered". I wonder what precipitated the change. The tracks I sampled on the album did sound pretty good. But if I remember correctly it was maybe the best sounding of the Doors albums. Of course that was back in the vinyl days. The DR Database indeed shows the early 1985 & 1986 CDs as having good dynamic range, and the 1989 CD as being especially good. This 2018 remaster matches some of the poorer reissues through the years. I can't make any sort of comparison as Spotify has dropped the "regular" version.