MEASUREMENTS: A look at DSD and using SoX-DSD as a standard for test signal conversion. The 1/10-decade Multitone 32 test. And retro-pop remixes... ;-)

Hey everyone, although I'm in the middle of the Topping D90SE review series, I thought this week I'd make a slight detour! Like I suggested last week, I'm planning to savor the D90SE measurements / discussion series and enjoy the DAC. No rush needed. I see this blog as more educational and philosophical than mere opinions about things you can just buy and try for yourself. ;-)

Over the years, the fun part of the hobby and blogging for me has been to "evolve". As an audiophile, it's fun taking on the challenge of examining this pursuit with a more objective lens. To do that, it has been good to see what others are up to and incorporate incremental improvements in the testing gear and techniques I use within the budget of a home audiophile.

Each of these "MEASUREMENTS" posts is like a mini-experiment that we as hobbyist/"citizen scientists" can do for ourselves. I trust that the information I post can be repeated and verified if you have the interest, time, and some know-how.

Inspired by discussions with Bennet Ng (aka Dtmer Hk recently and in the Topping D10B review), DSD measurements is something that I have seen little discussions of in reviews. For reference, in these blog pages, we have talked generally about DSD/SACD back in 2013, looked at conversion software (here, here), as well as talked about PCM --> DSD playback like with HQPlayer a couple years ago.

But I haven't really taken a more intentional look at my DSD testing regimen... Something I hope to rectify here and clarify the "standard" signals I'll use in the tests ahead.

I. A Little Background and on SoX-DSD:

I think among audiophiles, there remains still a bit of mystique about DSD. It was certainly hyped up by Sony back at the turn of the century (wow, Y2K was that long ago?) when SACD was introduced as having a special "analogue" sound. Even to this day, some audiophiles seem to feel that it's the "best" digital technology for music. I know there was a relatively brief push from 2014-2015 around the time of NativeDSD Music starting up and the idea of DSD downloads for audiophiles. Of course that was before the rapid decline of music downloads and the rise of on-demand streaming. In any event, DSD-playback capability, techniques like DoP, etc. are all basically ubiquitous these days even though actual DSD material is limited.

As you've likely read about, DSD is different from PCM in that it's another way to represent analogue music waveforms digitally. Instead of 16 or 24 or 32 multibit representation of the waveform output level (correlating with voltage out of the DAC), DSD utilizes 1-bit sigma-delta modulation.

The noise level of multibit PCM is consistent across the bandwidth of the signal - that is, bandwidth = samplerate/2, dynamic range = 6.02*bits + 1.76 dB. As you can imagine, a single-bit DSD signal only has the potential for 6.02dB of dynamic range! This would not be good enough to encode hi-fi music. The solution then is to increase samplerate ("oversample" to the MHz range) which will increase the bandwidth and utilize noise shaping to "push" quantization noise out of the audible 20Hz-20kHz range, important for human hearing. With noise "safely" in the ultrasonic frequencies, a filter can be applied to reduce the amount of that noise sent downstream to the rest of the audio components. [Of course, noise shaping can also be applied to multibit PCM using various algorithms like Weiss' POW-R, or iZotope's MBIT+, or Sony's Super Bit Mapping.]

As you probably are aware, SACD is based on 64x CD samplerate (at 1-bit); that's 44.1kHz x 64 = 2.8224MHz. Here are the main DSD samplerates supported these days in equipment specs:

DSD64/SACD "Standard rate" = 2,822,400 Hz (2.8 MHz)

DSD128 "Double-rate" = 5,644,800 Hz (5.6 MHz)

DSD256 "Quad-rate" = 11,289,600 Hz (11.2 MHz)

DSD512 "Octuple-rate" = 22,579,200 Hz (22.6 MHz)

DSD1024 "Sexdecuple-rate"(!) = 45,158,400 Hz (45.2 MHz)
Surely nothing needed more than "Sex-rate", right?! :-)

Just like the world of PCM music being primarily "standard" CD-resolution 16/44.1, most DSD music these days is of the "lowest common denominator" DSD64 (mostly imported or ripped from existing SACD data).

[More information here for bedtime reading. ;-)]

As I was mentioning, the job today is to review how I'm going forward with DSD measurements in my DAC playback reviews. For many years, I have just been using a JRiver 24 PCM to DSD conversion of test signals like the RightMark battery. As Bennet/Dtmer Hk mentioned in his comments, these days we can use open-source conversion of PCM to DSD with Måns Rullgård's modifications to SoX in 2015 - here's the source code. Unofficial SoX-DSD binaries available here and here. For reference, I will use the version online dated 2017-02-25 "sox-dsd-masr-git-x86_64-w64-mingw32.exe" for my testing.

Although there are nuances I've seen mentioned in various discussions for the conversion process (like -n 32 and -t 32 here), for all reasonable purposes, I'll convert my test signals starting as 32-bits integer PCM to the various DSD samplerates using SoX-DSD. For example, a RightMark 32/96 test signal can be converted to DSD64 using the command line:

sox RightMark32-96.wav RightMark32-96-DSD64.dsf rate -v 2822400 sdm -f sdm-8

This command tells the program to samplerate convert using the -v "very high quality resampling" to the DSD64 rate of 2822400. Notice the "sdm" setting which tells the software to create the sigma-delta modulated stream and allows us to specify various noise-shaped filter settings. In the example above, we're using the 8th order "sdm" noise shaper. Options are CLANS (Closed-Loop Analysis of Noise Shapers, an optimization method to synthesize lowpass "noise transfer function" [NTF]) and the "standard" SDM setting varying from 4th to 8th order.

[For those interested in more technical details, Måns tells me he used this Delta-Sigma Toolbox - have a look at the PDF documentation. The "SDM" series is based on using the "synthesizeNTF" function and of course "CLANS" settings using the "clans" function for the noise shaping.]

So here's the same command but creating a DSD128 (5644800) version and using the CLANS-4 noise shaping option:

sox RightMark24-96.wav RightMark24-96-DSD64.dsf rate -v 5644800 sdm -f clans-4

II. Signal Testing with example DAC:

Alright then. Let's test this out using this device:

As you can see, I'm running a "corner of the table" test here with my Topping DX3 Pro DAC currently playing "22.5MHz" DSD512 (more accurately 22.6MHz). For measurements, I'll use the E1DA Cosmos ADC, and there's my Surface laptop on the right for data capture/analysis.

The reason I'm using the DX3 Pro is because this is one of the best DSD-performing DACs I've run into over the years. The device has dual AKM AK4493 chips and I suspect the higher performance is due to the "DSD Direct" function which bypasses a digital multibit attenuator phase (ie. volume control) which then sends the data to the internal delta-sigma modulator thus adding another layer of processing and potentially more noise. Update: I did some further testing of the signals and it looks like the DX3 Pro is not using "DSD Direct" during playback even when set to "DAC" mode and the volume control has been turned off. The results below actually reflect this DAC's performance with the "normal" path from what I can tell with various amount of low-pass filtering applied. In fact, test results look better for the 384kHz signals with "DSD Direct" turned off in other testing using the RME ADI-2 Pro FS R Black Edition; I might show this another time.

First up, let's check out the various modulator options at the most common DSD64/SACD samplerate - let's choose CLANS-4, 6, 8 and SDM-4, 6, 8. For highest accuracy, let's use 32/96 RightMark signals fed into the SoX-DSD conversion. We can compare the playback using SoX-DSD with JRiver 24 DSD conversion which I have been using for years and also direct 24/96 PCM playback. Here's the "big table" of results:

As usual, click on the chart to zoom in. Basically what we see numerically is that the higher 8th order settings will improve measured noise level and dynamic range within the 20Hz-20kHz frequencies. SoX-DSD "SDM-8" setting gives us the highest results here compared to CLANS-8. JRiver 24 PCM-to-DSD conversion is quite good as well. We can see that SoX-DSD "SDM-8" DSD64 and direct 24/96 PCM are almost exactly the same with basically identical distortion results even though we know that conversion from PCM to DSD is not a completely "lossless" process.

Here's the frequency response:

PCM extends further than the DSD64 versions which are pretty well identical.

Noise level:

Note small amount of 60Hz mains hum picked up in the RCA out.

Overall the higher order noise shaping will provide a lower noise floor in the audible frequencies but you can see accentuation of the ultrasonic noise. We can see the JRiver conversion (yellow) is very similar to SoX-DSD SDM-8 (cyan). The JRiver conversion actually has noise increasing slightly below 20kHz while SDM-8 is nice in that noise level only rises above 20kHz.

As expected, 24-bit PCM maintains low noise floor across the whole bandwidth.

Here's a peek at intermodulation + noise distortion levels:

Again, higher order modulator settings result in less distortion below 20kHz. As the noise increases above 20kHz, so too the distortion amount. Likewise, the PCM measurement remains relatively flat for distortion across the full frequency range. Clearly there's a big difference between the 4th and 6th order settings.

With the E1DA Cosmos ADC (ESS ES9822 PRO ADC), we can now look at the ultrasonic frequencies up to 192kHz (384kHz samplerate) with a flat noise floor. Let's have a better look at the performance now between DSD64/128/256/512 SDM-8:

Interesting to see that the frequency response is a bit better with the DSD512 signal (orange) than even PCM (yellow).

Note that I'm showing the dynamic range graph because there seems to be an bug in the "noise level" graph in RightMark (unfortunately there are a few bugs here and there when displaying >96kHz test results). You can see that this test uses a -60dBFS 1kHz tone. There are a number of small harmonic peaks present. Notice particularly the amount and shape of the >20kHz ultrasonic noise due to noise shaping; highest with DSD64 > DSD128 > DSD256. The DSD512 noise floor was essentially identical to PCM up to 192kHz - nice!

RightMark is convenient, but we can also port some of my other test tones over into DSD playback. For example...

1kHz -0.1dBFS sine wave for THD+N - I reduced the amplitude a little for the DSD to avoid a tiny amount of clipping at 0dBFS. Converted 32/96 PCM to DSD:

Notice one nuance. The DSD256 1kHz tone is done with a 5th-order SDM setting. SDM can be tricky and with the higher-order modulators, when fed this static 1kHz high-amplitude tone, the signal became unstable after a few seconds (~10-15 seconds or so with sdm-8). I did not notice this issue with other test signals using sdm-8.

All results are "spitting distances" from each other. We see a little more low-level sidebands in the DSD64 and DSD128 FFT than DSD256 and PCM. But basically we're looking at insignificant +/-1dB variations at almost -110dB THD and THD+N levels of low distortion.

A friend asked me for the 1kHz signals I used. Here's a link to the files.

24-bit Jitter-Test:

The good ol' J-Test we still use to look at jitter was designed back in the days of AES3 and S/PDIF (Julian Dunn back in the early '90s). I think it's still useful when applied to interfaces like USB to look at low-level random timing anomalies ("skirting") and evidence of data-correlated jitter (sidebands).

Notice in the case of the Topping DX3 Pro, it's not the best performing asynchronous USB DAC I've measured and indeed there are some sidebands in the 24-bit J-Test evident in the PCM tracing. However, these are very low level anomalies below -130dB (as usual, I'm generally not worried about jitter audibility and the J-Test accentuates issues beyond what is found with actual music playback).

Like with some other DACs I've seen, running it in DSD mode often will show reduced J-Test anomalies. There's not much difference between DSD64 and DSD128. Not shown, the DSD256 FFT looks about the same as well.

1/10 Decade Multitone 32!

One more test! Over the last few months, I've received E-mails asking if I would incorporate something like the Audio Precision multitone test that's shown at places like ASR, etc. Okay, sure, no problem friends.

Let's do this. Instead of the same 1/3-octave multitone that AP uses, let's go with a 1/10 decade 32-tone signal starting around 17Hz with the 32nd tone up at 21.4kHz. Here's a multitone setting using REW's Signal Generator to create such a thing:

Notice the calculated "crest factor" is 13.3dB (similar to AP's 32-tone 1/3 octave multitone of 13.038dB, see user manual page 381). Also make sure if you're using this to attenuate the signal (-10.4dB in this case) so as to keep the peak level <0dBFS of course. Here's a peek at a zoomed-in portion of the waveform:

Notice the chaos in there! We're no longer "measuring" a single 1kHz sine wave like for THD(+N) but rather something more akin to the complexity of music (arguably even more complex on the microscopic level!) and the DAC will need to navigate through all those ups and downs in order to perform well without distortions.

Since this kind of test is pretty extreme, I figure I might as well incorporate <20Hz and >20kHz tones into it (hence 17.6Hz to 21.4kHz) which will cover not just human audible frequencies but also likely what "supertweeters" might reproduce and frequencies where metal tweeter "ringing" might occur. For the analysis, I'll run the ADC at 96kHz, use 1M-point FFT, 8 sample average scanning across 15Hz to 30kHz - I trust this is above and beyond any human claims to frequency bandwidth audibility - looking for distortions and noise anomalies!

So here's what this looks like with the Topping DX3 Pro with DSD64/128 and PCM (24/96), captured using the E1DA Cosmos ADC:

Notice with each of the FFTs above the distortion/noise-free range for this DAC is around 112dB. This is excellent. Notice the flat frequency response (flat peaks) from ~17Hz to above 21kHz. The highest noise/distortion peak is that 60Hz mains hum. Despite the complexity of the waveform, everything else is below that!

The main difference between the graphs is really just the increased >20kHz noise with DSD64 as expected. Otherwise, differences between DSD samplerates and PCM are rather subtle. I'll make sure to show you how the Topping D90SE handles this signal (the PCM at least) when I get to it next time.

III. In Summary:

With that, I think I've characterized with significantly more detail the DSD testing regimen I'll be using with the freely available SoX-DSD. In the absence of "standard" signals for DSD testing, I figure this is as good as any, and derived from freely available software. We'll start using these new SoX-DSD PCM-to-DSD test signals with the Topping D90SE next week. ;-)

Unless otherwise specified, I'll standardize on the "SDM-8" noise shaping setting which is capable of producing excellent low-noise resolution at the 20Hz-20kHz frequency range. The exception is my 1kHz tone converted with "SDM-5" due to instability with the 8th order setting. On a DAC like the DX3 Pro with excellent DSD performance, we see results that are very close, if not essentially identical to high-resolution 24-bit PCM even at DSD64.

As expressed over the years, I do like the "sound" of DSD but generally prefer good PCM for various reasons including the ease of using DSP (like room correction which makes a huge difference). My impression has been that with higher rate DSD (DSD128+), the sound becomes more like good PCM. In that regard, I don't "hear" anything all that special about what simply amounts to another way to digitally encode the analogue signal and can be just as "accurate".

As usual, I find it fascinating observing the audiophile culture and some of the beliefs/inconsistencies that people hold. For example, on the one hand you have folks venerating the sound of DSD and then there are those who suggest zero or low feedback is key to sound quality (certainly when it comes to amplifiers). Realize that DSD and the necessary noise shaping (especially strong for SACD/DSD64) requires significant amounts of negative feedback to achieve good dynamic range in the audio band. In fact, the "delta" component of sigma-delta modulator speaks to that feedback loop when the 1-bit data is generated. Yeah, it still sound great even with feedback, right?

For those who feel that negative feedback deteriorates sound quality (some people believe that feedback reduces the subjective "3D" sensation of audio, "more analogue"), they might prefer the sound of the lower order CLANS-4 or SDM-4 settings even though noise level is higher. I'll leave you to experiment for yourself.

[Check out Richard Murison's 3-part series on Sigma-Delta Modulators: Part I, Part II, Part III. Best material I've ever seen coming out of PS Audio! ;-)]

As a follow-up to this, also have a look at "Notes on DAC DSD (1-bit PDM) measurements going forward..." published in June 2022.

--------------------

To end, I heard this on the radio the other day:

Hmmm, that's quite some remix there. Not sure how I feel about all those synth effects. Do we need to make it more disco-like? Are the distorted background vocals necessary? Why's Whitney shouting at me through the whole song (sounds like compression used to the margin of gross distortion with added harshness and artificiality to her voice)? Cool to hear the pop sax still in there. ;-)

I could not help at the end of the song thinking that this is a remaster aimed at adrenaline junkies and filled with gimmicks to satisfy those with attention deficits (is this the state of modern pop?). Anyhow, interesting take on a 1985 pop classic. Will need to ask my teenage kids how they feel about this...

Regardless, it's actually fun hearing many of the old '80s/early '90s tunes redone including Whitney's version of Winwood's "Higher Love" a couple years ago, the recent Elton/Dua Lipa "Cold Heart" (psychedelic disco?) hitting the charts again. Won't be surprised to hear even more recycled retro-tunes in the years ahead.

Alright audiophiles, hope you're all doing well and enjoying the music whether it be in PCM, DSD, or anything else.

Happy October!

Addendum: October 5, 2021

In the comments, Unknown asked about looking at noise up to 100kHz. Alas, REW using ASIO4All only allows me to run the ADC input up to 192kHz for high quality FFT (would love to have the ComTrue 7601 ASIO driver that the E1DA Cosmos ADC work natively in REW so I can use it at 384kHz!). So at least we can see the noise up to ~96kHz. This will capture the strongest noise peaks with DSD64 and most of DSD128 at least:

Note that I did not bother optimizing the ADC level here hence the lower dBFS level.

Each of those multitone peaks are at -25dBFS so the peak of the DSD64 noise would be below -60dBFS (somewhere between -60 to -70dBFS) maxing out at 65kHz. Notice that this noise is seen already in the "Dynamic Range" graph above with the 384kHz samplerate test signal.

There is a point here about recognizing that this noise is being passed on to amps and speakers when we play DSD material - especially DSD64. At least with DSD128, the noise is pushed an octave above up to around 120kHz peak. High bandwidth amplifiers will try to reproduce this but it would be highly unusual for transducers (speakers, headphones) to get anywhere near this of course. I think it's rather unusual for any speakers to reproduce much content without very significant roll-off through 30-40kHz.

Just a point about sound quality. Is it possible that for those who really like DSD, filtering "too much" of that noise could subjectively reduce the perceived sound quality? Time domain characteristics like rise time and a very narrow impulse response thanks to DSD's very high speed sampling rate would be affected by filtering. [Personally, I'm not worried, but theoretically DSD-purists might have an opinion on this.]