Tuesday, 5 October 2021

MEASUREMENTS: DSD Testing with SoX-DSD Signals Part II (RME ADI-2 Pro FS R Black Edition, DSD Direct, Filter @ 50/150kHz) [Updated with 44.1kHz PCM --> DSD.]

 


Okay guys and gals, a rare "mid-week" update on DSD testing, a "Part II" follow-up to the previous article.

As I mentioned as an update in that article, I have a suspicion that the Topping DX3 Pro might not be using "DSD Direct" in its playback. However, I do have the RME ADI-2 Pro FS R Black Edition here as well. As you might know, I've been using the RME line of ADC/DACs for awhile now. These are certainly some very well thought out and high performance units with a ton of customization options.

Two of these options are shown in the screen in the picture above - "DSD Direct" and "DSD Filter". The "DSD Direct" is either OFF/ON, and "DSD Filter" either 50kHz/150kHz and is only activated when "DSD Direct" is ON, in recognition of course that DSD noise shaping does create significant ultrasonic content that in general probably should be filtered out; how much filtering is up to you.

Here's a snip from the AKM AK4493 datasheet:

Notice the highlighted items referring to the "DSDD" (DSD Direct) bit which allows the DSD data to be sent either thru the "Normal Path" to the DATT (Digital Attenuator) which implements volume control through a multibit mechanism then back to a delta-sigma modulator to the "switched capacitor filter" (SCF) from which the analogue output is generated.

The alternate pathway is through the "DSD Direct" path which bypasses the internal DATT and delta-sigma modulator and sends the signal directly to the SCF. Wonderful that the RME ADI-2 allows us to try this out and hear the effect for ourselves, and run measurements as well.

Okay, let's get to it! I'll try not to type too much text and let the data speak for itself.

I. Effect of the "DSD Direct" and 50/150kHz filter in RightMark

As mentioned, there are 3 possible setting:

  • A. DSD Direct ON + 150kHz Filter
  • B. DSD Direct ON + 50kHz Filter
  • C. DSD Direct OFF (50/150kHz switch greyed out)

A. First, let's just go wide open with "DSD Direct" ON and 150kHz filter, with 32/384 bandwidth test signal. This is "as direct" as one can get with these SoX-DSD converted test files on the RME:


Notice we have DSD64, DSD128, and DSD256 here. Notice the high frequency spikes in the SoX-DSD encoded signal for DSD64 and DSD128. For DSD64, don't expect accurate frequency response above 50kHz and for DSD128, stick with <100kHz.

At this high 384kHz samplerate and with high levels of DSD noise, I've found that the "Noise level" measurement and graphs can be inaccurate. Better to show the noise floor with the "Dynamic Range" plot which includes the -60dBFS 1kHz tone and aligns the noise floor appropriately.

As much as I would like to always use 384kHz, sometimes I run into issues and RightMark is much more stable at 96kHz. So if we run the test at various DSD rates using a 32/96 test signal, we get:



As you can see, I've included a 24/96 PCM measurement on the right. Obviously at 96kHz, the bandwidth is limited to 48kHz. Practically, this is probably more than good enough for human hearing although it would be nice to be aware of all that higher frequency content.

Note that with the sampling rate reduced to 96kHz, I can look at the "Noise level" graph, plotted accurately.

B. Time to switch the filter down to 50kHz... Here's what "DSD Direct" ON, filter at 50kHz measures like in 32/384:

In general, the numbers look better than the "DSD Direct" ON + 150kHz filter results above. Thanks to lower ultrasonic noise affecting the results either in the RightMark analysis or perhaps aliasing into the lower audio spectrum. Here are the graphs:

We can definitely see the effect of the filter as it smooths out the roll-off with DSD64 and DSD128 particularly, as well as in the shape of the ultrasonic "mountain" with DSD64 and DSD128. Again, I'm showing the "Dynamic Range" graph because the "Noise Floor" graph is inaccurate at 384kHz (likely as suggested by Bennet/Dtmer Hk in the comments link last time). 

Here are the results at 96kHz and we can show the actual "Noise level" graph, comparing with 24/96 PCM.


Results look good and personally I would have no issues sticking with this 50kHz filter and sparing the amplifier from trying to reproduce too much ultrasonic content.

C. Finally, the last of the 3 options. "DSD Direct" OFF (50/150kHz filter option inactive):

We're in the "Normal Path" (multibit) processing now with volume control turned on - I've still got the DAC set at 100% volume for the measurements of course. Numerically, noise level looks good. And likewise low distortion.

Looks like the "Normal Path" has resulted in filtering of the DSD64 and DSD128 streams, similar in fact if not identical to the 50kHz filter above.

Here's the 96kHz test with "DSD Direct" OFF:



From an objective perspective at least as it applies to RightMark's measurement algorithm using the AK4493 DAC chip, the multibit "Normal Path" performs very well. It appears to have even marginally better noise level and lower distortion than when we turn on "DSD Direct". Of course, subjectively one might listen and come up with a different impression!

II. 1/10 Decade Multitone 32 and DSD

Let's have a look at the multitone with the RME ADI-2 Pro FS R Black Edition, let's extend the FFT to 192kHz samplerate / 96kHz bandwidth (1M-point FFT, 8 average).

DSD64:

As in the tests above, we can see that the "DSD Direct" off condition also seems to introduce a similar if not exact 50kHz filter.

DSD128:

Much of the ultrasonic stuff extends above 100kHz now with DSD128.

DSD256:

Basically the vast majority of the noise pushed beyond 100kHz with DSD256. Notice that there are some low-level artifacts we can see here particularly with the 50kHz filter and with "DSD Direct" OFF from 20-40kHz. Not sure if that distortion from 65kHz up is a result of some sort of imaging created in the SoX-DSD conversion from 32/96 PCM source or a result of the DAC, maybe even from the Cosmos ADC's filter. Will need to chase this down another time. In any case, we're looking at stuff -100dB from the multitone peaks and way beyond audible frequencies.

Here's the same multitone test in PCM 24/96:


Notice that the RME ADI-2 Pro FS R BE has about 115-120dB of distortion-free range in this multitone test. The DSD256 playback looks more like PCM including some of the distortions in the ultrasonic range as we push the quantization noise even further up, leaving an overall cleaner noise floor. 

III. Summary

So what have we learned?

1. Regarding RightMark (I'm using version 6.4.5 Pro), let me just say it's a real pain measuring DSD at the 384kHz setting. There are unpredictable crashes, sometimes graphs don't plot out properly, and I suspect there's some kind of memory hole such that I have to close and restart once awhile otherwise it can hang or refuse to capture the data. At 384kHz, the "Noise Level" graph is also inaccurate so I've resorted to showing the "Dynamic Range" graph so the noise floor lines up properly with the -60dB tone.

Testing with SoX-DSD converted 96kHz signals is comparatively much more stable in RightMark whether in PCM or DSD. Arguably, so long as the results at 96kHz (48kHz bandwidth) look good, I don't think human ears will complain even though it would still be good to make sure the noise higher up isn't extreme.

Despite the bugs, RightMark is convenient as a simple, quick test battery. With a bit of experience, it's not hard to get consistent results. I've E-mailed the author with the hope of future updates and bug fixes but this probably will not be coming any time soon - 2018 was the last update! :-(

If anyone knows of an alternative, easily available software that can do what RightMark does, let me know.

2. The RME ADI-2 Pro FS line of devices (R Black Edition here) IMO remains a fantastic "audio toolbox" to explore features and tweaks like these DSD options. I know it takes some getting used to with the plethora of knobs, buttons, options and features with its heritage in the professional audio world. It's not hard once you get used to it and once you dial in the settings, there's probably no need to fool around too much.

As you can see, with the RME, there's the ability to dissociate "DSD Direct" and the ultrasonic filter (50/150kHz) to map out the effects. In the future, I can look at the results from other DACs and compare their graphs with the ones above to see if it looks like the device is passing along the DSD raw data or if there's low-pass filtering applied. "DSD Direct" playback using my SoX-DSD conversion of RightMark tests will result in those spikes in the 384kHz DSD64 and DSD128 frequency response graphs along with the noise patterns in the "Dynamic Range" / "Noise level" graphs.

If we now look back at the Topping DX3 Pro's performance from Part I:



Clearly, the DX3 Pro is not using "DSD Direct" + 150kHz filter given the smoothness of the DSD64 and DSD128 frequency response curves. At the very least, this is "DSD Direct" + 50kHz filtering, or likely the "Normal Path" ("DSD Direct" OFF). I'm still quite impressed by this little DAC's ability to maintain low noise, high dynamic range, low distortion as an unbalanced output device! I suspect the dual-AK4493 architecture has a large part to play in this level of performance. This DAC configuration paired with balanced outputs could be very impressive (did Topping ever made such a version?). Alas, AKM DACs are getting rarer these days on account of their factory fire in October 2020.

3. We can see the noise level in relation to the complex "1/10 Decade Multitone 32" signal up to 96kHz. The effect of the 50 vs 150kHz filter settings, the fact that higher DSD rates push noise further ultrasonic. I think it's interesting seeing the amount of ultrasonic noise here in the context of the multitone especially when we're playing DSD64.

Given the excellent AKM AK4493 performance for DSD playback, I certainly would not insist that "DSD Direct" must be turned on. I did not notice significant audible changes listening to the RME ADI-2 FS R BE as I played DSD64 tracks and fooled around with "DSD Direct" ON / OFF and the filter settings.

I hope this clarifies the DSD test signals and performance expectations further.

This weekend... A date with some Topping D90SE measurement results.

Addendum: October 8, 2021

Let's add one more section to this. As per Unknown's comments, let's have a quick peek at 44.1kHz signals, 16 and 24/32-bit, converted to DSD64 and DSD128 with SoX-DSD SDM-8. Realize that already DSD128 is rather excessive for low samplerate material so let's not worry about upsampling to DSD256.

The RME ADI-2 Pro FS R Black Edition was set to "DSD Direct" ON and 50kHz filtering which is typically what I use. Here's the RightMark summary table:


As you can see we have 16/44.1 and 24/44.1 signals, DSD64 and DSD128, as well as the original PCM. 

Numerically, even DSD64 is capable of delivering good 24/44.1-level performance very close to the PCM original. No surprise then that 16/44.1 is no problem for even "lowest common denominator" DSD64.


The graphs above show that at least with SoX-DSD, the DSD64 conversions do show a tiny bit earlier roll-off than DSD128 or PCM.

We can easily see the difference between 16-bit and 24-bit signals with the noise level, crosstalk, and IMD+N sweep. Notice how "tight" the 16-bit noise floor/crosstalk/IMD+N results are! Basically, 16-bit playback is really no problem with the DAC regardless of PCM or DSD, both DSD64 and DSD128 are able to play the signal just as accurately as the source PCM. It's only at 24-bits where some differences show up.

Okay, how about the 1kHz 0dBFS (or -0.1dBFS for the DSD conversions to avoid risk of clipping) tone?

Here's 16/44.1 and 24/44.1 PCM playback, I'm running the ADC at 192kHz to show up to 96kHz bandwidth:


We can see the "Sharp" filter that I'm using on the RME DAC with that roll-off in the 16-bit FFT.

Here's what the 16-bit signal converted to SoX-DSD SDM-8 looks like when played back in DSD64 and DSD128 (again "DSD Direct" ON, 50kHz Filter):


As you can see, SoX upsampling to DSD will implement its own steep filtering above 20kHz. Beyond that, there's the usual DSD64 > DSD128 noise shaping visible up to 96kHz. Notice from a measurement perspective that with 16-bit "standard res", there's really no meaningful difference between DSD64 and DSD128 for THD and THD+N.

How about a high-resolution 32-bit/44.1 1kHz tone?


Clearly DSD is able to provide better noise level as expected from a 32-bit source signal. Notice that DSD128 is a little bit better with lower THD although ultimately the noise level between 20Hz-20kHz is about the same as DSD64 and THD+N is only marginally improved.

Let's just end off with a quick look at the "1/10 Decade Multitone 32", we'll just look at the lower resolution 16/44.1 signal:


Notice that even with a signal as "basic" as 16/44.1, the ability to show the individual tones is still very good. The conversion from PCM --> DSD still looks clean with better than 105dB range between the peaks of the tones and noise/distortion. Don't try this with tape or LP playback. ;-)

You can see that the SoX-DSD upsampling filter has a steep roll-off beyond 20kHz which is why that last 32nd tone at 21.4kHz dropped by -24dB.

Okay, I hope the DSD testing is quite complete and anyone curious about DSD playback and what the ultrasonic noise looks like now has gained a satisfactory perspective.

13 comments:

  1. Excellent follow up Archimago, no one with an AP showed these things to me.

    I looked at the multitone graphs very carefully and put a horizontal line on the -130dB marker. The DSD64 plots for example indeed has a higher noise floor with the 150kHz filter. Also, the plot with DSD Direct off indeed has a cleaner noise floor by a pixel or two at around 2-17kHz, a sensitive area. So, while RMAA's noise level plots are botched beyond 96kHz, dynamic range plots are indeed accurate up to 384kHz.

    I believe these differences are caused by Cosmos ADC's aliasing rather than SoX, as SoX's rate effect needs an additional switch to allow aliasing/imaging. Also, mansr last year told me that rate -v is still not the highest one, there is a "secret command" called rate -u. However rate -v already has about 170dB artifact suppression so it is basically irrelevant in analog tests. Also, you can use "sox -V4 (other parameters)" to see more info like filter length when using "rate", you can see that rate -u will show a longer filter length than rate -v.

    While aliasing is a digital thing, in analog domain these wideband DSD noise can also cause IMD in the audible range and increase the noise floor. Of course, it all depends on how the downstream analog equipment handle these things.

    ReplyDelete
    Replies
    1. Thanks Bennet,
      You're the master of these details man and I appreciate the feedback ;-).

      Oh my, "-u" secret command eh? Well, might check that out at a later date given the resolution we're already talking about "merely" for the purpose of human hearing! Gotta finish the Topping D90SE review first :-).

      We'll see that the DSD testing is useful this weekend when we put the Topping D90SE on the test bench.

      Delete
  2. > no one with an AP showed these things to me.

    How could they? The AP system does not support DSD. It could do the analog analysis, but you still need a reference DSD signal to play into the DUT - and that doesn't exist. I applaud Archimago's work here by making these test signals and comparing their result on different DACs. This way we might get usable 'references' bypassing the sleeping measurement industry.

    ReplyDelete
    Replies
    1. Thanks Techland,
      All these years with various audiophiles swooning over DSD, it's unfortunate that we have so little discussed about DSD. Lots of opinions over the years about how some people seem to love the sound (I like it as well, but find PCM with the ability to do customized DSP superior), but it seems to be ignored in objective testing.

      Well, at least this is a little bit more... and will allow me to apply these tests to future DACs I get on the test bench.

      Delete
  3. Hi Archi, thanks for another great post.

    I'm a little confused by the use of PCM384k and PCM96k test signals.

    Especially PCM384k seems extra pointless?

    The most important sample rate to test would be PCM44.1kHz converted to the various DSD rates.

    I think the conclusion would be the same but presentation of results wouldn't look as 'messy' perhaps.

    Keep up the great work

    ReplyDelete
    Replies
    1. Thanks Unknown,
      Well the 384kHz is mainly so we have a look at all that noise up to 100kHz and beyond.

      I think this is important to document even if we can recognize that it's unlikely we'll hear it. As Bennet noted above, the noise could still result in stuff like IMD in the audio band - plus it's nice to have an awareness of the amount of noise up to 192kHz that's being fed to our (pre)amps.

      Delete
    2. Let's see, I'll show some 16/44.1 converted to DSD for you later tonight. Let's just say it won't be very exciting... :-)

      Delete
    3. 44.1kHz RightMark, 1kHz, and 32-multitone posted now in the Addendum.

      Delete
    4. Thanks for the Addendum Archi!

      Regarding your comment "Well the 384kHz is mainly so we have a look at all that noise up to 100kHz and beyond. I think this is important to document even if we can recognize that it's unlikely we'll hear it. As Bennet noted above, the noise could still result in stuff like IMD in the audio band - plus it's nice to have an awareness of the amount of noise up to 192kHz that's being fed to our (pre)amps."

      I don't think we need to fluff around with PCM192k or PCM384k test signals, to be able to see noise with up to 100kHz bandwidth... to do that, it is more important to capture at highest rate the A to D converter can support... that is not related to D to A side.

      Playing a test signal of PCM44.1kHz is more representative of 99% of music listening these days (90% in our audiophile niche too). Then you can capture up to PCM768kHz using ADI-2 Pro if you want to see 'out of band' noise that may impact amp/s downstream....

      Hence my comment above about being confused with PCM384k test signals... and even PCM96k it doesn't seem to add anything other than making overall presentation of results looks very messy , in my opinion anyway

      Keep up the great work

      Delete
    5. Unless I am confusing the sample rates being tested (including PCM to DSD test signals generated) versus being captured by A to D.

      My apologies if that is the case.

      Delete
    6. True, the reality is that a simple 44.1 or 48kHz samplerate capture is good enough for the vast majority of devices and uses.

      Over the years, I've pretty well "standardized" most of the RightMark tests at 96kHz because I think from the perspective of audiophiles, it's always good to go above and beyond to ensure we're not missing anything. I quite like the look of 24/96 results when they're in good shape ;-).

      The nice thing about 384kHz with the Cosmos ADC is that the device has a flat noise floor! Unfortunately the ADI-2 Pro's noise level above 96kHz sampling rate will have a rise due to the modulator noise so you won't be able to see lower level anomalies.

      Not so with the Cosmos ADC's ES9822. In that regard it's like the APx555 and its internal AK5394A as discussed here:
      http://archimago.blogspot.com/2020/07/pictures-audio-precision-apx555-b.html

      [The APx555's high resolution AKM ADC SNR ~120dB can go up to 192kHz (96kHz bandwidth) beyond which the AD7760 takes over with lower SNR ~100dB.]

      Delete
  4. Hi Archie.

    Thank you for the very serious DSD follow up. I haven’t done nor I haven’t seen this details before. I am not a great fan of DSD, as I use Digital Room DSP in nearly every of my HiFi systems, so I need PCM. But it is good to see such results, to have some background information. Thank you for that.

    Also thank you for the link concerning the ESS hump, in your former article. When I started with that, about a decade ago or so, it has taken me months of work, to get rid of the hump, because this was not only measurable, but also clearly audible, with less soundstage with this hump, compared to a non hump realization.

    Juergen

    ReplyDelete
    Replies
    1. Hi Juergen!
      Hope all is well with you and yours.

      Good to try (finally) being a bit more methodical with the DSD testing, using freely available encoding software! Likewise, I actually rarely listen to DSD music these days although I still have all my SACDs on the shelf and ripped copies on the music server.

      Agree, the ability to perform customized DSP in the audio system IMO has been a major factor in improving sound quality - alas, perhaps the one thing that many audiophiles have not had the chance to try (but should!).

      We're at the level of detail now demonstrable with testing where optimizing nuances whether it's the ESS hump or other stuff (like say the THD compensation parameters in ESS DACs) is the work that needs to be done to perfect performance.

      That's actually a great place to be in 2021 as audiophiles, I think!

      Delete