MEASUREMENTS & LISTENING: Topping D30 DAC by Greg Dunn. (And briefly, more USB cable nonsense.)

[Ed: Every once awhile, it's a pleasure to have guest posters adding their voice to this blog! There are many folks out there doing measurements, experiments, and I've certainly been in contact with a number of audiophiles with great stories about the gear they have, music they love, and the audiophile pursuit in general!

A few weeks back, Greg got in touch with me around the Topping D30 DAC that he has. This DAC was brought up in comment discussions back in 2018. The D30 measured quite well on Audio Science Review, but has been blasted as subjectively sounding bad - "like ass" as it were - on the Super Best Audio Friends forum. Elsewhere in that SBAF comment thread, the D30 was described as having "poor soundstage and poor timbre".

With this polarization of opinion, it's great to see Greg stepping up with his own opinion based on objective testing and subjective listening... Take it away, Greg!]

I recently had the opportunity to measure and audition several DACs varying from cheap to moderately priced, and learned quite a lot about my measuring system in the process. Not yet being ready to commit to (afford) a laboratory-grade setup, I decided to go with an inexpensive but capable external sound interface and see what I could find out about some of the electronics which I had access to. My choice, based on reviews, was a Focusrite Scarlett 2i2 (3rd gen) which seems to have quite respectable performance for the money. I realize I'm pushing its limits for some measurements, but since my goal was more to get a sense of how the equipment under test measured than to characterize it to ten decimal places, I’m pretty happy with the results. For vintage electronics and most modern stuff, it’s more than adequate.

I. Focusrite Scarlett 2i2 (3rd Gen) ADC Loopback

Unfortunately, even though it works perfectly with 24-bit and high rate audio signals, REW currently only displays down to -96 dB of relative input level which is a limitation in these tests (this will be changed in a future release).  I have done spot checks on the Scarlett and Topping showing good linearity well below these levels, but it’s harder to plot the data without an automated setup. I use 96k/24 bit mode for consistency between measurements and to ensure that there isn’t an artificial roll-off at 20 kHz which might bias the measurements.  Here's a plot, in loopback mode, of the Scarlett's frequency response. This is pretty uninteresting (good as a measurement device):

Next, the Scarlett distortion vs. frequency baseline:

Focusrite Scarlett 2i2 (3rd Gen), THD, noise level in loopback.

Distortion is well below 0.01% or better than -80dB with the input signal just below clipping. Noise is ever further down, by 20 to 30 dB. Not state of the art, but decent for an inexpensive tool and good enough to measure most of the equipment I want to look at (which is mainly speakers and power amplifiers). These numbers aren’t as good as other measurements of the Scarlett I’ve seen; it's likely possible to improve the performance of my bench setup with some effort. Since this is mainly for my own comparisons and to verify that my DUTs are working properly, I'm not too worried about not squeezing out the absolute best results from this setup. Maybe some day I can afford an RME Pro for testing, but not today. :-)

I recognize that most any really good piece of equipment will be limited by the Scarlett's limitations, so I do not claim the following numbers to be fully representative of the devices being tested – all I can say is that the device is at least this good, and probably much better.

For reference, the Scarlett 2i2 (3rd Gen) distortion vs. level:

If I had set this up so that the Scarlett did not clip, the distortion and noise floor measurements would have exactly paralleled each other.  In order to show the clipping behavior, I increased the gain on the Scarlett by about 10 dB for this measurement only. It was useful to have this info, so that we know the Scarlett is very clean right up to the point where its inputs are overdriven. For the Topping D30 measurements, the Scarlett's gain was adjusted so that it did not clip at all; any evidence of clipping would therefore be that of the Topping alone.

II. Topping D30 DAC

Now, on to the Topping D30 measurements. As you can see from the reviews discussing the DAC's features (such as this one), this device is rather flexible with USB input as well as both coaxial and TosLink SPDIF. Outputs are single-ended RCA.

First, let's have a look at the USB input frequency response:

Nothing to see here; ruler flat response to well beyond audibility.

Next, the distortion measurement (THD and noise) vs. frequency:

Much of this is limited by the Scarlett's internal distortion though there does seem to be a little added noise, more than likely below audibility.

Finally, the distortion vs. level at the Topping input:

It's apparent that the distortion (+ noise) decreases steadily as signal increases. Unfortunately, with the current version of REW it’s not possible to overdrive a purely digital input so 0 dBFS is the limit for testing (and neither device clips at this level). Again, behavior is limited mainly by the Scarlett’s input noise + distortion.

The optical TosLink input's measurements are so close to these values that there isn’t much point in showing them. The USB input might be 1-2 dB better at certain frequencies or levels, but for all practical purposes, results were identical.

III. Listening tests

Of course, the real purpose of this DAC is to listen to music through it, right? There have been tests which claim the D30 has an audibly different "sound" compared to other DACs, and even condemning it as sounding "bad". So ultimately, after finding that the device measured quite well and seemed to behave as a DAC should (within the limitations of the test equipment) the only thing left was to listen to it.

Sadly, I don't really have a double-blind test setup for comparison to another DAC, but I can still try my best to A/B compare using a switch box and a high impedance passive attenuator (to match levels). This will easily allow me to compare line-level devices, but there will be no easy way to randomize the selections. I was prepared to take more thorough measures in the event that there was an audible difference, but first it seemed prudent to try and see if such a difference existed.

For the test, which took most of an evening, I compared the D30 to my regular DAC, the SMSL M100. The SMSL actually measures about 10 dB worse in both SNR and distortion, though still quite good in overall sound. I found out when hooking them up that one device had higher output voltage at 0dBFS than the other, so I knew I was going to have to level-match them if there was any hope of achieving an accurate listening test. It turns out that the difference was on the order of 1.0-1.5 dB when connected to the stereo, so that needed to be dealt with using a test tone and a meter.  Unfortunately, unless the overall level was precise (volume control potentiometer linearity is not always perfect) there was still a slight difference in output between the DACs. Since sub-1dB differences are audible, I had to set the overall level with care to ensure essentially identical volume. This was aided by the in-line attenuator actually having detents on its rotary position so it could be precisely and consistently set, independent of the system volume control. Very handy as we'll see.

I tried both the TosLink optical and USB inputs for the DACs. Just to be clear, I did not notice any difference between the digital inputs which would affect the outcome of my listening test. Alas, I don't have a S/PDIF coaxial source to test that input.

I started playing music, comparing the two DACs. As it happened, I was using two identical RCA cables to connect the DACs to my stereo, and I actually forgot which pair was which after connecting them and putting the equipment all back in place on the shelf!  I decided this was fine, because even though I don't have the ability to easily randomize, at least not knowing exactly which wire was which provided a level of uncertainty to reduce bias.

In a nutshell, no matter what music I chose and which DAC was playing, I couldn't hear a difference! I tried to take my time and listen for several minutes at a stretch, as well as switching rapidly between DACs during parts of the test – which is also useful.  There are adherents to both methodologies, so I tried both – which did prolong the test somewhat, because I refused to rush the test just to get it completed. I tried rock, classical, and jazz selections; vocals and instrumentals; samples of the best recordings in my library. It didn’t matter; the two DACs remained, after the evening's work, as nearly identical as I could have expected.  I “thought” maybe I knew which one was which, based on the way I usually wired my inputs, but to be completely honest, it was totally a guess. Very interesting!

IV. Conclusions

At the conclusion of the formal test, I thought it was time to try something I had noticed in a blind test many years ago and which is mentioned but not talked about in many modern reviews. I bumped the stepped attenuator up one notch, making one of the DACs slightly louder (about 0.5 dB according to the meter). Instantly, one of the devices sounded “better” in a vague but consistent way – specifically, it didn’t sound louder, just different. I returned the attenuator to its original position and the difference vanished. Then I turned the attenuator down by the same amount – now the other device sounded better. The result was repeatable, with multiple iterations and multiple music types.  Yes, this was actually sort of a sighted test, but I still wasn't certain which DAC was which – only which input channel I was using.  In fact, when I unplugged them after the evening's tests, I purposely didn't note which cables went to which DAC so any sort of retrospective bias could be avoided. 

After the evening's tests, I came to two conclusions:

1) The Topping D30 (~US$120, Cirrus Logic CS4398 DAC chip) and SMSL M100 (~US$80, Sashi Kasei AKM4452 DAC chip) sounded identical in my system.

2) A very slight level difference of ~0.5 dB between the devices was immediately noticeable and significant in terms of perceived sound quality, at normal listening levels.

In my experience with this and another earlier test, my ears needed at least 1 dB of sound level difference before perceiving it as actual "loudness"; below that value, the louder device exhibited a consistent but vague "improvement" in sound quality, which correlates fairly well with other properly-conducted listening tests I’ve seen. The ear is an awful precision measuring device. Even though one of the DACs had about 10dB worse noise and distortion performance than the other (though still very good), that wasn't audible if levels were matched. Level differences totally biased the listening test results, as I had expected and was glad to confirm. If you can't match levels to the order of 0.1 dB, you’re likely introducing a bias to the test which can't be dismissed.

At the moment I'm writing this (actually before compiling all the test results) I still am not sure which DAC was which with certainty.  When I look at the objective data, of course, I’ll know for sure (because of the output level measurements) and that's why I'm writing this portion first even though it's at the end of the write-up. All I can say is that I'd be happy to have either DAC in my system; in fact, I have left the SMSL connected, in spite of its slightly worse measurements, because I like the digital display of bit rate on the front panel. It's as simple as that. Other than visual presentation, either one is satisfactory in my stereo and as far as I’m concerned, they’re both recommended.

--------------------

Thank you Greg for that excellent write-up which also included the loopback results from the Scarlett 2i2 DAC/ADC. The objective results of the Topping D30 certainly look good and I really like how you took the time to perform the listening tests! The fact that you were unclear about which input cable connected to which DAC is a nice twist on being "blinded" :-). Might be a good way to reduce bias if one can't get a friend or family member to help randomize for listening tests so long as levels were matched before hand.

Also, that qualitative comment about 0.5dB difference is an important one to keep in mind. Often we are told that a 1dB difference is the threshold of audibility for amplitude change. I agree with Greg that it's lower than that. Even if we cannot fully verbalize the experience, small differences like 0.5dB can be perceived as "different" and typically, a slightly louder sound of +0.5dB will be "preferred" as the better-sounder version in comparisons.

Obviously we must not disparage the importance of our ears and mind as the facilities from which we perceive the complex emotional tapestry of music. This is a fact to be celebrated. But as an actual "instrument", I agree with Greg that human hearing is understood to be far from a "precision measuring device".

I hope readers who might have strong beliefs in audible differences of high-fidelity DACs but have never tried a level-matched, preferably blinded, comparisons like this consider Greg's article as encouragement to experiment for yourself. Seems to me like the Topping D30 doesn't "sound like ass". :-)

Thanks again, Greg!

In other news this week... I see Darko is tripling down on the idea that "USB cables make a difference". As you probably know, I discussed this topic last week. Good to see that he's now arguing with some objective data like the 8kHz noise (which we've known about for years) that is not a problem with any decent DAC I've ever come across. Realize though that the argument was never about people hearing an 8kHz tone from their DACs (even that graph he posted showed a -105dB tone!). It was the idea that typically expensive digital cables (like these he reviewed a couple years ago) lowered audible noise, improved frequency effects ("filtering", better bass and treble), and improved jitter. Sure, USB data errors happen (in reference to Gordon Rankin's screenshot) - just get better cables that don't have to be expensive. Juergen Reis' quote in the article about PHY noise and jitter does not suggest that cables are the culprit nor that cables could "fix" a problematic, jittery DAC as far as I can tell.

The funny thing is that Darko is framing this subjective awareness of USB cable differences as some kind of transitioning "period pain". Perhaps insinuating that something like a "paradigm shift" is happening in digital audio thinking!? (Not working so well for MQA.) Supposedly, like the Earth is "spherical" vs. "flat", he reckons that his subjective opinion on USB cables (Industry-supported with interviews and such) leans toward the "spherical" side of the argument and will eventually be proven right? Well, I'll take my chances betting that audiophiles will turn to being "more objective" and discard the vast majority of cable beliefs, the "need" for expensive cables and associated groundless rationalizations fueled mostly by the advertising budgets of the last few decades. That kind of fantastical thinking is IMO regressive, a return to superstitions.

To me it's a bit disappointing that Simaudio would talk about stuff like USB cables (1:40) as I've always viewed them as quite reasonable fellow Canadians aiming to produce rational audio products. Looks like Costa Koulisakis, the "Director of Training" is more on the sales and marketing side than holding a technical portfolio in the company, thus not introduced as an "expert" in the article. As usual, we see more hand-waving around "listening tests" with zero details provided. Much of this is clearly nonsense and not applicable to the majority of audiophile systems.

The recognition that the Earth is spherical (or more accurately "oblate spheroid") belongs in the domain of natural sciences discovered and verified to be true by various objective means. Remember, USB cables (and digital audio in general) are products of human engineering with properties endowed by the USB Implementers Forum (USB-IF). Let's not be so grandiose as to suggest that the belief in fancy USB cables is in any way approaching a fundamental understanding of the natural world! Here's a thought, maybe Darko should interview some engineers in USB-IF companies to hear their thoughts on the "sound" of USB cables or even whether "bits are bits"... After all, they are the Creators of the various versions of the digital interface standard since 1996!

Eventually, knowledge like "the Earth is spherical" becomes self-evident for modern humanity; surely there's no need to call in an "expert" every time someone comes around and screams "The world is flat!", right?  Isn't this where we are with USB cables (and many other esoteric audiophile beliefs)? Experts need only be called if there is evidence to challenge the default belief, to determine the validity of such findings. Where is the evidence that one "needs" to or "should" spend more than a generous US$50 on USB cables to make any sonic difference?

The world is not black & white. Audiophiles are not either "experts" or "sheep". Just because one might not be an "expert" does not mean one is uneducated, cannot do a bit of research, or unable to activate one's critical thinking abilities. Those are I believe also the hallmarks of good, honest journalists. I suggest that Mr. Darko do a bit more research and soul-searching looking outside the audiophile bubble instead of following along just with Industry "experts" eager to give interviews. (I realize that I'm making an assumption here that he sees himself as a journalist. Perhaps in reality his role is as a "salesman" in which case he's probably doing a fine job for the companies.)

As usual, enjoy the music! Recently I've been digging into the works of Jerry Garcia's later years and his collaborations with David Grisman... There's the well known Garcia/Grisman (1991, available as SACD and 24/96, DR14), also 1993's Not For Kids Only, the jazzy Miles Davis cover So What, The Pizza Tapes with guitarist Tony Rice, and the posthumous Shady Grove that came out the year after Garcia's death. Generally, excellent acoustic recordings with good sound to feed the hi-fi. ;-)

And remember to do the blind test. Remember that I'm simply looking for honest results. Responses are collected completely anonymously, and a negative result is just as useful and as important as one who hears clear differences.