"High-End" DAC Blind Listening Results - PART I: Devices Unveiled!

The time has come friends. Over the last 6 weeks, I've been collecting the data from the 2024 "High-End" DAC Blind Listening Survey. The procedure I think is already well described in that test invite post so I won't spend time here recapping that. Thanks for everyone who participated in the blind listening!

As with many blind listening tests, unless it's literally obvious, I know it's not easy to flip back and forth and it's very common after a few A/B/C switches to second-guess oneself! I certainly appreciate the time that participants spent downloading the >500MB file, getting it onto your listening devices, and the listening itself.

For this post, let's talk about the test, some rationale, and let me unveil the 3 DACs that were used.

A. Why blind listening tests?

First, before dipping into the devices used, I want to just quickly talk about blind listening tests and the importance especially these days for audiophiles. But also limitations by doing this online.

I believe that as audiophiles, we do not perform enough blind listening before we declare all kinds of beliefs to the world. Despite the use of blinding as a core methodological principle in research to collect the most trustworthy human-response data (like in medical interventional research, or even wine tasting), blinding is rarely reported among audiophiles or used in written reviews. Needless to say, over the years, I have written a number of discussions on this point here as well as running blind listening surveys to address at times unlikely claims.

For those who might have missed them, here are the results over the last decade or so for various blind listening surveys/tests in no particular order:

Do digital audio players sound different playing 16/44.1 music? (2019)
Is high Harmonic Distortion in music audible? (2020)
MQA Core vs. Hi-Res Blind Test (2017)
24-Bit vs. 16-Bit Audio Test (2014)
24-Bit vs. Dithered 16-Bit Part Deux: Daft Punk Edition! (2023)
High Bitrate MP3 vs. Lossless FLAC Internet Blind Test (2013)
Linear vs. Minimum Phase Upsampling Filters Test (2015)
LP Needle Drop Blind Test (Roksan/SME vs. Technics SL-1200 system) (2014)

Blind listening tests represent the nexus between pure subjective opinions that could be highly biased (eg. claiming brand name "expensive" devices sound better even if there's no objective difference vs. generic "cheap" devices), and objective measurements that could be so detailed that they're documenting academic differences that no human could ever hope to hear in normal situations (eg. like THD+N differences between say -105dB and -120dB, or even a few hundred picoseconds of jitter).

As discussed last year by Taylor Christensen in his article The Different Types of Subjectivisms in Audiophilia, education and audibility data are critical elements to help us be "better audiophiles". There are now plenty of articles and videos out on the Internet to educate audiophiles basically about anything we care to study up on. However, audibility data is much harder to gather especially since many research papers are behind paywalls (like for the AES; I feel open access is the way to go). Audibility research requires work on the part of those researching as well as participants willing to put in their time and efforts. This paucity of publicly-derived data is why over the years I've been running these Internet Blind Tests online; after all, who else is going to do it? Would audiophile companies that sell fancy tweaks and cables? Would the music industry selling hi-res files want to test their 24-bit files publicly blinded? Would "hi-end", luxury companies want to put their product up for a blinded listening assessment and risk loss of their exotic mystique?🤔

As with any research protocol short of a highly controlled, well funded one, where subjects are matched and brought into a controlled lab environment, there are limitations to these Internet Blind Tests.

For this listening survey, on the side of the test procedure (my end), there is the assumption that the hi-res professional-quality RME ADI-2 Pro FS ADC is good enough to capture the audio output from the DACs transparently. Considering that the RME has the capacity to measure beyond CD resolution in tests over the years, has flat frequency response, and operating at 96kHz is a piece of cake, I see no reason to think this would be inadequate. Furthermore, this is the kind of device used in music studios to capture the sounds that we ultimately hear in the recordings we buy.

Unless one believes that there is a large component of non-electrical "magic" that eludes 24/96 capture, I think it's fair to accept that these files captured the sound coming out of the DACs up to about 48kHz (an octave over the 20kHz audible limit). This bandwidth will also capture much of the digital filter characteristic from the respective DACs.

As for the test subjects - you guys and gals doing the listening - there is obviously no way that I would be able to ensure that all of you listening have your systems set up optimally, that your devices (accuracy of your DAC, amps, speakers/headphones) are up to hi-fi standards, or that your environments (eg. room, ambient noise level) are capable of hi-fi quality. These are real-life limitations. Just have a look at the listening rooms of some of the audiophile reviewers out there (like this, or this) and it's obvious that many supposed Golden Ears are not exactly doing their auditions in ideal spaces either. Furthermore, many audiophiles choose devices like vacuum tube amplifiers that are non-linear. So rather than being a terrible limitation, the fact that I'm getting results from across the globe using different systems with the hi-res 24/96 test samples can be seen as more of a naturalistic result that reflects whether differences will be big enough to be beneficial.

At the very least, these results would be more powerful than YouTube blind listening because of much better volume matching and the use of lossless hi-res samples.

To some extent, I think the concerns above about optimal system setup are likely less of an issue because you're hanging around on this blog. I have faith that my readers here are probably better educated and know what to do with those 24/96 hi-res FLAC test files on their systems than the average Stereophile reader or YouTube viewer. 😁

B. About this listening test and expensive DACs.

Unlike the previous blind listening tests listed above, this one is unique because we're comparing the output from 3 different DACs of wildly different, up to even "aspirational" price points!

According to some, there is a belief that hi-fi, hi-res, DACs sound very different and that a US$10,000 "high-end" device should sound significantly better than something "cheap" like $100. For example, here's Jim Austin's take from his CH Precision C1.2 D/A processor (US$36,000) review in Stereophile, January 2023 (emphasis mine):

"So it's no surprise that you can buy very good Chinese-made DACs that measure very well, very cheaply. Those Chinese DACs are probably designed by first-rate engineers, and while extracting maximum technical performance from a good DAC chip requires care and attention, it isn't rocket science...

What, then, is the point in paying tens of thousands of dollars for a D/A converter?

It's a reasonable question, one that every DAC shopper must answer for themselves. Is extremely low measured jitter, noise, and distortion all that matters in a DAC? Is it sufficient assurance that it will sound "perfect," as good as a DAC can sound? Or is it possible to take this basic technology further, despite what the measurements show? It's easy enough to find people who are quite happy with their $1k DAC and smugly confident that they're getting the best possible sound. But in perfectionist audio (and certainly in this magazine), it's axiomatic that progress is always possible, that you can always do better, and that measurements—at least the easy and obvious measurements, such as S/N ratio, distortion level and profile, and Miller-Dunn J-Test jitter—don't tell the whole story. And if you listen with trained ears through topnotch audio systems well set up, it's frankly hard to miss the improvement in sound achieved by expensive DACs produced by companies committed to achieving the best possible digital sound.

And if you disagree? Then you just saved yourself a ton of money."

(You can see similar claims of audible benefit from a "high end" manufacturer like Paul McGowan of PS Audio - notice the lack of any substance in his response other than "it sounds better". Seriously guys, as a manufacturer who wants to make money on his products, is he even free to express any other opinion?!)

Whether we agree or disagree with the idea that expensive DACs can improve the sound, the more interesting point is not about saving money, but seeking the truth, not just having an opinion, Mr. Austin!

I for one am happy to spend ten thousand dollars on an expensive DAC if it can actually deliver sound quality that is audibly higher fidelity than something that costs $1000. Unless there is evidence, I don't see why any audiophile should believe that "progress is always possible" would be an audiophile axiom (axiom = "a statement that everyone believes is true... mathematicians use the word axiom to refer to an established proof"). This is certainly not well-established general truth since there is more than likely a ceiling effect based on limitations of hearing and cognition; perhaps the more important question is who these "audiophiles" are that believe such a thing. By claiming that it's an axiom does not make it so. In fact, I think it comes across more as a business pitch to maintain enthusiast interest for those who simply want to believe that every few months there's yet another "Best DAC Ever!" to grace magazine front covers.

On a personal level, we can each do our own listening (which I have done for myself over the years and Jim Austin offers his opinion above) but on a corporate level as a "body" of audiophiles, do we in fact have evidence that many of us could tell the difference between inexpensive and expensive DACs - and if we can, that it's "hard to miss the improvement" as Austin claimed?

For this test then, let's go "high-end". We're not talking about $500 vs. $10,000. How about we compare DACs that can be had for US$10 vs. US$20,000 (2,000x price differential) when new based on 2020 MSRP?! 😮

C. The Devices, Unblinded

Well guys and gals, as I mentioned in the listening invite, the order of the devices were picked out randomly by my daughter. It just so happened that they were in price order from least to most expensive. Here we go:

Sample A: Apple USB-C to 3.5mm Headphone Jack Adaptor (US$9, model A2049, released 2018)

Sample A with Radio Shack phono-to-RCA cable

Obviously, in order to play music, we would need to plug this dongle DAC into something like a smart phone or computer so the full cost would be higher than $9. For this test, I used my Huawei Matebook X (2018) Windows 11 laptop and foobar2000 to play the music, 100% volume (0dB). Regardless, this is still one of the cheapest DACs one can purchase today from a reputable brand (Apple). As I said in the test invite, I'm not using any Chinese brands here. Typical for Apple, this dongle was "Designed by Apple in California. Made in China."

There are already measurements elsewhere for this little device and in future posts, I might publish more details. For now, here's the 1kHz THD+N for this DAC (using the E1DA Cosmos Scaler and ADC):

As you can see, with a 1Vrms 1kHz tone (maximum output level this DAC is capable of), the THD+N is close to -100dB. Notice the somewhat broad (by today's standards) "skirt" at the base of the 1kHz tone along with a few symmetrical sidebands suggesting presence of jitter.

[As usual, I'll leave it to you to consider the effect of jitter on audibility.]

--------------------

Sample B: Linn Majik DS + Dynamik Power Supply (2008-2015) MSRP ~US$3,000

Sample B

Compared to our Apple dongle DAC, this is clearly a step up. It's an older model Linn streamer which they produced for 7 years based on the Wolfson WM8740 DAC. With an asking price of around $3000 back in the mid-2010's, it's certainly not cheap but within the Linn product lines, it's one of their least expensive devices.

Rear output features dual RCA and both S/PDIF Coax and TosLink digital outs allow this to perform as a transport:

Within the Glasgow, Scotland built Linn digital streamer product lines, this Majik would be the bottom tier; the three tiers are named Majik (base level), Akurate (mid-tier), and Klimax (top-tier). As you can see, Linn has an aversion to the letter "c", preferring the letter "k"... Cool in a retro kind of way - like Mortal Kombat.

This particular Majik DS model was upgraded with the Dynamik Power Supply a few years back which is their higher performing switch mode device (a few hundred bucks for the upgrade!).

Here's the 1kHz THD+N:

Interesting harmonic distortion pattern with primarily high 2nd and 3rd harmonics and suppression of the higher order distortions at least compared to the Apple USB-C dongle. Peak output level just shy of 2Vrms. Tiny hint of 60Hz mains hum.

--------------------

Sample C: Linn Klimax DSM/2 Katalyst (2016-2021) MSRP ~US$20,000+

Sample C

Of the three samples, this is the most expensive, clocking in at $20k asking price when new. Stylistically, this device looks nice and weighing in at around 20lbs with no physical buttons (there's a power switch located bottom right side), this feels like a solid chunk of metal!

This "Klimax" belongs in the top tier among Linn streamers. The "Katalyst DAC Architecture" is based on Linn's FPGA and uses the AKM AK4497 DAC chip. Since 2021, this device has been supplanted by the latest "Organik Architecture" and the DSM/3 (even more expensive device with a doubled asking price of $39k!).

Linn acronyms: DS = Digital Streamer I assume, DSM = Digital Streamer with Multimedia inputs (HDMI and S/PDIF).

A look at the rear:

As you can see, there's a good complement of ports back there including HDMI input and out, S/PDIF Coax and TosLink in, plus it also has balanced analogue XLR in and outs so can function as a DAC and preamp.

For consistency with the other devices above, I used the RCA output for this blind test. In theory, if the DAC architecture is audibly superior given the very significant jump in price, we should be able to hear the difference without needing to resort to balanced XLR output I would think.

Here's the 1kHz THD+N:

FYI: when tested with XLR out, THD+N improves to -110dB thanks to lower noise level.

Notice that objectively, we can see from the 1kHz THD+N graph that the Klimax DSM/2 is measurably better than the other two devices above (I separated R and L channels for a better look). There's lower THD+N overall, cleaner noise floor (well below -140dB), no "skirt" or sidebands (low jitter), no hint of 60Hz hum even with unbalanced RCA, highest harmonic distortion is the 3rd down at -105dB with excellent suppression of even harmonics.

At 0dBFS, the Klimax DSM/2 output level is slightly louder than the previous generation Majik DS so if you A/B compared the two of them using the same volume level through your streaming software like Roon, the much more expensive Klimax would be about 0.6dB louder. Due to psychoacoustics, the listener will likely prefer the sound of the Klimax if he did not compensate for this slight difference. BTW, I noticed across the Linn product lines that output level has increased over time so I assume this is by design.

Whereas I used the Windows 11 laptop and foobar software for the Apple USB-C dongle, ethernet streaming to both the Linn Majik and Klimax was with Roon done bitperfectly - no DSP or volume reduction in software or applied within the Linn set-up.

If you're wondering, to make extra sure that the sound was not being degraded, both Linn devices were captured using Ecosse's The Diva cables (precursor to the current Nu Diva Premier Plus silver plated and cryogenically treated interconnects - US$300/m) as opposed to the inexpensive Radio Shack 3.5mm phono-to-RCA cable used for the Apple USB-C dongle as shown.

--------------------

As you can see based on the THD+N graphs, from an objective perspective, there is some truth in what Jim Austin said about companies like Linn "committed to achieving the best possible digital sound" having made progress over the years. There is a jump in performance from the Majik DS to their next-gen Klimax DSM/2. We can say this for "Chinese-made DACs that measure very well, very cheaply" also over the years. But when it comes to whether measurements "tell the whole story" or not... Well, that's where we have to examine the audibility data to find evidence that audiophile listeners preferred the sound of the more expensive DACs. Only in doing so can we address the question of utilitarian value beyond the non-utilitarian benefits of owning a luxury device. If we can confirm that there's an audible difference that listeners prefer, then maybe we can tackle the question of benefit between $3,000 vs. $20,000 asking price. Neither Jim Austin nor the typical audiophile press (whether "professional" or "amateur" YouTubers) have the ability to answer these questions for us. Nor would they typically be motivated to do so if the objective is to help sell these products influenced by their reviews.

The listening survey has now closed. Thanks to the 105 of you from around the world who submitted your listening impressions. There's "power" in having numbers like this - I believe way more powerful than the mere opinions of individual unblinded claims and more honest than unblinded audio reviewer/manufacturer/dealer's opinions.

To keep myself as unbiased as possible in my discussions online, I have not looked at the results across this period of data collection. It's now time to look under the hood and do some number crunching with a summary for you over the next little while.

Finally, I would like to thank my buddy linnrd for hosting the listening, measurements, and audio recording sessions as we used his streamer and others he sourced from his friends for this test! 👍

--------------------

In other news this week, for those interested in multichannel/Atmos, here's an interview with Steven Wilson about Spatial Audio from the recent HIGH END Munich 2024:

There's an important reminder that multichannel is just the next step beyond mono → 2-channel stereo (which is also "spatial", as in being able to better convey the soundstage). He spoke of the concept that by spreading music beyond the two channels, this spacing can also help us appreciate details otherwise easily missed - "almost like sometimes music is too big for stereo to contain", referring to the nuances in complex studio productions. Also, the genre matters when remixing; for example, electronic pop being more abstract than say something like an actual live performance recording, the mixing engineer can be more "playful" and creative.

(Relevant discussions over the last few months here: Fidelity, Immersion, and Realism and Beyond "High Resolution" Recordings. Expanding the spatial aspects of 2-channel sound with crosstalk cancellation. Why there's no need to fear Atmos.)

For a taste of some interesting playful/creative pop remix, check out LSD's Labrinth, Sia & Diplo Present... LSD (2019, DR6 stereo, DR13 multichannel/Atmos mix).

Another example would be Billie Eilish's new album released this past week HIT ME HARD AND SOFT (2024, DR6 stereo 2-channel, DR12 multichannel/Atmos):

I'm just getting into this one with its complexities and nuances. Good that on the whole the album isn't as "emo" as the track "What Was I Made For?" from the Barbie soundtrack once we get past the first track "SKINNY"; I like my music a bit brighter than that. We saw some interesting sonic transformations on her previous album Happier Than Ever, and in this one there are some interesting genre-bending metamorphoses in tracks like "CHIHIRO" and "THE GREATEST". Finneas (Eilish's older brother) produces and cowrites these albums and clearly has a huge influence on the sound.

Until next time (Part II), happy listening, dear audiophile! Hope you're enjoying the music.