Over the years, we've seen websites, blogs, and videos try ways to demonstrate the sound of a hi-fi device or for the adventurous, even try to convey the sound quality of listening rooms. For example, there are binaural recordings at audio shows, soundroom demos, speaker comparison samples, etc. This is not an easy task because high-fidelity is about nuances and slight variations; not wholesale "obvious" differences for many devices like DACs or high performance amplifiers. Unlike what you might read in audio reviews, assuming you have a decent DAC already, a replacement would be unlikely to result in obvious changes in characteristics like bass response or claims that jitter effects are somehow obviously audible! Sorry folks, a lot of that kind of talk is just fantasy.
While it is convenient to view and listen to typical YouTube clips, I think we can all appreciate that sound quality would be highly affected by: the recording microphone, room acoustics, set-up quality, the lossy audio compression from YouTube among others that I may have missed. Dissociating the effect of the different components would be impossible. And obviously any time you use a transducer to convert the sound pressure into electrical signal (ie. speaker, microphone), there will be a significant reduction in resolution if we're trying to determine the effect of something like a DAC!
Then there's the issue of what music is being used? Is it music that audiophiles have general access to? Is it material that audiophiles/music lovers would even generally listen to? Obviously this bit is very subjective but I think there's something to be said about esoteric test material that might be recorded amazingly well, but just not adequately popular to have "mainstream" level acceptance. When "subjectivist" audiophiles complain that test tones are artificial and synthetic, is it that much different from listening to a handful of albums that barely anyone cares about? ;-)
While thinking about this recently, and having measured enough devices over the years and showed you numerous graphs and synthetic test material, why not expand this by giving you the opportunity to listen for yourself? Using the highest quality "recording" of devices that I can...
With that in mind let's lay down what I'm going to try to start doing with some of my future DAC/source measurements and reviews; I'll include a link to a hi-res recorded playback from the device so that you can listen for yourself and compare at home.
A few basic points about how I'm going to do this and then some actual recordings of devices:
1. I'll record in 24/96 direct from my RME ADI-2 Pro FS ADC which I know is capable of excellent resolution - at least capable of measuring down to -110dB THD+N with single-ended RCA input, and -113dB THD+N with balanced XLR. This is beyond the resolution from any speakers or headphones.
24/96 would be beyond human perceptual ability as well so let's not waste storage space with 24/192 or DSD.
The ADC will be set to linear phase, steep filtering. Given the 96kHz samplerate, we're looking at filter roll-off out beyond 40kHz, obviously an octave beyond even ideal (teenager, early-20's) hearing acuity.
2. Let's use a selection of test music, nothing terribly esoteric, at least good production quality, for subjective evaluation. Here are the tracks I selected based on some of the stuff I listen to:
A. Eiji Oue & Minnesota Orchestra – “Infernal Dance of the King Kashchei” from Stravinsky: The Song of the Nightingale / The Firebird Suite / The Rite of Spring (1996, DR14) – 24/88.2 à 16/44.1 using iZotope RX 8. Classical track, strong dynamics, listen for nuances, “microdynamics” in the orchestra. Fantastic production from Reference Recordings.
B. TakéDaké & John Kaizan Neptune – “Japanese Roots” from Asian Roots (1998, DR14) – ethnic wind and percussion instruments. Listen for dynamics, transients, smoothness as notes trail off and the fade to silence between passages. How "palpable" do the instruments sound through your headphones or in your room? Probably the most esoteric track here but it sounds pretty cool.
C. The Ray Brown Trio – “Summer Wind” from Live at the Loa: Summer Wind (1988, DR12) – I used the 2003 Concord SACD release DSD64 à 16/44.1 using Weiss Saracon - live, jazz, slightly longer than 2 minutes. Listen to the pacing, timing, flow. “Presence” and “atmosphere” of the audience, instrument spread over the soundstage, ability for DAC to separate bass, piano, percussion, and even the occasional background vocalizations.
D. Eva Cassidy – “Fields Of Gold” from Songbird (1998, DR11) – female vocal – listen for any excess sibilance, vocal presence, neutrality, and quality of the vocal reverb. Nice accentuated "attack" on the acoustic guitar on this cover of the Sting song.
E. Benjamin Clementine – “Winston Churchill’s Boy” from At Least For Now (2015, DR8) – male vocal – “body” of the voice, clarity of articulation, separation of voice from instruments, soundstage placement, tonality of the instrumentation, bass quality.
F. Dua Lipa – “Love Again” from Future Nostalgia (2021, DR5) – modern pop production – ability of the DAC to handle typical multilayered, “loud” / dynamic compressed modern Top-40 type music (+0.9dB “true peak” on this recording). [Notice the sampling of "My Woman" by Lew Stone & The Monseigneur Band from 1932, also used in White Town's "Your Woman", 1997.]
Age of the recordings range from 1988 to 2021. We have classical, world/ethnic, live jazz, female vocal, male vocal, and modern pop represented. As described, each will bring certain characteristics to listen for like the strong dynamics in the Stravinsky piece (this is not the HDCD version but sourced from 24/88.2 Reference Recordings hi-res), the clarity of "Japanese Roots", live ambiance of The Ray Brown Trio, guitar and voice on "Fields of Gold", bass authority on the Benjamin Clementine track, and louder compressed dynamics of "Love Again".
Some might wonder why I didn't include a techno or EDM track. Well, already the Dua Lipa track employs quite a bit of studio processing - artificial sweeteners for the ears, and maybe eyes as well but that's subjective ;-). In general, I think it's more useful to assess whether a DAC or CD player sounds "natural" as in the tonality of voices and acoustic instruments that can engage our judgements based on previous real-life experiences of these types of sounds. Playback quality of electronic music probably can already be assessed quite well with the results of measurements (synthetic test tones for synthetic music!).
3. ~90 seconds for each of the test samples above. This should be long enough to allow you to pick out interesting portions to focus on while at the same time provide a "gestalt" of the music. I increased the time to about 2 minutes for The Ray Brown Trio to appreciate the pace and flow of the live recording.
In total then, the composite test track lasts a bit over 9 minutes of audio. Here's a look at the waveform in Adobe Audition:
4. Let's stick with lossless 16/44.1 source. No need to argue or debate over hi-res these days I think (see here, and here). Even if you can hear a difference with hi-res, all indications point to the difference being minuscule. The reality is that for the foreseeable future, stuff that we consume will be CD-resolution 16/44.1 or equivalent (like upsampled and pseudo-hi-res). I believe there are 24-bit versions of Dua Lipa and Benjamin Clementine out there, but seriously folks, these kinds of low-DR albums (as discussed here years ago) do not warrant spending money on and would be an absolute waste of storage as well.
For future reference, I'll just call this test track and the way the DAC recordings performed the "AMPT". When doing the AMPT recordings, I'll make sure to include a few seconds of silence at the start to allow you to examine/hear the "sound of silence" from the device; very important to listen for hum for example. Also, in order to allow for easier "apples-to-apples" comparisons, the recorded track volume with be normalized to -27.2dB +/- 0.1dB average RMS amplitude using Adobe Audition 14 (2021); this is at -3dB of the source amplitude. This will allow for overhead such as the Dua Lipa track with +0.9dB true peaks. IMO, the high-resolution, very low jitter, studio-quality RME ADC (for the record 18kΩ XLR, 9kΩ RCA input impedance) will easily capture all that we need from these 16/44.1 digital-to-analogue conversions.
Here's the original track for those who want a download of the source 16/44.1 file:
AMPT - Original Test Track (16/44.1 ZIP, 53MB)
** VERY IMPORTANT: If you want to compare direct playback of the original track above on your system to some of the recordings of DACs I'll be posting below, make sure to increase the volume of the recording of the DACs by +3dB. I've noticed over the years of running blind tests that some people don't bother controlling volume and then make all kinds of claims of what they heard. Also, make sure to turn off any extra processing when listening (such as volume normalization, EQ and such which may affect quality).
As in the past with blind tests, the tracks are being used based on the principle of "fair use" for the purpose of education and testing. Only short portions of the music tracks are used (significantly <50% of the whole song). If you enjoy the music, purchase the full album and support the artists. I've included Amazon Affiliate links, the expectation is not one of making a sale but for convenience if you're interested in purchasing the music.
So, let me get you guys and gals started on "hearing" the output from a few DACs.
 |
| Here's a shot of the RME ADI-2 Pro FS R Black Edition being recorded. Using some inexpensive but good "generic" SRADIO 6' XLR cables I had around here. |
 |
| FFT of the RME ADI-2 Pro FS R BE recording - note the filter roll-off character and clean ultrasonic noise floor. |
 |
| D'oh! How did that Sleeman beer get in there? 3' generic shielded RCA cable used. |
 |
| Still big, bold and beautiful after all these years (bought in 2013)! "Sharp" filter used here as you can see. |
Logitech Squeezebox Touch (AKM AK4420 chip, RCA) AMPT
 |
| Notice the 60Hz mains noise and 180Hz 3rd harmonic using single-ended RCA. Quite a bit of higher order harmonics by today's standards. Noise level significantly higher than the TEAC above. |
 |
| Not the best DAC measurement results these days but still not bad and certainly better than majority of amplifiers. |
 |
| A decade back, "asynchronous" network audio devices like the SB Touch communicating with the server, even with WiFi, utilizing a good sized data buffer had no significant jitter issues. Notice that all the anomalies are below -120dBFS. |
 |
| A quick FFT similar to the RME ADI-2 Pro FS R Black Edition (not exactly same spot in the music). As you can see, a bit of extra ultrasonic content on this analogue out; clearly not as clean filtering as RME. |
Happy listening! Feel free to do quick A/B switching such as using foobar ABX Comparator. Software like DeltaWave could also be helpful to analyze for differences (like relative temporal drift between devices). If you look at the recording FFTs, you'll also get a sense of the variations in digital filter quality; particularly the roll-off around Nyquist (22.05kHz) and the presence of ultrasonic imaging artifacts from the DACs up to 48kHz. About a year back, we discussed the audibility (or non-audibility) of parameters like THD+N so the recordings here may or may not confirm such things if you look at the DAC measurements and listen to these samples. Ultimately, hopefully this continues to add to our understanding about the magnitude of differences between hardware like DACs these days without resorting to "magic" or impressionistic words (typically, hyped up descriptions from reviewers or in ads aiming to just sell hardware).
I'm reasonably technically competent but couldn't figure out how to download. Doing a right-click on the link and requesting download gets me a 120K HTML file. If I click on it I get into a directory-like thing but can't find anything to click on that yields the actual URL of the zip file.
ReplyDeleteIf this is causing grief with your hosting arrangements, I could arrange to host it in reasonably-efficient web space for free. Not looking for any money or credit or anything, just a URL that yields the zip.
Ah, figured it out. You have to click on the link, then in the directory thing, click on the file name, this takes you to a black screen that has a "Download" label at the top right corner. Click on that and you're good to go. Might want to put instructions in the piece?
ReplyDeleteThanks for this!
I think you just gave the instructions Tim ;-)
DeleteYeah, it's on the Amazon Drive. I figure for ease, I'll just put stuff over time on there and links. All in one place.
Hopefully no problems downloading for others!
Fascinating as usual, Archimago.
ReplyDeleteThis is off-topic but I wanted to say a word about Topping - not the devices so much as the company.
I ordered a D10S on your recommendation and could not be more pleased. It's stunningly good for the price. I won't get into the sound lest I provoke the subjectivist reflex; let's just say that in my view, its as good or better than any DAC I've ever had, including a couple costing thousands.
On to the company itself, and its representation in Canada.
The USB cable that came with the DAC didn't work. At first I thought I had a dead DAD, but when I switched out the cable a substituted one of my own, the D10S functioned perfectly. This was true with two computers and several cables.
I just threw out the cheapie and forgot about it. But then, I got something I've never seen before - a note from the company's Canada rep via Amazon, asking if I was satisfied with the service.
I explained the cable issue and she was very contrite. The next day she got back to say she had notified Topping in China of the problem, and they would ship another cable.
I said it wasn't important and not to bother, there was just no need.
Then she insisted on refunding the cost of tax on the sale. The repayment, a modest $5.73, was on my credit card the next day, again via Amazon. She said Topping is very focussed on quality control and would look into the problem.
I buy a fair bit on Amazon because of the convenience in time of COVID. I have never before had a retailer contact me for my opinion, let alone offer a refund when I didn't even request one.
Very impressive, I thought.
Obviously I meant DAC, not DAD!!'
DeleteThanks for the note Don,
DeleteFixed the repeats ;-).
Yeah, I've had that kind of experience from the representation here in Canada as well. Aoshida I know at least manages some of the sales here and have been very good at customer service in my dealing with them when I got the D10 Balanced as well.
[Disclosure, I did get the SMSL M100 MkII to test out from Aoshida back in January. This comment has nothing to do with that and as far as the Canadian operations is concerned, I'm just a customer...]
"I might be able to improve this on it's way... ;-)"
ReplyDeleteyou mean "its" don't you :D
Oops, thanks for that verifonix.
DeleteHad a nice glass of red in hand when typing that ;-).
Depending on the speed of shipping, might show a little preview next weekend.
Archimago wrote, "Yes. It sound fabulous.". Is it more fabulous than the others? :-) I listened to your test tracks and I don't think that I'd be able to tell a difference in a blind test. Couldn't really tell a difference in a sighted test. Impressive how good the SBT does. If there's one that I think I could pick as distinct then that would be the one.
ReplyDeleteAs I was listening to the tracks, I'd think that I had heard something different from one DAC to the other. When I would queue to the same passage and swap between them, then the differences became very elusive. This was a fun exercise; very creative. Keeps my feet on the ground when reading all the subjective viewpoints and when seeing measurements of technically superior new gear. Will be fun to, "hear", the differences with the new D90.
Yup Doug,
DeleteI think this is what's important to keep in mind and hence my comment at the start of the article. It's a fantastic fallacy and IMO disservice for audio reviewers every time they get a new "next generation" DAC to keep telling people just how amazing the new product is.
Yes, I know companies need to generate interest. Likewise, the YouTube guys, print magazine guys, bloggers, etc... have to promote interest to keep the $$$ flowing for themselves and the companies.
The science has consistently pointed in the same direction for years now. We are looking at mature 2-channel DACs which have hi-res levels of performance at least approaching if not easily exceeding the hearing abilities of typical audiophiles (who again are probably not teenagers or in their early 20's). This goes for the SB Touch even though of the group above, it is clearly the lowest performing!
The D90SE is without doubt supremely high-fidelity; this is a fact. I listened to it before running that simple THD+N (which literally takes 5 minutes). I can tell that this belongs in the family of highly "accurate" audio components. But to be honest, does it sound "more fabulous", or simply "better" than an already excellent RME ADI-2 Pro FS R in the sound room, or if I had the D10 Balanced in there, that I would "easily" be able to tell the difference? Of course not!
Value judgments will need to go beyond just what is heard because as a human being, my hearing is simply limited now that I'm almost 50 compared to 20 or 30 years ago!
I'm sure we'll talk more about this when I write about this DAC and think about the intersection of objective performance and subjectivity/psychology. Values, philosophy, and psychology play just as much as auditory perception at this level of performance!
Hi Arch,
ReplyDeleteNice test again, somehow like the previous blind one with complete pieces of equipment but now restricted to individual DAC. Good choice of material with the whole gamut of DR values.
What I find is, even with 24/96 ADC/DAC recording there is still a minute amount of quality lost from the original. Slightly more punch, more air, more details in the original… I guess that’s normal with every additional step taken, as good as it might be. Of course I increased the volume by +3dB to the recorded versions to be fair.
But comparing the recordings themselves shows little difference, except maybe when comparing directly the best (Topping D10 balanced) with the worst (Squeezebox), unless you played us a trick by mislabeling them ;-)
Summarizing, I find the Topping much more precise than the Squezebox whose sound is a bit muffled and soft in comparison. Not that it could be noticed if listened to by itself.
I concentrated on the dynamics of the Stravinski, and the floor hall noise that is heard between loud chords, also the flute and drums piece is very revealing for the reverb length and varying timbre and positionning of the percussion. Also I noticed variations in the sampling artefacts for the 1932 lo-fi strings in the last piece (good thing you mentioned it).
I could have done some difference file comparisons to see what is different at the bit level, but I guess that would be defeating the purpose in a way ;-)
It will be interesting to see if the coming Topping D90SE can still improve those results!
Nice listening impressions Gilles,
DeleteYeah, I find the same general impression. It's usually between the "worst" and the "best" that I find I think I can hear difference. The Touch isn't as "precise" or "tight". The decay isn't as "smooth" or extended. Acute dynamics not as impactful or "fast".
I can still psychological enjoy the Touch of course and tap to the rhythm or sing along.
No doubt there will be differences going between DA/AD steps... Inevitable.
When do I use Bluetooth? For years I used my phone to control Foobar2000 on my PC, but that went away when I got an LG G7 and an Apt-X HD receiver. There's just no penalty for using BT with my mostly 16 bit FLAC collection (I use the Musicolet app).
ReplyDeleteI've recently replaced the LG with a Galaxy S21 but with LDAC the story is the same. If a "golden ear" came over I guess I would run the G7 wired since it's DAC measures as well as any, but that would only be for their benefit.
Hi Dillisc,
DeleteYeah, I agree. Personally I would have no problems enjoying music over aptX (HD) or LDAC especially with 16/44 FLAC collection. They sound great and even on my main sound system, wireless is fine (for example playing to the Topping DX3 Pro).
For actual mobile use... Fuggedaboutit. I don't even know which wireless headphones with their little amps inside would be capable of true 16-bit performance (much less worry about the limitations of the drivers).
It occurs to me that you don't even need to listen. You have the original. After going through the DAC and then the ADC, you have a file that can be compared with the original. Subtract the double-converted file from the original and see if there is a difference (you could even listen to the difference). You could then analyse the difference for frequency content, transient differences, noise, etc. This removes the most unreliable analyser (the human) from the experiment.
ReplyDeleteSure ianrt,
DeleteBut what's the fun in that ;-).
Go listen man...
As usual a pure objectivist point of view that almost eveyone that follows your coments tunes in for. I concur with most of your points of view. I loved your view point on being mobile with aptx bluetooth as i use that alot in the Boat where i spend hours out in nature listening to the sounds of nature inbetween listening to music downloaded to my iphone 12 pro and my iphone second gen SE over Tidal Hifi. I am a fan of MQA although i failed Mark Walrep's test terribly as i was not familier with most of music until i bought many of his BlueRays and became very familier with many of the tracts that are STellar recordings. please excuse the typos as i an typing on a win 1o laptop not my usual osx mac keyboards. I concur that Sonos outputs a much stonger output than Tidal hifi and at a buds very expensive B$W system with multiple DB1's anything sounds good on that Brittish eqiptment. I have been weaned on high dynamic range systems my 5.1 Klipschorn based system and when you put a crap signal into my system is sounds like crap. when I put on Mark's blue rays through any of my old school blue ray players. I is like the band appeared in front of me. I personnal on my system that the soundstage does improve with MQA of old analogue tracks. But that is what I listen to 70% of the time so my Neuro pathways favor that sound. My brain gravitates to Dynamic music of any type that is Dynamic. JUst dialing in my sub placement since i have three three 15" Woofers on my front end, and the Cornwall in the middle is mind blowing for vocals. I have stuck with my original Bluesound Node and have auditioned many Better Dac's but it comes down to very little difference that is not audible on my setups as you point out. Thanks for the observations.
ReplyDeleteNice Glen,
DeleteThose Klipshorns are something else! Sounds like you're not missing any of the bass either ;-). And multichannel to boot... Awesome.
No worries about MQA. MQA has enough bits and pieces that some might like the sound. In fact, remember that our blind test here back in 2017 really didn't show any special preference one way or another (consistent with the McGill study results in 2018). The most disturbing part for me is that it does impose a resolution limit which I would rather not have applied to my music... And it's kinda silly to be paying the company for nothing that we really can't achieve otherwise with lossless compression these days.
Enjoy!
Me too. I'm grateful. Just posted a link to this article on my own (just started) blog Aurality. Thank you!
ReplyDeleteHi!Record the sound of Topping d90se for comparison.Everyone is interested in comparing the sound of the "best" dac.It is better to hear once than to read a hundred times.It may be interesting to compare just with this dac.
ReplyDeleteI made a cue sheet for your musical performance track.
ReplyDeletehttps://pastebin.com/N0vpEZwM
Hello Archimago, nice to see your results in testing DACs and other audio equipment. I appreciate it. I would like to ask if you ever consider the recording of the DAC-amplifier chain or (DAC-preamp-amplifier) by high-performance ADC like RME ADI 2 PRO with AMPT. Why Am I asking? The out-of-band noise of different DACs can affect the performance of the connected amplifiers. (And as known, the DS ADCs are very good in filtering the out-of-band-noise).
ReplyDeletePersonally, I used a Digilent Analog Discovery spectrum analyzer to inspect the out-of-band noise of the DACs. It has 100 Ms 14 bit ADC so I was able to observe differences between different DACs in the range of 100 kHz to 1 MHz.
Hello Archimago.
ReplyDeleteSadly I can't agree with your appreciation of what a good recording is.
Eva Cassidy'song has a very unnatural equalized voice. It has horrendous and disproportioned sibilants and unnatural reverb effect. Much in the style of typically disposable "audiophile" recordimgs, which in my humble opinion are unlisteneable to my ears. Musically wise it is, errr, I didn'heard any music at all...
Eiji Oue's rendition of Stravinsky's Firebird is a really bad take despite your praisings for "microdynamics" and nuances. It absolutely lacks hall ambiance and its sound completely fails to render any perspective and amplitude od a symphonic orchestra like what I feel when I am at a real hall. The sounds of this recording are easily localizated at the speakers itself, being unable to create any phantom scene decorrelated from the speakers placememt in the listening room. Close spot micing it is not the way to go, unless recordimg engineer knows really how to mix them with spaced microhones which capture the natural hall's reverberation. As an aside I can't bear any Reference Recordings take. They all sound samelesly flawed and unrealistic to me.
Japanese roots is an old song friend of mine. Nice attacks and decays with silences all the way. I think, although not sure about it, but except the pan flute, the percussion seems sinthesized. In any case, it sounds ok although lacking of a true reverbernt air surrounding the percussion instruments. It has not a deep soundstage, so not realistic at all.
I've never listened to Benjamin Clementine before. Thanks for mentioning him. Fairly good recorded, beautiful music and voice style. No ambieance sadly. Loved his songs. A little bit bassy but good in any case. Nevertheless his voice is not completeley believable as a real voice in front of me. For this, check Jacob Collier song Time to rest you weary head. Or the song O Letchto Kurko by Tcha Limberger Trio. Both recordings materialize a true and real human being singing in my room, undistingishable from the real event. It is necessary to take special care to find the optimum sound pressure level for any of them to make the magic happen.
Best regards