Over the years, we've seen websites, blogs, and videos try ways to demonstrate the sound of a hi-fi device or for the adventurous, even try to convey the sound quality of listening rooms. For example, there are binaural recordings at audio shows, soundroom demos, speaker comparison samples, etc. This is not an easy task because high-fidelity is about nuances and slight variations; not wholesale "obvious" differences for many devices like DACs or high performance amplifiers. Unlike what you might read in audio reviews, assuming you have a decent DAC already, a replacement would be unlikely to result in obvious changes in characteristics like bass response or claims that jitter effects are somehow obviously audible! Sorry folks, a lot of that kind of talk is just fantasy.
While it is convenient to view and listen to typical YouTube clips, I think we can all appreciate that sound quality would be highly affected by: the recording microphone, room acoustics, set-up quality, the lossy audio compression from YouTube among others that I may have missed. Dissociating the effect of the different components would be impossible. And obviously any time you use a transducer to convert the sound pressure into electrical signal (ie. speaker, microphone), there will be a significant reduction in resolution if we're trying to determine the effect of something like a DAC!
Then there's the issue of what music is being used? Is it music that audiophiles have general access to? Is it material that audiophiles/music lovers would even generally listen to? Obviously this bit is very subjective but I think there's something to be said about esoteric test material that might be recorded amazingly well, but just not adequately popular to have "mainstream" level acceptance. When "subjectivist" audiophiles complain that test tones are artificial and synthetic, is it that much different from listening to a handful of albums that barely anyone cares about? ;-)
While thinking about this recently, and having measured enough devices over the years and showed you numerous graphs and synthetic test material, why not expand this by giving you the opportunity to listen for yourself? Using the highest quality "recording" of devices that I can...
With that in mind let's lay down what I'm going to try to start doing with some of my future DAC/source measurements and reviews; I'll include a link to a hi-res recorded playback from the device so that you can listen for yourself and compare at home.
A few basic points about how I'm going to do this and then some actual recordings of devices:
1. I'll record in 24/96 direct from my RME ADI-2 Pro FS ADC which I know is capable of excellent resolution - at least capable of measuring down to -110dB THD+N with single-ended RCA input, and -113dB THD+N with balanced XLR. This is beyond the resolution from any speakers or headphones.
24/96 would be beyond human perceptual ability as well so let's not waste storage space with 24/192 or DSD.
The ADC will be set to linear phase, steep filtering. Given the 96kHz samplerate, we're looking at filter roll-off out beyond 40kHz, obviously an octave beyond even ideal (teenager, early-20's) hearing acuity.
2. Let's use a selection of test music, nothing terribly esoteric, at least good production quality, for subjective evaluation. Here are the tracks I selected based on some of the stuff I listen to:
A. Eiji Oue & Minnesota Orchestra – “Infernal Dance of the King Kashchei” from Stravinsky: The Song of the Nightingale / The Firebird Suite / The Rite of Spring (1996, DR14) – 24/88.2 à 16/44.1 using iZotope RX 8. Classical track, strong dynamics, listen for nuances, “microdynamics” in the orchestra. Fantastic production from Reference Recordings.
B. TakéDaké & John Kaizan Neptune – “Japanese Roots” from Asian Roots (1998, DR14) – ethnic wind and percussion instruments. Listen for dynamics, transients, smoothness as notes trail off and the fade to silence between passages. How "palpable" do the instruments sound through your headphones or in your room? Probably the most esoteric track here but it sounds pretty cool.
C. The Ray Brown Trio – “Summer Wind” from Live at the Loa: Summer Wind (1988, DR12) – I used the 2003 Concord SACD release DSD64 à 16/44.1 using Weiss Saracon - live, jazz, slightly longer than 2 minutes. Listen to the pacing, timing, flow. “Presence” and “atmosphere” of the audience, instrument spread over the soundstage, ability for DAC to separate bass, piano, percussion, and even the occasional background vocalizations.
D. Eva Cassidy – “Fields Of Gold” from Songbird (1998, DR11) – female vocal – listen for any excess sibilance, vocal presence, neutrality, and quality of the vocal reverb. Nice accentuated "attack" on the acoustic guitar on this cover of the Sting song.
E. Benjamin Clementine – “Winston Churchill’s Boy” from At Least For Now (2015, DR8) – male vocal – “body” of the voice, clarity of articulation, separation of voice from instruments, soundstage placement, tonality of the instrumentation, bass quality.
F. Dua Lipa – “Love Again” from Future Nostalgia (2021, DR5) – modern pop production – ability of the DAC to handle typical multilayered, “loud” / dynamic compressed modern Top-40 type music (+0.9dB “true peak” on this recording). [Notice the sampling of "My Woman" by Lew Stone & The Monseigneur Band from 1932, also used in White Town's "Your Woman", 1997.]
Age of the recordings range from 1988 to 2021. We have classical, world/ethnic, live jazz, female vocal, male vocal, and modern pop represented. As described, each will bring certain characteristics to listen for like the strong dynamics in the Stravinsky piece (this is not the HDCD version but sourced from 24/88.2 Reference Recordings hi-res), the clarity of "Japanese Roots", live ambiance of The Ray Brown Trio, guitar and voice on "Fields of Gold", bass authority on the Benjamin Clementine track, and louder compressed dynamics of "Love Again".
Some might wonder why I didn't include a techno or EDM track. Well, already the Dua Lipa track employs quite a bit of studio processing - artificial sweeteners for the ears, and maybe eyes as well but that's subjective ;-). In general, I think it's more useful to assess whether a DAC or CD player sounds "natural" as in the tonality of voices and acoustic instruments that can engage our judgements based on previous real-life experiences of these types of sounds. Playback quality of electronic music probably can already be assessed quite well with the results of measurements (synthetic test tones for synthetic music!).
3. ~90 seconds for each of the test samples above. This should be long enough to allow you to pick out interesting portions to focus on while at the same time provide a "gestalt" of the music. I increased the time to about 2 minutes for The Ray Brown Trio to appreciate the pace and flow of the live recording.
In total then, the composite test track lasts a bit over 9 minutes of audio. Here's a look at the waveform in Adobe Audition:
4. Let's stick with lossless 16/44.1 source. No need to argue or debate over hi-res these days I think (see here, and here). Even if you can hear a difference with hi-res, all indications point to the difference being minuscule. The reality is that for the foreseeable future, stuff that we consume will be CD-resolution 16/44.1 or equivalent (like upsampled and pseudo-hi-res). I believe there are 24-bit versions of Dua Lipa and Benjamin Clementine out there, but seriously folks, these kinds of low-DR albums (as discussed here years ago) do not warrant spending money on and would be an absolute waste of storage as well.
For future reference, I'll just call this test track and the way the DAC recordings performed the "AMPT". When doing the AMPT recordings, I'll make sure to include a few seconds of silence at the start to allow you to examine/hear the "sound of silence" from the device; very important to listen for hum for example. Also, in order to allow for easier "apples-to-apples" comparisons, the recorded track volume with be normalized to -27.2dB +/- 0.1dB average RMS amplitude using Adobe Audition 14 (2021); this is at -3dB of the source amplitude. This will allow for overhead such as the Dua Lipa track with +0.9dB true peaks. IMO, the high-resolution, very low jitter, studio-quality RME ADC (for the record 18kΩ XLR, 9kΩ RCA input impedance) will easily capture all that we need from these 16/44.1 digital-to-analogue conversions.
Here's the original track for those who want a download of the source 16/44.1 file:
** VERY IMPORTANT: If you want to compare direct playback of the original track above on your system to some of the recordings of DACs I'll be posting below, make sure to increase the volume of the recording of the DACs by +3dB. I've noticed over the years of running blind tests that some people don't bother controlling volume and then make all kinds of claims of what they heard. Also, make sure to turn off any extra processing when listening (such as volume normalization, EQ and such which may affect quality).
As in the past with blind tests, the tracks are being used based on the principle of "fair use" for the purpose of education and testing. Only short portions of the music tracks are used (significantly <50% of the whole song). If you enjoy the music, purchase the full album and support the artists. I've included Amazon Affiliate links, the expectation is not one of making a sale but for convenience if you're interested in purchasing the music.
So, let me get you guys and gals started on "hearing" the output from a few DACs.
|Here's a shot of the RME ADI-2 Pro FS R Black Edition being recorded. Using some inexpensive but good "generic" SRADIO 6' XLR cables I had around here.|
|FFT of the RME ADI-2 Pro FS R BE recording - note the filter roll-off character and clean ultrasonic noise floor.|
|D'oh! How did that Sleeman beer get in there? 3' generic shielded RCA cable used.|
|Still big, bold and beautiful after all these years (bought in 2013)! "Sharp" filter used here as you can see.|
|Notice the 60Hz mains noise and 180Hz 3rd harmonic using single-ended RCA. Quite a bit of higher order harmonics by today's standards. Noise level significantly higher than the TEAC above.|
|Not the best DAC measurement results these days but still not bad and certainly better than majority of amplifiers.|
|A decade back, "asynchronous" network audio devices like the SB Touch communicating with the server, even with WiFi, utilizing a good sized data buffer had no significant jitter issues. Notice that all the anomalies are below -120dBFS.|
|A quick FFT similar to the RME ADI-2 Pro FS R Black Edition (not exactly same spot in the music). As you can see, a bit of extra ultrasonic content on this analogue out; clearly not as clean filtering as RME.|
Happy listening! Feel free to do quick A/B switching such as using foobar ABX Comparator. Software like DeltaWave could also be helpful to analyze for differences (like relative temporal drift between devices). If you look at the recording FFTs, you'll also get a sense of the variations in digital filter quality; particularly the roll-off around Nyquist (22.05kHz) and the presence of ultrasonic imaging artifacts from the DACs up to 48kHz. About a year back, we discussed the audibility (or non-audibility) of parameters like THD+N so the recordings here may or may not confirm such things if you look at the DAC measurements and listen to these samples. Ultimately, hopefully this continues to add to our understanding about the magnitude of differences between hardware like DACs these days without resorting to "magic" or impressionistic words (typically, hyped up descriptions from reviewers or in ads aiming to just sell hardware).