INTERNET BLIND TEST: Do digital audio players sound different? (Playing 16/44.1 music.)

The other day, I was surfing the usual audiophile haunting grounds and came across this poll question on the Steve Hoffman Hardware Forum started a year ago. It reads: "Do You (Think) That Different CD Players Have Their Own Sonic Signature?"

Here are the results when I looked recently. I know it was closed last year, but it looks like it might be re-opened and still running into April if you want to cast a vote:

I thought it's worth spending some time to discuss this very simple question which of course is both understandable and asked frequently among newbies and longtime audiophiles alike. It's an important, and "core" question which can still be highly contentious after all these years! As suggested by the "blind test" title, by the end of this post, I will give you dear readers an opportunity to "cast your vote" and tell me what you think with a poll of our own using test samples to try (if you dare)!

 Let's make this our winter/spring research project...

First, notice the poll result above. Since when do we see audiophiles / music lovers vote 9:1 in the same direction? Almost never, right? So this is essentially a unanimous belief; survey says "Yes", CD players do sound different with their own sonic "signature" proclaimed by 263 of 296 total respondents.

Although the question is being asked about CD players, I think it's fair to suggest that this question applies to all digital players while reproducing 16/44.1 music which of course accounts for the majority of the digital music we're consuming today and for the foreseeable future.

The opinion expressed above is not unreasonable. We can easily come up with a few explanations as to why devices could perceptibly differ:

1. Different quality analogue outputs. Some will be noisier. Some will have different frequency response. Some (especially those with tube output stages) may also have higher measurable distortion (but distortion could be subjectively "euphonic"). 

2. Different DAC chip/circuits will also inherently vary as per (1). But from the measurements I've shown over the years, these differences tend to be small when looking at 16/44.1 test signals from good quality modern devices. The ability to resolve bit-depth will depend on whether the device is truly hi-res capable or not.

3. Depending on the digital input used and the circuitry, jitter will vary.  For example, back in the day I demonstrated the different jitter levels using SPDIF vs. USB interfaces. HDMI also will typically have higher jitter levels than asynchronous interfaces (check out the Oppo BDP-105 jitter with HDMI input for example though these days modern updates like the Oppo UDP-205 perform even better). I refer you again to the jitter demo I put up last year to consider for yourself whether jitter is a significant issue.

4. Different digital filters affect the analogue output. This is perhaps best demonstrated by results from filterless designs like the old "non-oversampling" (NOS) DACs and newer devices using weak filters. Examples over the years include the old Philips TDA154X NOS chips and the PonoPlayer with its Ayre-designed slow roll-off, minimum phase filter. Both of these will affect frequency response and a minimum phase setting will result in relative phase/time deviation across the spectrum which may or may not be of audible significance.

While we can say and know the above to be true, how many of us have had an opportunity to actually evaluate the impression that digital players have significant "sonic signatures" in any "controlled" fashion with our own ears? In other words, how many of us have actually evaluated the magnitude of these differences when volume is controlled, using the same amp and speakers, have the ability to quickly A/B switch so that we can compare within the limits of echoic memory? Just as important, how many of us have tried this evaluation in a blinded fashion, free from preconceived biases (price, brand, reputation, appearance)? I bet very few of us could say that we have taken the time to try listening with "controls" in place.

Here's your chance to put your ears to the test and listen in the comfort of your own homes! Time for one of our "Internet Blind Tests". Here is your mission, should you wish to accept it...

Ready for the test? Here's the procedure...

1. Download this big file. Make no mistake, this is a 470MB ZIP file. Within it contains 4 song samples captured using the professional grade RME ADI-2 Pro FS ADC at 24/96 from four different digital audio devices. Care was taken to ensure there was no clipping or other anomalies. I will not reveal the identity of these 4 devices yet, but they could be CD players, standalone DACs, mobile devices, network audio players, etc. What I will say is that when I reveal the identity of these devices, they will be recognizable as representative of their "class" of device and will not be esoteric audio players unfamiliar to music lovers and audiophiles. In fact, you might be a little surprised by the devices chosen.

The 4 songs all originated in 16/44.1 as verified "bit-perfect" CD rips. They were played as a playlist through the respective test devices, and then captured using the ADC in 24/96 quality at approximately the same volume. Afterwards I tweaked the files to make sure that the average loudness was almost identical:

I recorded the music in sequence using a playlist, totaling 6 minutes 32 seconds of audio for each device. With fine tuning of loudness in Adobe Audition (ITU-R BS.1770-2, amplitude processing with 32-bit resolution) afterwards, notice there is only +/-0.05dB difference between each recording's average RMS volume. At most, I had to boost the lowest amplitude device by +1.16dB, and lower the "loudest" device by -1.25dB. After this, I separated the recordings into individual songs. Notice the peak volumes have slightly larger range with 0.5dB between highest and lowest.

Obviously, I cannot ensure that you are hearing exactly what the devices sound like since we all have different playback set-ups and your DAC will have to convert the data back to analogue. What is more important I believe is that the direct recording of the analogue output derived from 16/44.1 playback into the same high resolution ADC at 24/96 is capable of capturing all the significant audible differences between the devices; or at least hopefully enough for your playback system to reproduce with high fidelity.

2. Listen to the samples. You will see that I have the music arranged in 4 directories corresponding to the device letters - Device A to D. To ensure that I did not consciously or subconsciously "rank" them by price or reputation, I had my son roll a dice to determine the order presented.

Within each Device's directory are the 4 sample tracks. The 4 musical samples I picked:

Maxi Priest - "Wild World"
- Well recorded pop song from the days when I was coming of age :-). Good DR13 recording without much dynamic compression, from the album The Best of Me (1991). 

Stephen Layton & Britten Sinfonia - "Chorus: For Unto Us A Child Is Born"
- Excellent classical / vocal recording from Handel Messiah. DR15 sample. Some strings, various vocal parts, delicately recorded. I know, we're past Christmas... Just work with me, alright? :-) 

Cécile McLorin Salvant - "Le Mal de Vivre"
- If you have not heard of Ms. Salvant, I think you're in for a treat. Let's not kid ourselves dear audiophiles, we love solo female vocals (eg. Diana Krall :-) and piano attacks and decays for sound evaluation. Here we have the combination of both in a nice DR14 recording from one of the best young jazz vocalists, IMO. This track can be found on For One to Love.

Joe Satriani - "Crowd Chant"
- For those who want to pump up the volume, here's a modern production weighing in at DR9. Rock with Joe as he hands out some distorted guitar licks while the crowd... er... chants through the "wall of sound". From Super Colossal.

Each track is a 1:30 to 2:00 excerpt. I don't expect you to have to listen to all 4 tracks with every one of the digital devices. Perhaps concentrate on the tracks that might appeal to you the most and that you feel would be the easiest to hear a difference with.

Can you hear the difference between the 4 devices? Is there a device that seems to sound best to you? Perhaps one device comparatively sounds poor? Grab a pen and paper and see if you can rate the devices from best sounding to worst and subjectively try to describe why.

Feel free to take the time to listen over a few sessions. Grab a friend / your kids / significant other. Maybe get a group of audiophiles together at the audio club meeting to listen and discuss.

As usual, I am using only a portion of the songs/tracks on the basis of fair use for the purpose of study / evaluation / research. I am not doing this for financial gain. Please delete these sample tracks once you are done with the evaluation. If you like the music, please purchase the albums as identified above.

3. Done listening? Submit your preference online! 
Before you click on the response link, remember that I'm not just interested in a simple "yes/no" answer since responses should be more nuanced. I'm interested in a little bit of demographics about you and why you respond the way you do... Here's what I want to collect:

- Your demographics - what part of the world, male/female, what decade of life? 

- Your sound system - briefly describe the system and approximate US$ MSRP cost.

- Did you use speakers or headphones (or both)? 

- Try to rank as best you can the 4 devices (Device A to D) in order from best to worst. 

- Think about how much difference you heard between BEST and WORST devices. 

- Think about how much difference you heard between BEST and SECOND BEST devices. (To get a sense of the difference between adjacent-ranked devices.)

- Did any specific track stand out as sounding different between devices?

- Let me know if you write / publish audio equipment reviews. This can be a blog, YouTube videos, all the way to having published magazine articles. Also, if you're a trained musician or have formal experience/training in sound evaluation, I want to know!

- Feel free to comment on subjective differences heard. Sense of "presence", "air", "frequency extension", "sound stage", "dynamics", "pacing"... All fair game for subjective descriptions.

If you spent some time listening to these tracks, then make sure to let your perceptions be counted. Whether you heard a big difference or very little difference or even no difference, I have provided the options for you to choose. Remember, all data is gathered anonymously for the results database although the poll site will prevent multiple entries from the same computer.

Ready? Submit your responses here.

By all means send this test out to all the audiophiles, head-fi folks, and "golden ears" you know who have the ability to play hi-res 24/96 music with good gear. Spread the link to this test around the audiophile forums. The more responses the better to get a good sense of the tested "real world" answer to the question being posed.

As usual, remember this is a listening test with the aim of gauging subjective preference. Please try not to load these tracks up into an audio editor for a peek until at least after submitting your responses. To be honest, I don't think looking at the files in an audio editor will help. Also, please let's not "share" answers which might bias others.

Unless something unforeseen comes up, let's run this test for 3 months - I'll close off the survey on April 30, 2019. Plenty of time to listen, consider, and submit results. As usual, afterwards I will compile the data and present them here. Let's see what people thought and if there appears to be any measurable subjective preference trends.

Thank you in advance for trying this out and getting me your survey results. Have fun and I hope you're enjoying the music!

February 1, 2019 - Fixed the slight pop in one of the tracks...

May 1, 2019 - Survey closed! Thanks for everyone who submitted their result... Will talk about results on the blog shortly. See results starting here.