MUSINGS / MEASUREMENTS: On why 24/96 sampling of the Blind Test of 16/44.1 devices is good enough. [On idealized filtering tests and "real life" music playback.]

As promised, I want to address a comment raised about the idea that the 24/96 recording using the RME ADI-2 Pro FS ADC was somehow inadequate for the 16/44.1 samples in the "Do digital audio players sound different?" blind test; that the 24/96 captures are somehow missing significant amounts of the output from the devices.

For reference, I've included the original comment here from Miska:

"It [the samples] just barely catches the first image band between 22.05 - 44.1 kHz and a little bit of next between 44.1 kHz and 48 kHz. One of the devices is clean though. The ADC anti-alias filter fixes rest by improving the reconstruction by removing further image frequencies. Strongest image for most current DACs is around 352.8 kHz.

Of course playback system then defines how much overlay of these come back again at different frequencies from the DAC, since it is now running at different rate family than the original source.

This doesn't really replicate real device playback performance, but at least it captures some apparent differences. So it gives kind of hint or shadow of how the device actually did."

Although the blind test is ongoing, I figure it's still good to talk about this while not revealing the devices used of course.

As a start, yes, of course poor filters can result in terrible looking spectra with all kinds of ultrasonic imaging beyond the first and second image band. Most infamously, we have seen these issues when we examined MQA in the past.

Remember that when we run objective measurements, we're often using extreme test signals to see the limits of the machine and whether some property of the device functions close to an ideal state. For example, when I run the "digital filter composite", I'm including the 19 and 20kHz -6dBFS tones to check for intermodulation distortion. There's a wideband noise with 0dBFS peaks allowing us to see if the filter's calculations overload. DAC/amplifier/speaker testing might also include the use of square waves running through them.

But these are not "normal" music signals by any means! For a device to pass with flying colors through these "torture tests", we are basically showing that the machine can achieve a certain level of idealistic performance which more than likely will not be tested with typical music. Or if it is tested, only rarely with certain types of recordings. When I evaluate devices, I certainly look for ideal performance since it tells us something about whether a device is capable of very high fidelity (which for me is what the "audiophile" pursuit is about).

Which brings us to the "real world" blind listening test. As you know, for this test, what I have done is used real music that were of reasonably good quality (notice I didn't pick nasty DR4 mastered pop music even though there are plenty around!). While the Joe Satriani track "Crowd Chant" did contain some clipped peaks in the digital signal itself, the other 3 tracks all were properly bandwidth limited. As such, the demands on the digital filter's performance are not extreme.

Let me show you what I mean using what I would consider are the 2 devices in the blind test with the poorest performing filters - let's call them Device X and Device Y so we don't make any associations with Device A/B/C/D in the actual blind test.

Let's start with simply playing a 16/44.1 1.0000kHz 0dBFS test tone (with TPDF dither) through these devices and I'll capture back the analogue output using the RME ADI-2 Pro FS at 24/384 for 192kHz of bandwidth:

That doesn't look so good at all. Device X is clipping the strong 0dBFS signal. As for Device Y, we can see imaging all the way out to almost 100kHz which a 24/96 capture indeed will not record.

Okay, let's get even more challenging... How about the -6dBFS 19 & 20kHz intermodulation test played back through these devices?

Ugly indeed especially for Device X where we have noise content going all the way to ~190kHz!

But remember, these are synthetic test signals... Do we listen to 1kHz 0dBFS tones? What music actually contains high amounts of 19 and 20kHz peaks? (Even if present, how many of us can honestly claim that we can hear such high frequencies?)

No friends, we listen to real music. And here is what real music "looks like" when reproduced with Device X and Device Y despite how awful the synthetic test tones appear in the graphs above:

Notice I added the noise floor graph in the "Crowd Chant" capture. The music signal was captured during one of the louder passages where there was some clipping showing excess signal above Nyquist. This is the only track of the 4 samples that does this.

What I've done is monitor the playback of the actual songs in realtime using WaveSpectra (ASIO ADC input at 24/384) and captured places during the playback where the ultrasonic defects looked the worst.

Quite a different "look" from the 0dBFS 1kHz and 19/20kHz synthetic signals above, right?

Knowing this, consider that when I created the test files, I captured the music at 96kHz using a sharp linear phase filter on the ADC. This means I've preserved almost all of the content up to 48kHz. For Device X, this means I have captured essentially all of that ultrasonic distortion clustered around 40-45kHz. For Device Y, the 96kHz capture would have missed out on the last bit of distortion closer to 50+kHz, but certainly kept a good amount of what's there.

Suppose you downloaded the Blind Test, and are about to listen to the sample tracks in the comfort of home through your laptop with an inexpensive hi-res USB DAC like the SMSL iDEA (previously measured) through your headphones.

Here's a comparison of what that SMSL iDEA DAC's analogue playback of the 24/96 test sample looks like compared to a direct feed from Device X/Y for the same 5-second segments (again, captured back at 24/384 through my RME ADI-2 Pro FS ADC):

Notice that I matched up the amplitude of the audible spectrum quite closely. As expected, since Device Y had a bit more ultrasonic content stretching out beyond 50kHz, the limits of a 96kHz samplerate with Nyquist at 48kHz would miss out on some of that higher ultrasonic stuff. As you can see, apart from that bit of "missing" ultrasonic data, a listener using something inexpensive like the little iDEA can still replicate the 16/44.1 playback quite well including most of the subtle, low level filter artifacts. Obviously each DAC will be a little different but I trust that most hi-res players can perform in a similar fashion to the <US$100 iDEA. I certainly do not believe that a substantial reason why players could/should/might sound different is because of some ultrasonic content out around 50kHz!

While I did not show measurements with the other 2 devices in the blind test, suffice to say that I have looked into this and I have even less concerns about "missing something" in the test samples with those other two devices.

In my opinion:

1. Sometimes I agree with the "subjectivists" :-). We don't listen to test tones and not all measurements will correlate with the sound of real music playback. While we can see marked imperfections when playing back test tones with Device X and Y, actual music playback shows much less ultrasonic artifact. Notice that both devices have relatively steep filtering that attenuate frequencies quite nicely by 25kHz, and the imaging artifacts out at 40-50kHz are at worst down at around -100dBFS.

2. There's a comment about the fact that since I'm recording at 96kHz, somehow using this different family of sample rates (48kHz base rate) is an issue. ("Of course playback system then defines how much overlay of these come back again at different frequencies from the DAC, since it is now running at different rate family than the original source".)

Remember that while the original data is 44.1kHz, this is hi-res 96kHz conversion of a continuous analogue signal... Not the same as non-integer conversion of 44.1kHz samples to 96kHz digital where perhaps some might take issue. In fact, it is because I used 96kHz that I am able to capture more of the artifacts than something like 88.2kHz. Sure, I could have captured the signal at 24/176.4 or 24/192, but as you can see, if I did that, most of the data would have been wasted bits and would have made the test files even larger than the already substantial 470MB package! I just didn't feel this was worth it for practical reasons.

If the only difference between something like a cheap $50 CD player and a $1000 device (not necessarily saying these are the devices I used, but similar :-) is because of some fragile low-level ultrasonic noise >48kHz, should we not automatically doubt the value of expensive devices?

3. There's an implication in the comment that speaks of the belief that filters are somehow causing a major difference between devices. Even to the point of suggesting the "Strongest image for most current DACs is around 352.8 kHz". Is ultrasonic content at 352.8kHz that important as to audibly differentiate expensive vs. cheap playback devices?! While I certainly hope not to see high amounts of noise at 352.8kHz or in the MHz range, I have seen no evidence that such high frequencies (at most around -50dBFS in measurements) coming out of typical DACs ultimately affect the sound from one's speakers/headphones.

I don't deny that anti-imaging filters are important. I don't like the sound of NOS with high levels of ultrasonic distortion, and I do not prefer weak Ayre/PonoPlayer filters that roll-off the highs. Furthermore, I see no point in promoting a system like MQA where the digital filters used are suboptimal. However, most of our playback devices are "somewhere in the middle" which could very well be "good enough" with real music.

Remember that CD and other digital audio players have deficits within the audible frequency that can be much more important than "good enough" filtering. For example, the noise floor will be clearly different between devices (hum especially tend to be high with cheap gear). Frequency responses vary. Crosstalk between the channels can be quite high at times. Not all devices are created equal when we compare harmonic and intermodulation distortions. Phase distortions may be present. Low level linearity errors diminish DAC resolution with cheap devices. Etc...

I would consider any of these factors above within the audible spectrum significantly more important than a few little ultrasonic imaging distortions with these Blind Test samples! A high quality, high-resolution capture of the 16/44.1 signal at 24/96 would most certainly retain these differences between devices - I can say this without hesitation because I have already measured the devices and can consistently demonstrate relative weaknesses.

In conclusion... What I have captured in these 24/96 test signals I believe is way more than just a "kind of hint or shadow of how the device actually did". While I respect Miska who wrote the comment (and as the author of HQPlayer discussed last week), I must disagree with this assessment.

Since the beginning, this blog is about being "more objective". While objectivity is obviously important especially since these days the mainstream audiophile press is strongly biased towards subjective reviews and clearly at times fanciful nonsensical tales told to audiophiles, there is inherently a balance we must seek. IMO, this balance is to be found when we correlate the objective facts with subjective experiences and adding some common sense into the equation.

Often common sense, knowledge of how things work, and a little systematic testing tells us that certain beliefs are clearly false (eg. the value of thousand-dollar power cables, or the notion of significant differences between otherwise error-free digital cables). In the other extreme, we can say that relying too much on objective findings - like making claims about sound quality based on pre-ringing of the impulse response - is folly as well. Either path alone leads us down a state of neurotic preoccupation as one can "miss the forest for the trees".

This is why I like running these blind tests once awhile. To give you, readers, an opportunity to add your voice to the discussion based on experience while attempting to reduce bias. I believe this is more powerful than the voice of certain "respected" opinions especially in the audiophile press over the years.

So, did different digital devices playing the same 16/44.1 music sound different? And by how much? Remember, you have until the end of April, so take your time and please share the test among audiophiles, music lovers, and those who just like testing things out (!) with time to compare the 24/96 samples and answer some straightforward questions.

-----------------------------------

I watched the Queen biopic Bohemian Rhapsody last night with my wife. Excellent flick and of course up for consideration this weekend at the Oscars; amazing acting by Rami Malek. The 4K UHD Blu-ray looks excellent with good HDR punch, natural colors and deep blacks. Alas, it looks like despite being filmed digitally in 3.4+K, the digital intermediate was still 2K - very common these days (remember I posted some discussions on the benefits of >2K movies awhile back). Nonetheless it looks sharp and clean. Impressive Dolby TrueHD Atmos soundtrack as well!

Hope you're all enjoying the music (and movies!) :-).