MUSINGS: Digital Filters Test discussion, and a 192kHz down-sampling setting suggestion...

Well, I guess I am flattered that the results of my recent Digital Filters Test got attention on AudioStream (Stereophile affiliate). The other day, Michael Lavorgna posted an entry entitled "The Trouble With Audio Tests" including a few quotes originating from my INVITATION to the test as well as my ANALYSIS posts.

I'm going to start today's entry addressing some of his thoughts on the matter and how I view tests like this.

First, let's just start with the quote from Ernest Rutherford (1871-1937) brought up by Mr. Lavorgna: "If your experiment needs statistics, you ought to have done a better experiment."


Well, it is a romantic notion isn't it that it could be so simple... That we don't need statistics to understand complex phenomena because somehow results end up being "black or white" or the conclusion just jumps out and declares itself. Perhaps in high school classrooms. Now I don't know enough about nuclear physics (to which Rutherford is known for) to provide any direct links, but I'd be surprised if some form of statistical analysis were not used in his famous Geiger-Marsden apparatus experiments to demonstrate alpha particle scattering, making sense of the empirical data, probabilities of deflection angles, and the margin of acceptable experimental error. Or whether modern particle physicists at CERN bother with statistics. But I do know that statistical analysis is essential in the biological and social sciences to make inferences out of observations of natural and social phenomena. It is acknowledgement that the natural world is "noisy" and that there is a "normal curve" to continuous variables out there in the "real" world. In research, we're trying to find answers based on whether a signal can be isolated from the noise. And this of course would include explorations into audibility and evaluation for the presence of "golden ears".

According to the post, Mr. Lavorgna was given the audio test samples in May as he states. He then did "one run-through" and "easily" picked out his preference for 2/3 samples. It now turns out he preferred 2 of 3 minimum phase samples and 1 of 3 for the linear filter.

I must say I am however disappointed! Why didn't you enter the results in my survey if you listened to these samples in May, Mr. Lavorgna? Remember, the test ended in late June (maybe you did but declined to indicate you were an audio reviewer?). Blind tests and claims of audibility are only good if one takes on the test and commits before the results are shown! You could have even added your subjective description as to what you heard so "easily", indicating strong confidence, and this 2/3 preference in the minimum phase filter could have pushed my results further towards an overall minimum phase preference for some of these samples. Alas, the opportunity is missed and what we end up with is a retrospective report - post hoc testimony.

He even goes so far as claim that he didn't even need to listen to the linear phase version before picking out the minimum phase one in the "second sample". ("So much so that I picked out the minimum phase filter in the second sample without having to hear the linear phase filter version.") If by "second sample" he meant the "GrandPiano" piece, recall that most respondents actually preferred the linear phase filter setting even to the point of a statistically significant p < 0.05 with headphone listeners! Impressive I guess that he was able to hear the difference so strongly. But also consider what this means if an audio reviewer's preference seems to be different from what most listeners seem to prefer...

I do agree with him that experimental results of group data may not apply to any one individual. It's not supposed to. But it does give us clues of trends and tendencies, what is important and what isn't. Sure, some unique individuals in a population will have remarkable abilities exceeding standard deviations of the "norm". These individuals have a "gift". Does Lavorgna have such a gift of having "golden ears"? I don't know nor does it necessarily matter. He only has to be true to himself just as I have to be true to myself and try my best to avoid my own preconceptions as best I can. This is just basic insight into human limitations and imperfections of the ear/mind mechanism. [Note that in this case, since the samples are all 24/176.4 and level matched, an honest ABX test with the foobar plugin will show just how "easily" the samples can be differentiated.]

What I can say is that out of 45 people who spent their time evaluating the test samples blinded (whom I wholeheartedly appreciate), I was not able to find a cohort who consistently selected preference of one filter type over the other (signal) beyond what was expected by chance (noise). The only significant result I could find was a preference for linear phase filtering with headphone users for one of the test samples, and that perhaps there is value in using the minimum phase setting with speaker listeners for 2 of the 3 samples based on subjective preference. This is all in the context of filter settings that exaggerate the amount of ringing due to an extremely steep "brick wall"! [That is, the vast majority of digital filters in use do not have close to this amount of ringing and I would expect audibility to be even less than this for most DAC filters.]

I certainly do not think the "answer" is at all "wishy washy" as Lavorgna claims. This is I believe a demonstration of reality, and a limited glimpse of the "forest" rather than individual trees to perhaps generate other hypotheses and further exploration. This is what you find when subjects are put to the test, when audibility between samples exist at the threshold of acuity. Not the remarkable "slam-dunk" subjective descriptions of "obvious", "clear" or "easily heard" comments made by almost every reviewer doing sighted evaluation as if "golden ears" were ubiquitous in the world.

One final comment before I end this segment. I do appreciate Mr. Lavorgna for admitting that he picked the linear filter in 1/3 samples. It's a demonstration that even if one believes there is an obvious difference, there could still be preference for the other filter setting in certain situations. Hence based on this empirical test, I would support DACs having switchable filters with both linear and minimum phase settings for audiophiles to choose their preference and do their own experimentation. Note that I'm not saying this should be a necessity since I personally feel the differences are at best subtle for the majority of listeners using typical filters with less ringing than the ones tested. And I think it would be presumptuous for any manufacturer to claim there is such a thing as an ideal filter setting to listen with. "Wishy washy" or realistic respect and appreciation for the subjectivity of human experience?

Mr. Lavorgna gave me a quote to start off discussions. Let me close with another I had mentioned previously courtesy of Karl Popper (1902-1994):

And who shows greater reverence for mystery, the scientist who devotes himself to discovering it step by step, always ready to submit to facts, and always aware that even his boldest achievements will never be more than a stepping-stone for those who come after him, or the mystic who is free to maintain anything because he need not fear any test?... All mystics, as F. Kafka, the mystic poet wrote in despair, "set out to say... that the incomprehensible is incomprehensible, and that we knew before."

(from The Open Society and Its Enemies, 1945, chapter 24 - Oracular Philosophy and the Revolt Against Reason)

---------------

I do not want to end this post with just philosophical debate. Rather, let's be practical and discuss a digital filter setting that I have been using over the last year for all my audio down-conversions.

I suspect many of you like myself may be a bit disappointed by all the hype around 192kHz samplerate material. Sadly, the majority of the 192kHz albums I have bought clearly have no "desirable" content at all in the ultrasonic frequency range (ie. most of the time the last octave is just low-level noise if anything). And they clearly don't sound any better than a lower samplerate version. Since I don't like to waste storage space, I routinely will downsample these to 96kHz if it looks like there is significant natural-looking material above 25kHz, down to 48kHz if it looks like it may have originated as that, and 44kHz if it looks literally like something upsampled from CD resolution. Even though disk space may be cheap, I'm still against wastage.

For a moment, let's not argue if we need anything more than 44/48kHz :-).

Suppose then, I want to down-convert 192kHz music to 44kHz, what filter setting would be good?

I've been using the excellent iZotope RX 4 software package for the last while. It's an excellent toolbox for experimentation and tweaking. I have been using the following setting for the lowpass resampling filter:

Pre-ringing at 1.0 means this is a linear phase setting - no phase shift, which I believe is preferable for the audio file itself (especially since it could be played back with a minimum phase filter DAC that would further distort phase). Filter steepness of 30 is moderate and while there is a little bit of aliasing, it's down at -28dB or so and above 20kHz. Finally the "cutoff shift" is down to 96%, lowering the passband slightly but reducing the amount of aliasing.

I find this setting works really well for me intellectually and it sounds just great :-). The reduced steepness correlates with less severe ringing / energy smearing at the Nyquist frequency. Pushing the cutoff shift down reduces frequency response only above 20kHz and inaudible for adults (and likely for everyone).

Impulse response for such a setting (using a 192kHz impulse downsampled to 44kHz):

Impulse Response

Just a tiny bit of low-amplitude symmetrical ringing (only about 4 cycles of any significance) with attenuated impulse at 22.05kHz.

And the spectral frequency display:

Spectral frequency display - Audition 3, Blackman-Harris windowing, 128 bands, 100% window width, 132dB range.

Nice linear response without group delays. No significant high-frequency smearing.

A similar setting if you're using the free SoX software would look like this:

sox.exe "Input_File" "Output_File" rate -v -a -b 91 44100

The -v flag is for "very high" quality accuracy, -a allows a small amount of aliasing, and -b 91 sets the passband to 91% or about 20.1kHz. For 48kHz, this would be pushed up to over 21.8kHz. Anyhow, a suggestion that works well for me, YMMV :-).

Frequency response of the SoX parameters downsampling 24/192 white noise to 44kHz - frequency response only rolls off over 20kHz.

---------------

The other day, Arny K. on the Squeezebox forum suggested having a look at the documentary An Honest Liar (2014) about James Randi. Even without the connection with audiophile cables, it's a fascinating look into an interesting fellow and his battles with charlatans and mystics over the years. Regarding the cable challenge that never happened almost 8 years ago, here's Mike Fremer's recollection of the story, and here's Randi's (long) side of the story [Oct 12, 2007], Oct 20, 2007 followup, and Oct 23, 2007. Not sure if much was said after that...

I guess we'll see what happened to the Ars Technica / Randi ethernet cable test soon... (AudioQuest Vodka teardown now posted.)

Time for summer holidays and a few weeks off with the family. Until next time... Enjoy the music!