MUSINGS: On the duplicity of hi-fi audio anti-measurers? The John DeVore example.

The Hall of Measurement Devices.

"When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meagre and unsatisfactory kind: it may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be." --- Lord Kelvin, Popular Lectures and Addresses vol. 1 (1889) ‘Electrical Units of Measurement’, delivered 3 May 1883

"What is measured, improves." --- Peter Drucker (from The Effective Executive) 

"When performance is measured, performance improves. When performance is measured and reported back, the rate of improvement accelerates." --- Karl Pearson

A longstanding debate over the years in the audiophile pursuit sadly is that of whether measurements are important in audio. Obviously, to me, and I think the vast majority of the world, the answer is "Of course it does!". One does not need to be a scientist of Lord Kelvin's stature or a statistician like Pearson, or a management consultant like Drucker to see the wisdom of such a basic idea which probably is one of the founding principles giving rise to our modern technologically sophisticated world.

Can you think of a single technological pursuit where the use of measurement instruments of some sort isn't beneficial to help gauge the effectiveness, efficiency, or value of a device which has been invented, produced, or manufactured? Is this not doubly important if that devices is supposedly built to a high "standard", reportedly made with quality and care? Typically at a higher cost like the things of "high end" audio?

Well, apparently, there are supposedly respected people in the audiophile world, designers even, who seem to think otherwise about the importance of measurements...

Consider this rant recently by John DeVore of DeVORE FIDELITY:

Assuming you got through the 36 minutes or so, I think you'll have noted some interesting historical stuff (which might not be fully accurate), interesting personal anecdotes (like the King Ding toy), somewhat relevant ideas about science and its evolution, and even interesting philosophical concepts about the mental constructs of "objects", "controlled hallucinations", "unconscious inference", and "the beholder's share".

However, beyond these vague, hand-waving, philosophical, more artistic thoughts, are there many useful truths we can hang on to from that monologue?

By the end, I was tempted to just ask - "so what?" - at least with regards to the anti-measurement rant mostly in the first part of the video. Let's delve into this rant for a moment.

To start, how are folks currently "weaponizing" (1:28) the use of measurements? Sure, in the '70s there were amplifier power and THD wars (as discussed before) but I think many of us still feel that those amps sounded great even if the power figures and 1kHz distortion levels were perhaps overly emphasized for advertising purposes. In fact, some contemporary dealers still sell these amps - like Just Audio showing recently at PAF'23. Just because some companies advertised based on a narrow set of measurements to catch the consumer's attention doesn't mean nobody measured products more broadly or understood the performance more holistically. I would argue that audiophiles back in the day knew a heck of a lot more about how the electronics worked than the average audiophile today no thanks to a dumbing down of the audiophile media for decades so I'm not sure too many were bamboozled by a single headline specification.

Let's go beyond "as if" history then, Mr. DeVore. Care to identify specifically which amplifiers/receivers you're talking about with amazing 1kHz THD and high power output but the company falsely "weaponized" these specs for audiophiles (and presumably the device didn't sound good)? Concrete examples would help drive home the point. The only really questionable stuff I am aware of are the false "Watts P.M.P.O." numbers that were targeted at the mass market like in the local K-Mart, and not really aimed at audiophiles or represented in our media.

Suppose as a hypothetical extreme we took away all the measurement tools in audio. Every speaker designer, DAC manufacturer, and amplifier company would now only use their ears to "tune" the sound. Would we end up with better or worse products than we have today? Would the consumers gain anything if this were to happen? Would audiophiles be any more enlightened by a measureless hobby without tools to verify and share what they've learned about objective accuracy?

No, I believe that would be awful. In fact, no matter how grandiose the rant, it's hard to imagine that guys like DeVore who designs products are serious about measurements not being useful, perhaps even vital! Does DeVore not perform measurements himself when designing his speakers or verifying that his crossovers perform as intended these days? I truly hope his products aren't all simply the result of "tuning by ear". (DeVore speaks about using measurement gear at 17:00 so he obviously must find them beneficial, also talked about measurements here.)

We clearly must rely to a significant degree on instruments for hi-fi testing. As we have seen in the last decade, we cannot just trust the few "golden ears" out there, especially those writing in audio magazines or in YouTube videos. Perhaps we would have handed over music distribution to MQA based on company propaganda already. I remember "golden ears" claiming how amazing SACD sounded because of DSD but over time we've learned that much of this library consists of PCM upsampled albums. Some indulged Neil Young's fantasies of how remarkably good 24/192 sounds even if it's just the same ol' dynamically compressed recordings. How many "golden ear" vinyl audiophiles clued in to the fact that MoFi used DSD in their vinyl production chain on so-called "One Step" pressings. Are we still to trust "golden ears" selling us 5-figure cables?

Whether in the production of the hardware, or as hobbyists trying to determine if we're actually being sold "high fidelity" items, measurements are a key component to keeping people honest. Objectivity plays an essential role as a kind of high-quality feedback loop that goes beyond the mere words that individuals like DeVore have to offer or the flowery prose of purely subjective audio reviewers.

While I have also warned against using "one number objectivism" like a single 1kHz THD+N/SINAD score, on the whole, I believe changes in the audiophile landscape with more hobbyists talking about measurements, and using objective means have been for the better. This has diversified the hobby by empowering those interested in seeking truth, to learn, and break through the myths propagated by those who would rather you accept their opinions on faith, often conveniently also associated with having something specific to sell. I can imagine that these individuals might not be happy with hobbyists swayed by objective-based evaluations.

As much as some complain that measurement devices like microphones do not function like human ears, well of course that's true! But isn't that also a good thing!? Measurement devices do not pass the input though the subjective filter of a human being and thus can typically go beyond what we can hear. In the frequency domain, let's see if anyone can draw a frequency response "by ear" with better accuracy than using REW. In the time domain, do you know anyone who can predict a speaker's step response accurate to sub-millisecond resolution? Show me an audiophile who can consistently detect, even quantify, the change introduced by 320kbps MP3 better than a machine. What is interesting about us as listeners is actually not that we have amazing hearing abilities, but it's the deficits we each have and the emotional emphasis we place on certain stimuli that actually define what sounds "good" to us individually.

As an aside, I certainly don't think DeVore's rant on YouTube is the worst; just one of many expressing typical sentiments, and audiophile myths. A worse example would be this recent Hans Beekhuyzen video claiming he can hear the effect of an ethernet filter "in seconds" presumably by changing a DAC's clock average jitter by less than a handful of picoseconds based on some work by Jaap Veenstra. Really? I will happily bet against his audibility claims as well. If it's truly that audibly obvious, I fail to see why he can't use a hi-res ADC, if not just a good mic to demonstrate that effect. After all, it looks like he has all kinds of measurement gear behind him, so I assume he must at some level trust that these are useful, speaking to the viewer so demonstrably bolstered by all that hardware as if this gives him credibility! He even claimed to have bought "about three generations" (4:45) of Audio Precision gear. Why bother wasting money if all he can produce are these kinds of chatty videos?!

At its core, people like DeVore and Beekhuyzen might portray themselves as draped with the veil of "science" by using the language of science only. Notice DeVore talking about stuff like Newtonian physics, Relativity, Quantum Mechanics as if these concepts confirm his rationale or somehow add to his understanding. Look below this façade and we see many claims are mainly anti-science, faith-based stuff.

They claim all kinds of things like "measurements don't correlate with what I hear", but when opportunities arise to really show if what they believe in is true, evidence offered remain purely within the uncontrolled, unblinded, subjective domain. Even when it comes to what should be factual claims, for example in that anti-measurement rant, DeVore believes that humans can hear above 20kHz (8:02) and apparently the developers of CD Audio at Sony and Philips "understood" this (of course, CD-DA can do up to 22.05kHz but that's just being picky). Well, who at Sony/Philips believed a clean 20kHz bandwidth was inadequate (see related discussions)? Where's the data for such a thing? Maybe he believes the questionable Oohashi stuff? Does DeVore himself think he could hear >20kHz? Beyond this, notice DeVore also touched on digital filters and the "hard edge" of brick-wall filtering at 11:00 into the video evoking certain myths about the "time domain". He then reminds us about jitter. All of these concerns are just the typical audiophile "boogeyman" anxieties also promoted by magazines with no in-depth discussions provided to help audiophiles understand or feel they can resolve the worries except for maybe spending more money.

He claims that measurements in the "wrong hands" would "mislead or oversimplify" (12:00). Sure, this could be true, but in 21st Century audiophilia, are consumers mainly misled by people publishing measurements? I would suggest DeVore look in the mirror at himself and the audiophile media and how they "weaponized" inaudible jitter, or the Gibbs pre-ringing seen in impulse responses with zero justification that this is audible from our DACs after all these years! And just look at how DeVore himself in this video glosses over everything by suggesting there are great mysteries out there in a rant that doesn't explore anything in depth, without pointing to any evidence, nor produce any links for further investigation, for most of what he believes to be true.

While there are multiple levels of dissonances in the rant, in the 21st Century, how can one speak of "high fidelity" audio (which can be scientifically defined), then go make hi-fi sound so complex as if impossible for the common audiophile to understand, and then question the important role measurements play which is basically the only thing that can help us resolve this supposed complexity?!

This is what I mean by duplicity - double-dealing, deceit - implicit in this kind of world view. Instead of questioning the importance of measurements in designing hi-fi products, the only reasonable thing for someone like DeVore to do is to actually assist by showing audiophiles what measurements he has found to be useful and what knowledge he has learned by doing so. Perhaps explaining how his designs overcome sonic shortcomings would be nice instead of ranting. And if indeed traditional measurements are inadequate, then go ahead and create new measurements others should consider using; spread knowledge. Of course, he is under no obligation to spill proprietary "trade secrets", if he has any.

The ultimate Hi-Fi experience... For him, I guess.

DeVore says this: "So science and its ability to measure have to continue to evolve in order to keep up with the increased body of experiential data" (13:10).

Sure, that could be true, but only if the "experiential data" is accurate. What if this listening "experiential data" is tainted and wrong based on psychological biases. For example, as far as I can tell, there's no experimental or experiential data confirming that bit-perfect digital streamers sound any different, yet this is what the magazine writers and audiophile salespeople have been telling us on auto-repeat for years while selling ever more expensive things (discussed last week). When one typically asks such individuals to put their hearing abilities to the test with a blind protocol (as is the way of science to control for variables so we can gather data), more likely than not, we hear the sound of crickets, or alternatively protests against why blind testing supposedly doesn't work. As we know, blind testing and ABX methods have been argued about for decades.

How curious and convenient that some believe the use of scientific methods apply to all other kinds of things but not to their own listening! Under such a constraint, should scientists spend time "evolving" measurement techniques and desperately search out new sets of data just because a group of people who never test anything under controlled settings say so? (Here's what happens when some pseudo-science audiophiles tried, resulting in ridiculous articles like this, and this about 10 years ago.)

I have never heard DeVore's speakers. Honestly, I am curious about how they sound. Notice that I have not said anything negative about his products. Just that I find his "rants on measurements in HiFi" rather hollow, meaningless, dressed up as false wisdom. There was a point at 18:00 in the rant where he gave an example of his own speaker being reviewed by the likes of Michael Fremer (here) subjectively agreeing with his views. So what? Fremer, like many subjective listeners, has made claims like this which made no sense, and are more than likely biased. To take a subjective listener at his word in an uncontrolled setting, as if infallible, and seeing if one could fit measurements to account for such mere opinions would be clearly non-scientific.

Yes, a scientist knows that there are often countless variables and not every one can be controlled, and many possible outcomes to any experiment (19:00), including surprising results. However, in 2023, when it comes to audio measurements, after decades of refinement by countless people, isn't it a little dramatic to call the measurement endeavor "dogmatic"? Measurement techniques do eventually need to be standardized for comparison purposes after all. (By using the word "dogma", his implication being that measurement-people are faith-based, I guess.)

I don't believe audiophiles who measure these days have closed their minds to understanding human experiences and areas of science DeVore and the audiophile industry typically avoid embracing - consumer psychology, power of the placebo effect, and psychoacoustics. In fact, much of DeVore's rant can be explored within these areas of inquiry, not the physical engineering science of frequencies, distortions, and power measurements. At one point he lamented on the fractured tribes of audiophiles with categories like "hedonistic horny hordes" and "fascist floydo toolies" and the "ordained order of first order... (whatever)" (21:15) - all of this yet again reminding us of the need for psycho/sociological exploration, not engineering or physics for many of his ideas. Is it any wonder that audiophile magazine writers, and DeVore might not want to go there?

Within the many words and recollections from 22:00 onwards, DeVore simply delves into the importance of psychoacoustics and his own psychology, again without just calling it for what it is. Rather, I guess he found that it sounded deeper and wiser to ruminate about the meaning of "consciousness", "theory of mind", how the sensory cortex "remaps", or philosophies of how the brain works - the "controlled hallucinations", and "beholder's share" of imagination.

Yes, fascinating, we can project all kinds of wishful ideas, but is this actionable to create better hi-fi gear bought with dollars, operating in the world of physics?

The Beholder's Share: But in the real world, other than bass boom, would a loudspeaker like this ever be able to approach the reproduction of what the man imagines? Arguably, with higher fidelity and better immersion, the demands on the "beholder's share" could be relaxed, right?

I find it interesting that DeVore's products are mentioned and reviewed quite a bit on Stereophile and by associates like Lavorgna or name-dropped by Guttenberg (apparently they were both salesmen at the same store at one time, interesting) yet I've never seen his speakers demo'ed at audio shows that I have attended over the years nor are they to be found at local show rooms. It doesn't take much searching to know that he's quite friendly with many of those reviewers in the magazine though. We can also clearly see that measurements of his speakers are not particularly impressive with often high cabinet resonances, wild off-axis frequency swings, and sometimes pretty grotesque on-axis frequency responses - see here, here, here, and here; thanks JA for the measurements at least. Perhaps that's why DeVore needed to put up the rant?

Clearly, the appearance of these products also might not be to the taste of those favoring a more modern look; they're more like the epitome of nostalgic "boxy" speakers in look and measured performance. I'm sure there are many who love the sound and appearance, the higher impedance and sensitivity, so to each his/her own.

Hope you're all enjoying December and some beautiful music, audiophiles! As usual, keep an open mind, but make sure the brain doesn't fall out.