Notice that the quest for high fidelity has always been, almost exclusively, a male pursuit. A picture from the Edison Diamond Disc and the "tone tests" done back in the day circa ~1916. |
I've been critical of many audiophile magazines, websites, and YouTube channels because it's hard to believe some of these individuals truly can hear what they report. I know, it's perhaps "rude" to question subjective claims, but unless one questions stuff like this, how are we to differentiate thoughtful critical listening from potentially biased claims of sound quality?
For example, I honestly doubt the folks who claim to hear big differences between cables. I think it's silly that some people insist they hear differences because of unsubstantiated "jitter" effects in digital audio. Those who think "bit-perfect" playback can sound very different have never over the years proved their case. Then there are those subjective reports that sing the praises of "hi-res audio" despite research never showing differences as being anything more than subtle. How about claims of the ability to hear the effect of "audiophile" ethernet switches, or USB noise/jitter devices?! As you can see in the links to these topics, I've tried to address them over the years so when questions come up, I can just point folks to areas covered already.
IMO, with time, more and more nonsense articles and claims have built-up in audiophile magazines and the Internet. I think young audiophiles must have difficulties making heads or tails about what is true anymore while negotiating this massive minefield of questionable information. I would not be surprised that some in this Industry actually prefer the presence of this "fog of subjectivity" because it allows people to make irresponsible claims about anything they want and not have to provide evidence so long as some key reviewers or readers have faith to repeat the belief over time!
That the ongoing majority of audio hardware reviewing these days is purely subjective is a bit of a problem in the context of what I believe to be very poor reliability.
Beyond the pure silliness of some of the bizarre claims above, as discussed a few years back, we do need to be aware of our hearing limitations and how this fits with writing reviews with conviction. Like it or not, a listener's age does play a role. Now that I have turned 50 years old myself, I am mindful of the effects on hearing (among other biological systems!). No, I do not believe I can hear as well compared to my 20's or 30's; to think otherwise would be ridiculous, uninsightful, and frankly dishonest. Looking around, there are some seriously old reviewers out there writing articles about the sound of hi-res DACs and $100,000 speakers, claiming the ability to hear nuances. Some might take offence to my skepticism as being "ageist", but isn't this simply what the science shows as we get older?
The problem with accepting subjective accounts at face value, whether concerns of the age of the individual or simply hard-to-believe claims (like cable differences), is that claims could all be wrong and there's typically no feedback mechanism to help correct these errors of "faith" in the audiophile world. Hey, If I'm going to be skeptical when someone tells me he saw a ghost, UFO or Sasquatch, then I think I have just as much right to question unlikely "audible" testimony!
Having said these things, it would not be fair of me to complain about others if I did not consider whether my own subjective impressions are correct! For this post, let's consider a comment I made a few weeks ago in the review on the Cayin RU6 R-2R DAC:
Putting on my "critical listening hat" for a bit, while usually not a problem, I noticed occasionally audible distortions with lower impedance headphones. Here's a concrete example I came across while rummaging through some music on my phone: with the closed-back lowish impedance AKG K371 headphones (measured here) in a quiet room, turn the Cayin to Low Gain, 90% volume (not too loud), NOS mode. Listen to the first few seconds of track 2 "A Million Dreams" on The Greatest Showman (2017, DR8) soundtrack. With this combination, I noticed during the lower-level piano intro, a subtle "haze"/"hash" has been introduced into the sound. Imagine something like the top end distortion of low bit-rate MP3 but transposed down an octave or two. It's not as strong with "OS" mode but still clearly not as clean as with a higher resolution DAC+amp (like say the Topping D10s with the Drop + THX AAA 789 headphone amp). I suspect this is an example where the non-harmonic distortions we see in the measurements surpassed the threshold for detection and I remember thinking "that sounds different...".
We can see the LH Labs Geek Out V2 in the right corner of the table further back. |
What you see is the Cayin RU6 plugged into the Raspberry Pi 4 "Touch" USB port (I'm running Volumio on the Pi). I have the track "A Million Dreams" (off The Greatest Showman CD) on a USB stick plugged into the Pi. The headphone output of the Cayin is connected to a test cable; one side (gold color) goes to the AKG K371 (rated 32Ω, see measurements for more details) and the other side is connected to the E1DA Cosmos ADC for recording, set to 2.7Vrms level or ~500-600Ω single-ended impedance - effective impedance seen by the headphone amp would be ~30Ω.
As you can see, I've then connected the Cosmos ADC to my Surface 3 computer where the audio is recorded at 24/96.
This test would be no good if I didn't give you a chance to hear this yourself! Here's the first 1:40 of "A Million Dreams" (portion of the music used as per "fair use" of copyrighted content for educational purposed) recorded from the Cayin RU6 with OverSampling (OS) and NonOverSampling (NOS) settings.
Download here:
"A Million Dreams": Cayin RU6 OS/NOS & LH Labs Geek Out V2, AKG K371 load
To get the best dynamic range in the recording, I used "High gain", 90% volume on the RU6. Also, I have included the Geek Out V2 DAC recording done at approximately the same output level. I did not adjust the volume between the OS and NOS recordings. The Geek Out V2 recording was normalized to the output level of the Cayin RU6 OS in Adobe Audition.
Here's a look at the 3 samples through the Dynamic Range Meter plug-in run on Foobar2000:
The Geek Out V2 recording was normalized in Adobe Audition to the Cayin RU6 OS recording using a slightly different algorithm than Foobar's RMS number, hence the slightly different average value. |
As you can see, all 3 tracks are the same 1:40 length, we see relatively close peak values, and RMS amplitude also similar - within 0.15dB. Average dynamic range is identical at DR12 between the different settings/DAC/headphone amps.
Now, let me cue up a couple of tracks at a time to listen through the ABX Comparator and see if I can ABX the difference:
Note: Since the tracks are already of similar amplitude, I did not use the ReplayGain setting in ABX Comparator to change volumes since I found that this exacerbated volume differences. |
Let's see the ABX results comparing Cayin OS vs. NOS modes, and between Cayin OS vs. Geek Out V2 (which is also an OverSampling DAC):
I used my desktop playback set-up which is a Topping D10 --> Drop + THX AAA 789 --> modded Dekoni Blue (anomalies easier to hear with Sennheiser HD800). Make sure your system has adequate resolution when listening for differences.
As you can see, differentiating the sound from the Cayin RU6 OS vs. NOS (left printout) could be done quickly - about 2 minutes for 16 trials with 14/16 correct.
I took more time to listen to a passage I thought differentiated the Cayin OS vs. Geek Out V2 (right printout) and the 16 trials took about 5 minutes total with 12/16 correct. Not perfect accuracy, but I think a reasonable demonstration that the files were not hard to tease apart and as expected, it was easier to tell the difference between Cayin's OS/NOS settings than between the more conventional sound of the Cayin OS vs. Geek Out V2.
You too should be able to hear the distortions with the NOS mode as a kind of distortion "haze" I described in the subjective review. It's not a static noise but fluctuates with the music itself, so it's not just an elevated noise floor (although the noise floor is higher). As for the Cayin RU6 OS vs. Geek Out V2, the difference is still present and you should be able to hear that the RU6's R-2R design is more noisy, definitely not as clean as a typical hi-res chip DAC such as the Geek Out V2's ESS SABRE9018AQ2M.
As expected, the Cayin NOS vs. Geek Out V2 was the easiest comparison to perform with 16/16 correct done a few days later even through DirectSound (set at 24/96 as per the test recording):
An interesting subjective effect I've described before with NOS playback is that it tends to sound "fuller", compared to the "leaner" OS mode. I believe this is because of the added harmonic and non-harmonic content to the signal with this DAC. Notice that the Dynamic Range Meter's results for the Cayin NOS recording showed a lower peak value and overall RMS amplitude despite the subjective impression of "fullness".
Another interesting effect I've heard with some NOS DACs is the impression that a reverb or fade trail seems extended or accentuated. Again this should not be interpreted as any kind of "accurate" sound even if some audiophiles might like this effect. I've often wondered with devices where the noise floor is higher (and this I think applies to vinyl as well), as the sound fades out quickly, maybe the mind is able to "fill in" the presence of that sound for a bit longer as it still detects some subtle noise/hiss. This is unlike a truly quiet device where the "fade to silence" is clearly perceived as quick and obvious, leaving more silence between notes and passages resulting in a "more hi-fi" or even "dry" sound. IMO, this is a good thing, it reflects the actual low-noise characteristic of the recording. And in those quiet intervals, we can appreciate the nuances thanks to the improved resolution.
Concluding thoughts...
Cayin RU6 with 1MORE Quad Driver IEM. |
So what does this little demo show?
1. Not all modern DACs sound the same! In particular, the R-2R architecture in the Cayin RU6 is relatively noisy and with the lower-impedance headphones putting more demand on the amp, this can be heard without great difficulty. Note that I have done other recordings with higher impedance headphones (like the AKG Q701) and the distortions are less audible.
The Cayin RU6 performance level (as can be seen objectively) is such that audibility is not difficult. However, most modern DACs have higher resolution and will be much more subtle if not impossible to hear differences when volume controlled.
2. NOS indeed sounds different. Lack of filtering allowing imaging artifacts to pass as well as a -3dB roll-off on the top end can be heard. Ultrasonic artifacts can (and will) cause issues like intermodulation to affect the audible frequencies, depending also on your playback system.
Let me reiterate the idea that "intentional distortions" such as NOS does qualitatively add a flavour to the sound even though it's not strictly "high fidelity". Whether you like this kind of sound is of course an individual choice. This theme is nothing new. In fact, the other day I was reviewing the very first blind test done on this blog back in 2013 comparing high bitrate MP3 vs. FLAC. Even back then, the results suggested some listeners preferred the sound of the MP3 tracks even though we know it's lossy and some nuances/details have been compromised.
My personal philosophy remains that of a preference for transparency when it comes to hardware. Even if we accept that imperfections, whether it be NOS, or high noise level, or elevated harmonic/intermodulation distortions might sound "better" to some audiophiles, for me, my "hi-fi" is still about primarily achieving lowest noise level, least distortion, and accurate temporal performance. If an artist intended the album to sound more "wet" than "dry", more noisy than pristine, then by all means he/she can do that. I just don't want my hardware to insist that all albums sound "wet" or have a uniform minimum noise level that's audible! Tonal colorations, distortions, and higher noise level are IMO antithetical to the goals of "high fidelity" reproduction. As such, the Cayin RU6 is not a "high fidelity" DAC/headphone amp.
3. See, objective measurements were able to demonstrate the differences between DACs, and we can actually hear it subjectively! Who says objective results are useless or don't correlate to sound quality? ;-)
I trust that listening to these samples now can give you some context into what the graphs and numbers were pointing to all along about this DAC if we now go back to the Cayin RU6 review.
4. Whew, I wasn't hallucinating when I wrote the subjective review! ;-)
I believe with a bit of work setting up the test conditions and making recordings of the output of devices like DACs, one can "capture" what was heard to show to readers/viewers if indeed there is something significant to hear. This goes beyond the usual publication of purely subjective claims or even measurements and graphs. I think it would be very cool if reviewers are able to chase down audible anomalies and demonstrate these with high-quality audio clips to contextualize what was heard.
For the vast majority of the time with reasonably high fidelity gear, I suspect differences will be subtle and will not correlate with some of the extremes of language used in the subjective descriptions (especially between devices like DACs). Apart from technical ability, I think pure-subjective reviewers likely will not want to do this as it weakens the prospect of generating a strong impression based on textual descriptions. Likewise, manufacturers looking for hype would likely be unable to achieve the excitement they're looking for.
Notice that I'm recording the demo tracks directly to 24/96, slight editing done in 32-bits, and making this available as a lossless file (like the AMPT recordings).
Many times, we see audiophiles wanting to show differences in sound using lossy YouTube audio. At a gross level, this might help give us a taste of the sound quality. These days, if we're intending to demo high fidelity gear, a YouTube video is obviously inadequate. For example, is this recording truly telling us anything about the equipment? Notice the nasty distortions at a few of the peaks, and lossy distortion artifacts.
My feeling is that subjective reviewers need to once awhile "prove" at least to themselves if not the readership that what they heard is indeed true. Just because a writer/reviewer may have had their hands on a bunch of gear over the years or had copious "experience" doesn't make the person necessarily a great listener nor should we think that age will be any kinder to self-proclaimed "Golden Ears".
I still think at the very least, there should be some kind of minimum standard "certification program" for subjective reviewers (like this). Not likely to happen ever, but if one is serious, listener training I think is a must.
-------------------
As a final test before returning the Caying RU6 DAC back to AudioPhil, I performed the stepped sine measurement to have a peek at distortion across output levels. I set the device to High Gain, 100% output level, connected to the E1DA Cosmos ADC at 2.7V reference (~600Ω single-ended impedance). Here are the OS and NOS graphs:
Since this is a USB loopback on my Intel NUC, 8kHz USB noise exacerbates the 8th harmonic. So I used 950Hz instead of 1kHz. |
This is the more "normal" pattern we see with high-resolution DACs these days.
Looking on the Internet (like here, here, here, here), we see many positive reviews of the Cayin RU6 with no recognition of rather serious fidelity limitations. Hey bros, do you guys ever give negative reviews or actually listen critically? Or are reviews just an exercise in marketing meant to push products and make a buck?
Behold the true nature of the Internet these days. Audiophilia being just a microcosm of much of the "information" fed to us in numerous other areas.
Don't worry guys and gals, I'm not paranoid or think they're "out to get us"; just a reminder that we need to be critical about all the stuff we read and see these days. We all need to make sure to sharpen our critical thinking skills, even more so when anyone can say anything online these days without needing to show evidence.
Recognition that the Internet in many ways is very much an advertising tool should not be shocking. Nothing here that would surprise Tim Berners-Lee or the idea that the Internet to a large degree has become a tool "about making markets" whether on a large scale with "Big Data" or implicitly on a micro level with content produced in one's YouTube broadcasts.
--------------------
On the muddled "Thoughts on reviewing"...
To end, I see that Jim Austin in a recent Stereophile editorial regarding "Thoughts on reviewing" continues to try to make a case that there is an apparent special need for a certain type of subjective evaluation of audio products. I believe this viewpoint is misleading and simply bad for the audiophile hobby.
Austin channels his theater professor and poses these apparent words of wisdom for reviewers: "Look inward before looking outward. How does it make you feel? Why?"
Immediately, I think many seasoned audiophiles will be a bit suspicious about such strong encouragement placed on emotions. Indeed, feelings are important and I certainly do not want to minimize this nor suggest life should be devoid of joys, sadness, or humor! However, when we open the pages of Stereophile to read about audio equipment reviews, are we sure that the process of writing reviews can fairly be lumped together with the output of "an aspiring playwright", the "theater critic", and a "composer"? These latter 3 individuals are either creating or writing about art. An audio equipment reviewer is writing about a technologically manufactured product. There is a wide gulf between sharing the emotional experience of a play, or enjoying an album, and discussing the merits and qualities of an inanimate product made in some kind of factory!
Yes, as humans we can share many experiences across cultures (Beethoven and the Beatles among them). Artistic appreciation is inherently qualitative although we can still measure certain aspects of music like tempo, or dynamic range, or complexity which might correlate to our "feelings" also. Is a product review in that same category as judging or critiquing art!? I don't think so!
The next part where Austin talks about "quantitative, scientific proof" is where the wheels really fall apart in this essay. Austin seems incapable of differentiating between the psychoacoustic science and research of Floyd Toole and quantitative measurements of products! By doing this, the article loses coherence and it becomes impossible to follow what his points are.
Yes, it's true that Toole's research helps educate by allowing for "human preference(s) made scientific". Sure, the scientific approach can produce reliable findings. But what's this about?
But there's a tradeoff. What it gains us in rigor and certainty, it gives up in specificity and incisiveness.
Huh? Tradeoff in what context? I thought we were talking about equipment reviewing here and just a moment ago weren't you expounding on there being a "shared humanity"? Isn't science that can illuminate reliably some parameters of how we as humans listen helping us understand that "shared humanity"? So why are we then pivoting to this:
First, what's measured is an average over a whole population, over all test subjects. Studying a population is necessary to achieve statistical significance—to be confident of the result to some stated precision—and to be certain your result has broad validity, beyond one individual. But what you learn from such research reveals little of interest about any individual in the group.
Suddenly, we're not "sharing" a commonality anymore, but complaining that scientific research (not equipment measurements necessarily) leaves out the "individual" within the group. As a reader trying to understand what the point is when it comes to equipment reviews, I'm confused!
Realize that complaining about not incorporating listener idiosyncrasies is a bit silly. When writing reviews, does the reviewer also not have to consider the audience? Does a reviewer write for the "individual" out there who only listens to death metal and demands >100dBSPL peaks (nothing wrong with this of course!)? Or does the equipment reviewing process also understands and targets the "average audiophile" with their tastes in music and desired sound quality? Within the pages of Stereophile, isn't it normal for a reviewer not to address the needs of that individual who listens only to death metal?
Then it gets even messier:
Second, in most such studies, human emotional response is reduced to a single, blunt concept: Which one do you prefer?
What does that teach us about love, or fate, or the human response to mortality—about deeper human things that almost anyone can learn about by watching a play or listening to a great piece of music? Not much.
Oi! What "such studies" are we talking about!? Since when did scientific studies become just about "a single, blunt concept" even? Did Austin ever actually read the studies and appreciate that research papers are way more nuanced than what he seems to be insinuating here? Don't research papers include statistical analyses and basic concepts like range, standard deviation, and confidence intervals to remind us that humans indeed do have different responses? To make it sound like science blandly is about providing "dull responses" (a phrase used later in the article) speaks volumes about Austin's lack of understanding as he creates the proverbial "strawman" that he tries to knock down; as if the sciences (especially biology, psychology, the social sciences) do not in their own ways also encompass an understanding of idiosyncrasies.
I've previously addressed thoughts like that second paragraph already - the concept of "emotional connection to music". If we're talking about equipment reviews here, tell me, what business does an inanimate piece of equipment, like say a DAC or even a nice tube amp have to do with teaching "love, or fate, or the human response to mortality"? Are those emotional, philosophical, even spiritual concepts something a piece of equipment is supposed to enhance or in any way affect? Or is it that those concepts are actually what the artist expresses in the music and our sound system's job is simply to "get out of the way", be "transparent"?
Personally, I will happily side with the latter idea that inanimate objects are simply incapable of especially enhancing these human thoughts and emotions. The idea that any amplifier can teach us "about love" or endows playback with some kind of "response to mortality" is bizarre and more than a little anthropomorphic even! Would an audio reviewer (based on his inner feelings of course) ever say that "This Pass Labs amplifier taught me more about my own mortality than that McIntosh."? Right about at this point in the article, I was reminded of that line by the antagonist Mugatu in Zoolander: "Doesn't anyone notice this? I feel like I'm taking crazy pills!" ;-)
Well, since he's in the zone, I guess he might as well continue to get even more metaphysical:
We read, or look, or listen, and we feel less alone. But to embrace those responses—to be a part of the human race—we have no choice but to abandon certainty. There are things we know about ourselves and others—about what it means to be human—that we'll never prove. We don't need to prove it because we know it.
My background in science sensitizes me to these issues—which many readers and other reviewers may never question....
I understand what he's trying to get at with the intangibles of human experience. But we're still talking about audio equipment reviews in the pages of Stereophile here, right? Are we sure he's not talking about a term paper on epistemology? I'm not sure how his background in science "sensitizes me to these issues". I have a background in science too and all I see are vague, hand-waving, somewhat disconnected comments with enough holes to make Swiss cheese jealous.
After all that, indeed it looks like he is talking about reviews and pivots back to this (emphasis mine):
Yes, our field—reproduced music—is underpinned by science. But that doesn't mean that the scientific approach to criticism is the only one that's valid. Music reproduction is based in electronics, but music itself, and our experience of it, is based in shared humanity.
Let me get this straight. While there are music/album reviews in Stereophile, most of the time, we're talking about audio reproduction hardware reviews, right? Audio hardware products, last I checked, are a type of consumer electronics (whether mass-produced or artisan-created, they're still a type of electronic machine for consumers). Since when should reviewing electronics hardware, and their ability to reproduce an electrical signal need to inherently include the "music itself"? DACs, amplifiers, speakers, computer streamers, even turntables can be considered works of art in their design, but is the "sound quality", often defined as how closely the device achieves a level of fidelity, worthy of artistic critique?
Consider for example if we read a review of an expensive, glorious, "high end" large screen flat panel OLED TV with amazing dynamic range, infinite contrast and ultra-realistic color tonality. Would an AV reviewer go beyond a "scientific approach" to that TV to also claim that viewing Schindler's List using the product "drove me to tears with such deep pathos in a way I had never experienced that movie before with any of my other large screen TVs!" Isn't that what expressing "feelings" and "shared humanity" might look like in such a product review?
Instead of appropriately attributing thoughts and feelings to the skills and artistry of Spielberg, or the actors, the screenwriter, sound editor, soundtrack composer, or the cinematographer, such an AV reviewer instead would be claiming that the object (TV) itself is the source of emotion and "humanity"! I trust that what the readers are actually looking for and the thrust of a TV review should be about its technical abilities. Nothing wrong with saying that the beautifully smooth black-and-white gradation in Schindler's List added to the ability for the reviewer to be drawn into the humanity of the film, but these "feelings" should not be a central rationale for why you, the reader should buy this phenomenal work of engineering!
As silly as this example might be, isn't this exactly what audiophile subjective reviews do (not just Stereophile, but also TAS, Hi-Fi+ and almost all the other magazines and YouTube videos)? Isn't this what Austin seems to be advocating? When month after month, we read about how these $50,000 speakers or that $20,000 computer streamer perpetually increases the emotional and even spiritual experience of the reviewer, at some point, I would hope audiophiles might just have had enough of this silly form of reviewing! Let's just get back to whether the device represented value for its build quality, usability, and of course technical performance.
The last paragraph in this article is interesting (again, emphasis mine):
The heart of this magazine, though, is in this other realm. It's based on a faith in shared human experience, a belief that what's true for the critic will be true for others, that humans share enough in common that not every insight requires proof. All that's needed is a certain sensitivity, seriousness, and goodwill.
Words like "heart", "faith", "belief", and "not every insight requires proof" all sound sweet. The kind of words the yoga instructor might evoke. Likewise, "sensitivity, seriousness, and goodwill" are fine human characteristics. But is there not a fundamental balance to be found? What about words that help us grasp rationality and reality: "mind", "facts", "knowledge", and "evidence"?
Earlier on in this article, Austin suggests to reviewers that:
Understanding your reaction requires a degree of self-reflection that's neither comfortable nor easy. Even if you're willing to engage in such self-reflection, answers don't come easily.
I would argue that to dig deep into truly honest, self-reflective reviewing, we must always maintain a balance with all those concepts in mind. Otherwise we'll drown in fantasies, wish-fulfillment, and neurotic preoccupations that might have nothing to do with the reality of these inanimate objects being reviewed. So are you truly willing to engage in self-reflection Mr. Austin? Is this comfortable or easy? I hope not because simply stating that "not every insight requires proof" seems to be a rather easy route for scammers and liars intent on financial gain regardless of truth. Let's face it, there are many "myths" in audiophilia that simply need to be addressed with "proof". There is already too much "fog of subjectivity" out there in audiophilia which has allowed the perpetuation of falsehoods, and hyped up questionable claims for years if not decades already. As I said near the start, I believe some in the Industry prefer it like this. Yet I believe this is unacceptable for the long-term health of the hobby, if not fundamentally flawed for honest journalistic ventures.
The implications I think are clear to most audiophiles these days that measurements and empirical examination of supposedly significant audible differences help anchor our beliefs within a reliable foundation. Austin mentions almost in passing the fact that at least Stereophile publishes measurements; the only reason I still have a subscription. Without that objective foundation, one is prone to lose track of reality and focus - as with Austin's IMO weak, and meandering "As We See It" article.
Stay rational, stay real, dear audiophiles. I hope you're enjoying the music - the art - with the full measure of emotions and humanity while using the engineered devices we are blessed to have at our disposal thanks to science.
-------------------------
As per the comment to G:
Spent about 30-40 minutes playing with Harman's "How To Listen" software... Achieved Level 6 with the "Band ID Training" at Trial 26, no practice, and just listening to the "How Long" track. Now that I know what's going on, should not be difficult achieving this with <25 trials the next time.
Would be curious whether others have tried this or what the "par" is for a 50 year old dude!?
Hmmm, it doesn't seem to be saving my progress running this on Windows 11. That means I won't really be able to use this seriously to "progress" over multiple listening sessions...
Also, the software definitely needs to allow the user to bring his/her own music. Gets really tiring listening to unfamiliar songs I might not like. :-(
ReplyDeleteI think subjective reviews can not only be entertaining (if one likes them) but *can* be of some use if well done. And by well done I mean if we get a well written sonic description by an astute listener who has capture the salient characteristics of the gear in question, in prose.
What I really dislike is when the emotions of the reviewer take center stage over description. I don't mind if some enthusiasm comes through in a review, but it's when the emotions of the reviewer is substituted for sonic description, e.g. "I LOVED how this system played my favorite Holly Cole albums" or "I found myself filled with emotion..."
I don't really care - taste being taste, what fills you with emotion may leave me flat, and visa versa. Just tell me how it *sounds.*
Worse is when the reviewer projects his own emotions on to the equipment itself (as you seem to be deriding here Arch). When, say, a speaker is described as "musical" or "emotional" or "producing profundity, or joy, or emotion" or whatever. This ascribing to audio gear the reviewer's emotions as if it were some objective quality of the device itself, that will produce the same reaction in all who hear it. Absolute waste of text wherever it appears IMO.
(I think there is a way in which it can be valid to describe how a system is capable of conveying properties of music. In other words, yes the emotional content comes from the music but a system can be better or worse in accurately delivering that content, e.g. the dynamic swells in Mahler or a John Bonham solo, which of course can push many people's emotional buttons. It's just when we anthropomorphize equipment as "emotional" it muddies the waters).
Hi Vaal,
DeleteYes, I agree. Subjective reviews can be entertaining and many of them can help us figure out if a certain device is worthwhile. I like when the subjective review is practical and the reviewer is able to show the utility of a product, or describing likes and dislikes of the design.
If a person has good ears, sure, the description of the sound quality can be enjoyable to read. Tell me if there's anything lacking in the breadth of frequencies this device delivers. Tell me how dynamic it sounds (those swells). Give me an example of which album resulted in an amazing soundstage in so I can maybe give it a try myself. Etc...
Indeed it's a bit painful when the description ends up being emotional like how much he "loved this" or evoked some description of joy or sadness without context to some property of the device. Even worse when the emotion is projected "into" the device instead of being derived from the music.
This last point is similar to the psychological "fetishism" idea where the emotional satisfaction comes from the inanimate object or ownership of that object.
I'm sure these age-old subjective reviewing motifs will continue although I hope audiophiles will grow tired over time.
ReplyDeleteHey Arch, I see you have spent far more time trying to understand and analyze Jim Austin’s article than I was willing to. Thanks for that. What did strike me as needing little analyzing on my part to disagree with was the same bit you included. First he praises the research done by Dr. Floyd Toole to “…uncover patterns in human emotional response.” And then he effectively dismisses its importance because “…what you learn from such research reveals little of interest about any individual in the group." Huh? For the purpose of the research, why do we need to know anything about the individuals other than they were a representative cross section of humans and the majority of them preferred a flat frequency response?
Then off he goes into the weeds talking about love, fate and mortality. Not sure what any of that has to do with reviewing electronic equipment designed for reproducing sound waves? For reviewing music and other forms of art, absolutely. But for my purposes of reading a review of equipment, the reviewers emotional response is not relevant to me. Because guess what? I have had emotional and sometimes spiritual responses to music on everything from a set of Apple earbuds connected directly to my iPhone (the horror), to a stack of McIntosh gear driving a couple of Paradigm Persona 9H’s.
Oh well, back to listening to some Jack White on a lowly Red Book CD, played on a second hand DENON DVD player, connected to my SVS Prime Active speakers. $650 of musical bliss.
Greetings Joe,
DeleteI think I spent way too much time on this Austin article ;-). But since he's the Editor-In-Chief of Stereophile, I just found the content too disappointing not to at least try to parse out the intent in more detail, especially as a general message aimed at reviewers at large!
That part about Dr. Toole is odd. I mean, it's one thing to criticize "objectivist" reviewers I suppose if he wanted to say that numbers and graphs alone as not representative of emotional response. But what Toole does isn't equipment reviewing!
Imagine writing a review for a very nice frying pan but complaining that the cookbook with various recipes that many people like doesn't apply given what I'm going to do with this frying pan. ;-)
What Toole's research does in setting out sonic preferences is on a whole different level than a mere review of a single device.
Likewise, I love music first. Even really crappy playback quality like SiriusXM in the car! The feelings for a song I love still come through, but the distortions get in the way of the material. Hi-Fi gear is by definition free of these distortions which then allows the music to simply be clearer. Appreciation of the stuff doesn't mean I have to fall in love with the stuff. ;-)
Those SVS Prime Actives sound like great units!
"praises the research done by Dr. Floyd Toole to “…uncover patterns in human emotional response.” And then he effectively dismisses its importance because “…what you learn from such research reveals little of interest about any individual in the group."" Those phrases triggered me too.
DeleteIt seems to be a common tool for laying baseless claims and recruiting disciples. Here, Austin cites legitimate research applicable to a wide swath of people and then speaks of his exceptionalism to that research. Who doesn't want to think of themselves as exceptional? We're all music enthusiasts, right? We listen to music for hours on end. We must be exceptional; well above the average. The reality of the fairly uniform performance envelope of human hearing, the demonstrably audible similarity and excellence of contemporary hifi gear, and affordability of that gear must really be a bummer. Chances are that any of us are going to be average and if one happens to be exceptional in hearing, they still aren't far from average.
I'm not surprised to see these sorts of tactics used in an enthusiasts publication. It really has become object adoration instead of performance appreciation. I hope that seeing through these sorts of ham handed logic games in this context can help us see through them in other contexts that have far greater world impact.
Yeah Doug, I was thinking the same thing (and you used a great word "exceptionalism")and how Jim Austin's writing naturally appeals to others that also like to think of themselves as exceptional. It allows them to justify $10,000 DACs and idiosyncratic designs that don't measure well.
DeleteYeah Arch, I bought my SVS actives about 9 months into the pandemic when it became apparent my working from home was going to be for an extended period of time (it will be 2 years at home in March). They serve me well as a pair of desktop speakers while I am working. I was considering a small pair of passive speakers and an inexpensive desktop amp, but after much research I decided active speakers would offer the best value. $600 for a DAC, 4x50 watt amplifier, WiFi for streaming, and a beautiful, very solid, white gloss painted cabinet. Oh, and I am impressed with the sound. Without getting too far into the audiophile vocabulary for describing sound, they sound clear, detailed, with a strong center image. I love em.
Yeah, I like the use of the word "exceptionalism" here as well, Doug.
DeleteThere is an air of superiority, elitism, self-importance among the gurus which by association gets endowed upon the acolytes who follow along. This relates as well to what AudioPhil below discusses about the tier system of Class A/B/C which provides a hierarchy to where one belongs within this organization with its form of the "priesthood".
I guess as humans, our psychology tends toward a desire for belonging, identification with out 'tribe', and seeking to find meaning. Nothing wrong with finding connectedness within the audiophile hobby or to make the assembling of a kick-ass audio system as an unspoken "goal" in life. For me, it's just important to be mindful of what we're doing. As a hobby, to make sure we put some rational, wise thought into this to escape from gross falsehoods and the "snakeoil" salesmen.
Great Joe, I bet many of us upgraded our audio systems and desktop devices as we switched to more "home office" duties.
Hey Arch, got a few thoughts I want to share first on my personal experience with the RU6 and on my interpretation of Jim Austin's article from another perspective.
ReplyDeleteFirst, thanks for some more rigorous testing of the RU6 to further elaborate on the distortions noted in your original review. I'd like to add a few more thoughts on my experience with it now that I've had it for a few more weeks.
To put things into context, allow me to share a little background. I may not speak for all users of the RU6, but for myself, the DAC is predominantly used by me at work sitting in front of a computer all day long. I listen to music streamed through TIDAL to help me stay focused on my work. Although I haven't been clinically diagnosed with ADHD, I often find it more difficult to concentrate without music playing in the background. My point is that I'm not intently focused on the quality of the music reproduction but over several hours a day, I have noticed that differently equipment I have tried in the past have caused me more listener fatigue, namely the Dragonfly Red and to a lesser extent the Chord Mojo. So based on the context of me trying to stay focused on work, the RU6 has been an easy and non-fatiguing listen.
Recently, I also acquired a used RME ADI-2 DAC that gives me a rather different listening experience. I'm glad that you've objectively proven that the distortions on the RU6 are audible in ABX testing because the last thing I want is two DACs that sound more or less the same. I actually listen to both for a few hours each day to change things up so I'm not listening to the same sound signature all day long. This helps to keep my brain fresh. And guess what? I actually like both. The ADI-2 gives me an unquestionably more resolving and accurate sound. It's all correct and faithful to the source and wows me with the details.
But like our appetite for food changes, I think it applies for music reproduction as well, and sometimes I just want to even knowingly see the world through rose-coloured glasses. I suspect this is the real reason why there are so many positive reviews from RU6 owners. It just sounds different from other more accurate DACs that they own. Although it goes against the audiophile philosophy of relentlessly pursuing accurate audio reproduction, we need to keep in mind that the RU6 never had such lofty aspirations.
The RU6 was never intended to be put through rigorous bench testing to show that it's not a "hi res" source any more than a 7 passenger luxury SUV was meant to taken out to the race track to measure their cornering ability or how a Formula 1 car would perform on a dirt road.
No RU6 owner would likely ever use it a main DAC to hook up with a high end speaker system or use it as a tool for critical testing. I reckon its main purpose is to provide a romanticized sound through a set of IEMs or headphones to make the workday go by easier.
Thanks for the note Phil,
DeleteGreat to hear you're having fun with the RME ADI-2 DAC. Definitely a different listening experience comparing that with the Cayin RU6!
I think you've touched on the fact that there are multiple intents that we as audiophiles can speak to. These are some of the important nuances which often get lost when we might talk about "sound quality" as a unitary phenomenon.
I've often wondered if as audiophiles maybe we should talk about INTENT first then discuss how closely the product REALIZED the target. So for the Cayin RU6 I might be tempted to say (and I agree with you):
INTENT: "24-bit", "high fidelity", "high resolution" sound quality - REALIZED: Unfortunately not capable of reproducing full 16/44.1 fidelity as a DAC and this is audible. Truthful marketing and hi-fi reviewers should be able to detect the sound as not quite "hi-fi". Cayin themselves have acknowledged that the RU6 is not the best when it comes to measured performance which is honest.
INTENT: Pleasant, non-fatiguing audio - REALIZED: Absolutely! I have no concern about this and as I mentioned in the previous review, it was more fun testing the Cayin than yet another AudioQuest Dragonfly. ;-)
I think it's a nice device to examine because it teaches us about the limits of our hearing, what is "euphonic" to our ears/mind, and ultimately what might be "good enough"!
IMO, there are of course many devices/technologies we can draw analogies to. Vinyl/LPs are a great example, not "accurate" when it comes to noise, frequency response, or wow-flutter time domain, but certainly very enjoyable. Many tube amps likewise have high distortion, but still very enjoyable even if not meeting up to a technical level of accurate resolution. Not to mention of course that as humans, our hearing systems are non-linear and there might be anomalies which are simply more "euphonic" among listeners.
Moving over to my take on Jim Austin's article, we need to consider that Stereophile audio equipment reviews on $50K speakers and $20K front ends serve a much different purpose than Consumer Reports reviews on toasters.
ReplyDeleteHaving gone down that rabbit hole to some extent earlier in my life, and to have evolved considerably over the past decade, I will say that for many "audiophiles" I have met along the way and for myself personally, the hobby wasn't really an interest in how audio equipment objectively measures and performs as a tool for audio reproduction. Rather, it is more about the (probably unhealthy) obsession with the actual pieces of equipment based on a prescribed hierarchy of desirability as dictated by the audio industry gurus who are of course, the audio reviewers themselves!
These audio reviewers, whose livelihoods come from the advertising revenue of hi-fi manufacturers and subscription revenue from their base of readers, have for decades acted as the leaders of this quasi-religion whose existence depends on the consumers continuing to fork over increasing sums of hard earned cash to reach the "next level" of audio nirvana. The hobby for the obsessed isn't simply building a musically satisfying system but more about saving enough pennies to move up from a Stereophile Recommended Class C system to a Class B and ultimately to the vaunted Class A or S systems.
Viewed in this context, we have to understand that the "irrational" audiophiles, if you may, don't really give a hoot whether they hear sonic improvements, or to be even more brutal, to actually LIKE the sound of their systems as long as it's been deemed worthy by the aforementioned audio gurus with the "golden ears". Does a $100K 4-stack dCS Vivaldi digital front end sound noticeably better than the one box $15K Bartok? Who cares, it sure looks more impressive, and most importantly, it buys you acceptance into the uppermost "inner circle" of the most accomplished and dedicated audiophiles on the planet. That's the endgame, baby! And that's what it's about. And when some misguided audio enthusiast has just dropped more bucks on his audio system than his car, he had better expect the reviewer to describe that equipment as a life changing "emotional" experience! Imagine if if Jim Austin describes the dCS nonchalantly describes the Vivaldi as “good” sounding, “reliable” and sounds just as good as any $1000 DAC out there. Do you think that would cut it? I reckon that’s the last review he’d be writing.
AudioPhil, I fully agree with your words. There is an unhealthy loop of reviewers "leading" the crowd and the industry, and at the same time being led by the marketing departments of the industry who suggest them which particular features of their expensive machinery to advertise. The result is a complete technical confusion for buyers. Here is an example that struck me recently: when talking about the "perfect" size for digital filters, here is a passage from the review of Chord Hugo M: "Every time you increase the number of taps, you improve the perception of pitch, timbre gets better—bright instruments sound brighter, dark instruments sound darker." So basically, the more taps you have, the "better" the filter. Now here is a statement from the review of CH Precision CD player: "Time smearing is basically if you put a single pulse through the system, if you have a filter with a very long impulse response, that single sample will extend over a large number of samples." They now state the opposite, and these writings are from the same magazine. The only fact all the reviewers are consistent about is that paying seriously more $$$ gives you much better sound, which is of course fine with the makers of this equipment.
DeleteHey AudioPhil,
DeleteWell, tell us how you really feel. ;-) I think as Voltaire once observed: "When it is a question of money, everbody is of the same religion."
Really the only way to be free to speak about this stuff without fears of repercussions is to ensure that there be no conflicts of interest, with financial reward being numero uno on a potentially long list of conflicts (interpersonal relationships, previous favours done being others).
Nice recent example Mikhail on taps, "blurring", and related filtering claims/beliefs (Feb 2020 on the Chord Hugo M, February 2022 on the CH Precision CD player). Truly unbelievable stuff.
Looks like Stereophile has no idea which comment is which, or at least is unable to take sides since the manufacturer said so, both companies presumably needing to be satisfied, neither to be questioned!
Fantastic "journalistic" follow-through! Excellent education for the consumer hobbyists! ;-)
As far as the 'next level' goes, I mentioned this on ASR, but it bears repeating. In his review of the ridiculously spendy CH D1.5 transport, Jim Austin lets slip a revealing comment, "One of the reasons that audio is so addictive—and takes such a big hit on our pocketbooks—is that we get used to a certain level of performance. We raise our systems to a certain level and acclimate. We stop noticing the surprise, and the music starts to bore us again."
DeleteOh dear. Once the lustre of a new shiny box wears off, the process of listening to music becomes a chore which can only be alleviated by spending even more on something shinier. Jim's confused ramblings about 'emotion' make a lot more sense when you realise he's really talking about gear fetishism.
I mostly listen to music on headphones and my system cost well under $1000 in total. Even though I think it sounds wonderful I'm occasionally plagued by the itch to 'upgrade' to something costlier. It's often hard to face the truth that in terms of real quality _this is it_: after a certain level I may change to something different, but it's not going to be better in any way that I can actually hear (and is very likely going to be worse). Needless to say, such a truth is a complete anathema to equipment makers who rely on fueling an endless cycle of consumption.
Well said Charles,
DeleteI agree, this is the awful "potential" that the advertising arm of basically any company or Industry would never want to be open about. As far as I can tell, 2-channel audio devices at least, especially DACs, CD players, streaming devices are already mature as technologies when it comes to accurate high-resolution reproduction devices. There is really no more "fidelity" performance to be had.
Yes, we can still improve in design, UI/ease of access, speed, features like DSP, low power, etc. But the idea of simply "higher fidelity reproduction", I would put money down to bet against anyone hearing a difference in a controlled blind test believing devices like that silly $$$$ CH CD player would make a difference.
I think the phrase serial hedonic adaptation is a nice one to describe this ridiculous upgrade cycle that some people get into:
Spend money --> feel the excitement! --> Get tired of it --> Some ad/reviewer says there's something even better --> You buy and feel happy for awhile... Rinse and repeat.
The whole cycle is indeed contingent on emotions. I suppose there's nothing wrong with that if we know that that's the hamster wheel we're revolving around. Of course, no audiophile magazine editor is ever going to admit this plainly. :-)
"I still think at the very least, there should be some kind of minimum standard "certification program" for subjective reviewers (like this). Not likely to happen ever, but if one is serious, listener training I think is a must."
ReplyDeleteDr Sean Olive et al of Harman International have developed just such a program to train the listeners used for their research (requiring passing at least level 8 in all tests), called 'How to Listen', which has actually been released to the public: http://harmanhowtolisten.blogspot.com/2011/01/welcome-to-how-to-listen.html
Unlike Philips' Golden Eears program, Harman's How to Listen has the backing of publicly published research demonstrating its efficacy in verifying and improving listening abilities, consistency and discrimination: http://seanolive.blogspot.com/2008/12/part-2-differences-in-performances-of.html
I've asked numerous reviewers to use this program and post their results. Not a single one has done so. Care to give it a try and post your results? ;)
Thanks G,
DeleteInteresting test software... I didn't do any pretraining and spent a few trials listening to the first track and got tired by Level 6 after 26 trials (start at Level 2 I think). There's only so long a guy can tolerate listening to the "How long has this been going on?" track. ;-)
Definitely will need to devote more time to this to "rise up the ranks". Also would love to easily get my own music on there which will help a lot when familiar with the song! I'm a bit unhappy that it seemed not to have save my progress?
Anyhow, for posterity, I attached the image of what this looked like above with 'watermarking'. I haven't seen results for others so don't know if achieving Level 6 with 26 trials in about 30-40 minutes is any good!?
Archimago, G, there is also great ear training software accompanying the book "Technical Ear Training" by J. Corey: https://sites.google.com/a/umich.edu/jason-corey/technical-ear-training. There is no "certification" as with Harman's program, however you can enjoy growth of your technical listening skills, and it allows using your favorite tracks.
DeleteGreat, thanks Mikhail,
DeleteWill keep an eye on that one as well. Nice seeing the various options out there. ;-)
Arch-
ReplyDeleteJust went and read the JA column. It's seems like some sort of new age nonsense. But I better understand now why he and his writers still support that failed format, MQA. His POV leads to nonsense conclusions and self-delusional responses like the ones over there to MQA.
Hey Unknown,
DeleteI've been criticized by some that I say too much about "mQa" in the blog which was why I avoided bringing it up here simply because the issues with Stereophile and how Austin, Atkinson, et al. think are IMO fundamentally flawed regardless of MQA.
You are right, the whole MQA debacle is a symptom of the deep-rooted distortion in that form of thinking and listening IMO. They promote things that are clearly untrue. They never seriously question the Industry representatives they come into contact with. They do not do their own investigation as "journalists". They do not perform honest (blinded) listening tests to confirm their sonic impressions.
Even with objective testing at their disposal, they never honestly tested MQA nor had the balls to "call a spade a spade" when it comes to nonsense like expensive bit-perfect computer streamers, power conditioners, snake oil cables and USB doohickeys even when it's obvious that they do not change the sound regardless of expense.
The same goes for most of the audiophile press. Even worse with those magazines that have zero objective testing capability and refuse controlled listening tests.
I don't think it's unreasonable to believe that the audiophile hobby would actually be better without these kinds of reviews and the misinformation they bring. For decades they have harmed the hobby by allowing Industry to create a market intent on extracting money with false claims of "better sound quality". I think that generation of viewpoints need to retire like so many of those reviewers.
As for "new age" ideas. Yeah, that's exactly the domain Austin appeals to with his "faith". I'm pretty sure the audio hobby at its heart isn't Yoga, Mindfulness, Religion, Naturopathy, Homeopathy, or Occultism. Those Edison Tone Test guys in blindfolds (our forefathers) in the picture above were aiming to obtain the leading technology for audio reproduction of their day. So too are audiophiles these days. I don't think they would have been impressed by Austin's article either.
"Thanks G,
ReplyDeleteInteresting test software... I didn't do any pretraining and spent a few trials listening to the first track and got tired by Level 6 after 26 trials (start at Level 2 I think). There's only so long a guy can tolerate listening to the "How long has this been going on?" track. ;-)
Definitely will need to devote more time to this to "rise up the ranks". Also would love to easily get my own music on there which will help a lot when familiar with the song! I'm a bit unhappy that it seemed not to have save my progress?
Anyhow, for posterity, I attached the image of what this looked like above with 'watermarking'. I haven't seen results for others so don't know if achieving Level 6 with 26 trials in about 30-40 minutes is any good!?"
I reckon that’s pretty good for a 50 year-old on your first try :) 2 levels below the minimum requirement of a trained listener according to Harman’s criteria. Kudos for being the first reviewer of many I’ve asked to post any kind of results!
You can use your own music, but only in ‘Practice’ mode:
https://www.audiosciencereview.com/forum/index.php?threads/harmans-how-to-listen-software.8262/#post-274427
Lossless files work for this, despite what that post says, although you might get an error when you try to select a previously added track in a new session after quitting and reopening the whole program, in which case you’ll have to add the track again.
The practice sessions aren’t saved, but the proper training sessions are, so it seems like you’ve only done the former so far. As it says on the left after you’ve logged into the program with your username and password, you begin a proper session by double-clicking on one of the session tasks in the list.
Thanks G, good to know that "Level 8" is what I'm aiming for as a start. ;-)
DeleteI'll have a peek at that ASR link and see if I can get my own music going. Still going to be arduous not allowing my own music for the formal (non-practice) sessions although I can understand why. I think it would be quite easy to cheat otherwise. Just wish there were more samples to choose from like a good classical orchestral track for example; even something out of a movie soundtrack.
Important discussion nonetheless! Ear training and testing are certainly not glamourous subjects nor going to sell products. Still, I feel that if a reviewer is going to be primarily subjective, doing the work of demonstrating listening ability is the price to pay to be "certified" in order to show readers that they're not actually deaf. (I have a suspicion that some out there might be.) ;-(
I find it funny that I'm the first "reviewer" to post any kind of result considering that I don't even consider myself much of a "subjectivist" reviewer. I would estimate that in the majority of my write-ups, maybe only 20-25% is actually about how something subjectively "sounds".
I no longer subscribe to any audio magazine and this article reminds me that I made the right decision.
ReplyDeleteI recently tried to determine which output would serve me best: Realtek Optical Digital Output (toslink) or Kernel streaming (USB). To switch from one to the other, I had to change the parameters in the playing software and select the appropriate DAC input: this takes about 30 seconds. I knew that the difference, if there is, should be very small. That 30 seconds delay made the task very hard and frankly, even if I have a very good system, a pretty good hearing for my age and of course, a very good memory of the exact timber of instruments, frankly, it was impossible for me to determine a winner. I know, both options are bit-perfect so, no difference should exists, but wait... they do not use the same data transmission technology and not all the same computer and DAC circuits, so there might be one, I just can't determine it objectively and, as opposed to reviewers, I admit it !
So, all this to say that if I can't remember exactly if DUT 'A' sounds better than DUT 'B' just after 30 seconds of silence, I just can take seriously reviewers that compares the SQ of a new version of a speaker or a CD player that they reviewed a few years ago: this is ridiculous and just proves that these guys writes reviews based on their feelings and trust that we will take all this bullshit for granted.
Hi DColby,
Delete"this is ridiculous and just proves that these guys writes reviews based on their feelings and trust that we will take all this bullshit for granted"
Yup, at the heart of it, it's mostly about "feelings" if we're all just being honest. Obviously, I cannot imagine Jim Austin (or the other usual suspects) ever agreeing with such a statement. So much of the audiophile magazine word-count is based on insistence of such subjective claims! To accept that much of it is likely just unreliable BS must clearly be defended against to the end (otherwise it would literally be the end of >1/2 of the "value added" content in reviews!).
This comment has been removed by the author.
ReplyDeleteWhat made me purchase the RU6 was the (however fallible) A-B testing here: https://youtu.be/nxrhRBiqt2I DSD type DAC simply sounds a small nuance drier.
ReplyDeleteAs for subjective testing the source makes some difference according to my hearing :) (Mobile phone is worse than Ipad Pro 2021 even with Dolby Atmos disabled on the phone which initially confused me). I think any device at max volume may not perform optimally, but anyway use it with Grado Sr80e with high gain, NOS and usually in the range 27 to 35 on the volume.
Positive example for the RU6 in my book is https://youtu.be/M38aWHxwtXE at 1:35 where the rawness and character in her voice comes across perfectly with the RU6 and less so with another DSD dac.