Saturday 13 January 2024

MUSINGS: On YouTube listening tests and the sound of "high-end" vinyl vs. streamed digital playback.

The Internet is wonderful, isn't it?!

In the span of a few decades, we can all make our presence known to the far corners of the world and express (almost) anything we want whether it's sharing what we had for lunch, whether we're "available", our political/moral/ethical affiliations, or even esoteric blog topics. ðŸĪŠ

Even better, we can project sounds and videos on YouTube and the like, opening up the opportunity to reach others though a modern "boob tube" where instead of the TV station programmer telling us what to watch next, good-ol' YouTube algorithms choose what we might desire from tracked search preferences. Amazing, if not also creepy...

Of course, if we have an entrepreneurial spirit, one could receive great rewards. Monetization potential can be impressive as witnessed by some of the elaborate content on YouTube channels! That's great so long as we're seeing knowledgeable, verifiable content presented in fair ways that can help teach and promote understanding. Not so great when information is perpetuating falsehoods, potentially destructive conspiracies and propaganda.

For this post, let's talk about something we've seen presented over the years on a number of audio channels. There is at least an implied idea out there, thanks to YouTube, we can now "hear" the sound quality of an audiophile system. Furthermore, that we can make comparisons of the sound quality. Is this true?

Over the recent holidays, a reader E-mailed me after seeing a series of A/B blind listening videos he came across and wanted to both vent as well as just talk about the problems of these kinds of tests. He is right to be concerned! Of course audio presented over YouTube as if representative of "hi-fi" sounds would be highly questionable.

He specifically pointed me to this video which he found "interesting":

Nice. No doubt, this fellow Jay's YouTube channel is fascinating. I recall mentioning this awhile back when discussing the Taiko computer - I think he's a dealer for the brand or something like that. The style of discourse and persona of the individual is interesting - arguably more interesting than the products he talks about!

Clearly on that channel, there seems to be a need to seek out very expensive gear in the "ultra-high-end" sandbox. It's a beautiful example of what I mean by the term "high-end" as different from the "hi-fi" hobby; there's a tendency to measure by price without any confirmation of elevated sonic fidelity needed (other than subjective opinion). As you can see, there are no actual tests being done nor any kind of objectivity maintained even though his sound room is being called his "lab". The value of that experience of the man playing with his gear is suggested as being translatable to knowledge worthy of him providing paid consultations.

[I noticed in some of the comments that there seems to be a rivalry of sorts between Jay's Audio Lab and OCD Hi-Fi Guy. As far as I can tell, both are audio gear dealers. Both sell/promote questionable stuff like expensive cables. Each hype about their stuff (what makes Playback Designs DACs all that great, are Authentic Audio Image cables special?). Each seem to have their "tribe" of followers. Both seem to promote highly entertaining audio subjectivisms in their passionate videos, even if at times they clearly don't know what they're talking about. Players in the same silly game, but wonderful drama I guess!]

Anyhow, as you can see in this video, we are presented with a kind of blind listening test to declare our preference for either digital streaming or vinyl playback. No doubt, he has put in a lot of money and effort into the room and components. Unfortunately, he doesn't list the components but it looks like this is a recording of the playback from Wilson Chronosonic XVX speakers (MSRP US$329k) amplified by what looks like his Gryphon Apex (MSRP $99k, hmmm, check out the measurements), I think he has that Taiko computer (MSRP $30k) streaming from presumably Qobuz to a MSB Select 2 DAC (MSRP $90k) over USB. The turntable is a Kronos Pro I think (MSRP ~$45k). Without including the phono cartridge (My Sonic Lab Signature Gold MC, US$9k), cables, preamp(s), power conditioner (including Stromtank), and "audiophile" network gear, we're already looking at a system "valued" at over US$550k.

I appreciate that he normalized the output levels to ~80dB SPL to keep "Presentation 1" and "Presentation 2" about the same volume; one of which being the digital playback, the other vinyl. We have some Sade "By Your Side" and "No Ordinary Love" as test samples in this video; reasonable choice of tracks given the clean production value.

Now, before talking about what can be heard, let's make sure we understand the context of such a "blind" test over YouTube:

1. The playback gear is expensive as noted above. No question, measured by price, this is up there with the "best" in the world as suggested by those familiar with the brands and products showcased in magazines at least.

2. We don't know what device he recorded the audio with. What camera? Was that a smartphone he used? Did he use a high quality external microphone, if so, what ADC?

3. Was the original recording lossy from that smartphone/camera? If lossless, was it at 24-bits? I assume there was some audio-video editing involved, was that done losslessly as well or was it re-encoded to another lossy file before uploading to YouTube?

4. Obviously, when we play back on YouTube, the audio would have been re-encoded in a lossy format then streamed. On a computer with good bandwidth, currently, YouTube sends out "high quality" Opus-encoded audio "opus (251)" which is variable bitrate up to 160kbps only. Given the improvements in psychoacoustic lossy compression over the decades, 160kbps Opus still sounds great.

5. Hopefully, on our playback end, we used a good DAC, amplifier, and speakers/headphones to listen with. Obviously, it could be hard to adjudicate sound quality if we're just listening off small laptop speakers.

As you can see, with each of those items from 2-4, risk of sonic deterioration compared to what was actually played in that room, is significant with potentially multiple signal changes between what was captured and the final YouTube playback. Clearly, given the variables, this would not pass adequate scientific rigor if we were to study the listening test results. Nonetheless, despite all these limitations, what do we hear in this vinyl vs. streaming comparison?

Well, although I would have no problem enjoying either presentation, let's be picky about the sound. Here's what I notice listening through a pair of Sony MDR-V6 headphones using the Drop+THX AAA 789 headphone amp on my desktop computer with basic, hi-res Topping D10s DAC (measured here) - nothing exotic needed:

1. The tonality isn't the same. Even if the source master is the same, for creating the vinyl version, application of RIAA EQ is necessary among other changes like making sure the bass isn't too "hot", possibly summing the bass frequencies to mono. I noticed that Presentation #2 has accentuated treble and a recessed midrange which some might prefer (the "smiley face curve"). Bass is fuller in Presentation #1, smoother midrange, more natural and potentially less harsh. Tonality varies more with vinyl playback depending on the phono pre-amp quality as well as the cartridge used.

2. There's a bit of "grittiness" to the noise floor that to me does not sound good in Presentation #2. On occasion we can hear other imperfections like crackling or anomalous "ticking" probably from some dust and light scratches. I noticed this more on "No Ordinary Love". Often when I hear this stuff on vinyl rips, I wonder if some listeners might mistaken the noise as if this is intended detail on the original recording. Higher frequency noise might sound like "air".

3. Louder portions sound a little harsh and kind of "flimsy" in Presentation #2. For example around 13:35 it didn't sound good to me with loss of definition during the louder and more complex passages.

4. Right-left balance slightly left-shifted in "By Your Side" on Presentation #2, especially the vocals. Overall, Presentation #1 sounds better centered and more stable.

5. For me, there's a "hollowness" to the sound of Presentation #2 which some might experience as more 'spatial' or even '3D'. I suspect this effect is from the tonality (1) with tipped-up treble plus imprecision of LP playback with higher crosstalk, resulting in a sound that seems to "fill" the room more at the expense of soundstage precision. 

Based on those impressions, I believe:

Presentation #1 = digital streaming
Presentation #2 = vinyl

The characteristics of Presentation #2 fit with the objective limitations of vinyl playback.

Many of the audible anomalies I listed above should be quite familiar to vinyl rippers. Typically, to do a good vinyl rip, one would need to meticulously clean the LP, make sure the cartridge and turntable are properly set up, and then make sure to use a good ADC. On the digital side once the audio is captured, fix anomalies like DC bias offset, adjust for channel imbalance, and software like Click Repair or Vinyl Studio or the professional iZotope RX can be used to reduce the noise and surface anomalies. Those are the processes and tools needed in order to make vinyl rips shine; it's quite a lot of work.

I see there's a second video with more A/B examples using older songs that are not particularly high quality from the late-'70s - early-'80s (Bowie/Moroder and Armatrading) that I find inherently more noisy. Unless I missed it somewhere, I don't think Jay has posted any reveal of the A/B test over the last 2 months. It would certainly be nice to get some confirmation. As we can see, a number of viewers offered their opinions in the comments. I don't see any consistent preference.

As I type this, I'm listening to the "No Ordinary Love" lossless rip from the Love Deluxe CD (1992, first release DR11) on my desktop computer and it clearly sounds much better than the YouTube video. This certainly isn't to imply that my desktop DAC or speakers are higher fidelity than the MSB and Wilsons, but rather a reflection of just how compromised the recording (from a smart phone or camera) is and once you go through the lossy encoding across YouTube, things likely got even a little more distorted.

Obviously, it would be impossible to fully gauge the sound of a hi-fi system over YouTube even if sometimes, comparisons like this could be useful.

I know, it's the age-old audiophile question: LP vs. CD, analog vs. digital - which is better?

As usual, they each have their pros and cons (discussed ages ago). But what is certain is that the differences are quite obvious with a little bit of critical listening. Even without listening, logically, one can deduce that for most modern albums like Sade which came out in the '90s, the recording and production were most likely done digitally anyways so it should not be controversial to suggest that staying digital would maintain the highest fidelity.

Note though that the opposite logic - suggesting that an analog recording should sound better as an analog LP - is not necessarily true! Analog recordings start as tape captures and they still need a lot of mastering work in order to sound good on vinyl. Characteristics like dynamic range typically is better with the professional tape than vinyl LP. In comparison, these days the analog-to-digital process can be done transparently with modern hi-res ADCs from the tape source (including to DSD). So long as the engineer is reasonably skilled, a CD/streamed/hi-res digital version of an analog recording can be way more accurate without all the processing and losses when converted to an LP (ostensibly this is why MoFi used DSD256 as their intermediate). There's also not nearly the amount of potential deterioration on the digital playback side from poor pressings, poor vinyl quality, dust, turntable issues, poor vibration isolation, tonearm & cartridge alignment, etc.

Regardless, beyond applying logic, one can still hear the difference between LP and digital as in this YouTube video because the difference is large. Despite the insistence and hype of vinyl advocates, these limitations can be heard without much difficulty; the change in fidelity such as the audible noise floor is higher than the lossyness of YouTube's Opus-encoded audio stream averaging around 128kbps (max 160kbps).

I know that some would use subjective words like LP's sound "smoother", "warmer", "musical", or has some kind of "analog sound". As far as I can tell, it's plain and simple EQ variation, higher distortion, noise, lower channel separation and at times audible wow and flutter that certain listeners seem to identify as preferable. I don't believe there's anything magical in those grooves, and remember not to get mentally trapped into the pitfall explanation of digital being "stair-stepped" as if all DACs are like the few NOS devices. Sure, these LP distortions can be subjectively "euphonic" to some tastes even though they add coloration to the sound and ultimately reduce fidelity. No reason to idealize turntable playback, even through a system that MSRPs for more than US$500k! (Kudos to Jay for putting his money down to demonstrate this for us.)

As suggested previously, when at audio shows, make sure to have a good listen to the difference between vinyl and digital versions of the same song since it's quite easy these days to find the digital track on streaming services. I contend that most of the time, the digital source will produce an obviously more accurate sound that's more impressive when showing off high-quality gear. In other words, the vinyl medium hinders the full potential of modern high-fidelity playback systems.

YouTube audio is not the best way to demonstrate hi-fi sound differences unless the change is quite significant like this digital vs. LP video. Best practice would be to make sure we're using low-noise microphones, high-quality ADC, and to record in hi-res losslessly (like 24/48). Make sure to edit and save losslessly, then upload to YouTube. While YouTube will still put the data through a lossy process (typically remaining at 48kHz), at least there will be less recompression and compounding of distortions (like temporal smearing) when transcoding. 

Alternatively, if we're intending to show the quality of the audio output from individual hi-fi components, as I do for my listening tests (like this), or AMPT recordings, capture the high-quality stream in 24/96 with a good ADC, let listeners download it and play as lossless hi-res FLAC files to the best of their ability.

Have a wonderful weekend everyone and I hope you're all enjoying the music!

Addendum 1:

A hilarious video on the vinyl vs. digital debate from the other day...



Addendum 2:

I figured it would be good to add this response to Doug's comment as part of the blog post since I think it belongs to the ideas in this broader text.

Nice comment and right to the point Doug,

When "pushing" to make a point, often people seem to lose track of the underlying nature of what they want others to believe. I tend to see 3 variations on this:

1. Objective Truth: This can be measured. Verified empirically to be true like with blind listening tests. And in the world of audio, fidelity to the source content is this factor that can be quantified and confirmed like the output from a hi-res DAC with typical resolution ideals the product is engineered for.

When trying to convey truth, at least in the world of engineered products, this is the highest level of discussion we should strive for in order to be sure we're well grounded in our beliefs.

2. Subjective Preference: This is what you're clearly noting. We sometimes like non-flat frequency responses. A little bit of noise can seem to add "atmosphere" in the room as if one is listening to some jazz in an intimate smoky nightclub after midnight. Distortions can add "air" to the room, and vinyl can sound good - to some people depending on situation and content. I agree, the trick is to make sure we express this insightfully and say things like "I prefer the extra distortion because it makes the sound more 'whole', rather than 'lean' on the more accurate digital gear". Nobody can disagree with that.

It becomes a problem if we pretend that our impressions are more "accurate", "hi-fi", or reflective of a truth that others should agree with; mixing personal preferences with #1.

3. Just Plain Bias: This one I find the most disturbing, potentially the most dishonest. Whereas one could measure and listen for accuracy (#1), and one could hear distortions and declare it to be "euphonic" (#2), bias happens when we declare something and there are personal benefits which one could consciously or unconsciously overlook. For example, this is what I believe happens with OCD Mikey where I have seen him declare that Playback Designs DACs sound like the greatest DACs in the world or something like that in some of his videos. Really? Based on what?

He's a salesman. He makes money off referrals, consultations, moving product, etc... I'm sure Playback Designs gear sounds fine, but clearly the bias "pushes" the product. I would bet that in a blind volume-controlled listening test, it would be highly unlikely that someone could differentiate a Topping, SMSL, other "cheap" Chinese brand in his eyes (looks like his personal bias is strong and IMO not totally fair when I've heard him talk about this once), with that fancy MPD-8 or whichever Playback DAC he's promoting. IMO, just another sales guy with a pitch, just like the expensive cable pushers (I think he does this as well). This is what gets us into the Snake Oil territory.

We see this kind of bias elsewhere. Just look at most politicians and what they "push"!

People are free to say whatever they legally want to on the Internet. The technology no doubt amplifies all kinds of ideas. It's up to us as knowledgeable people to parse out the difference, be wise about it, and develop the skills to elucidate truth.

Addendum 3:

Thanks to Mikhail for this objective analysis of the YouTube audio. See his comments below.

15 comments:

  1. Seems like people confuse objective measurements with subjective preferences and then try to push preference as a truth on others. This seems common in enthusiast endeavors. As you pointed out, objectively there's no contest in that CD quality or better is measurably superior to the LP format. That difference is so great that it is subjectively discerned by most people and could result in individual preferences. To then take one's preference and argue its superiority as a universal truth is where humans get silly. A preference is simply that, a preference. It may or may not agree with objective facts. Societies may be better off to recognize facts and acknowledge when preferences are not supported by those facts. It would be refreshing to me if a reviewer said that they love the sound of LPs with LP's more limited dynamic range, clicks, pops, wow and flutter over the measurably transparent and robotically perfect digital sources. Arguing subjective superiority of LP vs CD is as silly as arguing about blue sweaters being superior to green sweaters. Of course your LP sweater might be a little short top to bottom compared to the CD sweater and maybe have some bulges here and there but some people like them that way.

    ReplyDelete
  2. Hi Archimago! Thanks for a nice material to read. I decided to do some objective analysis of the recording—this gives a good chance to have a glimpse into these "high-end" setups. I used Smaart V8 software for that. This is a real-time acoustic analyzer that you might have seen used by sound engineers at live concerts and in theater halls. It employs what is called the "dual FFT" technique when the played material (tapped somewhere at output from the mixing console) is used as the test signal (instead of usual pink noise and measurement sweeps), so that it is possible to correct the transfer function of the PA system in the real time while the show is going.

    In this case, the reference signal is the digital version of the first song which I have captured digitally from Apple Music (lossless, 44.1kHz). Then I downloaded the m4a version of the audio from this YT video, at the highest bitrate available (127kbps, AAC). I downmixed both the reference and the recording into mono. This way, I've got a one channel reference and one channel "live" signal. This is what I have fed to Smaart. And this is what it has analyzed: https://t.ly/y6N91

    This is what it means. There are 4 graphs there: the magenta line which you can see around 0 dB is the transfer function of an ideal system that can be measured using this song, as you can see, it is almost flat. The green line is the transfer function of "Presentation 1" (of the first song)—that's this guy's speakers+room TF (and of the recording mic)! We can see that it is far from ideal, and sounding "bright." The orange line is the transfer function of "Presentation 2", you can see that it is closer to flat than the first one. I suspect, his system was tuned based on this source. The red line above shows the coherence of the second measurement—you can see that it is quite low (ideally it should be about 80% along the whole spectrum). While I was measuring the 2nd presentation, the timing of the measured IR was varying a log—I suppose this playback was from vinyl. The 1st presentation was much more stable in timing, and had good coherence (not shown).

    So, I agree with your hypothesis that presentation 1 is from streaming, presentation 2 is vinyl. I can also see that this guy does not use any room correction :) In general, such tests via YT have a little objective value and are mostly done to sell you something.

    ReplyDelete
    Replies
    1. Really nice work Mikhail! Thanks.
      Love it when the objective analysis helps build the case of the listening impression. Who said objective results don't reflect what we hear? 😁

      I've added text to your image for reference and posted as Addendum 3. Indeed, YT audio quality leaves *much* to be desired for listening tests...

      I see Jay uses acoustic panels in his room but no, I don't think there's anything more complex like DSP correction. The Wilsons likely could use a bit of DSP frequency response correction and time alignment, they're not "time coincident" (not sure what Wilson's speaker positioning calibration is aiming for).

      They also seem to be a bit large for his room size so I wonder about the driver integration at the listening position which I think is also against the back wall.

      Delete
    2. Thank you for showcasing my graph, Archimago!

      Looking at the IR of the digital playback session, there indeed seem to be strong nearby reflections (within 1 ft), and I suppose they cause the comb notches that we can see on the FR graph. Although, I can't say which objects cause them: the recording rig, seats, or walls.

      Speaking of driver alignment, it can be usually evaluated by looking at the group delay graph. In this particular case the graph is of course noisy. After smoothing it, I can see a peak in the group delay between 3 and 4.5 kHz which seems to cause that dip on the FR, and a bit of delay at 8.7 kHz. But overall it looks good. Sure, with this kind of a "measurement" setup, it's hard to make solid conclusions.

      Delete
    3. Wow Mikhail,
      You're the man for analyzing this recorded signal played off YT... :-)

      Thanks for the work!

      Delete
  3. I've been live streaming my vinyl setup (192/24) to my Chord Qutest Headphone setup for over a year now, and i just enjoy the music. I can sit at my desk, listening to my vinyl whilst i work and not disturb anyone else but also my head is permanently in the sweet spot (Phonitor 2 Crossover suite of controls). I still get the exercise by having to get up every 24 mins to flip sides. I enjoy the sound but i enjoy the music even more. I can switch to Roon listening of the same album to tell if its a vinyl or recording issue, generally with older recordings the tape noise is still present digitally but with digital the noise floor is lower. Sometimes i'm hard pressed to tell which prefer, but i reckon that's because the music just sounds good and i very quickly stop listening to the 'sound' and just enjoy the music.

    ReplyDelete
    Replies
    1. Thanks C3po,
      Yeah, at some point we should all just turn off the obsessive "hi-fi" analysis parts of our brain and just enjoy. :-)

      Having said that, I know that personally unless the vinyl is pristine and clean, I'd be too distracted by surface crackle, ticks, and pops when I've tried listening to LPs with headphones in the past.

      Curious, what are you using to live-stream your vinyl? I assume the turntable output to a high-quality ADC?

      BTW, the Phonitor 2 looks beautiful!

      Delete
    2. Hi Archimago, I'm Using the Wiim Pro Plus to ADC and then i feed the optical output to Chord Qutest. Even my old albums sound pretty good (but i do look after them and have a Project cleaner, some have been through a degritter). I'm currently listening to Spike Hughes and His All American Orchestra, a Mono Lp from 1957 and it is toe tappingly good, surprisingly very few albums are unlistenable due to crackles, this has become my favorite way of listening to albums (No room reflections to worry about and no person shouting "Turn that awful Music Off!!!"

      Delete
    3. I would add to stuff that needs doing to vinyl rips: high pass filter out TT rumble.

      Delete
  4. Luckily in my setup TT Rumble is sonically not an issue as i don't hear it across the majority of recordings.

    ReplyDelete
    Replies
    1. If you plan to do any upmix/demixing processing on the LP tracks, it needs to go. It's also 'stressful' to a system, as low bass tends to be. Old school receivers used to have a 'rumble cut' or 'infrasonic' toggle.

      Delete
    2. Yeah, good discussion guys. I agree, if there's significant infrasonic rumble, that's gotta go!

      Delete
  5. I had come across the following video by Michael Fremer directly before reading your post, and that fits right in here.

    https://www.youtube.com/watch?app=desktop&v=kdLbWQFdsPg

    In it he asks listeners their preference between the two different turntables he recorded.

    His statements regarding this seem to be:
    1. "HiFi" is the pursuit of the best possible playback of recorded music
    2. Vinyl can be more HiFi than digital, i.e. get closer to a correct reproduction of a recording
    3. The two turntables demoed are both top tier and their playback quality exceeds that of digital sources
    4. The two turntables sound audibly different, i.e. their playback differs from the recorded signal in different ways
    5. These differences are so large that they can be demonstrated in a YouTube video with lossy digital audio (specifically Opus codec at <=160 kbit/s)

    In the above either 2. is true and there should not be an audible difference between the two turntables in the YouTube video, or low-bitrate lossy digital audio can more faithfully reproduce a recording than vinyl.

    In either case Michael Fremer is a charlatan.

    ReplyDelete
    Replies
    1. Good one gzost,
      Over the years, I see Fremer has posted all kinds of videos and listening tests with vinyl gear. Specifically to that video, I don't think folks deny that different turntables can make things sound different. Heck I imagine every setup even with the same model turntable, tonearm, and cartridge will sound different due to geometry, and variation in the countless factors that go into the vinyl playback sound! Heck, even the LP itself changes over time. But overall, I would imagine turntables assuming otherwise good vibration isolation and speed stability could be the least significant.

      Great summary points!

      1. "HiFi" is the pursuit of the best possible playback of recorded music
      - Correct, "fidelity" means accuracy to the source material as laid out in the LP or digital file, etc. And we can define this objectively, and confirm with measurements that what's on those grooves or bits were reproduced accurately by our hardware.

      2. Vinyl can be more HiFi than digital, i.e. get closer to a correct reproduction of a recording
      - Absolutely wrong. LP cannot capture, store, nor reproduce the signal in any way, shape, or form that is more "faithful" to an original signal that was captured presumably in high resolution compared to digital. Yes, some people might prefer the nature of that playback or the processing that went into making an LP. Some CD masters can suck but that's not the fault of the playback system. We all have the subjective right to own our preferences and opinions of what sounds "good" to us. But we must be honest when using claims like "fidelity" or "accuracy" or "resolution" when we compare even an LP with the equivalent CD.

      3. The two turntables demoed are both top tier and their playback quality exceeds that of digital sources
      - As per the comment above, playback quality as defined by "high-fidelity" exceeding that of even just a reasonable digital source/playback of CD is not possible. But two turntables sounding different with different set-ups, calibrations, etc... sure.

      4. The two turntables sound audibly different, i.e. their playback differs from the recorded signal in different ways
      - Yup, clearly this must be so if it's audible. Which one is more "accurate"? Which one is "higher fidelity"? The only way to be sure is actually trying some objective testing comparing the output with the ideal output for the recording. Note thought that if it's so easy to change the sound, then clearly this must mean that tiny variations will affect fidelity and there's poor reliability from LP playback.

      5. These differences are so large that they can be demonstrated in a YouTube video with lossy digital audio (specifically Opus codec at <=160 kbit/s)
      - Yup, LP playback is so unreliable as a "hi-fi" system that even relatively low bitrate lossy codecs can convey the difference! By definition, LP playback must be lower resolution than Opus ≤ 160kbps which should not impress anyone.

      Delete