INTERNET BLIND TEST: Is high Harmonic Distortion in music audible? (plus Pet Shop Boys' newest, HRA needs HDR)

Another year, another Internet Blind Test, my friends! For me, doing this is important because it keeps us honest. It's worth reading this article in Audioholics published recently for an overview of the complexity of perception. I believe it's essential that audiophiles have an opportunity to participate in exercises of perception which I hope can enlighten ourselves and in aggregate, enlighten each other.

It's good to capture naturalistic data of audiophiles "in the wild" on top of information we may read from the "ivory tower" of Academia and profit-motive driven claims from Industry. Arguably, this is some of the most useful data. We can for example quite easily measure amazing levels of performance these days from our devices, but if we never correlate this to audibility, the data says nothing about relevance to humans, which at the end of the day is the only intent of these devices!

If you remember, last year we did the "Do digital audio players sound different?" test (the results systematically discussed starting here). This time around, in late 2019, Paul K [aka pkane] (who also wrote DeltaWave) contacted me to try out the early builds of his new software called "Distort"; aptly named software that allows us to purposely introduce anomalies into audio data. Among a number of distortions one could introduce is the one we've all heard off - harmonic distortion.

This is important and useful because other than noise level, probably the most common test done for all audio equipment is that of the "THD" (Total Harmonic Distortion). Remember that harmonics are mathematically related, integer multiples of the fundamental tone (or "first harmonic") in a wave. In the case of sound waves, harmonics are prevalent with natural instruments and human voices. Their presence, amounts, and relations to the fundamental and to each other will convey different "timbre" to the sound.

In the last while, I've been measuring amplifiers and no doubt you've seen THD(+N) graphs. The question is though, how well are we able to hear harmonic distortion? The answer to this question will help us contextualize the importance of this measurement when we're listening to gear. Let's try a blind test and see...

As you can imagine from the title of this test and that graph above, we're not talking "Can you hear 0.01% distortion?" We're talking samples with "high" THD. While I won't reveal the exact amount we're looking at in the test samples, it's something like what I show above... Some samples will be on the order of -50dB THD or 0.3% and possibly much more. :-)

For objectivists, 0.3% THD is rather poor if we're referring to most hi-fi components. A DAC with this much THD playing a 1kHz 0dBFS tone looking like the one above would be considered remarkably poor! Likewise, solid state amps usually are starting to clip by the time they hit this kind of distortion level. There are however tube amplifiers at 1W output into a typical 8Ω load that perform with even higher distortion and the occasional esoteric solid state amp as well (like the First Watt SIT-2 at my friend's place).

Speakers of course can have high distortion but for the sake of our blind test, let's mainly consider whether purposely distorted music can be heard and whether the sound of higher versus lower amounts of harmonic distortion can be differentiated using a few, what I believe are high quality test samples.

The procedure will be very similar to last year's blind test:

STEP 1. Download this big file. Notice that it's not too bad this year - "only" about 350MB in a single ZIP. Within it, like last year are 4 music samples. I'll go into detail with exact parameters used in Distort version 1.0.19 when I write up the results at the end.

Suffice it to say, I added specific amounts of harmonic distortion with no preference to odd or even order, aiming at certain target levels of THD including some that would be quite/very "high" for modern digital audio performance.

STEP 2. When you unZIP the files, you'll find the following tracks:

I. Clavier: "Prelude No. 19 in A major from the Well-Tempered Clavier Book I, BWV 864" - from Yo-Yo Ma, Chris Thile, and Edgar Meyer's excellent album Bach: Trios (2017). Notice that this is a high-resolution track in 24/96. This is also the only track in this test that started as high resolution. Track length is 1.5 minutes. As the album title suggests, this is a classical string trio recorded intimately. Lots of speed, low noise floor, listen to the instrument placement in the soundstage. Did the timbre of the string instruments change among the samples?

II. Horse: "Ballad of the Runaway Horse" is found on Jennifer Warnes excellent collection of Leonard Cohen songs (well known to audiophiles), Famous Blue Raincoat (1987). It's a great example of well-recorded female vocals with simple instrumental backing. Can you hear the change in the vocals and music introduced by harmonic distortion? Is there any change to the subjective "transparency" between tracks - any apparent "fog" or "dirt" between you the listener and Jennifer's performance?

III. Tootie: "Tootie" from Hootie & The Blowfish's Fairweather Johnson (1996) is another vocal track, but this time it's Darius Rucker's baritone voice taking center stage. Again, this is a good recording with tastefully done light rock instrumentation. Any change to timber of Darius' voice or instruments with the different amounts of harmonic distortion? Any apparent "artificiality" or dysphoric elements noticed?

IV. Rhapsody: "Hungarian Rhapsody No. 6 in D flat major" by Lang Lang from his album Liszt: My Piano Hero (2011). It has been claimed by many audiophiles that piano playback is difficult to reproduce well. Here is an example of a solo piano track with various amounts of harmonics added. Can you perhaps rank the samples from what you believe is "best" sounding to "worst", perhaps correlated to added distortion?

Notice that I have included Dynamic Range Meter data showing that each recording has good dynamic range (minimum DR11, up to DR15!). I've done some due diligence in ensuring that the average RMS amplitudes are within 0.01dB; this should be very nicely volume-matched when listening.

I limited all tracks to 2 minutes maximum which should be of adequate length for comparison. As usual, I'm using these music samples based on the principle of "fair use" for the purpose of research, discussion, and education. If you like the music, please purchase the albums and support the artists.

STEP 3. Listen to the test tracks!

As you can see, there are 4 variants of each music sample labelled A to D. Each letter represents the same amount of distortion added. So for example, "Clavier B", "Horse B", "Tootie B" and "Rhapsody B" would all have been processed with the same amount of harmonic distortion added. As a result, you can cross reference between different tracks to check if your impression is the same with the  different music.

As usual with these blind tests over the years, I recommend taking a pen and paper into the listening session to jot down impressions and make notes as to what you hear and what order you might consider sounds "best" to "worst". Consider also the question of harmonic distortion amount. For yourself, do you believe higher distortion is related to how you ranked which sounded "better"? You could very well have a separate list of which samples (A to D) you believe had more distortion added different from the subjective sense of which sounds "better" or "worse".

I'll ask about some of these subtleties when you submit your results.

Remember to take your time. This is not a speed contest :-). Also, no need to feel any pressure / anxiety. There is no "right" or "wrong" and results are submitted anonymously unless you want to identify yourself in the comments in case you want me to find your result later.

Notice that all tracks are 24-bits and 44.1kHz except for "Clavier", which is 24/96 so it's highly recommended that you use your highest resolution music system / speakers / headphones. One's system would impart its own distortion, but because of the amount of distortion added to some of these tracks, I suspect even with lower resolution DACs and "mid-fi" gear, much of the differences should still be there.

If you're wondering, randomization as to which distortion amount was linked to which sample (sample A to D) was done using multiple dice tosses with the help of my 12 year old to keep me impartial on the ordering itself. ;-)

STEP 4. Submit the results.

Alright, once you've had a good listen and decided on preferences, go here to submit your answers anonymously:

https://freeonlinesurveys.com/s/ZoxLngri

Like with my other blind tests, I will ask a bit about your demographics like which continent you're testing from, age (which decade), as well as experience such as whether you do music production, if you're a musician, and if you publish audiophile content. Also, I want to know about the system you used, approximate price, and would of course appreciate a description of the system components.

The heart of this survey asks you to rank from "best" to "worst" the samples from A to D for subjective sound quality. If you believe that harmonic distortion does not correlate with your subjective list of "best" to "worst", there is also a separate ranking if you want to list what you believe is the "lowest" to "highest" amount of harmonic distortion added.

I suspect you should be able to fill out the survey within 10 minutes if you already have a ranking in mind going in. There is also a place to indicate subjectively how much audible difference you heard, and as usual, it's certainly fine if you don't hear a difference - this too is important - make sure to let me know!

This blind test will run for 3 months - closing date APRIL 30, 2020. That should be plenty of time for everyone to test without pressure and send in results! Perhaps give it a try with family and especially audiophile friends.

Feel free to share this Internet Blind Test with other audiophiles, music lovers, and hi-fi enthusiasts. By all means, post the link to this test on social media and forums. Given the amount of distortion I have added to some of these test samples, the difference should be much more than something like a straight forward "hi-res vs. standard res" listening test (like this from 2014)!

Finally, needless to say, this is a LISTENING test :-). Don't be pulling out your audio editor to peek at the files before you actually listen and submit your results! If you do pull out that audio editor, perhaps refrain from posting pictures of FFTs and the like, and avoid discussions of results so you don't influence others... That would be much appreciated!

Have fun! Hopefully you'll enjoy the music and also the experience of being a participant, adding your voice to the final data set to be counted in this little study. A big thanks to Paul K for his phenomenal work on Distort plus wonderful discussions and assistance with this exercise!

Feel free to leave a comment or E-mail me (archimagosmusings@outlook.com) if you run into issues.

--------------------

I'm looking forward to spending more time listening to Hotspot, the latest release from the Pet Shop Boys this weekend (one of my favourite groups since the 80's). I've heard "Burning The Heather" and "Dreamland" already as singles. I agree with this BBC article about the duo's longevity.

I've already listened to the album once. As a music lover, I like it, but it's hard to be satisfied as an audiophile who cares about sound quality. While I've said this for years, I think it's important for the music industry and listeners to remember that sound quality was not always like the new albums these days. Some of the most popular albums from the PSBs sounded much more dynamic back in the day.

Here are waveform displays of "It's A Sin" (1987) and "Always On My Mind" (1988), truly classics that we can still hear on radio these days along with the simple but useful DR meter value showing very good DR13 and DR14 results:

In the 90's we see an intermediate state where things got louder but in the case of "Go West" (1993), the little "Postscript (I Believe in Ecstasy)" portion is allowed to maintain its softer, more gentle nature. DR values by this time in history started to drop quickly into the single digits. Notice what happened to this track when they remastered the album in 2001:

If you listen to the two versions back to back, the difference is obvious. There's an audible amount of distortion introduced in the 2001 remaster when they pushed the amplitude even higher and that "graininess" is even present in the "Postscript" as well. A nice (but unfortunate) example of why many modern remasters are not worth spending money on. I'm not saying that all "first pressing" albums sounded fantastic given limitations of the time like early ADCs used, but at least qualitatively, many do sound better by allowing one to turn up the volume to experience a more natural-sounding recording that doesn't restrict transients and allows the subtleties of soft passages to convey emotional intent.

By the second half of the 1990's, a song like "Se a vida é (That’s the Way Life Is)" (1996) shows that we're firmly entrenched in the "more modern" style of squashed dynamic range:

As we head into the new century, by the 2000's and 2010's, we can pretty well kiss punchy, dynamic albums goodbye with popular music as the discography almost permanently goes sub-DR8 with each album released:

Fast forward a little to today in 2020 and here's "Burning The Heather" off the new Hotspot album:

This song is a catchy, yet melancholic and sentimental track (which may have been referring to the suicide of this mystery case in 2015) - IMO, it could benefit greatly from more subtlety and sweetness. Alas, notice the low dynamic range. In fact, the album's average dynamic range is only at DR6 which is sadly quite typical of many mainstream new albums these days. I personally avoid any album averaging below DR6 as I almost invariably find them intolerably unpleasant sounding making them difficult with repeat listening when there's so much else I can be enjoying.

Sadly, if we look at the discography of almost all our favourite pop, rock, electronic acts over the decades, we'll likely find this pattern as average amplitude levels escalate and natural waveform extensions become arrested ("limited") at least on digital recordings. As audiophiles, regardless of how much fidelity we may achieve in our hardware, what's the point if what we listen to can never take advantage of those wonderful DACs, amps, speakers, and headphones? Is it any wonder that "new music" often doesn't satisfy those interested in high fidelity audio? (And also why I would never be able to include these dynamically compressed tracks when I run blind tests to see if people can hear increased distortion.)

To make matters worse, the music industry doesn't seem to recognize that their push into High-Res Audio (24-bits, >44.1kHz samplerate) is significantly hampered by the lack of High Dynamic Range material. Thankfully the professionals in the video world of Ultra High Definition actually understand the importance of High Dynamic Range contrasts and larger color gamut so that images "pop" with extended brightness, are capable of fantastic deep blacks, and have smooth gradations (rather than just pixel resolution). Unfortunately, it seems the audio intelligentsia missed the memo on what's important.

Of course, it's not like one cannot enjoy the music, I can still appreciate this Pet Shop Boys album despite low-DR; but why should we need to enjoy something despite extreme dynamic range compression? In 2020, who benefits from a DR6 album in a world where more and more of us stream online and each streaming service will apply their own form of ReplayGain-like volume normalization to achieve similar output levels?

Unfortunately, we as music lovers ultimately have had little control over any of this over the last generation. It's never too late for people like Neil Young to "get it" as a sound quality evangelist and the audio press should lay a bit more pressure on the music industry to maybe think about releasing "Advanced Resolution" versions of worthwhile albums.

Perhaps already since 2000, music lovers have voted with their cash. Don't just blame piracy. Don't just blame the loss of CD sales due to MP3. Certainly do not blame lossless CD-resolution (16/44.1) as somehow being inadequate for excellent sound quality. To some extent, perhaps unconsciously, I suspect many music lovers simply realized that the quality of music has dropped not because of the composition or lyrics, but rather the poor mastering quality which has nothing to do with bit depth or samplerate.

Presumably the artists too have no power over the final mastered sound quality. When opportunities arise, I think it's also worth asking our favourite artists like Neil Tennant and Chris Lowe:

"Why did it have to happen here? What have we done to deserve this? Left to your own devices, was this your intent as your sonic legacy? Is it hoping for a miracle to wish for opportunities of change? Seems to be so hard."


Listen to the blind test using well recorded music and get me results, everyone :-).

[2020-01-26] - Changed the download link to "Archimago's THD Blind Test Samples [Updated] (2020).zip" file. No change to test samples themselves so this will not change results if you're using the previous ZIP. Just added "README.pdf" file in the package to point to this blog post and procedure.

Test complete - see RESULTS HERE.