Saturday 25 January 2020

INTERNET BLIND TEST: Is high Harmonic Distortion in music audible? (plus Pet Shop Boys' newest, HRA needs HDR)


Another year, another Internet Blind Test, my friends! For me, doing this is important because it keeps us honest. It's worth reading this article in Audioholics published recently for an overview of the complexity of perception. I believe it's essential that audiophiles have an opportunity to participate in exercises of perception which I hope can enlighten ourselves and in aggregate, enlighten each other.

It's good to capture naturalistic data of audiophiles "in the wild" on top of information we may read from the "ivory tower" of Academia and profit-motive driven claims from Industry. Arguably, this is some of the most useful data. We can for example quite easily measure amazing levels of performance these days from our devices, but if we never correlate this to audibility, the data says nothing about relevance to humans, which at the end of the day is the only intent of these devices!

If you remember, last year we did the "Do digital audio players sound different?" test (the results systematically discussed starting here). This time around, in late 2019, Paul K [aka pkane] (who also wrote DeltaWave) contacted me to try out the early builds of his new software called "Distort"; aptly named software that allows us to purposely introduce anomalies into audio data. Among a number of distortions one could introduce is the one we've all heard off - harmonic distortion.

This is important and useful because other than noise level, probably the most common test done for all audio equipment is that of the "THD" (Total Harmonic Distortion). Remember that harmonics are mathematically related, integer multiples of the fundamental tone (or "first harmonic") in a wave. In the case of sound waves, harmonics are prevalent with natural instruments and human voices. Their presence, amounts, and relations to the fundamental and to each other will convey different "timbre" to the sound.

In the last while, I've been measuring amplifiers and no doubt you've seen THD(+N) graphs. The question is though, how well are we able to hear harmonic distortion? The answer to this question will help us contextualize the importance of this measurement when we're listening to gear. Let's try a blind test and see...

As you can imagine from the title of this test and that graph above, we're not talking "Can you hear 0.01% distortion?" We're talking samples with "high" THD. While I won't reveal the exact amount we're looking at in the test samples, it's something like what I show above... Some samples will be on the order of -50dB THD or 0.3% and possibly much more. :-)

For objectivists, 0.3% THD is rather poor if we're referring to most hi-fi components. A DAC with this much THD playing a 1kHz 0dBFS tone looking like the one above would be considered remarkably poor! Likewise, solid state amps usually are starting to clip by the time they hit this kind of distortion level. There are however tube amplifiers at 1W output into a typical 8Ω load that perform with even higher distortion and the occasional esoteric solid state amp as well (like the First Watt SIT-2 at my friend's place).

Speakers of course can have high distortion but for the sake of our blind test, let's mainly consider whether purposely distorted music can be heard and whether the sound of higher versus lower amounts of harmonic distortion can be differentiated using a few, what I believe are high quality test samples.

The procedure will be very similar to last year's blind test:

STEP 1. Download this big file. Notice that it's not too bad this year - "only" about 350MB in a single ZIP. Within it, like last year are 4 music samples. I'll go into detail with exact parameters used in Distort version 1.0.19 when I write up the results at the end.

Suffice it to say, I added specific amounts of harmonic distortion with no preference to odd or even order, aiming at certain target levels of THD including some that would be quite/very "high" for modern digital audio performance.

STEP 2. When you unZIP the files, you'll find the following tracks:
I. Clavier: "Prelude No. 19 in A major from the Well-Tempered Clavier Book I, BWV 864" - from Yo-Yo Ma, Chris Thile, and Edgar Meyer's excellent album Bach: Trios (2017). Notice that this is a high-resolution track in 24/96. This is also the only track in this test that started as high resolution. Track length is 1.5 minutes. As the album title suggests, this is a classical string trio recorded intimately. Lots of speed, low noise floor, listen to the instrument placement in the soundstage. Did the timbre of the string instruments change among the samples?

II. Horse: "Ballad of the Runaway Horse" is found on Jennifer Warnes excellent collection of Leonard Cohen songs (well known to audiophiles), Famous Blue Raincoat (1987). It's a great example of well-recorded female vocals with simple instrumental backing. Can you hear the change in the vocals and music introduced by harmonic distortion? Is there any change to the subjective "transparency" between tracks - any apparent "fog" or "dirt" between you the listener and Jennifer's performance?

III. Tootie: "Tootie" from Hootie & The Blowfish's Fairweather Johnson (1996) is another vocal track, but this time it's Darius Rucker's baritone voice taking center stage. Again, this is a good recording with tastefully done light rock instrumentation. Any change to timber of Darius' voice or instruments with the different amounts of harmonic distortion? Any apparent "artificiality" or dysphoric elements noticed?

IV. Rhapsody: "Hungarian Rhapsody No. 6 in D flat major" by Lang Lang from his album Liszt: My Piano Hero (2011). It has been claimed by many audiophiles that piano playback is difficult to reproduce well. Here is an example of a solo piano track with various amounts of harmonics added. Can you perhaps rank the samples from what you believe is "best" sounding to "worst", perhaps correlated to added distortion?
Notice that I have included Dynamic Range Meter data showing that each recording has good dynamic range (minimum DR11, up to DR15!). I've done some due diligence in ensuring that the average RMS amplitudes are within 0.01dB; this should be very nicely volume-matched when listening.

I limited all tracks to 2 minutes maximum which should be of adequate length for comparison. As usual, I'm using these music samples based on the principle of "fair use" for the purpose of research, discussion, and education. If you like the music, please purchase the albums and support the artists.

STEP 3. Listen to the test tracks!

As you can see, there are 4 variants of each music sample labelled A to D. Each letter represents the same amount of distortion added. So for example, "Clavier B", "Horse B", "Tootie B" and "Rhapsody B" would all have been processed with the same amount of harmonic distortion added. As a result, you can cross reference between different tracks to check if your impression is the same with the  different music.

As usual with these blind tests over the years, I recommend taking a pen and paper into the listening session to jot down impressions and make notes as to what you hear and what order you might consider sounds "best" to "worst". Consider also the question of harmonic distortion amount. For yourself, do you believe higher distortion is related to how you ranked which sounded "better"? You could very well have a separate list of which samples (A to D) you believe had more distortion added different from the subjective sense of which sounds "better" or "worse".

I'll ask about some of these subtleties when you submit your results.

Remember to take your time. This is not a speed contest :-). Also, no need to feel any pressure / anxiety. There is no "right" or "wrong" and results are submitted anonymously unless you want to identify yourself in the comments in case you want me to find your result later.

Notice that all tracks are 24-bits and 44.1kHz except for "Clavier", which is 24/96 so it's highly recommended that you use your highest resolution music system / speakers / headphones. One's system would impart its own distortion, but because of the amount of distortion added to some of these tracks, I suspect even with lower resolution DACs and "mid-fi" gear, much of the differences should still be there.

If you're wondering, randomization as to which distortion amount was linked to which sample (sample A to D) was done using multiple dice tosses with the help of my 12 year old to keep me impartial on the ordering itself. ;-)

STEP 4. Submit the results.

Alright, once you've had a good listen and decided on preferences, go here to submit your answers anonymously:


Like with my other blind tests, I will ask a bit about your demographics like which continent you're testing from, age (which decade), as well as experience such as whether you do music production, if you're a musician, and if you publish audiophile content. Also, I want to know about the system you used, approximate price, and would of course appreciate a description of the system components.

The heart of this survey asks you to rank from "best" to "worst" the samples from A to D for subjective sound quality. If you believe that harmonic distortion does not correlate with your subjective list of "best" to "worst", there is also a separate ranking if you want to list what you believe is the "lowest" to "highest" amount of harmonic distortion added.

I suspect you should be able to fill out the survey within 10 minutes if you already have a ranking in mind going in. There is also a place to indicate subjectively how much audible difference you heard, and as usual, it's certainly fine if you don't hear a difference - this too is important - make sure to let me know!

This blind test will run for 3 months - closing date APRIL 30, 2020. That should be plenty of time for everyone to test without pressure and send in results! Perhaps give it a try with family and especially audiophile friends.

Feel free to share this Internet Blind Test with other audiophiles, music lovers, and hi-fi enthusiasts. By all means, post the link to this test on social media and forums. Given the amount of distortion I have added to some of these test samples, the difference should be much more than something like a straight forward "hi-res vs. standard res" listening test (like this from 2014)!

Finally, needless to say, this is a LISTENING test :-). Don't be pulling out your audio editor to peek at the files before you actually listen and submit your results! If you do pull out that audio editor, perhaps refrain from posting pictures of FFTs and the like, and avoid discussions of results so you don't influence others... That would be much appreciated!

Have fun! Hopefully you'll enjoy the music and also the experience of being a participant, adding your voice to the final data set to be counted in this little study. A big thanks to Paul K for his phenomenal work on Distort plus wonderful discussions and assistance with this exercise!

Feel free to leave a comment or E-mail me (archimagosmusings@outlook.com) if you run into issues.

--------------------

I'm looking forward to spending more time listening to Hotspotthe latest release from the Pet Shop Boys this weekend (one of my favourite groups since the 80's). I've heard "Burning The Heather" and "Dreamland" already as singles. I agree with this BBC article about the duo's longevity.

I've already listened to the album once. As a music lover, I like it, but it's hard to be satisfied as an audiophile who cares about sound quality. While I've said this for years, I think it's important for the music industry and listeners to remember that sound quality was not always like the new albums these days. Some of the most popular albums from the PSBs sounded much more dynamic back in the day.

Here are waveform displays of "It's A Sin" (1987) and "Always On My Mind" (1988), truly classics that we can still hear on radio these days along with the simple but useful DR meter value showing very good DR13 and DR14 results:



In the 90's we see an intermediate state where things got louder but in the case of "Go West" (1993), the little "Postscript (I Believe in Ecstasy)" portion is allowed to maintain its softer, more gentle nature. DR values by this time in history started to drop quickly into the single digits. Notice what happened to this track when they remastered the album in 2001:



If you listen to the two versions back to back, the difference is obvious. There's an audible amount of distortion introduced in the 2001 remaster when they pushed the amplitude even higher and that "graininess" is even present in the "Postscript" as well. A nice (but unfortunate) example of why many modern remasters are not worth spending money on. I'm not saying that all "first pressing" albums sounded fantastic given limitations of the time like early ADCs used, but at least qualitatively, many do sound better by allowing one to turn up the volume to experience a more natural-sounding recording that doesn't restrict transients and allows the subtleties of soft passages to convey emotional intent.

By the second half of the 1990's, a song like "Se a vida é (That’s the Way Life Is)" (1996) shows that we're firmly entrenched in the "more modern" style of squashed dynamic range:


As we head into the new century, by the 2000's and 2010's, we can pretty well kiss punchy, dynamic albums goodbye with popular music as the discography almost permanently goes sub-DR8 with each album released:



Fast forward a little to today in 2020 and here's "Burning The Heather" off the new Hotspot album:


This song is a catchy, yet melancholic and sentimental track (which may have been referring to the suicide of this mystery case in 2015) - IMO, it could benefit greatly from more subtlety and sweetness. Alas, notice the low dynamic range. In fact, the album's average dynamic range is only at DR6 which is sadly quite typical of many mainstream new albums these days. I personally avoid any album averaging below DR6 as I almost invariably find them intolerably unpleasant sounding making them difficult with repeat listening when there's so much else I can be enjoying.

Sadly, if we look at the discography of almost all our favourite pop, rock, electronic acts over the decades, we'll likely find this pattern as average amplitude levels escalate and natural waveform extensions become arrested ("limited") at least on digital recordings. As audiophiles, regardless of how much fidelity we may achieve in our hardware, what's the point if what we listen to can never take advantage of those wonderful DACs, amps, speakers, and headphones? Is it any wonder that "new music" often doesn't satisfy those interested in high fidelity audio? (And also why I would never be able to include these dynamically compressed tracks when I run blind tests to see if people can hear increased distortion.)

To make matters worse, the music industry doesn't seem to recognize that their push into High-Res Audio (24-bits, >44.1kHz samplerate) is significantly hampered by the lack of High Dynamic Range material. Thankfully the professionals in the video world of Ultra High Definition actually understand the importance of High Dynamic Range contrasts and larger color gamut so that images "pop" with extended brightness, are capable of fantastic deep blacks, and have smooth gradations (rather than just pixel resolution). Unfortunately, it seems the audio intelligentsia missed the memo on what's important.

Of course, it's not like one cannot enjoy the music, I can still appreciate this Pet Shop Boys album despite low-DR; but why should we need to enjoy something despite extreme dynamic range compression? In 2020, who benefits from a DR6 album in a world where more and more of us stream online and each streaming service will apply their own form of ReplayGain-like volume normalization to achieve similar output levels?

Unfortunately, we as music lovers ultimately have had little control over any of this over the last generation. It's never too late for people like Neil Young to "get it" as a sound quality evangelist and the audio press should lay a bit more pressure on the music industry to maybe think about releasing "Advanced Resolution" versions of worthwhile albums.

Perhaps already since 2000, music lovers have voted with their cash. Don't just blame piracy. Don't just blame the loss of CD sales due to MP3. Certainly do not blame lossless CD-resolution (16/44.1) as somehow being inadequate for excellent sound quality. To some extent, perhaps unconsciously, I suspect many music lovers simply realized that the quality of music has dropped not because of the composition or lyrics, but rather the poor mastering quality which has nothing to do with bit depth or samplerate.

Presumably the artists too have no power over the final mastered sound quality. When opportunities arise, I think it's also worth asking our favourite artists like Neil Tennant and Chris Lowe:

"Why did it have to happen here? What have we done to deserve this? Left to your own devices, was this your intent as your sonic legacy? Is it hoping for a miracle to wish for opportunities of change? Seems to be so hard."


Listen to the blind test using well recorded music and get me results, everyone :-).

[2020-01-26] - Changed the download link to "Archimago's THD Blind Test Samples [Updated] (2020).zip" file. No change to test samples themselves so this will not change results if you're using the previous ZIP. Just added "README.pdf" file in the package to point to this blog post and procedure.

Test complete - see RESULTS HERE.

52 comments:

  1. Those DR results are really disturbing! Can you please explain how to calculate the DR value, as I'm now driven to go and measure my favourite trad folk music recordings.

    ReplyDelete
    Replies
    1. Hi Rob,
      I've just been using Foobar and the free "foo_dynamic_range_1.1.1" plugin all these years.

      See if you can find this and get it working... If not, let me know and I'll see about putting up a post with links.

      You might be OK with the traditional folk music. Likewise, some genres like classical are typically recorded well. Sadly dynamic range compression has infected soundtrack recordings these days.

      Delete
    2. Thanks, I will give it a try!

      Delete
    3. My go-to is the Internet DR dB: http://dr.loudness-war.info/

      Also, a mastering engineer has a website and blog dedicated to the topic of DR. https://productionadvice.co.uk/

      His more recent posts deal with all the streaming services enforcing their own loudness equalization, thus ending the loudness war by rendering it an entirely self-defeating exercise. But it may take a few years before everyone on the production side gets the memo. Also, it would likely mean everyone will produce to the same loudness/DR. He uses a different scale, which I'm not sure how it translates to DR, so I'm not sure what DR value we should expect in the end. (It could also back-fire if producers think they can sell a CD or iTunes version with higher DR than the services allow to stream as some sort of premium track: moar louder, moar better. -- I'm suspicious Tidal does that for "Masters" quality & MQA, btw.)

      His older posts go into some of the technical details on loudness and what it means. He even has some before and after comparisons. Pretty cool archives at his site. HTH.

      Delete
    4. Hi Allan,
      Thanks for the note and the reminder of Ian Shepherd's work. I see that Ian is typically referring to R128 LUFS reference values used with the streaming services. He has the "Loudness Penalty" analysis and plugin which looks like a good way for producers to double check what these online services are doing; and hopefully optimize dynamics.

      Delete
    5. If not obvious, I meant to say "sell a higher Loudness CD or hi-res version than streaming services allow."

      I can't imagine a sadder irony.

      Also, and on the other hand to my post, as Mark alludes the truce (much less victory) in the Loudness War keeps getting predicted by the losers, but so far as I can tell the winners haven't gotten the memo.

      Delete
    6. "Also, and on the other hand to my post, as Mark alludes the truce (much less victory) in the Loudness War keeps getting predicted by the losers, but so far as I can tell the winners haven't gotten the memo."

      That's very true. In principle, logic suggests that the "War" should be near the end or even "won". But habits die hard and inertia always takes time.

      Delete
  2. Just this morning I read an article in which the author said:
    "If the softest sounds are boosted to match the level of the loudest sounds we have a crushed mix with no dynamic range. This is one of the byproducts of the loudness wars – poor dynamic range. Fortunately, I think those times are now behind us."
    ( https://www.dagogo.com/appropriate-pre-master/ )
    Behind us??? Evidently not!

    ReplyDelete
    Replies
    1. Yeah Mark,
      Sadly I think it's still not exactly "behind us". :-(

      Maybe we don't have as many DR1 or DR0 type material these days?

      However, I think with the recognition and targeting as we move into more streaming as alluded to above, perhaps there's hope...

      Delete
  3. I'd like to see recorded music labelled with the Dynamic Range, like nutritional information for food products.

    ReplyDelete
    Replies
    1. Thanks for the note Richard,
      I agree. This would be very useful. It won't tell us anything about quality of the music itself, but can give us hints at least ;-).

      Delete
  4. Yay! I get to participate in an archimago blind test :-)

    ReplyDelete
  5. Arch, love it! re: ...but if we never correlate this to audibility, the data says nothing about relevance to humans... Too right man! This is the issue with most objective measures, i.e. the lack of correlation to what we can or cannot hear. This will be interesting :)

    Speaking of crushed recordings, mixes and masters: Dynamic Range: No Quiet = No Loud

    Keep up the great writings Arch!

    ReplyDelete
    Replies
    1. That was a great article, Mitch. I'd read it a while back, and enjoyed re-reading just as much. Thanks for the reminder.

      Delete
    2. Hey Mitch,
      Yup, very true about the lack of dynamic range.

      Have fun with the listening I hope :-).

      Delete
    3. Hi Allan, thanks for that! I don't know if it will ever get better with big studio declining recording budgets and more folks recording/mixing in their garage or basements with "studios in a box" using DAW's with "mastering" presets already set for DR6... However, their is still some goodness to be found in pop/rock recordings, especially those that follow Bob Katz's K-System.

      So Arch, I am still having fun listening, but sometime the drone of DR6 or lower sounds like AM radio no matter what gear one is using :-) But yah, still having fun! Will give your distortion tests a listen - this is really excellent as it will put a fine point on the audibility of harmonic distortion - I predict folks will be surprised at the results!

      Delete
  6. https://www.audiosciencereview.com/forum/index.php?threads/beta-test-distort-audibility-of-distortions.10163/post-277764

    ReplyDelete
  7. From what I understand the trick for mastering these days is to make a sound that can stand out amongst background noise. Most music is played through 1) low quality earbuds and transmitted usually at reduced bitrates through mobile phones by streaming services or 2) on low quality speakers in busy noisy environments, i.e. for playing in the kitchen with children running round or in the car. Very rarely (with exception of audiophiles) does critical listening happen in quiet optimum environments. The advantage low DR is that you are guaranteed your music will be heard; the ecosystem takes us to that point; the music industry does not want an artist's tracks to disappear into the background. What we are seeing is that the compression that radio stations have always undertaken of music now occurs upstream at the mastering stage because of how music is played, a DR5 song is simply more likely to be heard in a streaming list produced by a given streaming service. I would love the new normal, not to be MQA or whatever fashionable format it happens to be, rather it would be producing two versions of every album a low DR mass consumption version and a high DR critical listening version. Interestingly I find myself seeking out late 1980's music or classical music with higher DR for my critical listening moments, and often prefer the original versions to re-masters with higher DR numbers. I would even pay for the optimised version; ideally of course as a FLAC at 96khz preferably.

    ReplyDelete
    Replies
    1. Well said and succinct Nich,
      Yeah, compression is not ultimately an evil nor should be banned. It is useful and I find myself listening to music with strong dynamic compression in my car for example and find tracks with strong dynamics (like some live recordings or Broadway musical scores) to be difficult to hear in the soft parts due to high ambient noise level.

      The problem of course is when it's simply too much - for me that's typically <DR6. Sure would be nice to see the music industry "reverse course" somewhat and maybe look at something like DR9-10 as a reasonable compromise for pop/rock/electronic.

      Ideally, I believe having 2 versions, one of which more dynamic to serve those who listen in good environments would be the best way to go.

      I would also love to see the return of the "Loudness" button or function so folks can choose when they want compression added. I wondered if calling this function "Mobile Boost" or something like that might be more descriptive as I suggested here:
      http://archimago.blogspot.com/2016/08/musings-convenience-lossy-audio.html


      Delete
  8. Just had to laugh to myself, just checked what I was listening SACD flac rip of London Symphony Orchestra : Symphony No. 11 (Rostropovich, Lso) (2004)... a very humble DR21

    ReplyDelete
    Replies
    1. Wow. Intense... Will have to keep an eye out for that album!

      Delete
  9. Cool you are revisiting the loudness war! :) Reminds me of when Metallic released "death magnetic". Interestingly, at the time, the same release for the game "guitar hero" was not as compressed, a nice write-up here: http://recordinghacks.com/2008/12/20/metallica-wins-the-loudness-wars/.

    It seems that this is still a problem in 2020, the end listener should be in control of the volume and DR, luckily a tiny bit of hope comes from streaming services. For example, Spotify allows you to normalize the volume, and then also pick what kind of environment you are listening in.
    Spotify info for artists: https://artists.spotify.com/faq/mastering-and-loudness#what-is-loudness-normalization-and-why-is-it-used
    Spotify info about volume normalization for users: https://artists.spotify.com/faq/mastering-and-loudness#can-users-adjust-the-levels-of-my-music

    I will now go and try out the harmonic distortion samples, thanks!!

    ReplyDelete
    Replies
    1. Hi Steven,

      I think it's inevitable that we need to come back to the Loudness Wars every once awhile. For all the stuff that audiophiles talk about and battle over, without doubt in my mind, unless we take a stand on the poor sound quality of many modern albums, it's really all forgettable hot air!

      If audiophiles are to have a hand in changing something about audio reproduction and make it better, I think it's being vocal about this...

      Delete
  10. A casual quick first listen surprised me: nothing here sounds highly distorted. So it means some serious listening will be required to hear the differences, if I can hear them at all.
    Hopefully I'll get to it.

    ReplyDelete
    Replies
    1. Correct, none of the tracks are going to sound seriously distorted like... ohhh... DR1 music. That would be a slam-dunk! :-)

      However, the amount of distortion added is very significant when we start thinking about objective performance even with rather run-of-the-mill audio gear these days!

      Delete
  11. Very interesting test. My computer recently died so I'm not sure I'll be able to take part in the survey, but I am very interested in the results. As I look for a smaller solution to possibly replace my current receiver, I have noticed how manufacturers are cheating on the listed power output. One example is the ELAC EA101EQ-G integrated amplifier. I'm very intrigued by this amp as it includes DSP for blending a subwoofer and room correction. But I don't like the way they specify the power output.

    Power output Continuous:
    2 x 80 Watts 2 channels driven into 4 ohms @ 1kHz
    2 x 40 Watts 2 channels driven into 8 ohms @ 1kHz
    2 x 70 Watts 2 channels driven into 2 ohms @ 1kHz

    Power and THD+N, 1kHz:
    4 ohms 70 Watts @ .07%
    8 ohms 40 Watts @ .03%

    I don't know too many people who listen to 1kHz tones through their system. It's just a way for the manufacturer to inflate their specs. But maybe the power specifications as they relate to frequency and THD aren't as critical as I have been led to believe? Maybe the power that the ELAC can provide at a higher THD level would be perfectly fine. so much to learn.

    ReplyDelete
    Replies
    1. Hey Joe,
      Hope you get the computer fixed soon... Ya still have 3 months for the test ;-).

      Now as for power and amplifier, indeed it's hard to wrap one's head around what is advertised without more details and "standard" levels of expressing the ratings. Can't speak to that specific ELAC integrated amp, but I do like what Hypex has done with their datasheets for the nCore amps and the detailed disclosure of data! If every manufacturer could release results like that, it would really be useful I think!

      Delete
  12. Arch, I'm looking forward to another test session. Whilst researching how to put together an excellent system demo CD I came across this: https://harmanhowtolisten.blogspot.com/ which ties in nicely to your listening test and I thought (if you haven't mentioned it before) your readers might enjoy it too. First practice test in and I'm at only 70%...

    ReplyDelete
    Replies
    1. Thanks Giraffe,
      Missed that one all these years! I see it was posted back in 2011. Will have to give it a try ;-).

      Delete
  13. Why don't you randomize more the samples ?

    Just forgive my ignorance, I don't fully understand why the suffix letters imply the same distorsión for all the selected tracks. If I could perceive in one of them differences between two, I'd just be seriously influenced when listening the other ones. Anyway thanks a lot for this test !!! I'll take my time.

    ReplyDelete
    Replies
    1. Hi Gedeon,
      The reason I kept the same amount of distortion linked to the same suffix is because I do not want to stress listeners out too much. Imagine total randomization of 4 songs with 4 separate distortion levels especially when listeners are "forced" to give me a choice when the music is not a genre they enjoy listening to ;-).

      This way, I encourage more deliberate listening sessions I think with you the listener/tester having freedom to choose which tracks work best for you and correlating that with other tracks.

      Sure, this could influence opinions between tracks and decrease the opportunity to get a few more comparison results per person although I think it provides for more depth to the listening and what questions I can ask on the survey... All part of the balance one tries to strike with testing procedure!

      Delete
  14. It has sense. Thanks for the answer.

    ReplyDelete
  15. "imitations of the time like early ADCs used" -- I think this is perhaps too critical. I'm not sure your or I could tell the difference, blind, between those early ADCs and later ones, or if such difference would really be the cause of what you hear. If sound of those 80s CDs is subpar I'd put my bet at least equally on 1) use of a source other than original master tapes in good shape, or 2) poor transfer practice (tape playback needs proper head alignment, speed, Dolby calibration), or 3) questionable re-equalization/noise reduction choices during mastering for CD.

    ReplyDelete
  16. "Don't be pulling out your audio editor to peek at the files before you actually listen and submit your results! If you do pull out that audio editor, perhaps refrain from posting pictures of FFTs and the like, and avoid discussions of results so you don't influence others... That would be much appreciated!"

    Why not just ask people to use the ABX Comparator in foobar2000? https://www.foobar2000.org/components/view/foo_abx

    This would not only solve the problem of people 'cheating' by looking at the waveform, but if you set the threshold of passing the test as identifying 13 out of 16 trials, this will be solid statistical proof that the listener could actually differentiate between the samples, rather than just having a lucky guess. See here for details: https://wiki.hydrogenaud.io/index.php?title=ABX

    ReplyDelete
    Replies
    1. Hey b,
      One reason is that not everyone uses foobar and for folks with main systems not connected to a general computer, this would be rather cumbersome and limiting. I've always been happy when people submit their ABX log!

      Delete
    2. Then I suggest you add an edit to the main article asking people who have a computer to also submit Foobar ABX logs. Then you will have both a large sample of responses for the informal blind test, as well as true ABX tests that will have a higher degree of scientific accuracy, which could help corroborate the informal test findings.

      Delete
  17. So far I cannot hear a difference in these recordings, but I got a new pair of headphones today, the AKG K275s, and I was comparing them to the old MDR-V6 from Sony, and I noticed that with the Horse song from Jennifer Warnes in the test files that each of them have an obvious hiss or static sound (tape hiss?) in the background, whereas with the AKG headphones I did not notice it. It's barely there when I listen for it but I wouldn't have noticed it without the old, cheaper Sonys. The AKGs are smoothing over the sound instead of revealing it.

    ReplyDelete
    Replies
    1. Perhaps smoothed over is the wrong way to put it. The frequency response is different so some sounds are louder than others. the AKG might be a more neutral response that's not highlighting the tape hiss area as sharply.

      Delete
    2. Thanks for note.

      Yeah, different headphones will definitely give you significantly different results. The better ones should allow you to listen deeper and hopefully also identify the presence of distortions depending on the transducer's resolution.

      Make sure to submit your overall impressions!

      Delete
  18. yes foobar wit ABX plugin seems the only tool for ABX comparison.
    There was once an Mac software but I cant find it anymore. Sombebody knows about it.

    Anyway in foobar-ABX you can compare at once only two files.
    So for the 4 files you can do as follows.

    Compare:
    A-B, A-C, A-D
    Then B-C, B-D
    Then C-D

    With this each files against each other is heared.

    And yes its challenging to rate :)

    ReplyDelete
    Replies
    1. Thanks for the note Dipolaudio.
      For folks doing ABX listening, please feel free to copy and paste the results into the response box - the last item on the responses!

      Delete
  19. @Archimago said, "Thankfully the professionals in the video world of Ultra High Definition actually understand the importance of High Dynamic Range contrasts and larger color gamut so that images "pop" with extended brightness, are capable of fantastic deep blacks, and have smooth gradations (rather than just pixel resolution). Unfortunately, it seems the audio intelligentsia missed the memo on what's important."

    Yes. At some point sooner or later, the next paradigm shift of audio dynamic range and linearity may appear ("paradigm shift" as defined by a new technology causing 10X or greater performance improvement). We've seen three prior audio dynamic range paradigm shifts:

    1925: Acoustic to electric recording (+25dB)
    1950: Commercialization of mag tape (+25dB)
    1980: AD/DA recording (+25dB)
    XXXX: Long overdue

    The video analogy is spot on. The video world calls it HDR. The audio world is ready for HDR-A, with ADC and DAC performance leaping into the 27-bit domain. A critical clue is found in Arri's Alexa "dual gain lens architecture."

    https://www.videomaker.com/how-to/technology/what-is-hdr-video/

    ReplyDelete
    Replies
    1. Thanks for the historical background John,
      With hi-res recording and playback since the 90's and certainly with the "1st coming" of 24-bit back in late 90's and early 2000's with SACD and DVD-A, studios readily have better than 140dB range to play with.

      Tragically, at the same time since the mid-90's into the 2000's, we've retreated back in terms of audio dynamic range! While video has moved ahead and I can't imagine video dynamic range ever going backwards!

      "Mad (Audio) World".

      Delete
    2. It's an interesting conversation: dynamic range, linearity, and noise.

      When you look at combined signal path performance (mic > pre > adc > daw > dac > amp), today's high quality numbers (rarely achieved) are around:

      DR: 120dB or 20 bits
      Linearity: same
      Noise Floor: 10uVrms (broadband / unweighted)

      If we combine the "best of best" in each path element --- DPA or Lewitt Subzero or ribbon mics, Salzbrenner TrueMatch ADC, massively paralleled ESS DAC ICs, etc --- we can achieve slightly better broadband systemic performance:

      DR: 126dB or 21 bits
      Linearity: same
      Noise Floor: 4uVrms BB/UW

      Given recent work in multi-path audio, I think there's a good chance we're heading for the next "quantum" improvement (10X or better) in systemic DR, linearity, and noise floor. I would not be surprised to see 26-bit systemic performance (DR and linearity), via emerging multi-path topologies, in common use this decade.

      https://patents.justia.com/patent/9654134

      Delete
  20. does anybody know what happened to foobar2000 ABX Comparator foo_ABX component?
    Its not anymore on foobar's website

    ReplyDelete
  21. This article " this article in Audioholics published recently " is a bunch of crap.

    We're talking about listening to a hifi system either in the safest of environments imaginable: Our home, or alternatively at the hifi store, where we ought to feel just as safe. The mechanisms mentioned in the article, such a the brain filter is hardly active in such environments, except if we were to be presented with the "worlds best hifi system" inside a cigar box for 20 $.

    And the placebo effect... lol.

    Our ears don't lie to us. If we hear a change, it's there. If "scientific" methods can't explain why, the fault is not with our ears, but with the "scientific" methods.

    ReplyDelete
    Replies
    1. "If we hear a change, it's there"

      More than once have I reached for a 1dB boost at 12k, satisfied with the small change I heard ..... only later to realize that my EQ channel strip was OFF.

      "If "scientific" methods can't explain why, the fault is not with our ears, but with the "scientific" methods."

      The scientific method (in this case, usually large-population fast-ABX DBT) is in fact the only valid method for detecting change with a high degree of certainty. To think otherwise is religious faith. We've used fast-switched, blinded-AB successfully for 30 years of analog and digital R/D.

      But if you think you hear a difference in the stereo store, knock yourself out. Spend that extra $10k. We LOVE you.

      Delete
  22. This comment has been removed by the author.

    ReplyDelete
  23. I too have been working for some years on the study of harmonic distortions and the effects on perception. Here are a couple of posts about my studies that I think you might find useful:

    

1. Injecting harmonic distortions into a digital signal is not equivalent to the same distortions injected downstream from an analog amplifier. This post explains why:



    https://www.audiosciencereview.com/forum/index.php?threads/study-of-the-effects-of-nonlinear-distortion-on-the-perceived-sound-quality.24633/page-3#post-928830

    

2. The THD or IMD values, calculated with the classical methods, are not representative of the real entity of the non-linear distortions injected by our devices, even in the simplest cases. Here you will find the reasons:


    
https://www.audiosciencereview.com/forum/index.php?threads/thd-measurement-does-it-correctly-represent-the-actual-nonlinear-distortion.37561/

    ReplyDelete