INTERNET BLIND TEST: MQA Core Decoding vs. Standard Hi-Res Audio

Introduction:

Well my friends, the time has come... Yes, it's another Internet Blind Test!

As a "more objective" hobbyist blog, within these pages I try to demonstrate facts, figures, and opinions as best I can with the hopes that it educates the typical "audiophile" out there who loves music and wants to at the same time understand the hardware and technologies used in the world of high-fidelity. I do this for fun with the hopes that in time, as a group we can be "more rational", each of us better able to adjudicate what makes sense, what works, and what ultimately has either very little worth or should even be considered worthless "voodoo".

Over the last few years, as you've no doubt noticed, a number of my posts have been looking at MQA and the claims made. I'm not going to rehash much of that here (feel free to start on this page and check out the links at the bottom accumulated over time). As we've come closer to understanding details like the filters used and how the "Rendering" works, there is one very important piece that remains rather nebulous.

This last piece has to do with claims of time domain "de-blurring". The idea that some kind of DSP has been used to affect the sound quality, ultimately "improving the analogue-to-analogue performance" from the studio to one's own DAC output (ostensibly using various techniques including measuring and aligning impulse responses of the devices used in production and playback). How this works is of course proprietary and hidden in the encoding system which we as end-users have no access to.

While we might not know the mechanics of the system, we can at least try to judge the resulting sound. I have done my own listening and testing over many months and my opinions have been different from individuals like John Atkinson of Stereophile ("Whatever the provenance, a consistent factor in my auditioning of the decoded MQA files was a sense of ease to the sound."), John Darko of DigitalAudioReview (speaking of Ella and Louis: "The MQA version delivers fuller, more tonally satisfying bass notes and a better sense of the space surrounding Fitzgerald's voice."), JV Serinus also of Stereophile (speaking of Ray Charles & Natalie Cole's "Fever" - "the MQA version conveyed a more believably large soundstage"), Robert Harley of TAS ("MQA's dramatic superiority made the original high-resolution file sound like a pale imitation of the performance"), Chris Connaker of The Computer Audiophile (doing a blinded ABX - "I did this several times and immediately selected the correct MQA or non-MQA version of the track every time").

Most recently (August 2017 Stereophile, reviewing Brinkmann's Nyquist DAC) I read that mister analogue himself Michael Fremer endorses MQA - "Had this been CD sound in 1983, I'd still be an LP guy - but I'd also be all in with digital." Wow... Really? Consider that later on in the article he used the analogy of "Grand Canyon of analog-vs-digital" to describe the sonic divide to describe the difference; did MQA make that much difference!?

Depending on the quote, the writers above might be referring to specific tracks or portions of tracks  so I don't want to take anything grossly out of context. However, as you can see, they're often even referring to old analogue recordings processed through MQA. There seems to be almost unanimity among the mainstream audiophile press in recommending MQA; that this encoding system is noticeable and generally worthwhile for sound quality. Looking around elsewhere, instead of unanimity, we tend to see a mixed response from listeners on audio forums and even manufacturers like Linn, Benchmark, or Schiit for various reasons including questioning the sound quality.

Ultimately, like any new product, after the manufacturer has dispensed with the advertising promises and reviewers have said their piece, it is about whether the consumer is captivated by the value - what sounds good to you.  And this is exactly what I hope to investigate over the summer with this little test. In the process, I hope it also provides an opportunity for you to listen for yourself, think about the magnitude of differences, and ultimately perhaps formulate your own ideas about whether it's all worthwhile...

Purpose:

Here is the question I'm trying to answer:

Does the MQA stream decoded with MQA Core to 24/88 and 24/96, then "rendered" with a digital filter typical of an MQA DAC result in output preferred by audiophiles?

As you can imagine, in order to answer this question, I need to start with music that is available in both MQA and the original high-resolution to compare. As I have done on previous occasions, I will be using files from the 2L "free test bench" to perform this test. We are very fortunate to have such high quality samples available. These are true high-resolution, wide dynamic range recordings unlike the MQA versions of many old recordings available on TIDAL previously examined. Furthermore, given that these demos were released as showcases for MQA (in early 2016), it should mean that a good effort was made to improve the sound. The comment about "de-bluring" was for example referred to on the download page in fact.

Without getting into the specific details (which I will reveal after the testing is done), let's just say that I have selected 3 songs from the 2L repository, creating two variants of each song - samples A and B. One of these is an actual MQA Core digital decode, and the other is a resample of the original high-resolution file keeping other variables the same. The songs are:

Arnesen: Magnificat 4. Et miseracordia: (~1:53 sample) beautifully recorded classical vocal piece with orchestral accompaniment. Listen for the vocal placement, instrument soundstage depth and width, tonal quality in the voice, etc...

Gjeilo: North Country II: (~2:00 sample) a subtler piece consisting of primarily piano music with some understated accompaniment. Listen for the purity and sense of three-dimensional "realism" of the instruments. Listen to the temporal characteristics such as attack and quality of the decay of the notes.

Mozart: Violin Concerto in D maj (Allegro) [Original 2006]: (~2:00 sample) a lively and beautifully executed orchestral piece highlighting the violin of course. Great tempo, timre of instruments, attack, and "see" if you can delineate the spatial positioning of instruments in the soundstage. I would have loved to test out the 2016 MQA remix but there appears to be something wrong with the file as the MQA version would not decode properly!

I suspect that if there is a significant sonic difference between MQA compared to standard high resolution resulting in clear preferences for many listeners, the samples above should be able to tease out statistical significance in some form.

Pre-requisites:

First, this is what you'll need to get the job done right:

A. A good computer / streamer device to play back the hi-res FLAC files. These files are either 24/192 or 24/176.4kHz in order to minimize non-integer sample rate multiples. They are large files with a total download size of ~350MB for the test package.

B. A good high-resolution DAC capable of playing back the above in a bit-perfect fashion. No need for an MQA-capable DAC of course. Make sure that the DAC is not capped at 88.2/96kHz because these files capture the MQA-like digital filter effect with frequencies >44.1/48kHz in order to provide a reasonable facsimile of what an MQA Render DAC provides plus keeping the comparison "apples-to-apples". Also please make sure to turn off extra processing like room-correction, EQ, crossfeed, spatialization, normalization / ReplayGain.

C. A good system capable of high-resolution playback. Good (pre)amps are a must. Likewise, good speakers and/or headphones are essential!

D. Some free time (I suggest around 30 minutes) to run the test & great ears. :-)

Procedure:

If you have the above pri-requisites and are ready to give this a go, here's what you do:

1. Download the ZIP test package from one of the following:

Download from filehosting.org
Download from Amazon Drive
Download from Private Bits (thanks Ingemar!)

2. Play the music and compare A and B samples. Remember to make sure playback is bit-perfect - use ASIO or WASAPI on Windows. Feel free to use foobar2000 and the ABX Comparator.

If you need to, on the Mac make sure you have 24/176.4 or 24/192 in Audio MIDI set properly depending on the music if your player doesn't switch automatically (since these are FLAC files, I would advise using software like Decibel or Audirvana which will handle the samplerate changes).

While listening to each A/B pair, ask yourself:
     What differences do I hear?
     Do I have a preference?
     Am I enjoying more one of the samples over the other?
     How much of a difference am I detecting?

Treat each A/B pair independently. I have randomized A & B (literally by flipping a coin 3 times) so that while sample B may be the MQA decode in one of the tracks, this does not mean sample B is the MQA decode for the others.

Take your time - maybe set aside 30 minutes at least so there's no pressure. It obviously does not need to be done in one sitting. Grab a pen, write notes, maybe enjoy a beverage. The notes will be useful when it's time to complete the survey. Feel free to try it with your significant other or an audiophile friend!

If you're wondering, the songs are volume matched down to 0.01dB so there should be no bias due to amplitude differences:

3. Fill out the anonymous survey and tell me what you heard!

https://freeonlinesurveys.com/s/tkKU7UPY

Data collected will include:

• Demographics: gender, age range, which continent you're from, musical experience
• For each piece of music, which one sounded better / more natural / more realistic?
• Tell me how confident you are about your choice.
• Describe your audio system - estimated price (this does NOT need to be some $$$$ system, but I do want to see an estimate!), headphones and/or speakers.

For the sake of consistency and obtaining a good data set, I will insist that most of the questions must be answered. Give yourself maybe 10 minutes to fill in the survey as completely as possible. Thanks!

4. Let's make this a seasonal project. The survey will close on Friday September 8, 2017. In time for back-to-school after the summer holidays.

Remember this is a BLIND TEST. Use your ears :-). I have done what I could to anonymize the files so I would certainly appreciate it if testers do not open up the files with an audio editor at least until AFTER you've done the test and submitted your results. Do not base your decisions on things like file creation date, modification time, file size, tag information, etc... Don't think I haven't looked at these! Also, it's best not to share answers on message boards so as not to influence others.

Finally, just to be clear, I have no affiliations with any audio/hi-fi companies, have no commercial interests in the audio world, nor do I have any connections to the "mainstream" audiophile press. At most, over the years I've shared E-mails with those in the Industry as a matter of discussion around what I've written. I do receive a small amount of ad revenue (enough to buy a few CDs here are there) from this blog through AdSense and Amazon but unassociated with MQA. The music files used are available freely for testing purposes with only a portion of the tracks used in the blind test. The mechanism whereby I obtained the digitally decoded MQA stream will be described in detail later, but I can say that it was accomplished with free software; something that any of the thousands of TIDAL or Audirvana+ users can do for themselves already during the trial period. All I've done here is formalized the testing procedure and data collection. As such, I believe what I'm doing here is covered under fair use for the purpose of research with results potentially beneficial for public interest (at least among audio enthusiasts).

Let me know if you run into any troubles with the test procedure.

Have fun with these well recorded demo tracks! Like in the past, it's great to receive submissions from around the world. Feel free to share the test widely.

Thanks to my "beta testers" who have provided great feedback over the last week :-).

Cheers...

Addendum (July 17, 2017):
Every once awhile I'll just add to the addendum here when I see things interesting related to this test.

Feel free to follow the discussion on Steve Hoffman forum around this test. You'll see that forum member Testikoff posted his ABX (without revealing preference) and digital subtraction delta results on these files. Of course, try to do the blind test for yourself and submit survey results before delving deep into the objective measurements, thanks!

One thing I do encourage, if you use the foobar ABX Comparator, by all means include your log file in the survey results (you can just cut & paste to the "comments" field as appropriate) similar to what Testikoff posted. That would be another useful level of analysis.

Remember folks, do not be worried if you feel you have "cloth ears" rather than "Golden Ears" because you find it difficult. It's good to have the survey response from everyone with a reasonable audio set-up whether one is confident or not. It's also good to just get your intuitive subjective input even if consciously you can't be sure. Judgments like "positive" / "negative", "right" / "wrong" are irrelevant.

To capture as many subjective opinions as possible will provide us with statistical power in terms of drawing inferences and conclusions more potent than any comment from single audiophile writer, or studio engineer, or manufacturer claims IMO. I believe the results of a naturalistic survey like this is capable of shining light on "real life" value.

Addendum (September 11, 2017):
Thanks everyone for submitting results. In total I received 83 survey results. IMO that's excellent given the demands of the test! As usual, I'll be releasing details of the procedure, results, analysis and discussion over the next few weeks. Stay tuned!