Introduction:Well my friends, the time has come... Yes, it's another Internet Blind Test!
As a "more objective" hobbyist blog, within these pages I try to demonstrate facts, figures, and opinions as best I can with the hopes that it educates the typical "audiophile" out there who loves music and wants to at the same time understand the hardware and technologies used in the world of high-fidelity. I do this for fun with the hopes that in time, as a group we can be "more rational", each of us better able to adjudicate what makes sense, what works, and what ultimately has either very little worth or should even be considered worthless "voodoo".
Over the last few years, as you've no doubt noticed, a number of my posts have been looking at MQA and the claims made. I'm not going to rehash much of that here (feel free to start on this page and check out the links at the bottom accumulated over time). As we've come closer to understanding details like the filters used and how the "Rendering" works, there is one very important piece that remains rather nebulous.
This last piece has to do with claims of time domain "de-blurring". The idea that some kind of DSP has been used to affect the sound quality, ultimately "improving the analogue-to-analogue performance" from the studio to one's own DAC output (ostensibly using various techniques including measuring and aligning impulse responses of the devices used in production and playback). How this works is of course proprietary and hidden in the encoding system which we as end-users have no access to.
While we might not know the mechanics of the system, we can at least try to judge the resulting sound. I have done my own listening and testing over many months and my opinions have been different from individuals like John Atkinson of Stereophile ("Whatever the provenance, a consistent factor in my auditioning of the decoded MQA files was a sense of ease to the sound."), John Darko of DigitalAudioReview (speaking of Ella and Louis: "The MQA version delivers fuller, more tonally satisfying bass notes and a better sense of the space surrounding Fitzgerald's voice."), JV Serinus also of Stereophile (speaking of Ray Charles & Natalie Cole's "Fever" - "the MQA version conveyed a more believably large soundstage"), Robert Harley of TAS ("MQA's dramatic superiority made the original high-resolution file sound like a pale imitation of the performance"), Chris Connaker of The Computer Audiophile (doing a blinded ABX - "I did this several times and immediately selected the correct MQA or non-MQA version of the track every time").
Most recently (August 2017 Stereophile, reviewing Brinkmann's Nyquist DAC) I read that mister analogue himself Michael Fremer endorses MQA - "Had this been CD sound in 1983, I'd still be an LP guy - but I'd also be all in with digital." Wow... Really? Consider that later on in the article he used the analogy of "Grand Canyon of analog-vs-digital" to describe the sonic divide to describe the difference; did MQA make that much difference!?
Depending on the quote, the writers above might be referring to specific tracks or portions of tracks so I don't want to take anything grossly out of context. However, as you can see, they're often even referring to old analogue recordings processed through MQA. There seems to be almost unanimity among the mainstream audiophile press in recommending MQA; that this encoding system is noticeable and generally worthwhile for sound quality. Looking around elsewhere, instead of unanimity, we tend to see a mixed response from listeners on audio forums and even manufacturers like Linn, Benchmark, or Schiit for various reasons including questioning the sound quality.
Ultimately, like any new product, after the manufacturer has dispensed with the advertising promises and reviewers have said their piece, it is about whether the consumer is captivated by the value - what sounds good to you. And this is exactly what I hope to investigate over the summer with this little test. In the process, I hope it also provides an opportunity for you to listen for yourself, think about the magnitude of differences, and ultimately perhaps formulate your own ideas about whether it's all worthwhile...
Purpose:Here is the question I'm trying to answer:
Does the MQA stream decoded with MQA Core to 24/88 and 24/96, then "rendered" with a digital filter typical of an MQA DAC result in output preferred by audiophiles?
As you can imagine, in order to answer this question, I need to start with music that is available in both MQA and the original high-resolution to compare. As I have done on previous occasions, I will be using files from the 2L "free test bench" to perform this test. We are very fortunate to have such high quality samples available. These are true high-resolution, wide dynamic range recordings unlike the MQA versions of many old recordings available on TIDAL previously examined. Furthermore, given that these demos were released as showcases for MQA (in early 2016), it should mean that a good effort was made to improve the sound. The comment about "de-bluring" was for example referred to on the download page in fact.
Without getting into the specific details (which I will reveal after the testing is done), let's just say that I have selected 3 songs from the 2L repository, creating two variants of each song - samples A and B. One of these is an actual MQA Core digital decode, and the other is a resample of the original high-resolution file keeping other variables the same. The songs are:
Arnesen: Magnificat 4. Et miseracordia: (~1:53 sample) beautifully recorded classical vocal piece with orchestral accompaniment. Listen for the vocal placement, instrument soundstage depth and width, tonal quality in the voice, etc...
Gjeilo: North Country II: (~2:00 sample) a subtler piece consisting of primarily piano music with some understated accompaniment. Listen for the purity and sense of three-dimensional "realism" of the instruments. Listen to the temporal characteristics such as attack and quality of the decay of the notes.
Mozart: Violin Concerto in D maj (Allegro) [Original 2006]: (~2:00 sample) a lively and beautifully executed orchestral piece highlighting the violin of course. Great tempo, timre of instruments, attack, and "see" if you can delineate the spatial positioning of instruments in the soundstage. I would have loved to test out the 2016 MQA remix but there appears to be something wrong with the file as the MQA version would not decode properly!I suspect that if there is a significant sonic difference between MQA compared to standard high resolution resulting in clear preferences for many listeners, the samples above should be able to tease out statistical significance in some form.
Pre-requisites:First, this is what you'll need to get the job done right:
A. A good computer / streamer device to play back the hi-res FLAC files. These files are either 24/192 or 24/176.4kHz in order to minimize non-integer sample rate multiples. They are large files with a total download size of ~350MB for the test package.
B. A good high-resolution DAC capable of playing back the above in a bit-perfect fashion. No need for an MQA-capable DAC of course. Make sure that the DAC is not capped at 88.2/96kHz because these files capture the MQA-like digital filter effect with frequencies >44.1/48kHz in order to provide a reasonable facsimile of what an MQA Render DAC provides plus keeping the comparison "apples-to-apples". Also please make sure to turn off extra processing like room-correction, EQ, crossfeed, spatialization, normalization / ReplayGain.
C. A good system capable of high-resolution playback. Good (pre)amps are a must. Likewise, good speakers and/or headphones are essential!
D. Some free time (I suggest around 30 minutes) to run the test & great ears. :-)
Procedure:If you have the above pri-requisites and are ready to give this a go, here's what you do:
1. Download the ZIP test package from one of the following:
Download from filehosting.org
Download from Amazon Drive
Download from Private Bits (thanks Ingemar!)
2. Play the music and compare A and B samples. Remember to make sure playback is bit-perfect - use ASIO or WASAPI on Windows. Feel free to use foobar2000 and the ABX Comparator.
If you need to, on the Mac make sure you have 24/176.4 or 24/192 in Audio MIDI set properly depending on the music if your player doesn't switch automatically (since these are FLAC files, I would advise using software like Decibel or Audirvana which will handle the samplerate changes).
While listening to each A/B pair, ask yourself:
What differences do I hear?
Do I have a preference?
Am I enjoying more one of the samples over the other?
How much of a difference am I detecting?
Treat each A/B pair independently. I have randomized A & B (literally by flipping a coin 3 times) so that while sample B may be the MQA decode in one of the tracks, this does not mean sample B is the MQA decode for the others.
If you're wondering, the songs are volume matched down to 0.01dB so there should be no bias due to amplitude differences:
3. Fill out the anonymous survey and tell me what you heard!
Data collected will include:
- Demographics: gender, age range, which continent you're from, musical experience
- For each piece of music, which one sounded better / more natural / more realistic?
- Tell me how confident you are about your choice.
- Describe your audio system - estimated price (this does NOT need to be some $$$$ system, but I do want to see an estimate!), headphones and/or speakers.
For the sake of consistency and obtaining a good data set, I will insist that most of the questions must be answered. Give yourself maybe 10 minutes to fill in the survey as completely as possible. Thanks!
4. Let's make this a seasonal project. The survey will close on Friday September 8, 2017. In time for back-to-school after the summer holidays.
Remember this is a BLIND TEST. Use your ears :-). I have done what I could to anonymize the files so I would certainly appreciate it if testers do not open up the files with an audio editor at least until AFTER you've done the test and submitted your results. Do not base your decisions on things like file creation date, modification time, file size, tag information, etc... Don't think I haven't looked at these! Also, it's best not to share answers on message boards so as not to influence others.
Finally, just to be clear, I have no affiliations with any audio/hi-fi companies, have no commercial interests in the audio world, nor do I have any connections to the "mainstream" audiophile press. At most, over the years I've shared E-mails with those in the Industry as a matter of discussion around what I've written. I do receive a small amount of ad revenue (enough to buy a few CDs here are there) from this blog through AdSense and Amazon but unassociated with MQA. The music files used are available freely for testing purposes with only a portion of the tracks used in the blind test. The mechanism whereby I obtained the digitally decoded MQA stream will be described in detail later, but I can say that it was accomplished with free software; something that any of the thousands of TIDAL or Audirvana+ users can do for themselves already during the trial period. All I've done here is formalized the testing procedure and data collection. As such, I believe what I'm doing here is covered under fair use for the purpose of research with results potentially beneficial for public interest (at least among audio enthusiasts).
Let me know if you run into any troubles with the test procedure.
Have fun with these well recorded demo tracks! Like in the past, it's great to receive submissions from around the world. Feel free to share the test widely.
Thanks to my "beta testers" who have provided great feedback over the last week :-).
Addendum (July 17, 2017):
Every once awhile I'll just add to the addendum here when I see things interesting related to this test.
Feel free to follow the discussion on Steve Hoffman forum around this test. You'll see that forum member Testikoff posted his ABX (without revealing preference) and digital subtraction delta results on these files. Of course, try to do the blind test for yourself and submit survey results before delving deep into the objective measurements, thanks!
One thing I do encourage, if you use the foobar ABX Comparator, by all means include your log file in the survey results (you can just cut & paste to the "comments" field as appropriate) similar to what Testikoff posted. That would be another useful level of analysis.
Remember folks, do not be worried if you feel you have "cloth ears" rather than "Golden Ears" because you find it difficult. It's good to have the survey response from everyone with a reasonable audio set-up whether one is confident or not. It's also good to just get your intuitive subjective input even if consciously you can't be sure. Judgments like "positive" / "negative", "right" / "wrong" are irrelevant.
To capture as many subjective opinions as possible will provide us with statistical power in terms of drawing inferences and conclusions more potent than any comment from single audiophile writer, or studio engineer, or manufacturer claims IMO. I believe the results of a naturalistic survey like this is capable of shining light on "real life" value.
Addendum (September 11, 2017):
Thanks everyone for submitting results. In total I received 83 survey results. IMO that's excellent given the demands of the test! As usual, I'll be releasing details of the procedure, results, analysis and discussion over the next few weeks. Stay tuned!
Download in process, I'll give an honest evaluation playing on a quality hi-res system.ReplyDelete
Looking forward to the thoughts & experiences, guys & gals. I'm going to keep myself "blinded" as well without looking at results other than making sure the system is working OK. (I see a few submission coming in already... Nice.)
You have a problem with download links:ReplyDelete
"Download from filehosting.org
Download from Amazon Drive
Download from Private Bits (thanks Ingemar!)"
But all point to the last one "Private Bits" URL only...
All fixed... Must have got messed up with slight edits there...
Wow, wonderfull test setup! Will take my time to check it out!ReplyDelete
Tip: don't use Roon to play them back, it will disclose the formats...Delete
Haven't tried this but will suggest folks stay with the other playback software than Roon then...
Lionel, could you do me a favour and put in a survey entry identifying what Roon is disclosing? Make sure to identify it's you in the "Comments" at the bottom and I'll have a look.
At worst, if this compromises the file identities, might have to scrap the blind data collection but at least folks will have the test tracks for themselves to compared.
@Lionel about MQA identification in RoonDelete
Could you provide a screen shot, showing that Roon does identify part of these files as MQA, as I do not see this with these files.
With MQA files you are right. Roon does show me information like this: “FLAC 44.1kHz 24 Bit, MQA 352.8kHz”, but I do not see this in one of the above given files.
With Tidal Master, I do see in Roon the MQA information only when clicking on the “signal path” LED, showing me the “quality” of the stream, ...
… but with offline (stored) tracks, I see this additionally in the analyzed metadata, as Roon reads the MQA tags as does also analyze the audio content to find any possible mismatch between MQA tags and the MQA stream.
@Lionel about MQA identification in Audirvana or Mytek BrooklynDelete
Nor does Audirvana Plus identify one of the above files as MQA (as this software does detect MQA in Tidal streams as in Offline (stored) files).
Nor does the Mytek Brooklyn does light up any MQA sign with the above files, as the Brooklyn can, beside MQA Blue and Green, also detect MQA violet.
Thanks Juergen for the feedback!Delete
So, looks good then in terms of maintaining anonymity of these files.
I would certainly be impressed if any software has strong enough "AI" to identify one from the other given that the files are only portions of the original, upsampled, and slight fade-in/outs done at the terminal ends.
> No need for an MQA-capable DAC of courseReplyDelete
There you have it. This whole MQA stuff is so confusing that I don't understand anything anymore...it seems...
How can we compare the Hi-Res version with the MQA version without rendering it properly by an MQA capable DAC?
Yes you can. As you can see from previous posts, the MQA rendering component doesn't contain magic. It's just some filter settings and various dithering effects. By using the software MQA decoder to 88/96kHz and then upsampling that with a filter effect similar to what MQA does, we can get the essence of the decoding quality using something like TIDAL and a taste of the temporal characteristics of the filter. This is why I have these files as 176/192kHz.Delete
It might not be 100% of what a Mytek Brooklyn might sound like but it'll be very close in the digital data fed to the DAC; MQA decoded ultrasonics, the sort impulse response, and of course the aliasing inherent with the filter.
I agree with Techland. How on earth are you capable to do a blind test without proper use of a MQA decoder..? This is u meaningless, since there will be no end-to-end situation in which the impuls behaviour of the DAC itself will be included and compensated for as it should be with a proper MQA system..ReplyDelete
Not true Pedro.Delete
Remember, all kinds of people are already listening to TIDAL Master without an MQA-capable DAC. All kinds of people claim the MQA Core Decode sounds better than than lossless FLAC.
Not only am I digitally extracting the data from MQA Core, but in these 176/192 files, I'm upsampling with the impulse reponse settings typically used by these MQA DACs so you're hearing the MQA data go through a minimum phase, low-tap filter which MQA prefers.
Remember, even though MQA seems to position itself as a hardware solution, it's really just software. Manufacturers of DACs hand over their device to the MQA guys to produce a firmware which detects the incoming signal, "Decode" or "Render" the digital stream, and then feeds their data to the DAC. This is what I'm doing here... Grab the MQA software-decoded data, "render" it with a typical kind of MQA filter and send it to your DAC.
You tell me if this sounds any good :-).
I am sorry Archimago, with all respect to your research, a true and certified MQA decoder is certainly needed to reproduce the sound of the studio. Your experiments simply neglect the fact that the DAC itself which is used to play these files intrinsically has a contribution to the sound. True MQA encoding and decoding is an end-to-end algorithm in which the 'fingerprinted' behavior of the DAC which is auditioned is also part of the cumulative timing errors introduced by the ant-aliasing filters in the audio chain. In order to restore the native impulse response, it is evident that the DAC itself needs to be compensated for as well.Delete
All the evidence to date suggests there is no DAC-specific tuning. The only end-to-end thing in MQA is royalty payments.Delete
:-) Thanks Mans for being honestly blunt.Delete
Pedro, your comment is basically a reflection of the claims of the company. One which we have no real evidence of and if one were to tease apart the implications, appears rather unlikely. "De-blur" and "time-domain" claims are nice advertising talking points which perpetuates a type of mystique IMO. (Besides, the "end-to-end" analogue output that's important is not what comes out of the DAC, it's what comes out of your speakers/headphones! For some reason, MQA doesn't want to talk about that inconvenient truth for which they have no "de-blur" ability over... :-)
A much more practical consideration is whether MQA's lossy compression, potential dithering, and noise shaping makes a difference in the ultimate preference for the listener. I suspect MQA likes to de-emphasize these aspects which are tested in this blind comparison.
Remember, MQA have claimed that even an *undecoded* 24/44 or 24/48 MQA file sounds "better" than normal Redbook 16/44 because of some of the benefits of the de-blur process. If this is so, then surely with actual data decoding/decompression and use of an MQA-like minimum phase filter as I have done here would get significantly closer to the "MQA sound". As I said before, MQA is really just about software... And these test files have done the software part for public listening.
BTW Pedro, do you own an MQA-capable DAC?
Hi Archimago, I agree that a 'total' end-to-end should include the headphone characteristisch or ( even much mor complex..) the loudspeaker and room-acoustic characteristics.. But if one would regard those analog factors as 'constant'still a A/B comparison in AIR would be the ideal way of proving de-blurring functionality or not.. This is probably not feasible.. With regard to my personal situation, yes, I have a MQA certified DAC and streamer combo. It is NAD C390DD with BluOS streamer ( unfolding to 24/44.1 or 48,88,96,176 up to 192 max. 'vintage' Watt Puppy 5.0 loudspeakers which are still very time coherent and revealing. I am listening to MQA music since June 2016 and am very positive with what I hear and am able to compare both USB direct and Tidal streams. Most convincing are the remasters of recordings before 1960 and the recent release of The Nightfly is excellent demo material.Delete
?? Not sure what you mean by "comparison in AIR" ??Delete
Interesting that you're using the NAD BluOS system. I know that Mans knows a thing or two about that decoder software :-).
Thanks for the suggestion on the songs. Realize of course that those old 1960's recordings can sound great but not the same as modern hi-res captures. Also, I'm not sure how MQA would be able to de-blur those (remember, they claim to use impulse responses from the microphones and recording chain... How did they do that with recordings >50 years old!?).
Also, remember that The Nightfly was an early digital recording captured on a 3M multi-track digital. This is a 16-bit 50kHz system:
As you can see, since there were no true 16-bit ADCs at the time, it was actually captured with separate 12-bit and 8-bit merged portions.
Hmmm, how MQA can "de-blur" that accurately would be interesting to know! Obviously, The Nightfly is not a true hi-res recording either...
Hi Archimago, this suggested measurement of an A/B comparison between MQA (encoded-decosed) files and the non-MQA file in Air is a proposal to capture the differences in sound signature, especially to capture the difference in impuls response and time-smear. Since I am not-technical I do not know if such measurements using loudspeakers and microphones would be possible..Especially since both the loudspeakers and microphones and room acoustics will have a large influence . But if it might be possible, why not try this? Or are we dependent solely to our ears...? What intrigues me is that MQA claim to be capable to compensate for the CUMULATIVE ringing effects of the audio chain and that it is an end-to-end solution. So where are the limitations? I cannot accept that it is only 'psycho acoustics' what I hear and experience..Is it time-smear related or not? Is this measures let? Who can do such measurements?Delete
The claim that no 16 bit DACs were available in 1978 is false. I worked with a commercial product that relied on a 16 bit DAC that could handle 200,000 samples per second, and had better than 1 LSB linearity in 1973. It was alleged that the combination DAC in the 3M recorder could, with proper adjustment provide appropriate linearity. In theory, such a thing was possible.Delete
@Pedro: I see what you mean now. IMO this is not going to be accurate given the extra analogue noise and loss of resolution going through the air and transduction through the measurement microphone and yet another ADC.Delete
MQA claims to be "end-to-end". But it "ends" at the home user's DAC... As a result it cannot compensate for your speakers or your room which are very significant; just look at the step response with speaker measurements! This has been a clear issue from the start and one which begs for caution in accepting the benefits from the technique...
@Arny Krueger: Thanks Arny. Appreciate the personal experience! I assume you mean "ADC". In any event, achieving 16-bit @ 200kHz I suspect would have cost a "king's ransom" back in the 70's...
"reviewers have said their *piece*"ReplyDelete
Thanks for the fix Hal :-).Delete
Submitted my results a few hours ago. Nice work, Archimago!ReplyDelete
The magnitude of this test reminds me of the challenge to identify properly dithered '16/48000' files vs. the "original" Hi-Res ones. Or, to identify the effects of different high-quality dither + noise shaping applied [e.g., iZotope MBIT+ vs: SoX - available for foobar2000].
PS: BBC is currently offering "Radio 3 Concert Sound" in form of a trial/project. It is a lossless stream working on Firefox only. Found via + link:Delete
Cool. Thanks Daniel for the submission and BBC stream.Delete
Actually it's not only Firefox. I'm listening to it via LMS and Squeezelite and my Odroid XU4Delete
I'll give this a go, I need to read through your previous posts to have a look at your methodology, but I don't think it's correct to describe the MQA filter as being a low tap minimum phase filter.ReplyDelete
I have traveled to Huntingdon to Meridian where I met Bob Stuart and Spencer Chrislu, they tried to respond to your post but couldn't for some reason. They allowed me to compare the original high resolution version to the MQA version on their Ultra DAC in their special listening room. I found the sound to be a huge improvement over the Hi-Res originals for all the tests.
I still haven't responded to your previous response to my comments on your previous article about evidence that pre-ringing has an audible effect and that the temporal distortion degrades the sound. I believe there are AES papers with research on this.
I was going to post links from the bitperfectsound blog.
Great that you had a chance to hear the Ultra DAC. Yeah, let me know what you make of those filters if not minimum phase and of short tap length. Obviously many are not standard sinc filters.
Other than 1 response from Bob Stuart which the Blogger system flagged as spam, I have not seen others from Bob or Spencer Chrislu. As you can see, many other folks are posting so for sure they can try again and I'll make sure to check that it doesn't end up in the Blogger spam bit bucket!
Yes, please post links, I'm curious about the magnitude of the filter effects and would of course love to replicate and maybe even run blind tests like this one if I can...
You MUST read:ReplyDelete
[ Or nomad not yes man is Frederic Vanden Poel, designer of the 432 Evo music server of Belgian brand Klinktbeter. What follows landed in my email inbox...
Now the breaking discovery which has been tested on both the Mytek Brooklyn and Metrum Adagio: when you take any 24/44.1 MQA encoded Flac file from 2L.no and compare it with their 24/352.8 DXD version, the MQA file will sound less good than the DXD file. It will not sound as fluent and lack some stage/space. But once you upsample the MQA file back to 352.8kHz using the sox tool and a minimum phase vsM setting, the resulting upsampled file sounds identical to the DXD file... ]
I read through this article as well, and found it quite hilarous. I also checked an MQA vs PCM track from 2L using an MQA DAC and didn't notice a difference. But that is just my opinion, nothing scientific behind my comments.
I always thought that if MQA have to tell the DAC what to do after the "unfolding process, then all DAC's must be doing it wrong. It was also my opinion that recordings of the quality 2L produce would surely not need MQA assistance.
Seems this blind test is on the right track and it will be interesting to read the results.
Finally, the more I read and hear, it seems that good quality mastering is the key to high quality sound, regardless of the age of the material. Remastering also brings improvements, just listen to the SACD version of Albert King's I'll Play the Blues for you, something recorded in 1970, well by the sounds of it and the remaster is something to behold.
As Archimago's own findings have stated, these days good quality equipment can do the job without having to pay exorbitant amounts of money or use software that adds nothing to the original.
And back to the music
Hey Maty and Gordon,Delete
Yeah, I saw that article briefly this morning and looked into more detail tonight. :-)
SoX vsM eh? Obviously from the screen capture on the post, this filter has much stronger post-ringing than any of the MQA variants. In any event, the article seems to claim that you can take the 24/44 MQA file and just upsample without actual MQA Core decoding will sound like the original DXD. Hmmm, obviously I would not go that far! Whether one could hear the difference or not is in the "ear of the beholder"...
Interesting comment Godon about "It was also my opinion that recordings of the quality 2L produce would surely not need MQA assistance". Who knows if this is the case... A good hi-res recording like the 2L's at least have actual high frequency material for the compression system ("origami") to work with. Furthermore, modern recordings using contemporary gear allows MQA to actually obtain impulse responses and such which supposedly is the basis for the time-domain correction.
How would they "de-blur" say an old 1960's recording from Elvis? What kind of time domain correction can be done with an old standard resolution Dire Straits "Brothers in Arms"? These recordings were done decades ago and even if one were to drag out some old 3M digital recorders or 1960's Shure microphones to measure, who's to say the same corrections can be applied?
Archimago, since you mentioned 'Muddy Waters: Folk Singer' (originally released in 1964!) in some of your older posts; have you had a listen to the remastered by >Analogue Productions< album? This, and their remastered 'Nils Lofgren: Acoustic Live' are amongst my favorite old "new" releases.Delete
Note: 'Acoustic Live' was not recorded on analogue master tape; but originally at 44.1kHz/16-bit!
Yes, I have listened to the AP Muddy Waters but have not heard the remastered Nils Lofgren (thanks for the recommendation).
Yes, can totally agree with the Folk Singer SACD. The remastering effort is whorthwhile and can improve sound (in my case, compared to the old Classic Records HDAD in 24/192).
Proper remastering is a very different thing than just "de-blurring". Taking say an old analogue tape and running it though a modern high resolution ADC, having the leeway to repair imperfections in the old tape with modern digital tools, judicious noise reduction, better sounding EQ applied, can all of course work well and make the remastering a worthwhile effort.
But the MQA "de-blurring" is sold to us as some kind of time domain improvement which implies an ability for their software and DSP processing to "know" what amount and how to repair the anomaly. I can *maybe* be led to believe this can be done if one can measure the recording chain in a simple recording set-up with few microphones and standard ADC (which makes sense for 2L, hence the use of those demo tracks here rather than say a rip of Beyonce off TIDAL with MQA Decode).
As far as we can tell, MQA is not about a full remastering effort. Rather they're obviously batch converting pre-existing "hi-res" audio including stuff clearly NOT hi-res like Madonna, Beyonce, and Bruno Mars through their DSP encoder by the thousands (ie. Warner). As I expressed more than a year back, this "de-blur" DSP can be something sold to studios as part of the processing an audio engineer can choose to use. It's not something we as consumers need to be concerned / burdened / impressed by.
Of course, let's see if the blind test shows us any differently!
Hi Archimago, just want to tell you I can ABX your Gjeilo samples.ReplyDelete
foo_abx 2.0.2 report
File A: 03 - Gjeilo A.flac
File B: 04 - Gjeilo B.flac
WASAPI (push) : Speakers (Creative SB X-Fi), 24-bit
22:46:03 : Test started.
22:47:55 : 01/01
22:48:11 : 02/02
22:48:22 : 03/03
22:48:37 : 04/04
22:48:58 : 05/05
22:49:20 : 06/06
22:49:31 : 07/07
22:50:07 : 08/08
22:50:18 : 09/09
22:50:39 : 10/10
22:50:39 : Test finished.
Probability that you were guessing: 0.1%
-- signature --
It looks like you applied some fades in your samples. One sample faded slightly later than the other. As a result, by hearing the last piano note on both samples alone I can tell them apart. Of course, it has nothing to do with MQA or filtering at all, just want to tell you I can easily ABX them.
Hi Dtmer. Smart :-).Delete
Yes, indeed if you ABX'ed the 1st or last 2 seconds there were slight differences in the fades. Of course, the real question is which of the 2 you *liked*... (Don't answer that here of course. Let me know in the anonymous survey!)
Hi Archimago, I submitted my results yesterday - and - I posted a link to this page on a forum that I frequent quite a lot.ReplyDelete
One of the contributors over there "gizlaroc" doesn't think much of you lol.
Here's a link (relevant posts are at the bottom of the page)
He is clearly an MQA devotee.
Hey there Tony,Delete
Thanks for the link. Nah, I don't expect to be well liked by some in this hobby :-). That's fine, if everyone agreed, there would be no need to debate or try to change anything!
The thing is that intellectually, I think technical folks already suspect something not quite right in the claims. And when the company in its advertising goes so far as insist on very significant differences and certain audiophile reviewers (many of whom have very questionable worldviews about audio) seem to parrot these differences, it's bound to trigger exploration of claims. All the while, on the objective side, we can explore what MQA decode is doing both from the DAC output (like the impulse responses) and in the software (Man's work on the digital Render stream).
When "gizlaroc" says "He is just someone that has already decided it can't sound good/better.", I disagree. I have searched out opportunities to listen to it including at the local audio show but there was no A/B test. I've gone to a local audio store where the salesman was very positive about it and played me some tracks off a Bluesound Node 2 (sounded good but again no evidence that it's "revolutionary"). And of course I've listened for myself on TIDAL, compared with HDtracks songs I have, measured and explored the hardware output from MQA-capable DACs as well as the software digital output from TIDAL and Audirvana+ to explore the differences.
This blind test is a natural extension of that exploration for an opportunity so that others can listen for themselves. I am completely open to a scenario where the MQA Decode is preferred by a majority. No problem and we can have fun investigating why this is!
I can readily admit that the blind test is of course not an *exact* replica of the MQA process. Remember, 2 of the 3 tracks originated in DXD and I've obviously not upsampled them beyond 176/192. I will discuss this in greater detail in the write-up. But based on reasonable understanding of how well humans can perceive a sound, the idea here is to see if MQA Decode itself with a reasonable MQA-like filter facsimile adds anything compared to a direct resample of the original PCM. The test can easily answer that for us and I believe this will cover essentially all we need to know about sound qualitative diffences.
If MQA wants, there is nothing to stop them from releasing DXD PCM files comparing an MQA "deblurred" version and the original hi-res for all to hear. This would easily supersede what I'm doing here with official and more accurate samples. They have simply chosen not to.
As for "gizlaroc":
"I don't know about players I'm afraid, I use my iphone as I only ever listen to dance music with headphones, or I'm in the car where it doesn't matter so much."
Hmmm, doesn't sound like he's in the market for MQA? And obviously unless he's using an external DAC with that iPhone, he would not meet the pre-requisites for this test. Not sure why he's rendering an opinion...
I must confess that the vaste majoirity of my material is 16/44.1, with something in Hi-Res, but at 24/96 maximum.
Gien that I'm unable to play your samples at their full potential.
JRiver MC forces me to downsample to 24/96, even if it uses WASAPI.
Foobar 2000 refuses to play using WASAPI, it plays only with DS, and downsamples too.
It is the fault of the Audiolab 8200CD that accepts a maximum of 24/96 over the USB input, so, if I want to go beyond that limit, is a USB->S/PDIF converter advisable?
Are those devices jitter free? and, generally, what is their sonic character? should I spend a lot of money to get a good sound quality?
Yes, based on articles about the Audiolab 8200CD, it looks like it can accept up to 192kHz through the S/PDIF input (coaxial, but need to double check on the TosLink if it can handle it).
You can get an inexpensive asynchronous USB --> S/PDIF for this. For example years ago, I tested a CM6631A device:
Clearly cleaned up the jitter test spectrum compared to adaptive isochronous USB input.
These days we have USB --> S/PDIF interfaces using the newer CM6632 and XMOS chips which I'm sure will handle jitter extremely well.
"Sonic character"... It's really the sound of your DAC ultimately. They don't add or change the sound unless there's something noisy about the device that may be transmitted through the coaxial. TosLink is galvanically isolated but might have more challenges with higher bitrates and some cannot handle 24/192; you'll have to check your Audiolab. Jitter also tends to be lower with coaxial vs. TosLink but even this is not guaranteed and depends on your DAC.
No, don't spend a lot of money on the device; not for "sound quality" itself. A nice looking device with metal casing will cost more money of course. You should be able to get one for <$100.
Thank you, but ... is all that worthwhile?Delete
As far as I remember you said that going more than 24/96 is a waste of bytes.
More: MQA is a widely accepted standard? AFAIK only with TIDAL such material is available, but the albums in MQA are so few!
On the contrary with Qobuz Sublime+ you have (quite) one album over two in Hi-Res, at 24/82.2, at 24/96, no more than that.
By the way, I'm doing the test.
In order to be able to use WASAPI, I had to downsample to 88.2, 96 and 88.2 (using Foobar 2000 and the Poweramp resampler).
I hope that dividing the sample rate by two I made no "sonic" disasters!
Is this worthwhile?Delete
IMO. For listening to music out there... No. There's just nothing that needs 176/192kHz as I expressed previously:
Having said that, for this MQA test, in order to more closely control the sound of the digital filtering, I did upsample using the kind of filtering representative of MQA which only a DAC capable of the higher samplerate would be able to account...
Hi Archimago :)ReplyDelete
Thank you for this DBT possibility and the work you put into it. I found it very hard to hear differences, whether I listened blind or not blind. Which only shows that a good recording will always sound good, no matter the format (except on vinyl or bad tape).
For that reason, I have something to complain about: I would have preferred normal music, tracks anyone could buy, mainstream music. Music that sounds worse and isn't usually regarded as being "reference" class. Personally, I have often found that very good recordings can obscure format differences, just because they always sound good. Less good recordings or densely mixed recordings can reveal inherent sound problems a media/codec might produce.
Though I will admit that not everyone is used to this approach.
Yeah, I've heard that concern before as well...
Specifically for MQA, the problem is that there really isn't much music out there we can purchase in this encoded format. In fact, other than streaming off TIDAL, 2L is one of the main sites to get the music from.
Of course there's the matter of how to do this as transparently as possible which is why the 2L demos I felt was the most openly available. We have to be careful about copyright and such, but since these 2L tracks are available freely, I think it's fair to utilize these.
From my perspective, although well recorded music should sound good on good systems, if MQA insists that it's a "high resolution" format, then I must also try to find real high-res audio... Otherwise I think I'd get many complaints if I chose a Beyonce track or old remasters!
Just to say, I've had a listen on straight laptop speakers, comparing the piano samples only so far - and it's easy to pick the difference. One is clearly superior, to my ears, and is markedly so on this simple setup.ReplyDelete
I'll continue comparing, and post my findings - thanks for putting this up!
Please remember to put in your survey results!
thanks for this test, but I fail to see the point here.
Most objective folk already established this MQA is nothing more than a marketing hype. And the fanboys, who won't listen to reason, are going to love it regardless of results of this test.
So not sure who can benefit from such endeavour.
BTW, I'd love to take part, but as it happens, I have ran out of 24/192 DACs as they seem to be too much of an overkill even for recording vocals and instruments.
Sure, objective folks might have already passed on MQA, and fanboys already love it. At least that's what is said in public. But I think there is still value in providing an anonymous venue and opportunity for everyone to listen for themselves whether they're streaming TIDAL currently or have a decoding DAC. I'm sure there are many out there who don't know what to think about all this...
- Those who have written off MQA - have and listen to double check you might not actually end up having a subtle preference!
- Those who are fanboys - have a listen and confirm you're right... Challenge yourself with your perceived notion when I reveal the blind later.
- Everyone can also say they've listened to what MQA does for themselves. That experience itself will allow one to express an opinion with some level of confidence having taken the effort.
And finally, instead of me writing about just the technical, and objective elements, I want the HUMAN LISTENER to have their say. Ultimately, it is about the enjoyment of music and whether the encoding makes the sound better as claimed by many. Can this be shown in the data?
Thanks for your reply. Yes, from this perspective the prospect is sensible.Delete
I ABXed FLAC vs 150kbps VBR Opus some time ago and at that bitrate the difference was so minute, it was imperceptible to me, even in "hi-fi" headphones such as HD-800, HD-650 and LCD-X, so IMHO MQA is going to fare in similar fashion.
I'll try to find suitable DAC for the test in the meantime.
BTW, are you planning to cover ADCs as well? I have been looking for fine ADC that'd be built on the same principles as NwAvGuy's ODAC, but apart from audio interfaces meant for bedroom musicians, I couldnt find anything decent, UAC1 compatible and budget friendly. When enquired Schiit and JDS about the prospect - their reply was that the main focus were in DACs, which is maybe a bit disappointing, as home audio always included some sort of recording option (be it tapes, VHS, DVD recorders, R2R, 8 track and so on), not so much with current "smart" offerings.
Just a simple box with dual 1/4" TRS balanced input and +4dBu/-10dBV input sensitivity switch, nothing more.
Sorry, stirred a bit offtopic.
Thanks for the note. It would indeed be great to spend more time on ADC's as they all have different resolution limits, idiosyncrasies around subtle frequency responses and especially high frequency modulator noise floors.
In the real world of course, the number of consumers wanting to buy DACs way outnumber those interested in ADCs. Other than audio engineers who might look deeply into the different ADC offerings comparing the minutiae, I'm just not sure there's much revenue to be had for consumer-oriented companies. Already we know musicians themselves often aren't particularly interested in the best playback equipment... Not sure if those guys care that much about the ADC which ultimately isn't as important as choosing the right microphones, mixing technique or various DSP plugins to create the final sound.
I listened carefully on two occasions 12 days apart and using Oppo PM-3 headphones, a Cambridge DacMagic XS and a Win 10 laptop and Foobar with the appropriate USB 2 ASIO driver for the DacMagic. I was sure I heard slight differences. I made notes at both sessions, the 2nd session without refering back to the first set of notes. The two sets of notes largely matched, except that my final preferences had swapped over here and there. When I ran the test on my main loudspeaker-based system, which I'm certain has enough resolving power, I grew tired of trying to hear any differences and gave up. Maybe not in the right mood and will give that another go. FWIW my main system is Oppo BDP-103 (files played from a USB hard-drive), Benchmark DAC2 HGC & AHB2 power amp, Spendor D7 loudspeakers. An interesting, but tough, asignment. Thanks for your hard work Archi :-) Regards, DerekReplyDelete
Thanks Derek for the diligence :-).Delete
After decades of "perfecting" hi-fi equipment, it would not be fun if the assignment ends up being *too* easy! Younger or older, mainstream or upper echelon gear, cloth ear or Golden Ear, I want to see feedback from everyone who enjoys music and can appreciate higher quality reproduction & gear...
Folks, take this as a personal challenge as well. Make sure you write down what your preferences are so that when I reveal the results, you have an idea of your "Golden Ear" status. Like I have done in previous blind tests of this nature, I will also look at the groups who have all 3 selections consistent with either preferring MQA or the standard hi-res to see if there are significant correlations! That could be interesting especially around correlation of equipment costs...
Thanks to those who have submitted results already. Keep 'em coming.
How did you decode the MQA files? Did you use the decoding lib from BluOS or the Tidal desktop app, or did you capture the output from the Tidal desktop app?
Actually, none of the above :-).
It's capture of the output from my Mac and Audirvana+ 3. Latest version when the test was constructed.
I am looking for a solution that uses a lib, preferable the one from BluOS. I know Mans used that one, but he does not seem to have released his code.Delete
My code is here: https://code.videolan.org/mansr/mqaDelete
You need to acquire the libbluos_ssc.so file on your own.
Thanks! I had already acquired the lib some time ago.Delete
Submitted. Expect expectation bias ;)ReplyDelete
Quick question: how do I know if my data was submitted? I clicked "submit survey" but only see some progress animation of three balls jumping.ReplyDelete
Ok resubmitted. Perhaps you'll have me twice now in your DB.Delete
Hi Archimago - just read your note on the double submissions. Depending on if my first submission on Sept. 5. went through you will have three submissions with the same Linn-Kaber setup in your DB.Delete
One from the 5., a second identical one from the 6. where I resubmitted and a third one on the 7. where I asked my partner to listen through the test. She had a different preference with sample C but the info rest of the info is identical - apart from her gender ;) Also I forgot to add that she is a trained musician.
All hail to clean stats & thank you!
no news at all? 11 setptember :PReplyDelete
2020-11-12: Can you give us the link to the final results?ReplyDelete
Here are the results:
Part I: Procedure
Part II: Core Results
Part III: Subgroup Analysis
Part IV: Subjective Results & Wrap Up