Friday, 24 May 2013

MEASUREMENTS: Bit-Perfect Audiophile Music Players (Mac OS X).

Do bit-perfect Mac audio players sound the same?

Over the last few years, the list of "audiophile" audio players on the Mac has gradually increased. Do they sound the same if set to bit-perfect output? Let's have a look at the candidates I'll be considering here:

1. Decibel: I bought this program more than a year ago. It's a no-nonsense program that plays a nice range of file formats without fuss. It's able to take exclusive access of the audio device, and memory playback. As with all the commercial offerings, it can switch sample rate automatically. PCM only, no DoP for DSD at this time. I upgraded to the latest version 1.2.11 for these tests. Memory playback was activated.

2. Audirvana Plus: Current version is 1.4.6. I bought this one about 6 months ago. It's got a nice, fancy GUI. Able to handle DSD files with DST and was able to play DSD64 and DSD128 over the USB interface to my TEAC UD-501 without problem. "Under the hood", it's also got some extra features like memory playback, "Direct Mode" apparently bypassing CoreAudio as well as "Integer Mode". Since the software supposedly bypasses CoreAudio, I would have thought that "Integer Mode" would be an obvious given. They also talk about 64-bit processing which is great if one has need for the SRC and dithering (iZotope-based)... For these tests, I'm using Direct, Integer Mode with memory playback to the TEAC. The green "INT" indicator turns on. Also, I have SysOptimizer turned on (disables Spotlight, Time Machine, some USB tweaks).

3. JRiver Media Center for Mac - Well known media player originating from the Windows world. I measured the beta 18.0.177 build for this test. Bit-perfect from the start so I didn't fool with any of the default settings. It's capable of DSD playback to the TEAC using DoP.

4. Pure Music - I'm not as familiar with this one. I installed the trial version 1.89g. It literally "wraps" around the iTunes interface. Can handle DSD but I didn't bother trying since it looks like there were some contortions needed to get these files recognized under iTunes. "Memory Play" was activated for playback. My subjective opinion is that I did not like the UI and using iTunes means no native FLAC support.

5. TEAC HR Audio Player - Release version 1.0 for Mac. Just a freebie I can run with the TEAC DAC. Handles FLAC. Will do DoP for DSD playback. Unable to decompress DST though. Does have an "Expand to RAM" mode which I did not use for these tests.

6. iTunes 11.0.2 - The "standard" Mac music player. Should be "bit-perfect" so long as volume at 100% and none of the DSP plug-in's are activated. A lot of uncertainly out there about this program with folks jumping up and down with each version claiming that sound has changed for better or worse... Version 11 was released in November 2012 with some folks claiming volume and sound quality changes compared to version 10. The BIG negative about iTunes for audiophiles is the lack of automatic sample rate switching - need to go into the "Audio MIDI Setup" panel to change sampling rates and bit depth (yuck). IMO, the other BIG negative about iTunes is that it does not support FLAC...  Seriously, after 11 versions, to not support the universal lossless audio format is just stupid and has been a reason why I do not buy music from Apple.

Over the years I have tried Play, Amarra, and Fidelia as well, but figure the above was enough to look at for a sense of the field out there around Mac music players. I see there's also BitPerfect for iTunes - again, FLAC limitation sucks.

Setup:

(Note that this is same as previous DMAC Test.)

MacBook Pro (*running audio player*) --> shielded USB --> TEAC UD-501 DAC --> shielded 6' RCA --> E-MU 0404USB --> shielded USB --> Win8 laptop

MacBook Pro is the 17" early-2008 model previously described. Nothing fancy, and in fact relatively "old" 2.6GHz Core 2 Duo processor. Running OS X Mountain Lion with no OS tweak for audio.

Win8 laptop is the Acer Aspire 5552 which has been my measurement "work horse". Again, nothing fancy, just 2.2GHz AMD Phenom X4 processor to grab data from the E-MU 0404USB and process the data through DiffMaker, RightMark, or jitter FFT analysis.

Part I: RightMark 6.2.5 (PCM 16/44, 24/96, and DSD64)

All the measurements done with the test signal encoded as FLAC except for those based on iTunes (iTunes, Pure Music) where AIFF was used.

PCM 16/44:

Frequency Response:

Noise:

THD:

Stereo Crosstalk:

PCM 24/96:

Frequency Response:

Noise:

THD:


Stereo Crosstalk:

DSD64 (via DoP) - for the programs that support DSD:
Note that this is achieved using the 24/96 test signal encoded into DSD64 using KORG AudioGate, then played back to the TEAC UD-501 DAC using DSD Over PCM (DoP) protocol and measured with RightMark. As usual, we see the effect of noise shaping in DSD up in the ultrasonic range.


Frequency Response:

Noise:

THD:

Stereo Crosstalk:

Part II: Dunn J-Test for jitter (16-bits shown for brevity)

Decibel:


Audirvana Plus:

JRiver:

Pure Music:
Wanted to see if turning Memory Play ON / OFF had an effect.
 Memory Play OFF


Memory Play ON

TEAC:

iTunes:

No difference to see here folk... Didn't show the 24-bit test, but that was unremarkable as well.

Part III: DMAC Protocol

Time to let the machine have a listen to the music and see what kind of correlation it finds using the standard 24/44 audio sample with Audio DiffMaker... Reference for all these "correlational null depth" measurements is Decibel FLAC recording. Each audio player was measured 3 times.

I threw in comparison measurements for MP3 320kbps and 192kbps. Also to show what happens with some DSP processing - Pure Music with volume reduction of -1dB (with dither), and turned on the EQ in iTunes and dropped 8kHz slider just by 1 "click" lower.


I found it quite remarkable the drop in null depth by just turning on the iTunes EQ plug-in and using it to adjust just 1 notch (don't know how many dB's this is supposed to represent) [see addendum]! Pure Music -1dB volume control changed the measurement slightly but not much. DiffMaker has amplitude compensation so it is trying its best to compare the audio quality beyond the volume difference.

Part IV: Conclusion

Well everyone, unless I missed something obviously subtle here, what I see is that bit-perfect is indeed bit-perfect playing the audio through my TEAC UD-501 DAC with all these programs.

Now of course I cannot overgeneralize these findings to all Mac computers, all DAC's, all player programs, all drivers, all DAC's, etc... But I think I can say with some assurance given similar setups as mine that:

1. With bit-perfect playback, all the player software performed equivalently. This is supported by every measurement method used. Subjectively with headphones attached to the DAC, I did not notice a difference listening to the music being played back while doing the DMAC Test.

2. No evidence of anomalies in the Dunn jitter test signal. This is not surprising as I had already previously reported that I was unable to detect more jitter with increased processor load as some seem to believe. From what I can tell, jitter is primarily a hardware property and software timing issues lead to obvious audio drop-out rather than subtle pico- or nano-second changes in the audio output.

3. Although I did not do an equivalent DMAC (DiffMaker) test with the DSD audio, it looks like all 3 programs tested with DoP capability performed equivalently using the RightMark test. Still waiting for more DSD content for this to matter. :-)

4. I see no evidence that special features like memory playback, "direct mode", "integer mode", "SysOptimizer" made any difference compared to the output from the no-frills TEAC player where I did not even turn on the memory playback feature with the 2008 MacBook Pro.

Bottom line is that these programs work well to output bit-perfect audio. The MAIN feature over iTunes is the ability to automatically adjust the sample rate. Beyond that, I'm happy to own both Decibel for its simplicity and flexibility in playing all kinds of formats as well as Audirvana Plus for the full feature set including DSD playback and DST decoding. I just don't see any evidence that they sound any different...

Do bit-perfect Mac audio players sound the same? Yes, as far as I can measure and have personally experienced.

Again, let me know if you have any evidence otherwise.

I was E-mailed shortly after publishing the TEAC UD-501 review if I've tried JPLAY - not yet, but in the weeks ahead may find some time to hook up the Windows setup and have a look...  Until then, I recommend reading Mitch's excellent writeup between JPLAY and JRiver.

Enjoy the tunes :-).


Addendum - June 9, 2013:
To answer that question of why even just -1 click at 8kHz with the iTunes equalizer resulted in such a low DMAC correlation null... Here's the answer:
Yes, you can see a very small dip in the frequency response at 8kHz - the intent of the EQ. However, the high frequency gets rolled off very significantly as well from ~15kHz.

29 comments:

  1. Again, great to see an objective approach to this - I like the ability to use various SRC settings, etc., in Audirvana and in BitPerfect, but aside from that, and the ability to automatically change the MIDI setting to follow playback, really hear no differences between these, and no advantage over iTunes. If iTunes were to add automatic MIDI setting (even without any control over resmapling settings), it would be a no-brainer just to stick with iTunes.

    ReplyDelete
    Replies
    1. Exactly - that and FLAC.

      Delete
    2. Hi. Is there any disadvantage to have an ALAC based music library?

      I have encoded all my CDs directly in ALAC (XLD or iTunes rip) and I convert FLAC I buy quickly with XLD.

      That said, I agree, it is silly that Apple does not support FLAC to import existing libraries.

      Delete
  2. It is rather easy to find out how much correction one 'step' is in the equalizer... RMAA ?

    I agree with the findings and feel the tests are 'valid' but have the advantage to know what is actually tested and how it is. Most audiophools, however, have no technical background and will not understand what has been tested and simply say... the test is flawed or does not reveal what determines SQ simply because there is 'something or some aspect' the ignorant technical minded people do not KNOW about (yet) and drag in the middle ages and what they thought they knew back then and how wrong they were knowing what we know now and stating we still have LOTS to learn.

    So when subjectivists get wind of this blog they will disregard all that is done here based on:

    A: They believe the quite obvious differences can not be measured (yet).
    B: Subjectivists (especially the < 1 ps jitter crowd) believe the changes they can perceive are in the -130 dB region and the measurements appear to differ in the -90dB region already.
    C: Subjectivists always trust their ears over any instrument.

    the list of excuses is endless and replies to Mitchco's en NwAvGuy's attempts are living proof of it.

    It is too bad none of these golden ear oriented characters can actually demonstrate what they claim they in front on a less subjective person that tests them. There is always an excuse mostly about 'familiarity with the system', 'not revealing enough system' or 'pressure to perform' which seems to be able to, temporarily, completely loose the ability to discern, the otherwise always clearly present, differences under these (or other) flags. The few that a tested were never able to prove what they claimed (and still claim) they CAN do.. just not when I am around... stress ?

    Of course I can't discern differences when testing blind but sometimes think I hear differences when testing sighted. A well performed control test always proves that not to be the case. For a lot of 'objective minded' audio-aficionados that sometimes doubt if, what subjectivists say with such conviction, might actually have some truth in it or think they might not have the same hearing abilities/gear as 'them', these tests might give them a push in the back.

    ReplyDelete
    Replies
    1. Thanks for the comments Frans.

      Yes, RIAA would show what the equalizer did... I might have a look next week after my vacation :-)

      Agree on the general audiophile-folk comments. I think (hope) it's a matter of education. After decades of various audiophile magazines promoting the purely subjective perspective with no way to "fact check", I suspect many audiophiles have lost touch with the technical/engineering side. I do appreciate John Atkinson's measurements in Stereophile and see his measurements section as absolutely the most useful portion of the magazine.

      Imagine where we would be if all of engineering were performed without properly controlled tests!

      I *try* to avoid arguments with hard-core subjectivist folks. Pointless. I'd probably have better luck with the evangelists that come by my door once awhile. I usually just remind them that I'm after accuracy from the gear; nothing more, nothing less - ALL of that can be measured by instruments more accurate than my ears/brain.

      Delete
  3. Great site - a quick question as I couldn't find it in any of the posts, when performing the Dunn J-test you appear to be using an analyser other than the Rightmark suite. What are you using and is it free?

    ReplyDelete
    Replies
    1. Hi "Me".

      Yup, the Dunn test is surprisingly easy to do. All you need to know with links:

      Go here:
      http://www.diyaudio.com/forums/digital-source/183110-jitter-test-signal-j-test-signal-matlab-2.html
      Get the Dunn jitter .exe to create the WAV file for either 24-bit or 16-bit tests.

      Go here:
      http://www.ne.jp/asahi/fa/efu/soft/ws/ws.html
      and grab WaveSpectra which I use to perform the FFT in realtime.

      Go into settings in WaveSpectra and choose 131072 points under the FFT tab (this is the max), and I also use Blackman-Harris for the windowing function.

      Now play the Dunn jitter track into your analysis hardware and watch the waveform analysis in WaveSpectra... Of course the resolution of your measurement will be dependent on the device used but even with my humble E-MU 0404USB, it's good enough for at least gear comparisons.

      Happy sideband hunting.

      Delete
  4. Hi, trust my ears and my expensive gear : use the TEAC player with your TEAC DAC (rather than any A+ PM Amarra etc) ; it's even worth the pain to extract the dff from SACD isos (over A+).... the music is faster, livier ; it sounds right. Is this subjective ? no more than reducing the complexity of the involved physics to mere 70's hifi criteria such as FR... Thanks anyway

    ReplyDelete
    Replies
    1. Since I believe DSD should always be compressed (with DST) to save >2:1 storage, the TEAC player doesn't work for me with most of my SACD rips.

      "mere 70's hifi criteria such as FR" - hmmm... Unless human ears have evolved in the last few decades or physics has altered, I'd say FR remains pretty important :-)

      Delete
  5. This is a bit off topic...

    I recorded 60 second samples of 24/96 material played by JRiver, Foobar and XMPlay using GoldWave 5.69. They were recorded to 24/96 PCM Stereo. I then tried to use those samples in Audio DiffMaker. The files would load but you could not play them - the program would crash. You could load the files and try to difference them but again it would just keep crashing.

    I tried the same thing this time using 16/44 source recorded to 16/44. No problems at all.

    So I have a question: have you used DiffMaker on 24/96 files and if so, what is the trick?

    ReplyDelete
    Replies
    1. Yes - I'm using 24/96 for the DMAC test.

      You have to limit the length down to ~30-40 seconds otherwise it seems to overload the program... When it tries to run >4,000,000 FFT's on the sample, it crashes for me.

      Delete
    2. Okay great! Thanks for the tip. Can I download your test file? If not can you email it to me - my reply ID at gmail.

      Delete
  6. I'll see about putting the little file up on filepost or something similar tonight...

    ReplyDelete
    Replies
    1. Hi folks - I just updated the "PROTOCOL" page for the DMAC test to include a link if you want to get the test to try yourself.

      http://archimago.blogspot.ca/2013/05/protocol-diffmaker-audio-composite-dmac.html

      Remember that for my results, I played the 24/44 file and recorded it back at 24/96 for the comparison... I just found 24/96 more of an optimal sample rate for my gear and gave me somewhat more consistent results.

      Enjoy.

      Delete
    2. Using your sample, I am still unable to use the 24/44 samples that I record with Goldwave with Audio DiffMaker. Weird eh? DiffMaker crashes. I can play the recordings back with JRiver and Foobar but not DiffMaker.

      Delete
    3. Strange, weirdo12.

      Did you make sure there wasn't too much silence before and after the recording to keep the duration essentially just the test audio?

      Delete
    4. Yep the sample is about 36 seconds. In fact I trimmed much of the silence off of you sample file. Here's on of the files I recorded - maybe you can play it:

      https://docs.google.com/file/d/0B56jXUfrXyKdbDhRMHJ2V0FubUE/edit?usp=sharing

      It's something to do with the way GoldWave saves the WAV file I suppose. The reason I am using it to record is that it will start recording when it see's the audio and I can end end up with virtually identical samples.

      Delete
    5. After doing some trial and error and getting GoldWave and DiffMaker working together at 24/96, I finally found this on the GoldWave form from back in 2008:

      "Microsoft change the RIFF Wave specification to add the WAVE_FORMAT_EXTENSIBLE format tag. All PCM formats with greater than 16 bit quality are now covered by that tag. Any files using the old WAVE_FORMAT_PCM format tag for high quality are considered obsolete."

      GoldWave uses the WAVE_FORMAT_EXTENSIBLE tag unless you specifically change the settings to use the obsolete tag. If you do use the obsolete tag, data recorded with GoldWave works correctly in Audio DiffMaker.

      http://www.goldwave.ca/forums/viewtopic.php?f=1&t=4289

      http://www.goldwave.ca/forums/viewtopic.php?f=1&t=2569

      Delete
  7. This comment has been removed by the author.

    ReplyDelete
  8. This comment has been removed by a blog administrator.

    ReplyDelete
  9. Pure Music does support both FLAC and DSD. Since it uses iTunes as the library UI, you simply have to drag and drop any FLAC or DSD files/folders to Pure Music and it inserts a small bookmark file in the folder for each album. After that, the albums will show up in iTunes but can only be played through Pure Music. I've loaded tons of FLAC and DSD files into my library that way.

    ReplyDelete
  10. If I'm reading this right, you are testing bit perfection through an analog loop. There are interfaces, ranging from $30 solutions, like the FiiO D5 that provide SPDIF out, and recording-oriented interfaces often offer SPDIF in (like the Focusrite Scarlett 6i6 and 8i6). After recording the input from the digital input, you can align and do a bitwise comparison of original and recording.

    That said, I _have_ run those tests in the past, and in my tests, most players on mac were bit perfect, with the exception of Audirvana Plus, which would suffer dropouts of a few samples here and there.

    ReplyDelete
    Replies
    1. Do you need a good music player for Mac? I'm using Macgo free media player, which can play music smoothly. Besides, it can support almost all media categories and formats, including DVD, VideoCD, MOV, MKV, AVI, FLV, WMV, MP4, MPEG, RMVB, MP3, WMA, AAC, AC3, etc. It can play not only movies, videos, audios but also musics and photos on MacBook Air, Pro, Mac mini, iMac and Mac Pro, with Mac OS X 10.10 Yosemite. http://www.macblurayplayer.com/mac-media-player.htm

      Delete
  11. Just wanted to thank you for this article, all your hard work in getting all this data is much appreciated. As someone who went from windows coupled with foobar and asio drivers to Mac a few years ago I wish I had this comprehensive resource earlier to ensure I was getting the best possible sound out of my Mac. From what I understood the main advantage in this realm that OS X has over Windows is that it is -inherently- capable of bit perfect audio playback. I experimented some in the past but found good old iTunes with volume at 100 and eq turned completely off offered great results. Thanks again for the page, will be keeping an eye on your blog

    ReplyDelete
  12. Thanks for review, it was excellent and very informative.
    thank you :)

    ReplyDelete
  13. One of the very few measurements I know showing a difference.
    Post 24 in http://www.avsforum.com/forum/91-audio-theory-setup-chat/1889169-continuing-crisis.html

    ReplyDelete