MEASUREMENTS: Bit-Perfect Audiophile Music Players (Mac OS X).

Do bit-perfect Mac audio players sound the same?

Over the last few years, the list of "audiophile" audio players on the Mac has gradually increased. Do they sound the same if set to bit-perfect output? Let's have a look at the candidates I'll be considering here:

1. Decibel: I bought this program more than a year ago. It's a no-nonsense program that plays a nice range of file formats without fuss. It's able to take exclusive access of the audio device, and memory playback. As with all the commercial offerings, it can switch sample rate automatically. PCM only, no DoP for DSD at this time. I upgraded to the latest version 1.2.11 for these tests. Memory playback was activated.

2. Audirvana Plus: Current version is 1.4.6. I bought this one about 6 months ago. It's got a nice, fancy GUI. Able to handle DSD files with DST and was able to play DSD64 and DSD128 over the USB interface to my TEAC UD-501 without problem. "Under the hood", it's also got some extra features like memory playback, "Direct Mode" apparently bypassing CoreAudio as well as "Integer Mode". Since the software supposedly bypasses CoreAudio, I would have thought that "Integer Mode" would be an obvious given. They also talk about 64-bit processing which is great if one has need for the SRC and dithering (iZotope-based)... For these tests, I'm using Direct, Integer Mode with memory playback to the TEAC. The green "INT" indicator turns on. Also, I have SysOptimizer turned on (disables Spotlight, Time Machine, some USB tweaks).

3. JRiver Media Center for Mac - Well known media player originating from the Windows world. I measured the beta 18.0.177 build for this test. Bit-perfect from the start so I didn't fool with any of the default settings. It's capable of DSD playback to the TEAC using DoP.

4. Pure Music - I'm not as familiar with this one. I installed the trial version 1.89g. It literally "wraps" around the iTunes interface. Can handle DSD but I didn't bother trying since it looks like there were some contortions needed to get these files recognized under iTunes. "Memory Play" was activated for playback. My subjective opinion is that I did not like the UI and using iTunes means no native FLAC support.

5. TEAC HR Audio Player - Release version 1.0 for Mac. Just a freebie I can run with the TEAC DAC. Handles FLAC. Will do DoP for DSD playback. Unable to decompress DST though. Does have an "Expand to RAM" mode which I did not use for these tests.

6. iTunes 11.0.2 - The "standard" Mac music player. Should be "bit-perfect" so long as volume at 100% and none of the DSP plug-in's are activated. A lot of uncertainly out there about this program with folks jumping up and down with each version claiming that sound has changed for better or worse... Version 11 was released in November 2012 with some folks claiming volume and sound quality changes compared to version 10. The BIG negative about iTunes for audiophiles is the lack of automatic sample rate switching - need to go into the "Audio MIDI Setup" panel to change sampling rates and bit depth (yuck). IMO, the other BIG negative about iTunes is that it does not support FLAC...  Seriously, after 11 versions, to not support the universal lossless audio format is just stupid and has been a reason why I do not buy music from Apple.

Over the years I have tried Play, Amarra, and Fidelia as well, but figure the above was enough to look at for a sense of the field out there around Mac music players. I see there's also BitPerfect for iTunes - again, FLAC limitation sucks.

Setup:

(Note that this is same as previous DMAC Test.)

MacBook Pro (*running audio player*) --> shielded USB --> TEAC UD-501 DAC --> shielded 6' RCA --> E-MU 0404USB --> shielded USB --> Win8 laptop

MacBook Pro is the 17" early-2008 model previously described. Nothing fancy, and in fact relatively "old" 2.6GHz Core 2 Duo processor. Running OS X Mountain Lion with no OS tweak for audio.

Win8 laptop is the Acer Aspire 5552 which has been my measurement "work horse". Again, nothing fancy, just 2.2GHz AMD Phenom X4 processor to grab data from the E-MU 0404USB and process the data through DiffMaker, RightMark, or jitter FFT analysis.

Part I: RightMark 6.2.5 (PCM 16/44, 24/96, and DSD64)

All the measurements done with the test signal encoded as FLAC except for those based on iTunes (iTunes, Pure Music) where AIFF was used.

PCM 16/44:

Frequency Response:

Noise:

THD:

Stereo Crosstalk:

PCM 24/96:

Frequency Response:

Noise:

THD:

Stereo Crosstalk:

DSD64 (via DoP) - for the programs that support DSD:
Note that this is achieved using the 24/96 test signal encoded into DSD64 using KORG AudioGate, then played back to the TEAC UD-501 DAC using DSD Over PCM (DoP) protocol and measured with RightMark. As usual, we see the effect of noise shaping in DSD up in the ultrasonic range.

Frequency Response:

Noise:

THD:

Stereo Crosstalk:

Part II: Dunn J-Test for jitter (16-bits shown for brevity)

Decibel:

Audirvana Plus:

JRiver:

 

Pure Music:

Wanted to see if turning Memory Play ON / OFF had an effect.

 Memory Play OFF

Memory Play ON

TEAC:

iTunes:

No difference to see here folk... Didn't show the 24-bit test, but that was unremarkable as well.

Part III: DMAC Protocol

Time to let the machine have a listen to the music and see what kind of correlation it finds using the standard 24/44 audio sample with Audio DiffMaker... Reference for all these "correlational null depth" measurements is Decibel FLAC recording. Each audio player was measured 3 times.

I threw in comparison measurements for MP3 320kbps and 192kbps. Also to show what happens with some DSP processing - Pure Music with volume reduction of -1dB (with dither), and turned on the EQ in iTunes and dropped 8kHz slider just by 1 "click" lower.

I found it quite remarkable the drop in null depth by just turning on the iTunes EQ plug-in and using it to adjust just 1 notch (don't know how many dB's this is supposed to represent) [see addendum]! Pure Music -1dB volume control changed the measurement slightly but not much. DiffMaker has amplitude compensation so it is trying its best to compare the audio quality beyond the volume difference.

Part IV: Conclusion

Well everyone, unless I missed something obviously subtle here, what I see is that bit-perfect is indeed bit-perfect playing the audio through my TEAC UD-501 DAC with all these programs.

Now of course I cannot overgeneralize these findings to all Mac computers, all DAC's, all player programs, all drivers, all DAC's, etc... But I think I can say with some assurance given similar setups as mine that:

1. With bit-perfect playback, all the player software performed equivalently. This is supported by every measurement method used. Subjectively with headphones attached to the DAC, I did not notice a difference listening to the music being played back while doing the DMAC Test.

2. No evidence of anomalies in the Dunn jitter test signal. This is not surprising as I had already previously reported that I was unable to detect more jitter with increased processor load as some seem to believe. From what I can tell, jitter is primarily a hardware property and software timing issues lead to obvious audio drop-out rather than subtle pico- or nano-second changes in the audio output.

3. Although I did not do an equivalent DMAC (DiffMaker) test with the DSD audio, it looks like all 3 programs tested with DoP capability performed equivalently using the RightMark test. Still waiting for more DSD content for this to matter. :-)

4. I see no evidence that special features like memory playback, "direct mode", "integer mode", "SysOptimizer" made any difference compared to the output from the no-frills TEAC player where I did not even turn on the memory playback feature with the 2008 MacBook Pro.

Bottom line is that these programs work well to output bit-perfect audio. The MAIN feature over iTunes is the ability to automatically adjust the sample rate. Beyond that, I'm happy to own both Decibel for its simplicity and flexibility in playing all kinds of formats as well as Audirvana Plus for the full feature set including DSD playback and DST decoding. I just don't see any evidence that they sound any different...

Do bit-perfect Mac audio players sound the same? Yes, as far as I can measure and have personally experienced.

Again, let me know if you have any evidence otherwise.

I was E-mailed shortly after publishing the TEAC UD-501 review if I've tried JPLAY - not yet, but in the weeks ahead may find some time to hook up the Windows setup and have a look...  Until then, I recommend reading Mitch's excellent writeup between JPLAY and JRiver.

Enjoy the tunes :-).


Addendum - June 9, 2013:
To answer that question of why even just -1 click at 8kHz with the iTunes equalizer resulted in such a low DMAC correlation null... Here's the answer:

Yes, you can see a very small dip in the frequency response at 8kHz - the intent of the EQ. However, the high frequency gets rolled off very significantly as well from ~15kHz.

MEASUREMENTS: Do lossless audio formats sound different? (WAV vs. AIFF vs. FLAC vs. ALAC vs. APE vs. WV vs. lossy MP3 & AAC...)

The year is 2013.

Digital audio has been around for a long time. The CD 16/44 PCM format has been the de facto standard of audio delivery for 3 decades now. For the last decade at least, many of us have been involved in computer audio of one form or another. Personally I started seriously archiving all my CD's with bit-perfect rips since 2004 and conversion of all my PCM audio to FLAC by 2005/2006.

In all these years, I do not believe I have ever felt that playback of a compressed lossless format like FLAC compromised sound quality. Yet, if you look around the Internet at the various audiophile forums, you hear from all kinds of folks how uncompressed formats like WAV and AIFF "sound better" than the lossless compressed formats like FLAC, Apple Lossless (ALAC), WavPack (WV), and Monkey's Audio (APE).

Let's have a look...

Setup:

MacBook Pro (Decibel player) --> shielded USB --> TEAC UD-501 DAC --> shielded 6' RCA --> E-MU 0404USB --> shielded USB --> Win8 laptop

MacBook Pro is the 17" early-2008 model previously described. Nothing fancy, and in fact relatively "old" 2.6GHz Core 2 Duo processor. Running OS X Mountain Lion with no OS tweak for audio. Decibel version 1.2.9 (haven't upgraded to latest version yet). For Decibel I did not even turn on the "load file to memory" option so the lossless decompression is happening real-time.

Win8 laptop is the Acer Aspire 5552 which has been my measurement "work horse". Again, nothing fancy, just 2.2GHz AMD Phenom X4 processor to grab data from the E-MU 0404USB and process the data through DiffMaker.

Procedure:

I encoded the DMAC Test using dBPowerAmp 14.3 from FLAC (which I used to standardize the test results) into the various formats supported natively: WAV, AIFF, ALAC. I downloaded the binaries for APE (v.4.11) and WavPack (v. 4.60.1 Windows) from the official web sites respectively. I used the highest compression level available for each - level 8 for FLAC, -hh "very high quality" for WavPack, "Insane" for APE.

All files were transferred to the Mac and played back off the machine's 240GB SSD drive. I ran 3 iterations with each file format to account for some inter-test variability. DiffMaker comparison was made between my "standard" FLAC recording and each of the test recordings.

Results:

All the lossless formats scored within a narrow range. Correlated null depths across the board were in the 80-90dB range for the lossless formats. As expected, the lossy formats (MP3 and AAC) did not score as well. Also as expected, AAC 192kbps showed less variance (spectrally more accurate) than the equivalent MP3 encoded at 192kbps - AAC is newer and clearly better at lower bit rates.

Conclusion:

A couple observations...

Firstly, notice the greater variability in numerical results for the lossless formats (but remaining in the 80-90dB reference range). Remember that the correlation scale is measured in dB's - it's logarithmic. With "bit-perfect" measured correlations up around 90dB's, sensitivity is very high and it doesn't take much difference to alter the measured value. Measurements with results lower down like in the 60's and 50's tend to show less inter-test variability.

Secondly, when I listen to the "difference" WAV file produced by DiffMaker of 80-90dB correlated null depth, I need to turn up the headphone volume on the TEAC (listening with Sennheiser HD800) to maximum where it still sounds soft. With normal audio, this would be uncomfortably loud. Sonic differences therefore would be orders of magnitude softer than the normal music itself.

Bottom line. The measured variance from the TEAC DAC analogue output between lossless file formats decoded using an older Core 2 Duo computer without decoding into RAM first is extremely low - basically, there's no difference in the sound.

Do lossless compressed formats all sound the same? YES, they should, and in this test, they do.

Based on what I'm hearing and measuring, it's obviously not hard to get good bit-perfect sound. If a piece of equipment is producing audibly different output from say WAV vs. FLAC (that is, assuming the difference isn't cognitive/perceptual bias), then I think there's something wrong with the setup since this was not the intent of the creators of lossless compression. Either the settings are wrong (eg. transcoding to lossy format, ReplayGain tags being applied, or DSP turned on) or there's something 'broken' in the decoding process (eg. CPU too slow, data transfer speed issue, or poor software unable to keep up with the relatively low processing demands). This is a problem and diagnostics should be run to determine how to fix it.

As usual, please feel free to drop me a note or link to good evidence if you run across any information contrary to these test results and opinion.

Enjoy the music...


--------------------------------
Addendum (those interested in spectral plots of the difference between FLAC reference and test file):

FLAC / APE / WV / ALAC / WAV / AIFF all look somewhat like this - not much to see. Note: There's always a little bit of noise in measuring the analogue output plus limitations of the E-MU ADC.:

This is what lossy looks like in comparison - quite striking how much can be "reconstructed" and still sound good!
MP3 320kbps:

AAC 320kbps:

MP3 192kbps:

AAC 192kbps:

PROTOCOL: [UPDATED] The DiffMaker Audio Composite (DMAC) Test.

This summary is not available. Please click here to view the post.

MEASUREMENTS: TEAC UD-501 DSD Performance (Part 3)

Okay, this is the 3rd (and likely) last part of my TEAC UD-501 review.

This will be the second time I measure a true DSD device. The first time was about a month ago when I checked out the beta firmware for the Oppo BDP-105 at a friend's house. Unfortunately, although we seemed to be able to play some DSD128 samples from 2L properly in DFF format, I was unable to play the DSD128 encoded test signals properly, so this will be the first time on this blog I can show the improvements between DSD64 and DSD128.

Setup:

The setup is the same as with my PCM tests.

MacBook Pro --> shielded USB --> TEAC UD-501 --> 6' shielded RCA --> E-MU 0404USB --> shielded USB --> Win8 laptop

DSD playback software: JRiver Mac alpha 18.0.177.

I used the same KORG AudioGate 2.3.1 DSD-encoded test signals as I did previously with the Oppo tests. Basically, I took the highest resolution 24/192 synthetic test signal produced by RightMark 6.2.5, ran them through AudioGate to produce the equivalent DSD64 and DSD128 versions to test. Of course, doing this involves a transcoding step so the result at best is that of the PCM signal minus small losses due to the transcoding using AudioGate. These tests will not demonstrate the best that the TEAC (or whichever DSD device tested) can do, but rather, hopefully a reasonable approximation. Doing this also implies the possibility that different conversion software could produce different results.

As you likely are aware, 1-bit quantization results it lots of noise. DSD deals with this by virtue of the high sample rate - DSD64 at 2.8MHz and DSD128 at 5.6MHz along with noise shaping to shift the noise up into the ultrasonic parts of the spectrum. By doing this, DSD is capable of high signal-to-noise ratio in the audible spectrum rivaling that of 24-bit PCM. However, this noise floor is not flat like for PCM, but rather will gradually increase higher up in the spectrum and then escalate rather quickly by 20kHz for DSD64.

As I showed in the Oppo test, when you look at something simple like a 1kHz sine wave through DSD64, you can actually make out this high-frequency noise. Here is how it looks coming out of the RCA output from the TEAC:

DSD64 1kHz sine wave at -6dBFS:

DSD128 1kHz sine wave at -6dBFS:

PCM 16/44 SHARP digital filter, 1kHz sine wave:

 

As you can see, the DSD128 and PCM (with digital interpolation filter) waveforms look nice and smooth compared to the DSD64 output. Good "concrete" example of the improvement with DSD128.

As I mentioned, noise shaping will move the excess noise up into the ultrasonic spectrum. Due to concerns that this ultrasonic signal may cause some amps to oscillate, an analogue lowpass filter is used to remove some of this noise. The finite impulse response (FIR) filter is selectable for the TEAC based on the options in the PCM1795 datasheet:

FIR1: fc = 185kHz, gain = -6.6dB

FIR2: fc = 90kHz, gain = +0.3dB (default)

FIR3: fc = 85kHz, gain = -1.5dB

FIR4: fc = 94kHz, gain = -3.3dB

Now before one gets overly impressed, realize that these filters operate in the ultrasonic range... That is, you're not going to hear the difference other than the potential gains or attenuation as it may affect the audible frequencies. However, those gain values are very audible indeed with FIR2 sounding clearly loudest and FIR1 softest (again, like for the PCM filters, just a turn of the knob allows you to instantaneously A-B the difference on the TEAC).

DSD64 (1-bit, 2.8MHz):

Let's get to it then, RightMark 6.2.5 results playing the DSD64 transcoded 24/192 test tone:

The first 4 columns are the TEAC with FIR filters 1-4. The fifth is the TEAC playing the original PCM test with the SHARP filter (best measuring of the three). Finally the last column is the Oppo BDP-105 playing from a USB stick.

Notice the default FIR2 filter performed the best.

Here's what the frequency response looks like:

Notice how little difference there is between the 4 FIR filters! I believe this is to be expected since the analogue filters as I mentioned are operating way in the ultrasonic spectrum and differences are seen only above 20kHz. In comparison, the PCM test behaves appropriately (yellow), and the Oppo (red) interestingly has a filter that measurably starts rolling off by 10kHz (no biggie, still only down by -0.7dB at 20kHz).

Noise Level:

THD:

It's quite evident from the graphs above just how noisy DSD64 is above 20kHz. The 24/192 PCM test signal (yellow) is cleaner beyond 20kHz. The noise characteristics of the TEAC and Oppo are about the same with a hint that the Oppo's analogue lowpass filter is stronger.

DSD128 (1-bit, 5.6MHz):

Finally, I get to have a look at what DSD128 measures like. Theoretically, 1-bit sampling going from 64x to 128x should allow the machine to push the noise an octave higher. Therefore, we should see the noise start ascending from 40kHz onwards. Likewise, the SNR in the audio spectrum should improve as well but this would be already beyond the E-MU's ability to measure.

First, the Summary:

Hmmm, nice. No deterioration in noise or dynamic range compared to DSD64 (like I said, these should be even better at DSD128 but will need a better measurement device). Again, we see the fluctuations in measured values between FIR1-4. The 5th column again is the native PCM 24/192 SHARP filter result.

Frequency Response:

Interesting, once we hit DSD128, they're all essentially identical. This too could be just a limitation of the E-MU hardware and RightMark software.

Noise:

THD:

In both the graphs above, we see that DSD128 indeed does push the noise floor out to ~40kHz as predicted and from there, the level rises quite significantly. Although the noise level isn't as high as DSD64 by 100kHz, it is still significantly higher than with PCM (yellow).

Dunn J-Test:

As I've mentioned before with the Oppo tests, you cannot apply the J-Test to DSD and expect to stimulate jitter since this was designed for PCM with 16/44kHz or 24/48kHz sampling rates in mind; specifically for digital S/PDIF or AES/EBU interfaces between transport and DAC. Nonetheless, here's what they look like through the TEAC in DSD64 (DSD128 looks about the same).

16-bit J-Test:

24-bit J-Test:

Looks perfect (as it should with DSD).

Summary:

Well everyone, there you have it. DSD64 and DSD128 off the TEAC UD-501. Contrary to the French Qobuz review where the authors have suggested that DSD was being converted to PCM internally, the test results here are consistent with direct DSD decoding. For a comparison, look at the results with the Pioneer DV-588A where internally 24/88 PCM conversion was being done on DSD64 (look at the unusual noise spectrum, PCM-type frequency response, and J-Test showing the jitter pattern found in PCM).

The TEAC UD-501 performed essentially the same as the Oppo BDP-105 from what I see here. This is good and provides a nice comparison with the level of performance out of the SABRE32 DAC. Again, this level of performance certainly is consistent with the subjective listening of DSD64 and DSD128 material I reported earlier.

About those FIR filter settings. There really is no difference in sound other than the differences in gain from my listening. I'd just stick with FIR2 unless there's a reason you would want to attenuate the signal (for example, FIR1 with -6.6dB allowed me to measure the TEAC through the XLR output without clipping the E-MU 0404USB - results are good BTW and shows the benefits of balanced interconnects).

Considering the ultrasonic characteristics of these filters for a moment, FIR1 with a cutoff frequency (fc) of 185kHz basically will allow all the DSD noise to pass through unattenuated. So, if you figure you have an amplifier that can handle all that ultrasonic noise feel free to give this setting a try... It's like the old "custom" filter setting of the first Sony SCD-1 (vs. the stronger "standard" filter) where some audiophiles felt the sound became more "open" and "airy" with the weaker filter. FIR3 is the strongest filter out of the 4. Personally I'm happy with FIR2 with its fc of 90kHz. Look at the PCM1795 datasheet for some nice graphs for these filters.

Benefits of DSD128 over DSD64 are clear in the measurements - you can see it in the cleaner sine wave above, and it pushes the ultrasonic noise out to ~40kHz. Subjectively, it clearly has the potential to sound fantastic if the recording is up to par. I guess we'll see in the days ahead just how much commercially available material gets released...

Bottom line: For the price, the TEAC UD-501 DAC offers up a lot of value and fantastic set of features. Sonically, I believe this DAC can easily trade punches with the best out there - whether PCM or DSD.

Enough with testing... Time to enjoy the music!

----------
NB:

Before I end off, I just want to make a general comment & plea about the state of DSD computer audio now that I've got a chance to try it out.

The DFF and DSF file formats are inadequate. Compared to FLAC, APE, or ALAC, these DSD file formats feel geriatric! Seriously, PLEASE get the file format right.

Firstly, we need good tagging features - all the more important for DSD since much of the excellent material consists of classical music where it's important to document conductor, orchestra, composers, title, year of performance and composition, etc... Please let me be able to use something universal like the excellent Mp3tag to manage all my PCM and DSD files.

Secondly, a standard DSD file format NEEDS lossless compression. DSD is extremely compressible - using DST with DFF files, I regularly see compression ratios >2.5:1 losslessly, getting up to 3:1 in some tracks. This becomes even more useful for DSD128 where the space savings are very substantial. By doing this, DSD64 can be compressed to file sizes overall smaller than 24/88 encoded with FLAC with equivalent (some would say better) sound quality... I'd certainly be happy with that! Lossless compression would also save file transfer times and cost of storage for the music producer, distributor, and of course consumer. Seriously, what other modern hi-resolution media format doesn't allow for at least lossless compression?

Over the months, I have heard DSD apologists talk about how you don't need compression or native tagging because "hard drives are cheap" and "JRiver can tag with its internal database". Sorry... That's not good enough. This is a foundational matter and will impact future generations of products, so it's important to get it done properly instead of rely on work-arounds.

Please guys, now that you've gotten together to define DoP and manufacturers to make these DAC's, lets get it done right with a standard DSD format that's fully capable, preferably "free" as in "open". Maybe some enterprising coder like Josh Coalson of FLAC fame can apply their expertise!

Thanks in advance. ;-)