MEASUREMENTS: Do lossless audio formats sound different? (WAV vs. AIFF vs. FLAC vs. ALAC vs. APE vs. WV vs. lossy MP3 & AAC...)

The year is 2013.

Digital audio has been around for a long time. The CD 16/44 PCM format has been the de facto standard of audio delivery for 3 decades now. For the last decade at least, many of us have been involved in computer audio of one form or another. Personally I started seriously archiving all my CD's with bit-perfect rips since 2004 and conversion of all my PCM audio to FLAC by 2005/2006.

In all these years, I do not believe I have ever felt that playback of a compressed lossless format like FLAC compromised sound quality. Yet, if you look around the Internet at the various audiophile forums, you hear from all kinds of folks how uncompressed formats like WAV and AIFF "sound better" than the lossless compressed formats like FLAC, Apple Lossless (ALAC), WavPack (WV), and Monkey's Audio (APE).

Let's have a look...

Setup:

MacBook Pro (Decibel player) --> shielded USB --> TEAC UD-501 DAC --> shielded 6' RCA --> E-MU 0404USB --> shielded USB --> Win8 laptop

MacBook Pro is the 17" early-2008 model previously described. Nothing fancy, and in fact relatively "old" 2.6GHz Core 2 Duo processor. Running OS X Mountain Lion with no OS tweak for audio. Decibel version 1.2.9 (haven't upgraded to latest version yet). For Decibel I did not even turn on the "load file to memory" option so the lossless decompression is happening real-time.

Win8 laptop is the Acer Aspire 5552 which has been my measurement "work horse". Again, nothing fancy, just 2.2GHz AMD Phenom X4 processor to grab data from the E-MU 0404USB and process the data through DiffMaker.

Procedure:

I encoded the DMAC Test using dBPowerAmp 14.3 from FLAC (which I used to standardize the test results) into the various formats supported natively: WAV, AIFF, ALAC. I downloaded the binaries for APE (v.4.11) and WavPack (v. 4.60.1 Windows) from the official web sites respectively. I used the highest compression level available for each - level 8 for FLAC, -hh "very high quality" for WavPack, "Insane" for APE.

All files were transferred to the Mac and played back off the machine's 240GB SSD drive. I ran 3 iterations with each file format to account for some inter-test variability. DiffMaker comparison was made between my "standard" FLAC recording and each of the test recordings.

Results:

All the lossless formats scored within a narrow range. Correlated null depths across the board were in the 80-90dB range for the lossless formats. As expected, the lossy formats (MP3 and AAC) did not score as well. Also as expected, AAC 192kbps showed less variance (spectrally more accurate) than the equivalent MP3 encoded at 192kbps - AAC is newer and clearly better at lower bit rates.

Conclusion:

A couple observations...

Firstly, notice the greater variability in numerical results for the lossless formats (but remaining in the 80-90dB reference range). Remember that the correlation scale is measured in dB's - it's logarithmic. With "bit-perfect" measured correlations up around 90dB's, sensitivity is very high and it doesn't take much difference to alter the measured value. Measurements with results lower down like in the 60's and 50's tend to show less inter-test variability.

Secondly, when I listen to the "difference" WAV file produced by DiffMaker of 80-90dB correlated null depth, I need to turn up the headphone volume on the TEAC (listening with Sennheiser HD800) to maximum where it still sounds soft. With normal audio, this would be uncomfortably loud. Sonic differences therefore would be orders of magnitude softer than the normal music itself.

Bottom line. The measured variance from the TEAC DAC analogue output between lossless file formats decoded using an older Core 2 Duo computer without decoding into RAM first is extremely low - basically, there's no difference in the sound.

Do lossless compressed formats all sound the same? YES, they should, and in this test, they do.

Based on what I'm hearing and measuring, it's obviously not hard to get good bit-perfect sound. If a piece of equipment is producing audibly different output from say WAV vs. FLAC (that is, assuming the difference isn't cognitive/perceptual bias), then I think there's something wrong with the setup since this was not the intent of the creators of lossless compression. Either the settings are wrong (eg. transcoding to lossy format, ReplayGain tags being applied, or DSP turned on) or there's something 'broken' in the decoding process (eg. CPU too slow, data transfer speed issue, or poor software unable to keep up with the relatively low processing demands). This is a problem and diagnostics should be run to determine how to fix it.

As usual, please feel free to drop me a note or link to good evidence if you run across any information contrary to these test results and opinion.

Enjoy the music...


--------------------------------
Addendum (those interested in spectral plots of the difference between FLAC reference and test file):

FLAC / APE / WV / ALAC / WAV / AIFF all look somewhat like this - not much to see. Note: There's always a little bit of noise in measuring the analogue output plus limitations of the E-MU ADC.:

This is what lossy looks like in comparison - quite striking how much can be "reconstructed" and still sound good!
MP3 320kbps:

AAC 320kbps:

MP3 192kbps:

AAC 192kbps: