Saturday, 27 July 2013

MEASUREMENTS: DAC "Waveform Peeping" - the -90.3dB 16-bit LSB Test...


When it comes to technological "toys", I've vacillated over the years between the accumulation of digital photography gear and audio stuff... As I'm sure many of you know, "pixel peeping" is the act of "using 100% crops and similar techniques to identify flaws that have no effect on the photograph under real-world conditions" (Google web definition). Back in the "old" days (like a decade ago), the act of pixel peeping wasn't all that unreasonable since the differences visible could be demonstrated on photo-enlargements. When I was using my old Nikon D70 with 6 megapixels, sharpness at the pixel level was a significant consideration with moderate enlargements like 13"x19"; imperfections like moiré could be seen in the final product as well. Monitor resolution wasn't that high back then either so fine details were easily obscured.

Fast forward these days and I'm now using the Nikon D800. At 36 megapixels viewed on a >2MP monitor; unless I'm printing huge enlargements, there really is little need to "zoom" down into the 1:1 pixel level to appreciate a high quality image... Sure, sometimes it's just fun to see how much detail has been captured especially when evaluating different lenses or to show off each hair follicle, but for the most part, "pixel peeping" has become quite unnecessary.

Although in daily usage, one might not need to "peep" anymore, if one were to publish camera body or lens reviews, any reviewer these days "worth their salt" would run the images through objective tests; including highly detailed "pixel level" tests or compare 1:1 images between cameras or lenses. Dynamic range, ISO-noise interaction, color accuracy tests, distortion characteristics (for lenses), effect of file formats (JPEG vs. RAW) of course all serve to complete the evaluation. The quality is so high these days among high-end cameras (SLR's, medium format digital backs...), it is with these detailed tests that we can fully appreciate the qualitative differences between top contenders. Subjective opinions in terms of the camera's touch-and-feel and user interface are important of course, but if you care about the potential image quality that can be captured, then objective tests are really really important. If you haven't already done so, just have a look at the camera reviews on www.dpreview.com and see how much work actually goes into what I respect as proper reviews of well engineered equipment! Also of interest, Hasselblad is trying to market "exotic" cameras at high prices by appealing to aesthetics (just look at the responses to see how people feel about that!). [Here's another one.] Is this what happens when technology matures and companies have difficulty competing on primarily technological merits?

I've often wondered why in the audio world, objective measures have so often been left out as part of the review process - especially as it comes to line-level devices like DACs. Maybe it's because digital audio matured earlier and we're going to see the same outcome with cameras one day. Around some forums, the mere mention of objective measures seems to be scoffed at - as if objectivism with audio gear is either "obsolete" or the sole domain of "high end" manufacturers with arcane tests out of reach of mere mortals. I know I'm digressing into "MUSINGS" territory here, but IMO, a good review needs to dig into the gear's objective properties so the reader can truly appreciate how it compares with other similar gear in order to have an informed opinion and gauge value as a (hopefully) well engineered piece of technology... Let's get back on track then with some "MEASUREMENTS".

For me, one of the most interesting "waveform peeping" tests consistently done by Stereophile over the decades on digital gear has been the undithered 1kHz sine wave test at -90.3dBFS. This is one of the most "microscopic" tests of DAC performance. It's simple and the result basically answers the question "can this DAC accurately reproduce the least significant bit (LSB) in a 16-bit audio signal?" At a glance one can tell at least 3 things:
1. Is the DAC "bit-perfect" down to that last 16th bit? (Assuming everything upstream is set up properly, you should see something resembling the 3 quantization "steps".)
2. Is the dynamic range at least 16 bits? If not, the waveform becomes obscured by excessive noise.
3. Are there anomalies to the waveform morphology to suggest "DC shifts" leading to "tilting" of the waveforms (power supply related issues). (For a good example of 60Hz low frequency noise effect, see the measurement of the Philips CDR880 Figure 7.)

The Stereophile website archive provides some lovely examples of this test dating back to the late 1980's such as the Philips LHH1000 from 1989 (check out Figure 5). How about the Naim NA CDS from 1992 (Figure 6) or the $8000 Mark Levinson No.35 DAC from 1993 (Figure 6, still not good). By 1995, we saw excellent performance like with the $9000 Krell KPS-20i (Figure 5). In a few years, by 1998, reasonable priced gear like the California Audio Labs CL-15 CD player was capable of similar accuracy at the $1500 price point. Since the millennium, this level of performance can easily be achieved within the $1000 price point and below (eg. the Rega Apollo from 2006). These days, the little Audioquest Dragonfly can do a reasonable job USB-powered at <$250 retail.

On the whole, this test has demonstrated the progression of improved accuracy over the years. State-of-the-art DACs like the MSB Diamond DAC IV (Stereophile October 2012, not on website) and Weiss DAC202 (Figure 6) are great examples of what this level of accuracy looks like (as opposed to expensive gear of questionable technical ability which I will not mention). IMO, well engineered CD/DVD/SACD/Blu-Ray/DACs these days claiming to be "high resolution" really should pass this test without issue. Nonetheless, there are recent devices apparently incapable of a low noise floor for whatever reason (eg. Abbingdon DP-777 Figure 15, surprisingly the recent Wadia 121 Decoding Computer Figure 6 didn't fare too well either).

I was curious whether I could run a similar test using my simple test gear... After all, so long as the DAC and measurement device can achieve >16-bits dynamic range reliably, one should be able to obtain a reasonably good set of measurements. So far, from what I've seen in the other tests, I should be able to reproduce this test with the E-MU 0404USB!

Here goes... Setup and procedure for the various DACs/streamers:
Test DAC --> shielded RCA --> E-MU 0404USB ADC --> shielded USB --> Win8 laptop

- I created an undithered 1.1025kHz sine wave at -90.31dBFS at 16/44. This is what an "ideal" waveform would look like with the usual Gibbs phenomenon (ringing) due to bandwidth restriction.
- Green is LEFT channel, Blue is RIGHT channel. Notice the phase inversion between the channels.

- For comparison, I also created the equivalent at 24-bit quantization:

- Capture the above at 24/88 with Audacity using the E-MU 0404USB. From previous tests, the E-MU functions very well at 2x sample rates (88 & 96kHz) with optimal dynamic range. Although not as good as a high precision oscilloscope used by Stereophile, this should be adequate to allow relative comparisons between different DACs. I used the analogue preamp on the E-MU to boost the signal by about 18dB to give me "more" amplitude to capture.
- As you can see above, I decided to plot the channels overlaid and inverted to compare precision of timing and amplitude.

Here are the results of this test on the various DACs I have around here:

TEAC UD-501 [2x BB PCM1795 circa 2009] SHARP filter:
16-bit undithered:


24-bit:

Clearly the TEAC has no problem with reproducing that least significant bit in the 16-bit signal. Also, obviously the resolution has improved significantly by going to 24-bits.

ASUS XONAR Essence One [2x BB PCM1795 c. 2009] (opamps upgraded to all LM4562):

Very nice... Notice a wee bit of channel imbalance - the left channel (blue) seems consistently louder than the right. Same internal DAC chip as the TEAC so similar level of performance expected.

Logitech Squeezebox Transporter [AKM4396 c. 2004]:

Nice! Not bad for a discontinued device from a computer peripheral manufacturer released in 2006, eh? ;-)
Of course, the Stereophile review demonstrated this nicely already...

Logitech Squeezebox Touch [AKM4420 c. 2007]:
WiFi (only 30% signal strength 2 floors up from router!):

Ethernet:

24-bit:
Three observations:
1. Clearly the Touch is noisier than the better DACs above. It's still capable of >16-bit dynamic range though.
2. Some DC shift is evident - look at the upward slope with the 24-bit sine wave and compare to the Transporter above. Maybe this could be improved with a better linear power supply than the stock switching wallwart I used... Not sure if an improvement would be audible however.
3. No substantial difference between WiFi and Ethernet. (No surprise; just thought I'd have a look to see if WiFi added much noise down at this level.)
N.B. Remember that this is still a pretty good result - we are looking at a waveform down at -90dBFS, or ~90 microvolts! Nice correlation with what Stereophile found (Figures 5 & 6) in terms of the Touch being a 'touch' more noisy than better DACs.

AUNE X1 Mark I [BB PCM1793 c. 2003] (using CM6631A USB-to-Coaxial S/PDIF, ASIO driver):

This is what can be achieved by a <$175 DAC off eBay direct from China these days (I bought this unit in early 2012). Notice that it's able to produce a cleaner analogue output than the Touch. But it's also not quite up to the standard of the TEAC, ASUS, or Transporter. Notice both a slight channel imbalance as well as mild amplitude fluctuations (again, possibly due to cheap wallwart). Hopefully the following zoomed out screenshots illustrates this well for comparison:
 AUNE X1 - left channel (blue) noticeably louder and notice the amplitude fluctuations over time.

Touch - Notice it's more noisy with unpredictable amplitude spikes occurring in both right & left channels.

TEAC UD-501 - more stable, clean, uniform waveforms in comparison.

As much as it's great to see the level of performance afforded by DACs these days, I'm very impressed by the level of performance of this old E-MU ADC! As I have stated before, one of the reasons I put up these posts is to demonstrate that it doesn't take megabuck equipment to test out audio gear objectively. A lot can be "known" about the performance of a piece of hardware rather than depending on only subjective "opinion".

The other test I would categorize as the equivalent of "pixel peeping" is the Dunn jitter test where we're "peeping" into a small part of the audio spectrum around the 11 or 12kHz primary signal and scanning for sideband anomalies. IMO, neither the jitter nor this undithered LSB test really are that important for audio quality. Random noise affecting the 16-bit least significant bit would sound like some form of dither (eg. like what happens when an HDCD with embedded LSB data is decoded by a non-HDCD player). Likewise, my feeling is that even a "moderate" amount of S/PDIF jitter (like say 1ns) isn't going to intrude into my listening pleasure. (Maybe one could make the argument that the details and nuance of sound/music can reside in these microscopic domains but I have yet to see any proof...)

Assuming the digital player/DAC is meant to be faithful to the source signal and doesn't implement a DSP known to affect the LSB data, to be able to measure and verify precision down to these levels I believe would be a reasonable pre-requisite in achieving high-fidelity. It's a test of how well the hardware was designed and implemented than necessarily how it "sounds". Just like knowing if a hi-res digital camera is capable of the resolution it claims... You might never need 36 megapixels for a slideshow or in print, but it's good to know that the camera was capable of delivering on the claims! Likewise, if I'm going to spend a good amount of money on high-fidelity gear, I'd certainly like to know that precision engineering went into it by the results of tests like this among others already discussed over these months.

Let's throw some nostalgia in. Here's what the MUSE Mini TDA1543 x4 NOS DAC looks like down at -90dB (using the CM6631A USB-to-S/PDIF coaxial interface):

Party like it's 1991! Ugly... Clearly it's incapable of accurately reproducing the 16-bit LSB undithered tone.

Zoomed out (24/44) - still ugly:

Using a 24-bit signal makes no difference since this is a 16-bit DAC and the lower 8 bits get truncated. The Philips TDA1543 DAC chip was introduced back in 1991 according to the specs sheet... Thankfully, it looks like DAC designs have improved somewhat since then at least in this characterstic :-).

A stroll down memory lane... The top 3 highest grossing movies of 1991: Terminator 2, Robin Hood: Prince of Thieves, Disney's Beauty And The Beast. Top 3 songs (Billboard): (Everything I Do) I Do It For You, I Wanna Sex You Up, Gonna Make You Sweat (Everybody Dance Now). Hmmm...  Good year :-).

------------------------------------------------------------------

Folks, much of the measurements above were done about 3 weeks ago but I'm putting this post up behind the "Great Firewall" while on vacation (hey, >10hr plane flight gave me plenty of time to do some writing!). I know a few people are trying to get hold of me by E-mail. Unfortunately my VPN + Outlook is a bit finicky so other than more important work related matters as I check the E-mail every few days, I will likely not be responding until mid-August.

Time to go enjoy some good food... And snap some pictures of course... :-)


BTW: For those interested in some light non-fiction summer reading, consider picking up Chuck Klosterman's "I Wear The Black Hat: Grappling With Villains". An enjoyable, thought provoking social commentary.

Friday, 12 July 2013

LIST: Suspected 44 or 48kHz PCM upsampled SACDs.

 The sentence says "supported by Japanese SACD manufacturer" (whatever that means!). An example of how the term SACD gets thrown around in cheap domestic and pirated Asian markets (this wasn't a true XRCD either)...

As I mentioned previously in my post on SACD (and DSD), there are a number of SACDs I have digitally ripped over the years that appear to be sourced from 44kHz PCM. This is of course the same sample rate as good ol' RedBook CD and therefore it's unlikely that these titles should sound any "better" than the CD release since the PCM-to-DSD conversion process will add some distortion to the original signal.

It's not difficult to detect these releases because of the "brickwall" loss of frequencies beyond 22kHz. Note that this list is of course unofficial and even though there's evidence that these come from 44 or 48kHz PCM, it's still possible they're from 24-bit data which could still be "hi-res".

An example of the 22kHz brickwall - Thelonious Monk's "Between The Devil And The Deep Blue Sea" off Straight, No Chaser SACD. Notice the typical ultrasonic 'noise shaped' SACD quantization noise from 22kHz up - filtered off in this case before 40kHz.

Sunday, 7 July 2013

MUSINGS: Is CD sound quality (16/44 PCM) good enough?

According to some... CD sound quality is inadequate.

In the comments to a previous post, Fabio Zolli asked:

I would like to ask your opinion about high resolution audio files like 96/24 or 192/24 (I know it's off topic but I didn't know where to post). I think there isn't a really audible improvement among standard 44/16, taking into account also your mp3 test. Thanks for your attention.

Thanks for the note Fabio.

The MP3 test shows that even if we do hear a difference, at the level of quality tested (~320kbps MP3), without comparing the "studio master" sound, it's virtually impossible to know which is better/more accurate. Amazing given just how much data is being thrown out by MP3 encoding!

Personally, up to now I have not been able to ABX a difference between properly dithered 16/44 and the 24/96 source. Others have also done some fantastic testing around this like Mitchco at Computer Audiophile with different test methodologies.

It has been 30 years since the introduction of the CD (PCM 16/44) standard for mass consumption. The fact that we have not seen good controlled tests to show that 16/44 is somehow "lacking" (when listening to music at normal volumes) compared to 24/96+ or SACD tells me that 16/44 is most likely good for any circumstance. Much debate for example was sparked in 2007 by Meyer & Moran when they published their report based on work with the Boston Audio Society when listeners could not detect a significant difference between hi-res (DVD-A and SACD) analogue output versus the output from a 16/44 A/D/A loop.

Knowing the above, reading stuff like this should make an unsuspecting audiophile wonder why the inferiority of 16/44 wasn't proven ages ago! The author writes: "For me its clear that 24/96, 24/192 and DSD are superior to 16/44.1 in many meaningful ways..." Really?

The only rational issue I have come across "against" the 44kHz sampling rate is that with older DACs, the Nyquist frequency at 22kHz is too close to the potentially audible 20kHz upper range and the steep "brickwall" filter may cause audible effects. But this issue was addressed with upsampling and slower roll-off filters long ago (I know... yet another contentious issue for some).

Admittedly, I will seek out 24/96 audio for favored albums and rip my SACDs as 24/88. My rationale's simple... Since storage is cheap these days, I consider doing this as reasonable "insurance" to guarantee that everything that science tells us is humanly audible is captured in the digital sampling with plenty of headroom above 20kHz. I believe this is a reasonable price to pay in terms of storage even if I do not ultimately get an audible benefit because my ears aren't good enough for the task (like one pays for insurance never knowing if it's actually needed). Furthermore, objectively, all my DACs seem to be optimal at these 2x sampling rates as well showing nice frequency extension and excellent dynamic range measurements.

As for 24/192...  Christopher "Monty" Montgomery at xiph.org has written an excellent review of this topic including examples of how this could be detrimental to sound quality (like intermodulation distortion which also can be shown with 24/96). Likewise, Dan Lavry has written about the technical issues of 24/192 in his whitepapers like this one.

Personally, I have yet to find music I thought could benefit from this high sampling rate (how many microphones can even accurately record >40kHz?). Even though storage is cheap, I'm not convinced that there's anything to be gained going from 96kHz to 192kHz theoretically or otherwise. The cost-benefit ratio is hard to justify when benefit seems to be zero! (BTW I have come across some albums like Carmina Celtica from Canty off the Linn download site at 24/192 with unusually high noise level above 45kHz which I suspect could create some very nasty intermodulation distortion if not properly filtered.)

Bottom line:
I'm happy with 16/44 if the music is well recorded and mastered. For my favorite tunes, I'll go for 24/88 or 24/96 if available to "ensure" that I'm not missing something. To date, I have happily downsampled many 24/192 albums to 24/96 (or 24/176 to 24/88) without any reason to think that I'm somehow "missing out" (rather than gaining space for more music!). As for DSD, I only have a few albums verified to be sourced from an original DSD recording.

As many others have commented before, the main "enemy" to good sound these days is not about the audio format - yes, even the much-maligned-by-audiophiles MP3 can sound excellent if the underlying content is good. Rather, the way the album was recorded and mastered is more important. For example, the loudness war has caused more damage to sound quality than we can ever gain going from 16/44 to any high-resolution format. I believe sound quality "evangelists" like Neil Young would do well to "wage heavy peace" on that silly "war" and in the process stay relevant.

Feel free to leave me a note especially if there are good reasons to consider keeping 24/192 music on my server system! :-)

One more thing... There is one situation where you'd definitely want to either upsample or run a higher sampling rate - if you're using a NOS DAC. However, this is more to do with reducing aliasing distortion in the audible spectrum.

Musical selection tonight: Going to have a listen to k.d. lang's Ingénue (1992) again... It's been awhile!

Sunday, 30 June 2013

MUSINGS: The Squeezebox Family...

Well, after finishing the last couple of posts, I sent the Squeezebox Receiver back to fordgtlover. Thanks for the opportunity to test out another member of the SB family! I very much appreciate the generosity!

Before the Receiver left on the long trip back to Australia; I decided to take a family portrait:


I remember patiently awaiting the arrival of the Slim Devices Squeezebox 3 back in late 2005 after putting in the pre-order. It's thanks to this little device that my audio listening habits changed forever away from physical media. Despite the ups and downs with various versions of the server software, these little boxes have certainly made access to my own music remarkably easy in a way almost unimaginable previously. My father has access to my music library across the city through his Touch and wherever I go in my travels, music streams to the PC/Mac as well. The stability of my server these days with >5000 albums is fantastic with uptimes of months (this is with Windows, and the only reason it goes down these days is because I install Windows updates every few months).

Sadly, the Squeezebox line was discontinued in August 2012 by Logitech.

Looking back, I'm still impressed by the technology put together by the Slim Devices team to get the infrastructure in place, and later with the Logitech team and especially the Touch. Even today, the objective analogue audio output performance of the Transporter remains superb (it came out in September 2006) and as I look at all that can be done with the Touch (came out April 2010) including the ability to handle 24/192 (EDO kernel), communicate with a number of USB DACs, transport DSD64 via DoP, it's amazing the power of what can be done with an Open Source architecture where the "community" is empowered to maximize the machine's capabilities. Have a look at the used prices of the Touch these days, it's a reminder of just how much value it offered audiophiles. The computer audiophile community needs more devices of this caliber; devices that can "raise the bar" and do it without need for valuation as a luxury or "artisan" item when it's just "good sound" and "good functionality" that I believe many of us are after.

Looking ahead, it's great to see ongoing development in the Squeezebox community towards replacement hardware and software - check out the communitysqueeze.org FAQ. Wonderful to see the DIY machines being put together out there (check out the picture gallery!). With the availability of small, low powered, but reasonably fast computers like the Wandboard and the already ubiquitous ports of client software like Softsqueeze/Squeezeplay/Squeezeslave, Squeeze Player (Android), Squeezecast (iPod/Phone), I have a feeling that I'll be running my Logitech Media Server (or some equivalent) for the forseeable future...

... Now, if they could support multichannel FLAC/WAV and native stereo/multichannel DSD (I still think there's a need for better file formats than .dsf and .dff), I think we'd have all of audio covered. :-)

Vive le Squeezebox!

----

Well, it's summer time in the northern hemisphere and that means the kids are out of school. The heat wave finally arriving here in Vancouver. Time for some vacation, BBQ's, camping, lazy afternoons, general R&R, and of course good summer tunes (Katrina & The Waves I'm Walking On Sunshine is playing in the back currently). :-)

I'm also starting to go through Ethan Winer's recent 2012 book "The Audio Expert: Everything You Need To Know About Audio". Only started and already it's a good read written in an accessible manner for those interested in the science and technical aspects of audio without going deeply into the math. The Kindle edition (IMO a bit pricey) along with some mindless fiction like Dan Brown's Inferno will likely accompany me on the summer trips coming up.

As always, enjoy the music everyone!

Friday, 28 June 2013

MEASUREMENTS: Do bit-perfect digital S/PDIF transports sound the same?


Using suggestions from this page, the Touch can be used to transport DSD to the TEAC as DoP wrapped around a 24/176 FLAC file through Triode's USB kernel. Neat hack - proof of bit perfect transmission. Unfortunately the files are huge, so I likely will await an efficient solution. Note that this is NOT the test setup described below, just something cool. :-)

Let's talk about digital transports for a bit.

Most of us I'm sure remember those days when the only way to get digital data to an outboard DAC was through a CD transport. Although we can still resort to a CD reader, I suspect that many of us here have gone mostly into the computer audio realm with data on hard drives or flash/SSD devices. Thankfully gone are the days when the data being read off the CD could be inaccurate and interpolation may be needed in realtime, or be susceptible to mechanical failures of CD drive mechanisms (though hard drive failures and need for backups present another challenge).

I have shown that bit-perfect data can be transferred and played back without any concern off a USB asynchronous interface (eg. the various Mac and Windows software players and laptops). While I have the Squeezebox Receiver still on hand, let us have a look at the effect of using different transport devices with S/PDIF interfaces objectively.

Remember to keep in mind that the S/PDIF interface, unlike packetized asynchronous USB (or ethernet) conducts its data transfer in a unidirectional serial fashion formalized around 1985 when the AES3 (AES/EBU) standard was also laid down. As I think many of us have read, S/PDIF combines the data and clock signals using "biphasic mark code" and it is this "feature" of the interface that has resulted in many an audiophile nightmare regarding timing issues - jitter concerns especially well publicized. This is to a large part the basis for the well known paper by Dunn and Hawksford in 1992 asking "Is The AESEBU/SPDIF Digital Audio Interface Flawed?". (Of course the topic of jitter can be very complex as described in the paper going far beyond what we need to concern ourselves with here.)

With that said, let's be practical and see what the results looks like with the different devices using TosLink and coaxial interfaces (with comparison to asynchronous USB)...

Setup:

For this round of testing, I decided to take a break from the TEAC UD-501 DAC and go back to the ASUS XONAR Essence One for a bit. Although I have been listening to the TEAC a lot in the last couple months, on a daily basis, I still use the ASUS Essence One at my computer workstation and it was just more practical to run these tests there. Despite my concerns around the upsampling feature of the ASUS, it measures well and sounds excellent.

Here's the hook-up:
* Transport device * -> Coaxial/Toslink/USB cable -> ASUS Essence One -> Shielded 3' RCA -> E-MU 0404USB -> 6' Belkin Gold USB -> Win8 laptop

Coaxial cable = 6' Acoustic Research
TosLink cable = 6' Acoustic Research
USB cable =  6' Belkin Gold

Transport devices tested:
1. Squeezebox 3 -> Coax / TosLink
2. Transporter -> Coax / TosLink
3. Receiver -> Coax / TosLink
4. Touch -> Coax / TosLink
5. Laptop -> CM6631A Async USB to S/PDIF -> Coax / TosLink
6. Laptop -> Async USB direct to ASUS Essence One

As you can see I've got the host of Squeezebox devices on the test bench along with the usual two ways to connect the computer's USB port to DACs (direct or through USB-S/PDIF converter).

I. RightMark Analysis:

Since there are so many devices/combinations, I'll show the results a few at a time to demonstrate what was found. Let us start with the results of the four Squeezebox devices. I decided to "max out" the capabilities of the SB3 and Receiver by using 24/48 sampling rate:

Numerically, not much difference...  From my subjective listening during the tests, I would agree with these numbers in saying that "it sounds like the Essence One DAC"; there are more similarities in the sound than subjective differences.

Let's have a look at the frequency response graph because there does appear to be some difference - here's coaxial interface only:

Hmmm, interesting! Small differences at the top end. Let's zoom into that top end and have a good look:


Notice the shape of the curves suggest slightly earlier roll-off with some devices. The flattest, most extended frequency response (and possibly most "accurate") is the Transporter, followed by SB3, then Touch, and Receiver. Remember, we are talking only about 0.15dB difference between the Transporter and Receiver at 20kHz; not perceptible IMO but since we're looking for evidence of a difference, useful to note.

Let's now include the TosLink measurements:



Even though we ran out of colors, it doesn't matter because there are still only 4 curves. There is no difference between coaxial and TosLink; they overlay on top of each other essentially perfectly.

Let us now add the computer-USB interfaces (take away the TosLink since no difference):



As you can see, the flattest response curves come from USB direct and Transporter. Here's the ranking: Transporter, ASUS USB direct, SB3, Touch, CM6631A, and Receiver. We'll talk more about this later, just keep in mind then that frequency responses are *slightly* different between transports despite bit-perfect settings...

The rest of the RightMark graphs - no significant difference:




II. Jitter Analysis:

Dunn J-Test stimulation of jitter. To keep it more manageable, I'll group them into 16-bit and 24-bit side-by-side first, let's just look at the coaxial interface here:

A. Squeezebox 3 (16-bit / 24-bit):

B. Transporter (16-bit / 24-bit):

C. Receiver (16-bit / 24-bit):

D. Touch (16-bit / 24-bit):

E. Laptop -> CM6631A USB to coaxial (16-bit / 24-bit):

F. Laptop -> USB direct (16-bit / 24-bit):

Objectively, it looks like the CM6631A USB-to-S/PDIF and USB direct 24-bit graphs are cleaner, and of the Squeezebox devices, the Transporter on the whole seems to have the least data-correlated jitter. Even at its worst, the sidebands for the SB3 around the primary signal is down around -120dB. Is this a problem? I doubt it since auditory masking will easily make this inaudible (assuming one could even hear down that low around 12kHz pitch). Furthermore, in theory, the J-Test should create a "worst case scenario" for jitter which is unrealistic in real music.

Here's the difference between coaxial vs. TosLink:

A. Squeezebox 3 24-bit, Coaxial vs. TosLink:

B. Transporter 24-bit, Coaxial vs. TosLink:

C. Receiver 24-bit, Coaxial vs. TosLink:

D. Touch 24-bit, Coaxial vs. TosLink:

E. CM6631A USB to S/PDIF 24-bit, Coaxial vs. TosLink:

In general, we can say that indeed TosLink is worse (remember however TosLink is immune to electrical noise with galvanic isolation so there are some positives in this regard). Interestingly this is very clear with the Transporter! However, increased jitter with TosLink is not a given because the SB3 and Receiver seem to behave in the opposite fashion and show less jitter artifacts with the TosLink interface.

Remember that all of these jitter graphs are indicative of the interface between the transport device connected to the Essence One DAC. The graphs could be different with another DAC since much of the result will depend on the accuracy of the DAC in extracting the clock information and what other steps it might take (eg. data buffering, reclocking) to further stabilize the timing - it's not just about the transport device.

III. DMAC Protocol

So, up to now we can see differences between bit-perfect devices with RightMark, and obvious differences with the J-Test. How do they sound? Let's see what the computer "hears". For this test, I am using the Transporter playback as reference against which all the others are being compared.

First, I must admit that I'm not as confident about these numbers as I am of the graphs and plots above simply because it was really tough getting this done properly! From previous experience with the Audio DiffMaker program, results can vary depending on environmental factors like temperature of the equipment and subtle "sample rate drift" over time. With each transport measured, cables needed to be reconnected, settings needed to be changed, and for each condition, I ran the test 3 times to get a sense of the "range" of results. Admittedly, I made an error with the 'USB direct' measurements and did not realize this until after the fact so did not include the results here (foobar was accidentally set to output 16-bit instead of 24-bit).

The bottom line is that the results suggest that each device "sounded" different according to the computer. Instead of the usual high "correlated null depth" like in my previous tests with player software around 80-90dB (similar to the Transporter tested against itself above), we're seeing numbers in the 60-80dB range between transports. The computer thought the Squeezebox 3 sounded the most different from the Transporter. Good to see that it was able to detect the Receiver playing 320kbps MP3 as "most different" (ie. lowest correlation) to provide a point of reference. A reminder, this measurement is logarithmic so the actual mathematical difference between the MP3 sample compared to the others is larger than what it might look like on the graph.

Remember that this is a measurement of the difference between each device and the Transporter connected to the Essence One. There is no implication here of whether one sounds "better" than another since that would of course be the listener's subjective judgment call.

IV. Summary

Let me see if I can summarize this based on the results here along with what I know/believe over the months of testing as applicable... Q&A format:

Q: Do all bit-perfect transports sound the same?
A: Based on the results, not exactly. Even though bit-perfect (I have verified this with the Touch, Transporter, SB3, CM6631A, ASUS USB direct with ASIO), small differences in frequency response can be measured. Furthermore, jitter analysis clearly looks different between devices and this also varies between coaxial and TosLink interfaces (with TosLink generally worse than coaxial for jitter). Likewise, the DMAC test also suggests the level of audio correlation when playing musical passages is not as high as previous tests with bit-perfect software or decoding lossless compression. Within the Squeezebox family, not surprisingly the Transporter performed the most accurately with flattest frequency response and lowest coaxial S/PDIF jitter, although I was quite surprised by the stronger TosLink jitter.

Q: Why do you think the frequency response varies?
A: My belief is that this is not a jitter issue. The reason I say this is that there appears to be no difference between coaxial and TosLink even though jitter varies between the two interfaces as demonstrated by the Dunn J-Test. I believe that this is the result of mild clock speed / data rate differences of the transport devices. Since the word clock has to be recovered from the S/PDIF signal, clock accuracy is dependent on the transport's internal clock - some transports may be timed a little quicker, some a littler slower and the DAC has to adjust to this (of course the E-MU 0404USB ADC measuring the audio has a part to play in setting where it believes the roll-off should be). This frequency roll-off variability is not seen with laptops connected to an asynchronous USB device for example (that's of course the point of being asynchronous; not time-coupled to the data sender by having the recipient working off its own clock and telling the sender to speed up or slow down if necessary).

Q: But surely different/better/more expensive digital S/PDIF cables can help?
A: No. I don't think so. As I have measured and discussed before, digital cables make no substantial difference to timing/jitter as far as I can tell. Even though very long or poorly constructed cables may add to the jitter, the difference IMO is much less than what I'm showing here and as far as I can tell is irrelevant for a reasonable length of decently constructed coaxial/TosLink.

Q: Those jitter plots look nasty... I bet I can easily tell the transports apart!
A: Of course, anyone can claim anything over the Internet or in print since there are rarely if ever any actual "double checking" with sound methodology or formal peer review in the case of print magazines (obviously these are not scientific journals). Although I have shown these measurable differences, as a (currently) 41 year old male who works in an office environment, have generally avoided very loud concerts, and have a hearing frequency threshold around 16kHz, I do not believe I would be able to differentiate any of these bit-perfect transports in controlled testing with the same ASUS Essence One DAC.

Q: Surely you just need better gear to hear it!
A: The data correlated jitter with any of these devices would be >100dB below the primary signal. The frequency response difference is less than 0.15dB at 16kHz (my frequency threshold). Unless there's some significant interaction that causes anomalies in the output significantly beyond what I measure here, these difference would be inaudible to me irrespective of the quality of the sound system. Of course if you have younger ears and better hearing, this could be different. I believe speakers and headphones would introduce much more distortion and change to the frequency response than what I'm measuring here with a good modern DAC.

Q: Well, if that's the case, then I might as well go for the cheapest digital transport/streamer I can find, right?
A: Well, maybe, maybe not. When it comes to sound quality, I think a digital transport would have to be quite incompetent to sound poor (eg. non-bit-perfect, horrific jitter or imagine if the frequency response rolled off way too early because of severe S/PDIF timing inaccuracies). Therefore, spending more on a digital transport is IMO not primarily about sound quality but rather features and the aesthetic "look and feel" you're after (eg. better remote, can handle higher sampling rates, more reliable, fits into the decor...). Sound quality IMO is better served by putting the money into good speakers/room treatments/amp/DAC. Back in the "old days" of CD spinners, better mechanics with higher reliability and accuracy just cost more money. Even then it's not a given; I remember spending five times more on a higher model Harmon/Kardon CD player 20 years ago and that failed within three years whereas an inexpensive JVC from Costco with digital out still runs fine today. I have not had occasion to try the "low end" devices like the <$100 media streamers (eg. WD TV)  to see how those compare to the Squeezebox dedicated audio units.

BTW: If you're not aware, the Squeezebox devices by nature are asynchronous since they receive the data through WiFi or ethernet from Logitech Media Server and buffered with a decent amount of internal memory. You can see that the TosLink and coaxial connections have worse jitter than what's measured directly off the analogue outputs (eg. look at SB3, Touch, Transporter, Receiver jitter measurements).

As usual, feel free to comment or link to any good data you may have come across regarding this topic especially if conclusions are different from what I've presented.

Musical selection this evening:
Philippe Jaroussky - Carestini (The Story of a Castrato) (Virgin Classics, 2007) - amazing vocals and fascinating musical history. ("It's a man, baby!" -- Austin Powers)

Happy listening! ;-)

Monday, 24 June 2013

MEASUREMENTS: Squeezebox Duet - Receiver & Controller (Analogue Output).

A number of moons ago (December 2012), this blog was started to obtain data on ~320kbps MP3 vs. lossless audio. Near the end of data collection for that study, I started performing some tests and began posting them on the Squeezebox forum for discussion. 

Over the ensuing months, I started posting data on the family of Squeezebox devices which I collected over the years as the foundation of my sound system at home. Up to this point, I have had the opportunity to measure ALMOST the whole family of products from the SB3 onwards...


Obviously missing from this lineup is the Duet - a combination of the Receiver and the LCD-endowed Controller which I have always wanted to try but never saw a unit locally to buy back in the day. I want to send a big thank-you to 'fordgtlover' from the Squeezebox forum for taking the time and expense to send me the Squeezebox Duet for testing - all the way from Australia no less!

Here she is:

The Controller comes with a nice weighted metal stand which also plugs in as a recharger. The LCD screen is reasonably bright and easy to read. Buttons have a nice click and selection is made with the turn wheel. One could ask for a smoother feel to the wheel, but as is, it's reasonably functional. Even though you can connect the Receiver via ethernet, the Controller obviously needs a functioning WiFi network.

The Receiver is a relatively non-descript black box with just a lit wireless icon on the front to indicate power & device status. Getting it connected up was quite simple and quick using the Controller - see here for the details.

A look at the back ports on these devices. From left to right of the Receiver: analogue stereo, TosLink digital, coaxial digital, Ethernet (10/100), and wallwart power (9V, 0.56A). You can make out on the top of the Controller a phono plug as well for audio output to headphones on the go.

I: Receiver Measurements

Internally, the Receiver uses the Wolfson WM8501 which is spec'ed at 100dB SNR and although it can go to 24/192, the Receiver is limited at 24/48 (as are all the other Squeezeboxes except Transporter and Touch).

Okay, lets get into it. First I wanted to have a look at the analogue outputs of the Receiver. Since this is a loaner unit, I'll try to extract as much as I can before sending it on the long journey home to Australia...

Lets pull up an oscilloscope reading* - 1kHz 0dBFS square wave (24/44):
The yellow channel is the right, and blue left. Notice there is some channel imbalance on this unit with the left channel a bit louder. Left channel peak voltage measures at 2.47V, right channel 2.40V.

The square wave itself looks nice. Minimum noise, good contours with minimal ringing (compare with the Controller below). From this, one would suspect that the usual RightMark analysis would show good results.

Here's the impulse response (as usual, 16/44 impulse recorded with the E-MU 0404USB at 24/192):

A standard linear phase filter response. Absolute polarity is maintained. We can therefore expect a sharp roll-off at Nyquist.

A. RightMark results:
The usual setup:
SB Receiver --> Shielded RCA --> E-MU 0404USB --> Shielded USB --> Win8 laptop
Firmware: 77
Receiver volume 100%.

Cables were the same for all these measurements (a 3' shielded RCA cable from Radio Shack, shielded 6' USB cable).

I tried both ethernet and WiFi. Since I could find no difference, I'll just present the WiFi data here. I noticed that the WiFi in the Receiver was able to achieve better signal strength than the Touch at the same location. 2 floors up from the ASUS RT-N56U router, my Touch got ~65% strength, while the Receiver was up at ~75% (Controller also about 75%).

Where I did the measurements, the Receiver was up around 85% WiFi signal strength.


As you can see, I measured the Receiver against the Touch all the way up to the max sampling rate of 24/48. Also, I included the 24/44 data from the Squeezebox 3/Classic as comparison. The graphs below are derived from the 24/44 data. Notice that we gained 4dB from 16 to 24-bits using the Receiver...  Not much.

Frequency Response:

Noise:

THD:
Relatively stronger odd-order harmonics up at 3, 5, and 7kHz compared to the Touch or SB3.

Stereo Crosstalk:

As you can see, a mixed bag. The Receiver's frequency response is similar to the Touch; both are flatter than the SB3 which tends to roll off the deep bass a bit more. The Touch clearly beats the other two in terms of noise performance (SB3 and Receiver about equivalent here), but scores lower with stereo crosstalk (not really an issue IMO). Total harmonic distortion was higher with the Receiver.

B. Dunn Jitter Test (16 & 24-bits):
16-bits:

24-bits:
Low jitter using this test in the analogue output.

II: Controller Measurements

The controller is a neat device. As shown above, there is a metal charging base and it's motion sensitive so will automatically wake when picked up (a few seconds wait time to wake up if not in the charger). It's pretty good at doing what it's meant to - as controller for the various Squeezebox units (not just the Receiver). The scroll wheel allows accelerated item selection, but these days, I mainly use a tablet to control the Squeezebox devices and it's of course quicker to type in search items than scrolling through letters.

The Controller has a standard 3.5mm phono plug up top for headphones. Although it can also play back audio, as far as I know, this feature was implemented at a "beta" level only. You have to go into Settings -> Advanced -> Beta Features -> Audio Playback to turn this on. Because the unit has a little speaker inside, I also needed to download the "Headphone Switcher" app from Applet Installer which adds an item in the "Extras" menu to turn the headphone on. Eventually I was able to get the unit to automatically switch between the speaker and headphone after some fiddling and rebooting the device (honestly I don't know which step was the key - the nature of 'beta' I guess). I did notice some slowdown to the UI responsiveness when streaming audio to the Controller.

According to the SB Hardware Wiki, the Controller has a Wolfson WM8750 inside. Understandably, this is a low power chip meant for portable devices. Data sheet lists DAC SNR up to 98dB. Maximum sample rate goes up to 24/48.

Setup:
Controller --> phono-to-RCA shielded Radio Shack cable (3') --> E-MU 0404USB --> shielded USB --> Win8 laptop
Firmware: 7.8.0-r16739
Controller volume 100%.

Here's what a 1kHz square wave, 0dBFS, 24/44 looks like under the oscilloscope playing off the phono plug:

In comparison to the Receiver above, it's clear that this isn't going to be "hi-fidelity". There's a bit of ringing and noise evident. Peak voltage is around 800mV and the two channels are reasonably balanced.

Here's the impulse response:
Upside down - the Controller inverts polarity. Standard linear phase digital filter.

A. RightMark results:


Comparison is made with the Receiver and Touch. Numerically, the thing that really sticks out is that massive intermodulation distortion up at 4%! I'm a bit surprised how the program did not calculate a higher THD as well given how nasty the graph looks - this is why I post up the graphs as well... I checked and double checked the testing to make sure settings were not clipping the signal. Made no difference - it is what it is!

No meaningful improvement with going to 24-bits.

Frequency Response:
Odd noise spikes from 5kHz up. Have not seen this before in my other tests. I did the test 3 times and this appears to be a real finding. Also, earlier roll-off down to -3dB by 20kHz.

Noise:

THD:
This looks bad but the score was about the same as the Receiver. RightMark is probably just looking at the odd and even harmonics for the calculations.

Intermodulation Distortion + Noise:
Ouch. Non-linearity is an issue.

Stereo Crosstalk:

As I said earlier - the Controller isn't high fidelity :-).

B. Dunn J-Test:
16-bit

24-bit

Very strong jitter! Many many nanoseconds of jitter :-). Again, this is not distortion from clipping.

III: Summary (Analogue Output)

Receiver: Reasonable analogue output from the device. It performs similar to the old SB3 in terms of dynamic range and noise floor. I hooked up the Receiver to my bookshelf system upstairs ('vintage' Sony MHC-1600 from university days powering some Tannoy mX2's) for a few days to listen. Very enjoyable - got a chance to listen to the Into Darkness soundtrack, the recent HDTracks release of Cole Español, and stroll down pop memory lane with Samantha Fox's Greatest Hits :-). I pulled out the Squeezebox 3 to compare and subjectively, I agree that there's a bit more bass with the Receiver and Touch than the SB3. While that channel imbalance was measurable, I didn't find any gross anomaly with the speaker system but I could hear the difference with the AKG Q701 headphones for example listening to where Ella Fitzgerald's voice was centered on Ella Sings Gershwin, I'd say it's subtle so would not affect my listening pleasure (could also be placebo since I was specifically listening for this!).

Controller: Firstly, how does it sound? Well, perhaps surprisingly OK :-). It's low powered so I listened with a pair of JVC HAFXC80 IEM headphones. I would compare the sound output to what I hear off my Samsung Galaxy S2. If you have a Controller, have a good listen - that's what a high jitter device sounds like (I have not ever seen this many side bands in all the testing so far!). For me, the best I can describe is that the Controller sounds "distant" even with volume pushed up and slightly "hollow" compared to listening through the headphone out from the Touch. The percussion at the start of Star Trek Main Theme from the "Star Trek: Into Darkness" soundtrack for example sounded less defined and spatially smeared, getting worse as the orchestral dynamics build up as if there's some low-level static in the background. How much of this can be definitively attributed to the high jitter is hard to say since the unusual frequency response, noise levels, intermodulation distortion and headphone output limitations are all significant factors in the sound. I highly recommend just streaming 192kbps MP3 as this will improve the responsiveness of the Controller and you're not going to hear a difference.

"fordgtlover" wanted to know how well the Receiver serves as a digital transport - good question! This then will be the topic for the next installment along with comparisons with the Touch... Stay tuned... Something tells me this is going to be complicated and hopefully quite interesting!