Saturday, 21 February 2015

MEASUREMENTS: The Intercontinental Internet Audio Streaming Test...

Time to go intercontinental. :-) [Scene from the old movie War Games.]

After the ethernet cable results last week and the absence of any difference, there was discussion about an extreme internet music server test. What if instead of maybe 100 feet of ethernet cable from server to player, we had thousands of miles of cables in between?

With help from Mnyb from the Squeezebox Forum, we were able to orchestrate a test to demonstrate the extremes of measured performance with the server system basically on the other side of the world. He lives out in Västerås, Sweden approximately 100 km from Stockholm. A direct distance of:


More than 7300 km away from my home in Vancouver, Canada. This test will require the data to be transferred across the Transatlantic internet "backbone" and across North America to get to the west coast of Canada. Considering that the internet infrastructure cables are not straight lines, I suspect the data probably is traveling a substantially longer distance to reach my home than 7300 km.

Undersea Internet backbone map circa 2011.
I can tell that we're traveling a substantial distance based on the results of data latency. Using the paping (port ping) program, I can 'ping' the TCP port of Mnyb's Logitech Music Server in Sweden to see how long 2-way data takes between the two locales. First within my home, here's how it looks:

As you can see, my server is located internally at 192.168.1.80, and it takes on average <1 millisecond to go from the media room to my music server and back. Port 9000 is the LMS server control port if you're wondering. But when we reach out to Sweden:
IP address removed of course...

It now shows an average of 213ms data latency. Note that this amount of latency is problematic for real-time interactive data transfer like playing a first-person video game off the server... I'd get slaughtered if it took 1/4 of a second to tell the server what I'm doing in a high speed game of Call of Duty! But remember, music streaming is about bulk data transfers.. Let's see if I can find any measurable issue.

I. Set-up

My local system is set up exactly as described last week. I'll use the green 20' standard CAT 6 UTP cable I measured last week to connect my local switch with the Transporter player.

Mnyb's Server in Sweden <--> generic ethernet patch <--> Netgear GS108 gigabit switch <--> 20' flat ethernet cable <--> Netgear GS105 gigabit switch <--> generic patch <--> Linksys WRT1900AC <--> patch cable to wall socket <--> THE INTERNET >7000 km <--> 3' Cat 6 UTP patch cable <--> NETGEAR Nighthawk R7000 router <--> 30' Cat-6A STP Gigabit cable (in wall) <--> 6' Cat 6A STP patch cable <--> TP-LINK TL-SG1008D gigabit switch <--> 20' Cat 6 UTP cable <--> Logitech Transporter streamer


Whew... As for Mnyb's server computer:
HP ProLiant MicroServer N36L - released in 2010, dual-core 1.3GHz AMD Athlon II NEO. 1GB DDR3 RAM, gigabit port, 250GB OS hard drive, Western Digital "Green" 2TB drive.
Running ClearOS 6 (Linux), Logitech Media Server 7.9.

Note that his server machine is less powerful than what I'm using in my tests last week with those different ethernet cables (AMD A10-5800K, 16GB RAM). Not a worry though since serving music is a rather straight forward task not needing much CPU speed; especially with an efficient OS like Linux.

Mnyb's ISP is rated 100Mbps down/10Mbps up. Mine is a 50Mbps down/5Mbps up.

II. RightMark Audio Analyzer (24/96)

Summary of the calculated values. The first 3 rows consist of measured results from last week using various ethernet cables, the fourth row is the result of the "Intercontinental Test":

As you can see, there's really not much difference! Remember that these tests are very sensitive so little variations can happen just because of cables being moved for example. Furthermore, the "Intercontinental Test" was done a week later after I had put everything away from last week's test. Of interest and importance is that distortion results (THD, IMD) were essentially the same and certainly no worse than having a server in the next room. Again, these results are for a 24/96 "high resolution" audio test.

Some graphs then - same set as last week's:
Frequency Response: Essentially a perfect overlay representative of the Transporter device.
Noise level: There is a small spike at 120Hz I didn't see last week. Little spurious noise like this can be seen due to the sensitivity of the system. In any case, there's the 60Hz power line noise, and nothing else measures greater than -110dB.

IMD: Nothing unusual to write home about!

Stereo crosstalk: Very minor differences with tests run 1 week apart.

III. J-Test (24-bit)

I took the J-Test composite from last week and overlaid the result from today's "Intercontinental Test":

You can make out the result today as slightly brighter than last week's plots. Again, this is a pretty close overlay of the 24/48 Dunn J-Test spectrum. No evidence of any significant jitter being added to the signal during the >7000 km journey.

IV. Conclusion / Discussion

As you can see, objectively, there is no evidence that locating the server >7000 km away did anything to the output quality of the Transporter. Neither the RightMark results nor the Dunn J-Test suggests any change beyond what I normally expect from inter-test variation.

Despite the tests completing without issue, I did have trouble streaming the 24/88 FLAC-encoded Shelby Lynne song "Pretend" from the Just A Little Lovin' SACD rip which I had included in the directory for Mnyb to put up on his server... Because the test signals were short and consisted of quite a bit of silence, they were easily compressed by FLAC and transferred easily with an average data rate of <1Mbps. No issues with keeping the Transporter's buffer adequately full. However, a full length 24/88 song has a much higher average bitrate (>2.5Mbps) and this resulted in the need for rebuffering about every 15 seconds streamed across from Europe. Other than the unfortunate buffer under-run every 15 seconds, the song sounded excellent subjectively during playback.

Is this surprising? Of course not! Like I said last week, computer networking protocols are robust in terms of data error correction. When it comes to TCP/IP, internet, and ethernet data transmission, everything happens asynchronously and in a bit perfect way. This is what digital is good at - getting data transmitted perfectly with no degradation. And for asynchronous interfaces like ethernet and asynchronous USB where there is bidirectional flow control, there is furthermore no "clock recovery" process like in older digital interfaces like S/PDIF where jitter can be introduced in the source clock (remember, jitter is a property of the digital source and DAC themselves, not the cable assuming we're looking at reasonable cable lengths as far as I can tell). This is why the objective results look fine even though the server is a continent away. Even though the latency was high as demonstrated by the "paping" data above, it doesn't matter so long as the Transporter's data buffer did not "run dry" as it did when I tried playing a full song in high-resolution resulting in annoying pauses. Considering that both his and my ISP are usually able to manage specified speeds (100 Mbps down/10 Mbps up, and 50 down/5 up), it's unfortunate that I wasn't seeing such throughput during the test (maybe because it was ~9:30PM local time and about 6:30AM in Sweden on the first day of the Lunar New Year?!).

There are at least a couple important implications. First, this test again reiterates the idea (fact) that ethernet cables will make no difference. Why bother with expensive wires for the last number of feet leading to the player device if there is no evidence that data transfer over >7000 km using standard cables makes any difference when played back? Note that along the way, there has likely been a number of conversions between electrical conduction and optical lines. Even the most expensive ethernet cable will of course not speed up transfers and avoid buffer under-runs.

Second, there's no issue with jitter - bits are bits as it applies to asynchronous interfaces and so long as bit-perfection is achieved and data transfer rate is good enough for the local buffer, there's nothing to worry about. This is good as audio consumption gradually transitions to streaming services for many music lovers / audiophiles. The most that will happen are pauses from connection speed issues rather than qualitative difference to the sound during playback assuming the software isn't somehow messing things up!

(BTW - ever wonder why some audiophiles and reviewers obsess over a few feet of digital cable - especially blaming jitter despite really no evidence to show that it makes a significant difference? Yet these days, with the advent of digital streaming and advertising dollars from places like Tidal, nobody talks about the "potential" for jitter when data is being streamed from miles and miles away? Instead there's just isolated talk about ethernet cables this and that to sell expensive stuff yet no consideration for the truly big picture! Obviously there's something wrong with this whole cable "industry".)

A final thanks to Mnyb for "opening up" his LMS server for me to tap into and helping out with this test! Nice music collection you got there, buddy :-). I managed to stream a couple 16/44 FLAC songs off there with very minimal buffer under-runs (3 rebuffers/song rather than every 15 seconds with the 24/88 tune), and no rebuffering issues at all with 256kbps MP3 which still sounded great. (Don't forget the result of the MP3 test in 2013! High bitrate MP3 sounds excellent and still has a role to play when data speed is restricted or unreliable.)

Enjoy the music as we end off February 2015... Happy lunar new year for all celebrating!

18 comments:

  1. Now maybe i should pay for an 100mB uplink :) that would make streaming trouble free from where ever i'm at ,

    ReplyDelete
    Replies
    1. .. But it proves a piont everything is fine until it is not , as long as the buffer is full it works , no subtle degradation . Ethernet is a very different thing it's designed to work this way , even heavy data loss would only result in resends . Cheers

      Delete
    2. Let me know when you get that 100Mbps uplink... I'll make sure to stream a few tunes and add to your value! ;-)

      Cheers!

      Delete
  2. I love your blog but I think you don't need to spend your time measuring Ethernet cables (unless you are comparing a cheap one with a 10k on just to prove a point) or measuring intercontinental ballistic music streams. You can simply post links to HTTP over TCP/IP RFCs. These are well known, proven protocols that the entire world economy relies on. It makes no difference if the stream is served from my neighbor or from across the world, except for latency. This is why the thing we know as Internet, works.

    Cheers

    ReplyDelete
    Replies
    1. Of course Vijay...

      However, certain audiophiles *insist* that "bits are not just bits" and obsess over timing "issues" despite the fact that it's all buffered and reclocked. The only ways to demonstrate the fallacy of this belief is to demonstrate that the analogue output is not affected despite the latency numbers and distance traveled... Hence the measurements...

      Delete
    2. I understand and I have tried this on an "audiophile" forum by providing measurements. Measurements were rejected on the premise that they don't capture everything and that the human ear can perceive things the measurements can't. Faith is much stronger than fact and that is the unfortunate fact.

      Delete
  3. In a similar vein, someone I know who manages a small own-brand record label in the UK invited me to a recording session in a church several months ago. This friend and I often discuss the ridiculous claims of audiophiles regarding cables, and he pointed out that this recording session - with some 50 microphones, being recorded in 5.1 at 24/96 - just used "miles of plain copper cables." You'd think the audiophiles would also obsess about the cables used during recording sessions, but since they can't change that, they simply ignore them.

    ReplyDelete
    Replies
    1. Yes indeed. I guess the "audiophile studios" could advertise that they use Nordost, or AudioQuest, or Monster or whatever cables... But the majority I suspect would just be regular stuff or higher quality pro brands - eg. Mogami, Neutrik, Canare, etc.

      I've certainly had a number of friends "convert" from high-priced audiophile brands and happily switch over to Mogami for example and just recognize that their music was "made" with this anyways! I personally use Canare cables for my speakers as well as some Mogami AES/EBU digital cables with the Transporter. More than good enough ;-).

      Delete
    2. Same here, Archimago. I am lucky to Blue Jeans cable local here. All my wiring is from them and XLR stuff is from Mogami.

      Delete
  4. Very informative as usual Archimago.
    Kirkmc I couldn't agree more.

    ReplyDelete
  5. And my paragraphs-long comment got swallowed into the ether...

    Just to bottom-line it, until you perform double-blind _listening_ tests on 'premium' cables, you do not have the data to demonstrate that they can not make an audible difference on your system. Measurements are great, but you can only infer audible effects from the numbers. I have a _belief_ that ethernet cables do not affect music audibly, but do not have the DBT listening tests to state that definitively for my system.

    ReplyDelete
    Replies
    1. Is there anything for which you think double blind listening tests might be a waste of time? If a company suggested that reciting a $1000 book of spells over your CD player could enhance the sound, would you accept that as a possibility, and that until you had performed test you would not feel able to dismiss it? You could spend the rest of your life performing listening tests on more and more outlandish claims. Where do you draw the line?

      To an engineer who understands digital audio, a "premium cable" is just like that book of spells.

      Delete
    2. Further to "therationalaudiophile"s post, indeed we can do DBTs and such which of course would give us data as to whether the human ear/brain can detect something different that the machines don't pick up in measurements. But through experience, I can say that a good enough ADC (in my case even the inexpensive E-MU and I tried the TASCAM back in December) can show me things that I personally *know* I cannot hear.

      For example, that high frequency noise from the TASCAM recently picked up by my E-MU I would not be able to hear. I cannot hear a significant difference between linear and minimal phase filters used in upsampling. Nor can I detect the jitter from TosLink vs. coaxial vs. ethernet with my Transporter even though I can measure. (I can of course hear the wow & flutter anomalies with my turntable and test LP!)

      Furthermore, the blind tests I conducted here with MP3 vs. lossless and 24-bit audio - if I ran the test tones through these procedures, I would easily be able to measure the difference; yet people could not reliably hear a difference...

      All of this is to say that machine "hearing" and measurements can easily produce results around accuracy demonstrating more sensitivity than human ears/brain when put to the test. As such, when it comes to tests like this, I'd rather take an hour to measure than having to endure a DBT unless there's good evidence to say human ears are better than test equipment as it applies to what we're trying to study...

      Of course, if the question were one of "subjective preference" like between solid state amp vs. valves and we just want to know if most people prefer one to another, then listening tests is the way to go... But with these tests of *accuracy*, I think measurements are the way to go and if some audiophiles truly feel the results are in error, then they're welcome to respond with DBT or some other controlled testing to provide evidence!

      Delete
  6. Great article Archimago, thank you very much.

    So now I can be confident that the music I listen to here in Australia, which is downloaded from N America or Europe (or wherever), is identical to that same music you may download from the same source when you listen to it in BC. That's a relief, we don't need to add to the difficulties of intra-regional distribution.

    I'm still puzzled at how that last metre or two of USB cable, or Ethernet cable, which winds it's way round my listening room, could possibly reveal what 7,000km of cable couldn't mask?

    This digital malarkey sure has turned out to be a curved ball for audiophiles.

    ReplyDelete
  7. This comment has been removed by the author.

    ReplyDelete
  8. Nice post! I think i'm really OK with my 10 US$, 5 meter Ethernet cable :)

    By the way, i know that you already measured HDMI cables and various devices looking for jitter, but i think that it would be interesting to do a comparsion between a S/PDIF optical out from a regular modern PC Motherboard and compare that to the HDMI audio from a modern discrete videocard from AMD or Nvidia.

    Best regards! Keep up the good work!!

    ReplyDelete
    Replies
    1. Yeah, I'm sure the $10 cable is fine :-).

      I guess I could check out the jitter... However, my current receiver with both HDMI and TosLink isn't that great for jitter suppression of HDMI so I'm afraid I could just be seeing an idiosyncratic difference simply because of the Onkyo TX-NR1009 I have rather than demonstrating actual differences from the PC motherboard.

      One day, if/when I get a better receiver that actually advertises low jitter digital inputs, I will definitely do this test!

      Delete
  9. Cat5E & Cat5E patch networking patch cable basically use high functioning and durability.
    Thanks for the sharing.

    ReplyDelete