Home Network: Update on mixed 10 gigabit network, Roon stability & value? (And a quick look at Pi-Hole.)

Happy 2023, dear audiophiles!

While I typically publish articles like measurements of audio hardware, discuss albums (usually as part of reviews or examination of things like dynamic range), and offer critiques about extreme "High End" audiophilia, every once awhile I'll talk more generally about my computer hardware and local area network (LAN). I see the network stuff as part of a broader foundation of the modern digital audio system which simply has to work reliably and efficiently for digital streaming/computer audiophiles. Like other parts of the "perfectionist" audiophile system (like say the room which is often also neglected in audiophile hardware talk), this should be optimized for best performance such that those using it will not have to suffer from speed issues or dropouts. Sometimes it's easier said than done given potential interoperability nuances of the multitudes of network hardware!

Apart from silly "audiophile ethernet switches", or "audiophile ethernet cables", there's generally nothing specific to sell to audiophiles when it comes to home network hardware. As a result, it's atypical to see long articles at audiophile websites or in the magazines about this stuff despite the importance.

As a reminder, in my experience, unless we're dealing with faulty gear, "Bits Are Bits" when it comes to sound quality - that's just reality if you take the time to analyze the audio signal objectively or run some controlled listening trials. Despite all kinds of subjective voices/claims out there otherwise, high performance modern computer networks are simply not going to change the sound of audio playback so long as they're reliably "bit-perfect". After all these years, I find no reason to be concerned about ethernet cables, or switches transferring packet data around. Whether the data is of audio content or otherwise is irrelevant - the network doesn't care whether it's audio meant for the DAC or pixels aimed at the printer.

Digital data, given how it's packaged and transmitted, when corrupted will result in rather obvious audible anomalies if not corrected (like an error-prone USB cable). You will not hear subtle changes (like "more bass" or "better soundstage" or even more vaguely "improved presence") as some reviewers seem to promote when hyping nonsensical claims of what they supposedly "heard".

Sometimes, I publish articles like this one today as a kind of journal entry for future reference as the system here at home evolves. I want to document settings on my home network that currently "works" very well not just for general family computing, or file serving, but also stable for Roon streaming which has been an intermittent annoyance over the last few years getting to work without glitches whenever I change network hardware (we'll talk more about Roon later).

Consider this article a case study of a specific network system. Perhaps some of what I discuss here and the settings can be helpful to you and your set-up.

My most recent article about the home computer network focused on improving the stability and issues I had with the ASUS XG-C100C 10GbE network cards (Aquantia/Marvell AQC107-based chips). Since the last major update to my network in April 2021, I've made a few further hardware tweaks and you can see the current network map in the image above. It's actually not a very complicated network even though I do use a few things like the "VPN Fusion" feature of the router for IoT devices. I suspect if you map out your own home network, it might also look quite complex as well! Thankfully, most devices these days are "plug-and-play".

The multigigabit QNAP QSW-M2108 managed switch in my basement electrical panel sits right at the heart of the home network and is the main departure point between 1Gb, 2.5Gb, and 10Gb-compatible devices. Since wired ethernet is simply much more reliable, I've made sure that all the main rooms have access. It was a pain, but well worth it, to tunnel the ethernet cable into and across my basement "man cave" when I first moved into the house.

The ASUS ROG GT-AX11000 wireless router sitting centrally in the main floor of my home remains a phenomenal performer with good WiFi connectivity and I can't imagine needing anything more at this time.

These days, 1 gigabit ethernet systems are ubiquitous and should not be challenging to set-up. However, over the last while I have been tweaking the settings for my 10GbE system and figuring out the best parameters on my ASUS (AQC107) network cards, QNAP switch and the ASUS AX11000 router. Here are my current settings achieving both speed and stability:

ASUS XG-C100C (Aquantia/Marvell AQC107) Network Card:

See previous article for more details like firmware and driver versions. I updated some settings over the Christmas holidays:

Downshift retries: Disabled
Energy-Efficient Ethernet: Disabled
Flow Control: Disabled
Interrupt Moderation: Enabled
Interrupt Moderation Rate: Adaptive
IPv4 Checksum Offload: Rx & Tx Enabled
Jumbo Packet: 9014 Bytes (make sure switches and router compatible)
Large Send Offload (all IPv4 and IPv6 options): Enabled
Receive Buffers: 2048 (max 4096 in multiples of 8)
Maximum number of RSS Queues: 8 (max 8, correlate to number of CPU cores)
Receive Side Scaling: Enabled
Recv Segment Coalescing (IPv4 and IPv6): Disabled

Speed & Duplex: 10 Gbps Full Duplex (no need to auto-negotiate if we know it works)
TCP/UDP Checksum Offload (IPv4 and IPv6): Rx & Tx Enabled
Transmit Buffers: 4096 (max 8184 in multiples of 8)
Log Link State Event: Disabled

Using jumbo frames of 9k, going beyond 512 receive and 768 transmit buffers did not result in further speed increase or significant change in CPU utilization so the buffer settings above are higher than needed. Just make sure your switches and router are able to handle jumbo frames and likewise ideally all the high speed endpoint devices. Otherwise, the system will fragment packets when necessary which would reduce efficiency somewhat but it all still works fine here. By the way, the standard CRC-32 error detection algorithm loses efficacy with higher MTU (jumbo frame) values, don't go too high and stick with 9k bytes which is basically the "de facto" value these days beyond basic 1500.

For those wondering, RSC (Receive Segment Coalescing) is a mechanism to offload some of the data reception to the NIC hardware. In tests, I can see about 1-2% reduction in my Server's Intel i7-7700K CPU load if I turn this on; no big deal to leave off as it can improve stability with these AQC107 cards (interestingly, this feature seems stable at 9k jumbo frames so I might try leaving it on for a bit).

Notice that I've left Energy-Efficient Ethernet disabled for the 10GbE network. Speed consistency was better with EEE off especially on the Workstation-to-Netgear GS110MX leg. Also at times, EEE seems to throttle the speed a little bit (just ~5% in my tests). With the speed of 10Gbps, it's unlikely one needs the full bandwidth so it's OK to turn this on from a power-to-performance balance (just make sure you don't run into stability issues).

QNAP QSW-M2108-2C Managed MultiGigabit Switch:

RSTP - ON

LLDP - ON

IGMP snooping - ON

     Multicast flood blocking - OFF

     IGMP querier - ON

     Make sure to select the port leading to router

Port Configuration → Flow Control - OFF (let TCP manage itself)

Port Configuration → Speed - fixed at "10Gbps FDX" for the high speed ports

No issues with these settings on the QNAP affecting 10GbE stability. Very sporadic glitching affected torture testing if I set speed to "Auto" with EEE on - I wonder if occasionally it temporarily wanted to change speed for power saving?

ASUS ROG GT-AX11000 router:

Turned on IGMP helped with Roon stability. And also make sure to turn on jumbo frames support the same as my NIC cards:

Make sure to check out Roon's "Network Best Practices" page to see if they have any recommendations for your specific device. The ASUS router runs smoothly using the higher speed 2.5GbE port connected to 10GbE switch.

Windows (10, 11, Server) OS:

For completeness, you might want to specify in/outbound rules in Windows Firewall for Roon specifically.

Kept open private inbound UDP 9003 and TCP 9300-9400 ports.

Test time!

Let's test the speed and stability with iperf 3:

Transfer rate over 5 minutes to examine maximum transfer speed. An average of 9.9Gbps over the 10Gbps system - can't complain about 99% of expected!

Of course, a short 5 minute test isn't indicative of real-world robustness! Let's run a "torture test"...

Excellent stability over 12 hours (43,200 seconds) running at full-tilt (10Gbps transfer between Server to Workstation computers) with 2 concurrent Roon lossless streams playing across the network plus playing a 4K/HDR10 movie with 0 errors/dropped frames on repeat through that time. In total, >48TB of data transferred between the 2 computers during those 12 hours (not including the audio and video streams).

I can look at the QNAP router's error statistics - port 10 is my computer located across the house and upstairs which is the most distal 10GbE connection to the QNAP (thus most prone to transmission errors when sending all that data):

That's an average of 1 packet error every 0.65TB of data or so, running full speed for half a day. Nothing to be concerned about through error-corrected TCP connections.

Finally for completeness, let's see what UDP (User Datagram Protocol) "connectionless" transfers look like:

Server computer (iperf client): "iperf3.exe -c 192.168.1.10 -u -f g -t 600 -b 200M -i 60 -l 16k". Workstation computer (iperf server): "iperf3.exe -s -f g -i 60".

200Mbps UDP stream over 10 minutes sent as 16kB "datagrams". As you can see, 1 datagram got lost out of 915,000. Looks good, but it's a reminder that UDP has been called "Unreliable Datagram Protocol" compared to the error-correcting TCP. The above test was performed also while Roon was streaming to a couple of endpoints from the Server computer and I'm playing a 4K/HDR10 movie from the shared Samba drive.

These days, we see UDP streaming being used for things like real-time gaming, VOIP and Zoom calls where speed is more important (eg. gaming) but noticeable errors (like 5+% losses resulting in voice dropouts and distortions when making phone calls) would not be disastrous. Furthermore, 200Mbps as tested here is extremely high for these kinds of transmissions. Modern audio-video streaming, where we expect a better quality experience like YouTube, Netflix, even Spotify use TCP these days. [For context, a 4K/HDR/lossy multichannel Netflix stream typically is <20Mbps.]

I find it useful to look at UDP data as a way to examine the quality of one's network since unlike TCP, we can actually see how much error is present in the form of lost packets which can be from unreliable hardware or network congestion. Within a wired LAN, a reliable network should show <<1% loss and low jitter (<<1ms) assuming not too much else is loading the network at a reasonable speed like 50Mbps. Amount of dropped datagrams can be very unpredictable at higher speeds. I noticed more dropped packets when running Oracle VirtualBox for my LMS VM with "Bridged Adaptor" sharing the ethernet on my Server computer. Also expect higher losses with WiFi.

Finally, a quick test of transfer speed at MTU 1500 "non-jumbo frame" then turning on 9k jumbo frames from my Server set to MTU 9k from one of the other computers on the network:

As you can see, the larger frame size resulting in less packet fragmentation at 9k improves throughput of the 1 gigabit transmission by 40Mbps - just 4% change. Nothing scary these days and it all happens in the background without functional issues the user would notice.

Overall, I'm quite happy with the performance and stability of the network system. Even the often finnicky Roon seems happy. Happy enough until the next time I upgrade something at least. ;-)

Quick look at Pi-Hole...

By the way, you can see in my network map that I put an old Raspberry Pi 3 to good use as a Pi-Hole DNS server. This provides an extra level of protection, control and potentially improved speed as a local DNS cache. It's also got a nice GUI to monitor activity over time.

If you don't want to use your ISP's DNS server, 9.9.9.9 (Quad9) is a good choice with higher security. Fastest public DNS is Cloudflare's 1.1.1.1 currently but check on the speed rankings over time! Alternatively, for the power-user, measure with Namebench yourself.

Very cool looking Star Trek theme BTW. The low activity hours to the left in the bar graphs above were to examine my Server computer running Roon and LMS to make sure nothing suspicious was happening. Most of the DNS requests were originating from Roon Core. Activity increased as I pointed all the DHCP devices to the Pi-Hole at 10:00 and at around 18:00, I activated advertising/malicious website blocking at which point we start seeing the red bars indicating blocked queue requests (see top bar graph).

During the test period, ~11% of the DNS requests were satisfied within the cache. On average I bounce between 10-20% cache hit rate among the users here at home.

Every once awhile, it's also a good idea to have a peek at some of these blocked DNS queries and see which device within the network they're coming from.

Explanation of Type designation.

It looks like most of those are originating from web surfing activity or phone apps like Whatsapp. The only real surprise has been the high number of DNS requests from the AppleTV 4K "phoning home" even when it's supposedly asleep! Interesting...

Final little tip, I underclocked my Raspberry Pi 3 as per the "CRAAP" settings (edit config.txt on the microSD card). Dropping "arm_freq = 800" along with some under-volting reduces the CPU temperature by about 2-3°C and I imagine running the box 24/7 will save a few watts over time. Even with underclocking, I've rarely seen the CPU load go above 0.5. DNS transactions are slow (<20ms would be quite fast already) and only about 500 bytes are communicated each time so even the Pi 3's 100Mbps ethernet speed would be more than enough.

-------------------------

To end, I thought I'd offer a few thoughts about Roon now that I've been using it for a number of years, tested the recent additions to the software, while observing price changes.

1. I still like the GUI and think it's "best in class" as an audiophile music player. I enjoy Roon Radio, the deep metadata is excellent even though one better have quite a powerful server with good amounts of storage. Performance on my Windows Server 2019 machine with i7-7700K CPU, 32GB RAM remains excellent with >15,000 albums currently (>200k tracks) accumulated over a lifetime in the library. I have been using Roon essentially daily for years now.

2. Roon ARC works although not without occasionally unexplained halts and finnicky network performance issues. It complains of "poor connection" at my office even when all I'm asking is for it to stream at low-bitrate Opus with "Bandwidth Optimized" (~128kbps) setting. I've had situations where it refuses to play through the office WiFi to my phone (but okay through 4G data), yet will run without complaints in BlueStacks Android emulation on Windows. Alas, I don't have the time to troubleshoot whatever might be causing these episodic anomalies. I trust over time bug fixes will work things out.

Well, when it works, ARC is great and for the most part, I have no problems playing to multiple devices concurrently.

3. Have a look at the top music streaming services - at least with the data we have up to 2021 globally:

From BusinessOfApps.

While I'm sure there will be some shifting of the pie-chart wedges in 2022, Roon needs to get on that pie chart with support for Spotify, Apple Music, or Amazon Music if there's a desire to broaden user base. The top 3 has command over 65% of the pie. I know this is going to be difficult since the music services are unlikely to want users to access an external GUI like Roon, giving up the opportunity to push content to their users or making in-app purchases. Furthermore, we don't know how many Roon users there are, but I suspect it's just a drop in the bucket for the music services.

IMO, among the streaming services Roon supports, it's sad that Tidal went the way of MQA all these years (which for the informed audiophile implies a loss of potential hi-res fidelity) despite being one of the first to stream lossless 16/44.1. And Qobuz still is not as broadly available around the world (including Canada). Tidal & Qobuz are fine services for Roon to get started with but music streaming has evolved and it's a shame that Roon has not been able to partner with the leaders in the music streaming space. I still hold out hope that at some point Tidal might dump the expensive MQA tier and get with the program with true lossless 2-channel hi-res, ideally adding more multichannel/Spatial Audio/Atmos (lossy) streams.

4. Roon - what is it with all these network hassles?! If you do a Google search on Roon and network issues, notice just how many people have had problems with playback instabilities, unexpected stops, possible router compatibility issues, DNS hammering with metadata updates, etc. Sure, we can blame the end-user's network system in many cases, but when it seems like many users are experiencing issues with otherwise well-performing networks for other applications, perhaps Roon should look a little more into its fault tolerance. Nothing is 100% reliable so little glitches will happen here and there and it would be nice for the software to be able to handle small bumps more gracefully.

As an obvious example, notice in the Roon Networking Best Practices guide, they mentioned the D-Link DGS-105 unmanaged switch as one that could be problematic because of the QoS feature. Those little metal switches are quite popular and have been available since 2011 and I presume the vast majority of users have no issues with them. So if Roon has problems, isn't it more important for Roon to figure out why and fix their software to be more compatible? Otherwise, maybe explain the bug so D-Link can fix the problem, instead of just saying the hardware "may cause issues".

Beyond network glitches, I really hope Roon is careful about making sure that they don't make unforced self-defeating choices. For example, the recent revelation that Roon 2.0 requires continuous Internet access to even just play from one's own library is silly and the excuse about having 2 search engines likewise is a stretch. It's not unreasonable to suspect that the "need" for a constant Internet connection has something to do with licensing or security checks rather than placing the interests of the users first. I for one don't use Tidal or Qobuz so if/when my Internet Service Provider goes down for a few hours (which happed for about 24 hours last year after a storm), the idea that I cannot play my local music from the Roon streamer is just silly. Even if I don't have some fancy search function (simple search of titles and artists is all I need most times anyways), shouldn't I still be able to at least access my playlists? Terrible decision on Roon's part and I don't think this would help increase sales for those sitting on the fence.

Another thing I wondered about was whether in 2022, RAAT should still be transmitting audio data without any form of lossless compression*. Given the amount of computing power available these days in tiny devices, lossless decompression would not be taxing and could help those who are using Roon over WiFi. Since the RAAT protocol is said to be extensible, perhaps this could be easy to implement. For example, low-computational demand FLAC-like compression should save 25+% PCM network traffic - Squeezebox/Logitech Media Server endpoints have been able to natively decode FLAC since 2005. In fact, why not make the lower bitrate ARC modes like "Balanced" (256kbps Opus) and "Bandwidth Optimized" (128kbps Opus) available in the home as well just in case one is trying to stream to a device located where there is less-than-optimal WiFi signal strength?

For the same reason, I wondered why for some devices like my WiFi Chromecast Audio, Roon doesn't allow me to limit the playback bit-/samplerate to 16/44.1. Even though it's placed close to my WiFi router, I notice that it sometimes will choke while playing 24/96. These days, I hope Roon recognizes that rational audiophiles are likely perfectly happy about 16/44.1 being more than good enough for most situations, and lossy high bitrate encoding (like 256+kbps MP3/AAC/Opus...) sounds very good. We're not always streaming to the high-fidelity soundroom so don't necessarily demand the best quality playback at every Roon endpoint in the home. Sure, go ahead and warn users that changing settings could result in "lossy" downsampling; no biggie.

DSD lossless compression would be more complex and processor-demanding so maybe just leave it uncompressed for the likely few users that care about DSD.

Some level of compression would also help multichannel transmissions. And then there's the question of whether extensions may at some point allow us to bitstream Atmos to a compatible decoder/receiver; that would be nice! (Even better of course would be the ability to decode all channels including height channels!)

[* As you may recall, a decade back, there were many audiophiles saying that lossless compression "sounded different" between formats like ALAC, FLAC, APE, WV, etc... A fallacy and "urban legend" of course, not hard to disprove with some objective testing.]

5. Roon price increased in 2023. As of January 1, the price of Roon has increased to US$12.49/m billed annually, $14.99/m billed monthly, and lifetime $829.99.

Well, I know prices of many things have gone up but the $12.50/14.99 monthly price will make Roon equivalent if not more expensive than Qobuz $13/month, Spotify $10/m, Apple Music $11/m, and Amazon Music at $9-10/m. Tidal HiFi is $9.99/m, and the poorly priced Tidal HiFi+ lossy MQA stream costs $19.99/m if you feel charitable. Since Roon doesn't actually provide a music library to listen to, I think for most music enthusiasts, this automatically pushes Roon into more of a "luxury" tier than a product that competes on the basis of value.

As usual, I'll let you determine whether the price reflects adequate value in your use case. I'm glad I got my lifetime membership for Roon back in 2019 at US$499; not cheap at that time either as audio playback software goes - but look at that 66% increase in less than 4 years! Has there been 66% increase performance or functionality? Given the history of technology getting faster and usually cheaper over time, this kind of price increase is quite unusual. I don't think my monthly Netflix rate or any of the music streaming services have increased like this.

At least when it comes to the large increase in the lifetime price, it's a push to have more people stay with the monthly subscription which over time could be more lucrative for Roon assuming they're still around in 2028. I personally would not be interested in a monthly subscription for playback software like this to access my own music library.

--------------------

One last thing about ethernet I want to mention is that (ironically) I get the sense that subjective audiophiles seem to like big numbers even though they profess to only assessing quality "by ear". ;-)

What's with claims that Cat 7 and Cat 8 cables "sound better" for example? Yeah, I know companies like Wireworld and AudioQuest will happily sell these grades of cables at a mark-up (unclear sometimes if these cables actually conform to the standards). As you can see in the picture above, I've got a few Cat 6A, 7, and 8 cables here (some tested awhile back) which are "Shielded Twisted Pair" (STP) designs with a metal shield around the wires and at the connector ends. In contrast, the top cable I'm holding is a simple Cat 6 "Unshielded Twisted Pair" (UTP) cable. That Cat 6 UTP cable is already capable of transmitting 10GbE.

Cat 6A, 7 and 8 cables are more expensive, thicker, and heavier. Yes, they're also capable of higher data rate with Cat 7 pushing 10GbE over 100m (vs. 55m with Cat 6) and certified Cat 8 able to push 25/40GbE over 30-36m - good luck finding network hardware, and make sure to double check on whether the cable's legit!

Realize though that with the STP cables, you have to make sure that grounding is done properly, otherwise it's possible that you could run into or create higher ground noise through your computer/streamer which would not be a good thing. I see debates about whether the grounding should be done on one end or both. In testing, I've actually found that a thinner (easier to work with) and less expensive 15' UTP cable can achieve subtly less speed fluctuations compared to an equivalent 15'  Cat 7 through my Ryzen 9 Workstation computer connected to the 10GbE Netgear GS110MX switch. This is why these days, unless I have a good reason, I'm happy to use Cat 6 UTP for most home situations that don't need more than say 25m.

Unless you live under a radio transmitter or insist on running your ethernet cables beside the washer and dryer ;-), I'd recommend testing to confirm improved performance before spending a lot on ethernet cables. If in doubt, best not to waste money.

Alright folks, that's all for now.

I think 2023 will be another interesting year for audiophilia. I'm curious if the anticipated decline in advertising budgets especially in the technology and digital marketing spaces will have effects on the glossy magazines, ad-supported websites and YouTube channels in the small audiophile niche over the next while. With recent articles like this and this, I see it as recognition that the health of the "High End" industry is not particularly encouraging at least here in North America. Perhaps totally justified. We will see.

Again, I hope you're all having a great 2023 so far... And enjoying the music of course!