Saturday, 20 June 2020

MUSINGS/POLL: A lifetime of digital audio storage? Enterprise / Data Center hard drive update.


For this week's post, I thought I'd spend some time talking about music storage, a recent upgrade, and some thoughts while reviewing my own music collection. A potpourri of observations and ideas, ending with a little poll...

A few weeks back, I saw this "extreme" server system described on Audiophile Style. I certainly agree that a machine like this is very much extreme for simply audio playback or as a music media server (presumably it will not even be used for something more demanding like a higher bitrate 4K/HDR movie/video player)!

What got me thinking about my own server system was not whether one needs something like this for "good sound" - of course one does not. I thought it was very cool though that the machine is capable of such a large amount of SSD storage - up to 24TB. Basically, what they're doing is putting up to 3 ASUS Hyper M.2 X16 PCIe (~US$60-70) cards in the machine. The user can then install up to four M.2 SSDs within each card (a single 2TB WD Blue 3D can be had for about US$230 currently, 2TB WD Red for around US$280). So with each ASUS card filled up, we have up to 8TB SSD storage, and 3 of these populated ASUS card results in that 24TB potential.

In 2020, SSDs are still expensive so a full 24TB storage with 3 of these ASUS Hyper M.2 X16 PCIe cards and 12 of the 2TB WD Blue SSDs will cost about US$3,000. In the years ahead we will no doubt see lower priced, larger multi-terabyte SSDs (you can currently get 4TB M.2 SSDs like this Sabrent Rocket but at high price of US$720, or 8TB for US$2000 - ouch!).

I suppose in the future, SSD drives are the way to go. They have the speed advantage, smaller, lower power consumption, and should have great reliability over the years. The only thing that I have found worrisome with SSDs is that when they fail, they tend to go completely kaput with little warning; back-ups are mandatory as usual!

For the time being however, the best dollar-per-terabyte storage is still with hard drives.

I've been collecting CD's since 1988 - the first CD I ever bought was Rod Stewart's Out Of Order from a little long-gone music store at the local shopping mall paid with newspaper delivery money after school. Before then, I had a large collection of mainly 80's pop cassettes, many of these have been digitized into my library as well. Through the 1990's, my CD library grew as a bunch of used record stores opened; I remember hanging out after university classes at a long-closed store called Tramp's in Calgary, Alberta during those years. By 2000, I started buying SACDs and DVD-A's, many with multichannel content that over the years have been ripped and converted to 5.1 FLAC.

The process of bit-perfect ripping of all the digital media I had until that time began around 2001, completed by 2004. But of course this was not the end of the collection process by any means...

The music library expansion got even larger by the early 2010's as lossless hi-res digital downloads got going. I suspect like many audiophiles, I became curious about all the content available through HDTracks and went a bit wild with acquiring my favourites. Although I had a number of vinyl rips from friends prior to 2014, I finally got my own turntable and ripping set-up then, capturing the audio in 24/96 (because everybody claimed vinyl playback was "hi-res" :-).

These days, DSD is essentially extinct for new music, there was of course a brief time around 2014-2015 when the Industry tried unsuccessfully to revive this encoding system with DSD downloads primarily based on SACD's DSD64 but with some albums released as DSD128 and DSD256. I still have a number of .dsf in my collection from this time period which I've converted to PCM for compatibility (conversion is transparent, no sonic difference with good hardware).

Finally, over the years I have made recordings of live performances of friends and their bands as part of my library. Most of these done in 24/96 of course which provides latitude in the future for editing and resampling.

While I have been making sure the music is properly tagged (takes a bit of discipline I must admit!) with a consistent directory structure for various genres, as I survey the collection, this lifetime of music "data" collection has resulted in a hodgepodge of directories spread over 6 hard drives (3 + 3 backup) in my home network. I noticed recently when upgrading to Windows Server 2019 that some of the drives were getting rather old, including a couple of 2TB Western Digital Greens from 2010! Too disorganized for my taste and time to refresh and consolidate...

So I got these:


As you can see, these are a pair of 12TB Seagate EXOS Enterprise X14 (~US$300 each) drives. In theory at least, these "enterprise" drives meant for data centers are designed to be more robust than the typical desktop storage. They're faster than typical WD Red NAS drives and have a 7200rpm spin speed with 256MB cache. I'm seeing about 20-30% real-life faster copy speed. These newer generation drives also are helium filled, run cooler, uses less energy per terabyte (about 5W idle power usage for 12TB, up to 10W in use). While reliability is always a bit of a lottery with spinning drives, these come with 5 year warranty (starting from manufacture date so more like 4.6 years since mine were made in late 2019), and are rated at 2.5M hours MTBF (mean time before failure) which is 2.5x that of something like the WD Red hard drives.

For my Server machine, once installed, the main storage HDs are then a combination of these Seagate EXOS and a couple of 6TB WD Golds from a few years back inside:


A friend asked me the other day to compare the WD Gold vs. EXOS X14 drives. I didn't notice too much difference. The newer EXOS ran cooler overall, data transfer speed was about the same. Remember that "enterprise" / "data center" drives are meant to be robust but not particularly quiet! The EXOS X14 is a bit louder than the WD Gold with spin-up noise. Both have head seek noise more noticeable than with typical desktop hard drives from the last 15 years. It's probably best not to have these on all night in a bedroom and definitely no spinning HDs in the audiophile sound room. However, this kind of noise is nothing of concern compared to loud hard drives of yesteryear for those of us old enough to remember pre-Y2K SCSI models and drives in the 386/486 era :-). Anyways, the drive noise didn't sound any different between the Seagate and WD Gold once inside my computer tower case and was well attenuated in the Mediasonic 8-bay USB 3.1 enclosure.

This is the first time in more than a decade I've tried Seagate drives in my machines so hopefully they don't let me down!

A side note: I tried a 10TB HGST Ultrastar HE10 a little while back. I found it too much of a hassle with the SATA power pin-3 issue related to Power Disable on some models. It's a useful feature to remotely power-cycle a drive instead of manually sending someone to do the job in a data center, but of no real value on desktop or home server machines. Furthermore, for some reason, that HGST drive did not like my Server computer's motherboard SATA controller. It worked well on my other computer though but I returned it due to these hassles/compatibility issues.

Since the hard drive update, all my albums have now been consolidated into a single 12TB drive with the second drive acting as mirror back-up. All the old WD Green and Red drives have been retired to storage. The new drives feel slightly faster, certainly more convenient to have all audio data together, lower total power utilization, cooler, and probably overall also quieter (since each EXOS replaced 3 drives). Remember though that there is a benefit with smaller drives in that failure at least will not potentially take out as much data at once. However, if the future is massive SSD capacity, I just need these enterprise HDs to last the next 10 years or so until cheaper SSDs are available. :-)

In total, I see that I have amassed close to 7.5TB worth of music data over a lifetime, mainly as 2-channel FLAC, with some DSD/SACD rips, and a decent amount of 5.1 multichannel as FLAC also. Here's a peek at my Roon front panel the other day.


About 13.5k albums these days total on my local library, not including multichannel/DSD stuff which Roon isn't asked to deal with. In reality many of those "albums" are actually singles. Furthermore, some titles are duplicates since I have a few versions of Kind of Blue, Dark Side, Dylan, Beatles discographies, etc. (pretty typical of audiophiles, right?!). Notice that I had been fooling around with my Springsteen collection the day before I took that screenshot with some 16/48 resampling - we'll talk about this another time.

7.5TB is admittedly a lot of data. As someone approaching 50 years old, have really little interest in subscription-based streaming music (I can basically preview anything I want on YouTube already and prefer to just buy what I want!), and have noticed that little of the "new music" these days really interest me, album purchases have dropped dramatically compared to 20 years ago when I would visit stores and routinely pick up 10 albums a week easily. I suppose that's what happens as we get older, solidify our interests, and develop other pastimes, plans, and responsibilities. As I mentioned years ago, there's already a lot of music out there; easily a lifetime can be spent mining the good stuff of previous decades! I'm still discovering artists from the 80's even - for example, I listened to some Brenda Russell and Paul Davis the other night I had missed all those years ago.

Now that both my kids are teens and love listening to music, I notice that they have little interest in collecting music. Compared to when I was that age, neither they nor their friends seem to care about new album releases, have little interest in being "fans" of artists, and appear to be completely content with free online streaming options. Regardless, they're welcome to my collection of digital "dad music", CD/SACD/DVD-A's, and vinyl when I'm done with all this. :-)

BTW, happy Father's Day as I post this weekend.

Let's end off with a fun bit of interaction I hope!

Tell me, how much digital audio storage do you have? And below that, a similar question since one could have tons of storage but little music, currently, how much digital music do you have? I've added my votes already with 12TB storage (more like 11TB once formatted and all) and about 7.5TB of music (so "10-12TB" storage, and "6-8TB" music).

All data collected anonymously of course. Nobody's going to report you for an episode of A&E's Hoarders. As you can see, I'm already in rather bad shape! :-)


How much storage do you have for your digital audio (not including backups)?

less than 500GB
500GB - 1TB
1 - 2TB
2 - 4TB
4 - 6TB
6 - 8TB
8 - 10TB
10 - 12TB
12 - 15TB
15 - 20TB
>20TB!
Created with Poll Maker


Approximately how much digital audio data do you have (including PCM, DSD, hi-res, multi-channel, MP3, AAC, etc...)?

less than 500GB
500GB - 1TB
1 - 2TB
2 - 4TB
4 - 6TB
6 - 8TB
8 - 10TB
10 - 12TB
12 - 15TB
15 - 20TB
>20TB!
Created with Poll Maker

NOTE: I created these polls more than 2 weeks back so the number of days of voting should be subtracted by 17 days in the results.

20 comments:

  1. Thanks for, as usual, a very interesting article. I would take issue with one point you make...SACD/DSD is not extinct for new music if you listen to music for adults. :-) There are a number of European labels that produce new SACD recordings of old "classical" music and new, composed music for trained performers, as well. (BIS, Pentatone, Chandos, Channel Classics, 2L and probably other labels whose recordings I haven't yet purchased) I know that there are descriptions of how to rip SACD and Blu-ray music to files but I am a newbe at that and am reluctant to even temporarily reprogram my now irreplaceable Oppo for that purpose. I, for one, would greatly appreciate any recommendations on how safely to rip and convert multichannel SACD and Blu-ray to 5.1 flac files.

    ReplyDelete
    Replies
    1. Yeah, I see there are still SACD/DSD being released.

      I wonder, does anyone have numbers of how many total albums from companies like the ones listed above are still being released as SACDs. I see that downloads for example from Channel Classics can also be downloaded as FLAC/PCM.

      Of course, so long as we can still play the music, it'll never truly be "extinct". Until my last DVD-A player dies, it lives although these days for me there's no point spinning the disk.

      As for digitally ripping SACD, there are a number of ways. Old-skool PS3 (my brother still has one of these with old firmware; here's a HOW-TO: https://www.instructables.com/id/Introduction-54/), Oppo with ISO2DSD, and there are a number of devices which can use SACDExtract more recently:
      https://hifihaven.org/index.php?threads/rip-sacd-with-a-blu-ray-player.3652/

      As for conversion, although it has been awhile, I will take the ripped .iso SACD file, extract the 2.0/multichannel as .dff files. Convert either with foobar or Saracon to .wav. I'll have a look at the output to see if the audio is worth keeping as hi-res. Remember, many SACDs are just PCM 44/48:
      http://archimago.blogspot.com/2013/07/list-suspected-44-or-48khz-pcm.html

      Then from there might resample or convert to FLAC with dBPowerAmp typically. Sometimes it's good to check the conversion isn't clipping the audio or sometimes need to add a few dB if the audio is a bit soft. Don't know if there's a recent HOW-TO for this part.

      Delete
  2. Just like you did, I took a look at my collection when one of the NAS drives started to fail... did the math and decided that a Tidal subscription would be cheaper, and would come with the benefits as less hassle, better tagging, etc. Told my son (who's 21) to stay with Spotify, as managing a music collection is not worth it.

    After 3 months of Tidal I returned to my collection, mainly because reaching the music I wanted proved to be a bigger hassle!

    Maybe some combination of Alexa/Siri, Roon and much better streaming solutions might be what I need, but I think that 1/ the technology is not yet there (I've been streaming music for 13 years, and have been disappointed by vendors small and large in this space), and 2/ I'm too old and too well set in my ways to learn new tricks.

    My music collection is like my marriage: after a lifetime together, after miriad ups and downs, we are together because there's simply no better alternative!

    ReplyDelete
    Replies
    1. LOL In the Cold,
      Love the story and the marriage analogy :-).

      Yeah, there's a certain connection and sense of "commitment" isn't there with one's music collection? This collection of music has gone through life with me. The albums have represented my time on this Earth... From the artists I've loved, to the songs themselves. Some are tagged with the memories of decades; high school, prom, college, dating, marriage, even special songs that came out when my kids came.

      Before Tidal or Qobuz or Spotify or Google Music, I spent time on curating the collection and I guess in many ways see the collection as the universe of songs I've found to be of value in some way.

      Another of the ways we can appreciate and even "love" the music subjectively...

      Delete
  3. I thought I had a fairly OK size music collection, but it came to 206 GB, so I answered less than 500 GB on both questions. There's no chance that I'll need more in the foreseeable future. It might be too early to conclude, but it looks like less than 2 TB works for 2/3.

    ReplyDelete
    Replies
    1. Yup cfuttrup,
      I'm not surprised! Very large collections I suspect are the obsession of the minority. Another of the dimensions in which audiophiles can be "extreme" in.

      Over the years, I have suggested that audio people come in 2 variants... The "music lover" with their collections and range of artists, and the "hardware audiophiles" who collect all kinds of reproduction hardware.

      Perhaps there's some kind of ratio one can calculate of "albums owned":"DAC/players owned" to help define "What type of audiophile are you?" :-)

      Delete
  4. Hi Arch
    I only have about 500GB music data at the mo (still have a few yards of CDs to burn) that I store on a 1TB SSD on my media PC and backup all over the place.
    My main backup is a Drobo NAS with RAID (5?) where I also store my video files. I'm pretty happy with this set up: I have the silent, fast, and compact SSD and reliable backups for peace of mind. I wouldn't rely on SSD alone, nor would I rely on mag drives without RAID.
    I think this will do me for the foreseeable future. I'm buying less music these days: with Spotify at $10/month I can preview just about everything and only buy albums with really good sound (or surround) that I want to play often, usually in 24 bit format where available (even if I can't hear the difference.) If it's something mediocre in sound quality, I'll just stream it (not going to buy Desmond Dekker or vintage Furtwaengler in 24/96!). Also I'm not sure if I'd subscribe to a more expensive higher res streaming service like Qobuz or Amazon if they ever come here: current practice works fine and is economical.
    Cheers
    Phil

    ReplyDelete
    Replies
    1. Hi Phil,
      Sounds good. Love the backup RAID. I'm with you man. Indeed, if I do go with the monthly subscription solution, good chance that Spotify is all I need unless the services start doing more surround content (Tidal Atmos is a good start).

      Yup, be careful with 24/96 "vintage" music :-).

      Delete
  5. Hi,

    My music collection is close to 1tb, have it on an ssd and also bought the same HDD (12TB Seagate EXOS) mainly for my photo backup. Good, fast drive.

    ReplyDelete
    Replies
    1. Great that the EXOS is working out for you!

      My photo collection is on those WD Golds ;-).

      Delete
  6. Greetings. A long time fan here. My approach is a little bit different: I buy a lot of music but listen to it mostly on streaming services (yes,the same one that I already bought and imported to Roon). My reasoning is simple: I want to support the artist whichever way possible. Since Tidal provides me with wanted fidelity (though I'm perfectly happy with my Spotify also) I am able add another couple of cents to artists check without the smallest compromise.

    ReplyDelete
    Replies
    1. Hi Andreas,
      That's really kind of you and thoughtful support of the artists! Awesome, will have to think on that further!

      Delete
    2. I tend to listen via Qobuz and buy vinyl!
      Supports the artist and is an "object of desire".

      Delete
  7. Now that are some large HDDs!
    You forgot to mention that the music soundstage is now way more spacious. While it is widely known that a 7200rpm drive really lifts some veils, a true connoisseur should better strive for those 10K SAS beauties ;)

    On a more serious note, I inherited tapes and vinyl from my parents, but as my turntable was low grade, I was too frustrated with pops and clicks to even consider the medium.
    My "collection" started around '95 when I got my hands on some mp3's, and since then, it was/is exclusively digital.

    As I don't have time to search actively for new music, I subscribe for a streaming service and run it in background in my office. If my ears catch something interesting, i get it for my collection.

    The "best of" of my inventory ends up on a 32GB usb drive that accompanies me in my car.

    Thinking of it, it turns out that my main listening room is actually my car, as only there i can properly focus on the music during the commute.

    ReplyDelete
    Replies
    1. Whoa Turrican... A "car audiophile"!

      Hope you have a kick-ass bass system in there :-).

      Yeah, maybe some 15krpm SCSI drives from the good 'ol days might sound even more spacious! I'm sure with those, 2.0 music must sound like a live recording in a full concert hall.

      Speaking of family, I forgot to note that while most of the music is mine, a substantial proportion of the music also comes from family contributions. Growing up, my brother and sister also bought many CDs in parallel. They've contributed to certain genres like country music from my brother who lives in Texas these days and soundtracks from my sister. Also, I've incorporated CDs from my dad over the years with some of his classic pop and classical.

      While I listen to Roon here at home mainly, I still run a VM on the server with LMS as discussed here:
      http://archimago.blogspot.com/2016/03/howto-logitech-media-server-brutefirdrc.html

      Through that, I have access at work and even in the car if I want and the family to have access to their music as well, making the collection not just "mine", but also a "family collection". (Typically, I stream at 256kbps MP3 for the remote listeners to save data and more reliable as well.)

      Delete
  8. "About 13.5k albums these days total on my local library... As someone approaching 50 years old..."

    Archimago, do the math: if you live until 85, you've got 35 years (12775 days) to live.

    Even if you listen 1 album a day until the end of your life and never listen to the same album twice, your collection (13500 albums) is already more than you can handle, man...

    It's called an obsessive-compulsive audio collector's syndrome... I've got the same thing (3600 albums and counting, I am 46 y.o.).

    I was so hungry in my childhood for new music, the choice was limited, me and my friends listened to the same album dozens of times and we read in magazines, drooling, lots of reviews of the albums that we knew we were never going to get a hold of... This hunger still lives in me, I cannot stop collecting.

    ReplyDelete
    Replies
    1. Hey fgk,
      Tell me about it :-). I was reminded how bad this was when moving house a few years back and it took ages to pack up the CDs!

      Absolutely, the "hunger" lives. :-)

      Delete
    2. Oh my. and I thought it was bad enough with almost 9k albums and 7.5TB of music on my NAS. It's good to know there are others like me out there. Blame it on being a poor student while having very limited funds to purchase music. Now at 45, as the story goes, I will not hoard anything else BUT music...

      Delete
    3. Hey Fai :-).

      You're getting there - once you go past 10k, you know you've "arrived"!

      The question after that is... "Now what?"

      Enjoy!

      Delete



  9. I started reading about that Extreme audio PC until i got to where he said:

    "Overall each case takes multiple days to CNC mill, but the end result is simply spectacular."

    ... which is complete bullsh*t. So i closed the page.

    ReplyDelete