Here are the graphs direct from the survey site. Feel free to compare / contrast with those in Part II in the "Demographics" section.
95% males with 1 woman in the group. Not unexpected.
Average age calculated from the median age in each category: 45 years. Essentially the same as the 44 years old average for all 140 respondents.
24% musicians as compared to 22.1% for all respondents.
30% does audio "engineering" as compared to 24.3% for all respondents.
None of these who got all the choices correct identified themselves as an audio hardware reviewer. Since this was an optional item, 1 individual did not answer the question.
11/19 (58%) were Windows users (almost exactly the same as the full group at 60%). Again, we see external USB/Firewire DAC being used mainly (13/20 - 65%). The only thing that seemed quite different was a higher proportion of headphone users here at 12/19 (63%) versus about 50% in the total group. Note that there was 1 individual who got all 3 24-bit selections correct who did not go into detail with his reporting, hence the total of 19 since he did not click either the speaker options or headphone option.
Average cost of the systems (based on median price in each price category) used by those who got all 24-bit samples correct: ~$6600. This is lower than the $8160 for all respondents. It's a smaller sample so I wouldn't put too much into this... No evidence in any case that these folks on average had more expensive systems. Like in the main group, the $1000-3000 segment was most common.
The person who used the "$100-$250" system was running what looked like a basic Windows PC with some Sennheiser HD433 (~$25) plugged in. (Remember, I dissuaded folks from listening directly off the computer since often the on-board DACs are poor - worked out well in this case!)
The person using the "$50,000-$100,000" system included the use of a Squeezebox Touch --> Berkeley Alpha DAC, Cary CAD 120S tube amp, Revel Ultima Studio 2s. Not sure about the preamp function. Looks like a nice setup...
Other gear used in this subgroup as I browse through them: balanced cable Sennheiser HD800 headphone, Grado SR80 + Bose QC15 headphones, Parasound amp & preamp, PSB speakers, Senn HD800 connected to Woo Audio WA7 amp, Senn HD600 connected to AudioQuest Dragonfly 1.0, NuForce DDA-100 to KEF Coda 70 bookshelf speakers, Arcam rDAC to Arcam DiVA AVR280 receiver, KEF iQ30 bookshelves, Benchmark DAC2 HGC with Grado RS1i headphones, Wyred4Sound DAC-2 to Senn HD800. Certainly some excellent gear there but not really exotic.
So... How "confident" were they of the answers picked?
In summary, this demographic doesn't look very different from the general group of all respondents - perhaps proportionally more headphone listeners. Remember however that I was not able to find a significant improvement in general accuracy of identifying 24-bit audio comparing all the headphone-using respondents in the overall group.
Of interest also was the fact that the average cost of the hardware used by these respondents was no higher than that calculated for the total group.
Although this subgroup was accurate in their choice of which sample was the 24-bit audio, confidence overall remained relatively low with 20-25% admitted to being "guesses".
Part III: SUBJECTIVE COMMENTS & FINAL THOUGHTS
On behalf of the 'I heard it correctly' group (I wasn't amongst them, not surprisingly) I would say thanks for making this subcategory.
ReplyDeleteJudging from the confidence level, about half of the 'winners' were more than just confident and the other half admitted they were more or less guessing, or at least unsure and guessed correctly 3 times. (not that implausible)
So out of the select 19 that's about 10 of the 140 that were pretty sure they heard it correctly AND actually heard (or guessed) correctly as well. Still a group to be reckoned with.
Good news is (when indeed this group heard correctly) you do NOT need very expensive audio gear to be able to hear these (to most inaudible) differences AND it seems USB DAC's appear to be good enough to display those differences.
Hooray for 'bad sounding' USB interfaces as most golden-eared subjectivists already found that USB, and USB cables sound really bad and need lots of conditioning before it even sounds acceptable.
One other thing to contemplate when reading hardware reviews on the Internet, and I suppose magazines as well, is that none of the participating reviewers had 'better' hearing abilities than the select group of about 10/140.
Now … what would be interesting is to test these contestants with a larger amount of, to be identified correctly, tracks and would recommend the Bozza track for this as that seemed to be the one that gave the impression to be the most 'audible' one.
The question is which these 10 'diamond eared' (as golden eared obviously isn't enough) contestants would pick them all correctly (without cheating) and have officially 'Californium 252 ears'.
Those that can pick them reliably could be valuable subjects to help improve audio reproduction and should be recognised.
On average I don't think 16 or 24 bits is that relevant for the majority of people (and music lovers). After all those last 8 bits can only describe signals < -90dB more accurately which would be probably be lost in the background noise of even the best recordings anyway.
IMO those extra 8 bits are highly overrated and may take up more space on your storage medium than they do good. At least that will be true for the majority of people.
Yup.
Delete:-)
This comment has been removed by the author.
ReplyDeleteYup. MkII :-)
ReplyDeleteSolderdude Frans you must be really beloved on every "more subjective based forum" as you said :-)
As for the *diamond eared* I'll just copy/paste/edit my comment on Archimago's previous post:
"It would be nice if some of the "golden 20" especially "four & five stars", writes a few words (Stephen Ward thnx for your comment) about the procedure of listening and maybe point out few details about the differences between samples, or share some tips and tricks.
Maybe I have "Golden Ears" but don't know how to use them."
Seriously, anyone?
I'm not sure I'm qualified to reply. Only on one of the tracks was I reasonably confident of my answer (Goldberg Aria) and statistically speaking that confidence is no proof of anything! Any subjective assessment of perceived differences is meaningless unless I can repeat the result many times over, under the same test conditions.
ReplyDeleteGreat test and highly interesting outcome !!
ReplyDeleteI'm not sure what my choices were (can't find the piece of paper on which I wrote down the result :-(), but after rerunning the test for myself (and not looking for the right answers), I assume I answered B-B-A. I find the second track the most difficult to hear any difference, so I guess that one made me hesitate the most during listening. Do I have "almost" golden ears? No, certainly not. My hearing is already decaying for at least a decade or so.
I bought a couple of 24-bit albums in order to experience if I would hear any difference. Some sound different, but I don't know if this is caused by the way of (re)mastering or some other process that makes the sound different between 16-bit or 24-bit. It's a bit like the old discussion between LP and CD: some hear much difference and some don't (or don't bother). I am a frequent record-listener, but not because I don't like CD (I have lots or them) or streaming audio. I also don't think that LP is the better medium for music (although a least you're listening to infite-bit recordings ;-)).
Music is all about emotion and listening to whatever kind of medium should be irrelevant. A relative of mine is a concert-pianist with a very nice sounding grand piano in his home, but his stereo-equipment is from the Stone Age. But his way of listening is so much different from mine (I'm just a music-novice).
Did the test prove if 24-bit is irrelevant? Technically speaking I think it did, but when recordings are made in the best way to let us hear the difference and our home equipment can handle it well then it makes sense.
The whole 32/24/16 bit and 44.1 to 352 sampling rate (as well as various DSD levels) debate is a bit silly and irrelevant.
ReplyDeleteIn the end does it really matter if SOMEONE ELSE gets more enjoyment from hires files and yet someone else is perfectly happy listening to the same music on MP3, another (lossy) format, a radio broadcast, or vinyl ?
In any case, those that care about these (undeniably) higher resolution recordings, which may sound better, should buy them.
Certainly if it brings them more listening pleasure.
There is nothing wrong with wanting the best possible 'something' for your whole reproduction chain nor with listening to lesser quality (MP3 or analog) for that matter.... IF that rocks your boat....
I see VERY good reasons to record in the highest possible quality and master in it as well.
IMO it is not needed to reproduce that final result in a similar high resolution... in MOST cases, but also see NO reason to not make this higher quality format available to the public that WANTS... no.. DEMANDS this studio quality.
With 'most cases' I mean there is little point in listening to those high resolution files on lo-fi/mid-fi transducers. That would equate to taking a picture with a 50 Mega-pixel camera and printing it on 10x15cm paper on your (non calibrated) $50.- colour printer.
IF someone is really pleased with hires files and feels it is well worth the effort/price I am all for it.
I don't see it as a scam but as an opportunity for an extra market to serve those that ask for it.
IF they feel the need to publicly express their appreciation for it and state it sounds soooo much better to them... then I AM happy for them but personally will spend my money on other things instead.
Archimago's test did show that for the majority of people that took the test (130 of the 140 people) it really didn't matter if they listened to the 24 or 16 bit version regardless if they felt they could tell them apart.
That doesn't mean an album in hires sounds exactly the same as another 'copy' on CD, MP3 or vinyl as the mastering quality CAN (and often does) differ.
This better mastering quality may be the best reason to listen to those hires files, because they MAY be mastered 'better' OR your mind is 'eased' by knowing you are listening to the best quality available.
If you feel the need to save some storage space by down-sampling THAT good sounding master or compressing it (lossy or lossless) and cannot detect it yourself than I see NO reason to NOT listen to that 'lower quality copy' either.
Nor should anyone else feel that I must/should care, just because they do.
I certainly hope nobody else minds that I can thoroughly enjoy listening to 320kbs MP3 just as much as a 24/96 recording IF that album/song happens to be recorded very well.
Live and let live... . enjoy YOUR favourite music in the quality YOU prefer, played on YOUR beloved equipment in YOUR favourite format is all that matters in the end. Not what your neighbour or someone else on the other side of the world thinks/feels YOU should be owning/trying/buying/testing/hearing.
Yes. Some good points there Frans.
DeleteUltimately, of course it's a free world and anyone can do what they want with these hi-res offerings. As I expressed in Part III, for me there are clear issues with "value" and how this is defined which only the individual music listener/consumer can answer for him/herself before they hand over that credit card number.
The test I hope provides some useful information for consumers about "real-world" expectations rather than the usual advertising spin. In all these years of reading the audiophile press (print & the usual online news sources), I do not believe I have ever come across a decent article questioning/discussing the value of high resolution downloads (despite the fact that there are some very clearly questionable offerings out there). At least a publication like Hi-Fi News and Record Review will test out hi-res music downloads in their reviews. The reason for this is quite obvious I suspect in this day when so-called "reviews" read like ad copy. The only place where anyone ventures to question if the "emperor is naked" is in independent blogs or forums where the residents dare to question or test for themselves. Studies like that Boston Audio Society experiment is also good to look at.
I truly hope that in the days ahead, the music producers can produce albums catering to what audiophiles really want - better sound. To not hang their heads on silly beliefs (Cookie Marenco's WAV sounding better than FLAC for example) and specs like 24-bits, 192+kHz sampling rate, DSD, etc... Consumers clearly need to look below the surface where true value lies just like any other purchase; not how big and shiny the "box" looks (oooohhh, 24-bit box... shiny chrome DSD detailing...).
I was in the 20 (Dragonfly 1.0 and Sennheiser HD -600) and learned from this test that the difference between 16 and 24 bits is small. Lately I have been moving away from hi-res audio to streaming (MOG/Beats). I'm buying only the best sounding albums which I really want to see in my iTunes library.
ReplyDeleteNice Olah. And good job on the test!
DeleteApart from some vinyl collecting (more on that in the days ahead!), I too have been exploring musical landscapes with streaming...
I see you mentioned one of the bookshelf speakers KEF iQ30 bookshelves, but the link is incorrect. It must have been outdated.
ReplyDelete