Part II: RESULTS & CONCLUSIONS
Part IIa: The 20 Correct Respondents
Welcome to Part III; the last in my reporting of the 24-bit vs. 16-bit blind test. In this segment, I want to spend time looking at the subjective differences as reported by respondents to the survey. I also want to spend a few moments offering a few personal observations and thoughts at the end.
I. Subjective "differences" between 24-bit vs. 16-bit audio?
First, I want to present verbatim the comments I received when the respondents were asked to describe what difference they heard between what they believe as the 24-bit and 16-bit samples:
| soundstage, instruments and voices in 3d space, edginess of crescendos, how relaxed I was in listening to the music |
| Depth, instrument brass and breathing of the voman |
| nothing specific - overall clarity and air and space |
| The 24-bit tracks sound more realistic, but the difference is very small. Some tips would be to use the best equipment you can, and do not listen for too long at a sitting. |
| Space, separation, intensity of instruments |
| Much more clarity and width/depth |
| Small parts (10-15 secs) comparison. 24bit was to me which sounded the more colorful |
| Clarity of sound |
| Dynamics, detail. In my experience if tracks are produced and mastered in the same manner there is no significant audible difference in audio format. Differences in producing and mastering are what makes the difference. |
| Do not listen to a section of any more than ten seconds. Use good headphones and a good DAC. |
| Detail, space. |
| Tried to listen for the usual audiophile adjectives, but was ultimately unsuccessful in picking out those qualities. |
| Use headphones absolutely. Bozza - A Sample seemed more alive and enthralling. Vivaldi - A Sample seemed clearer. Goldberg - B Sample seemed richer and complete. |
| Listened for imaging, soundstage, clarity, decay of piano notes. |
| a mix of distortion and resonances |
| string instruments, background noise "air", human voice neutrality, brass instruments, bells... |
| Started out listening for/expecting the stuff you specify in Q's 1-3, but mostly could not discern significant differences. In one case, on a different system, I got the opposite answer with the same notes (more "room" and "more resolved". I went into it not at all confident that I would be able to discern the difference between a properly down-and-upres'd file and the original. Truncating or dithering-down a good quality recording mostly results in removing very little information. Upres can "smooth" things out (e.g. Cambridge Audio 840C - I sold it). I suspect if this test were done with the sample rate rather than the bit rate, it would be more audible. I also think a more meaningful comparison might be to simultaneously record a performance in 16 and 24 bit with the same ADC, though my current feeling is that it's best to record and play back at the "native" rates of the ADC/DAC. |
| Dynamic details, piano string more ohysical reality |
| Listening for instrument/vocal placement in the soundstage, transient dynamics, general recording ambience (such as reverb decays) and tonal qualities of instruments. |
| I already played this game with recent recordings that I own in both qualities. The listening is more intense and discerning at night. The morning is the worst moment of the day when everything sounds equally and sadly flat. |
| The Dynamik, the Clearness, at the End: The Music The Piano is easy, you hear definetly more mechanical Noises |
| I tried to listen to dynamic and spatial reconstruction |
| 1. With 24bit more reverberation can be heard. 2. Cymbals or other high tone percussion instruments have more clarity and remain clear when other instruments play loud (e.g. at the end of Bozza) 3. Piano tones have more information in the upper frequency range, especially in the attack. |
| dynamics and upper mid range 1 khz - 3 khz. |
| For the first sample it was the cymbals. For the second and third, it was what sounded more live and natural. |
| Generally listen for clarity of the highs, separation of instruments and soundstage. I'm pretty new to this though. Really couldn't tell much of a difference in the three samples. |
| I do attend classical music concerts regularly and just try to compare how the instruments sound on the recordings to how i remember them sounding live - nothing more sophisticated than that.in truth. |
| I ythought the '24bit' recordings sounded sharper with more precision especially at the top end of the frequency range. But I might be wrong. |
| small clues such as sound of triangle, fullness of the sound, how fast the transient response is, room ambiance, decay of the piano, etc |
| I tried to focus on the clarity, on the precision and dynamic of the presentation, but honestly I cannot identify a single difference, even a very subtle one. No way to distinguish from the two versions. |
| Fullness of sound. Decay of notes. |
| If I'm right then I would bescribe the 24-bit sound less "spikey" and rounded. Less focused on left/right but more coherent and staged in the middle. I hate to say it, but "more analog" would describe it for me. (If I'm wrong with my choices then I will not buy any 24-bit music ... !!) |
| The A tracks seemed more life-like and engaging. The B tracks in comparison sounded flatter. |
| I listened for ringing on piano, realism in voices and "pressence" in all intruments and voices |
| More "airy", increased clarity and realism. |
| Smoothness and depth. |
| I felt the tracks I picked for 24bit had more open dynamics, the 16bit tracks sounded a bit constrained. |
| Honestly i couldn't tell between them, for "Bozza - La Voie Triomphale" i thought sample A had a bit more fullness/detail, but the other 2 sounded the same to me, if the difference can be heard it doesn't seem my equipment is good enough for me to tell (or my ears i suppose. haha) |
| High frequencies, airiness, etc |
| A more dimensional sound, one of the samples usually sounded "flatter" than the other. |
| I listened to resoluion and depth at upper midrange and high frequencies to specify some differences. Imagine a bright clear stary night sky. Guess how many gloomy stars our eyes are capeable of. Which sky might be brighter to your perspective. What about your reproduction system, will it be able to resolve the higher density transparently and deliver it to your ears. 24bit is nice, not necessary. |
| This was my first time ever comparing the two. I don't recall listening to a 24-bit recording before. The 24-bit version, at least what I thought they are, had deeper bass (noticeable in Vivaldi's), richer highs (around 1.5 minute into Bozza's), and crisper notes (Goldberg). |
| I looked for the air around the instruments and whether the notes lingered a bit longer or ended abruptly (ie not naturally). I tried to also notice the violin, guitar plucks and the piano to see if they sounded more natural. |
| Sounds closer miked than the 16 bit track, livelier, more rough. The 16 bit sounds more polished. The 16 bit might be a little bit more boring and easier to listen to. |
| Transparancy |
| Presence. |
| more detail overall In particular the transients and decays are much easier to follow on the tracks I've identified as 24 bit |
| richer sound |
| separation realism |
| I just felt I detected more detail, "air" |
| More relaxed, dimensional and resolved. |
| Nothing particular. I thought it sounded richer. |
| What sounded better! The last track was the hardest. |
| Nothing specific...just straight listening and looking for any possible difference whatsoever. |
| 24 Bit: more present, livelier, 3D space, subjectively more dynamical, more resolution of the polyphony more details of airy sounds. While listening You forget about the media 16 Bit: flat, artificial |
| I was unable to honestly discern any difference between samples. My markings were a total guess! That has been my previous experience when comparing 16 to 24 bit from the same source. |
| Mostly imaging and top end clarity. |
| The sustain of notes and tonal quality |
| The highs specifically cymbals and sibilance in vocals. That is the usual way of telling a really low bitrate MP3 from lossless. Couldn't tell a difference. |
| Smoother, more lifelike and natural sounds to the instruments/vocals. |
| I tried to listen to then deepness of the sound, especially the bass |
| I listened for dynamic range sound stage/spaciousness attack and sustain I am not an experience classical music listener, so not familiar with the instrumentation of these sample audio tracks. on Goldberg. I preferred B, which I think was 16 Bit |
| Clarity of tone. |
| openness of sound seemed to me different. |
| a bit smoother and less harsh on the cymbals |
| High resolution; as when listening to a 64 kbit stream and the obvious difference when compared to CD quality |
| quiet passages |
| The 24-bit sounds "thinner" because of better separation of instruments, sound is clearer, in particular bass is cleaner and more natural. Have heard each piece once (no ABXing or similar), and decision was clear. Did a second round to double check, with same result. |
| I felt the tracks I identified as 24-bit had a *shade* more bass "impact" and less "edge" on the notes (e.g., "rounder", "smoother" sound). It was certainly very subtle in my setup (Geek Out 720, HE-500 headphones). |
| Smoother and less harsh sound. |
| vocals, piano warmth trebble |
| With the first track I listened to the difference between the bass drums. The 24-bit track had a clearer "bang" here and the overall room between the instruments was larger. The 2nd track was much harder to identify but also on certain passages the 24-bit gave it away through more details and more room. The third track was the hardest and at the end I identified the 24-bit by the details of the side noise such as the breathing of the artist. |
| Greater attack, more space between instruments, more tonal colour. The first track had a much grander scale than the second. The third track held the attention much better and allowed the listener to see into the recording more - notes hung and decayed more realistically. On the piano track, the 16-bit sounded as if there was a blanked stuffed inside the piano in comparison with the 24-bit version. |
| Quietest passages. Headphones. |
| Too low a bit depth "flattens" the music. It will sound less airy, less dynamic. |
| I could not identify any differences between tracks. I listened for transparency, potential harshness, resolution and such things. |
| percussion; voice; extend/sustain |
| details |
| originally though i could hear a more realistic sound on piano/voice. i was exactly 50% for all three tracks with 10 trial ABX |
| At first listening i thought i got it which one is some what "fresher", but after several times played i get confused TBH. Did not perfome a/b-ing. Got no reason for that. If I can't tell difference within half a minute between two listenings then it is good enough for me. Was picking sollely by impression of freshness. Not a native english orator so i can't figure more apropriate words. Must say both Goldbergs is somewhat different sounding compared to my copy of the same recording and that is without a doubt. Unfortunatly got no ma own copy's of other two recordings to make that comparisson as well. Know that was not the question you asked, but thought you might find it interesting enough to mention. |
| Transients, tonality, space around the music. |
| no help here. Just listening for some sort of extra clarity for lack of a better word. |
| I would not expect 24 bit to be different to 16bit, personally, but had a clear preference for A, B and B which surprised me. Smoother with more air and better intelligibility. |
| complete |
| Just initial impressions. Most probably I picked more B tracks since they were the second listen of the musical piece! |
| Imaging mostly, the ones I prefered had a better, more detailed and stable soundscape. The most obvious (to me) is the Vivaldi. The singer really stand out more in A while in B she is drowned in the accompaniment. The piano is also more natural and stable in A, even though I hear notes bouncing all over the place in both, a normal phenomenom with a non-point source as the sounboard is, when closely miked. The band was more difficult, I listened mostly to room decays, and better definition of the instruments. Of course, I may be completely wrong... |
| Spaciousness. The sound of the room more than the instruments. The interaction of different sounds. The difference with the Bozza was vastly more obvious so I assume it may have been a trick question with the lower being deliberately knobbed. IMHO, the best of 24-bit will come when producers etc. use less dynamic range compression. I look forward to that day. |
| Maybe a bit fuller and smoother. |
| Smoothness and presence |
| track b's were less pleasant to listen to, sounding thin. |
| fullness of sound. soundstage imaging smoothness |
| The A tracks sounded consistently more strident, less focused and less coherent. Vocalist and piano was more diffuse on A tracks. Piano was duller sounding. The B tracks were slightly more dynamic, more focused and richer in tone. The vocalist was more 3-D. |
| Upper bass. It is lighter in the 24 bit version. |
| air, spaciousness around decaying notes and transients |
| I was listening for any difference in overall listening experience. Did one sound more natural. I was particularly paying attention to the dynamics to see if that extra headroom allowed for greater impact and contrast from the soft to loud sections.I payed attention to noise floor etc. I didn't expect to hear a difference but i tried my hardest to find one and i honestly couldn't . at first i thought track 1 sample B was slightly softer in the quiet sections and had more impact in the louder sections but i used abx test and at best got 60%. |
| I listened loud obviusly :) tried to hear the ambience but also the 3d space and deep bass and the silence ! if it sounded differently . But... this is so hard |
There were a handful of comments about not hearing a difference which I've left out from the list above since I wanted to focus on the subjective experience of those who believed/heard a difference. Not surprisingly, the respondents utilized the standard lingo of subjective audio evaluation with the typical adjectives (I of course offered some of that when I asked which sample they thought was 24-bit earlier in the survey). There are >80 comments in that list above yet we know overall there is no evidence that despite these subjective impressions, the respondents were statistically able to discern a difference as a group.
II. How "easy" or "hard" was this test?
One of the last questions asked of the respondents was whether they felt the test was easier or harder than expected:
| I had no real expectations. I found it impossible to hear any differences on first listen, but after approximately 30 minutes or so, began to sense a bit more air and relaxed presentation with of A samples. It also seemed that I could relax into the music with the As. It will be fascinating to know whether I just made that up in my head or actually heard a difference. |
| Clearly harder than I thought, although I'm not really used to listening for those small details - I usually just enjoy the music... |
| harder |
| Thanks for putting together this test. You obviously put a lot of effort into it. Here's some advice on how to give the results. Document with as much detail as you can the procedure used to create the test files. That way, people can duplicate your test files exactly and show that you weren't trying to cheat them in any way. A lot of trust is required for this kind of test to have any kind of validity, and many audiophiles are paranoid. |
| Thank you for doing this. Really made me re-think how I feel about high resolution audio. Going into the tests I thought there was not a lot but some clear differences. After the tests I would probably place more emphasis on finding the best recording/mastering of a particular Album rather than just buying the highest resolution files of that Album. |
| You manipulated this music, when I understand it corretly there could have been |
| Diffmaker shows only a negligible difference, still hearing difference |
| Exactly as I expected ;) |
| It was approximately as simple as expected. |
| Again THANKS TO YOU looking forward to the seeing the results. Playing Bass since 1968. |
| About what I expected. Did not expect to be able to tell much of a difference at all. I thought i heard differences the first time through, but the differences became smaller as I became more familiar with the music. |
| That more difficult than was expected. Thank you ! |
| I expected there to be no difference, and I heard no difference. |
| Thanks for putting together this valuable survey. I have long suspected there was no difference between hi-rez and redbook formats. Would like to see the same thing with 96 or 192 sample-rates vs. redbook. |
| It was hard to listen any difference. Could you manage a similar test between PCM and DSD in the future. |
| Hmm, without quick comparison AB tracks seem to be the same :-) |
| As I mentioned, I was not at all sure that chopping off bits and upres'ing them would be audible. And I'm not at all sure what it illustrates. I'm not confident that it proves that there is no difference between 16 and 24 bit recording. |
| The test was difficult. I'm by no means certain of any of my answers. I have yet to reach the stage where I will habitually buy a 24bit version of an album in preference to the CD, though that's partly because of the price difference! |
| Assuming that my answers are correct I did find the test a lot easier than what I expected. |
| It was easy |
| I expected it would have been hard, it was indeed :) |
| Harder than expected. I believe a very good DAC and amplifier are needed to make the difference hearable. It is much easier with headphones than with speakers. |
| harder than expected. |
| Not easy. I think a ABX test to determine if people can actually hear a difference would be useful. Sorry I don't have to software to do the ABX switching you talked about. If there were three sample for each song and Q1 would be which are the same, Q2 would be which do you think is the 24 bit. I am curious to know if I got anything right! |
| It was hard, just like expected. I played with open windows and an open balcony door which raises the noise level in the room. But either way, I would be surprised if I got all three correct. I didn't spend much time on this. I would have preferred being able to more easily forward both to a passage in the sample where more "is happening" and then AB test them quickly with a press on a button. Those times I've been able to distinguish MP3 in 256kbps vs lossless, I have found it easier when entering a passage in the music where there's for instance more high frequencies. |
| F'ing hard! |
| Came with no preconceptions - on one hand hoping 24 bit might offer an inexpensive upgrade on CD quality but likewise if there is no benefit to the "extra 8 bits" why waste disk-space and extra cost of 24 bi files. Honestly couldn't tell much if any difference. All samples sounded very well recorded - better than many recordings I have bought from major record lables. Looking forward to seeing the results ! |
| Slightly harded than expected. I thought I'd be confident about hearing a difference between 16 and 24 Bit. Thank you for setting this test up. I will be very interested to read the results. John Allen (UK) |
| I got the MP3 vs lossless response right. But my confidence is much lower with this test. The difference is too minute for my ears/equipment to reliably detect the difference. Anyway, thank you for putting this together! |
| The third sample was more difficult. |
| This test confirmed what I'm sure about from a long time: that I cannot hear any difference from 16/44 to 24/96 or more files. No way, 16/44 seems to be enough, at least for my hearing system :D For future tests I suggest you leave the files in wav format uncompressed. I know this is a waste of storage but someone could be influenced by the different file sizes of a flac compression (even though they're not correlated to the original file resolution). Anyway great test!! Useful for people to understand what they can and what they cannot really hear! PS: I don't care anonymity so if needed this is my email address: mag1ster@alice.it |
| Easier than expected. I found the classical pieces chosen to be very revealing- more so than some rock selections I have tried this test with in the past. |
| Currently I use a Raspberry Pi + HifiBerry DAC. There is much debate about powersupplies (walwart vs BOTW). I would like to see a test about the differences between the use of a "stabilized" PSU vs an ordinary "2-dollar" PSU. Is this auditable. |
| This was easier than I expected. You don't mention how you dithered the files. I'm wondering whether the differences would be less pronounced with a different or more sophisticated dither algorithm. |
| harder than expected |
| very hard |
| Great fun, thanks for your effort. |
| I can't tell if it was easy or not because I don't know if I'm right yet XD |
| I think I can tell them apart when listening side by side but it would be very difficult to guess a track on it's own. The 3rd pair was hard to tell because the recording wasn't as detailed to begin with. |
| Much harder than expected. I still think the raw material and capture of true ambiance is much more important than the final resolution of the music tracks. I've heard some great CD's and some very bad supposed to be HDTracks |
| Harder than expected! |
| for me, it was't easy to detect subjectivly the higher resolution examples. yet I only suppose less or more that I'm right |
| In terms of how you ask the questions. It might have been better to have a 'not sure' option instead of 2 because that might skew your data analysis. You could have combined both first questions as having 5 options: Definitely A, Probably A, Not sure, Probably B, Definitely B. Any questions/feedback, feel free to reach me at ezzatelhalabi@yahoo.com.au Really keen on the results of this survey. Regards and best of luck. |
| It was hard. The differences are not huge (to my ears). Clips should be shorter. |
| I made a point not to look at file sizes or any other file property. I cwitched the output to 96khz on the Dragonfly. I am not sure I could consistently differentiate between the samples in a real blind AB test. |
| Much harder than expected. |
| This was harder than expected. Please do more of these tests in the future :D i enjoyed it greatly |
| much easier than expected |
| bit easier! |
| Easier. (assuming I was correct...!) Thanks for going to the trouble of providing this survey. |
| A little harder than I thought it would be. |
| Differences were subtle. Was the 16 bit file sourced from 24 bit file and downsampled. Or was it a native 16 bit track. |
| Harder. |
| The tests were easy but trying to find any difference between 24 and 16 bit was very difficult to the point where my answers were complete guesses. In other words, can't tell the difference. |
| Easier to decide than thought |
| I appreciate your ongoing efforts to clarify the vast amount of misinformation surrounding digital audio. I belong to that subset that finds 16/44 more than adequate. |
| I didn't expect to be able to hear a difference. The unsure results were as I expected. |
| Harder |
| Not a big proponent of high-res, at least not on normal gear. I think its benefits are marginal, making this sort of evaluation as challenging as I expected it to be. |
| Great work |
| Is there a recommended BC wine to consume whilst performing the test. |
| Thanks for doing this. |
| HARDER! |
| Enjoyed it and looking forward to your findings. Would really like to know if someone can hear the difference without doing A/B'ing instantaneously. |
| Well prepared test. But was not difficult for me (as I expected). Not sure, if I could distinguish files after high quality up- and down-sampling (or down- and up). |
| harder than expected. I just could slightly notice the dynamics more clear but wasnt 100% sure... |
| Harder - not very confident on any of the samples. But did not expect it to be "night and day", as some "audiophiles" say it is. |
| Samples 2 and 3 have lost resolution in editing and have become noisy. (digital resolution have been limited) |
| I just felt curious. I don't think I have "golden ears", so my ABX tests are just from an untrained person. I am considering upgrading my gear to a pair Sennheiser HD650 headphones in the future, since the sony's are not great for mid and high frequecies according to many reviewers and experts. Also, I am unsure wether a dedicated headphone amp would better help to discern each version. |
| Assuming that my results are correct I found it so-so. See my comments above. But I knew before that the differences between 16-bit and 24-bit recordings can be quite subtle and therefore not so easy to identify. |
| Much easier than expected. I was expecting to have to play the tracks a number of times to discern any differences, but I played each track through only once and could pick up the differences within seconds of the second track starting. I'd like to see more of these tests. Too many people saying that high-resolution is pointless and tests like these prove otherwise. |
| easier than expected - nothing to hear, move on :) |
| I switched between A and B extensively on all three tracks, I could not discern even the slightest difference. Not even a hint. Therefore, I didn't even complete the test by picking X = A or Y = A. It would have been pure guesswork. |
| dear archimago first i would like to congratulate you for your awesome work. I greatly enjoy your damn fine blog. You are a champion of rational thought and enlightment in this hifi world which unfortunately is plagued by snake oil, hocus pocus and gullible fools (I once was one myself) please go on with your excellent work! |
| This was interesting. It would be so much more with some other equipment... |
| i expected to get about 50%. |
| In case my findings are relevant in reality i could confirm my phylosophy about audio - if you must to perform intensiv listening on your thoes in order to find some pluses or minuses then those differences are irelevant when you actually listen to the music itself. Big enough diffs will be transparent from the go and that is when you should think about upgrade or trying another recording of the same peace. YMMV |
| I found this very hard. After one or two passes I formed a view which I didn't change, but I know if I'd been listening properly blind, I wouldn't have been able to ABX the tracks successfully. Thanks for setting this up! |
| Difrents is small. |
| I was not expecting anything. The differences were small but I am fairly confident in noticing them. |
| easier |
| Easier, but still the difference is quite small. I think the sound engineering quality of a recording is the most important aspect. Some very well recorded 16/44.1 CDs sound as good or better than some multi-channel SACDs, DSD being somewhat equivalent to 24/96 PCM. But I also have some amazingly realistic SACDs... |
| One seemed very obvious but the other two were both very subtle. I wouldn't be surprised if I got them wrong. You should really try the demo CD (yes - CD) from http://www.earthworksaudio.com/products/microphones/ it's a great demo of how better quality kit shines even through a 'bottle neck'. Personally I think it's more to do with accurate timing than high frequencies. |
| Just as I expected. |
| Very nice |
| Probably some people (who also are certain about their abilities) are going to answer right by chance. Without proper ABX those are of course worthless data points. It has become clear to me that even really rigorous scientific testing doesn't apply to audiophiles or marketeers - people who have a "I want to believe" poster up in their heads just ignore facts and continue spreading myths no matter what. |
| This is very hard (read impossible). I would like to se a test where CD quality is compared to hirez. |
| About what I expected. |
| Differences were subtle, but noticeable without doing fast A/B switching. Particularly with Bozza, fatigue would set in with track A and not so much with B version. I think some more musical tracks would be nice, something that most would want to listen to and keep in their collection. More percussion and maybe string bass would be helpful. A mix of Jazz and classical and acoustic guitar would also be helpful. |
| the file sizes for A and B are different- maybe this factored into people's judgements. as hard/easy as expected. |
| You can so a similar test like I made where the ultrasonics are removed only. So convert 96/24 to 48/24 and back to 96/24. Make sure to use a good down-upsampler. Info for this here: http://src.infinitewave.ca |
| thank you for your hard work in finding out the truths within this myth filled hobby of ours. Your Blog is a breath of fresh air and is truly excellent. Thank you once again, regards, James. |
| Impossible but i expected that . But i actually tried hard . |
As you can see, there were a variety of experiences described in the comments above. Some felt it was harder, some felt it was as expected, and some seemed surprised by the difficulty. Interestingly a number thought it was easier and expressed high confidence in their ability to discriminate the sonic difference. Since this is all anonymous anyhow, let's correlate a few scores to the comments above: the person who said "easier" was 1/3 correct, "much easier than expected..." 1/3, "it was easy" 0/3, "easier to decide than thought" 1/3, "bit easier!" 1/3, "it was easier than I expected..." 2/3, "Easier than expected." 0/3, and "It was approximately as simple as expected." 3/3. Easier? Really?
III. Final Thoughts & Personal Impressions...
After reading the testimony of the respondents above, I think it's just as fascinating watching again the promo video for Pono where Neil Young apparently wows his buddies with his car stereo system allegedly on account of fantastic sounding high-resolution audio (Neil of course seems to have a "thing" for 24/192).
Given the 24-bit vs. 16-bit audio test results, unless high samplerate (ie. 44kHz vs. 192kHz) is to explain the striking difference so dramatically captured in that video (take any 24/192 track and down sample it to 24/44, tell me if you were WOW!ed by the difference), I honestly wonder what those celebrities are talking about. In my opinion, if these dramatic Pono testimonies are to be believed as genuine, then Neil Young must either be playing different masterings (eg. distorted vs. better mastering, playing them at different volumes), and/or he's playing some ridiculously data compressed track (64kbps MP3?) versus high-resolution to get that kind of reaction. I'm sure this can be easily answered if Pono would just release a couple of minutes of what was used on those celebs - I'm sure the rest of us would love to experience the apparent glory.
On a side note, CNN listed Pono as a "game-changing gadget" of 2014. Yeah... I guess we'll see about that...
On a personal note, I did try the blind test myself on two systems:
- Windows 8.1 PC --> ASUS XONAR Essence One via USB (ASIO) --> Sennheiser HD800
- Windows Server 2012 R2 or Win 8.1 PC --> Squeezebox Transporter or TEAC UD-501 DAC --> Emotiva XSP-1 pre-amp --> Emotiva XPA-1L monoblock amps in 35W Class A bias mode --> Paradigm Signature S8 + SUB1 (balanced XLR interconnects, 4' 12G OFC speaker cables; <30dB SPL quiet sound room at night, room correction DSP off)
Total cost of the systems above would be in the $10,000 - 20,000 range. I used Foobar ABX on the PC with HD800 headphones and achieved 6/10, 6/10, and 4/10 correct in identifying the 24-bit sample over 3 listening sessions with each musical piece (that would be 2/3 "correct" I suppose). I could not tell the difference with the Transporter or TEAC DAC played through the full-sized speakers with sub and would easily grade my level of confidence as a "guess" or at best slightly "more". I'm currently 42 years old.
Over the 2 months that I was gathering data for the survey, I also tried this test with friends and family of various age and both males and females (results not entered in the survey). Never did anyone express the opinion that differentiating sample A vs. B was "easy".
I do remain open minded, however. Although I have not met anyone who could easily and accurately detect 24-bit vs. 16-bit audio file differences in a controlled setting, I'm also not saying it's impossible. Who knows, maybe the fellow above who responded "It was approximately as simple as expected." and got 3/3 is one of these. Humans are capable of amazing feats after all... However, I do believe hearing acuity of this magnitude would at best be rare and I suspect most reasonable individuals would recognize this once they try an ABX for themselves or logically figure this out based on understanding of the science. Listening volume would also be a consideration and most people would understand that at normal listening levels, the extra 8-bits would be highly unlikely to be of any benefit, especially if audible ambient noise is present.
Finally, someone asked me the other day whether I thought 24-bit music was therefore some kind of a "con". Well, no, not necessarily. Assuming an album was recorded, mixed, and mastered well with extremely high-resolution equipment, then one could be buying music of the highest fidelity/accuracy (more dynamic range if ever needed, more "complete" ultrasonic frequency and low level details captured from the recording session). It would be hypocritical of me to desire a high-end DAC capable of >16-bit resolution but turn my nose up against a truly high resolution album, wouldn't it?
The key of course is that first, the 24-bit/high-resolution audio file must be actually of high quality (accurate digital chain, superb microphones, music worthy of the dynamic range and frequency response, expert engineer doing the job, all processing maintaining at highest level of resolution). Secondly, that the album was one I truly love and desire the best resolution version in order to "go the extra mile" in terms of finding the hi-res version. Fulfilling these criteria, I would personally find some value in the purchase (how much $$$ over the same CD resolution version is another matter requiring consideration!). The pragmatic reader could just as easily ask "what's the point at all if we can't hear the difference?" and I would not argue with that either. For me, this is still about "perfectionist audio" and I believe one is allowed a certain level of neuroticism in this (and any) hobby... :-)
[Speaking of assessing value, obviously when it comes to music, this is totally a subjective personal matter. But let us also keep in mind that high-resolution 24-bit downloads (any music downloads) are intangibles. There is no "street value" attached (I don't even know if it's legal to sell them 'used' - presumably the laws could be different depending on country). Let me know what's on offer at the local pawn shop if you ever bring over to them a USB stick with your beloved 24/192s, JPEG "covers", and PDF "booklets" assuring them that these are your last copies and transferring all rights/privileges of ownership. In this fashion, "collecting" high-resolution music downloads is quite fundamentally different than having a library of stamps / books / spirits / wines / paintings / vases / cigars / cars / CDs / LPs... To me, there does not appear to be any material "store of value" in the digital music download collection from a monetary perspective.]
What I think consumers should not have patience for is hyped up talk about high-resolution audio applied to questionable old recordings that never had more information in them than what a 16-bit CD was always capable of encoding. Or even worse, new volume compressed recordings and remasterings of low dynamic range especially when sold as 24-bits as if this somehow magically makes it better (like this). Sadly the above conditions cover the majority of what I believe are current "high-resolution" offerings (at least in the pop/rock genres). Remember too that there could be a different mastering used in the high-resolution version like they did for the SACD of Joe Satriani's Engines Of Creation documented in my upsampled SACD list (eg. more dynamic, less clipped or volume compressed) which would make it the preferable one to get as an audiophile... But this would not necessarily be because it's in a 24-bit format.
I suspect the marketplace will figure this all out in time, as it did for the likes of most DVD-A and SACD "high-resolution audio" over the last decade and a half.
A final thanks to all those who helped me put together this test and to all the folks who gave the test a try. I appreciate your willingness to participate in this little experiment! I hope it provided some personal insights beyond the conclusions presented as a group.
Until next time, enjoy the music.
