Previous -
Part III: Discussion
I know some of you have been wondering if the dataset contained descriptions of the
subjective experience of respondents between the two sets. Compiled below are the respondents who commented with a subjective description - I have removed any identifying information in the spirit of maintaining anonymity and only included the responses where there was a suggestion of what was heard/not heard that may have allowed the respondent to choose.
There were understandably few comments from those that did not hear an audible difference.
IMO, whether the commenter was correct in identifying the lossy compressed sample or not is not as important as the thoughtful consideration and comments in participating in what I believe is a difficult task.
As a reminder, the question was which 'Set' was felt to be 'inferior'. Set B was the actual lossy compressed option.
Those who chose Set A:
"Although I could easy hear the differences. It's very hard too discribe the difference. They are very small. There is some glare, a little bit harsness, a liitle less ambiance, ect. Most easy was it too recognize the differences on voices on the church track."
"Set B sounded more "there" to me, for each track. In particular, bass
sounded more present, and treble sounded sweeter. Interested to see the
final results"
"I could hear the most difference at the top end. Symbols/High hat and
"s" seemed subdued in set A. However I am not sure if the conversion
process to MP3 didn't enhance/reveil the top end. I struggled to detect
any difference on the Megaherz tracks. I'll try again on a good HIFI
later as I am sure my old scud of a laptop's sound card isn't up to
much."
"I sort of tried to deduce. In term of preference, I have no preference
over either (not my kind of tunes). Set A just sounds a tiny bit louder
and given they seem to have been treated to be at equal loudness, I just
thought a bit louder = lower dynamic range = must be compressed (the
same way my AVR makes sound seem louder by reducing the dynamic range).
:D"
"Much more difficult to tell a difference than I thought. Shorter song
selections might have made it easier (hearing memory isn't very long). I
hope my choice was correct! I just thought the B selections sounded
better. Can't wait to see the results."
"Well, took me a while to tell the differences but indeed there are
differences especially in terms of details, attack and decay. Easily
identifiable on large loudspeakers rather than on headphones. I am quite
surprised that they are very close. A better comparison than this would
have been a high res studio master/higher bit rate vs MP3 320kbps. In
the first place, the music information should have been there rather
than just an upsampled CD/MP3."
"It comes with experience. Harmonics isn't easy to listen out for in the first try."
"Bass drum @ ~2 minutes of Church Distortion guitar and cymbals at start of Keine Zeit"
"Very difficult to tell the difference - in fact, it seems that
it's just an impression that set B sounds better, slightly clearer,
maybe. But listening to he samples more and more seems to even them out.
Can't tell if there is more bass or treble here or there. It's just an
overall impression. Thanks for putting this survey together. I'm curious
to see the results! All the best!"
"I noticed a difference within a minute or so on my computer which
has a small Hi-Fi system with bookshelf speakers. I confirmed it in my
listening room and my wife concurs. By the way, you did not ask age,
but for reference, I have 65 year old ears."
"Listened for the deeper base and the higher sparks. Pink Floyd's
Time and Lyle Lovett's Church made it easier to find those missing bits
that MP3's remove due to general midrange equipment cannot reproduce
those extreme lows and highs. Those details are retained in the FLAC
file. The bass went deeper with the subwoofer for set B and the piano
sounded more full bodied with on the headphones."
"Set B sounded sharper if you listen carefully a few times, but the
third song were hard due to the drum"
"I can't pinpoint exactly what's different, but set B sounds more open and sparkles more than set A to my ear."
"Extension of frequency by comparison, like if lossy, u won't hear very low or high frequency."
"'Time' set A actually sounded superior to me. But overall, B superior."
"I made the test two times. At the very beginning of the survey in
good and silent conditions and 2nd time today January 27 with windows
open and neighbors drilling behind the wall and electricity polluted.
Both time the results are the same and are manifested with MUCH more
air, natural details of female voices, clapping less harsh cymbals and
generally more separated sources (voices and instruments). The DSOTM was
the hardest, because my home version sounds slightly different. The
metal band was easiest, because mp3 turns the cymbals into a constant
dirty sheen."
"B just sounds more "real" especially in the percussion on
Megahertz - listen to the hi-hat @ 1:14 (i'm a drummer); but I could be
just be guessing ... it is VERY difficult to tell the difference! thanks
for putting this together, looking forward to "hearing" the results!"
"obvious only on Keine Zeit (i.e. need complex sounds)"
"I wouldn't use the terms "inferior" or "superior" or "better" or
"worse" - I'd just ask "Which one did you like better?" "Time" is one
of my all-time favorite songs. I heard a difference immediately between
"A" and "B" - but the funny thing is, I couldn't tell you what the
difference is. Tried listening with the "...is it the bass? The high
end? Better depth? Better tone?..." kind of mindset, but that was
hopeless. What it comes down to is "B" sound more "solid" and "3
dimensional" and "real" to me than "A" does... but I can't tell you why.
It just does. I don't know if it's because of the difference digital
datastreams, or because of the processing you did to *GENERATE* the
different data streams. For all I know, the "B" track might be the MP3,
but whatever you did to transcode the "A" to the "B" made me like "B"
better. Who knows? Thanks for putting this together, was quite
interesting."
Those who chose Set B:
"With the good DAC this was REALLY easy to discern. I could tell within a second which was which. It's interesting it did not take a megabuck rest of the system to hear the difference. With the SBT analog outs the difference was much closer, but still there."
"I wonder if the you should have used tracks with "more going on" to help the listener? Tracks with more prominent percussion for longer periods of time, for example, since that's where it was fairly easy to detect mp3's in the past. Interesting test... it's my first listening test. Thanks!" (Selected "no difference")
"Although picked B as mp3, I'm not really that sure. Only a few things
made me *think* this is true (e.g., piano). But I wouldn't be surprised
to hear that I was wrong. mp3 is at worst very transparent here."
"the music was flat vs open i also tried with (cheap) headphones on a
laptop and was almost impossible to hear a difference"
"Presence."
"There is a common
sound to PCM processed with floating point math, also dither (especially
with noise shaping) and MP3 (worst of all). All these processes smear
the precision of the sounds in time and stereo position, likely
introducing quantisation errors that the designers of AD and DA
converters tried hard to avoid :)"
"I'm
67 and sadly, my high end rolls off @13khz. I found the highs on the
bells & chimes in "Time" to be harsher and more strident on the B
version"
"I used ABXer for the test. I scored 100% on Church, 60% on Keine Zeit, and a big zero on Time."
(Ed: Thank you for spending the time!)
"I listened to the cymbals. HF content, easy clue for spotting bad
compression. Cymbals seemed more mellow/relaxed/natural in A, but if
there was a volume difference, that probably fooled me. I heard very
little difference."
"I'ts a very difficult test. I've recognized difference only on the
first Time track. Others are indistinguible for me."
"great test this has helped me realize that i don't have those golden
ears and i should just simply enjoy the music"
"not
sure if i'm correct as my audio set-up is not exactly "hi-fi", but the
most audible difference i heard was the hi-hat in the ending part of
church, in B, it sounded more synthetic compared to A. I guess MP3s are
still awesome when space is a constraint, i don't strain my ears so hard
when listening normally and so do most people in order to enjoy music!
(right?) very glad to help you with the survey, hope i have been of some
help!"
"hard edge at top end"
"Not much difference really. Maybe headphones thru a laptop isn't
much but noticed in Lyle Lovett's Set B, his vocal's seem more
'pronounced'"
"For me, the one thing I noticed is the the sound stage was
completely different with the clocks on Time. One set had a real
definied space for each clock, where the other was a lot tougher to
pinpoint the clocks actual location. In fact, I would have honestly
said it was two different masters moreso than one being compressed or
another. I've not experienced anything like that with my LAME Mp3
settings... my mp3s match my FLAC files to the point no one has ever
been able to guess which is which. This has been a very interesting an
fun thing! Next time let me compress the files and give them to ya! ;)"
"There is a distinct difference between the sharpness of both
encodings but whether the sharpness is considered better quality or
inferior quality is up to our tastes. I chose the sharper quality as
the better recording."
