Saturday, 17 March 2018

MUSINGS: A quick discussion on what's OBJECTIVE and what's SUBJECTIVE (alas, more MQA-related stuff).

Infographic, from pediaa.com.

Hey guys, very busy this past week and about to go out of town... Nonetheless, it has been fun participating in discussion forums this week. Therefore, I thought I'd just fire out a "quick" post this morning.

I see some rather disturbing back-and-forth arguments on the recent Stereophile post on MQA from folks like dalethorn and boulderskies that IMO reflects a fundamental disagreement on the differences between what the words "subjective" and "objective" means when we use them to address ways of understanding audio (among other ways of discerning knowledge in this world). Obviously, unless we are all on the same page with definitions, we won't be going anywhere with debates and arguments!

At first, I was going to just make this comment a forum post, but given the length and for future reference, I'll just put it here instead of a typical comment buried deep in a thread. We can potentially get deeply philosophical about this, but let's see if we can just hit the main points at least...

First, let's get some dictionary definitions out there (as per Google):
OBJECTIVE:
(of a person or their judgment) not influenced by personal feelings or opinions in considering and representing facts.
SUBJECTIVE:
based on or influenced by personal feelings, tastes, or opinions.
Boulderskies bluntly said in one of the comments:
Objective means graphs & charts; Subjective means, "Does it sound good?"
Clearly it's not so simple. When someone shows graphs and charts based on what a machine was able to measure, we can say at some level that this is "objective" because the data is likely a quantitative result that originated apart from direct human input. Not all charts and graphs are objective though because they could just be a reflection of opinion (eg. graphing a bunch of subjective scores like "4/5" for "recording quality" comparing album reviews). Of course not all machine-based results are accurate based on whether the test is appropriate and the operator had performed the test soundly - bias can creep into measurements. Furthermore, measurements can show variability, and issues like inter-test reliability may need to be considered as well. Rigorous set-up, using a measurement device of adequate resolution, and appropriate use of test signals must be followed in order to ensure objective tests reflect a non-biased result.

In the case of MQA, with objective testing, using known signals, we can detect subtle distortions and show imaging artifacts with MQA filtering (as reviewed recently). Likewise, we can detect the limitation of resolution with MQA by using actual MQA encoded/decoded music and measuring the bit-depth reduction. These things are facts, replicated by others and not just my opinion. (In "fact", this morning, I received some data from others having done the null measurements showing bit-depth loss with MQA from about 10 albums.)

From the facts, we can deduce some inner workings of a system and form opinions around quantitative and likely qualitative implications since we also have knowledge of human auditory perceptual abilities we can try to map the findings to. Furthermore, we can figure out for ourselves how MQA works independent of what MQA (or Bob Stuart) tells us because we can use our knowledge of audio and computer science fundamentals to understand the mechanisms being employed, although not down to every proprietary detail.

Dale's PDF notes on MQA and the comments by boulderskies invoke subjective "analysis". They pertain to expressions of personal tastes and opinions either when listening individually or desiring to know what others opine.

We all have opinions. But what can be deduced from opinions is a different level of "knowledge" than objective analysis. Personal opinions will not allow us to understand with precision how MQA functions and the comparison results Dale brings up in his text are his opinions based on personal experiences and cannot be thought of as universal. For example, when he (and subjective reviewers) says something sounds "darker", this is a subjective quality which doesn't have a precise definition, likely would need trained listeners to achieve agreement, and has no clear point of reference. This property of sound (qualia) could be experienced very differently for me if I were to hear that same passage. While it could be "true" for him, this is not a "fact" that applies to everyone nor is it necessarily a property of MQA itself (ie. we need to be sure that what was heard was an apples-to-apples comparison, the only difference being MQA processing).

We could still try to be objective with "measuring" human preferences. Again this involves control and rigor. For example, we can do blind testing with music known to be of high quality, the same mastering, A/B between the original 24-bit master vs. MQA decoding within the limits of making auditory comparisons, etc... This is of course the intent of my "Internet blind test" many months back.

I look forward to the results from McGill University's testing of MQA, curious to know what controlled methodology they employed and with what audio stimuli (eg. are we talking about test signals, actual music, what equipment, etc...)

Ultimately, it depends on what one is aiming for. For any of us, we can absolutely decide that all that matter is "what sounds good to me" (I think this is boulderskies' perspective). If so, then by all means the only testing needed is personal experience and one's own subjective opinion rules supreme. In this context, it really doesn't matter what levels of distortions are present or if concepts like "fidelity" or "accuracy" to the original encoded signal were achieved in playback. It's important however to have insight and acknowledge that the generalizability of such opinions could likely be limited to oneself...

Unlike subjective preferences, objective facts can be replicated by others and often can be accomplished with different methods. Facts are "universal" (within reasonable context of course).

There is finally one last important difference I'll point out between the subjective and objective. As powerful as objective facts can be, they can also be proven wrong. That is the intrinsic "risk" of claiming generalizable truth. We cannot prove opinions wrong because they are by nature idiosyncratic - "true" to the individual. Whether a man or woman prefers "the sound of MQA" is along the same lines as "what is your favourite music genre?". No point arguing about this because in a free society, we can accept a multitude of opinions.

As humans, for any concept, both objective facts and subjective opinions (eg. "feelings about that fact") need to coexist at some level when it's something that matters to us. Who knows, in the days ahead, maybe MQA can show us in controlled tests that "most people" prefer the sound of processing through the MQA encoder/decoder; some evidence of subjective preference in its favour. But this will not diminish the existence of measurable distortions; it just means that the levels of the distortions are below audible thresholds of objection or some people might like them (we can say the same for preferences around vinyl playback or tube devices that add distortion).

None of this changes the non-audio factual findings like the presence of the cryptographic signature embedded in MQA files. This is of course a separate entity using the knowledge of which we can form other subjective opinions...

-------------------------------

Have a great week ahead everyone :-).

Enjoy the music...

BTW: If you're looking for another high image quality UHD Blu-Ray movie to check out - Murder On The Orient Express looks great! Originally filmed in 70mm (65/5-perf).


21 comments:

  1. It behooves one to point out that your statement, "Dale's PDF notes on MQA and the comments by boulderskies invoke subjective analysis." is not using the term "analysis" as it is currently understood to be. No definition of the word "analysis" covers the scenario "I heard it and liked it." Your reasons for using the word may stem from an attempt to be non-confrontational (perhaps even polite), but this is a post about definitions and your use, as such, may blur (pardon the term) the issue.

    In an attempt to elevate the discussion in good faith, one runs the unfortunate risk of having the unintended consequence of lowering bar for the discussion. Put simply, ascribing a value of "analysis" to an "opinion" (or, even worse as in this context, anecdote) has the propensity to devalue the term. At no point has the plural of anecdote been, or ever will be, defined as data (a prerequisite for analysis).

    To paraphrase the philosopher, McCoy, "It's analysis Jim. Just not as we know it."

    Respectfully,
    T.S. Gnu

    PS: I notice my unfamiliarity with the editing interface has resulted in my ignominious debut with a deleted post.

    ReplyDelete
    Replies
    1. Good point Gnu.

      In my haste to post up the reply yesterday, I clearly did not read deeply enough into the exact intended wording. You might be correct that perhaps subconsciously at a certain level I was hoping not to be too confrontational.

      Dale's document clearly does not represent analysis but rather a variety of opinions, some related to sound quality, others appear to be vague, tangential and poorly developed thoughts.

      I will leave the wording but adjust it to "analysis" (with quotations) to suggest "so called" analysis and leave the reader to see your comment and reference to the esteemed philosopher McCoy...

      Delete
    2. It appears that my original post edit did not take. Before I incur the deserved wrath of the acolytes, I would like to attribute the above reference to the great philosopher Spock. I hope that this restores order in the universe and prevents any further sharpening of the bat'leth.

      Delete
  2. It seems much worse than this. For all those snake oil believers, USB cable sound difference hearers and (insert your favorite nonsense) the word 'objective' means nothing else than 'I tried it personally, so I can give objective information, comments and even reviews'. 'Subjective' for them means 'I just read it elsewhere'.

    This way 'subjective' unexpectedly (for me) turns into 'objective'.

    Confronted with the obvious error in defining their (not existing) reliable test methods, you get some hilarious replies. Here is an example I just stumbled upon a few days back:

    https://www.head-fi.org/threads/rme-adi-2-pro-dac-headamp-an-a-lot-of-more-stunning-features.837026/page-17#post-14104377

    https://www.head-fi.org/threads/rme-adi-2-pro-dac-headamp-an-a-lot-of-more-stunning-features.837026/page-18#post-14106023

    After competely realizing what I just had witnessed I was left quite shocked.

    ReplyDelete
    Replies
    1. Thank you for the links Techland.

      Wow, those are clearly inappropriate use of the adjective "objective" (whether implied as actually objective or even in that second link admission of "not purely objective").

      I find it fascinating how the writer, in the second link said "expectation-bias is for me at least not really present" but does not imply at all that the testing he/she did was actually done in any blinded fashion! Obviously the writer is grossly minimizing the power of subconscious effects even though he/she may genuinely be trying to consciously resist such biases...

      This is why I used the sentence:
      "It's important however to have insight and acknowledge that the generalizability of such opinions could likely be limited to oneself..."

      That ability to develop "insight" is important and I think is related in some ways to a sense of humility. To recognize that because I am human, I am very much prone to my own conscious or unconscious biases of thought, and limitations of the listening/hearing (ear/mind) apparatus.

      Just because I honestly may "believe" I hear something doesn't mean this is actually true. This insight I sense may be lacking in many "golden ears" who are certain about their beliefs, especially those who tend to get upset when faced with skepticism of their claims

      Delete
    2. In the second link (https://www.head-fi.org/threads/rme-adi-2-pro-dac-headamp-an-a-lot-of-more-stunning-features.837026/page-18#post-14106023), we have the fascinating quote:

      "expectation-bias is for me at least not really present"

      This is a stunning example of expectation bias as well as a breakdown of fundamental logic. i.e. a person expecting themselves not to have expectation bias. At this level it isn't even worth having a debate about the material presented or the experimental acumen. There are problems at a much lower level, and tackling methodology and lack of definitions is at a level beyond that. Until these issues are brought to light and addressed and we are given internally self-consistent comments in e-space, there is little to discuss about "analyses" that were carried out in meat-space.

      It is useful to think about how easily we have just overlooked the inconsistency WITHIN "expectation-bias is for me at least not really present" and moved on to actually debating this when it is, in essence, a non-sensical statement. Perhaps a more closer, critical look as to whether it is even possible to engage in meaningful debate in these instances is in order.

      Another good example is this very column. We are attempting to establish common definitions, while overlooking that Thorn has quoted within the space of an hour:
      "Quotes around "fact" refer to the fact that what you state as a fact isn't necessarily a proven fact, instead it's an asserted fact."
      https://www.stereophile.com/content/mqa-benefits-and-costs#0PBbWjWSAvuoXysa.99

      ""Even [sic] fact can be proven false, in error, or wanting in some way."
      https://www.stereophile.com/content/mqa-benefits-and-costs#0PBbWjWSAvuoXysa.99

      ...(mis)leading at least one person to believe, "As least you appear to know what science is, unlike many non-scientists." https://www.stereophile.com/content/mqa-benefits-and-costs#0PBbWjWSAvuoXysa.99, when in actuality a more appropriate response would be, "What are you even talking about? And...which one is it?" Instead, that reader is caught up in discussion that is built on a nonsensical premise. I posit that more vigilance is required at a more undamental level — akin to your present rhetorical question on blurring, i.e. Is this even a thing?

      *Incidentally my response, similar in content and milder in tone, has not yet showing online, apparently "awaiting moderator's approval." It would, at first glance, appear that one simply needs to pay the cost of entry and "be a customer" as has been mentioned by a poster there; unfortunately as far as the pursuit of clarity is concerned, it seems that while the cost of entry is low, the price is a but too high.

      Delete
    3. Thoughtful analysis Gnu.

      Yes, there are fundamental issues here of agreed-upon definitions that we all can apply to "meat-space". Clearly what you're also getting at beyond the content of thought is a concern around certain individual's thought form, the ability for insight and by extension applying this to logical discussion of the abstract.

      It is indeed possible that some individuals may not be able to do this, and hence discussions can never go beyond a certain concrete level... I honestly hope that this assessment is not true for some of these individuals and that in fact they are paid for their commentary and arguments rather than an apparent cognitive limitation!

      Delete
  3. Head-fi is notorious for this type of thing. Remember, the NWAVGuy got banned from that site for being objective, vendor critical.

    ReplyDelete
    Replies
    1. Didn't know he got banned from there, BJRMD.

      I suspect hard-core Flat Earth Society meetings would not desire to entertain the thoughts of "Round Earthers" either. Hopefully over time more will recognize the importance of objective methods to discern what is ultimately truth beyond one's own "truth".

      Delete
  4. I think there is a 'third way' that is neither fact nor opinion. It could be called logical deduction, or rationality, or 'Thought Experiment'. Or maybe just 'ideas'.

    It seems to me that most people don't think in these terms, and seek to reduce everything to facts or 'personality projections'. The potential for a useful outcome is limited...

    Facts and opinions could not take you from monophonic reproduction to stereophonic, or from analogue to digital. Progress depends on ideas, logic, and finally experiments. You can't measure something that doesn't even exist yet. And an opinion won't devise for you your first experiment.

    ReplyDelete
    Replies
    1. Well said therationalaudiophile,
      Indeed I also believe there is a balance to strike and what you use the word "rationality" to describe is a level of "informed reality" I like to see.

      For me, I've always thought that in life as well as audiophilia, we must have a basic level of being "more objective" in being sure we understand and "know" what's going on! Without technical knowledge, the audiophile stands on shifting sand with no foundation for what should be a hobby based off engineering and physics. Discussions end up being unfruitful if there is no basis for "truth". I find arguments with folks who subscribe to the value of exotic cables are of this nature. Like you suggest, discussions end up being in the realm of "personality projections" essentially based on nothing of substance. Worse, it allows a perpetuation in questionable companies to operate in this space and if this is a substantial portion of the hobby, it loses its educational value and respect from the broader technical public.

      The technical side ultimately must serve the subjective side. The subjective informs the ideals which we aspire to... I've always seen hi-fi equipment as serving the role of achieving a desire towards reproducing "reality"; that sense of "accuracy". Stereophonic is "more real" and "accurate" than mono. Multichannel when done right IMO is the next level but realistically tougher to implement well much of the time. Digital allows us to record and transmit with greater accuracy and reality by achieving greater dynamic range, time, and frequency domain accuracy than the inevitable noise and signal losses of analogue (yeah, I know the vinyl people will disagree... so be it...).

      Subjectivity also allows us to be free to choose those things which may not be viewed as ideal but idiosyncratic. The key is of course to be insightful enough that we can appreciate that idiosyncratic preferences are not things we need to fight for as characteristics key to sonic reproduction that everyone has to agree to. I might subjectively like rolled-off treble from 10kHz but this is not something I would fight for from manufacturers. But it might be something I'll program a room correction DSP to perform for my pleasure. Likewise, there's generally no factual accounting for very expensive speakers, cables, luxury electronics, etc. from the perspective of audio quality. That again is fine for me if one prefers something based on subjectivity so long as the hobbyist is insightful about the numerous reasons from which pleasures were derived.

      Delete
  5. I thought the interesting thing about Iverson's piece in Stereophile was that it focused not on the technical aspects of MQA, or on the subjective question of whether it "sounds better," but rather on the prospect of a proprietary format monopolizing the market for digital audio distribution.

    That's where the real danger lies. We can banter all day about the technical merits or demerits, or why it is that some people claim to prefer the sound of MQA (heck, some people prefer the sound of vinyl played back through tube gear), but if there comes a time when when all the music you want is available only in MQA format, those discussions will be moot.

    ReplyDelete
    Replies
    1. Exactly, there is danger here if the "business model" extends too broadly and affects consumer options. This was the basic rationale for the 3rd point in my Computer Audiophile article.

      The bottom line is potentially as simple as protection of "crown jewels". The purposeful restriction from allowing consumers that option of acquiring true "studio master" 24/96 and 24/192 recordings under the guise of something "better" in MQA. Might as well throw in some cryptography while they're at it of course :-).

      Delete
  6. A valiant attempt, but the 'subjectivists' should know all this by now...certainly this has all been explained before to multitudes of them in multitudes of forums.

    One thing I disagree with. Opinions *can* be 'wrong', or maybe more accurately, 'not even wrong'. An example comes from a phantom switch test, where the same device/stimulus is presented twice, though the subject doesn't know it. Not infrequently a subject will be of the opinion than presentation A and B sounded different, and one sounded better than the other.

    There are some audio fans that I know of who've gone from subj --> obj stances...often due to an accidentally self-administered 'phantom switch' experience. Experiencing how convinced they can be of hearing 'difference', when incontrovertably there was *none*, can shake a believer to his core.

    Opinions can be 'wrong' when their underlying premises are factually incorrect.

    IMO!


    ReplyDelete
    Replies
    1. Thanks for the note Steven,
      Yes, that is a good point that subjective opinions can be adjudicated as "wrong" in phantom switch situations. But that is assuming that the person himself/herself has insight!

      For example someone running a test with some exotic speaker cable and claiming to hear a difference may in truth discover later that he didn't switch the cable and was listening to 14G zip cord. But he/she might not be able to see the logical conclusion of this mistake... We might look that that situation and state that the individual is not being logical, or perhaps even not honest with himself. But based on "subjective" belief, the man might never agree with this possibility or then plug in the exotic cable and still claim it sounds better. Dishonest? Delusional? Do we have the right to judge? (I think we do, but some people might say we never have a right to judge anyone's character or mental state...)

      Sadly, IMO there are times when personality factors do play a huge part in the disagreements and arguments in life and maybe especially online!

      Delete
  7. Hi Archimago,

    I have read many of your posts, and come to a conclusion that the only components that matter are speaker, amp and DAC. Which is a very good news for me :)

    Does it means what people hear from network players like (eg: Linn Klimax, Lumin D2) is the product's DAC implementation, and digital transport (eg: dCS Network Bridge) should perform exactly the same as my Pi?

    Is there any parts that might affect performance, like the master clock? (eg: Mutec REF10)
    Haven't seen you write about clocking yet. This does look like sciences and more than jitter, I am looking forward to see some measurement of it.

    Best and thanks,
    KY

    ReplyDelete
    Replies
    1. Having an external master clock is really only important for studio usage where effect or any other DSP, and/or more than one ADC and or DAC need to sample at the exact same time.

      That's why those clock inputs exist and master (word)clocks are available.

      You need very expensive specialised test equipment to measure clock frequency deviations and irregularities accurately. A decent ADC/DAC won't be able to show the differences.
      Perhaps when you find any expect them to be around -140dB to -150dB.

      Those where expectation-bias is not really present may well hear obvious sonic improvements with their own ears though.

      Delete
    2. Thanks for your opinion.

      I have read RME ADI-2 DAC's document, a pro gear, it explains clocks in this way: "Usually a clock section consists of an analog PLL for external synchronization and several
      quartz oscillators for internal synchronization. " (page 53, https://www.rme-audio.de/download/adi2dac_e.pdf)

      If I am understanding this correctly, one think one component (ie DAC) could have multiple clock inside it.

      Delete
    3. They do.

      How much different clock signals there are (including derived or synchronised) depends on the internal circuitery and functionality.

      There are audiophiles that claim to hear differences between clocks.

      All digital circuits have 1 or more clocks. They determine the pace in which the digital signals pass through the digital circuits.

      The option of an external clock is to ensure several devices all 'sample' or 'process' at the exact same time which is not the case when several pieces of equipment are all on running on their own clocks.

      Clocks in the middle to higher priced gear usually is of excellent quality and won't improve in stability etc. when an external clock is used. The relevant internal clocks will be synchronized with that external clock signal.
      Some clocks (for say a display or other tasks) won't be synchronized nor do they need it.

      Delete
    4. Hi Ka Yue,
      I agree with Solderdude. Devices these days typically have multiple clocks but of course many subsystems are asynchronous. Ultimately, what is important for audio is reasonable accuracy of the clock used by the DAC in terms of target frequency so the correct pitch is achieved and of course making sure the clock stability is accurate (low jitter).

      Over the years, I have seen some variability in clocks between devices such as when I measure and null the analogue outputs from different DACs; such as this:
      http://archimago.blogspot.ca/2017/04/comparison-mqa-authentication-sound.html

      The tool Audio DiffMaker is used for this in order to subtly correlate the samples in order to perform the subtraction task. Microsecond differences in speed can be seen. In fact, I can see these tiny fluctuations as well depending on whether a DAC is just turned on and after it warms up for an hour.

      These days, the clocks are very accurate so the only thing we worry about is the variation that can affect the audio output - hence the focus on jitter.

      I have not played with wordclock syncs but from a consumer playback perspective, I really have not heard of a good rationale.

      Delete
    5. Indeed the analog null-tests show small differences caused by 2 clocks not being in sync. Both the playback and recording clocks are 'free running' and thus mostly do not 'sample' at the exact same time causing small differences in the actual 'null' which would not have been there had both the playback and recording been done with equipment that was connected to a master clock.
      That's what such a master clock can be used for.
      It won't make the sound any different, only the null would be more 'constant' during the entire test track.
      The final test results would have been even better and more conclusive.

      The results of these null test thus show minute frequency accuracy (speed/frequency) differences between 2 clocks as amplitude differences.
      It won't show which clock was more 'accurate' nor shows phase noise or other jitter effects of any one of the clocks.

      To measure the (long and short time) accuracy of a single clock/DAC specialised (very expensive) test equipment is needed.

      But like Archimago also said ... clocks aren't a problem in any decent to excellent DAC's these days.
      Manufacturers of high-end DAC's and audiophools will have different opinions of course.

      Delete