The False Balance

[This is also posted at the Canberra Skeptics Blog]

It would be odd to expect unanimous appraisal for any idea, even among experts. Humans aren't perfect computers, they bring their own biases and influence into what they do. So while there might be overwhelming consensus for a given idea among experts, chances are there will be dissenting voices.

Science is no different to any other discipline, while objectivity is the name of the game it must be remembered that science is a human enterprise. People make mistakes, see patterns that aren't really there, impose their beliefs into their interpretations, etc. But while there is always this problem, the process helps make it as objective as possible - evidenced by the totality of science on our daily lives.

Because we are all not experts in nearly everything, we rely on expertise to help us make as informed a choice as possible. But of course we as individuals also have biases too, so how do we overcome them when looking to experts for information?


A second opinion?
Imagine upon results from a doctor hearing you have lung cancer. She explains to you what those tests were and why the results indicate the problem. This might all be compelling but surely you'd want to make sure this is true, so you go to another doctor, this time a lung cancer specialist. He runs the tests and confirms with your first doctor that you do have lung cancer. And then just to be sure you go to a third who again confirms the diagnosis. And then a fourth, and a fifth, and a sixth - all confirming lung cancer.

Eventually you come across a doctor that says you don't have cancer, that those other doctors are paid by pharmaceutical companies to peddle their expensive cancer treatment drugs. Instead the tests are just showing inflammation which would go away with some echinacea and some reflexology.

Now all of those doctors might have similar expertise in terms of qualifications, and there are two different diagnoses as to what it could be. On the one hand you have cancer, on the other inflammation. In the absence of any medical training yourself, how can you trust one over the other?

Say there were 10 doctors: nine who would interpret the evidence as cancer and one who would see it as inflammation. If you went to two doctors, there are 45 different combinations you could end up with. Only 9 of those 45 combinations would have the inflammation diagnosis. In other word, 80% of people if they went to two doctors would not even know that there would be a disagreement. If someone went to see 3 doctors, there would be 120 combinations of which 36 would have inflammation as a minority explanation and 0 that have it as a majority. 70% of people would still be unaware of the potential diagnosis.


Voicing the contrarian
But what's the harm, I hear you ask, of putting alternate opinions of experts on the table? After all, the 1 doctor might be on the fringes but he is still a doctor. He's spent time studying medicine and the human body, even if he might disagree with others he is still qualified as an expert.

The harm is what it does is loft the opinions of one expert at the expense of many others. It's saying that an expert's opinion is proportional to how fringe that opinion is among other experts. In the case above, the opinion of the inflammation doctor is worth 9 times as much as one who thinks it is lung cancer because there are 9 times as many doctors who would think its lung cancer.

So if on a subject like evolution that there are something like 99% of biologists who support evolution and 1% who are creationists, then holding creationism and evolution as equal ideas is giving the opinion of creationists 99 times as much weight as those who oppose it.


The importance of expertise
The false balance comes by taking views espoused by experts as equal regardless of where the weight of expertise lie. It's not to say that the doctor saying inflammation is wrong because he is in the minority, but that us as non-experts should take into account where the weight of expertise lies. Otherwise we run the risk of choosing which experts to listen to based on our own bias, which is as useful as not having any experts at all.

It's easy to see obvious bias where an expert is up against a non-expert (climatologist vs politician, biologist vs preacher, doctor vs naturopath) but when its two experts up against each other, there is a grave disservice done to the audience. It's not taking a random sample of two experts from the field, it's deliberately seeking out dissenting voices as if having disagreement means balance. Fringe views do not mean controversy, it just means creating a false balance and does a disservice to the general population and to the disciplines being misrepresented.