Saturday, August 08, 2009

Bias

Thought-provoking article in The Guardian today, about how even medical research can end up with an unreliable consensus skewed by influential reviews:

A small number of review papers funnelled large amounts of traffic through the network. These acted like a lens, collecting and focusing citations on the papers supporting the hypothesis.

Worse, science can be "spun":

One paper reported no beta amyloid in three of five patients with IBM, and its presence in only a "few fibres" in the remaining two patients; but three subsequent papers cited this data, saying that it "confirmed" the hypothesis.

This is an exaggeration at best, but the power of the social network theory approach is to show what happened next: over the following 10 years these three supportive citations were the root of 7,848 supportive citation paths, producing chains of false claim in the network, amplifying the distortion.

This leads one on to consider the implications of social network theory. I suppose it's a talent for this that helped Mao and Stalin rise, but also it may explain how people in other fields (e.g. finance and banking, the media) can be both successful and dangerously dumb. (Remember Mao's bright idea of 1958, culling sparrows because they ate crops? The resulting explosion in the crop-gobbling insect population forced him to ask Russia for thousands of birds to restock).

And have you watched the celeb version of "Who wants to be a millionaire?" and been struck by the ignorance of some of them? Yet they know enough (of what they need to know) to make a sight more than most of us. The technique seems to be, get the job first, then learn how to do it from those around you. Duffers try to learn first, then apply for the post, by which time it's gone. Look at chancer Blair as against plod-towards-it Brown. (Some say that Blair has never read a book; but then, he doesn't need to. As Disraeli said, "When I want to read a book, I write one.")

Connected to social network theory is Cass Sunstein's notion of "group polarisation", where like-minded people get together, not only reinforcing their views but making them more extreme. I suppose this has implications not just for political caucuses and media advisers, but for how we choose our newspapers.

And what blogs we read.

2 comments:

James Higham said...

That's so. I wrote a derivative piece two years ago based on an excellent article on the fallacy of research.

Paddington said...

My money is still on the scientific process to determine facts (not truth) min the long term. It's humanity's greatest invention, which attempts to eliminate wishful thinking from actual data.