Saturday, May 02, 2015

A lot of what is published is incorrect


Via the k2p blog. The Lancet recently published a piece about a symposium on the reproducibility and reliability of biomedical research, held at the Wellcome Trust which begins :-

“A lot of what is published is incorrect.” I'm not allowed to say who made this remark because we were asked to observe Chatham House rules. We were also asked not to take photographs of slides. Those who worked for government agencies pleaded that their comments especially remain unquoted, since the forthcoming UK election meant they were living in “purdah”—a chilling state where severe restrictions on freedom of speech are placed on anyone on the government's payroll. Why the paranoid concern for secrecy and non-attribution? Because this symposium—on the reproducibility and reliability of biomedical research, held at the Wellcome Trust in London last week—touched on one of the most sensitive issues in science today: the idea that something has gone fundamentally wrong with one of our greatest human creations.

Every now and then we hear these whispers about the untrustworthy nature of science and scientists, how too much scientific research is junk aimed at more funding and fashions rather than the advancement of human knowledge. Understandably the problem seems to be causing significant anxiety in medical fields, hence the symposium and the Chatham House rules.

Yet it is extremely difficult for anyone to put some kind of scale on the problem. There is a problem I'm sure, but how significant is it? To my mind it's another of those areas where we should do our own research and reach our own conclusions. Here's another quote:-

The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. As one participant put it, “poor methods get results”.

All the scientists I ever knew were decent people who would not compromise sound science. Times change though. During my working life bureaucracy, political fashions and the power of money became ever more important. 

Good scientists retired and numerous external pressures began to dominate the agenda. The integrity of the individual scientist gradually became unimportant, ineffective against a swelling tide of political, bureaucratic and financial exigencies. Finally :-

The good news is that science is beginning to take some of its worst failings very seriously. The bad news is that nobody is ready to take the first step to clean up the system.


All original material is copyright of its author. Fair use permitted. Contact via comment. Unless indicated otherwise, all internet links accessed at time of writing. Nothing here should be taken as personal advice, financial or otherwise. No liability is accepted for third-party content, whether incorporated in or linked to this blog; or for unintentional error and inaccuracy. The blog author may have, or intend to change, a personal position in any stock or other kind of investment mentioned.


Paddington said...

Read David Colquhoun's column at for more details. His claim is that the junior researchers are being pushed into dubious behaviour by the senior ones, thanks to the stupid rules at some universities on 'required' levels of funding. Strangely, the people who create and enforce those rules have often never produced anything at all.

Sackerson said...

Sounds like the world of compulsory education, too.

A K Haart said...

Paddington - thanks, I've bookmarked the link.

A related problem which appeared in my field shortly before I retired was the professional manager.

An MBA was deemed necessary and sufficient for a person, often a comparatively young person, to manage engineers and scientists without having the faintest idea what those engineers and scientists actually did.

So their management imperatives were not derived from engineering or scientific excellence. How could they be?

Sackers - some critics refer to it as an effect of managerialism, which may be so but do we need yet more isms?