In my post last week about Strelitzia reginae (the Bird of Paradise plant), I wrote about how it had been initially mistaken as another plant, then later reclassified as a new species. One of the documents I found that supported this fact (and several others) was a collection of Dutch history with an extraordinary long title (the relevant part of the document was graciously translated by Grayson Morris). This correction got me thinking about the nature of error and validation in modern science. I started doing some research, and what a rabbit hole I’ve fallen into!
Published science has a variety of problems. It always has. I’ve seen a lot of headlines from the past few years floating around with titles like the optimistic “A Sharp Rise in Retractions Prompts Calls for Reform” from the New York Times, the neutral “How Science Goes Wrong” from the Economist, and the pessimistic “Trial and Errors: Why Science is Failing Us” from Wired. The content and tone of these articles vary, but when seeing these and similar articles all together, it is easy to start thinking something is wrong at a fundamental level.
I’ve seen editorial retractions of controversial papers that overstate their results (GMO corn causes tumors in rats). I’ve seen a paper with obvious errors accepted to more than 150 open access journals. I’ve seen a computer generated paper published at a spurious Indian engineering seminar/conference, and similarly generated papers accepted to a journal in Russia and a Romanian Magazine. I’ve seen a highly touted study published with exaggerated results from a major research institution get lost in a scandal including a LOT of red tape and the death of one of the researchers. And I’ve seen the story of a Dutch psychologist who had been faking papers for twenty years.
How are these studies even accepted in the first place? How can made up studies even get past any scrutiny? If problems like these happen in labs at some of the greatest institutions in the country, how widespread are they? In the face of these many examples of scientific blunder, it is hard not to look for a single problem. It is hard not to think the scientific method is seriously flawed and its results shouldn’t be trusted if I can’t verify them myself.
What I take away from all of this, however, is not distrust in science. The processes of science is still as it always has been (at least in my lifetime): empirical study and verification of knowledge. Trust, but verify. What I take away is an appreciation of the complexity of problems that emerge when science is combined with the reality of human error and competition.
In my next post, I want to delve deeper into some of these emergent problems. I want to tease apart the processes of science (including research, analysis, and publishing), see what’s being done poorly, look at what is being done to correct it, and explain why published science is still worth trusting.