Practicing Safe Science
The scientific method, as with any process, is not immune to adverse human influences.
Journalism has a persistent bias for the new and exciting. They sell in pop culture, and as it turns out, they sell in scientific culture too. This creates unintended consequences.
Unlike pop or mainstream journalism, objectivity and peer review form critical cornerstones of science’s scientific method. Summarizing “Journalistic Deficit Disorder” (The Economist, September 22, 2012 edition) and “The Truth Wears Off” by Jonah Lehrer (The New Yorker, December 13, 2010 edition), scientific journals tend to prefer studies that:
- Will sell more publications
- Explore popular fields
- Produce exciting, outlying results
- Prove their hypotheses
- Are new, not reruns of previous studies
- Produce supporting results for a new, fashionable paradigm
- Have substantial corporate investment or interest
These tendencies pressure scientists and researchers whose careers, reputations, incomes and funding depend on publicity their works receive. Several consequences undermine the credibility of science and research as a result:
- Emphasis on proving outlandish hypotheses
- Diminished importance of peer review
- Increased biases in interpreting data and statistics
- More focus on confirming popular findings or those with substantial financial backing
- Defunding contrarian work
- Skewing results toward extremes
Exciting often means extreme. In science it’s outlying results such as found in the bell curve. As Lehrer writes, since outliers receive the press, duplicating results is often difficult. Therefore, while hypotheses might be true, they’re not as true. However, as is more often the case, results are wrong, caused by inadequate research methodology, poor statistical analysis or normal human biases.
In other words, we can’t practice safe science by simply relying upon the scientific method. Human nature is too strong, even in scientists and trained researchers. We need to provide our own protection. That means educating ourselves on the scientific method and on the questions to ask. It also means taking nothing on blind faith . . . even science.
- Problems with Science
- Practicing Safe Science
- Science’s Subjective Birth
About Post Author
Very interesting “Economist” and “New Yorker” articles – fascinating to this former engineering researcher AND recommended to readers for their consideration.
I’m a believer that MOST researchers make an effort to be as transparent, thorough, and honest in their efforts – including documenting / reporting. Of course, there are those who are not. I recall my FIRST conference attended as a PHD student. A researcher presented slide after slide of the complicated mathematics for a solution to a differential equation. Upon completion, the first question asked was this: “Are you aware that the differential equation you discussed has been shown not applicable to the (theme of the conference)?” To which the presenter replied: “Yes I am; and I have an even better solution to be published in two months!”
That was about fifty years ago… Relevency and applicability are typically demanded today. And on another related point, I’m wondering about the strict use of “The Scientific Method” currently. Scientist or engineer, I think the justification of proposed work and the complexity of the real world (with impact so well documented in the two articles) make the research seeking the most defendable and USEFUL outcome(s) doable whereas proving or disproving a hypothesis highly unlikely – despite best efforts!!!
Correct answers always exist of course BUT WE VERY RARILY KNOW WHAT THEY ARE!!! We quote some values as constants (e.g., the universal gas constant) but only because they have been measured so many times that the biases in any measurement have been minimized and the overall uncertainty in the value approaches zero. Far better is reporting outcomes accompanied by full honest discussion of methods used AND including honest estimates of uncertainty. The desire is that many applications of these outcomes will be USEFUL in addressing important situations (some not useful however)! Non-usefulness can happen because the uncertainties of the outcomes are too large OR that the conditions not addressed in developing the methods (but appropriate to the application) are significant.
My honest but not intentionally pessimistic advice to students is this: Expect any currently USEFUL approaches / research results to eventually be NOT useful; the real world we live in is simply too complex…
John, I agree that most are as you say. Still, they are not immune to unconscious influences. I do agree, if I’m interpreting your comments correctly, that the scientific method has been extended and applied for some dubious purposes. Yes, I too, am concerned about complexities, uncertainties and uniqueness in life’s events. For example, a friend said, “Mike, if I let go of my coffee cup, it will fall to the floor. That will happen every time.” I replied, “That is true, but it and the coffee in it won’t fall exactly the same way and produce the exact same event each time.” Life always applies some unique element that will make everything and every moment different from the one before and the one after. It’s this uniqueness that also afflicts experiments and research especially in the soft sciences.
I like your last paragraph, John, so true. New research is always seemingly overturning existing thought. Thank you again. ~Mike
Extending your comments on my last paragraph: Even the incomplete understanding of any research because of the complexity insures that our ever expanding use of that understanding will lead to “the usual approach” no longer being useful / helpful.