“We don’t know how the results were obtained. The post-doc who did all the work has since left to start a bakery” reads a tweet with the #overlyhonestmethods hashtag. The hashtag is being used for scientists to discuss the elements of their methodology that do not get discussed in “proper” scientific papers.
In response to this series of tweets, others have been reassuring readers that #overlyhonestmethods is a “‘joke” hashtag, and should not be construed to reflect the actual state of scientific work. Why? What’s the big deal?
Part of it is about the ways in which we like to consider science. The societal discourse is that science (particularly lab science) represents a “pure” form of knowledge, unbiased by human perceptions, relationships, and pragmatism.
In some ways, that may be true (if I mix Flourine and Francium, for example, the result is likely to be explosive whether I believe it to be or not), but that does not mean science isn’t shaped by social, cultural, and institutional forces.
For example, the choice of what to research is highly political. During wartime, scientific research is devoted to things that may aid the war effort, from weaponry, to vehicles, to food preservation. Political priorities in certain regions, likewise, direct research dollars into forestry management instead of ecological preservation. The scientists who do this research direct their efforts in this way because that is the research they can get funded.
#Overlyhonestmethods is, among other things, exposing the very real social nature of scientific research, pointing out that scientists may time their experiments so as to avoid being the lab on evenings and weekends. Or that it is sometimes difficult to know how certain results were obtained because people leave the profession and can’t tell you.
These concerns — about recording knowledge, and people’s quality of life at work — exist in every other profession, but in most cases we don’t need to discuss those statements as a “joke.” This is because most other professions do not make the claim of presenting absolute truth. In telling the “unpublished” stories of scientific research, #overlyhonestmethods makes it obvious that scientists are people who face constraints — personal, relational, practical, and institutional — potentially shaking the trust people put in science to offer “the” Truth.
Anastasia Kulpa teaches Sociology at Grant MacEwan University in Edmonton, Alberta, Canada. Her research interests include the sociology of post-secondary classrooms and cultural vehicles for transmitting ideology (class, music, television, etc).
Comments 42
Methodological Honesty | Sean Carroll — March 5, 2013
[...] A provocative hashtag: #overlyhonestmethods. For when you want to know how science is really done. (Via Sociological Images.) [...]
Elena — March 5, 2013
Previously on PHD Comics. Also, have a Lady Gaga parody. You'd be amazed at how many song parodies by maths and chemistry people are on Youtube.
Yrro Simyarin — March 5, 2013
I think one of the most important concepts in science is to remember that it can never be considered "The Truth", but simply, "The Best We've Come Up With So Far."
Yannick — March 5, 2013
I can tell you that from my experience, it's more true than not. It's just unsettling because it's a behind-the-scenes look at how people work. I'm sure we don't want to hear what surgeons banter to each other either, or how cops talk shop on their days off.
At the end of the day, scientists do their best to produce good research, and of course there are hiccups along the way. Sometimes these hiccups aren't worth throwing away good science. It honestly looks worse than it really is.
Tusconian — March 5, 2013
The last one really gets to me. Not because I fear those scientists who may not be using reliable sources or reading the sources fully, but because I know how frustrating it is to run into that, even though I studied different subjects. Sometimes, your only options are wikipedia, unreliable articles (even news articles tend to be unreliable on most subjects), and stuff that's behind a paywall. Often enough, finding a book on the topic is improbable, or the book will only have a blurb on the topic, which is usually something that a college student, much less someone doing research at the graduate level or beyond, is supposed to already consider a given. And what use is a source that tells you less than a textbook? I can't really fault the sources, as I bet that's how they make most of their profit, but it seems like most decent news and scholarly material needs to be paid for, even though it's generally being used by people who are already paying a lot to do the work they're doing. IMO, it puts even more of a class barrier up to legitimate education than the cost of university.
Sartora — March 5, 2013
"In some ways, that may be true (if I mix Flourine and Francium, for example, the result is likely to be explosive whether I believe it to be or not), but that does not mean science isn’t shaped by social, cultural, and institutional forces."
I think a lot of the confusion results from the use of the term "science" as a blanket term for the people involved, their methodologies, their results, their theories and the phenomenon they are trying to explain.
If you take "science" in (A) It is scientifically proven that Flourine and Francium, when mixed, are explosive, and (B) Science is shaped by social, cultural and institutional forces, people will often assume you are implying that (C) The fact that Flourine and Francium become explosive when mixed is shaped by social, cultural and institutional forces.
In reality, of course, the term "science" and its derivatives means very different things in the two contexts. If everyone were clearer about what exactly they are talking about, we could probably make more headway in raising awareness of the effects social structures have on research and the dissemination of results, and thus combatting the warping influence certain power structures tend to have.
LynneSkysong — March 5, 2013
As an engineer that's done a good bit of R&D, I can confirm that paywalls, time management issues, product availability, quirky machines, and reagent reclassification have all affected how I've gone about doing an experiment. I call this stuff life, because science doesn't happen in a Utopian bubble.
myblackfriendsays — March 5, 2013
Glad to see some people being honest about this.
Lunad — March 5, 2013
I find most interesting the influences that are invisible to the scientists themselves. I wish I knew the source, but I remember reading about how medical textbooks tend to describe the egg as passive and receptive, and the sperm as active and aggressive. New research is calling this view more and more into question: so now the egg is a predator that conforms to a different female stereotype.
Djiril — March 5, 2013
I wonder if some of the anxiety is that it might give more fuel to people who want to cut funding for scientific research.
On another note, I hadn't heard of this hashtag before and I love it. Thanks for the laugh!
Vee — March 5, 2013
For some of these the joke is along the same lines as the old joke about the woman cutting the final two inches off every joint of meat because her mother did it and her grandmother before her, the punchline being that it all started because the original oven was very small.
It often doesn't matter if the PCR is 25 cycles or 26 or even 30, but scientists are trained to give precise details of their methods so that other people can replicate them and these protocols then develop their own mythology as they get handed down. The art comes in working out which aspects are essential and which can be varied (a really experienced scientist will be able to adapt their protocol to fit in a tea-break).
MPS — March 5, 2013
This is a keen post but I'm not sure the hashtag is playing of the discrepancy between actual science and the public perception of it. I think it's playing off the discrepancy between actual science and how it is presented in academic journals.
My impression is scientists are happy to be sincere about science: they are excited about what they study it and they really believe in what they do. (There are various cultural elements that select for this.) However academic journals / peer review impose strict constraints on how science is communicated. I think this creates the tension that the hashtag plays off of.
[links] Link salad thinks about clinical trials | jlake.com — March 6, 2013
[...] Protesting Too Much About #OverlyHonestMethods — This is both funny and not. Which is kind of the point of the article. [...]
kali — March 6, 2013
There's a difference between science as a method and science as a practice. That difference is not recognized very often and leads to people talking at cross-purposes. Science as a method is pretty solid, but science as a practice is flawed like any human endeavor would be.
Methodological Honesty | PERCEPTION MEDIA — March 11, 2013
[...] A provocative hashtag: #overlyhonestmethods. For when you want to know how science is really done. (Via Sociological Images.) [...]