By Ashley Moses, edited by Andrew S. Cale Each year, millions of scientific research papers are published. Virtually none of them can…
Recognizing “Spin” in the Scientific Literature
“The wheel is turning
and you can’t slow down
You can’t let go
and you can’t hold on
You can’t go back
and you can’t stand still
If the thunder don’t get you
then the lightening will”
— Robert Hunter lyrics, “The Wheel” (Grateful Dead)
Facts are facts and data are data—both reveal objective truths. But, subjectivity is introduced the moment those facts and data are interpreted. In other words, they are susceptible to spin, or the biased interpretation of facts and data to persuade opinion in favor or against those facts and data. Therefore, individuals who are not entrenched in the science community are at a severe disadvantage when interpretations of data and fact are spun. Such individuals will take interpretations at face value without the training to analyze the hidden meanings or view topics from different angles. This disadvantage is further underscored when such individuals have little to no background in science and don’t speak the language of jargon. Instead, the public relies on scientists to relay truth instead of being skeptical interpreters. And this has led to a large anti-intellectual movement among Americans who do not trust scientists or their interpretations of data. For example, see how the same climate change data lead the vast majority of climate scientists to implicate humans as a direct cause of climate change, whereas political pundits on the Right are skeptical and use the same data (or cherry-picked subsets of the data) to spin their interpretation that humans are not responsible. How can American citizens trust scientists in their community, and how will they be able to tell who is a trusted source of information? This is a critical question in the current political environment, and one recently examined by Kellia Chiu and colleagues in a 2017 paper in PLOS Biology.
Should there be a specific criterion that members of the community should be able to draw their conclusions from? Are there some scientific topics that are more susceptible to spin than others? Furthermore, how is spin defined by the non-scientific community and by those who are in the science community?
How is Science Spun?
Spin is a common tool of propaganda, often employed by media outlets to push an agenda. Within science, it is commonly used to generate ‘science hype.’ In these cases, the importance of scientific findings is overstated and relayed in an inappropriate manner. Spin has gained renewed interest by researchers as they have noticed that findings—sometimes preliminary ones—are being put forth in positive light so that they can be viewed favorably by the public. This begs the question: what motivates scientists to spin their results? What are their goals? Are they mainly grant-oriented, or career-advancing, or are there other goals they are pursuing from overstating findings? Perhaps, another question that should be asked is whether the prevalence of spin renders entire groups of findings as false, and whether this makes scientific findings with spin questionable material for public consumption. If members of the public take what they read at face value, how are they to know any better?
The Effect of Spin
Chiu and colleagues investigated the effect of spin in science and how it impacts public opinion. Additionally, they investigated the following question: what is spin’s impact on evidence-based policy decisions? For example, politicians of all ideologies often distort scientific information in order to achieve specific goals. If citizens are not aware of what scientists are discovering, they will not be able to make informed decisions, and as a result, citizens cannot pressure their representatives to do the same degree of thorough investigation of scientific findings to make informed policy decisions. The paper by Chiu et al. analyzed 35 reports that had investigated the concept of spin and its effects on clinical trials, observational studies, diagnostic accuracy studies, systematic reviews, and meta-analyses. Chiu et al. concluded that spin varied in different studies, and that the level or intensity of spin may be different in relation to the amount of funding that each study received, although there was inconclusive evidence to support this claim.
Types of Spin
So… how do we identify spin? As Chiu and colleagues point out, spin can take a variety of forms, including: 1) “inappropriate study given study data;” 2) “inappropriate extrapolations or recommendations for clinical practice;” 3) “selective reporting;” 4) and “more robust or favorable data presentation.” Let’s unpack these a little bit.
The first of these types, “inappropriate study given study data,” occurs when findings simply are interpreted incorrectly. Specifically, Chiu et al. found that this type of spin is commonly used in conjunction with casual (or colloquial) language, which in many cases has the potential to alter interpretation of the data. So how do researchers strike a balance between using scientific jargon and communicating an idea in language understandable by those not in the science community? Should there be a glossary of scientific definitions provided at the end of an article which readers can refer to?
The second type of spin, refers to using statistical evidence—unintentionally—to support a statement when the evidence does not point to the conclusion. For example, how often do we lament the weather forecast being incorrect? From a statistical standpoint, we shouldn’t ever be surprised when the forecast doesn’t match the weather because the forecast is based on extrapolation of data, and extrapolated data are really just estimates. So, the relationship between final conclusions and the extrapolated data upon which they are based is weaker than if the data had been measured directly.
The final two examples of spin are related in the sense that “selective reporting” of some (but not all) data can lead to “more robust or favorable data presentation.” In other words, leaving out some key pieces of information can enhance the interpretation of data and shed then in a positive light when they may not be favorable when viewed in total.
Where is the evidence of Spin?
Chiu et al. found evidence of spin in nine reports examined (9/35), across eight scientific findings that included spin. These researchers examined the conclusions of these scientific trials and found that conclusions were heavily spun the data were inconclusive to answer the question being asked. By saying inconclusive, we mean to say that the results neither confirmed nor disputed particular findings. Other studies suggested that high spin can be characterized by lack of (statistical) uncertainty when framing conclusions, no recommendation for further trials, or no acknowledgement of the statistically nonsignificant outcomes. Another study by Clement Lazarus and colleges reported that 128 abstracts of nonrandomized interventional studies included “tones” that suggested the outcomes came as a result of the intervention or used strong language to relate as much, without any actual evidence of a causal relationship. These “tones” included phrases such as: ‘the study shows that’ or ‘the results explain,’ even when there was no real evidence to support such claims. Therefore, spin such as that described here, can mislead the public and other researchers who may not read critically a paper that has been peer-reviewed in a major journal. The types of reports that were deemed most severe were concluding recommendations for clinical practice when not supported by results, titles that claim that the treatment is beneficial when not supported by the results, selective reporting, or overemphasizing results that favor a specific beneficial intervention.
What does this tell us?
What do these reports tell us the prevalence of spin in the scientific literature? Should readers place all their faith in one article they read? Or is it better for lay people to gain knowledge pertaining to specific findings from different reputable sources? Just as we colloquially advise to never place all eggs in a single basket, so it with scientific literature–reading more than one source is always better than relying on a single source. This is true for researchers and the public alike.
About The Author
Tasnim Elmamoun, an undergraduate student at IUPUI, is double majoring in Secondary English Education and Psychology, while also minoring in Arabic and a certificate in Pre-Art Therapy. She hopes to write, teach, and participate in community work in her future. Tasnim is working on SciCommPLOS through the IUPUI Life-Health Sciences Internship program, which connects undergraduates to high-quality, structured internship experiences that encourage career exploration, skills development, and professional growth.
Edited by Jason Organ, PhD, Indiana University School of Medicine.
Want to stay connected with #SciCommPLOS or pitch an idea for a blog post? Tweet us at @SciCommPLOS or email us at scicommplos@gmail.com.
“For example, see how the same climate change data lead the vast majority of climate scientists to implicate humans as a direct cause of climate change, whereas political pundits on the Right are skeptical and use the same data to spin their interpretation that humans are not responsible.”
Not true. They almost always use cherry-picked subsets of the same data, or confuse what the data is, which lead to one conclusion that the rest of the data taken as a whole do not support. Example: “satellites shows the upper atmosphere is COOLING, so global warming isn’t happening”. (That doesn’t show what they think it shows.)
Good point. We have edited our statement to reflect this. Much appreciated.
All my life I have been told about imminent crises that will soon befall us. The Population Bomb and Peak Oil are two that come to mind. They are always pushed by progressives who say Man is the problem, and that only by growing the Government and giving it more power can it be solved.
Progressives are the Boy that Called Wolf, and I for one will remain skeptical until someone I trust provides the evidence.
If you have any question about the author being progressive, just Google her.
Why do you not trust the scientists who study climate change? Are they not the experts? How can we restore the public trust in science?
Reason and the laws of thermodynamics tell us that every activity of mankind warms the surroundings. No one on the Right disputes this. The question is the degree if any that the climate is changing, and if so, how much is natural, and how much is man caused, and what should be done about it.
The author embraces the left’s spin that if anyone disputes or even questions that we must immediately disrupt our entire way of existence while giving the government the power to enforce it, even while “the scientists” cannot tell us what will be the benefit, then they are “Climate Change Deniers.”
In fact, this article is an attack on the right by a leftist author. I notice that no example of biased interpretation by the left was cited. Just to give one, there is the denying of racial differences, or that race is anything but a “social construct.”
Getting back to my original comment, during my lifetime I have seen the left run around screaming “The Sky is Falling!” many times, over some issue or other, quoting expert after expert, but it all came to naught, except for expanding the government’s power over us. I have come to think that this is their goal after all.
Is the left right about catastrophic climate change after being wrong for so long? Perhaps, but I don’t want to be fooled again, with a hellacious cost.
I still don’t trust the subjectivity of their interpretations.
This blog post is a review of an article published in PLOS Biology. You are correct that a single example is given here — that of climate change. But to label this post as “an attack on the right by a leftist author” is simply misguided. You may want to re-read this post to see that the author reviews how the original science posted in PLOS Biology examines “spin”. And in fact, to your final point that you still don’t trust the subjectivity of data interpretations… YES! That is exactly what this blog post is directed toward.
For some reason my last reply is not posted. You or someone else must not like the points I attempted to make. So to tiptoe around their sensibilities I will address one question you asked.
“Why do you not trust the scientists who study climate change?”
The two papers the article cites are an examination of the spin and distortion by researchers on the results of their own studies and trials.
Apparently not all “Experts” can be trusted. Especially when another layer of spin is place on the results by others with their own agendas, as this article does.
Don’t be disingenuous, Berkley. You are aware of the reason that your last post was not published because I detailed the reason in a respectful email to you. I will not publish ad hominem attacks on the authors of blog posts. You have been warned privately that this will not be tolerated. Now I am warning you publicly.
But to your point in this most recent comment… You have completely missed the point of what our author has written. She has reviewed a paper in PLOS Biology that investigates the prevalence of “spin” in the scientific literature.
The statement that has upset you is the following: “For example, see how the same climate change data lead the vast majority of climate scientists to implicate humans as a direct cause of climate change, whereas political pundits on the Right are skeptical and use the same data (or cherry-picked subsets of the data) to spin their interpretation that humans are not responsible.”
The point the author is making here is that the SAME DATA are used by opposite sides of the “argument”… NOT the INTERPRETATION of those data, but the data themselves. They are interpreted differently, and the interpretations are polar opposites: humans are causing climate change vs. humans are NOT causing climate change. So, your point about not trusting experts who spin their interpretations is invalid here, and is exactly the point that our blog post author was making: data are objective, interpretations are subjective. But when the objective data are spun in opposite directions, how can the public trust the spin?
So, please take a step back and realize that you and the author are on the same page here. But your biases are getting in the way of clear reading comprehension.
Again, we welcome you to make any comment about the validity of the SCIENCE we discuss. But ad hominem attacks on our authors will get you banned from making further comments on this blog. Am I clear?