Menu Home

Retractions and academic misconduct: Coming soon to a discipline near you?

The subject of retractions has been gaining a lot of steam in the media recently, with several recent studies (outlined nicely in this New York Times article) showing that retractions are on the rise, and misconduct and falsifying data are one of the most common reasons. Yesterday on Twitter, I noted that, while the language used to describe these retractions was very inclusive (“scientists,” “science articles,”) the fields that the retractions were in were largely limited to the biomedical and life sciences. A discussion ensued, with folks like Ed Yong pointing out that lack of evidence doesn’t necessarily indicate lack of fraud in other fields. At this stage, it would be useful to know whether this is a real or a perceived difference in misconduct rates, not only because the culture of how we do science may be relevant, but also because we need to make sure that the public doesn’t lose trust in science and scientists in general.

“Scientists” are not a monolithic group, and it generally irks me when we’re represented that way. Still, I’m increasingly convinced that misconduct is a problem that all of us should start taking seriously. As these retraction announcements have been coming out, I found myself shaking my head and chalking up the difference to biomedical and life science institutions and cultures. I was making the assumption that falsifying data was not something that was going to happen in my fields; it just didn’t fit with the ecologists, geologists, and climate scientists I know. I couldn’t imagine a world where people who cared about their research and its implications to the environmental problems we face would want to falsify results– it seems so counterintuitive. I trust my colleagues, for one. “I know people are collecting field data,” I thought. “I have access to their code. I just don’t see it happening.”

But after yesterday’s discussion, I’m thinking more deeply about this. Data can be manipulated. Statistics can be tweaked. I don’t want to mistrust my friends and fellow researchers, but a healthy dose of skepticism has helped me realize that “oh, it’ll never happen in my field!” is precisely the wrong way to think about this issue. As Ed Yong pointed out yesterday, it’s not like folks in retraction-heavy fields were thinking, “we’re really at risk and need to be on our guard against things like this happening!” There’s also a large continuum of misconduct. On one end, you’ve got falsification of entire datasets. On the other end, you delete a data cell or drop a sample from an analaysis because it stands between you and statistical significance (I’ve never, ever done this, but it’s not difficult to imagine). You know your data– that data point is weird, and it’s ruining the last year’s worth of work you’ve done. It’s such a simple fix, right? But it’s still wrong.

Obviously, this is a good time for a lot of self-reflection within the scientific community as a whole, and its many branches. One thing I pointed out is that there are very large institutional and cultural differences amongst various scientific disciplines when it comes to funding, publishing, important metrics for hiring, collaborations, data sharing, etc. In my fields (ecology and earth science), single-investigator studies (and single-authored papers) are rare, and open data is more common than in some other fields. If there is an actual (and not just perceived) difference in misconduct, do these cultural differences play a role? Not everyone publishes 170+ papers in twenty years (like anesthesiology researcher Dr. Yoshitaka Fujii, who is believed to hold the record number of retractions). Do things like research output expectations, cutthroat grant requirements, or other pressures contribute to increasing fraud? Are biomedical retractions more common because the immediate stakes– human health and safety– so much higher than, say, a new fossil discovery, and therefore biomedical papers are under more scrutiny, so falsifications are more likely to be caught? Have things always been this way, but improved communication is allowing us to catch cheaters more?

On one level, I hate this. I hate that there are people out there who are shaking the public’s trust, when that trust has been so hard-earned to begin with (and is often shaky anyway). As someone for whom research is an act of love, it makes me sick that people would just make things up for personal gain; it holds up the entire pursuit of learning about the universe (other than the fact we already know, that people lie). On the other hand, maybe science is due for a good shake-up. If this helps us take a good look at how science is funded, reviewed, published, disseminated, and shared, that’s a good thing.

And finally, I can’t talk about retractions without a nod to one of my favorite sites, Retraction Watch. I wish we didn’t need it, but I’m very glad it’s there.

Categories: Academia Commentary

Tagged as:

Jacquelyn Gill

6 replies

  1. I have an old post asking if misconduct is rare in ecology and evolution. The answer, as you note, is that it’s hard to say, because ecology and evolution studies are so rarely replicated (often, replication is impossible).

    https://dynamicecology.wordpress.com/2011/11/03/is-scientific-misconduct-especially-rare-in-ecology-and-evolution/

    Also, ecology and evolution studies don’t usually involve figures where falsification can be detected by careful inspection (manipulated gel images are one big way data fiddling gets detected in biomedical fields).

    Like

  2. Thanks, this is a great post! I have been very interested in open data and included the R scripts as a supplement to my last submission (I’m re-writing others or an upcoming submission right now), with the idea that it’s the analysis that we most need to be careful about, but the fact is, in paleoecology, there’s very little stopping someone from falsifying count data from a pollen record. It is time consuming to count a sample, how likely is it that the sample would be re-counted? That’s the premise behind the Dirk Smeesters retractions as well (e.g., http://retractionwatch.wordpress.com/2012/06/25/following-investigation-erasmus-social-psychology-professor-retracts-two-studies-resigns/ ), he concocted survey data that simply didn’t exist, but to revisit the data would be time consuming. Certainly, including scripts would help in determining when data were over-used or mis-used, but they don’t get at the underlying problem that the counts might be falsified. When think about that, and all the data in Neotoma (httlp://www.neotomadb.org) it makes me sick to my stomach.

    Like

    1. I think you’re right– and, honestly, what’s there to stop people from falsifying data from any field-based study? Spending a year in the field measuring some variable is time-intensive, and not everyone has access to the same study sites. Even more fundamentally, who has the time and money to replicate studies? I’m fortunate that the grant that funded me was set up precisely to reinvestigate old sites, but that’s pretty rare (which was definitely brought up in the grant reviews). Does collaboration help? Or having a lab structure with a lot of vertical integration from the undergraduate tech to the PI getting funded? Or do we need to revisit how publishing, tenure, and funding happen? And how likely is THAT? Should we build more cross-checks into our projects, along with broader impacts? Should we be sending samples across labs for cross-validation? I think definitely yes, to the latter, and I’ve done a bit of that (on both ends). Still, it’s really hard to shake that sense of not being my problem. I don’t want to think that I can’t trust people.

      Like

      1. Not sure how much collaboration helps. If one person is the source of the raw data, then the collaborators pretty much have to trust that person. As noted in my old post and the comments on it, and in downwithtime’s comment, plenty of people who’ve been caught falsifying data, including a few who’ve been caught doing so in ecology and evolution, were part of collaborative projects. Like downwithtime, I doubt that having collaborators, or having mandatory datasharing, is much of a deterrent to people who are otherwise inclined to falsify data. I suspect the bottom line is that, for better or worse, we’re all just going to have to keep trusting each other. Because the kind of checks that would prevent falsification of raw data are *really* onerous (I mean, you’d have to somehow ban things like people collecting data on their own, which is totally infeasible). Plus no one sees it as their job to develop and enforce such checks.

        Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: