When editors mistake direct criticism for discourtesy, science loses – Retraction

Jasmine Jamshidi-Naeini

In January 2022, motivated by our experience with eClinicalMedicine, we wrote about the mishandling of published errors by journal editors. We noted that the methods used to analyze a cluster randomized trial published in the journal were invalid. Using a validation approach, we reanalyzed the raw data that were shared with us by the original authors. The results of the trial were overturned.

As Retraction Watch readers may recall, we subsequently submitted a manuscript describing why the original methods were invalid, what a valid analysis should look like, and our results after performing a valid analysis. After initially rejecting the bureau “in light of [the journal’s] pipeline” and further correspondence, the journal shared our findings with a statistician involved in the original review and the original authors and sought their responses.

After receiving the responses, both of which contained factually incorrect statements, the editorial team ultimately suggested that we summarize our full manuscript as a 1,000-word letter for submission to the journal. We did not agree that a letter would allow us to fully communicate our methods and reanalysis. Thus, in order to comply with the journal’s word limit while fully presenting our arguments, we published our additional points as a preprint and cited the preprint in a letter we sent to the journal.

Then we ran into another obstacle to correcting the literature.

Colby Forland

We received the first edit request from the editorial team with suggested tracked changes to the wording of our letter. We were asked to remove any wording referring to the original analysis as ‘invalid’ or ‘incorrect’ and any wording referring to our re-analysis as ‘valid’, ‘correct’ or ‘legitimate’. These edits downplayed our reanalysis and the conclusions drawn (which overturned the results of the original analysis), referring instead to “a different interpretation (with a different analysis).”

The editorial team had changed our main message about the incorrectness of the original analysis and conclusions and the correctness of our reanalysis approach. The editorial team also removed citations to a link to supplementary information where we provided depth and clarity to interested readers, including our reanalysis statistical code.

Over two rounds, we tried to meet the editorial team’s expectations by partially accepting their proposed revisions and providing our arguments for why other revisions would change our intended message and were therefore unacceptable to us. Our responses were rejected and the editorial team required the exact same revisions each time. It sounded like an ultimatum to us: either publish with the corrections the editors wanted, or don’t publish the corrected results at all.

Although we disagreed with their edits, we were eventually forced to accept them. The letter was finally published on October 6, 2022. The original document remains uncorrected.

Andrew Brown

This was not the first time we were asked to revise phrases such as “the analysis was incorrect” or “the results were overturned by using a valid analysis” into phrases such as “an alternative analysis showed something different”. Calling a “valid alternative” analysis an “alternative” analysis does not make it clear to readers that the original analysis was clearly incorrect by any reasonable standard of statistical knowledge.

When we identify unambiguous errors in the published literature, we often report them to journal editors. Some journal editors see what we consider simply clear and straightforward statements about the correctness, or lack thereof, of such analyzes as somehow impolite, unfair, or as a reviewer of one of our manuscripts describes, “unnecessarily pejorative.”

There are certainly cases of rude behavior or worse in science. Michael Lauer, NIH’s deputy director for extraordinary research, recently shared some true stories of NIH staff and review committee members facing “inappropriate and discourteous behavior” from applicants. Lauer’s examples include using an aggressive tone, condescending, or conducting offensive correspondence. This is all rude, impolite and dishonest behavior.

David Allison

However, there is nothing impolite or unfair about saying that a particular analysis is incorrect, flawed or invalid and therefore that the conclusions drawn from it are either invalid or unfounded. Editors’ struggle to distinguish impoliteness from directness may be partly related to an idea we call the “second demarcation problem”: Some editors have difficulty (or are unwilling) to distinguish unambiguous errors from matters of subjective scholarly opinion. The former needs to be corrected, while the latter deserves scholarly debate.

On the other hand, those engaged in public research criticism to promote rigor, reproducibility, transparency, and trustworthiness in science sometimes interpret the encouragement to be polite and courteous as an encouragement to remain silent.

Lillian Golzari-Arroyo

But one can criticize professionally, politely, constructively, and as noted earlier,directlywithout being silent. We should not be silent when we see gaps in the research literature. We need to have a dialogue and point out the mistakes we find. However, we should not allow the need not to be silenced by our critics to be a license for rudeness or personal attacks.

The clear distinction between errors and legitimate scientific debate should not be undermined under the guise of politeness. Obscuring an obvious error by not admitting its incorrectness, or demoting a valid reanalysis by calling it an alternative analysis, creates the impression that the invalid approach can be considered correct. This undermines the integrity and reliability of science.

There is no passive magical process by which science corrects itself as an anthropomorphized nebulous figure. Maintaining the self-correcting nature of science requires scientists to correct science from the field of science, and scientists can be polite, courteous, constructive, and direct when doing so.

Jasmine Jamshidi-Naeini is a postdoctoral fellow and Colby J. Vorland is a research assistantt at the Indiana University School of Public Health in Bloomington, where David B. Allison is dean and Lillian Golzari-Arroyo is a biostatistician in the school’s Biostatistics Advisory Center. Andrew W. Brown is an associate professor at the University of Arkansas for Medical Sciences.

Like Retraction Watch? You can do a tax-deductible contribution to support our workFollow us on Twitterlike us on Facebookadd us to yours RSS readeror subscribe to our daily digest. If you find a retreat, that is not in our databaseyou can let us know here. For comments or feedback, email us at [email protected].

Leave a Comment