3 like 0 dislike
in Open Science by (190 points)
Assume you submitted a manuscript to a respectable journal. You are close to acceptance, but reviewers/editor write: "We will happily accept your manuscript for publication; you only need to ..." and then suggest things that you clearly identify as p-hacking/ questionable research practices.

You already wrote a response where you argue why you'd rather not p-hack. But despite your elaborated arguments, the editor insists on doing it.

How can you react?

[Please write different approaches in separate answers]

3 Answers

3 like 0 dislike
by (40 points)
I think all analyses are fine as long as you characterize them what they are (confirmatory, exploratory, fishing expedition). So, in the interest of conflict avoidance, I'd conduct and run the analyses and make sure that in the manuscript I qualify them as what they are (e.g., "On an exploratory note we deleted participants with high values from condition A. Eliminating five people yielded a significant effect. As this analysis was not pre-registered, however, we caution to base strong inferences on this exploratory finding."). If editors insist you should run these analyses and disguise them as if they were confirmatory, I would contact the editor in chief.
1 like 0 dislike
by (40 points)
If you are going to do as Roland suggests, then I would recommend using Bonferroni correction, to adjust for all the possible analyses that you could have done. But sometimes that is hard to estimate with a flexible analytic pipeline.
1 like 0 dislike
by (40 points)
Another option would be to send the editor to a Gulag for re-education. I would volunteer to be chief guard.

Ask Open Science used to be called Open Science Q&A but we changed the name when we registered the domain ask-open-science.org. Everything else stays the same: We are still hosted by Bielefeld University.

If you participated in the Open Science beta at StackExchange, please reclaim your user account now – it's already here!

E-mail the webmaster

Legal notice

Privacy statement