3 like 0 dislike
asked in Open Science by (163 points)
Assume you submitted a manuscript to a respectable journal. You are close to acceptance, but reviewers/editor write: "We will happily accept your manuscript for publication; you only need to ..." and then suggest things that you clearly identify as p-hacking/ questionable research practices.

You already wrote a response where you argue why you'd rather not p-hack. But despite your elaborated arguments, the editor insists on doing it.

How can you react?

[Please write different approaches in separate answers]

1 Answer

2 like 0 dislike
answered by (30 points)
I think all analyses are fine as long as you characterize them what they are (confirmatory, exploratory, fishing expedition). So, in the interest of conflict avoidance, I'd conduct and run the analyses and make sure that in the manuscript I qualify them as what they are (e.g., "On an exploratory note we deleted participants with high values from condition A. Eliminating five people yielded a significant effect. As this analysis was not pre-registered, however, we caution to base strong inferences on this exploratory finding."). If editors insist you should run these analyses and disguise them as if they were confirmatory, I would contact the editor in chief.

Ask Open Science used to be called Open Science Q&A but we changed the name when we registered the domain ask-open-science.org. Everything else stays the same: We are still hosted by Bielefeld University Library.

If you participated in the Open Science beta at StackExchange, please reclaim your user account now – it's already here!

e-mail the webmaster