.
Kent Staley
Professor
Department of Philosophy
Saint Louis University
Commentary on “The statistics wars and intellectual conflicts of interest” (Mayo editorial)
In her recent Editorial for Conservation Biology, Deborah Mayo argues that journal editors “should avoid taking sides” regarding “heated disagreements about statistical significance tests.” Particularly, they should not impose bans suggested by combatants in the “statistics wars” on statistical methods advocated by the opposing side, such as Wasserstein et al.’s (2019) proposed ban on the declaration of statistical significance and use of p value thresholds. Were journal editors to adopt such proposals, Mayo argues, they would be acting under a conflict of interest (COI) of a special kind: an “intellectual” conflict of interest.
Conflicts of interest are worrisome because of the potential for bias. Researchers will no doubt be all too familiar with the institutional/bureaucratic requirement of declaring financial interests. Whether such disclosures provide substantive protections against bias or simply satisfy a “CYA” requirement of administrators, the rationale is that assessment of research outcomes can incorporate information relevant to the question of whether the investigators have arrived at a conclusion that overstates (or even fabricates) the support for a claim, when the acceptance of that claim would financially benefit them. This in turn ought to reduce the temptation of investigators to engage in such inflation or fabrication of support. The idea obviously applies quite naturally to editorial decisions as well as research conclusions.
Mayo’s “intellectual” COIs differ from this familiar case. The relevant interests of (in this case) journal editors are not financial, but concern policies governing the conduct of science itself.
One might object that journal editors are entrusted with decision-making power precisely to adopt and act upon such policies, and this distinguishes intellectual COIs from financial ones. Journal editors, according to this view, are responsible for making informed and reasoned judgments about the standards that distinguish credible research conclusions. They cannot do so if they are barred from adopting standards in accord with their personal judgments. To have an intellectual interest in a policy is simply to think that it is a good idea, and shouldn’t journal editors act on good ideas when they (think that they) have them?
To continue the objection, take an example from the field of particle physics: The editors of Physical Review D surely ought to be free to impose the requirement that claims to have “observed” a new phenomenon cannot be published unless the putative signal for that phenomenon constitutes at least a 5s departure from the null hypothesis prediction. They ought to have the ability to impose such a requirement even though there are some members of the particle physics community who are critical of that policy, or who (perhaps because they prefer Bayesian analyses) reject even the use of significance calculations as a requirement of discovery claims.
Perhaps such an objection might be encouraged by the idea of an intellectual COI, but I think it misses the point of Mayo’s argument. The dispute within the statistical community over significance testing, and the “statistics wars” more generally, is fundamentally a philosophical one, or at least involves, in Mayo’s words, “philosophical presuppositions.” These presuppositions concern such fundamental aspects of scientific inquiry as “what is the purpose of a statistical test?” and “do the beliefs of investigators matter to how the results of inquiry are characterized, and if so, how?” Philosophical disputes tend to have a bad reputation among non-philosophers because they are often thought to be never-ending or even unresolvable in principle. Perhaps some are, but even in those cases (and I don’t think this is one), there is at least the possibility for progress in terms of clarifying what is at stake and eliminating non-viable positions from consideration. In any case, so long as competing methodological approaches in a given field rest upon differing philosophical presuppositions, about which there is legitimate and ongoing disagreement, to preclude the use of one of those approaches as a matter of editorial policy would be to foreclose on the possibility of engaging that philosophical dispute at the level of scientific practice. The consequences of that foreclosure for the scientific discipline itself would be impoverishing.
All commentaries on Mayo (2021) editorial until Jan 31, 2022 (more to come*)
Schachtman
Park
Dennis
Stark
Staley
Pawitan
Hennig
Ionides and Ritov
Haig
Lakens
*Let me know if you wish to write one