Comment on Replacing Statistical Significance… Ross D Neville Sportscience 26,
sportsci.org/2022/rdn.htm, 2022 I very much welcome
and appreciate this article on sampling uncertainty, both for clarifying and
updating the current status of the MBI/MBD vs NHST debate, and also for
situating both MBI/MBD and NHST within the broader suite of current
state-of-the-art approaches to interpreting effects. The first line says it
all really: "a sample provides only an approximate estimate of the
magnitude of an effect". Sport scientists therefore need to understand,
be upfront about, and know how to effectively report and interpret sampling
uncertainty in their manuscripts. As I read through the
article, my first thoughts were related to marketing: who is this article
aimed at? I think it will benefit anyone interested in alternatives to
statistical significance, be they students or experienced researchers. In
this respect, the article is more valuable than an adversarial piece written
to put "establishment" detractors of MBI/MBD in their place. The article
is about sampling uncertainty, not primarily about NHST or MBI/MBD. Rightly
so. Other thoughts were
related to the philosophy of science. Will states that "authors
interpret significance as real, meaningful, worthwhile, important, useful,
beneficial, harmful, or otherwise substantial". This list encompasses
different philosophies of science (e.g., realism, pragmatism,
instrumentalism, consequentialism) that should not all be lumped together.
Furthermore, I don't think that Will’s comment about people regarding a
significant effect as "real" accurately captures the attitude of
most sport scientists towards their data. It’s probably more like "I
need significance so I can publish", evidence of which is the upward
bias of magnitudes of published statistically significant effects in small
samples. So, their motivation may be more about outputs than understanding. Confirmation
bias is another issue here, and p <0.05 is an easy way for authors to "confirm"
their preconceived beliefs: "I expected an effect, I found it, I published
it, and the finding is attributed to me." Finally, I’m not
qualified enough to emphatically state, as Will and many others have, that
NHST should be retired. But having read this article, I think that sport
scientists should be convinced (as I am) that NHST provides neither necessary
nor sufficient evidence when assessing effect magnitudes. Sport scientists
should therefore analyze and present their data (as I do) using alternative
approaches. Back
to Replacing
Statistical Significance and Non-Significance… Published August
2022. |
