Commentary
on Spreadsheets
for Analysis of Controlled Trials Amanda J Cox Sportscience 10, 64, 2006
(sportsci.org/2006/ajc.htm) |
|
The
limitations of analytical approaches based exclusively on statistical significance
have been well documented (Batterham and Hopkins, 2005). Use of standardised mean changes
and qualitative inferential outcomes offers an attractive alternative to
researchers, including those in the field of exercise immunology. With invasive
sample collections and expensive analytical methods, most studies in this
field suffer from small sample sizes. The large within- and between-subject
variability frequently observed in immune and endocrine measures further
limits interpretation of outcomes solely reliant on statistical significance.
The initial spreadsheets developed by Hopkins (2003; 2005) offered researchers in this field
with a useful alternative to traditional analysis based on statistical
significance. One
limitation of the original spreadsheets was the inability to include a covariate
in the analysis. The newly modified version with adjustment for a subject
characteristic provides researchers with a means of reporting treatment/intervention
effects without the confounding influence of additional variables. This is
particularly useful for addressing the issue of regression to the mean,
arising when the error of measurement is substantial in comparison with the
between subject variability, as is often the case for immunological measures.
By including the pre-test values as the covariate this confounding effect can
be adjusted for and outcome effects less likely to be over- or under-estimated.
I found the following specific features of the spreadsheet particularly
useful: • The large number of inserted
comments provide the user with information for easy use and customisation of
the spreadsheet. • Single entry of the level of
confidence and thresholds for substantial change eliminate entry of these
into multiple result panels • Inclusion of graphs of raw and log
transformed data. • Summarised mean effects as
percentage and fold changes and now as standardised mean effects for each of
the groups. • The plots of covariate values
verses change scores are useful for examining any influence of the covariate
on the treatment effect. Inclusion of the mean covariate value on the plots
is particularly useful in indicating how responses of the groups differ at
this value. That is, whether there are substantial effects of the treatment
once the confounding influence of the covariate has been adjusted for. • Automatic generation of the
qualitative outcomes into the bottom of each of the results panels simplifies
the task of making an inference. Complicated
study designs involving multiple groups will require the generation of
multiple spreadsheets for each level of comparison. While this approach may
become labour intensive, it remains a practical option for researchers familiar
with the use of the spreadsheets. More
complex analysis may require more sophisticated approaches such as mixed
modelling. Data analysis through use
of the spreadsheet will require careful description in the methods section of
submitted manuscripts to satisfy those reviewers committed to more
traditional statistical significance. Batterham AM, Hopkins WG (2005). Making meaningful inferences about magnitudes. Sportscience 9, 6-13 Hopkins WG (2003). A spreadsheet for analysis of straightforward controlled trials. Sportscience 7, sportsci.org/jour/03/wghtrials.htm (4447 words) Hopkins WG (2005). A spreadsheet for fully controlled crossovers. Sportscience 9, 24 Published Dec 2006. Back to article/homepage |