Inadequate Post-hoc Statistical Power Analysis in Journal of the Korean Society of Emergency Medicine

Ryou, Hyun Wook; Lee, Mi Jin; Kim, Jong Kun; Choe, Michael Sung Pil

Abstract

PURPOSE
Negative studies provide valuable information. However, conducting studies with inadequate power is unethical and an inefficient use of resources. Moreover, inaccurate interpretations from underpowered studies result in false conclusions that alter clinical interventions and deter further research. The purpose of this study was to determine the prevalence of negative studies with inadequate power in the Journal of the Korean Society of Emergency Medicine (JKEM).
METHODS
We assessed all papers in JKEM from 2009 to 2012. We sought published evidence that a post-hoc power analysis had been performed in association with the main hypothesis of the paper. All clinical research studies containing the phrase "no difference" were identified. Data necessary for power calculation were extracted from applicable studies.
RESULTS
There were a total of 351 papers in which a statistical comparison was undertaken. Out of 351 original articles, 170(48.4%) were negative studies that contained enough information for analysis. Out of 126 negative studies in JKEM, only 21((16.7%) had performed a power analysis demonstrating adequate sample size. In addition, only 6.3% of dichotomous variable articles and 10.3% of continuous variable articles had adequate power. Levels of adeadequate power in negative studies did not improve over time (p=0.148).
CONCLUSION
Many negative studies in JKEM are inconclusive because they lack the adequate power to detect even large differences between groups. Therefore, it is imperative to consider power when interpreting the literature. When designing future research, power calculations should be performed to ensure sufficient patient recruitment to attain clinically meaningful results.