Anesth Pain Med.  2021 Apr;16(2):138-150. 10.17085/apm.21038.

Trial sequential analysis: novel approach for meta-analysis

• Kang H

• KMID: 2515539
• DOI: http://doi.org/10.17085/apm.21038

Abstract

Systematic reviews and meta-analyses rank the highest in the evidence hierarchy. However, they still have the risk of spurious results because they include too few studies and participants. The use of trial sequential analysis (TSA) has increased recently, providing more information on the precision and uncertainty of meta-analysis results. This makes it a powerful tool for clinicians to assess the conclusiveness of meta-analysis. TSA provides monitoring boundaries or futility boundaries, helping clinicians prevent unnecessary trials. The use and interpretation of TSA should be based on an understanding of the principles and assumptions behind TSA, which may provide more accurate, precise, and unbiased information to clinicians, patients, and policymakers. In this article, the history, background, principles, and assumptions behind TSA are described, which would lead to its better understanding, implementation, and interpretation.

Keyword

Interim analysis; Meta-analysis; Statistics; Trial sequential analysis.

Figure

• 

  Fig. 1. Trial sequential analysis graph. The graph presents monitoring boundaries, futility boundaries, conventional boundaries and required information size. The graph is divided by monitoring boundary and futility boundary into four zones: area of benefit, area of harm, inner wedge, and not statistically significant zone.

• 

  Fig. 2. Probabilities according to the number of analyses. Dark line represents probability of overall Type I error and gray line represents probability of accepting null hypothesis.

• 

  Fig. 3. Trial sequential analysis graph and monitoring boundary. (A) The last point of Z-curve stays within the monitoring boundaries. (B) The last point of Z-curve stays within the monitoring boundaries after new study is added. (C) The last point of Z-curve stays outside of the monitoring boundary.

• 

  Fig. 4. Trial sequential analysis graph and futility boundary. (A) The last point of Z-curve stays outside futility borders. (B) The last point of Z-curve stays gets within the futility borders after adding the study. (C) The last point of Z-curve stays outside of futility borders.

Reference

1. Wald A. Contributions to the theory of statistical estimation and testing hypotheses. Ann Math Stat. 1939; 10:299–326.

2. Armitage P, McPherson CK, Rowe BC. Repeated significance tests on accumulating data. J R Stat Soc Ser A. 1969; 132:235–44.

3. Ahn E, Kang H. Introduction to systematic review and meta-analysis. Korean J Anesthesiol. 2018; 71:103–12.

4. Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions version 5.1.0. Cochrane [serial on the Internet]. 2011 Mar [2021 Mar 15]. Available from www.handbook.cochrane.org.

5. Kang H. Statistical considerations in meta-analysis. Hanyang Med Rev. 2015; 35:23–32.

6. Ye XF, Yu DH, He J. The rise in meta-analyses from China. Epidemiology. 2013; 24:335–6.

7. Lee JM, Cho YJ, Ahn EJ, Choi GJ, Kang H. Pharmacological strategies to prevent postoperative delirium: a systematic review and network meta-analysis. Anesth Pain Med (Seoul). 2021; 16:28–48.

8. Ioannidis JP. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016; 94:485–514.

9. Turner RM, Bird SM, Higgins JP. The impact of study size on meta-analyses: examination of underpowered studies in Cochrane reviews. PLoS One. 2013; 8:e59202.

10. Pereira TV, Ioannidis JP. Statistically significant meta-analyses of clinical trials have modest credibility and inflated effects. J Clin Epidemiol. 2011; 64:1060–9.

11. Afshari A, Wetterslev J, Smith AF. Can systematic reviews with sparse data be trusted? Anaesthesia. 2017; 72:12–6.

12. Borm GF, Donders AR. Updating meta-analyses leads to larger type I errors than publication bias. J Clin Epidemiol. 2009; 62:825–30. e10.

13. Chan JSK, Harky A. Trial sequential analysis in meta-analyses: a clinically oriented approach with real-world example. J Thorac Cardiovasc Surg;2020. doi: 10.1016/j.jtcvs.2020.06.063. [Epub ahead of print].

14. De Cassai A, Pasin L, Boscolo A, Salvagno M, Navalesi P. Trial sequential analysis: plain and simple. Korean J Anesthesiol;2020. doi: 10.4097/kja.20637. [Epub ahead of print].

15. Wetterslev J, Thorlund K, Brok J, Gluud C. Estimating required information size by quantifying diversity in random-effects model meta-analyses. BMC Med Res Methodol. 2009; 9:86.

16. In J, Kang H, Kim JH, Kim TK, Ahn EJ, Lee DK, et al. Tips for troublesome sample-size calculation. Korean J Anesthesiol. 2020; 73:114–20.

17. Brok J, Thorlund K, Wetterslev J, Gluud C. Apparently conclusive meta-analyses may be inconclusive--trial sequential analysis adjustment of random error risk due to repetitive testing of accumulating data in apparently conclusive neonatal meta-analyses. Int J Epidemiol. 2009; 38:287–98.

18. Imberger G, Vejlby AD, Hansen SB, Møller AM, Wetterslev J. Statistical multiplicity in systematic reviews of anaesthesia interventions: a quantification and comparison between Cochrane and non-Cochrane reviews. PLoS One. 2011; 6:e28422.

19. Peto R, Pike MC, Armitage P, Breslow NE, Cox DR, Howard SV, et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. Br J Cancer. 1976; 34:585–612.

20. O'Brien PC, Fleming TR. A multiple testing procedure for clinical trials. Biometrics. 1979; 35:549–56.

21. Demets DL. Group sequential procedures: calendar versus information time. Stat Med. 1989; 8:1191–8.

22. Kim K, DeMets DL. Confidence intervals following group sequential tests in clinical trials. Biometrics. 1987; 43:857–64.

23. Lan KKG, Hu M, Cappelleri JC. Applying the law of iterated logarithm to cumulative meta-analysis of a continuous endpoint. Stat Sin. 2003; 13:1135–45.

24. Hu M, Cappelleri JC, Lan KK. Applying the law of iterated logarithm to control type I error in cumulative meta-analysis of binary outcomes. Clin Trials. 2007; 4:329–40.

25. Sidik K, Jonkman JN. A comparison of heterogeneity variance estimators in combining results of studies. Stat Med. 2007; 26:1964–81.

26. Brockwell SE, Gordon IR. A comparison of statistical methods for meta-analysis. Stat Med. 2001; 20:825–40.

27. Sweeting MJ, Sutton AJ, Lambert PC. What to add to nothing? Use and avoidance of continuity corrections in meta-analysis of sparse data. Stat Med. 2004; 23:1351–75.

28. Wetterslev J, Jakobsen JC, Gluud C. Trial sequential analysis in systematic reviews with meta-analysis. BMC Med Res Methodol. 2017; 17:39.

29. Shah A, Smith AF. Trial sequential analysis: adding a new dimension to meta-analysis. Anaesthesia. 2020; 75:15–20.