Cardiovasc Prev Pharmacother.  2020 Jul;2(3):85-98. 10.36011/cpp.2020.2.e12.

Pragmatic Clinical Trials for Real-World Evidence: Concept and Implementation

• Jeong NY
• Kim SH
• Lim E
• Choi NK

• KMID: 2536954
• DOI: http://doi.org/10.36011/cpp.2020.2.e12

Abstract

The importance of real-world evidence (RWE) has been highlighted in recent years, and the limitations of the classical randomized controlled trials, also known as explanatory clinical trials (ECTs), have been emphasized. Post-marketing observational studies have several problems, such as biases and incomparability between patient groups, and RWE can only be obtained after a certain period. Therefore, pragmatic clinical trials (PCTs) have garnered attention as an alternative to obtaining scientifically robust RWE in a relatively short time. PCTs are clinical trials that have a pragmatic concept, i.e., the opposite of ECTs and are intended to help decision makers by evaluating the effectiveness of interventions in routine clinical practice. The characteristics of PCTs are the inclusion of various patients in clinical practice, recruitment of patients in heterogeneous settings, and comparison with actual clinical treatments rather than a placebo. Thus, the results of PCTs are likely to be generalized and can have a direct impact on clinical and policy decision-making. This study aimed to describe the characteristics and definitions of PCTs compared with those of ECTs and to highlight the important considerations in the planning process of PCTs. To perform PCTs for the purpose of obtaining RWE, the contents covered in this study will be helpful.

Keyword

Causality; Epidemiologic studies; Patient outcome assessment; Pragmatic clinical trials

Figure

• 

  Figure 1. Relationships among the explanatory clinical trials, observational study, and pragmatic clinical trials, including hypothetical PRECIS-2 diagrams. PRECIS-2 diagrams in the figure is referred to Bevan et al.20)PRECIS = PRagmatic Explanatory Continuum Indicator Summary.

Reference

1. Silverman SL. From randomized controlled trials to observational studies. Am J Med. 2009; 122:114–20.

2. Treweek S, Zwarenstein M. Making trials matter: pragmatic and explanatory trials and the problem of applicability. Trials. 2009; 10:37.

3. Gill J, Prasad V. Improving observational studies in the era of big data. Lancet. 2018; 392:716–7.

4. Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. N Engl J Med. 2000; 342:1878–86.

5. Grimes DA, Schulz KF. Bias and causal associations in observational research. Lancet. 2002; 359:248–52.

6. Zuidgeest MG, Goetz I, Groenwold RH, Irving E, van Thiel GJ, Grobbee DE; GetReal Work Package 3. Series: pragmatic trials and real world evidence: Paper 1. Introduction. J Clin Epidemiol. 2017; 88:7–13.

7. Patsopoulos NA. A pragmatic view on pragmatic trials. Dialogues Clin Neurosci. 2011; 13:217–24.

8. Dal-Ré R, Janiaud P, Ioannidis JP. Real-world evidence: how pragmatic are randomized controlled trials labeled as pragmatic? BMC Med. 2018; 16:49.

9. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis. 1967; 20:637–48.

10. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003; 290:1624–32.

11. Califf RM, Sugarman J. Exploring the ethical and regulatory issues in pragmatic clinical trials. Clin Trials. 2015; 12:436–41.

12. Gamerman V, Cai T; Elsäßer A. Pragmatic randomized clinical trials: best practices and statistical guidance. Health Serv Outcomes Res Methodol. 2019; 19:23–35.

13. Zwarenstein M, Treweek S. What kind of randomized trials do we need? J Clin Epidemiol. 2009; 62:461–3.

14. Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Oxman AD, Moher D, CONSORT group; Pragmatic Trials in Healthcare (Practihc) group. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ. 2008; 337:a2390.

15. Coats TJ. Future research in emergency medicine: explanation or pragmatism? Large or small? Simple or complex? Emerg Med J. 2011; 28:1004–7.

16. MacPherson H. Pragmatic clinical trials. Complement Ther Med. 2004; 12:136–40.

17. Borgerson K. Are explanatory trials ethical? Shifting the burden of justification in clinical trial design. Theor Med Bioeth. 2013; 34:293–308.

18. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015; 350:h2147.

19. Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, Chalkidou K. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009; 62:464–75.

20. Bevan A, Biedenbach P, Berger A. Pragmatic randomized trials: considerations for design and implementation [Internet]. Bethesda, MD: Evidera;2019. [cited 2020 May]. Available from https://www.evidera.com/pragmatic-randomized-trials-considerations-for-design-and-implementation/.

21. Kalkman S, van Thiel G, van der Graaf R, Zuidgeest M, Goetz I, Grobbee D, van Delden J. The social value of pragmatic trials. Bioethics. 2017; 31:136–43.

22. Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise JM, Santa J, Conway PH, Daudelin D, Morrato EH, Leslie LK. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J Gen Intern Med. 2012; 27:985–91.

23. Johnson KE, Tachibana C, Coronado GD, Dember LM, Glasgow RE, Huang SS, Martin PJ, Richards J, Rosenthal G, Septimus E, Simon GE, Solberg L, Suls J, Thompson E, Larson EB. A guide to research partnerships for pragmatic clinical trials. BMJ. 2014; 349:g6826.

24. Fraser J, Moloney R, Tambor E, Tuzzio L. Building partnerships to ensure a successful trial: deciding who to engage. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials [Internet]. Bethesda, MD: NIH Health Care Systems Research Collaboratory;2020. [cited 2020 May 30]. Available from https://rethinkingclinicaltrials.org/chapters/engaging-stakeholders/deciding-who-to-engagev2/.

25. Chambers D, Coronado G, Green B, Jarvik J, Septimus E, Tuzzio L, Zatzick D. Designing with implementation and dissemination in mind: key considerations. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials [Internet]. Bethesda, MD: NIH Health Care Systems Research Collaboratory;2018. [cited 2020 May 30]. Available from https://rethinkingclinicaltrials.org/chapters/design/designing-implementation-dissemination-mind-top/designing-with-implementation-and-dissemination-in-mind-2/.

26. Simon G, Ramsberg J, Coronado G, Septimus E, DeBar L, Vazquez M, Dember LM, Vollmer WM, Green B, Zatzick D, Huang SS, Hernandez A, Jarvik J, Platt R, Mor V. Data sharing and embedded research: introduction. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials [Internet]. Bethesda, MD: NIH Health Care Systems Research Collaboratory;2019. [cited 2020 May 30]. Available from https://rethinkingclinicaltrials.org/chapters/dissemination/data-share-top/data-sharing-andembedded-research-introduction/.

27. Welsing PM, Oude Rengerink K, Collier S, Eckert L, van Smeden M, Ciaglia A, Nachbaur G, Trelle S, Taylor AJ, Egger M, Goetz I; Work Package 3 of the GetReal Consortium. Series: pragmatic trials and real world evidence: Paper 6. Outcome measures in the real world. J Clin Epidemiol. 2017; 90:99–107.

28. Curtis L, Hernandez A, Weinfurt K. Choosing and specifying endpoints and outcomes: meaningful endpoints. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials [Internet]. Bethesda, MD: NIH Health Care Systems Research Collaboratory;2020. [cited 2020 May 30]. Available from https://rethinkingclinicaltrials.org/chapters/design/choosing-specifying-end-points-outcomes/meaningful-endpoints/.

29. Murad MH, Shah ND, Van Houten HK, Ziegenfuss JY, Deming JR, Beebe TJ, Smith SA, Guyatt GH, Montori VM. Individuals with diabetes preferred that future trials use patient-important outcomes and provide pragmatic inferences. J Clin Epidemiol. 2011; 64:743–8.

30. Basch E. New frontiers in patient-reported outcomes: adverse event reporting, comparative effectiveness, and quality assessment. Annu Rev Med. 2014; 65:307–17.

31. Walton MK, Powers JH 3rd, Hobart J, Patrick D, Marquis P, Vamvakas S, Isaac M, Molsen E, Cano S, Burke LB; International Society for Pharmacoeconomics and Outcomes Research Task Force for Clinical Outcomes Assessment. Clinical outcome assessments: conceptual foundation-report of the ISPOR Clinical Outcomes Assessment - Emerging Good Practices for Outcomes Research Task Force. Value Health. 2015; 18:741–52.

32. Worsley SD, Oude Rengerink K, Irving E, Lejeune S, Mol K, Collier S, Groenwold RH, Enters-Weijnen C, Egger M, Rhodes T; GetReal Work Package 3. Series: pragmatic trials and real world evidence: Paper 2. Setting, sites, and investigator selection. J Clin Epidemiol. 2017; 88:14–20.

33. Bulpitt CJ. Randomised Controlled Clinical Trials. New York, NY: Springer Science & Business Media;2012.

34. Oude Rengerink K, Kalkman S, Collier S, Ciaglia A, Worsley SD, Lightbourne A, Eckert L, Groenwold RH, Grobbee DE, Irving EA; Work Package 3 of the GetReal consortium. Series: pragmatic trials and real world evidence: Paper 3. Patient selection challenges and consequences. J Clin Epidemiol. 2017; 89:173–80.

35. Cook AJ, Delong E, Murray DM, Vollmer WM, Heagerty PJ. Statistical lessons learned for designing cluster randomized pragmatic clinical trials from the NIH Health Care Systems Collaboratory Biostatistics and Design Core. Clin Trials. 2016; 13:504–12.

36. Godwin M, Ruhland L, Casson I, MacDonald S, Delva D, Birtwhistle R, Lam M, Seguin R. Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity. BMC Med Res Methodol. 2003; 3:28.

37. Fransen GA, van Marrewijk CJ, Mujakovic S, Muris JW, Laheij RJ, Numans ME, de Wit NJ, Samsom M, Jansen JB, Knottnerus JA. Pragmatic trials in primary care. Methodological challenges and solutions demonstrated by the DIAMOND-study. BMC Med Res Methodol. 2007; 7:16.

38. Jiang D, Chen K, Mukhopadhyay S, Katta N, Zhang L. Review of Statistical Issues in Pragmatic Clinical Trials in Current Drug Development Environment. In: Contemporary Biostatistics with Biopharmaceutical Applications. Cham, Switzerland: Springer;2019. p. 285–98.

39. Gupta SK. Intention-to-treat concept: a review. Perspect Clin Res. 2011; 2:109–12.

40. Sugarman J, Califf RM. Ethics and regulatory complexities for pragmatic clinical trials. JAMA. 2014; 311:2381–2.

41. McKinney RE Jr, Beskow LM, Ford DE, Lantos JD, McCall J, Patrick-Lake B, Pletcher MJ, Rath B, Schmidt H, Weinfurt K. Use of altered informed consent in pragmatic clinical research. Clin Trials. 2015; 12:494–502.

42. Kim SY, Miller FG. Informed consent for pragmatic trials--the integrated consent model. N Engl J Med. 2014; 370:769–72.

43. Wendler D. “Targeted” consent for pragmatic clinical trials. J Gen Intern Med. 2015; 30:679–82.

44. Faden R, Kass N, Whicher D, Stewart W, Tunis S. Ethics and informed consent for comparative effectiveness research with prospective electronic clinical data. Med Care. 2013; 51:S53–7.

45. Faden RR, Beauchamp TL, Kass NE. Informed consent, comparative effectiveness, and learning health care. N Engl J Med. 2014; 370:766–8.

46. D'Agostino RB, Sullivan LM, Massaro J. Wiley Encyclopedia of Clinical Trials. Hoboken, NJ: John Wiley & Sons, Inc;2007.

47. Ellenberg S, Sugarman J, Zatzick D. Data and safety monitoring: which PCTs should have a DMC? In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials [Internet]. Bethesda, MD: NIH Health Care Systems Research Collaboratory;2020. [cited 2020 May 30]. Available from: https://rethinkingclinicaltrials.org/chapters/design/planning-data-safety-monitoring/which-pcts-should-have-a-dmc/.

48. Food and Drug Administration. FDA Guidance for Clinical Trial Sponsors On the Establishment and Operation of Clinical Trial Data Monitoring Committees. Rockville, MD: Food and Drug Administration;2006.

49. DeBar LL, Jarvik JG, Tuzzio L, Vazquez MA. Assessing feasibility: introduction. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials [Internet]. Bethesda, MD: NIH Health Care Systems Research Collaboratory;2018. [cited 2020 May 30]. Available from https://rethinkingclinicaltrials.org/chapters/conduct/assessing-feasibility/assessing-feasibility-introduction/.

50. DeBar LL, Jarvik JG, Tuzzio L, Vazquez MA. Assessing feasibility: developing the trial documentation. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials [Internet]. Bethesda, MD: NIH Health Care Systems Research Collaboratory;2018. [cited 2020 May 30]. Available from https:// rethinkingclinicaltrials.org/chapters/conduct/assessing-feasibility/developing-the-trial-documentation/.

51. DeBar LL, Jarvik JG, Tuzzio L, Vazquez MA. Assessing feasibility: pilot testing. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials [Internet]. Bethesda, MD: NIH Health Care Systems Research Collaboratory;2020. [cited 2020 May 30]. Available from https://rethinkingclinicaltrials.org/chapters/conduct/assessing-feasibility/pilot-testing/.

52. DeBar LL, Jarvik JG, Tuzzio L, Vazquez MA. Study startup: introduction. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials [Internet]. Bethesda, MD: NIH Health Care Systems Research Collaboratory;2020. [cited 2020 May 30]. Available from https://rethinkingclinicaltrials.org/chapters/conduct/startup/startup-introduction/.

53. DeBar L, Jarvik J, Tuzzio L, Vazquez M. Study Startup: Implementation Readiness Checklist. In: Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials [Internet]. Bethesda, MD: NIH Health Care Systems Research Collaboratory;2020. [cited 2020 May 30]. Available from https://rethinkingclinicaltrials.org/chapters/conduct/startup/startup-implementation/.

54. Cook DJ, Ferguson ND, Hand L, Austin P, Zhou Q, Adhikari NK, Danesh V, Arabi Y, Matte AL, Clarke FE, Mehta S, Smith O, Wise MP, Friedrich JO, Keenan SP, Hanna S, Meade MO; OSCILLation for ARDS Treated Early Investigators; Canadian Critical Care Trials Group. Coenrollment in a randomized trial of highfrequency oscillation: prevalence, patterns, predictors, and outcomes. Crit Care Med. 2015; 43:328–38.

55. Welch MJ, Lally R, Miller JE, Pittman S, Brodsky L, Caplan AL, Uhlenbrauck G, Louzao DM, Fischer JH, Wilfond B. The ethics and regulatory landscape of including vulnerable populations in pragmatic clinical trials. Clin Trials. 2015; 12:503–10.

56. CRASH-2 trial collaborators, Shakur H, Roberts I, Bautista R, Caballero J, Coats T, Dewan Y, El-Sayed H, Gogichaishvili T, Gupta S, Herrera J, Hunt B, Iribhogbe P, Izurieta M, Khamis H, Komolafe E, Marrero MA, Mejía-Mantilla J, Miranda J, Morales C, Olaomi O, Olldashi F, Perel P, Peto R, Ramana PV, Ravi RR, Yutthakasemsunt S. Effects of tranexamic acid on death, vascular occlusive events, and blood transfusion in trauma patients with significant haemorrhage (CRASH-2): a randomised, placebo-controlled trial. Lancet. 2010; 376:23–32.

57. Wada H, Thachil J, Di Nisio M, Mathew P, Kurosawa S, Gando S, Kim HK, Nielsen JD, Dempfle CE, Levi M, Toh CH; The Scientific Standardization Committee on DIC of the International Society on Thrombosis Haemostasis. Guidance for diagnosis and treatment of DIC from harmonization of the recommendations from three guidelines. J Thromb Haemost. 2013; 11:761–7.

58. Hunt BJ, Allard S, Keeling D, Norfolk D, Stanworth SJ, Pendry K; British Committee for Standards in Haematology. A practical guideline for the haematological management of major haemorrhage. Br J Haematol. 2015; 170:788–803.

59. Rossaint R, Bouillon B, Cerny V, Coats TJ, Duranteau J, Fernández-Mondéjar E, Filipescu D, Hunt BJ, Komadina R, Nardi G, Neugebauer EA, Ozier Y, Riddez L, Schultz A, Vincent JL, Spahn DR. The European guideline on management of major bleeding and coagulopathy following trauma: fourth edition. Crit Care. 2016; 20:100.

60. National Clinical Guideline Centre. Major Trauma: Assessment and Initial Management. London: National Institute for Health and Care Excellence;2016.

61. Choudhry NK, Avorn J, Glynn RJ, Antman EM, Schneeweiss S, Toscano M, Reisman L, Fernandes J, Spettell C, Lee JL, Levin R, Brennan T, Shrank WH; Post-Myocardial Infarction Free Rx Event and Economic Evaluation (MI FREEE) Trial. Full coverage for preventive medications after myocardial infarction. N Engl J Med. 2011; 365:2088–97.

62. Fröbert O, Lagerqvist B, Olivecrona GK, Omerovic E, Gudnason T, Maeng M, Aasa M, Angerås O, Calais F, Danielewicz M, Erlinge D, Hellsten L, Jensen U, Johansson AC, Kåregren A, Nilsson J, Robertson L, Sandhall L, Sjögren I, Ostlund O, Harnek J, James SK, TASTE Trial. Thrombus aspiration during STsegment elevation myocardial infarction. N Engl J Med. 2013; 369:1587–97.

63. Bakerly ND, Woodcock A, New JP, Gibson JM, Wu W, Leather D, Vestbo J. The Salford Lung Study protocol: a pragmatic, randomised phase III real-world effectiveness trial in chronic obstructive pulmonary disease. Respir Res. 2015; 16:101.

64. Vestbo J, Leather D, Diar Bakerly N, New J, Gibson JM, McCorkindale S, Collier S, Crawford J, Frith L, Harvey C, Svedsater H, Woodcock A; Salford Lung Study Investigators. Effectiveness of fluticasone furoate-vilanterol for COPD in clinical practice. N Engl J Med. 2016; 375:1253–60.

65. Kumar A, Rana V. Pragmatic trials: importance and impact. J Endocrinol Thyroid Res. 2017; 2:555594.

66. Shah AS, Anand A, Strachan FE, Ferry AV, Lee KK, Chapman AR, Sandeman D, Stables CL, Adamson PD, Andrews JP, Anwar MS, Hung J, Moss AJ, O'Brien R, Berry C, Findlay I, Walker S, Cruickshank A, Reid A, Gray A, Collinson PO, Apple FS, McAllister DA, Maguire D, Fox KA, Newby DE, Tuck C, Harkess R, Parker RA, Keerie C, Weir CJ; Mills NLHigh-STEACS Investigators. High-sensitivity troponin in the evaluation of patients with suspected acute coronary syndrome: a stepped-wedge, cluster-randomised controlled trial. Lancet. 2018; 392:919–28.