J Korean Med Assoc.  2011 Apr;54(4):419-429. 10.5124/jkma.2011.54.4.419.

Tools for assessing quality and risk of bias by levels of evidence

• Lim SM
• Shin ES
• Lee SH
• Seo KH
• Jung YM
• Jang JE

• KMID: 2190428
• DOI: http://doi.org/10.5124/jkma.2011.54.4.419

Abstract

Tools for assessing methodological quality or risk of bias in randomized controlled trials (RCTs) and non-randomized studies (NRS) were reviewed. The van Tulder scale and Cochrane's assessment of risk of bias are the two most useful methodological quality evaluation tools for RCTs. Cochrane's tool includes sequence generation, allocation of sequence concealment, blinding, incomplete outcome data, selective outcome reporting, and other potential sources of bias. The Cochrane Collaboration Group recommends the Downs and Black instrument and the Newcastle-Ottawa Scale for evaluating the quality of NRS. In conclusion, this study offers useful information to physicians about tools for assessing the quality of evidence in clinical guidelines. Further research is needed to provide an essential core for evidence-based decision making regarding levels and/or grades of recommendations.

Keyword

Jadad's scale; van Tulder scale; Cochrane's tool; Downs and Black instrument; Newcastle-Ottawa Scale

MeSH Terms

Bias (Epidemiology)
Cooperative Behavior
Decision Making

Reference

1. Higgins JP, Green S, editors. Cochrane handbook for systematic reviews of interventions. Ver. 5.1.0. The Cochrane Collaboration. 2011. cited 2011 Jan 7. Available from: http://www.cochrane-handbook.org.

2. Chalmers TC, Smith H Jr, Blackburn B, Silverman B, Schroeder B, Reitman D, Ambroz A. A method for assessing the quality of a randomized control trial. Control Clin Trials. 1981. 2:31–49.

3. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials. 1995. 16:62–73.

4. Barratt A. Evidence based medicine and shared decision making: the challenge of getting both evidence and preferences into health care. Patient Educ Couns. 2008. 73:407–412.

5. Kranke P. Evidence-based practice: how to perform and use systematic reviews for clinical decision-making. Eur J Anaesthesiol. 2010. 27:763–772.

6. Parrilla-Castellar ER, Almeyda R, Nogales E, Velez M, Ramos M, Rivera JE, Da Vila B, Torres V, Capriles J, Adamsons K. Evidence-based medicine as a tool for clinical decision-making in Puerto Rico. P R Health Sci J. 2008. 27:135–140.

7. Tiburi MF. Evidence-based medicine as viewed by key decision-makers of health plans in southern Brazil. Health Serv Manage Res. 2008. 21:185–191.

8. Stolba N, Nguyen TM, Tjoa AM. Towards sustainable decision-support system facilitating EBM. Conf Proc IEEE Eng Med Biol Soc. 2007. 2007:4355–4358.

9. Wyer PC, Silva SA. Where is the wisdom? I. a conceptual history of evidence-based medicine. J Eval Clin Pract. 2009. 15:891–898.

10. Collins J. Evidence-based medicine. J Am Coll Radiol. 2007. 4:551–554.

11. Anastasiu M, Strâmbu V, Popa F. Evidence-based medicine, conceptual challenge or the future of daily practice? Chirurgia (Bucur). 2007. 102:527–530.

12. Carter MJ. Evidence-based medicine: an overview of key concepts. Ostomy Wound Manage. 2010. 56:68–85.

13. Maluf-Filho F. The importance of evidence-based medicine concepts for the clinical practitioner. Arq Gastroenterol. 2009. 46:87–89.

14. Borgerson K. Valuing evidence: bias and the evidence hierarchy of evidence-based medicine. Perspect Biol Med. 2009. 52:218–233.

15. Isaac CA, Franceschi A. EBM: evidence to practice and practice to evidence. J Eval Clin Pract. 2008. 14:656–659.

16. Kruer MC, Steiner RD. The role of evidence-based medicine and clinical trials in rare genetic disorders. Clin Genet. 2008. 74:197–207.

17. Rogers W, Ballantyne A. Justice in health research: what is the role of evidence-based medicine? Perspect Biol Med. 2009. 52:188–202.

18. Kitto S, Petrovic A, Gruen RL, Smith JA. Evidence-based medicine training and implementation in surgery: the role of surgical cultures. J Eval Clin Pract. 2010. 08. 04. [Epub]. DOI: 10.1111/j.1365-2753.2010.01526.x.

19. Carretier J, Bataillard A, Fervers B. The patient's role in evidence-based medicine. J Chir (Paris). 2009. 146:537–544.

20. Soll RF. Evaluating the medical evidence for quality improvement. Clin Perinatol. 2010. 37:11–28.

21. Deeks JJ, Dinnes J, D'Amico R, Sowden AJ, Sakarovitch C, Song F, Petticrew M, Altman DG. International Stroke Trial Collaborative Group. European Carotid Surgery Trial Collaborative Group. Evaluating non-randomised intervention studies. Health Technol Assess. 2003. 7:iii–x. 1–173.

22. MacLehose RR, Reeves BC, Harvey IM, Sheldon TA, Russell IT, Black AM. A systematic review of comparisons of effect sizes derived from randomised and non-randomised studies. Health Technol Assess. 2000. 4:1–154.

23. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials. 1996. 17:1–12.

24. van Tulder M, Furlan A, Bombardier C, Bouter L. Editorial Board of the Cochrane Collaboration Back Review Group. Updated method guidelines for systematic reviews in the cochrane collaboration back review group. Spine (Phila Pa 1976). 2003. 28:1290–1299.

25. Newell SA, Sanson-Fisher RW, Savolainen NJ. Systematic review of psychological therapies for cancer patients: overview and recommendations for future research. J Natl Cancer Inst. 2002. 94:558–584.

26. Scottish Intercollegiate Guideline Network. SIGN 50: a guideline developer's handbook 2008 [Internet]. 2008. cited 2011 Jan 28. Edinburgh: Scottish Intercollegiate Guidelines Network;Available from: http://www.sign.ac.uk/pdf/sign50.pdf.

27. National Institute for Health and Clinical Excellence. The guidelines manual 2009 [Internet]. 2009. cited 2011 Jan 27. London: National Institute for Health and Clinical Excellence;Available from: http://www.nice.org.uk/media/5F2/44/The_guidelines_manual_2009_-_All_chapters.pdf.

28. Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998. 52:377–384.

29. Wells GA, Shea B, O'Connell D, Peterson J, Welch V, Losos M, Tugwell P. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses [Internet]. c1996-2010. cited 2011 Mar 28. Ottawa: Ottwa Hospital Research Institute;Available from: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp.

30. Reisch JS, Tyson JE, Mize SG. Aid to the evaluation of therapeutic studies. Pediatrics. 1989. 84:815–827.

31. Thomas H. Quality assessment tool for quantitative studies: effective public health practice project. 2003. Hamilton: McMaster University.

32. Zaza S, Wright-De Agüero LK, Briss PA, Truman BI, Hopkins DP, Hennessy MH, Sosin DM, Anderson L, Carande-Kulis VG, Teutsch SM, Pappaioanou M. Data collection instrument and procedure for systematic reviews in the Guide to Community Preventive Services. Task Force on Community Preventive Services. Am J Prev Med. 2000. 18:1 Suppl. 44–74.