J Korean Med Sci.  2016 May;31(5):682-687. 10.3346/jkms.2016.31.5.682.

Methodological Quality Appraisal of 27 Korean Guidelines Using a Scoring Guide Based on the AGREE II Instrument and a Web-based Evaluation

• Chang SG
• Kim DI
• Shin ES
• Jang JE
• Yeon JY
• Lee YS

• KMID: 2373663
• DOI: http://doi.org/10.3346/jkms.2016.31.5.682

Abstract

This study evaluated the methodological quality of CPGs using the Korean AGREE II scoring guide and a web-based appraisal system and was conducted by qualified appraisers. A total of 27 Korean CPGs were assessed under 6 domains and 23 items on the AGREE II instrument using the Korean scoring guide. The domain scores of the 27 guidelines were as following: the mean domain score was 82.7% (median 84.7%, ranging from 55.6% to 97.2%) for domain 1 (scope and purpose); 53.4% (median 56.9%, ranging from 11.1% to 95.8%) for domain 2 (stakeholder involvement); 63.0% (median 71.4%, ranging from 13.5% to 90.6%) for domain 3 (rigor of development); 88.9% (median 91.7%, ranging from 58.3% to 100.0%) for domain 4 (clarity of presentation); 30.1% (median 27.1%, ranging from 3.1% to 67.7%) for domain 5 (applicability); and 50.2% (median 58.3%, ranging from 0.0% to 93.8%) for domain 6 (editorial independence). Three domains including scope and purpose, rigor of development, and clarity of presentation were rated at more than 60% of the scaled domain score. Three domains including stakeholder involvement, applicability, and editorial independence were rated at less than 60% of the scaled domain score. Finally, of the 27 guidelines, 18 (66.7%) were rated at more than 60% of the scaled domain score for rigor of development and were categorized as high-quality guidelines.

Keyword

AGREE II Instrument; Quality Appraisal; Clinical Practice Guidelines; Rigor of Development; Scoring Guide

MeSH Terms

Humans
Internet
*Practice Guidelines as Topic
*Program Evaluation
Quality Assurance, Health Care
Republic of Korea

Figure

• 

  Fig. 1 A framework for the systematic web-based quality appraisal of CPGs in Korea.

Reference

1. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Hanna SE, Makarski J. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ. 2010; 182:1045–1052.

2. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Hanna SE, Makarski J. Development of the AGREE II, part 2: assessment of validity of items and tools to support application. CMAJ. 2010; 182:E472–8.

3. Brouwers MC, Kho ME, Browman GP, Burgers JS, Cluzeau F, Feder G, Fervers B, Graham ID, Grimshaw J, Hanna SE, et al. AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ. 2010; 182:E839–42.

4. Makarski J, Brouwers MC, Enterprise AG. The AGREE Enterprise: a decade of advancing clinical practice guidelines. Implement Sci. 2014; 9:103.

5. Jo MW, Lee JY, Kim NS, Kim SY, Sheen S, Kim SH, Lee SI. Assessment of the quality of clinical practice guidelines in Korea using the AGREE Instrument. J Korean Med Sci. 2013; 28:357–365.

6. Ahn HS, Kim HJ. Development and implementation of clinical practice guidelines: current status in Korea. J Korean Med Sci. 2012; 27:Suppl. S55–60.

7. Lee YK, Shin ES, Shim JY, Min KJ, Kim JM, Lee SH; Executive Committee for CPGs. Korean Academy of Medical Sciences. Developing a scoring guide for the Appraisal of Guidelines for Research and Evaluation II instrument in Korea: a modified Delphi consensus process. J Korean Med Sci. 2013; 28:190–194.

8. Oh MK, Jo H, Lee YK. Improving the reliability of clinical practice guideline appraisals: effects of the Korean AGREE II scoring guide. J Korean Med Sci. 2014; 29:771–775.

9. Sinclair D, Isba R, Kredo T, Zani B, Smith H, Garner P. World Health Organization guideline development: an evaluation. PLoS One. 2013; 8:e63715.

10. Tudor KI, Kozina PN, Marušić A. Methodological rigour and transparency of clinical practice guidelines developed by neurology professional societies in Croatia. PLoS One. 2013; 8:e69877.

11. Gagliardi AR, Brouwers MC. Do guidelines offer implementation advice to target users? A systematic review of guideline applicability. BMJ Open. 2015; 5:e007047.

12. Jokhan S, Whitworth MK, Jones F, Saunder A, Heazell AE. Evaluation of the quality of guidelines for the management of reduced fetal movements in UK maternity units. BMC Pregnancy Childbirth. 2015; 15:54.

13. Sabharwal S, Patel V, Nijjer SS, Kirresh A, Darzi A, Chambers JC, Malik I, Kooner JS, Athanasiou T. Guidelines in cardiac clinical practice: evaluation of their methodological quality using the AGREE II instrument. J R Soc Med. 2013; 106:315–322.

14. Burgers JS, Cluzeau FA, Hanna SE, Hunt C, Grol R. Characteristics of high-quality guidelines: evaluation of 86 clinical guidelines developed in ten European countries and Canada. Int J Technol Assess Health Care. 2003; 19:148–157.

15. Al-Ansary LA, Tricco AC, Adi Y, Bawazeer G, Perrier L, Al-Ghonaim M, AlYousefi N, Tashkandi M, Straus SE. A systematic review of recent clinical practice guidelines on the diagnosis, assessment and management of hypertension. PLoS One. 2013; 8:e53744.