J Educ Eval Health Prof.  2007;4:5.

Effects of Rating Training on Inter-Rater Consistency for Developing a Dental Hygiene Clinical Rater Qualification System

• Park JR
• Oh JS
• Chae M
• Jung JY
• Bae SS

Abstract

We tried to develop itemized evaluation criteria and a clinical rater qualification system through rating training of inter-rater consistency for experienced clinical dental hygienists and dental hygiene clinical educators. A total of 15 clinical dental hygienists with 1-year careers participated as clinical examination candidates, while 5 dental hygienists with 3-year educations and clinical careers or longer participated as clinical raters. They all took the clinical examination as examinees. The results were compared, and the consistency of competence was measured. The comparison of clinical competence between candidates and clinical raters showed that the candidate group's mean clinical competence ranged from 2.96 to 3.55 on a 5-point system in a total of 3 instruments (Probe, Explorer, Curet), while the clinical rater group's mean clinical competence ranged from 4.05 to 4.29. There was a higher inter-rater consistency after education of raters in the following 4 items: Probe, Explorer, Curet, and insertion on distal surface. The mean score distribution of clinical raters ranged from 75% to 100%, which was more uniform in the competence to detect an artificial calculus than that of candidates (25% to 100%). According to the above results, there was a necessity in the operating clinical rater qualification system for comprehensive dental hygiene clinicians. Furthermore, in order to execute the clinical rater qualification system, it will be necessary to keep conducting a series of studies on educational content, time, frequency, and educator level.

Keyword

Clinical competence; Evaluation; Criteria; Item; Difficulty; Inter-rater consistency

MeSH Terms

Calculi
Clinical Competence
Dental Hygienists
Education
Humans
Mental Competency
Oral Hygiene*

Reference

1. National Health Personnel Licensing Examination Board. A study on issues and possible improvements of current national dental hygienist examination. Seoul (KR): National Health Personnel Licensing Examination Board;1998. Korean.

2. National Health Personnel Licensing Examination Board. Analysis on works of dental hygienist. Seoul (KR): National Health Personnel Licensing Examination Board;2000. Korean.

3. Korean Dental Hygienists Association. An investigation and research report on the extension of dental hygienists’ works. Seoul (KR): Korean Dental Hygienists Association;2004. Korean.

4. Jung JY. Dental hygienists’ awareness on switchover and occupational activities. Suwon Women’s College Theses Collection. 1998; 24:127–148. Korean.

5. Kim ES. A study on possible associations between some dental hygienists’ educational experiences and task performance capacity. Dent Res. 1998; 44:53–4. Korean.

6. Hwang MY. Possible approaches to consolidated curriculum for dental hygienists. J Korean Acad Dental Hyg Educ. 2003; 3:59–72. Korean.

7. Kwon HG. Actual profile and possible developments of dental hygienists’ working conditions. Symposium on fostering and employing alternative human resources for dental care and operation. Seoul (KR): National Health Personnel Licensing Examination Board;2002. p. 43–5. Korean.

8. Jang GW, Lee JB. Practical skill education and rating for qualifying dental hygienists and dentists. Seoul (KR): National Health Personnel Licensing Examination Board;2003. Korean.

9. National Health Personnel Licensing Examination Board. Exploration into possible developments of national examination for health and medical personnel: Clinical practical skill test-practical skill education and rating for dental hygienists. Seoul (KR): National Health Personnel Licensing Examination Board;2003. Korean.