J Educ Eval Health Prof.  2017;14:28. 10.3352/jeehp.2017.14.28.

Developing a framework for evaluating the impact of Healthcare Improvement Science education across Europe: a qualitative study

• Lillo-Crespo M
• Sierras-Davó MC
• MacRae R
• Rooney K

• KMID: 2406691
• DOI: http://doi.org/10.3352/jeehp.2017.14.28

Abstract

PURPOSE
Frontline healthcare professionals are well positioned to improve the systems in which they work. Educational curricula, however, have not always equipped healthcare professionals with the skills or knowledge to implement and evaluate improvements. It is important to have a robust and standardized framework in order to evaluate the impact of such education in terms of improvement, both within and across European countries. The results of such evaluations will enhance the further development and delivery of healthcare improvement science (HIS) education. We aimed to describe the development and piloting of a framework for prospectively evaluating the impact of HIS education and learning.
METHODS
The evaluation framework was designed collaboratively and piloted in 7 European countries following a qualitative methodology. The present study used mixed methods to gather data from students and educators. The framework took the Kirkpatrick model of evaluation as a theoretical reference.
RESULTS
The framework was found to be feasible and acceptable for use across differing European higher education contexts according to the pilot study and the participants' consensus. It can be used effectively to evaluate and develop HIS education across European higher education institutions.
CONCLUSION
We offer a new evaluation framework to capture the impact of HIS education. The implementation of this tool has the potential to facilitate the continuous development of HIS education.

Keyword

Europe; Health personnel; Curriculum; Delivery of health care; Students

MeSH Terms

Consensus
Curriculum
Delivery of Health Care*
Education*
Europe*
Health Personnel
Humans
Learning
Pilot Projects
Prospective Studies

Figure

• 

  Fig. 1. Example of the minimum data set that was developed.

• 

  Fig. 2. Development of the HIS evaluation framework. HIS, healthcare improvement science; MDS, minimum data set.

• 

  Fig. 3. Examples of the online healthcare improvement science learning evaluation framework, level 1 (A) and level 5 (B).

Reference

References

1. Zander B, Aiken LH, Busse R, Rafferty AM, Sermeus W, Bruynell L. The state of nursing in the European Union. EuroHealth. 2016; 22(1):4–8.

2. MacRae R, Rooney KD, Taylor A, Ritters K, Sansoni J, Lillo Crespo M, Skela-Savic B, O’Donnell B. Making it easy to do the right thing in healthcare: advancing improvement science education through accredited pan European higher education modules. Nurse Educ Today. 2016; 42:41–46. https://doi.org/10.1016/j.nedt.2016.03.023.

3. Skela-Savic B, Macrae R, Lillo-Crespo M, Rooney KD. The development of a consensus definition for healthcare improvement science (HIS) in seven European countries: a consensus methods approach. Zdr Varst. 2017; 56:82–90. https://doi.org/10.1515/sjph-2017-0011.

4. University of the West Scotland. Improvement Science Training for European Healthcare Workers (ISTEW) Project [Internet]. Paisley: University of the West Scotland;[cited 2017 Nov 02]. Available from: https://www.uws.ac.uk/research/.

5. Swensen SJ, Shanafelt T. An organizational framework to reduce professional burnout and bring back joy in practice. Jt Comm J Qual Patient Saf. 2017; 43:308–313. https://doi.org/10.1016/j.jcjq.2017.01.007.

6. Reeves S, Clark E, Lawton S, Ream M, Ross F. Examining the nature of interprofessional interventions designed to promote patient safety: a narrative review. Int J Qual Health Care. 2017; 1–7. https://doi.org/10.1093/intqhc/mzx008.

7. Rouse DN. Employing Kirkpatrick’s evaluation framework to determine the effectiveness of health information management courses and programs. Perspect Health Inf Manag. 2011; 8:1c.

8. Armstrong L, Lauder W, Shepherd A. An evaluation of methods used to teach quality improvement to undergraduate healthcare students to inform curriculum development within preregistration nurse education: a protocol for systematic review and narrative synthesis. Syst Rev. 2015; 4:8. https://doi.org/10.1186/2046-4053-4-8.

9. Kirkpatrick DL, Kirkpatrick JD. Evaluating training programs: the four levels. San Francisco (CA): Berrett-Koehler;2012.

10. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Medical Research Council Guidance. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008; 337:a1655. https://doi.org/10.1136/bmj.a1655.

11. Larkins SL, Preston R, Matte MC, Lindemann IC, Samson R, Tandinco FD, Buso D, Ross SJ, Palsdottir B, Neusy AJ. Training for Health Equity Network Thenet. Measuring social accountability in health professional education: development and international pilot testing of an evaluation framework. Med Teach. 2013; 35:32–45. https:/doi.org/10.3109/0142159X.2012.731106.

12. Ovretveit J. Understanding the conditions for improvement: research to discover which context influences affect improvement success. BMJ Qual Saf. 2011; 20 Suppl 1:i18–23. https://doi.org/10.1136/bmjqs.2010.045955.

13. Tackett S, Grant J, Mmari K. Designing an evaluation framework for WFME basic standards for medical education. Med Teach. 2016; 38:291–296. https://doi.org/10.3109/0142159X.2015.1031737.

14. White M, Wells J, Butterworth T. Leadership, a key element of quality improvement in healthcare: results from a literature review of “lean healthcare” and the productive ward: releasing time to care initiative. Int J Leadersh Pub Serv. 2013; 9(3):90–108. https://doi.org/10.1108/IJLPS-08-2013-0021.