New education method or tool
Open Access

Patient Safety and Quality Improvement: A Curriculum for Physical Medicine and Rehabilitation

Micheal Murphy[1], Stephen Bai[1], Jeffrey Barnett[1], Sarah Eickmeyer[1]

Institution: 1. University of Kansas Medical Center
Corresponding Author: Dr Micheal Murphy ([email protected])
Categories: Teaching and Learning, Postgraduate (including Speciality Training), Curriculum Evaluation/Quality Assurance/Accreditation
Published Date: 14/04/2021

Abstract

Introduction:

Quality improvement (QI) is an imperative aspect of healthcare and resident education, and as such, an emphasis has been placed on the development of QI curricula. We set out to describe and analyze the effectiveness of the PSQI Curriculum within the PM&R Residency Program at the University of Kansas Medical Center during the academic years of 2017-2020.


Methods:

The curriculum includes nine mandatory institute for healthcare improvement modules, the university’s GME core conference series, the departments patient safety conference series, a PSQI lecture series, and annual resident-led QI projects. Resident PSQI knowledge was assessed before and after completion of the annual curriculum through an internally generated questionnaire with optional resident feedback.

 

Results:

A total of 16 PM&R residents participated in this curriculum for a minimum of one or more years during the academic years of 2017-2020. We found a significant difference in 9 of 12 domains during the 2017-2018 academic year, 10 of 12 domains during the 2018-2019 academic year, and 10 of 12 domains during the 2019-2020 academic year. Aggregating for all academic years revealed a statistically significant difference across all 12 domains. 


Discussion:

In order to graduate from an ACGME-accredited residency program, residents must exhibit proficiency in PBLI and SBP, which includes the domain of quality improvement. The continuous QI curriculum within the PM&R Residency Program at KUMC has proven to be effective in improving residents’ familiarity and comfort with quality assessment and improvement and may serve as a model for other PM&R programs around the nation. 

 

Keywords: quality improvement; patient safety; medical education; physical medicine and rehabilitation

Introduction

Quality improvement (QI) is an imperative aspect of healthcare and resident education, and as such, an emphasis has been placed on the development of QI curricula in residency training programs. The Clinical Learning Environment Review (CLER) program was founded in 2012 by the Accreditation Council for Graduate Medical Education (ACGME) as a means to examine and subsequently ensure the quality of training in residency programs. The CLER National Report in 2016 found that many academic institutions were routinely failing to engage trainees in patient safety and quality improvement (PSQI) opportunities (Nasca, 2016). As a result, the ACGME mandated that QI instruction and participation be included in residency and fellowship education (ACGME, 2020). The CLER went a step further to include PSQI in its assessment domains for resident and fellow learning environments (Weiss, Wagner and Nasca, 2012). Given that the clinical patterns practiced during residency and fellowship often predict how physicians will care for their patients post-training (Asch et al., 2009) it is inherently essential that training programs implement specialty specific QI curricula that will transcend their trainees’ education (Weiss, Bagian and Nasca, 2013). 

 

Currently, no standard exists for quality improvement curricula regardless of level of training or specialty. Studies have shown a wide variation in the content and degree of exposure to QI principles and participation in QI projects among residency programs (Parker et al., 2015; Craig et al., 2014; Mann, Craig and Moses, 2014; Wong et al., 2010; Philibert et al., 2014). Furthermore, the CLER program reported that trainees and program leadership were often distant participants (Schumacher and Frohna, 2016). Some have called for the implementation of QI curricula that involve more active participation of trainees in QI efforts and more institutional support for such endeavors (Liao, Co and Kachalia, 2015) which is essential to optimize resident and fellow education during training.

 

In order to graduate from an ACGME-accredited residency program, residents must exhibit proficiency in practice-based learning and improvement (PBLI) and systems-based practice (SBP) (ACGME, 2020). Quality improvement and patient safety have come to be included among these domains. We developed a PSQI model designed to meet ACGME requirements and CLER expectations. Our model’s foundation is a continuous QI curriculum that recurs annually and is designed specifically for Physical Medicine and Rehabilitation (PM&R) residents. It focuses on core PBLI and SBP competencies while engaging residents in quality improvement initiatives that foster better patient care. While there are QI curricula published (Keefer et al., 2016; Stewart et al., 2016; Wener, 2017; Vinas et al., 2018; Kiger and Bertagnoli, 2018),  this resource represents the first QI curriculum specific to PM&R residents.

Methods

The patient safety and quality improvement curriculum within the PM&R Residency Program at the University of Kansas Medical Center (KUMC) was implemented in 2015. The initial (2015) version of the curriculum has undergone marked changes over the ensuing years, reflecting evaluations from learners and faculty instructors in addition to evolving literature in the area of medical education. A total of 16 PM&R residents have participated in this QI curriculum for a minimum of one or more years. Medical students rotating through were invited but not required to participate in QI efforts. 

 

Curriculum structure:

Institute for Healthcare Improvement (IHI)

The IHI is the international leader in the realm of PSQI and has developed an online Open School that is available free of charge to any medical student, resident, fellow, or faculty member who is involved in teaching health professionals. Their modules incorporate content with video clips, interactive questions, and quizzes in an effort to educate healthcare professionals on the foundational concepts of PSQI. There are nine required modules for all residents at KUMC, five within the quality improvement domain and four within the patient safety domain (Table 1).

 

Table 1: Required PSQI IHI Modules

Lecture Number

Title

QI 101 

Introduction to Healthcare Improvement 

QI 102

How to Improve with the Model for Improvement

QI 103

Testing and Measuring Changes with PDSA Cycles

QI 104

Interpreting Data: Run Charts, Control Charts, and other Measurement Tools

QI 105

Leading Quality Improvement

PS 101

Introduction to Patient Safety

PS 102

From Error to Harm

PS 104

Teamwork and Communication in a Culture of Safety

PS 105

Responding to Adverse Events

 

There are numerous supplementary modules available to our residents in PSQI in addition to opportunities in the areas of medical education, leadership, and patient care. While our residents are encouraged to complete all of the IHI modules by graduation; currently, we are only mandating completion of the nine specified modules in table 1 for all PGY-2 residents by the end of their second year of residency. The IHI modules are introduced to our residents during the first PSQI lecture of the year, which is described further below. Quarterly reminders are subsequently sent to all second-year residents to help ensure completion in a timely fashion. 

 

University of Kansas Graduate Medical Education (GME) Core Competency Conference Series

The GME’s primary responsibility at any given institution is to provide an organized educational program that fosters resident and fellow education while safeguarding patient care (Armstrong et al., 2012). KUMC’s conference series provides accessible and valuable instruction aligned with the ACGME competencies and supports the University of Kansas Health System’s quality and safety initiatives. All residents are required to attend at least 75% of KUMC’s GME core conference series, held on the second Tuesday of every month, throughout their residency. Residents on rotations off the main campus are afforded the opportunity to attend remotely or watch the recorded lectures at a later date. 

 

Departmental Patient Safety Conference

Every year, each resident is required to present a patient case, with faculty mentorship, at our departmental patient safety conference held once per month. There are 30 minutes allotted for each presentation with the first 10 minutes designated for the case presentation and respective literature review and the remaining 20 minutes designated for discussion. During the discussion phase of the presentation, residents utilize the Healthcare Matrix developed at Vanderbilt University Medical Center to demonstrate the opportunities for improvement in their patient case. This Healthcare Matrix is a valuable teaching instrument in resident education as it allows for patient care assessment using ACGME and Institute of Medicine (IOM) core competencies (Quinn et al., 2009). Of note, the literature review will reflect the presenting resident’s interpretation of the matrix prior to group discussion and should synthesize the current literature trying to address this specific problem. 

 

PSQI Lecture Series and Resident-led Projects

At the start of each academic year, residents form three teams, each comprised of at least one PGY2, PGY3, and PGY4 resident, respectively. Each team nominates a faculty mentor to help them identify and complete a PSQI project throughout the academic year. We elected to utilize this structure for two reasons. First, as residents advance through their training, they experience a tremendous amount of growth in clinical knowledge and skills which should be reflected with gradually increased responsibility in both medical education and patient care. This structure affords senior residents an opportunity to exercise leadership and share their knowledge with junior residents. Secondly, this structure allows for more cohesive implementation of the Plan-Do-Study-Act (PDSA) cycle from one year to the next should non-graduating residents opt to continue with their previously identified project.

 

The resident PSQI lecture series is divided into four quarterly lectures with the addition of two supplemental lectures comprising a total of six distinct lectures. Each lecture is 30 minutes in duration except for our Mock Root Cause Analysis lecture which is allocated 60 minutes of lecture time. Lectures are given by faculty members throughout the course of the academic year. The focus is on high yield PSQI topics and development of resident led QI projects. The topics are outlined in Table 2. 

 

Table 2: PSQI Lecture Series

Date

Time (minutes)

Topic

QI Worksheet (questions requiring completion prior to lecture)

First quarter

30

Introduction and Construction of a QI Project*

Questions 1-4

Second quarter

30

Intro to PDSA Cycle*

Questions 5-6

Third quarter

30

Modeling Processes*

Questions 7-8

Third quarter

60

(Mock) Root Cause Analysis

-

Fourth quarter

30

Making QI Work for You*

Question 9

Fourth quarter 

30

Prepare QI Project Presentations

-

*Represents capstone quarterly lectures 

 

 

QI and PDSA worksheets (Supplementary File 1 and 2, respectively) are provided at the beginning of the academic year following the introduction and construction of a QI project lecture. Each lecture correlates to specific questions within the worksheets, and residents are expected to have completed the correlating questions prior to lecture within their respective project teams as detailed above. The first 10-15 minutes of each 30-minute lecture is spent on the presentation of a new topic. The remaining 15-20 minutes is spent reviewing QI worksheet updates with the faculty member to ensure appropriate implementation of newly learned topics and timeliness of QI project completion. Following the completion of the PSQI lecture series and projects, each team then presents their project to the department at the end of the academic year during our final departmental patient safety conference. 

 

Resident feedback

An anonymous pre- and post-PSQI self-assessment form (Supplementary File 3) was completed by residents at the beginning and end of each academic year starting in 2017-2018 when the first cohort of residents to participate in the curriculum started their senior year. Residents were asked about their previous exposure to QI, their confidence in implementing QI methodologies in future practice, and their confidence in their ability to use QI principles to improve patient care. Additionally, they were asked for qualitative feedback in regard to questions and concerns pertaining to patient safety and quality improvement. The survey is considered exempt by the KUMC Institutional Review Board, and the results are continuously utilized to modify and improve our QI curriculum. 

 

Faculty training and commitment

The primary faculty mentor for the patient safety and quality improvement curriculum has had specific training in QI methodology in their residency program and has completed institutional QI training. Department leadership provides sufficient administrative time for the development, implementation and continuous improvement of this curriculum. On average the faculty mentor spent 2 additional hours of work each month developing content in addition to coordinating and following up resident efforts over the first year of curriculum initiation. This time occurred during previously appointed administrative time. Over the next four years of the curriculum, the faculty mentor spent on average 2 hours of work each quarter updating lectures and coordinating resident efforts.

Results

Resident feedback was gathered via Likert scale relating to how comfortable each resident was with various aspects of PSQI. The following connotations were used on pre and post assessments: 1 - not at all comfortable, 2 - slightly comfortable, 3 - moderately comfortable, and 4 - extremely comfortable. Question statements which residents were asked to rate can be found in Supplementary File 3.

 

Our data was collected over three academic years, spanning 2017-2020, with a total of 16 different residents (PGY-2 through PGY-4) having completed the pre and post assessment. We had a 100% completion rate during this three-year collection period (n=29). Importantly, because this curriculum occurs annually, PGY-2 residents during the 2017-2018 academic year went on to complete this questionnaire a second and third time during 2018-2019 and 2019-2020, respectively. The same holds true for PGY-3 residents who completed the questionnaire a total of two times during their residency. The responses were aggregated for each academic year. Given that our survey responses were completely anonymous, we were unable to stratify results based on the number of years a resident had completed the curriculum. 

 

The results of the pre-curriculum responses were compared to the results of the post-curriculum responses for each academic year. As the respondents were the same for each pre-and-post test, a paired t-test was used to compare resident comfort with QI pre-and-post curriculum. We expected the curriculum to have a positive impact on resident familiarity with QI, thus a one-tailed test was utilized (Table 3).

 

Table 3: PSQI pre-post difference by academic year for all assessment questions

Question statement (On a scale of 1-4, how comfortable are you with each of the following aspects of quality improvement?)

One-sided P-values from t(v) distribution: P[t(v) > u]

 

2017-18 (n=9)

2018-19 (n=10)

2019-20 (n=10)

All years combined

Writing a clear problem statement

0.0249

0.005

0.0839

< 0.001

Applying the best professional knowledge

0.0403

0.007

0.1114

0.00144

Using measurement to improve your skills

0.0249

0.012

0.0223

< 0.001

Studying the process

0.0249

0.005

0.0691

< 0.001

Making changes in a system

0.0557

0.002

0.0004

< 0.001

Identifying whether a change leads to an improvement 

0.141

0.026

0.0112

< 0.001

Using small cycles of change

0.012

0.002

0.0075

< 0.001

Identifying best practices

0.052

0.052

0.0016

< 0.001

Implementing a structured plan to test a change 

0.011

0.012

0.0119

< 0.001

Using the PDSA model as a systematic framework

0.009

< 0.001

0.0048

< 0.001

Identifying how data is linked to specific processes

0.0249

0.500

0.0004

< 0.001

Building your next improvement upon success/failure 

0.007

< 0.001

0.0048

< 0.001

 

 

We found a significant difference in 9 out of the 12 domains in our QI curriculum during the 2017-2018 academic year, 10 out of 12 domains during the 2018-2019 academic year, and 10 out of 12 domains during the 2019-2020 academic year. Specifically, making changes in a system, identifying whether a change leads to an improvement, and identifying best practices were not statistically significant for the 2017-2018 academic year. Identifying best practices and identifying how data is linked to specific processes were not significant for the 2018-2019 academic year. Writing a clear problem statement and applying the best professional knowledge were not significant for the 2019-2020 academic year. However, aggregating for all academic years, as shown in Figure 1, revealed a statistically significant difference across all 12 domains. 

 

Figure 1: PSQI pre-post difference aggregated for all years

Discussion

We set out to describe and analyze the effectiveness of our PSQI Curriculum within the PM&R Residency Program at the University of Kansas Medical Center over the academic years of 2017-2020. Analysis revealed significant improvement between pre and post assessment in 9 out of 12, 10 out of 12, and 10 out of 12 domains for the academic years of 2017-2018, 2018-2019, and 2019-2020, respectively. Making changes in a system, identifying whether a change leads to an improvement, identifying best practices, identifying how data is linked to specific processes, writing a clear problem statement and applying the best professional knowledge were all areas that residents failed to demonstrate significant improvement at some point during our data collection period. However, there was year-to-year variance pertaining to which domains failed to demonstrate significant improvement. In fact, only identifying best practices spanned multiple academic years, 2017-2018 and 2018-2019. Moreover, aggregated analysis for all academic years demonstrated significant improvement across all domains. 

 

Given that our assessment tool was a questionnaire, it is possible that questionnaire bias impacted our results. Bias may have arisen from the way individual questions were designed and thus interpreted by our residents. This may explain why there was year-to-year variance in regard to which domains residents failed to show significant improvement. On the contrary, it is possible that there are deficiencies in our residents’ knowledge, and thus in our curriculum, which would necessitate further modifications to address these discrepancies. However, given the overall improvement seen across all domains for the aggregated data, it would appear this is less likely. 

 

There were two major limitations in this study: 1) the use of a non-validated questionnaire 2) the inability to track individual residents as they progressed through the curriculum from year-to-year. Future considerations would include adapting a validated evaluation tool to be relevant for PM&R, such as the Quality Improvement Knowledge Application Tool (QIKAT), which has previously been described (Kiger and Bertagnoli, 2018). This tool would allow for more accurate assessment of resident QI knowledge and skills. Adjustments could also be made to the wording and design of the pre- and post-PSQI self-assessment form to minimize bias. Allowing for qualitative feedback at the end of the survey in regard to the strengths and weaknesses of the QI curriculum could prompt additional necessary revisions in the curriculum. Lastly, tracking residents as they progress through the curriculum throughout their residency training would provide valuable information on the curriculum's effectiveness from year-to-year. Intake could remain anonymous, but a tracking system nonetheless would need to be implemented. 

Conclusion

With oversight from the ACGME and CLER program, residency programs must afford trainees the opportunity to engage in meaningful patient safety and quality improvement curricula. Subsequently, trainees must demonstrate competency in these domains for graduation. In order to meet this expectation, our program implemented a continuous PSQI curriculum, which has proven to be effective in improving our residents’ understanding, familiarity and comfort with quality assessment and improvement. Additional revisions to not only our curriculum but also our evaluation tools will need to occur in order to further improve resident knowledge acquisition, skills, and attributes as it pertains to PSQI. We hope that our curriculum can serve as a model for other PM&R programs throughout the nation.

Glossary of Terms

Acronym

Definition

ACGME

Accreditation Council for Graduate Medical Education

CLER

Clinical Learning Environment Review

GME

Graduate Medical Education

IHI

Institute for Healthcare Improvement

IOM

Institute of Medicine

KUMC

University of Kansas Medical Center

QI

Quality improvement

QIKAT

Quality Improvement Knowledge Application Tool

PM&R

Physical Medicine and Rehabilitation           

PBLI

Practice-based learning and improvement

PDSA

Plan-Do-Study-Act

PGY

Post graduate year

PSQI

Patient safety and quality improvement

SBP

Systems-based practice

Take Home Messages

Following completion of the curriculum, learners should:

  • Understand the basic principles and common techniques used for patient safety and quality improvement in health care
  • Identify areas and potential interventions for quality improvement projects
  • Design and implement meaningful tests of change
  • Propose, implement, and evaluate plan-do-study-act cycles
  • Develop goals for action teams to facilitate successful changes in system processes and quality of patient care

Notes On Contributors

Micheal Murphy, M.D. is currently a resident physician in the Department of Physical Medicine and Rehabilitation at University of Kansas Medical Center. He received his M.D. degree from the University of Kansas School of Medicine.

Stephen Bai, M.D. is currently a resident physician in the Department of Physical Medicine and Rehabilitation at University of Kansas Medical Center. He received his M.D. degree from the University of Kansas School of Medicine.

Jeffrey Barnett, D.O. is currently a resident physician in the Department of Physical Medicine and Rehabilitation at University of Kansas Medical Center. He received his D.O. degree from Kentucky College Of Osteopathic Medicine.

Sarah Eickmeyer, M.D. is currently an Associate Professor, Program Director, and Medical Director of Inpatient Services at University of Kansas Medical Center. She received her M.D. degree from Northwestern University and completed her residency training in Physical Medicine and Rehabilitation at McGaw Medical Center-Northwestern University-Chicago.

Acknowledgements

The authors are the source and copyright owner of Figure 1.

Bibliography/References

Accreditation Council for Graduate Medical Education (ACGME). (2020) ACGME common program requirements. Available at: https://www.acgme.org/What-We-Do/Accreditation/Common-Program-Requirements (Accessed: 30 January 2021).

 

Armstrong, G., Headrick, L., Madigosky, W. and Ogrinc, G. (2012) ‘Designing education to improve care’, Joint Commission Journal on Quality and Patient Safety, 38(1):5-14. https://doi.org/10.1016/S1553-7250(12)38002-1

 

Asch, D. A., Nicholson, S., Srinivas, S., Herrin, J., et al. (2009) ‘Evaluating obstetrical residency programs using patient outcomes’, JAMA, 302(12):1277-1283. https://doi.org/10.1001/jama.2009.1356

 

Craig, M. S., Garfunkel, L. C., Baldwin, C. D., Co, J. P., et al. (2014) ‘Pediatric resident education in quality improvement (QI): a national survey’, Academic Pediatrics, 14(1):54-61. https://doi.org/10.1016/j.acap.2013.10.004

 

Keefer, P., Orringer, K., Vredeveld, J., Warrier, K., et al. (2016) ‘Developing a quality improvement and patient safety toolbox: the curriculum’, MedEdPORTAL,12:10385. https://doi.org/10.15766/mep_2374-8265.10385

 

Kiger, M. E. and Bertagnoli, T. (2018) ‘A project-based, resident-led quality improvement curriculum within a pediatric continuity clinic’, MedEdPORTAL, 14:10738. https://doi.org/10.15766/mep_2374-8265.10738

 

Liao, J. M., Co, J. P. and Kachalia, A. (2015) ‘Providing educational content and context for training the next generation of physicians in quality improvement’, Academic Medicine, 90(9):1241-1245. https://doi.org/10.1097/ACM.0000000000000799

 

Mann, K. J., Craig, M. S. and Moses, J. M. (2014) ‘Quality improvement educational practices in pediatric residency programs: survey of pediatric program directors’, Academic Pediatrics, 14(1):23-28. https://doi.org/10.1016/j.acap.2012.11.003

 

Nasca, T. J. (2016) 'Introduction to the CLER National Report of Findings 2016', Journal of Graduate Medical Education, 8: 7-9. https://doi.org/10.4300/1949-8349.8.2s1.7

 

Parker, S. L., McGirt, M. J., Asher, A. L. and Selden, N. R. (2015) ‘Quality improvement in neurological surgery graduate medical education’, Neurosurgery Clinics of North America, 26(2):231-238. https://doi.org/10.1016/j.nec.2014.11.012

 

Philibert, I., Gonzalez del Rey, J. A., Lannon, C., Lieh-Lai, M., et al. (2014) ‘Quality improvement skills for pediatric residents: from lecture to implementation and sustainability’, Academic Pediatrics, 14(1):40-46. https://doi.org/10.1016/j.acap.2013.03.015

 

Quinn, D. C., Reynolds, P. Q., Easdown, J. and Lorinc, A. (2009) ‘Using the healthcare matrix with interns and medical students as a tool to effect change’, Southern Medical Journal. 102(8):816-822. https://doi.org/10.1097/SMJ.0b013e3181ad5fe7

 

Schumacher, D. J. and Frohna, J. G. (2016) ‘Patient safety and quality improvement: a “CLER” time to move beyond peripheral participation’, Medical Education Online, 21(1):31993. https://doi.org/10.3402/meo.v21.31993

 

Stewart, D., Lye, C., Lopez, M., Mothner, B., et al. (2016) ‘Engaging learners through modules in quality improvement and patient safety’, MedEdPORTAL, 12:10482. https://doi.org/10.15766/mep_2374-8265.10482

 

Vinas, E. K., White, A. B., Rogers, R. G., Ridgeway, J. J., et al. (2018) ‘Developing best clinical practices through outcomes improvement: an ongoing quality improvement curriculum for faculty and residents’, MedEdPORTAL, 14:10676. https://doi.org/10.15766/mep_2374-8265.10676

 

Weiss, K. B., Bagian, J. P. and Nasca, T. J. (2013) ‘The clinical learning environment: the foundation of graduate medical education’, JAMA, 309(16):1687-1688. https://doi.org/10.1001/jama.2013.1931

 

Weiss, K. B., Wagner, R. and Nasca, T. J. (2012) 'Development, Testing, and Implementation of the ACGME Clinical Learning Environment Review (CLER) Program', Journal of Graduate Medical Education, 4: 396-98. https://doi.org/10.4300/JGME-04-03-31

 

Werner, J. A. (2017) ‘An integrated, multimodal resident curriculum in patient safety and quality improvement’, MedEdPORTAL, 13:10641. https://doi.org/10.15766/mep_2374-8265.10641

 

Wong, B. M., Etchells, E. E., Kuper, A., Levinson, W., et al. (2010) ‘Teaching quality improvement and patient safety to trainees: a systematic review’, Academic Medicine, 85(9):1425-1439. https://doi.org/10.1097/ACM.0b013e3181e2d0c6

Appendices

None.

Declarations

There are no conflicts of interest.
This has been published under Creative Commons "CC BY-SA 4.0" (https://creativecommons.org/licenses/by-sa/4.0/)

Ethics Statement

Formal IRB review was requested at the University of Kansas Medical Center through submission of the Quality Improvement Determination Form (http://www.kumc.edu/human-research-protection-program/institutional-review-board/forms-templates-and-resources.html) and IRB review was deemed to not be required. Authorized 27/10/20 by Karen Blackwell, Director, HRPP.

External Funding

This article has not had any External Funding

Reviews

Please Login or Register an Account before submitting a Review