Continuous Review of Accreditation Standards and Quality Improvement in a Medical Education Program – Practical Recommendations for Implementing a Program

Background: Accreditation reviews of medical education program standards are becoming more common throughout the world. The consequences of unfavorable decisions on a school’s accreditation status can be significant. As accrediting bodies have become more rigorous in applying the standards required for medical schools to maintain their accreditation, a school’s ability to ensure compliance with standards and act upon identified problems should not be left until the school’s preparation for the next accreditation review cycle. Aim and Method: Based upon a review of the literature related to accreditation standard review and continuous quality improvement (CQI), regulatory documents of agencies impacting accreditation requirements, and our own experience with developing and implementing a process of continuous review of accreditation standards and quality improvement of the medical education program, we propose practical recommendations for schools interested in initiating such a program or modifying an existing program. Results: A program of continuous review of accreditation standards and quality improvement of the medical education program can be implemented at an institution by clearly defining the expectations for such a program by institutional leadership, by assigning a faculty and staff member the responsibility of facilitating the implementation and evaluation of the program, by utilizing a practical approach of regular review of entire standards and specific topics within standards deemed to be of critical importance to the school, by involving a committee of key leaders of the medical education program to oversee the evaluation of the review program and link its results and decisions to action plans and CQI efforts, and by utilizing common work flow processes and software available at most institutions. Conclusion: Establishing a regular review process of the medical education program’s accreditation standards and Clinch C, Violato C MedEdPublish https://doi.org/10.15694/mep.2016.000072 Page | 2 linking the review process to CQI goals of the medical education program (based upon the findings of an institutional self-study) is a manageable endeavor, will provide updated data to the medical education program’s leadership for the purposes of strategic planning and CQI effort prioritization, will lead to the creation of a continually updated self-study data collection instrument that facilitates the process of the next complete self-study, and may lead to fewer negative accreditation decisions by a program’s accrediting organization.


Introduction
The actions of the Liaison Committee on Medical Education (LCME) and the Committee on Accreditation of Canadian Medical Schools (CACMS) have gained greater public and professional visibility in the past several years partly because they have put several prestigious medical schools (educational institutions that lead to the MD degree) on probation.In 2015, the programs leading to the MD degree at McGill University Faculty of Medicine and Wayne State University School of Medicine were both given probationary status by CACMS and LCME, respectively.Similarly, Baylor College of Medicine's program was put on probation by the LCME in 2014.Since 2004, 15 medical schools (approximately 10%) under the jurisdiction of the LCME or CACMs (United States, Canada) have been given probationary status by these organizations.The public response from officials of the schools and the mass media is frequently one of incredulity and confusion, wondering how such schools could be put on probation (Krupa, 2011).There appears to be, however, a growing consensus that the accrediting bodies have become more rigorous in applying the standards required for medical schools to maintain their accreditation.
In the United States and Canada the accreditation of medical schools is both a process and a status (Hunt, Migdal, Eaglen, Barzansky, & Sabalis, 2012).The process aims to improve the program quality by providing review of the program using a set of standards set by peers.The status is that a program either receives accreditation (perhaps with some conditions) or it does not.Hunt et al (2012) identified six possible actions that can be taken by the LCME or CACMS based upon their findings.The first two actions relate to continuing accreditation (perhaps with some minor conditions such as follow-up reports) but actions three through six are considered "severe".These are: "undetermined accreditation period requiring a follow-up visit", continue accreditation but place the program on "warning of probation", continue accreditation but place the program "on probation" (an action that must be publicly announced), and "withdraw or deny accreditation", a penalty that would effectively cripple a program's existence.
Severe action decisions by an accrediting body can be very significant for a medical school because this information becomes public (i.e., newspapers, television, etc.) leading to embarrassment and other consequences such as student and faculty anxiety, potential reduction of future applicants, possible loss of funding opportunities, and the threat of the ultimate possible catastrophe -the loss of accreditation of the medical education program.How can medical schools hope to avoid receiving severe actions by the LCME/CACMS (or other similar accrediting bodies in their region) during accreditation reviews?An important solution is to implement a strategy of continuous monitoring of accreditation standards.
Continuous monitoring of accreditation standards related to a medical education program can pose a challenging dilemma for faculty, staff and medical education program leadership.Determining what to monitor and how frequently to conduct monitoring has a direct impact on the resources needed to perform continuous monitoring as well as the ability of an institution to implement timely continuous quality improvement (CQI) measures related to accreditation standards.In this paper we provide practical recommendations to assist those developing a process for continuous monitoring of accreditation standards and connecting that process with their CQI efforts.

Framing the Issue
Why is continuous monitoring of accreditation standards important?In a follow-up to their previous research, Hunt and colleagues revealed that between October 2004 to June 2012, the LCME issued 40 Severe Action Decisions, and, as described above, these decisions included either granting an established medical education program an unspecified or shortened term of accreditation instead of a full eight-year term, placing a program on warning status, placing a program on probation status, or withdrawing a program's accreditation (Hunt, Migdal, Waechter, Barzansky, & Sabalis, 2016).Among the issues associated with a Severe Action Decision, the top two were insufficient response to the self-study data collection instrument (16/40; 40%) and chronic non-compliance (9/40; 23%), defined as one or more standards determined to be non-compliant in both the current and previous accreditation review cycle.In logistic regression analyses, the Odds Ratio (OR) for an insufficient response to the self-study data collection instrument leading to a Severe Action Decision was 7.30 (95% CI=2.38-22.46)and the OR for chronic non-compliance was 12.18 (95% CI=1.91-77.55).Both of these issues require ongoing attention by a school during the period between accreditation cycles.Reviews of accreditation standards that occur only periodically between cycles of accreditation may not be optimal for ensuring the prompt identification of and remediation of problem areas (Barzansky, Hunt, Moineau, Ahn, Lai, Humphrey, & Peterson, 2015).

Regulatory Requirements
Among the 12 LCME standards and their associated 95 elements, the very first element (1.1) is entitled, "Strategic Planning and Continuous Quality Improvement", which requires that, "A medical school engages in ongoing planning and continuous quality improvement processes that establish short and long-term programmatic goals, result in the achievement of measurable outcomes that are used to improve programmatic quality, and ensure effective monitoring of the medical education program's compliance with accreditation standards."(Liaison Committee on Medical Education, 2016a) While the LCME's authority for accreditation of medical education programs leading to the MD degree is limited to the United States (U.S.) and Canada, twenty-five percent of the U.S. physician workforce is comprised of international medical graduates (IMGs), defined as physicians graduating from medical schools outside of the U.S. or Canada.(Educational Commission for Foreign Medical Graduates, 2015) Certification by the Educational Commission for Foreign Medical Graduates (ECFMG) is the standard for evaluating the qualifications of these physicians before they are able to enter the U.S. graduate medical education system and to take Step 3 of the United States Medical Licensing Examination® (USMLE®).In September, 2010, the ECFMG has required that, effective in 2023, in order to be considered for ECFMG certification, a physician must have graduated from an appropriately accredited medical school.This requirement may be satisfied only if the physician's school is, "accredited through a formal process that uses criteria comparable to those established for U.S. medical schools" by the LCME "or that uses other globally accepted criteria" (Educational Commission for Foreign Medical Graduates, 2015).Whether or not the accrediting organization utilizes the LCME standards, the Basic Medical Education Global Standards for Quality Improvement (2015 Revision) of the World Federation for Medical Education (WFME), or a separate set of accreditation standards, developing an approach to continuous monitoring of accreditation standards and CQI will be of importance to one's institution (Liaison Committee on Medical Education, 2016a; World Federation for Medical Education, 2015).Developing processes to regularly review accreditation standards will help ensure that medical schools do not lose track of their ability to assess and document compliance with accreditation standards in between cycles of accreditation.Additionally, schools can demonstrate their ability to connect the review of accreditation standards with quality improvements they have planned and achieved.

Suggesting an Approach
A school should begin with a charge from the medical education program's chief academic officer (typically the dean of the medical school) to establish a process for continuous monitoring of accreditation standards and demonstration of CQI in the medical education program.To ensure a high degree of visibility and accountability surrounding continuous monitoring of accreditation standards, establish an institutional committee consisting of the leaders with responsibilities related to the components of the accreditation standards.For example, the medical school dean or a designee could chair a committee consisting of those responsible for the medical education curriculum, student affairs, student admissions, faculty affairs, graduate medical education (called "postgraduate medical education" by the WFME), continuing medical education (called "continuing professional development" by the WFME).This committee can also include leadership from the following dedicated functions in the institution if they exist: leaders responsible for academic accreditation, diversity and inclusion, medical education administration, and continuous quality improvement (CQI) project management.
If a complete institutional self-study of accreditation standards has not been recently performed, the next step is to initiate a "no risk" self-study of compliance with all of the applicable accreditation standards to document a baseline database of information upon which to make an assessment of compliance and from which to initiate the program of continuous monitoring and CQI.The term, "no risk", refers to performing the self-study for one's own purposes, not specifically to submit to the accrediting body.Utilize the above-referenced committee, hereafter referred to as the Continuous Accreditation Review and Quality Improvement (CARQI) committee, to lead and accomplish the "no risk" self-study.The self-study findings should be presented to the medical school dean by the CARQI committee and decisions related to how to attain or maintain compliance with accreditation standards should be documented.Metrics surrounding the decisions of the CARQI committee should be included in the quality improvement efforts for the medical education program.For further information about conducting a self-study, see the LCME's "Guide to the Institutional Self-study for Full Accreditation" (Liaison Committee on Medical Education, 2016b).
Following an initial institutional self-study, determine which standards are non-compliant, which standards may be compliant but require a high degree of diligence to maintain compliance (e.g., documentation of direct observation of physical examination skills of medical students by faculty members), and which standards require surveying of individuals in order to obtain data to assess compliance.
With respect to those standards (or items within standards) that require a high degree of diligence to maintain compliance, develop a process of monitoring those specific items on a regular basis (e.g., monthly or quarterly) and present the results of that monitoring in the form of a dashboard to the medical education program's leadership.An example would be the creation of a dashboard that displays the results of the quarterly performance of clerkships with respect to student achievement on assessments (institutional and/or national), the percentage of grades submitted within a required timeframe, the percentage of students receiving mid-rotation feedback, the percentage of students attesting to having been observed by a faculty member while performing components of the history and physical or mental status examination (as measured on an end-of-clerkship questionnaire), the number of reports of student mistreatment (as measured by a student mistreatment monitoring process), and any other item(s) the institution deems critical to monitor and report regularly.
To provide regularity to the review of accreditation standards, one method is to implement a quarterly review process with three complete standards being fully reviewed and updated each year; the first quarter of each year being used for an overall review of the monitoring and CQI process and for planning and alignment upon the next three standards to be fully reviewed.(See Image 1) Given that the period between accreditation review cycles among accrediting bodies may range from 4 -12 years, and that there are 12 LCME standards and 9 WFME standards, this process will allow each standard to be reviewed up to twice prior to the next accreditation review cycle.(Barzansky et al., 2015) Additionally, since the data and narrative responses related to each standard will have been reviewed and updated regularly, the process of completing the self-study with the full set of standards will be much more streamlined.

Additional Process Tips
The process involved in the quarterly review of a complete standard can proceed as outlined in the checklist provided (see Appendix 1).The dean must appoint a leader and an administrative staff member to ensure responsibility for facilitating the items on the checklist.An example of a "Review Table" mentioned in the checklist is provided in Appendix 2. While third-party vendors offer resources to facilitate the process of review of accreditation standards either as a subscription or a purchase (e.g., Activity Insight by DigitalMeasures; Compliance Assist by CampusLabs; Xiatrics Standards by Xiatrics; others), a review of those resources is beyond the scope of this paper.The process described in the checklist (see Appendix 1) utilizes software typically available to most institutions.

Final Steps
The final step in this review and CQI process is to document improvement against any goals set for the medical education program.Establish thresholds that reflect the attainment of success in meeting or exceeding goals during the planning quarter in the review process.This will allow demonstration of those goals met or exceeded and, if goals were not achieved, this will allow for the development of an action plan to demonstrate how goals will be achieved over the upcoming academic year.
While other methods of performing continuous monitoring of accreditation standards may vary the frequency with which standards are reviewed/updated from more frequently (e.g., one standard each month) to less frequently (e.g., all standards at the mid-point of an accreditation review cycle), the method described above provides a practical approach that can assist those interested in initiating a continuous monitoring process and linking the data obtained through such a monitoring process to institutional CQI efforts and reporting.

Conclusions
Establishing a regular review process of the medical education program's accreditation standards and linking the review process to CQI goals of the program (based upon the findings of an institutional self-study) is a manageable endeavor.It will allow the medical education program's leadership to be well-informed about the school's compliance status and be armed with data to better inform strategic planning and CQI efforts.Additionally, updating and archiving the school's response to accreditation standards as part of a regular review cycle will allow for the creation of a continually updated self-study data collection instrument and facilitate the process of the next complete institutional self-study.
Establishing a regular review process of accreditation standards and linking this review to institutional CQI efforts may also result in fewer Severe Action Decisions (or their equivalent among other accrediting bodies) related to an inadequate response to the self-study data collection instrument and chronic non-compliance with a standard(s).A reduction in Severe Action Decisions is, however, an empirical question that requires further study and research.

Take Home Messages
Establishing a process of continuous, regular review of accreditation standards begins with a charge from the 1.
medical education program's leadership to demonstrate the importance of this process.
Utilizing a committee composed of institutional leaders with responsibilities corresponding to the key 2.
components of accreditation standards allows for high visibility of the importance of achieving and maintaining compliance with accreditation standards and connecting quality improvement effort priorities to the standards.
Assigning a faculty and staff member the responsibility to ensure the continuous review and quality 3.
improvement process is implemented and monitored (with the support of the medical school dean) will provide safeguards against the process becoming neglected.

Declaration of Interest
The author has declared that there are no conflicts of interest.

Figure 1 :
Figure 1: Continuous Accreditation Review and Quality Improvement Cycle

Follow a checklist
to ensure all components of an accreditation standard review are accomplished (