Report of meeting or workshops
Open Access

Approaches for Curriculum and Assessment in Leader and Leadership Education and Development Programs in American Medical Schools

Erin Barry[1], Neil Grunberg[1], Hannah Kleber[1]

Institution: 1. Uniformed Services University of the Health Sciences
Corresponding Author: Ms Erin Barry ([email protected])
Categories: Assessment, Curriculum Planning, Educational Strategies, Teaching and Learning
Published Date: 25/10/2018

Abstract

Problem: There is a growing call to add leader and leadership education to undergraduate medical education (UME). Yet, there currently are no established standards, competencies, curricula, or requirements for UME leader and leadership education and development (LEAD) programs. The Uniformed Services University of the Health Sciences (USU) F. Edward Hébert School of Medicine LEAD program hosts annual Summit and Working Group meetings to address issues and to share experiences about LEAD programs.

Approach: Based on survey results following the 2017 USU LEAD Summit, working group participants reported that the meeting was valuable, should be repeated, and should address the specific topics of curriculum and assessment. Therefore, the 2018 Summit’s goal was for participants to share experiences, ideas, and ways forward regarding leader and leadership curricula and assessment measures for UME. Themes from working groups were compiled and reported.

Outcomes: Themes within LEAD curriculum include: (1) what to teach: relevant knowledge, skills, and abilities/attitudes (KSA) for specific topics; (2) when to teach: a life-cycle program woven through UME into graduate medical education and beyond; and (3) how to teach: near peers, development of mentors, and near-term, practical applications of skills. Themes within LEAD assessment include: (1) what to assess: alignment with program goals and curriculum within a positive culture of assessment and trust; (2) when to assess: occur at times that are consistent with the learning objectives and curriculum to provide information on incremental “growth” of students and the program; (3) how to assess: use formative and summative, qualitative and quantitative measures that are reliable and valid.

Next steps: Based on feedback from working group participants at the 2018 Summit, the USU LEAD team will host a third Summit in April 2019 focusing on leader and leadership education and development across the healthcare work force life cycle.

Keywords: Leader and Leadership Education and Development; Leaders; Leadership; Medical Leader and Leadership Education and Development; LEAD

Problem

Undergraduate medical education (UME) requirements already are extensive and the continuous introduction of new procedures, instruments, medications, and record systems keep adding more to learn. In addition, there is a growing clarion call to add leadership education to UME (Webb et al., 2014; Frich et al., 2015; Neeley, Clyne and Resnick-Ault, 2017; Grunberg et al., 2018). According to the AAMC, leadership is “the most critical component for success (Association of American Medical Colleges).” Yet, there currently are no established standards, competencies, curricula, or requirements for UME leader and leadership education and development (LEAD) programs (where leader refers to human capital and leadership refers to social capital (Day, 2001)). An increasing number of medical schools are offering LEAD programs (Hargett et al., 2017; Barry et al., 2018; Hopkins et al., 2018) and many are sharing information relevant to their LEAD curriculum and assessment instruments. Among these efforts, the Uniformed Services University of the Health Sciences (USU) F. Edward Hébert School of Medicine LEAD program – that trains healthcare professionals, officers, and leaders for the United States Air Force, Army, Navy, and Public Health Service – hosts annual Summit and Working Group meetings (beginning in 2017) to address issues and to share experiences about LEAD programs. 

Approach

The annual USU LEAD Summit and Working Group meetings are convened for faculty and staff from U.S. Schools of Medicine, service academies, and related professional organizations to address issues relevant to health workforce leader and leadership education and development. At the 2017 Summit, participants shared opinions, experiences, and best practices regarding medical school leadership programs (Grunberg et al., 2018). Based on survey results following the 2017 Summit, participants reported that the meeting was valuable, should be repeated, and should address the specific topics of curriculum and assessment. Therefore, the 2018 Summit’s goal was for working group participants to share experiences, ideas, and ways forward regarding leader and leadership curricula and assessment measures for undergraduate medical education (UME).

The Summit began with an overview of the schedule, purpose, and introductions of participants. The morning session focused on small working group discussions of curriculum followed by a networking opportunity and then a plenary discussion of major points raised in the small group discussions. The afternoon session focused on small working group discussions of assessment followed by further networking and then a plenary discussion of major points raised in the working groups. The meeting concluded with a large group discussion of next steps.

Similar to the 2017 Summit, (Grunberg et al., 2018) working group participants included individuals from public and private U.S. medical schools engaged in leader and leadership education and development. Working group participants also included individuals from: United States Military Academy (USMA), United States Naval Academy (USNA), United States Naval War College, and Marine Corps University. In addition, representatives from the Association of American Medical Colleges (AAMC) and Veterans Administration (VA) participated (see Table 1).

Table 1: List of working group participants and their affiliations.

Working Group Participants

Institution

Gene Andersen, MA

United States Naval War College

Erin Barry, MS*

Uniformed Services University

Eric Bean, DO, MBA

University of South Florida, Lehigh Valley

Andrew Bergemann, PhD

University of Texas at Austin

Mary Brueggemeyer, MD

Uniformed Services University

Ronald Cervero, PhD

Uniformed Services University

Matthew Clark, PhD, PMP

United States Army

Joe Doty, PhD

Duke University

Kathryn Eklund, MS

Uniformed Services University

David Fessell, MD*

University of Michigan

Tanja Fessell, MS

University of Michigan

Alexander Galifianakis, MD

Uniformed Services University

Craig Goolsby, MD, MEd

Uniformed Services University

Neil Gunberg, PhD*

Uniformed Services University

Nathan Hudepohl, MD, MS, MPH*

Brown University

Nancy Hueppchen, MD, MSc

Johns Hopkins University

Hannah Kleber, BA*

Uniformed Services University

Shari Lawson, MD

Johns Hopkins University

Lauren Mackenzie, PhD

Marine Corps University

Ronald Massey, MA, MPA

Veterans Administration

Christopher Mattos, MA

United States Military Academy

John McManigle, MD

Uniformed Services University

Katherine McOwen, MSEd

Association of American Medical Colleges

David Musick, MD

Virginia Tech University

Francis O'Connor, MD, MPH

Uniformed Services University

Kathleen Ogle, MD

George Washington University

Louis Pangaro, MD

Uniformed Services University

Lisa Perla, MA

Uniformed Services University; Graduate School of Nursing

Penny Pierce, PhD

Uniformed Services University; Graduate School of Nursing

Joann Quinn, PhD

University of South Florida

Stacey Rizza, MD

Mayo Clinic

Douglas Robb, DO, MPH

Uniformed Services University

Carol Romano, PhD, RN

Uniformed Services University; Graduate School of Nursing

Eric Schoomaker, MD, PhD

Uniformed Services University

Amy Smith, PhD

University of South Florida, Lehigh Valley

Steven Spector, PhD

University of South Florida

Erin Sullivan, PhD

Harvard University

Dean Taylor, MD

Duke University

Suzanne Templer, DO*

Nova Southeastern University

Joslyn Vaught, BS

Mayo Clinic

David Wallace, PhD

United States Naval Academy

Joseph Weistroffer, MD

Western Michigan University

Dean Winslow, MD

Stanford University

Melanie Wong-Dodge, MBA

United States Military Academy Preparatory School

*indicates working group facilitators

There were six working groups composed of six to seven participants representing different organizations, demographics, and perspectives (e.g., physicians, nurses, psychologists, health profession educators, researchers, administrators). Each group had a facilitator who was given a list of questions to help guide the discussions (see Tables 2 and 3). Both morning and afternoon small group sessions lasted 60 minutes; networking sessions were 30 minutes; large group discussions were 30 minutes.

Table 2. List of Curriculum questions for facilitators to guide working group discussion

  • What are your top examples of curricular content?
  • Where do you believe the leadership curriculum best fits within the overall medical school curriculum?
    • How do you sequence leadership curriculum into medical school curriculum (pre-clinical and clinical)?
    • Do you envision your leadership program for all students or for some students?
  • Do you primarily use leadership activities (that may or may not be related to other leadership activities)?
  • Do you find that your leadership curriculum is supported by the rest of the student experience?
  • What educational methodologies do you use most in your program (e.g., small group, didactic)?
    • What resources are required (or do you envision having) for your session?
  • What leadership sessions have been most well-received by your students?
  • What experiences have you had with sessions that were not received well?
    • Why did you believe it would be well-received? What were the unanticipated problems with this session?
  • What feedback have you received from students regarding curriculum topics and method delivery?
  • What are student views on leadership in the medical school curriculum?
  • Have you attempted any faculty development? What approaches have you used (if any) for fostering faculty development in leadership?

 

Table 3. List of Assessment questions for facilitators to guide working group discussion

  • What do you currently assess?
    • What are you trying to assess?
    • Is assessment a requirement for your current program? If so, in what way?
  • How do you currently assess?
    • Students
    • Faculty
    • Overall Program/curriculum
    • Do you currently have any specific tools that you use to assess (i.e., quiz questions, observations, surveys, qualitative or quantitative data)?
  • Are your assessment goals short-term or long-term?
  • What role does assessment play in your current program?
  • Have you been able to measure the effect of your leadership program (either before or after your students have left)?

 

Facilitators recorded the main points from each discussion and submitted these notes to the Summit organizers. Three coders (who were small group facilitators – a senior faculty member, a junior faculty member, and an education specialist) independently analyzed the submitted notes from all six working groups. Coders were instructed to identify three to five themes into which the individual points regarding curriculum could be categorized and three to five themes into which the individual points regarding assessment could be categorized. The three coders met to compare and discuss their initial classification results. There was complete agreement among the three coders that the points raised in all working groups could be classified into what, when, and how for curriculum and for assessment. There was substantial agreement among the three coders with regard to the classification of the many individual points recorded by facilitators; differences were resolved by discussion and majority consensus among the three coders. The outcomes (see below) reflect these findings.

Outcomes

Curriculum

What to teach. All working group discussions indicated that LEAD curriculum should teach knowledge, skills, and abilities/attitudes (KSA) relevant to effective and successful performance as a leader. KSA topics most frequently mentioned were: professional identity as healthcare leaders, medical ethics, emotional intelligence, and teamwork.

When to teach. LEAD must be approached as a life-cycle program and should start early – even as early as high school and undergraduate college education.  This life-cycle approach should be woven through UME and continued during graduate medical education (GME) and beyond in a coherent manner. The life-cycle approach should guide when particular KSAs are taught.

How to teach. The working groups indicated the importance of: the inclusion of near peers (as small group facilitators, discussion leaders, etc.) in teaching; (2) development of mentors (including faculty and peers as mentors for students); and (3) providing relevant, near-term, practical applications of KSAs.

 

Assessment

What to assess. It is important that program goals, curricula, and assessment are well aligned. A positive and supportive culture of assessment and trust needs to be fostered, cultivated, and emphasized. Establishing positive culture and trust with assessment takes thoughtful effort and time to implement.  

When to assess. Assessment measures should occur at times that are consistent with the learning objectives and curriculum and should assess students, instructors, and programs by students, instructors, and stakeholders. There should be consideration of where each student is in the LEAD education life-cycle for meaningful interpretation of current status and incremental “growth” of individual students and of the program. It is important to have buy-in from students, instructors, and stakeholders throughout the education life-cycle (UME, GME, and beyond).

How to assess. Assessments should use formative and summative, qualitative and quantitative measures that are reliable and valid. These measures should be based on where the student is within the life-cycle of education and development. Additionally, Kirkpatrick levels of training evaluation (Reaction; Learning; Behavior; Results) are useful to assess students and programs (Kirkpatrick and Kirkpatrick, 2016). It is critical to include faculty development so that instructors understand the program’s philosophy, curriculum, goals, and assessment strategies, and how to optimally assess students. 

Next Steps

Curriculum

What to teach. LEAD programs should identify specific KSAs relevant to professional identity as healthcare leaders, medical ethics, emotional intelligence, and teamwork, and design sessions to address them.

When to teach. A life-cycle approach would allow programs to teach KSAs in a laddered manner to allow students to gain better understanding and application of the KSAs. This approach would allow students to incorporate skills into practical applications.

How to teach. Students learn in a variety of ways with the greatest impact derived from hands-on experiences and relevant, practical examples in educational sessions.  Near peer facilitators and sound mentor relationships can enhance students’ leadership development.

 

Assessment

What to assess. Establishing a culture of assessment that emphasizes mutual respect, trust, and meaningful critiques (i.e., indicating successes and what to continue as well as what to improve) are essential to assess students, faculty, and programs.

When to assess. Regular and expected feedback provide measures of incremental growth and remind students of the value of the KSAs. Additionally, it is important to evaluate the incremental growth of the program to improve the experience for students.

How to assess. Reliable, valid assessment measures that are aligned with the goals of the program and which span UME to provide information for students about their incremental growth are essential. Kirkpatrick levels of training evaluation will allow programs to assess students, faculty, as well as the effectiveness of the program.

 

Future Summit

Based on feedback from working group participants at the 2018 Summit, the USU LEAD team will host a third Summit in April 2019. The 2019 LEAD Summit will focus on leader and leadership education and development across the healthcare work force life cycle.

Take Home Messages

  • Working group meetings focused on medical leader and leadership education and development (LEAD) are valuable.
  • LEAD curricula should consider what, when, how; more specifically: relevant KSAs; life-cycle programs woven throughout UME, GME, and beyond; near peers, mentor development, and near-term, practical applications.
  • LEAD assessments should consider what, when, how; more specifically: alignment with program goals and establishment of a positive assessment culture; occur consistently with the learning objectives and curriculum to provide information on incremental “growth” of students and the program; use of formative and summative, qualitative and quantitative measures that are reliable and valid.

Notes On Contributors

Erin S. Barry, MS, is a Research Assistant Professor in the Department of Military & Emergency Medicine (MEM), Research Associate for Leader and Leadership Education and Development (LEAD), and doctoral student in the Health Professions Education Program, Uniformed Services University of the Health Sciences (USU). ORCID: https://orcid.org/0000-0003-0788-7153.

Neil E. Grunberg, PhD, is a Professor in MEM and Director of Research & Development LEAD, USU.

Hannah G. Kleber, BA, is an Education Specialist for LEAD, USU.

Acknowledgements

Conflicts of Interest: We declare that the authors have no conflicts of interest

Disclaimer: The opinions and assertions contained herein are the sole ones of the authors and are not to be construed as reflecting the views of the Uniformed Services University of the Health Sciences or the Department of Defense.

Acknowledgements: We thank all working group participants who attended the 2018 Summit. We especially thank Drs. David Fessell, Nathan Hudepohl, and Suzanne Templer for serving as working group facilitators. We thank LTG (US Army, Ret.) Mark Hertling for sharing his experience with leader and leadership development. We also thank the Uniformed Services University administration and staff for supporting this event.

Bibliography/References

Association of American Medical Colleges  About leadership development. Available at: https://www.aamc.org/members/leadership/about/.

Barry, E. S., Grunberg, N. E., Kleber, H. G., McManigle, J. E., et al. (2018) 'A four-year medical school leader and leadership education and development program', International journal of medical education, 9, p. 99. https://doi.org/10.5116/ijme.5abe.12d2

Day, D. V. (2001) 'Leadership development: A review in context', The Leadership Quarterly, 11(4), pp. 581-613. https://doi.org/10.1016/S1048-9843(00)00061-8

Frich, J. C., Brewster, A. L., Cherlin, E. J. and Bradley, E. H. (2015) 'Leadership development programs for physicians: a systematic review', Journal of general internal medicine, 30(5), pp. 656-674, https://doi.org/10.1007/s11606-014-3141-1

Grunberg, N. E., Barry, E. S., Kleber, H. G., McManigle, J. E., et al. (2018) 'Charting a Course for Leader and Leadership Education and Development in American Medical Schools', MedEdPublish, 7(1). https://doi.org/10.15694/mep.2018.0000037.1

Hargett, C. W., Doty, J. P., Hauck, J. N., Webb, A. M., et al. (2017) 'Developing a model for effective leadership in healthcare: a concept mapping approach', Journal of Healthcare Leadership, 9, p. 69. https://doi.org/10.2147/JHL.S141664

Hopkins, J., Fassiotto, M., Ku, M. C., Mammo, D., et al. (2018) 'Designing a physician leadership development program based on effective models of physician education', Health care management review.

Kirkpatrick, J. D. and Kirkpatrick, W. K. (2016) Kirkpatrick's four levels of training evaluation. Association for Talent Development Press.

Neeley, S., Clyne, B. and Resnick-Ault, D. (2017) 'The state of leadership education in US medical schools: results of a national survey', Medical education online, 22(1). https://doi.org/10.1080/10872981.2017.1301697

Webb, A. M., Tsipis, N. E., McClellan, T. R., McNeil, M. J., et al. (2014) 'A first step toward understanding best practices in leadership training in undergraduate medical education: a systematic review', Academic Medicine, 89(11), pp. 1563-1570. https://doi.org/10.1097/ACM.0000000000000502

Appendices

None.

Declarations

There are no conflicts of interest.
This has been published under Creative Commons "CC BY-SA 4.0" (https://creativecommons.org/licenses/by-sa/4.0/)

Ethics Statement

The present manuscript reports the findings based on a US National Conference therefore no ethics approval is necessary.

External Funding

This paper has not had any External Funding

Reviews

Please Login or Register an Account before submitting a Review

P Ravi Shankar - (08/11/2018) Panel Member Icon
/
I enjoyed reading this report about the 2018 LEAD summit. Leadership education is important for doctors but the primary focus and role of a leader may differ according to the vision and mission of the university/institution. The working groups’ deliberations provide inputs regarding leadership education and assessment. Looking at Table 1 I note that participation was restricted to certain US medical schools. This could be a limitation. The summit was held at the Uniformed Services University and the majority of participants were from the military. This could have an influence on their perspectives about leadership and influenced the deliberations. The working groups provide a general perspective on teaching and assessing leadership skills in medical education. This brief paper will be of interest to medical educators in a variety of settings. The context and the vision and mission of the school and the region or country could and should influence the leadership education curriculum.
Tan Nguyen - (26/10/2018) Panel Member Icon
/
This work offers an excellent example of cross-institutional collaborations on key important issues relevant to medical education, which could be adopted by other health disciplines. I think there will be still a substantial body of work to be done to identify/manage the expectations and develop a program that will address the learning needs of the delegates at the Summit. An evaluation of the LEAD program could be developed to articulate how the program impacts on learning. I would be great to discuss how these themes identified may or may not be similar to those reported in the medical education literature on leadership.
Richard Hays - (26/10/2018) Panel Member Icon
/
This paper addresses a topical issue that should be of interest to many readers. We are in an era when leadership is spoken of often as a need to lead health care through challenging times. A common response is to teach and assess leadership, often from the beginning of medical education. While this is sound, in principle, it opens big questions, such as 'how much more can we fit into the curriculum', 'how does this integrate with learning clinical medicine', and 'how do we assess something that is so hard to measure?'. Some useful resources are available, such as the Leadership Development Framework in the UK, and this document will be another useful resource. However, many approaches and frameworks assume that medical doctors will be the leaders and in fact can be from a young age with little or no preparation. At risk of sounding like an 'over the hill' red wine, my experience has shown that leadership develops with experience, based on reflection, self-awareness, humility, values, respectful communication and learning from mistakes. I am not sure how this journey can be shortened, but to me that is the main purpose of leadership frameworks. Start slow, with awareness and discussing examples of strong, sound and perhaps just different leadership styles. The best time to expand may be the late undergraduate/early GME periods, when individuals often feel least empowered, with leadership experience, reflection, feedback and assessment in-context. I worry that we may be asked to deliver another linear, milestone-assessed curriculum theme, with teaching and assessment methods that are not well suited to this task. This may be too important a topic to risk engendering in early career doctors a sense of: 'leadership? passed that, I deserve promotion'. What I particularly enjoyed about this paper is that the authors show awareness of these concerns, but leave open how to address them beyond principles. More work is needed to work out the details of just what, how and when to teach and assess the elements of leadership, which would see me award the extra star. Perhaps the next stage in the journey?
Possible Conflict of Interest:

I am the editor of MedEdPublish

Felix Silwimba - (25/10/2018)
/
This is a very good report of LEAD in UME. I only wish low and middle income could use a similar approach in addressing the challenges of UME. The take home messages are clear and well-articulated.