New education method or tool
Open Access

Four Years of Experience Using Problem-Based Learning as a Platform to Develop Student Teaching Skills: A Preliminary Report

Carla Lupi[1], Melissa Ward-Peterson[2], Christian Castro[3], Ansley Splinter[4]

Institution: 1. Department of Obstetrics and Gynecology & Office of Medical Education, Herbert Wertheim College of Medicine, Florida International University, 2. Department of Epidemiology, Robert Stempel College of Public Health & Social Work, Florida International University, 3. Madison Teaching and Learning Excellence, Collaborative for Advancing Teaching and Learning, University of Wisconsin—Madison, 4. Department of Pediatrics, The Ohio State University College of Medicine
Corresponding Author: Dr Carla Lupi (clupi@fiu.edu)
Categories: Educational Strategies, Students/Trainees, Teachers/Trainers (including Faculty Development), Undergraduate/Graduate
Published Date: 26/07/2018

Abstract

Introduction: A graduating medical student should be competent in both teaching and communication skills. This concept is supported by the Accreditation Council for Graduate Medical Education and the Association of American Medical Colleges. Our study describes how we utilized problem-based learning as a platform for developing student teaching skills and to examine preliminary outcomes.

 

Methods: Since 2013, third-year medical students at Florida International University Herbert Wertheim College of Medicine have participated in a mandatory problem-based learning course in parallel to their clinical rotations. During the course orientation students have been led through interactive sessions on writing learning objectives and methods for effective micro-teaching sessions. During seven subsequent sessions, trained faculty members have assessed and provided narrative comments on students’ “Ability to Teach Peers” using an anchored developmental scale rubric. Data from four academic years were available for analysis. The Wilcoxon signed-rank test was used to test differences between the initial and final sessions.

 

Results: At the initial session, 39.0% (n=147) received ratings of “mastering.” By the final session, 62.6% (n=236) received ratings of “mastering.”

 

Conclusion: Our preliminary work demonstrates that a brief orientation to micro-teaching followed by repeated mandatory practice and feedback within our problem-based learning curriculum may serve to build students’ teaching skills.

 

Keywords: teaching skills; medical students; problem-based learning; feedback; peer-teaching

Background

The importance of learning to teach as a key outcome in medical education now has widespread acceptance. The Accreditation Council for Graduate Medical Education (ACGME) competency of practice-based learning and improvement requires that trainees “participate in the education of patients, families, students, residents and other health professionals” (ACGME, 2009). Indeed, teaching and communication skills underlie three of the Association of American Medical Colleges (AAMC) Entrustable Professional Activites (EPAs), including EPA 6 (provide an oral presentation of a clinical encounter), EPA 8 (give or receive a patient handover to transition care responsibly), and EPA 11 (obtain informed consent for tests and/or procedures) (AAMC, 2014). The General Medical Council of the United Kingdom states that a graduating student should be able to “work effectively and appropriately as a mentor and teacher for other learners in the multi-professional team” (General Medical Council, 2018).

 

In a national survey conducted in 2008, 43 of 99 US medical schools reported offering a formal student-as-teachers programs with the majority occurring as fourth-year electives (Soriano et al., 2010). Our review of subsequent literature on this topic revealed several additional reports of elective experiences occurring in the fourth year (Shah et al., 2017; Yeung et al., 2018; Yoon, Blatt and Greenberg, 2017).  While these learning opportunities will support the development of skills among students who choose them, the challenge of promoting competency in all students calls for changes to the mandatory curriculum. With time constraints limiting curricular expansion, educators might identify opportunities in existing curricula to provide students both practice to teach and feedback on their teaching. One such opportunity includes student presentations incorporated within active learning pedagogies such as problem-based learning (PBL).

 

In a literature review using three databases (PubMed, ERIC, and EMBASE) with the terms “medical education, undergraduate”, “students, medical”, “teaching”, “feedback”, and then same with “problem-based learning,” we found several references to students as near-peer and peer teachers (Nelson et al., 2013), to electives and modules specifically designed to develop educator skills (Dandavino, Snell and Wiseman, 2007), and to programs for developing students as PBL tutors (Blatt and Greenberg, 2007; Clark et al., 2008). However, we found no literature on the use or impact of feedback to students within problem-based learning curricula to improve their teaching skills.

 

The objectives of this study are to describe how we utilized PBL as a platform to develop student teaching skills and to examine preliminary outcomes.

Activity

Since 2013, a third-year mandatory PBL course running parallel to clinical rotations has served as a platform to utilize a rubric and provide feedback on student teaching skills. During the course student orientation, we conducted interactive sessions on writing learning objectives and structures and guidelines for effective micro-teaching sessions. The latter consisted of a simple structure emphasizing the importance of engaging the audience through initial interaction to activate prior knowledge and ensure connection to the case, use of visual aids (pictures, diagrams, algorithms, etc.) rather than text on PowerPoint to present information, and a closing summary to test understanding and highlight take-away points. Students then attended seven, one-day PBL sessions over the course of a year, each divided into three segments: the morning for case dissection and generation and selection of learning objectives; a break for research on objectives and preparation of teaching; and the afternoon for individual presentations of approximately 10 minutes each. Faculty facilitators assessed and provided narrative comments on students’ “Ability to Teach Peers” using the following anchored developmental scale rubric:

 

  • Emerging (1-2): failed to relate learning issue back to case; rote recital of information from notes and/or PowerPoint; did not engage audience or check for understanding.
  • Acquiring (3-4): learning objectives stated or presented in beginning; organized; presented learning issue at appropriate level; related learning issues back to case.
  • Mastering (5): engaged audience through the use of frequent questions, visual diagrams, handouts or other activities; checked understanding of audience.

This rubric was discussed in detail at the faculty course orientation, with specific behavioral anchors of student performance at each level.

 

Data were available for a total of four classes (Classes of 2015-2018). Descriptive statistics included frequency distributions, means, and standard deviations on student performance at the initial and final sessions; tthe Wilcoxon signed-rank test was used to test differences between the initial and final sessions. The mean of student responses to a Likert-type item included in the post-course survey related to this activity (“The PBL sessions improved my ability to teach effectively”) was also examined from two classes; ratings for this item ranged from 1 (Strongly Disagree) to 5 (Strongly Agree). All analyses were conducted using Stata 14 (College Station, Texas).

 

This research was deemed exempt from review by the Florida International University Health Sciences Institution Review Board.

Results

Data were available for both the initial and final sessions for a total of 377 students. Results for these students are shown in Table 1. At the initial session, 39.0% (n=147) received a rating of “mastering”, with a mean score of 4.17 (SD=0.81; range: 1-5). By the final session, 62.6% (n=236) were rated as “mastering”, with a mean score of 4.58 (SD=0.58; range: 3-5). Results of the Wilcoxon signed-rank test indicated a significant difference between the initial and final sessions.

 

Table 1. Four classes of performance data on student ability to teach peers at initial and final sessions in a third-year problem-based learning course

Class

Performance at Initial Session

Performance at Final Session

Mean (SD)

Percent Rated As “Mastering”

Mean (SD)

Percent Rated As “Mastering”

Class of 2015 (n=77)a

4.14 (0.94)

44.2%

4.61 (0.61)*

67.5%

Class of 2016 (n=99)b

3.99 (0.79)

26.3%

4.54 (0.56)*

56.6%

Class of 2017 (n=92)c

4.41 (0.76)

56.5%

4.66 (0.54)*

69.6%

Class of 2018 (n=109)d

4.14 (0.72)

32.1%

4.52 (0.62)*

58.7%

Overall (n=377)e

4.17 (0.81)

39.0%

4.58 (0.58)*

62.6%

SD = Standard Deviation

*Statistically significant improvement (p-value less than 0.05).

aTotal class size: 78; data available for 98.7%. bTotal class size: 114; data available for 86.8%. c Total class size: 113;

data available for 81.4%. dTotal class size: 120; data available for 90.8%. eTotal number of students: 425; data available for 88.7%.

 

Post-course survey data were available for a total of 186 students from two classes. For the Class of 2016, the mean rating of the statement “The PBL sessions improved my ability to teach effectively” was 4.1 (SD: 0.75; n=83; response rate: 72.8%). The mean rating for the Class of 2018 was 4.0 (SD: 0.96; n=103; response rate: 85.8%). For logistical reasons, data were unavailable for the Classes of 2015 and 2017.

Discussion

Our preliminary findings show that initial instruction in basic principles followed by narrative feedback within a problem-based learning course may contribute to improving students’ teaching skills. The lower standard deviation at the final session indicates that ratings of student performance became more standardized over the academic year.

 

The strengths of our work include the novelty of using PBL as a platform to assess student teaching skills as well as our large sample size. Another strength is the consistent finding of improvement over the academic year in each of the four classes. An important limitation of this study is our lack of comparison groups that did not engage in the practice or in the instruction on microteaching, and/or receive faculty feedback provided on their teaching. Neither can we isolate effects of the PBL course from the clerkships where students may receive feedback on didactic presentations. However, the PBL sessions serve as the most standardized and frequent didactic teaching practice our students receive in their third year. Additionally, student responses in the post-course survey support their perception that the PBL sessions played a key role in improving their teaching skills. Data for this question from all four of the classes would have strengthened this support.  Finally, we have not yet been able to systematically explore possible validity evidence for our rating scale; we plan to do so in future studies.

Conclusion

Our preliminary work demonstrates that a brief orientation to micro-teaching followed by repeated mandatory practice and feedback within our problem-based learning curriculum may build students’ teaching skills, as judged by faculty ratings as well as student perception. Future research should explore the content of narrative feedback from faculty with respect to effectiveness and developmental appropriateness. Our work has also uncovered the need to devise rating systems for use in the PBL setting with validity evidence to judge the quality and impact of student teaching and provide rigorous outcome assessment for future work in this area.

Take Home Messages

  • To our knowledge, this is the first report on the use of mandatory PBL curricula to develop medical student teaching skills.
  • Mandatory PBL instruction may serve as an efficient platform for the development and improvement of medical student teaching skills.
  • Medical students perceive that a problem-based learning course offers the opportunity to develop teaching skills.
  • Orientations to micro-teaching can be easily incorporated into a PBL course orientation; quantitative and qualitative ratings of student micro-teaching performance can be easily incorporated into PBL rubrics.

Notes On Contributors

Dr. Carla S. Lupi is the Associate Dean for Faculty and Professor of Obstetrics and Gynecology at Florida International University Herbert Wertheim College of Medicine, where she also serves on the team working with the AAMC Pilot for the Core Entrustable Professional Activities for Entering Residency.

 

Dr. Melissa Ward-Peterson is a Postdoctoral Associate in Department of Epidemiology, Robert Stempel College of Public Health and Social Work, Florida International University. She is currently supported by the National Institute of Minority Health and Health Disparities grant (1U54MD012393-01) for FIU-RCMI.

 

Dr. Christian Castro is Assistant Director for Teaching and Learning, Collaborative for Advancing Teaching and Learning, University of Wisconsin—Madison.

 

Dr. Ansley Splinter is an Assistant Professor of Pediatrics at Nationwide Children’s Hospital and The Ohio State University.

Acknowledgements

Bibliography/References

Association of American Medical Colleges (AAMC). (2014) Core Entrustable Professional Activities for Entering Residency: Curriculum Developer’s Guide. Available at: https://members.aamc.org/eweb/upload/Core%20EPA%20Curriculum%20Dev%20Guide.pdf (Accessed April 1, 2018).

 

Accreditation Council for Graduate Medical Education (ACGME). (2009) Practice-Based Learning and Improvement: Common Program Requirement. Available at: http://www.acgme.org/Portals/0/PDFs/commonguide/IVA5c_EducationalProgram_ACGMECompetencies_PBLI_Explanation.pdf (Accessed April 1, 2018).

 

Blatt, B. and Greenberg, L. (2007) 'A Multi-Level Assessment of a Program to Teach Medical Students to Teach', Advances in Health Sciences Education, 12(1), pp. 7-18. https://doi.org/10.1007/s10459-005-3053-2

 

Clark, L.C., Eddy, W.S., Stewart, J.E., Stone, S.M., et al. (2008) 'Fourth-year medical students as PBL tutors', Medical Science Educator, 18(1), pp. 33-37.

 

Dandavino, M., Snell, L. and Wiseman, J. (2007) 'Why medical students should learn how to teach', Medical Teacher, 29(6), pp. 558-565. https://doi.org/10.1080/01421590701477449

 

General Medical Council. (2018) Outcomes for graduates. Available at: https://www.gmc-uk.org/-/media/documents/dc11326-outcomes-for-graduates-2018_pdf-75040796.pdf (Accessed April 1, 2018).

 

Nelson, A.J., Nelson, S.V., Linn, A.M., Raw, L.E., et al. (2013) 'Tomorrow's educators … today? Implementing near-peer teaching for medical students', Medical Teacher, 35(2), pp. 156-159. https://doi.org/10.3109/0142159X.2012.737961

 

Shah, K.P., Kohn, J.R., Goyal, S. and Stewart, D.E. (2017) 'Medical students as teachers: hands-on quality improvement education', Medical Education, 51(11), pp.1177-1178. https://doi.org/10.1111/medu.13446

 

Soriano, R.P., Blatt, B., Coplit, L., ChichoskiKelly, E., et al. (2010) 'Teaching medical students how to teach: a national survey of students-as-teachers programs in U.S. medical schools', Academic Medicine, 85(11), pp. 1725-1731. https://doi.org/10.1097/ACM.0b013e3181f53273

 

Yeung, C., Friesen, F., Farr, S., Law, M., et al. (2017) 'Development and implementation of a longitudinal students as teachers program: Participant satisfaction and implications for medical student teaching and learning', BMC Medical Education, 17(1), p. 28. https://doi.org/10.1186/s12909-017-0857-8.

 

Yoon, M.H., Blatt, B.C. and Greenberg, L.W. (2017) 'Medical students' professional development as educators revealed through reflections on their teaching following a students-as-teachers course', Teaching and Learning in Medicine, 29(4), pp. 411-419. https://doi.org/10.1080/10401334.2017.1302801

Appendices

None.

Declarations

There are no conflicts of interest.
This has been published under Creative Commons "CC BY-SA 4.0" (https://creativecommons.org/licenses/by-sa/4.0/)

Ethics Statement

This research was deemed exempt from review by the Florida International University Health Sciences Institution Review Board, protocol number IRB-18-0002.

External Funding

This paper has not had any External Funding

Reviews

Please Login or Register an Account before submitting a Review

Trevor Gibbs - (13/08/2018) Panel Member Icon
/
An interesting although a rather confusing paper.

I have not seen a "PBL" course that has been shortened to one day and like my colleagues I would not call what the authors describe here as a PBL session. It would have been helpful to see some of the various scenarios used to differentiate them from a case-based learning approach
That said, I also worry that the authors directly attribute the improvement in the students teaching to the PBL, given that they already had attended an orientation session on teaching and learning, had seven "goes" at teaching or presenting ( as the authors state", and had received some form of feedback through that seven opportunities time. I feel that there needed to be at least a comparative group who had not received those extra inputs to see if it was the PBL that brought about improvement.
How we improve "teaching" within our graduates is very important since in everyday of our professional lives we teach, to patients, co-workers, peers, near-peers and such like. How we develop that is often found in our communication skills training, but we need to look at other methods. I congratulate the authors in attempting to combine teaching development with curricula learning, although in this case I remain to be convinced.
Sateesh Babu Arja - (05/08/2018) Panel Member Icon
/
I did read this paper with much interest. The way authors described the way of conducting PBL is away from the normal process of conducting PBL. The normal process of PBL as described includes seven steps; identifying and clarifying the keywords, define the problem, analyze the problem, brainstorm the solutions, develop learning objectives, self study, and then sharing the information and synthesis and evaluation. May be it was implemented peer teaching at the level of sharing the information. It seems interesting. But I didn’t see that time was given for self-study or gathering the information as authors described their PBL sessions as one day sessions. The writing style is simple and easy understandable for academic faculty members.
P Ravi Shankar - (03/08/2018) Panel Member Icon
/
This is an interesting and well-written manuscript. It is important for medical students to be able to teach peers, colleagues, patients and the public as mentioned in the article. Third year medical students used PBL sessions to teach other members of the group and their performance were assessed by faculty members. All students were assessed and the results show a significant improvement in student performance as teachers. I assume that the faculty comments were based on the student presentations during the PBL. The authors have mentioned the limitations of their study and the take home points. I would be interested in greater information about the rubric used for assessing the student performance. The article will be of broad interest to medical educators.
Neslihan Onder-Ozdemir - (28/07/2018) Panel Member Icon
/
Thank you very much for your manuscript that focused on Problem-Based Learning (PBL) as an ‘education strategy’ in medical education. The gap regarding PBL in medical education has been reported very well: there is a paucity of study on “the use or impact of feedback to students within problem-based learning curricula to improve their teaching skills”.
Below are my comments to improve the content.
Methods section in the Abstract:
-In the Methods section in the abstract, the number of the students who participated in the PBL should be stated given that abstract is crucial, and PBL is usually performed with a small group of learners but also a larger group.
Methods section in the manuscript:
- Some information is needed whether Developmental Scale Rubric reported in this study has been developed by the researchers themselves. If so, some references could be provided how the three criteria have been chosen: Emerging, Acquiring, Mastering. This rubric could be beneficial for medical teachers if they want to use.
-Given that PBL is based on clinical problem, to contextualise this research and to show how this rubric can be used, a problem scenario from the research setting can be reported.
- In the Discussion part, authors reported as follows: “Our preliminary findings show that initial instruction in basic principles followed by narrative feedback within a problem-based learning course may contribute to improving students’ teaching skills.” In the Abstract: “rained faculty members have assessed and provided narrative comments on students’ “Ability to Teach Peers”. Please provide some data regarding faculty members’ assessment given that this research highlighted the significance of the use or impact of feedback to students within PBL.
-There is a need to report about the resource materials used to use PBL (handouts or instructions during the teaching?)
All in all, specific examples in the Methods section could make the PBL more concrete for the medical teachers.

Michelle McLean - (27/07/2018) Panel Member Icon
/
Thank you for asking me to review this short submission.

As a someone who believes that we should all be on the same page when we talk or write about PBL or TBL (highly developed educational approaches underpinned by a series or steps or processes), there is not enough in this article to convince me that the method being described is PBL, i.e. Case Opening with five steps, a period (often a week) of self-study (Step 6), then returning for the Case Closing or Wrap-up (final one or two steps, with Step 7 being elaboration and application and Step 8 being reflection). PBL was designed as a whole curriculum approach so the fact that there were only seven one-day sessions in the academic year suggests that what is being described as PBL may not in fact be PBL in terms of the intended approach?

The exercise of formally getting students to 'teach' is, however, well described and is something that could be introduced anywhere in the curriculum where the focus is on patient cases. The template and rubric could be shared with the medical education community.

I could, however, see how students could contribute to peer learning during Step 6 by providing the 'expertise' in a hybrid PBL program, i.e. the supplementary sessions to assist learners to meet the learning issues they identified in Steps 1-5 as they worked through a patient presentation. How this currently works in one day (as described) is a difficult to imagine.