Research article
Open Access

Geriatric Medicine Resident-as-Teachers

Karen A. D'Silva[1], Natasha Gakhal[2], Dov Gandell[3]

Institution: 1. University of Toronto, 2. Women's College Hospital, 3. Sunnybrook Health Sciences Centre
Corresponding Author: Dr Karen A. D'Silva ([email protected])
Categories: Curriculum Planning, Teachers/Trainers (including Faculty Development), Postgraduate (including Speciality Training)
Published Date: 03/06/2016

Abstract

Introduction

Subspecialty residents devote considerable time to teaching junior trainees; however, few studies have documented and evaluated resident-as-teacher (RaT) curricula for this group.  The purpose of this study was to (1) assess the need for formal teaching instruction among subspecialty geriatric medicine residents and (2) implement and evaluate a RaT curriculum based on the Stanford Faculty Development Program (SFDP). 

Methods

Survey-based needs assessment of subspecialty geriatric medicine residents and geriatric medicine faculty.  A single-group, pretest-posttest design used to evaluate the impact of a three-hour teaching seminar on residents’ teaching effectiveness and attitudes toward teaching.

Results

Residents reported limited knowledge of core teaching principles and limited exposure to formal teaching instruction.  However, residents expressed strong interest in improving their teaching skills.  Evaluation of the impact of the seminar intervention on residents’ attitudes toward teaching and teaching performance showed statistically non-significant results.

Conclusions

Study results were statistically non-significant limited by the design and sample size and future study would benefit from an experimental design with larger sample population.  They do, however, highlight the potential benefits of formal teaching instruction for subspecialty medical residents, an area that has not yet been well-explored in the medical education literature.   

Introduction

In most academic centers a significant portion of teaching, whether to more junior trainees or allied health professionals, is accomplished by resident physicians.  By some estimates, twenty to twenty-five percent of resident work hours are spent teaching (Brown, 1970).  Teaching is also a skill required by North American regulatory authorities, such as the Royal College of Physician and Surgeons (Frank, Snell et al., 2015).  Past research identifies many benefits from resident teaching.  Junior trainees rate residents as contributing significantly to their learning (Bing-You & Sproule, 1992), and students report that residents’ contribution may exceed that of faculty members (Seely et al., 1999).  Studies indicate that medical student rotation satisfaction increases as a result of resident teaching (Snell 2011; Huynh et al., 2011).  Medical students frequently view residents as role models for both professionalism and career choice (Sternszus et al., 2012; Chang, Odrobina  &, McIntyre-Seltman, 2010).

In an effort to formalize instruction and support for residents in the domain of teaching, there has been a growing interest in and presence of resident-as-teacher (RaT) training programs over the last 45 years (Hill et al., 2009).  Although programs have been very heterogeneous in terms of content, mode of instruction and duration, they consistently report positive outcomes for participants (Hill et al., 2009).  These include improvements in teaching skills and behaviours, enhanced participant attitudes toward and perceptions of teaching, as well as more positive student evaluations (Lawson & Harvill, 1980; Jewett, Greenberg, & Goldberg, 1982; Edwards et al., 1988; Greenberg, Goldberg, & Jewett, 1984; Bing-You & Greenberg, 1990; Dunnington & DaRosa, 1998; Morrison et al., 2004).

A limited body of literature describes RaT curricula for residents in their subspecialty or fellowship years, the last formative stage before transitioning to staff positions with teaching responsibilities.  RaT programs have been associated with improved perceptions of a specialty (Ogburn et al., 2005) and of the resident as clinician (Wamsley, Julian & Wipf, 2004). Moreover, it is likely that improving residents’ teaching skills also benefits patient care by enhancing confidence and communication skills (Snell, 2011).

As educators within the Division of Geriatric Medicine at the University of Toronto, our main objective was to improve the teaching skills of residents in the program.  This initiative was structured in two parts: 1) needs assessment of geriatric medicine residents and faculty; and 2) development, implementation, and evaluation of a three-hour interactive seminar to instruct geriatric residents how to teach using an established method, the Stanford Faculty Development Program (Skeff et al., 1992)(SFDP). Based on Kirkpatrick’s model (Kirkpatrick, 1994), we focused on changes in learner attitudes towards teaching (Kirkpatrick level 2; Table 1) (Hill et al., 2009) and changes in behaviours (Kirkpatrick level 3; Table 1) (Hill et al., 2009) to evaluate educational outcomes.  This model has been widely used to evaluate the impact of educational outcomes of RaT programs in the medical education literature (Hill et al., 2009).  We hypothesized that participation in the interactive seminar would improve residents’ teaching effectiveness and attitudes toward teaching.

Methods

Needs Assessment

The needs assessment was conducted using an anonymous, online, cross-sectional survey (Appendices B, C).  We developed the survey questions based on a review of the literature (Sanchez-Mendiola et al., 2010; Hannon, 2000; Keister & Grames 2012; Grant, 2002; Ratnapalan & Hilliard, 2002) which included questions pertaining to teaching experience, comfort, willingness to improve skills, and self-rated knowledge of the seven categories of clinical teaching from the SFDP (Skeff et al., 1992). The survey was piloted with four geriatric medicine residents, the postgraduate residency program director, and division director of geriatric medicine at the University of Toronto to informally assess clarity and face validity.  Several sets of revisions were made in order to accommodate the feedback and suggestions of all parties.  The University of Toronto Research Ethics Board approved the research protocols for the needs assessment and teaching intervention.

We emailed a description of the project and a link to the online survey on fluidsurveys.com to all geriatric medicine residents in Canada (n = 25) and all faculty members in the Division of Geriatric Medicine at the University of Toronto (n = 27) in October, 2013.  As incentive, we offered a $5 coffee shop gift card to participants.  We evaluated frequency data from the needs assessment to confirm the need for clinical teaching instruction among subspecialty geriatric medicine residents.  The data was analyzed by simple proportions.

Intervention: Three-Hour Interactive Seminar

We used a single-group, pre-test post-test design to evaluate the impact of a three-hour teaching seminar on residents’ teaching effectiveness and attitudes toward teaching.  The seminar was based on the SFDP educational framework (Skeff et al., 1992) and consisted of a description of each of the seven SFDP components, discussion of each component, role-play, and group and instructor feedback about their performance in the role-play.  It was held at the Li Ka Shing Knowledge Institute in December, 2013.  First, each SFDP component, including learning climate, control of session, goal setting, promotion of understanding and retention, evaluation, feedback, and self-directed learning, was defined and the components of each skill explained.  Residents were then asked to role play common clinical scenarios with one resident taking the teacher role.  The rest of the group was then asked to provide feedback on the teacher’s performance with respect to each SFDP skill.  Drs. D’Silva and Gandell then highlighted the important elements of each SFDP skill to reinforce the components of each skill.

Residents’ teaching effectiveness was assessed via two methods: indirect observation through videotaping of teaching performance and survey-based self-assessment.  These methods were chosen because they were commonly used in RaT curricula, based on recent systematic reviews (Hill et al., 2009;Wamsley, Julian, & Wipf, 2004), and because they were considered most feasible based on available program resources. For the indirect observation, geriatric medicine residents were videotaped before and after participating in the seminar, during their required rotational teaching of junior trainees.  Teaching topics included common geriatric syndromes such as falls, polypharmacy, and delirium, and were taught in an oral presentation format.  The videotaped sessions ranged in length from 10-20 minutes in length and were evaluated by two University of Toronto clinician teachers trained in the SFDP.  The evaluators were blinded to timing (pre/post intervention) and used a standardized and validated 25-item tool, based on the Stanford Educational framework, to evaluate residents’ teaching effectiveness (Litzelman et al., 1998).  Items corresponded to the seven SFDP categories and included “Encouraged learner to participate actively in discussion” (Learning Climate), “Stated relevance of goals to learners” (Communication of Goals), and “Evaluated learner’s ability to apply medical knowledge to specific patients” (Evaluation).  Agreement between the two evaluators was calculated using Spearman’s rank correlation coefficient. Correlation estimates could not be calculated for some of the questions because the data lacked variance (i.e., evaluators gave the same score to all five residents).  For each of the 25 items, we calculated residents’ average score from the two clinician teachers and then averaged the item scores for each of the seven categories.  Paired t-tests were used to compare the average pre-test and post-test scores for each of the seven categories.

Subspecialty residents’ assessments of the intervention focussed on self-evaluation of teaching skills and attitudes toward teaching pre- and immediately post-intervention.  Resident evaluations of the teaching seminar itself were not examined beyond one item in the post-intervention survey seeking opinion about the seminar usefulness.  This decision was made based on the Kirkpatrick model hierarchy in which participants’ views of the learning experience are considered less effective (level 1) as a measure of educational outcome than changes in attitudes or perceptions among residents toward teaching (level 2; Table 1) (Hill et al., 2009).

Residents rated their own teaching effectiveness pre- and immediately post-intervention by indicating their agreement on a five-point scale, with 11 statements about their teaching skills including, “In clinical teaching, I generally evaluate learners’ ability to apply knowledge” and “In clinical teaching, I generally give corrective feedback to learners” (see Table 4 for a complete list of items).

Residents’ attitudes toward teaching were assessed before and after the intervention via self-report.  Using a five-point, Likert scale, residents indicated their agreement with five statements about teaching; including “I enjoy teaching” and “Teaching learners is an important part of my job” (see Table 5 for complete list of items).

Results

Needs Assessment: Residents

The needs assessment survey was completed by 17 of 25 geriatric medicine residents (68% response rate).  Fewer than half of the subspecialty geriatric medicine residents surveyed (47%, 8/17) had been taught how to teach in clinical settings, although many (65%, 11/17) had received feedback from faculty on their teaching.  Residents displayed limited knowledge of the SFDP teaching categories, with only two of seven principles “known well enough to implement” by a majority of respondents: communication of goals (67%, 10/15) and feedback (53%, 8/15; Table 2).  In addition, fewer than half of residents knew the principles of writing learning objectives (41%, 7/17).

Subspecialty residents reported limited experience teaching large groups of medical students and junior residents, while teaching at the bedside occurred more frequently.  Most residents (71%, 12/17) did not have a collection of cases to illustrate their teaching objectives.  Despite somewhat limited teaching experience, 88% (15/17) of residents said they ‘often’ or ‘always’ feel comfortable teaching junior trainees about geriatric medicine and 94% (16/17) said they were comfortable with case-based teaching.  Comfort decreased, however, when teaching core geriatric medicine topics to juniors who were in post-graduate year three, or just below them, in training.

The needs assessment identified an interest in teaching among subspecialty geriatric medicine residents.  The majority of residents surveyed (76%, 13/17) planned to make teaching a large part of their career and all residents (17/17) wanted to acquire new teaching skills.  Residents’ preferred modes of teaching instruction were mentorship pairing (59%, 10/17), lectures/workshops by experts (53%, 9/17), small-group interactive discussions (53%, 9/17), and online tutorials (47%, 8/17).

Needs Assessment: Faculty

The needs assessment survey was completed by 16 of 27 faculty members (59% response rate). Approximately half of the faculty members surveyed (47%, 7/15) held positions as clinician educators.  Two faculty members had completed an undergraduate or graduate degree in education or teaching, whereas 27% (4/15) had completed a teaching certificate program and 53% (8/15) had participated in an individual class or professional development seminar. Compared to residents, faculty members displayed greater knowledge of the SFDP teaching categories (Table 3).  Among faculty, the best-known principles were communication of goals (71%, 10/14), evaluation and feedback (64%, 9/14), learning climate (57%, 8/14), and self-directed learning (57%, 8/14).  A majority of faculty members knew the principles of writing learning objectives (60%, 9/15).

More than two-thirds of the faculty members surveyed (69%, 11/16) had assessed residents’ teaching on six or more occasions and eighty percent of faculty members (12/15) said they “sometimes” or “often” provide feedback to residents on their teaching.  Only 7% (1/15) said they “always” give feedback.  Almost all faculty members described residents’ teaching skills as meeting expectations (93%, 13/14), as opposed to below (0/14) or exceeding expectations (7%, 1/14).

The most common approach faculty used to teach residents how to teach was observing and offering feedback (53%, 8/15); however, one-third of faculty members (33%, 5/15) said they did not have a specific method for teaching residents how to teach.  Only one of the fifteen faculty members surveyed (7%) reported having prepared sessions about teaching how to teach.

Faculty members often selected interactive workshops as the curriculum addition most likely to enhance the teaching skills of subspecialty geriatric medicine residents (80%, 12/15).  Faculty members were less likely to agree on the most important skill, for a subspecialty resident to possess to be an effective teacher.  The most frequently chosen response—‘imagination’—was selected by 29% of faculty members, while ‘public speaking/oral communication’ was selected by 21% of faculty (3/14) and ‘passion’ by 14% (2/14).

Faculty members reported having used a variety of strategies and resources to improve their own teaching skills, the most common of which were observation/discussion with skilled teachers (73%, 11/15), practice (73%, 11/15), and formal courses/programs in education (53%, 8/15).  In addition, a majority of faculty members (87%, 13/15) expressed interest in continuing to improve their teaching skills.  Faculty members’ preferred modes of teaching instruction were small-group interactive discussions (71%, 10/15), critiqued teaching (57%, 8/15) and lectures/workshops by experts (43%, 6/15).

Intervention: Videotaped Assessment of Teaching Effectiveness

A total of ten subspecialty geriatric medicine residents participated in the seminar at the Li Ka Shing Knowledge Institute in December, 2013.  Six of the participating residents were enrolled in the Geriatric Medicine program at the University of Toronto and four residents were enrolled in the Care of the Elderly program at the University of Toronto.  Of the ten residents who participated in the seminar, five geriatric medicine residents completed both the pre- and post-seminar videotaping.

For each teaching effectiveness category, post-intervention scores were not significantly higher than pre-intervention scores (Table 3).  Overall, there was considerable agreement (positive correlation) between the evaluators in some of the 25 questions; however, there was also considerable disagreement (negative correlation).  The number of disagreements was higher for the post-intervention evaluations.

Intervention: Resident Self-Assessment of Teaching Effectiveness

Compared to the pre-test, post-test sores were higher for ten of the eleven teaching effectiveness items in resident self-assessment surveys (Table 3).  However, only one item showed a statistically significant increase: “In clinical teaching, I generally evaluate learners’ ability to apply knowledge” (p = .01).

Intervention: Attitudes toward Teaching and Satisfaction with Seminar

None of the post-intervention questionnaire ratings for residents’ attitudes toward teaching showed a statistically significant change from pre-intervention (Table 4).  All residents rated the seminar intervention as useful to their practice, with a Likert scale rating of four (20%, 2/10) or five (80%, 8/10 residents) on the 5-point scale.

Discussion

Despite growing interest in residents’ role as clinical teachers, few studies have focused on the development and evaluation of resident-as-teacher (RaT) curricula for trainees in their subspecialty or fellowship years.  To address this gap in the literature, we first conducted a survey-based, teaching-focused needs assessment of subspecialty geriatric medicine residents and geriatric medicine faculty members. We then implemented and evaluated a three-hour, RaT training program based on the Stanford Faculty Development Program (Skeff et al., 1992).  

Needs Assessment

The resident and faculty surveys had response rates of 68% and 59% respectively.  These rates are generally consistent with the published mean response rate among surveys in medical journals (Asch, Jedrziewski, & Christakis, 1997).  Although non-response rates of about 30-40%, such as these, do reduce the sample size and limit the strength of conclusions, overall survey results supported a need for formal teaching instruction among subspecialty geriatric medicine residents.

Residents reported limited knowledge of core teaching principles, including evaluation and writing learning objectives, and had limited exposure to formal teaching instruction.  Although residents received regular feedback on their teaching, most had not been taught how to teach in a clinical setting.  In addition, a large number of faculty members reported that they did not have prepared sessions on teaching residents how to teach.  Residents were experienced in teaching at the bedside but had infrequently taught large groups of trainees. Despite gaps in knowledge and experience, residents were generally comfortable teaching more junior trainees and expressed strong interest in improving their teaching skills.  Residents preferred to learn about teaching through mentorship pairing, lectures/workshops led by experts, and small-group interactive discussions.  Faculty members also indicated that interactive workshops would be the curriculum addition most likely to enhance residents’ teaching skills. 

Intervention: Videotaped Assessment of Teaching Effectiveness

Our focus for this initial study based on scope and available population sample, was to impact geriatric medicine resident attitudes, knowledge and skills (Kirkpatrick Level 2A, 2B) (Hill et al., 2009; Table 1).  Based on the needs assessment results, we expected residents would respond positively to our workshop-based teaching curriculum.  However, the blinded videotape assessor ratings of resident teaching pre- and post-curriculum implementation were non-significantly different in all of the seven teaching categories.  Moreover, there was significant variability in agreement between the two raters.  This may have been due to a lack of standardized training in using the evaluation tool in addition to limitations of study design.

The main limitations of this study were small population size and the lack of experimental design.  Unfortunately, small numbers are currently a limitation of any individual postgraduate Geriatric Medicine program in Canada.  Therefore, future investigators may consider combining residents from all postgraduate Geriatric Medicine programs within a particular region or including more than one type of subspecialty program, to increase sample size and, thereby, significance of results. 

Given the single-group, pre-test post-test design (i.e., no control group), it was difficult to establish a causal relationship between exposure to the intervention and the observed outcomes.  Even if post-intervention scores were significantly higher, they may still only reflect only natural growth in skills as residents became more experienced teachers over time.  It is also possible that residents were exposed to other sources of teaching instruction during the study time frame which could have confounded residents’ scores at post-intervention.  Completing the pre-intervention survey may have also sensitized residents to the purpose of the study and influenced post-intervention survey responses.  Future research on RaT curricula would benefit from a controlled design to help establish a cause-and-effect relationship.

Intervention: Attitudes toward Teaching and Satisfaction with Seminar

Residents’ attitudes toward teaching (Kirpatrick Level 2A) (Hill et al., 2009; Table 1) are difficult to interpret based on our study results. Their self-reported teaching effectiveness and attitudes toward teaching post-intervention were generally higher, although only one of the 17 paired t-tests conducted was statistically significant. This result that could be attributed to chance given the 95% confidence level. 

Beyond the limitations of study design and sample size, the lack of change in attitude may be explained by several other factors.  First, it may be unrealistic to expect marked changes in teaching effectiveness following a three-hour seminar.  Residents may require a longitudinal curriculum in order to experience significant benefit from a training program.  However, allocation of resources for this considerable undertaking would need to be weighed against potential gains, particularly in postgraduate programs such as Geriatric Medicine, in which resources are limited by relatively small numbers of trainees.  Interpretation of post-intervention survey results for teaching attitudes also should take into consideration that the pre-intervention scores were quite high and thus ceiling effects may have prevented significant gains.  Finally, our seminar was designed for and attended by geriatric medicine residents, which may limit the generalizability of results to residents in other subspecialty programs.  The seminar, however, was largely focused on the SFDP teaching principles which have been used for and found relevant to, other medical specialties (Litzelman, et al., 1998; Johansson, Skeff, & Stratos, 2009). 

Despite the limitations of our study, the results were encouraging toward further development of an RaT curriculum for subspecialty residents. A relatively brief seminar providing a framework of teaching skills was received positively among our geriatric medicine residents who universally identified the seminar as “useful to their practice.” Residents at this last formative stage prior to transitioning to staff positions with teaching responsibilities expressed a need to have a structured approach to improve their skills.   The literature has shown that junior trainees can certainly benefit from learning from better resident teachers both in terms of knowledge gained and overall satisfaction with any given rotation (Snell 2011; Huynh et al., 2011).  Moreover, faculty members expressed a need to improve their own skills and have the potential to play a more active role in teaching residents how to teach by participating in RaT seminars.  If our RaT seminar were to be used as a model for further curriculum building, further consideration should be given to duration of training required to achieve desired learning outcomes and ways to optimize reliability and validity of evaluators’ assessments.  

Conclusion

Our research was an initial study that identified a need for, and interest in, formal teaching instruction among subspecialty residents.  Exposure to a brief teaching seminar based on the SFDP teaching principles can have a positive outcome on resident attitudes toward teaching; however, future study would benefit from an experimental design and a larger sample population in order to broaden applicability and interpretation of results.

Take Home Messages

  • There is a clear need for formal teaching instruction among subspecialty geriatric medicine residents.
  • Even a brief instructional seminar on teaching can have positive outcomes for residents.
  • Future study would benefit from an experimental design and a larger sample population in order to broaden applicability and interpretation of results.

Notes On Contributors

Dr. D’Silva is a Lecturer in the Department of Medicine at the University of Toronto and full-time clinician teacher in the Division of Geriatric Medicine, St. Michael’s Hospital, Toronto, Ontario, Canada.

Dr. Gakhal is a Lecturer in the Department of Medicine at the University of Toronto and full-time clinician in quality and innovation in the Division of Rheumatology, Women's College Hospital, Toronto, Ontario, Canada.

Dr. Gandell is an Assistant Professor in the Department of Medicine at the University of Toronto and full-time clinician teacher in the Division of Geriatric Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada.

Acknowledgements

The authors thank Dr. Savannah Cardew, MSc, MD (Faculty of Medicine, Division of General Internal Medicine, University of Toronto, Toronto, Canada) for her help with the assessment of the trainee videotaped teaching samples and Jemila Hamid, MSc, PhD, (Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Canada) for her assistance with the statistics used in this manuscript as part of her normal duties. We also thank Ariel Chernin, PhD, (Annenberg School for Communication, University of Pennsylvania, Philadelphia, Pennsylvania) for help with editing early versions of the manuscript.

Bibliography/References

Asch, D. A., Jedrziewski, M. K., & Christakis, N. A. (1997). Response rates to mail surveys published in medical journals. Journal of Clinical Epidemiology, 50(10), 1129-1136.

http://dx.doi.org/10.1016/S0895-4356(97)00126-1

Bing-You, R. G., & Greenberg, L. W. (1990). Training residents in clinical teaching skills: A resident-managed program. Medical Teacher, 12(3-4), 305-309.

http://dx.doi.org/10.3109/01421599009006635

Bing-You, R. G., & Sproul, M. S. (1992). Medical students' perceptions of themselves and residents as teachers. Medical Teacher, 14(2-3), 133-138.

http://dx.doi.org/10.3109/01421599209079479

Brown, R. S. (1970). House staff attitudes toward teaching. Journal of Medical Education, 45(3), 156-159.

http://dx.doi.org/10.1097/00001888-197003000-00005

Chang, J. C., Odrobina, M. R., & McIntyre-Seltman, K. (2010). Residents as role models: The effect of the obstetrics and gynecology clerkship on medical students' career interest. Journal of Graduate Medical Education, 2(3), 341-345.

http://dx.doi.org/10.4300/JGME-D-09-00070.1

Dunnington, G. L., & DaRosa, D. (1998). A prospective randomized trial of a residents-as-teachers training program. Academic Medicine : Journal of the Association of American Medical Colleges, 73(6), 696-700.

http://dx.doi.org/10.1097/00001888-199806000-00017

Edwards, J. C., Kissling, G. E., Plauche, W. C., & Marier, R. L. (1988). Evaluation of a teaching skills improvement programme for residents. Medical Education, 22(6), 514-517.

http://dx.doi.org/10.1111/j.1365-2923.1988.tb00796.x

Frank JR, Snell L, et al. (2014). Draft CanMEDS 2015 Physician Competency Framework – Series I. Ottawa: The Royal College of Physicians and Surgeons of Canada, Accreditation Council for Graduate Medical Education.

Grant, J. (2002). Learning needs assessment: Assessing the need. BMJ (Clinical Research Ed.), 324(7330), 156-159.

http://dx.doi.org/10.1136/bmj.324.7330.156

Greenberg, L. W., Goldberg, R. M., & Jewett, L. S. (1984). Teaching in the clinical setting: Factors influencing residents' perceptions, confidence and behaviour. Medical Education, 18(5), 360-365.

http://dx.doi.org/10.1111/j.1365-2923.1984.tb01283.x

Hannon, F. B. (2000). A national medical education needs' assessment of interns and the development of an intern education and training programme. Medical Education, 34(4), 275-284.

http://dx.doi.org/10.1046/j.1365-2923.2000.00512.x

Hill, A. G., Yu, T. C., Barrow, M., & Hattie, J. (2009). A systematic review of resident-as-teacher programmes. Medical Education, 43(12), 1129-1140.

http://dx.doi.org/10.1111/j.1365-2923.2009.03523.x

Huynh, A., Savitski, J., Kirven, M., Godwin, J., & Gil, K. M. (2011). Effect of medical students' experiences with residents as teachers on clerkship assessment. Journal of Graduate Medical Education, 3(3), 345-349.

http://dx.doi.org/10.4300/JGME-03-03-35

Jewett, L. S., Greenberg, L. W., & Goldberg, R. M. (1982). Teaching residents how to teach: A one-year study. Journal of Medical Education, 57(5), 361-366.

http://dx.doi.org/10.1097/00001888-198205000-00002

Johansson, J., Skeff, K., & Stratos, G. (2009). Clinical teaching improvement: The transportability of the stanford faculty development program. Medical Teacher, 31(8), e377-82.

http://dx.doi.org/10.1080/01421590802638055

Keister, D., & Grames, H. (2012). Multi-method needs assessment optimises learning. The Clinical Teacher, 9(5), 295-298.

http://dx.doi.org/10.1111/j.1743-498X.2012.00549.x

Kirkpatrick, D. L., & Kirkpatrick, J. D. (1994). Evaluating training programs: The four levels (Third ed.). California: Berrett-Koehler Publishers.

Lawson, B. K., & Harvill, L. M. (1980). The evaluation of a training program for improving residents' teaching skills. Journal of Medical Education, 55(12), 1000-1005.

Litzelman, D. K., Stratos, G. A., Marriott, D. J., & Skeff, K. M. (1998). Factorial validation of a widely disseminated educational framework for evaluating clinical teachers. Academic Medicine : Journal of the Association of American Medical Colleges, 73(6), 688-695.

http://dx.doi.org/10.1097/00001888-199806000-00016

Morrison, E. H., Rucker, L., Boker, J. R., Gabbert, C. C., Hubbell, F. A., Hitchcock, M. A., & Prislin, M. D. (2004). The effect of a 13-hour curriculum to improve residents' teaching skills: A randomized trial. Annals of Internal Medicine, 141(4), 257-263.

http://dx.doi.org/10.7326/0003-4819-141-4-200408170-00005

Ogburn, J. A., Espey, E. L., Dorin, M. H., Ming, C., & Rayburn, W. F. (2005). Obstetrics and gynecology residents as teachers of medical students: Predictors of excellence. American Journal of Obstetrics and Gynecology, 193(5), 1831-1834.

http://dx.doi.org/10.1016/j.ajog.2005.07.074

Ratnapalan, S., & Hilliard, R. (2002). Needs assessment in postgraduate medical education: A review. Medical Education Online, 7(8), 1-8.

http://dx.doi.org/10.3402/meo.v7i.4542

Seely, A. J., Pelletier, M. P., Snell, L. S., & Trudel, J. L. (1999). Do surgical residents rated as better teachers perform better on in-training examinations? American Journal of Surgery, 177(1), 33-37.

http://dx.doi.org/10.1016/S0002-9610(98)00306-7

Skeff, K. M., Stratos, G. A., Berman, J., & Bergen, M. R. (1992). Improving clinical teaching. evaluation of a national dissemination program. Archives of Internal Medicine, 152(6), 1156-1161.

http://dx.doi.org/10.1001/archinte.1992.00400180028004

Snell, L. (2011). The resident-as-teacher: It's more than just about student learning. Journal of Graduate Medical Education, 3(3), 440-441.

http://dx.doi.org/10.4300/JGME-D-11-00148.1

Sternszus, R., Cruess, S., Cruess, R., Young, M., & Steinert, Y. (2012). Residents as role models: Impact on undergraduate trainees. Academic Medicine : Journal of the Association of American Medical Colleges, 87(9), 1282-1287.

http://dx.doi.org/10.1097/ACM.0b013e3182624c53

Wamsley, M. A., Julian, K. A., & Wipf, J. E. (2004). A literature review of "resident-as-teacher" curricula: Do teaching courses make a difference? Journal of General Internal Medicine, 19(5 Pt 2), 574-581.

http://dx.doi.org/10.1111/j.1525-1497.2004.30116.x

 



 

Appendices

Table 1: Kirkpatrick’s Model for Evaluating Educational Outcomes (9)

Level 1

Reaction

Participants’ views of the learning experience, its organisation, presentation, content, teaching methods, and quality of instruction

Level 2A

Learning – change in attitudes

Changes in attitudes or perceptions among participant groups towards teaching and learning

Level 2B

Learning – modification of knowledge or skills

For knowledge, this relates to the acquisition of concepts, procedures and principles

 

For skills, this relates to the acquisition of thinking and problem-solving, psychomotor and social skills

Level 3

Behaviour – change in behaviours

Documents the transfer of learning to the workplace or willingness of learners to apply new knowledge and skills

Level 4A

Results – change in the system or organizational practice

Refers to wider changes in the organisation attributable to the educational programme

Level 4B

Results – change among the participants’ students and peers

Refers to improvement in medical student or peer learning or performance as a direct result of the educational intervention

 

 

 

Table 2 description. Needs assessment survey results showing Geriatrics subspecialty residents and faculty self-rated knowledge (phrased as “known well enough to implement”) of the SFDP categories.

Table 2. SFDP Teaching Categories Known Well Enough to Implement

 

Residents

(n = 15)

Faculty

(n = 14)

Learning climate

27%

57%

Control of session

20%

50%

Communication of goals

67%

71%

Promotion of understanding and retention

27%

36%

Evaluation*

40%

64%

Feedback

53%

n/a

Self-directed learning

27%

57%

1For faculty, “evaluation and feedback” were listed as one item on the survey rather than as two separate categories.

 

 

 

Table 3 description. Geriatrics residents teaching (videotaped samples) scores determined by two clinician teachers, before and after the intervention. For each of the 25 items on the evaluation tool, the residents’ average score was calculated from the two clinician teachers’ individual scores and the item scores were then averaged for each of the seven SFDP categories. Paired t-tests were used to compare the average pre-test and post-test scores.

Table 3. Geriatrics Residents’ Teaching Evaluation Scores Pre- and Post-Intervention (n=5)

SFDP Category

Mean  Score [95% CI]

P-value*

 

Before

After

 

Learning climate

4.2 [3.6, 4.8]

4.4 [4.2, 4.6]

0.40

Control of session

3.5 [3.2, 3.8]

3.3 [3.0, 3.5]

0.13

Communication of goals

3.7 [3.4, 4.07]

3.4 [2.7, 4.1]

0.38

Promoting understanding and retention

3.5 [3.1, 4.0]

 

3.4 [3.1, 3.8]

 

0.70

 

Evaluation

3.9 [3.4, 4.4]

4.0 [3.6, 4.4]

0.61

Feedback

3.2 [2.6, 3.7]

3.3 [3.0, 3.6]

0.39

Self-directed learning

3.2 [2.0, 4.5]

3.1 [2.0, 4.1]

0.23

* p-value is calculated using paired t-test

 

 

Table 4 description. Geriatrics residents mean self-ratings of teaching effectiveness toward teaching, before and after the intervention.

Table 4. Residents’ Self-Reported Teaching Effectiveness Teaching, Pre- and Post-Intervention (n = 10)

Survey item

Mean Rating [95% CI]

P-value*

 

Pre

Post

 

In clinical teaching I generally. . .

 

 

 

Stimulate learners' interest

4.6 [4.2, 5.0]

4.6 [4.1, 5.1]

1.00

Motivate learners to learn on their own

3.6 [2.9, 4.3]

3.8 [3.2, 4.4]

0.59

Explicitly state goals for learners

3.8 [3.0, 4.6]

4.1 [3.7, 4.5]

0.47

Explain material clearly

3.9 [3.3, 4.5]

4.2 [3.7, 4.7]

0.47

Use visual aids

4.0 [3.6, 4.4]

4.2 [3.7, 4.7]

0.35

Make efficient use of teaching time

3.6 [3.1, 4.1]

3.9 [3.4, 4.4]

0.34

Am willing to say "I don't know”

4.8 [ 4.5, 5.1]

4.9 [4.7, 5.1]

0.59

Evaluate learners' knowledge

3.3 [2.8, 3.8]

3.7 [3.4, 4.0]

0.10

Evaluate learners' ability to apply knowledge

3.1 [2.7, 3.5]

3.8 [3.3, 4.3]

0.01

 

Give positive feedback to learners

4.1 [3.6, 4.6]

4.3 [3.8, 4.8]

0.62

Give corrective feedback to learners

3.5 [2.9, 4.1]

4.0 [NA]

0.10

 

 

 

Table 5 description. Geriatrics residents’ mean self-ratings of attitudes toward teaching, before and after the intervention.

Table 5. Residents’ Self-Reported Attitudes Toward Teaching, Pre- and Post-Intervention (n = 10)

Survey item

Mean  Rating [95% CI]

P-value*

 

Pre

Post

 

Teaching learners is an important aspect of my job

4.7 [4.4, 5.0]

4.8 [4.4, 5.1]

 

0.80

 

I enjoy teaching

4.5 [4.0, 5.0]

4.5 [4.0, 5.0]

1.00

I would like to improve my teaching

4.7 [4.4, 5.0]

4.6 [4.2, 5.0]

0.73

I need to improve my teaching

4.1 [3.5, 4.7]

4.4 [3.9, 4.9]

0.47

It is important to have a program for teaching residents to teach

4.7 [4.4, 5.0]

4.8 [4.4, 5.1]

0.68

 

 

Declarations

There are no conflicts of interest.
This has been published under Creative Commons "CC BY-SA 4.0" (https://creativecommons.org/licenses/by-sa/4.0/)

Reviews

Please Login or Register an Account before submitting a Review

Trevor Gibbs - (08/06/2016) Panel Member Icon
/
With increased pressure upon teaching faculty, the use of residents in reducing number of faculty employed to teach and the increased need to balance service and research commitments with teaching and learning activities, this well-written paper relates to a very important area- the use of residents in training as teachers. I would be at odds with the authors in that there is limited evidence as to the value of residents as teachers- see the excellent AMEE Guide 103 by Ramani, Mann, Taylor and Thampy. However I would agree that there is limited research that explores the area of need in including residents within faculty training programmes.
Given the negative results from this intervention, I would have liked to have seen a much more critical analysis of the outcome shown, be it of the methodology, the content of the three hour training session and its impact or what a bigger and more longer-term project might have produced
Richard Hays - (08/06/2016) Panel Member Icon
/
The concept behind this paper is sound, but the results are disappointing. It would be nice to see a measureable improvement in attitudes towards and effectiveness of faculty development. The survey was based on a validated faculty development framework, analysis of videotaped encounters is well established and validated, and the intervention (a single three-hour session) is a familiar approach, although often difficult to carve out of time focussed on clinical duties. It is therefore disappointing that the faculty development provided had such little impact. This study may be an example of one that is conceptually sound, uses appropriate methods, has appropriate analysis, and yet is ‘negative’ for meaningful outcomes. This may be related to relatively small numbers of participants – more could be added over time to address that – or perhaps the framework, interventions and measurements have problems. The alternative view may be that much more faculty development is needed, and the culture of teaching more strongly embedded, to achieve the desired outcomes of faculty development. A three-hour session may be a pragmatic, but relatively weak intervention. The authors have identified the need for further work to clarify these issues.
Style: Well written and easy to read.
Target audience: Of most interest to faculty developers.