Geriatric Medicine Resident-as-Teachers

Subspecialty residents devote considerable time to teaching junior trainees; however, few studies have documented and evaluated resident-as-teacher (RaT) curricula for this group. The purpose of this study was to (1) assess the need for formal teaching instruction among subspecialty geriatric medicine residents and (2) implement and evaluate a RaT curriculum based on the Stanford Faculty Development Program (SFDP). Methods Survey-based needs assessment of subspecialty geriatric medicine residents and geriatric medicine faculty. A single-group, pretest-posttest design used to evaluate the impact of a three-hour teaching seminar on residents’ teaching effectiveness and attitudes toward teaching. Results Residents reported limited knowledge of core teaching principles and limited exposure to formal teaching instruction. However, residents expressed strong interest in improving their teaching skills. Evaluation of the impact of the seminar intervention on residents’ attitudes toward teaching and teaching performance showed statistically non-significant results. Conclusions Study results were statistically non-significant limited by the design and sample size and future study would benefit from an experimental design with larger sample population. They do, however, highlight the potential benefits of formal teaching instruction for subspecialty medical residents, an area that has not yet been well-explored in the medical education literature.


Introduction
health professionals, is accomplished by resident physicians.By some estimates, twenty to twenty-five percent of resident work hours are spent teaching (Brown, 1970).Teaching is also a skill required by North American regulatory authorities, such as the Royal College of Physician and Surgeons (Frank, Snell et al., 2015).Past research identifies many benefits from resident teaching.Junior trainees rate residents as contributing significantly to their learning (Bing-You & Sproule, 1992), and students report that residents' contribution may exceed that of faculty members (Seely et al., 1999).Studies indicate that medical student rotation satisfaction increases as a result of resident teaching (Snell 2011;Huynh et al., 2011).Medical students frequently view residents as role models for both professionalism and career choice (Sternszus et al., 2012;Chang, Odrobina &, McIntyre-Seltman, 2010).In an effort to formalize instruction and support for residents in the domain of teaching, there has been a growing interest in and presence of resident-as-teacher (RaT) training programs over the last 45 years (Hill et al., 2009).Although programs have been very heterogeneous in terms of content, mode of instruction and duration, they consistently report positive outcomes for participants (Hill et al., 2009).These include improvements in teaching skills and behaviours, enhanced participant attitudes toward and perceptions of teaching, as well as more positive student evaluations (Lawson & Harvill, 1980;Jewett, Greenberg, & Goldberg, 1982;Edwards et al., 1988;Greenberg, Goldberg, & Jewett, 1984;Bing-You & Greenberg, 1990;Dunnington & DaRosa, 1998;Morrison et al., 2004).A limited body of literature describes RaT curricula for residents in their subspecialty or fellowship years, the last formative stage before transitioning to staff positions with teaching responsibilities.RaT programs have been associated with improved perceptions of a specialty (Ogburn et al., 2005) and of the resident as clinician (Wamsley, Julian & Wipf, 2004).Moreover, it is likely that improving residents' teaching skills also benefits patient care by enhancing confidence and communication skills (Snell, 2011).As educators within the Division of Geriatric Medicine at the University of Toronto, our main objective was to improve the teaching skills of residents in the program.This initiative was structured in two parts: 1) needs assessment of geriatric medicine residents and faculty; and 2) development, implementation, and evaluation of a three-hour interactive seminar to instruct geriatric residents how to teach using an established method, the Stanford Faculty Development Program (Skeff et al., 1992)(SFDP).Based on Kirkpatrick's model (Kirkpatrick, 1994), we focused on changes in learner attitudes towards teaching (Kirkpatrick level 2; Table 1) (Hill et al., 2009) and changes in behaviours (Kirkpatrick level 3; Table 1) (Hill et al., 2009) to evaluate educational outcomes.This model has been widely used to evaluate the impact of educational outcomes of RaT programs in the medical education literature (Hill et al., 2009).We hypothesized that participation in the interactive seminar would improve residents' teaching effectiveness and attitudes toward teaching.

Needs Assessment
The needs assessment was conducted using an anonymous, online, cross-sectional survey (Appendices B, C).We developed the survey questions based on a review of the literature (Sanchez-Mendiola et al., 2010;Hannon, 2000;Keister & Grames 2012;Grant, 2002;Ratnapalan & Hilliard, 2002) which included questions pertaining to teaching experience, comfort, willingness to improve skills, and selfrated knowledge of the seven categories of clinical teaching from the SFDP (Skeff et al., 1992).The survey was piloted with four geriatric medicine residents, the postgraduate residency program director, and division director of geriatric medicine at the University of Toronto to informally assess clarity and D'Silva K, Gakhal N, Gandell D MedEdPublish https://doi.org/10.15694/mep.2016.000004

Methods
face validity.Several sets of revisions were made in order to accommodate the feedback and suggestions of all parties.The University of Toronto Research Ethics Board approved the research protocols for the needs assessment and teaching intervention.We emailed a description of the project and a link to the online survey on fluidsurveys.comto all geriatric medicine residents in Canada (n = 25) and all faculty members in the Division of Geriatric Medicine at the University of Toronto (n = 27) in October, 2013.As incentive, we offered a $5 coffee shop gift card to participants.We evaluated frequency data from the needs assessment to confirm the need for clinical teaching instruction among subspecialty geriatric medicine residents.The data was analyzed by simple proportions.Intervention: Three-Hour Interactive Seminar We used a single-group, pre-test post-test design to evaluate the impact of a three-hour teaching seminar on residents' teaching effectiveness and attitudes toward teaching.The seminar was based on the SFDP educational framework (Skeff et al., 1992) and consisted of a description of each of the seven SFDP components, discussion of each component, role-play, and group and instructor feedback about their performance in the role-play.It was held at the Li Ka Shing Knowledge Institute in December, 2013.First, each SFDP component, including learning climate, control of session, goal setting, promotion of understanding and retention, evaluation, feedback, and self-directed learning, was defined and the components of each skill explained.Residents were then asked to role play common clinical scenarios with one resident taking the teacher role.The rest of the group was then asked to provide feedback on the teacher's performance with respect to each SFDP skill.Drs.D'Silva and Gandell then highlighted the important elements of each SFDP skill to reinforce the components of each skill.Residents' teaching effectiveness was assessed via two methods: indirect observation through videotaping of teaching performance and survey-based self-assessment.These methods were chosen because they were commonly used in RaT curricula, based on recent systematic reviews (Hill et al., 2009;Wamsley, Julian, & Wipf, 2004), and because they were considered most feasible based on available program resources.For the indirect observation, geriatric medicine residents were videotaped before and after participating in the seminar, during their required rotational teaching of junior trainees.Teaching topics included common geriatric syndromes such as falls, polypharmacy, and delirium, and were taught in an oral presentation format.The videotaped sessions ranged in length from 10-20 minutes in length and were evaluated by two University of Toronto clinician teachers trained in the SFDP.The evaluators were blinded to timing (pre/post intervention) and used a standardized and validated 25-item tool, based on the Stanford Educational framework, to evaluate residents' teaching effectiveness (Litzelman et al., 1998).Items corresponded to the seven SFDP categories and included "Encouraged learner to participate actively in discussion" (Learning Climate), "Stated relevance of goals to learners" (Communication of Goals), and "Evaluated learner's ability to apply medical knowledge to specific patients" (Evaluation).Agreement between the two evaluators was calculated using Spearman's rank correlation coefficient.Correlation estimates could not be calculated for some of the questions because the data lacked variance (i.e., evaluators gave the same score to all five residents).For each of the 25 items, we calculated residents' average score from the two clinician teachers and then averaged the item scores for each of the seven categories.Paired t-tests were used to compare the average pre-test and post-test scores for each of the seven categories.Subspecialty residents' assessments of the intervention focussed on self-evaluation of teaching skills and attitudes toward teaching pre-and immediately post-intervention.Resident evaluations of the teaching seminar itself were not examined beyond one item in the post-intervention survey seeking opinion about D'Silva K, Gakhal N, Gandell D MedEdPublish https://doi.org/10.15694/mep.2016.000004 the seminar usefulness.This decision was made based on the Kirkpatrick model hierarchy in which participants' views of the learning experience are considered less effective (level 1) as a measure of educational outcome than changes in attitudes or perceptions among residents toward teaching (level 2; Table 1) (Hill et al., 2009).Residents rated their own teaching effectiveness pre-and immediately post-intervention by indicating their agreement on a five-point scale, with 11 statements about their teaching skills including, "In clinical teaching, I generally evaluate learners' ability to apply knowledge" and "In clinical teaching, I generally give corrective feedback to learners" (see Table 4 for a complete list of items).Residents' attitudes toward teaching were assessed before and after the intervention via self-report.Using a five-point, Likert scale, residents indicated their agreement with five statements about teaching; including "I enjoy teaching" and "Teaching learners is an important part of my job" (see Table 5 for complete list of items).

Needs Assessment: Residents
The needs assessment survey was completed by 17 of 25 geriatric medicine residents (68% response rate).Fewer than half of the subspecialty geriatric medicine residents surveyed (47%, 8/17) had been taught how to teach in clinical settings, although many (65%, 11/17) had received feedback from faculty on their teaching.Residents displayed limited knowledge of the SFDP teaching categories, with only two of seven principles "known well enough to implement" by a majority of respondents: communication of goals (67%, 10/15) and feedback (53%, 8/15; Table 2).In addition, fewer than half of residents knew the principles of writing learning objectives (41%, 7/17).Subspecialty residents reported limited experience teaching large groups of medical students and junior residents, while teaching at the bedside occurred more frequently.Most residents (71%, 12/17) did not have a collection of cases to illustrate their teaching objectives.Despite somewhat limited teaching experience, 88% (15/17) of residents said they 'often' or 'always' feel comfortable teaching junior trainees about geriatric medicine and 94% (16/17) said they were comfortable with case-based teaching.Comfort decreased, however, when teaching core geriatric medicine topics to juniors who were in postgraduate year three, or just below them, in training.The needs assessment identified an interest in teaching among subspecialty geriatric medicine residents.The majority of residents surveyed (76%, 13/17) planned to make teaching a large part of their career and all residents (17/17) wanted to acquire new teaching skills.Residents' preferred modes of teaching instruction were mentorship pairing (59%, 10/17), lectures/workshops by experts (53%, 9/17), smallgroup interactive discussions (53%, 9/17), and online tutorials (47%, 8/17).Needs Assessment: Faculty The needs assessment survey was completed by 16 of 27 faculty members (59% response rate).Approximately half of the faculty members surveyed (47%, 7/15) held positions as clinician educators.Two faculty members had completed an undergraduate or graduate degree in education or teaching, whereas 27% (4/15) had completed a teaching certificate program and 53% (8/15) had participated in an individual class or professional development seminar.Compared to residents, faculty members displayed greater knowledge of the SFDP teaching categories (Table 3).Among faculty, the best-known principles were communication of goals (71%, 10/14), evaluation and feedback (64%, 9/14), learning climate (57%, 8/14), and self-directed learning (57%, 8/14).A majority of faculty members knew the principles of writing learning objectives (60%, 9/15).

Results
More than two-thirds of the faculty members surveyed (69%, 11/16) had assessed residents' teaching on six or more occasions and eighty percent of faculty members (12/15) said they "sometimes" or "often" provide feedback to residents on their teaching.Only 7% (1/15) said they "always" give feedback.Almost all faculty members described residents' teaching skills as meeting expectations (93%, 13/14), as opposed to below (0/14) or exceeding expectations (7%, 1/14).The most common approach faculty used to teach residents how to teach was observing and offering feedback (53%, 8/15); however, one-third of faculty members (33%, 5/15) said they did not have a specific method for teaching residents how to teach.Only one of the fifteen faculty members surveyed (7%) reported having prepared sessions about teaching how to teach.Faculty members often selected interactive workshops as the curriculum addition most likely to enhance the teaching skills of subspecialty geriatric medicine residents (80%, 12/15).Faculty members were less likely to agree on the most important skill, for a subspecialty resident to possess to be an effective teacher.The most frequently chosen response-'imagination'-was selected by 29% of faculty members, while 'public speaking/oral communication' was selected by 21% of faculty (3/14) and 'passion' by 14% (2/14).Faculty members reported having used a variety of strategies and resources to improve their own teaching skills, the most common of which were observation/discussion with skilled teachers (73%, 11/15), practice (73%, 11/15), and formal courses/programs in education (53%, 8/15).In addition, a majority of faculty members (87%, 13/15) expressed interest in continuing to improve their teaching skills.Faculty members' preferred modes of teaching instruction were small-group interactive discussions (71%, 10/15), critiqued teaching (57%, 8/15) and lectures/workshops by experts (43%, 6/15).Intervention: Videotaped Assessment of Teaching Effectiveness A total of ten subspecialty geriatric medicine residents participated in the seminar at the Li Ka Shing Knowledge Institute in December, 2013.Six of the participating residents were enrolled in the Geriatric Medicine program at the University of Toronto and four residents were enrolled in the Care of the Elderly program at the University of Toronto.Of the ten residents who participated in the seminar, five geriatric medicine residents completed both the pre-and post-seminar videotaping.For each teaching effectiveness category, post-intervention scores were not significantly higher than preintervention scores (Table 3).Overall, there was considerable agreement (positive correlation) between the evaluators in some of the 25 questions; however, there was also considerable disagreement (negative correlation).The number of disagreements was higher for the post-intervention evaluations.Intervention: Resident Self-Assessment of Teaching Effectiveness Compared to the pre-test, post-test sores were higher for ten of the eleven teaching effectiveness items in resident self-assessment surveys (Table 3).However, only one item showed a statistically significant increase: "In clinical teaching, I generally evaluate learners' ability to apply knowledge" (p = .01).Intervention: Attitudes toward Teaching and Satisfaction with Seminar None of the post-intervention questionnaire ratings for residents' attitudes toward teaching showed a statistically significant change from pre-intervention (Table 4).All residents rated the seminar intervention as useful to their practice, with a Likert scale rating of four (20%, 2/10) or five (80%, 8/10 residents) on the 5-point scale.

Discussion
development and evaluation of resident-as-teacher (RaT) curricula for trainees in their subspecialty or fellowship years.To address this gap in the literature, we first conducted a survey-based, teachingfocused needs assessment of subspecialty geriatric medicine residents and geriatric medicine faculty members.We then implemented and evaluated a three-hour, RaT training program based on the Stanford Faculty Development Program (Skeff et al., 1992).Needs Assessment The resident and faculty surveys had response rates of 68% and 59% respectively.These rates are generally consistent with the published mean response rate among surveys in medical journals (Asch, Jedrziewski, & Christakis, 1997).Although non-response rates of about 30-40%, such as these, do reduce the sample size and limit the strength of conclusions, overall survey results supported a need for formal teaching instruction among subspecialty geriatric medicine residents.Residents reported limited knowledge of core teaching principles, including evaluation and writing learning objectives, and had limited exposure to formal teaching instruction.Although residents received regular feedback on their teaching, most had not been taught how to teach in a clinical setting.In addition, a large number of faculty members reported that they did not have prepared sessions on teaching residents how to teach.Residents were experienced in teaching at the bedside but had infrequently taught large groups of trainees.Despite gaps in knowledge and experience, residents were generally comfortable teaching more junior trainees and expressed strong interest in improving their teaching skills.Residents preferred to learn about teaching through mentorship pairing, lectures/workshops led by experts, and small-group interactive discussions.Faculty members also indicated that interactive workshops would be the curriculum addition most likely to enhance residents' teaching skills.Intervention: Videotaped Assessment of Teaching Effectiveness Our focus for this initial study based on scope and available population sample, was to impact geriatric medicine resident attitudes, knowledge and skills (Kirkpatrick Level 2A, 2B) (Hill et al., 2009; Table 1).Based on the needs assessment results, we expected residents would respond positively to our workshopbased teaching curriculum.However, the blinded videotape assessor ratings of resident teaching preand post-curriculum implementation were non-significantly different in all of the seven teaching categories.Moreover, there was significant variability in agreement between the two raters.This may have been due to a lack of standardized training in using the evaluation tool in addition to limitations of study design.The main limitations of this study were small population size and the lack of experimental design.Unfortunately, small numbers are currently a limitation of any individual postgraduate Geriatric Medicine program in Canada.Therefore, future investigators may consider combining residents from all postgraduate Geriatric Medicine programs within a particular region or including more than one type of subspecialty program, to increase sample size and, thereby, significance of results.Given the single-group, pre-test post-test design (i.e., no control group), it was difficult to establish a causal relationship between exposure to the intervention and the observed outcomes.Even if postintervention scores were significantly higher, they may still only reflect only natural growth in skills as residents became more experienced teachers over time.It is also possible that residents were exposed to other sources of teaching instruction during the study time frame which could have confounded residents' scores at post-intervention.Completing the pre-intervention survey may have also sensitized residents to the purpose of the study and influenced post-intervention survey responses.Future research on RaT curricula would benefit from a controlled design to help establish a cause-and-effect  1) are difficult to interpret based on our study results.Their self-reported teaching effectiveness and attitudes toward teaching post-intervention were generally higher, although only one of the 17 paired t-tests conducted was statistically significant.This result that could be attributed to chance given the 95% confidence level.
Beyond the limitations of study design and sample size, the lack of change in attitude may be explained by several other factors.First, it may be unrealistic to expect marked changes in teaching effectiveness following a three-hour seminar.Residents may require a longitudinal curriculum in order to experience significant benefit from a training program.However, allocation of resources for this considerable undertaking would need to be weighed against potential gains, particularly in postgraduate programs such as Geriatric Medicine, in which resources are limited by relatively small numbers of trainees.Interpretation of post-intervention survey results for teaching attitudes also should take into consideration that the pre-intervention scores were quite high and thus ceiling effects may have prevented significant gains.Finally, our seminar was designed for and attended by geriatric medicine residents, which may limit the generalizability of results to residents in other subspecialty programs.The seminar, however, was largely focused on the SFDP teaching principles which have been used for and found relevant to, other medical specialties (Litzelman, et al., 1998;Johansson, Skeff, & Stratos, 2009).Despite the limitations of our study, the results were encouraging toward further development of an RaT curriculum for subspecialty residents.A relatively brief seminar providing a framework of teaching skills was received positively among our geriatric medicine residents who universally identified the seminar as "useful to their practice."Residents at this last formative stage prior to transitioning to staff positions with teaching responsibilities expressed a need to have a structured approach to improve their skills.The literature has shown that junior trainees can certainly benefit from learning from better resident teachers both in terms of knowledge gained and overall satisfaction with any given rotation (Snell 2011;Huynh et al., 2011).Moreover, faculty members expressed a need to improve their own skills and have the potential to play a more active role in teaching residents how to teach by participating in RaT seminars.If our RaT seminar were to be used as a model for further curriculum building, further consideration should be given to duration of training required to achieve desired learning outcomes and ways to optimize reliability and validity of evaluators' assessments.
Our research was an initial study that identified a need for, and interest in, formal teaching instruction among subspecialty residents.Exposure to a brief teaching seminar based on the SFDP teaching principles can have a positive outcome on resident attitudes toward teaching; however, future study would benefit from an experimental design and a larger sample population in order to broaden applicability and interpretation of results.
There is a clear need for formal teaching instruction among subspecialty geriatric medicine  1 For faculty, "evaluation and feedback" were listed as one item on the survey rather than as two separate categories.
Table 3 description.Geriatrics residents teaching (videotaped samples) scores determined by two clinician teachers, before and after the intervention.For each of the 25 items on the evaluation tool, the residents' average score was calculated from the two clinician teachers' individual scores and the item scores were then averaged for each of the seven SFDP categories.Paired t-tests were used to compare the average pre-test and post-test scores.

D
'Silva K, Gakhal N, Gandell D MedEdPublish https://doi.org/10.15694/mep.2016instructionalseminar on teaching can have positive outcomes for residents.Future study would benefit from an experimental design and a larger sample population in order to broaden applicability and interpretation of results.Dr. D'Silva is a Lecturer in the Department of Medicine at the University of Toronto and full-time clinician teacher in the Division of Geriatric Medicine, St. Michael's Hospital, Toronto, Ontario, Canada.Dr. Gakhal is a Lecturer in the Department of Medicine at the University of Toronto and full-time clinician in quality and innovation in the Division of Rheumatology, Women's College Hospital, Toronto, Ontario, Canada.Dr. Gandell is an Assistant Professor in the Department of Medicine at the University of Toronto and full-time clinician teacher in the Division of Geriatric Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada.MedEdPublish https://doi. org/10.15694/mep.2016.000004