Research article
Open Access

Uncovering Hidden Scholar Feedback with Field Notes

Gill Kazevman[1], Jessica L. Marshall[2], Ben Shachar[1], Morgan Slater[3], Fok-Han Leung[4], Charlie B. Guiang[4]

Institution: 1. University of Toronto - Faculty of Medicine, 2. Univeristy of Toronto, 3. Health Services and Policy Research Institute, Queen’s University., 4. Department of Family and Community Medicine, University of Toronto
Corresponding Author: Mr. Gill Kazevman ([email protected])
Categories: Assessment, Learning Outcomes/Competency, Scholarship/Publishing, Postgraduate (including Speciality Training), Research in Health Professions Education
Published Date: 10/06/2021

Abstract

Background

The CanMEDS Scholar role remains one of the most neglected and difficult to assess roles in medical education literature. Our study analyzes 'field notes' (FNs) - frequent formative written feedback in postgraduate family medicine, and examines whether the Scholar role feedback can be found within the feedback for the Medical Expert role.

 

Methods

A deductive analysis was conducted on FNs generated by resident preceptors in the Department of Family and Community Medicine (DFCM) at the University of Toronto (UofT) between April 2017 and July 2017. We focused on Medical Expert role feedback. Scholarly roles derived from the CanMEDS Scholar role description were used to identify Scholar role feedback in these FNs.

 

Results

1809 FNs providing feedback to the Medical expert role were analyzed. Of these, 7.6% definitely described scholarly feedback. Two scholarly roles were highlighted - Evidence-based decision-maker and Lifelong learner.

 

Discussion

This study highlights aspects of the Scholar role that can overlap with Medical expert role feedback. To address this overlap, feedback tools should allow learners to receive comprehensive feedback in their learning encounters. Moreover, more attention should be given to better prepare preceptors to evaluate and categorize these roles in the clinical setting.

 

Keywords: CanMEDS; Scholar; Medical Expert; Family Medicine; Field Notes; Medical Education

Introduction

CanMEDS is a competency-based medical education framework that was formally adopted in Canada in 1996 and implemented worldwide. CanMEDS is composed of seven distinct roles, each imperative to physician training. These roles include – Communicator, Leader, Professional, Collaborator, Health advocate, Scholar and the encompassing role Medical expert (Frank, Snell and Sherbino, 2015). The CanMEDS framework is used for evaluating and providing feedback for medical learners, through assessment of each individual role within the academic and clinical settings (CanMEDS, 2000). At the Family Medicine (FM) residency program at the University of Toronto (UofT), postgraduate learners are evaluated using an assessment tool called “Field notes”. When used appropriately, comments in the form of field notes (FN) can provide necessary feedback for learner reflection and growth (Donoff, 2009).

 

One of the CanMEDS roles that has been recently described as “neglected” is the Scholar role (Ologunde, Di Salvo and Khajuria, 2014). CanMEDS states that as scholars, “physicians must demonstrate a lifelong commitment to excellence in practice through continuous learning and by teaching others, evaluating evidence, and contributing to scholarship’ (The Royal College of Physicians and Surgeons of Canada, 2020b). CanMEDS further describes that as a ‘scholar’, a physician must uphold four key scholarly roles: Lifelong learner, Evidence-informed decision-maker, Teacher, and Researcher (The Royal College of Physicians and Surgeons of Canada, 2020b). While two scholarly roles, the teacher and the researcher, have been reported as easy to assess, assessment of the other roles has proven more challenging (Kassam et al., 2013). The inability to assess and provide feedback to learners pertaining to certain elements of the scholar role may be the explanation why many residents view this role as one of the least important ones (Ringsted et al., 2006).

 

As the ‘Medical expert’ role has been referred to as the “central and integrative role” (Whitehead, Austin and Hodges, 2011), we hypothesized that feedback relating to the other CanMEDS roles may be found in feedback evaluating the ‘Medical expert’ role. In this study we aim to discover whether or not learners, in receiving feedback on the ‘Medical Expert’ role in their field notes, are also receiving feedback on the intrinsic ‘Scholar’ role at the same time. Identifying and understanding the extent of the overlap of these two roles in field note assessments can inform educators on ways to enhance feedback on the Scholar role.

Methods

We extracted de-identified data from encounter-based FN written by multiple clinical preceptors from 14 different training sites affiliated with the UofT Department of Family and Community Medicine between April-July 2017. Four reviewers with different levels of medical education (GK, FL, BS, and JM) each independently coded 25% of the field notes. Following organized mixed-method approaches (Lockyer et al., 2018), each FN was coded into one of three categories - Definitely scholar, Somewhat scholar, or Not scholar. Code designation relating the relevance of the content of a free text comment to the scholar role was accomplished by referring to the CanMEDS Scholar role description as a guide. “Definitely scholar” notes were those easily related to one of four key scholarly roles, and the scholarly feedback was the focus of the comment. These notes were further mapped onto one of the four key scholarly roles - Lifelong Learner (LL), Evidence-based Decision-maker (EBDM), Teacher and Researcher. “Somewhat scholar” notes were identified as relating to 1 of the 4 key scholarly roles, but had a stronger relationship to other CanMEDS roles. “Not scholar” notes were those identified as not relating to any of the scholarly roles. Following completion of the coding process, all coders reviewed the code assignment and disagreement was resolved through consensus. Finally, internal reliability was tested with one of the initial reviewers (JM) through a code agreement test of a random subset 10% of comments. The number of definitely scholar identified FNs mapped onto each of the four scholarly roles was tabulated. The data were analyzed using Dedoose 8.0.35 web application (Los Angeles, CA). Approval for this study was obtained from the UofT Research Ethics Board (REB). 

Results/Analysis

A total of 5224 field notes were written of which 3.0% (N=155) were identified by preceptors as ‘Scholar’ role feedback and 34.6% (N=1809) of the field notes written were identified by preceptors as ‘Medical expert’ feedback. During the analysis, we found that 7.6% (N=138) of all ‘Medical expert’ recorded feedback pertained to the ‘Scholar’ role, identified by the code “Definitely Scholar”. While the data analysis generated 3 groups of codes (Definitely scholar, Somewhat scholar and Not scholar), we used only “Definitely Scholar” comments as they truly represent feedback related to the scholar role.

 

Evidence-based Decision-maker

EBDM was the most identified key scholar role in the free text comments, seen in just over half of the “Definitely scholar” ‘Medical expert’ field notes. This role encompasses how well learners are able to identify and consolidate the best quality evidence into their practice. Frequent topics mentioned were proper utilization of common diagnostic resources and the ability to in prior knowledge to support decision making. One occasion this role was identified can be seen in the feedback of a preceptor to a resident that:

“Although [the learner] had some knowledge limitations given the unique scenario, knowledge was appropriate for level of training and knew how to access resource to get information”

 

Lifelong Learner

This was the second most common scholar role identified in the examined field notes. This role entails feedback describing learners’ reflection of their personal practice after a patient interaction and for times when learning plans were made to improve knowledge in the future. One example of reflection and learning plan creation can be seen in the following feedback:

“wrong immunization given (adacel-polio rather than adacel); please review adult immunization schedules as per TPH resources/guidelines”

Many of the comments pertaining to the LL role include specific resources that should be learnt by the resident. This can be seen in one preceptor’s feedback:

 “Suggested atrovent treatment for asthma in a child. Review pediatric asthma guidelines re diagnosis/ treatment (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2831604/)”

 

Teacher

Only two comments were related to the ‘Teacher’ role, which pertains to times when a learner educated a medical student, or other allied healthcare professionals. One example is seen in a preceptor’s feedback on a resident’s procedure:

“Excellent teaching- bringing a medical student to tongue tie procedure on a newborn with this condition…. Competent to do procedure on own next time”

 

Researcher

This role describes times when learners were involved in the creation or distribution of novel research. None of the free text comments in our analysis fulfilled this description.

Discussion

Only 3% of field notes written by FM preceptors were designated as feedback regarding the ‘Scholar’ role, showing that feedback related to this role is uncommon. However, an almost equivalent amount of feedback pertaining to the ‘Scholar’ role was found to be hidden within the ‘Medical expert’ labeled feedback. In particular, while feedback for the Researcher and Teacher key scholarly roles was rarely found in our analysis of the ‘Medical expert’ labeled FNs, the EBDM and LL key roles appear to be more commonly found in these evaluations.

 

The presence of EBDM comments in the ‘Medical expert’ feedback can be easily explained when examining the description for the ‘Medical expert’ role. CanMEDS describes that among other expectations, a medical expert must “Apply knowledge of the clinical and biomedical sciences relevant to their discipline” (The Royal College of Physicians and Surgeons of Canada, 2020a). This description is very similar to the ‘Scholar’ role’s CanMEDS description stating that as scholars, physicians must “Integrate best available evidence into practice” (The Royal College of Physicians and Surgeons of Canada, 2020b). This overlap, together with the perceived importance of one role over the other, may be the reason that ‘Scholar’ role feedback is found within the ‘Medical expert’ feedback. On the other hand, the presence of LL feedback within the ‘Medical expert’ comments is harder to explain. Lifelong learning has been proven to be easily taught and evaluated within learning environments, such as with problem-based learning exercises often used in medical school education (Berkhout et al., 2017). Yet, it has been highlighted that teaching and evaluating this scholarly role within the clinical setting is much more difficult (Bammeke et al., 2015). In the academic context learners focus on learning, whereas in the clinical context, learners are focused on providing patient care which supports their learning process. Therefore, we believe that feedback for this key scholarly role will always be intertwined with medical expert feedback, as advice and guidance will arise after providing care for a patient.

 

In recent years, several publications have pointed out the importance of the existence of physician scholars (Bammeke et al., 2015; Puddester et al., 2015), and the growing concern among physicians for the fate of the scholar role (Ologunde, Di Salvo and Khajuria, 2014). Yet, in these publications, the term “Scholar” was often synonymous for “Researcher”. This dissonance between the perceived and the actual description of the scholar role may be the reason why this role is less visible in learners’ feedback. A study published in 2012 proffered that while the generation of new medical knowledge is an important competency, we must remember that it only represents one aspect of the scholar role (Koo et al., 2012). Our analysis suggests that in order to allow for more comprehensive scholar role feedback for learners, an assessment should be done of the tools used to evaluate performance in the scholar role. Specifically, we recommend that the way “evidence-based decision-making” and “lifelong learning” are evaluated should be addressed when modernizing the evaluation system. Lastly, we recommend adjusting evaluation tools to allow more nuance, such as those that allow preceptors to provide feedback for multiple roles witnessed in a single encounter.

Conclusion

This study highlights aspects of the Scholar role that can overlap with Medical Expert role feedback. In order to address this overlap, feedback tools should allow learners to receive comprehensive and rich feedback in their learning encounters. Moreover, more attention should be given to better prepare preceptors to evaluate and categorize these roles in the clinical setting.

Take Home Messages

  1. The Scholar role remains one of the least evaluated competencies in post graduate medical education.
  2. Preceptors percieve significant overlap between the Medical Expert role and the Scholar role.
  3. Assessing postgraduate medical learners on their abilities to meet the scholar role criterias must describe more than just their abilties as researchers and teachers.
  4. More support must be provided to preceptors in order to improve the accuracy of assesment of the Scholar role in the postgraduate medical education setting.
  5. Better tools should be used in order to accuratly describe postgraduate medical learners' abilities in meeting expectations as scholars in a postgraduate medical program. 

Notes On Contributors

Gill Kazevman is a third year medical student, University of Toronto Faculty of Medicine, Toronto, Ontario. ORCiD: https://orcid.org/0000-0002-3185-1655

 

Jessica Marshall is a recent graduate, University of Toronto, Scarborough, Ontario. ORCiD: https://orcid.org/0000-0003-4375-5721

 

Ben Shachar is a fourth year medical student, University of Toronto Faculty of Medicine, Toronto, Ontario. ORCiD: https://orcid.org/0000-0002-0354-7330

 

Morgan Slater is a Research Scientist, Health Services and Policy Research Institute, Queen’s University, Kingston, Ontario. ORCiD: https://orcid.org/0000-0003-3967-2501

 

Fok-Han Leung is an associate professor, Department of Family and Community Medicine, University of Toronto, Toronto, Ontario. ORCiD: https://orcid.org/0000-0001-8886-3625

 

Charlie Guiang is an assistant professor, Department of Family and Community Medicine, University of Toronto, Toronto, Ontario. He is also the Physician Co-Lead at The Health Centre at 410, and the Resident Academic Project Coordinator for his Department. ORCiD: https://orcid.org/0000-0003-0545-854X

Acknowledgements

None.

Bibliography/References

Bammeke, F., Liddy, C., Hogel, M., Archibald, D., et al. (2015) ‘Family medicine residents’ barriers to conducting scholarly work’, Can Fam Physician, 61(9), pp. 780-787. Available at: https://www.cfp.ca/content/61/9/780.long (Accessed: 30/08/2020).

 

Berkhout, J. J., Helmich, E., Teunissen, P. W., van der Vleuten, C. P., et al. (2017) ‘Context matters when striving to promote active and lifelong learning in medical education’, Medical Education, 52(1), pp. 34–44. https://doi.org/10.1111/medu.13463.

 

CanMEDS (2000) 'CanMEDS 2000: Extract from the CanMEDS 2000 Project Societal Needs Working Group Report', Medical Teacher, 22(6), pp. 549–554. https://doi.org/10.1080/01421590050175505.

 

Donoff, G. (2009) ‘Field notes’, Can Fam Physician, 55(12), pp. 1260-1262. Available at: https://pubmed.ncbi.nlm.nih.gov/20008606/ (Accessed: 20/08/2020).

 

Frank, R., Snell, L., Sherbino, J. (2015) ‘Physician Competency Framework’, Ottawa: Royal College of Physicians and Surgeons of Canada, (2015) pp. 1-17. Available at: http://canmeds.royalcollege.ca/en/framework (Accessed: 29/08/2020).

 

Kassam, A., Donnon, T., Cowan, M. and Todesco, J. (2013) ‘Assessing the scholar CanMEDS role in residents using critical appraisal techniques’, Canadian Medical Education Journal, 4(1). https://doi.org/10.36834/cmej.36599.

 

Koo, J, Bains, J, Collins, M. B., Dharamsi, S. (2012) ‘Residency research requirements and the CanMEDS-FM scholar role’, Can Fam Physician, 58(6), pp. 330-336. Available at: https://www.cfp.ca/content/58/6/e330.long (Accessed: 30/08/2020).

 

Lockyer, J. M., Sargeant, J., Richards, S. H., Campbell, J. L., et al. (2018) ‘Multisource Feedback and Narrative Comments: Polarity, Specificity, Actionability, and CanMEDS Roles’, Journal of Continuing Education in the Health Professions, 38(1), pp. 32–40. https://doi.org/10.1097/ceh.0000000000000183.

 

Ologunde, R., Di Salvo, I. and Khajuria, A. (2014) ‘The CanMEDS scholar: the neglected competency in tomorrows doctors’, Advances in Medical Education and Practice, p. 383. https://doi.org/10.2147/amep.s71763.

 

Puddester, D., MacDonald, C. J., Clements, D., Gaffney, J., et al. (2015) 'Designing faculty development to support the evaluation of resident competency in the intrinsic CanMEDS roles: practical outcomes of an assessment of program director needs', BMC Medical Education, 15(1). https://doi.org/10.1186/s12909-015-0375-5.

 

Ringsted, C., Hansen, T. L., Davis, D. and Scherpbier, A. (2006) ‘Are some of the challenging aspects of the CanMEDS roles valid outside Canada?’, Medical Education, 40(8), pp. 807–815. https://doi.org/10.1111/j.1365-2929.2006.02525.x.

 

The Royal College of Physicians and Surgeons of Canada (2020a) CanMEDS Role: Medical Expert. Available at: http://www.royalcollege.ca/rcsite/canmeds/framework/canmeds-role-medical-expert-e (Accessed: 02/09/2020).

 

The Royal College of Physicians and Surgeons of Canada (2020b) CanMEDS Role: Scholar. Available at: http://www.royalcollege.ca/rcsite/canmeds/framework/canmeds-role-scholar-e (Accessed: 31/08/2020).

 

Whitehead, C. R., Austin, Z. and Hodges, B. D. (2011) ‘Flower power: the armoured expert in the CanMEDS competency framework?’, Advances in Health Sciences Education, 16(5), pp. 681–694. https://doi.org/10.1007/s10459-011-9277-4.

Appendices

None.

Declarations

There are no conflicts of interest.
This has been published under Creative Commons "CC BY-SA 4.0" (https://creativecommons.org/licenses/by-sa/4.0/)

Ethics Statement

Ethical approval for this study was obtained from the Research Ethics Board (REB) at the University of Toronto, on 13/08/2019 (REB ID 37665).

External Funding

This article has not had any External Funding

Reviews

Please Login or Register an Account before submitting a Review

Juliana Sa - (12/07/2021) Panel Member Icon
/
The document brings an interesting topic that is not frequently discussed, additionally the use of field notes is also an interested source of information. The scholar role is well described; however, “other possible roles” are mentioned but are not explore what they could be, the use of field notes could be useful to explore this approach. Methodology could be more explicit about how the mix-methods were used and the importance of several observers.
Ben Canny - (30/06/2021) Panel Member Icon
/
This manuscript emphasises the observations that assessment of the Scholar role within the CanMEDS framework remains elusive. While the paper draws some observations regarding the ways in which this can be done, this reader was left a little uncertain about whether assessment of the role was important, and how assessment could be enhanced or changed. It is noted that increased feedback regarding the Scholar role can be hidden within other roles, but even with the most generous collation of the data, only 5.6% of all reports (293 of 5224) make reference to the Scholar role. This forces the questions of either how important is the Scholar role, or how can we encourage both candidates and preceptors tp place more importance on the role. The article may have been enhanced by a clearer discussion of this dilemma.