Personal view or opinion piece
Open Access

Rethinking Assessment in Medical Education in the time of COVID-19

Saniya Sabzwari[1][a]

Institution: 1. The Aga Khan University
Corresponding Author: Dr Saniya Sabzwari ([email protected])
Categories: Assessment, Learning Outcomes/Competency, Undergraduate/Graduate
Published Date: 27/04/2020

Abstract

The COVID-19 pandemic has impacted the design, delivery and processes in medical education. While the crisis continues, it is important to develop creative ways of assessment that continue to maintain the standards of medical education and accommodate the present environmental and social limitations brought on by COVID. This crisis also offers an opportunity to transition programs to a competency based curriculum.

The transition should be comprehensive but staged to include current and prospective learners.  

It is important to first consider the existing framework of assessment and work backwards on design and process. As an immediate and first step would be to add multiple opportunities for formative assessment into the new approaches adopted for teaching/learning (T/L). Standard assessment formats in medical school that include written exams (like MCQs, SAQs and practical skills via OSCE) will need to be re-imagined and re-planned.   

It may now be time to consider shifting focus of assessment to attainment of competencies. This is a longer term process but can be phased in. As a start using low stakes assessment as part of entrustable professional activities that are task directed and encompass domain of knowledge, skills and attitude.

Revisiting older modalities and introducing new ways that accommodate distant engagement may need consideration for e.g. structured viva, use of log books for assessment, virtual patients, e-portfolios.

Thus an emphasis in assessment should be on using multimodal tools for formative and summative assessments that focus on mastery in clinical reasoning, problem solving and decision making skills allowing transition to a competency based curriculum.

The key task will be the timely creation and monitoring of a balanced face to face and online assessment program keeping in mind resource availability, faculty buy-in and support of regulatory and licensing bodies.

Keywords: Assessment; undergraduate program; competency based medical education

Background

As of April 16th 2020, 210 countries have been affected by COVID-19 (Worldometer). Most countries are at various trajectory points in this crisis. Some may have flattened the curve, others are in the midst and some still awaiting the full onslaught of the virus. Therefore predicting its end may be difficult. Furthermore the ongoing concerns about acquiring immunity and development of an effective vaccine lend support to this being a longer term scenario for most of the world.

 

During this COVID lockdown, medical colleges have prudently adopted alternative ways of teaching and learning(Ahmed et al., 2020). Adjustments in assessment are also evident (Boursicot et al., 2020). However a comprehensive strategy is required to gauge not just the effectiveness of these new teaching/learning strategies but to also uphold the standards of medical education.

 

As COVID may be the first among similar such pandemics (Daszak et al., 2020) we must consider this event as a reset point and an opportunity to transform medical education. While asynchronous and synchronous ways are being readily adopted for teaching/learning (T/L), can we rely on the honesty and honor code of young minds to ensure fairness and transparency using current modalities of assessment? A lot more deliberation will be needed to ensure that we develop safe physicians who are ready for a world altered by a single event.

Assessment redesign: rationale and strategies

It is important to first consider the existing framework of assessment. Formal assessment in most medical colleges is largely centered on satisfactory attainment of knowledge and skills. Students nowadays have multi-source access to knowledge in today’s digital world therefore knowledge testing should be embedded in problem solving with increasing complexity overtime (Humphrey-Murto et al., 2017). It may also be time to perform an in-depth review of the most commonly used physical exam skills by practicing physicians to allow a strategic selection and assessment of those very skills. While physical examination can never be completely replaced, COVID-19 has enhanced the role of telemedicine (Leite et al., 2020) that limits the role of physical examination. This practice of distant disease management may continue to grow even after COVID gets contained, transforming medical care for the long term. Now more than ever before, it is important to shift our focus to attaining competencies that allow future physicians to combat a rapidly changing health milieu.

 

So how should the focus of assessment shift? One hopes that the turmoil seen with COVID does not occur on the side of education and a deliberate and staged plan is developed to achieve the desired outcomes. While assessment is often the endpoint in a curriculum design, but certain immediate steps are required as COVID-19 continues.

 

The plan should include not only current but prospective learners as well. This is also an opportunity to keep the “end in mind” and gradually introduce strategies used in competency based medical education (CBME).

 

At an entry level, intake exams maybe forsaken for pre-existing college scores or high school performance and feedback based on a format similar to MSPE (Andolsek, 2016) from previous institutions.  Now more than ever may be the time to consider more uniform use of testing for traits of resilience (Howe et al., 2012), problem solving and self-efficacy. With growing risk to the mental well-being of healthcare workers such traits (Greenberg et al., 2020) become essential to ascertain in prospective physicians. Gathering evidence of being socially responsive is another domain worth exploring.

 

During the 4 or 5 years of medical school, the framework of assessment needs meticulous planning to ensure that basic principles of assessment are met. Assessments must continue to be valid, reliable and feasible(General Medical Council, 2009). Standard assessment formats in medical school that include written exams (like MCQs, SAQs and practical skills via OSCE) will need to be re-imagined.

 

Most medical schools currently use physical spaces for summative written examinations that often house large number of students. Such face to face assessments are conducted primarily to ensure integrity and security of examination. Institutions may have to consider creating larger venues that allow physical distancing or plan a greater number of venues to achieve the same goal. A recent article described the specifics of how an OSCE was conducted during the COVID crisis (Boursicot et al., 2020).

 

It may also be useful to consider strengthening processes for ongoing assessment. As a first step adding multiple opportunities for formative assessment into smaller and larger T/L activities.  These opportunities can be created remotely using institutional learning management systems wherever available, to place activities that allow formative feedback. This regular and consistent e-feedback would give students more direction and assume responsibility of their own learning journeys.

 

Introducing entrustable professional activities (EPAs) may allow (distant) monitoring of some pre-defined tasks e.g. history taking (Sohrmann et al., 2020). Overtime as competency of a pre-defined task is achieved tools like MiniCEX could be used (done either face to face or remotely) for formative, ongoing and final assessment (Mortaz Hejri et al., 2020).

 

Use of e-portfolios has thus far been limited. This can create opportunities for formative and summative assessment and reflection and greater faculty student engagement (Chertoff et al., 2016). Such portfolios can include activities like students adding videos specific tasks performed as part of formative or ongoing assessment that is seen over time.

 

Some end of module/clerkship assessment strategies can be converted to non-face to face without compromising integrity or feasibility of assessment. As an example older modalities like viva may need to be revisited (Akimov and Malin, 2020). Using structured viva formats allows it to remain a reliable tool (Ganji, 2017).

 

Another possible tool to consider is log books. Log books are currently used to monitor the spectrum of patients seen by students’ over time (Denton et al., 2007). Using a student’s own patient log could be used for assessment via creation of structured questions on the particular disease/s seen by a students to gauge knowledge, critical thinking and clinical reasoning skills across the variety of patients seen by the individual student.

 

Practice OSCEs could be introduced where examiners could observe candidates perform a series of assigned tasks of history and examination skills allowing assessment of communication and clinical reasoning skills while allowing repeated opportunities for practice and mastery.  Similarly, high fidelity mannequins could be utilized for physical examination skills and identification of abnormal findings (Tsai et al., 2003).

 

Virtual patients (VPs) have been used both for formative and summative purposes (Poulton and Balasubramaniam, 2011), with value for knowledge application across differing scenarios and clinical reasoning and as another practice tool that allows opportunities for mastery (Consorti et al., 2012) (Berman et al., 2016). The value of using VPs now becomes more apparent than ever, as a modality to assess critical reasoning and decision making skills. While lots of virtual patient programs exist, locally developed VPs that depict regional disease patterns would make learning and assessment more contextual. This could also lead to the development of virtual rounds, wards and virtual communities.

 

In summary a greater emphasis in assessment should be on using multimodal tools for low stakes and high assessments that focus on mastery in clinical reasoning, problem solving and decision making in different clinical scenarios and simultaneously allow transition to a competency based curriculum. While some of these strategies may be transient until physical distancing is required, others may become a more permanent part of the assessment program.

 

What is imperative in the design and placement of such activities is resource availability, faculty buy-in and support. This may also be an opportunity for countries with limited resources to collectively develop and share their online resources. The key task will be the timely creation and monitoring of a balanced face to face and online assessment program that is acceptable to all stakeholders including regulatory and licensing bodies.

Conclusion

The COVID-19 pandemic has necessitated the development and implementation of new T/L and assessment modalities. It also offers an opportunity to improve design, planning and implementation of an assessment program that aligns to provision of competency based medical education. 

Take Home Messages

  • Timely modification of low stakes and high stakes assessment is important during the ongoing COVID-19 pandemic
  • This time also presents an opportunity to transition to a competency based curriculum

Notes On Contributors

SANIYA SABZWARI is an Associate Professor in the Department of Family Medicine at the Aga Khan University (AKU), Karachi. She has a Masters in Health Professions' Education. She is currently the Chair of the Undergraduate Medical Education Examination and Promotion Committee at AKU and has an interest in curriculum development and assessment, program evaluation, and student/faculty engagement. ORCiD: https://orcid.org/0000-0002-1163-5277

Acknowledgements

None.

Bibliography/References

Ahmed, H., Allaf, M. and Elghazaly, H. (2020) 'COVID-19 and medical education', The Lancet Infectious Diseases. https://doi.org/10.1016/S1473-3099(20)30226-7          

Akimov, A. and Malin, M. (2020) 'When old becomes new: a case study of oral examination as an online assessment tool', Assessment & Evaluation in Higher Education, pp. 1-17. https://doi.org/10.1080/02602938.2020.1730301

 

Andolsek, K. M. (2016) 'Improving the medical student performance evaluation to facilitate resident selection', Academic Medicine, 91(11), pp. 1475-1479. https://doi.org/10.1097/ACM.0000000000001386

Berman, N. B., Durning, S. J., Fischer, M. R., Huwendiek, S., et al. (2016) 'The role for virtual patients in the future of medical education', Academic Medicine, 91(9), pp. 1217-1222. https://doi.org/10.1097/ACM.0000000000001146

Boursicot, K., Kemp, S., Ong, T. H., Wijaya, L., et al. (2020) 'Conducting a high-stakes OSCE in a COVID-19 environment', MedEdPublish, 9. https://doi.org/10.15694/mep.2020.000054.1

Chertoff, J., Wright, A., Novak, M., Fantone, J., et al. (2016) 'Status of portfolios in undergraduate medical education in the LCME accredited US medical school Status of portfolios in undergraduate medical education in the LCME accredited US medical school'. Medical Teacher, 38, 9.  https://doi.org/10.3109/0142159X.2015.1114595

 

Consorti, F., Mancuso, R., Nocioni, M. and Piccolo, A. (2012) 'Efficacy of virtual patients in medical education: a meta-analysis of randomized studies', Computers & Education, 59(3), pp. 1001-1008. https://doi.org/10.1016/j.compedu.2012.04.017

Daszak, P., Olival, K. J. and Li, H. (2020) 'A strategy to prevent future pandemics similar to the 2019-nCoV outbreak'. Biosafety and Health. https://doi.org/10.1016/j.bsheal.2020.01.003

Denton, G. D., Hoang, T., Prince, L., Moores, L. et al. (2007) 'Accuracy of medical student electronic logbook problem list entry', Teaching and Learning in Medicine, 19(4), pp. 347-351. https://doi.org/10.1080/10401330701542560

 

Ganji, K. K. (2017) 'Evaluation of Reliability in Structured Viva Voce As a Formative Assessment of Dental Students', Journal of Dental Education, 81 (5) 590-596. https://doi.org/10.21815/JDE.016.017 

General Medical Council (2009). Available at: https://www.gmc-uk.org/-/media/documents/Assessment_in_undergraduate_medical_education___guidance_0815.pdf_56439668.pdf (Accessed: 14 April 2020).

Greenberg, N., Docherty, M., Gnanapragasam, S. and Wessely, S. (2020) 'Managing mental health challenges faced by healthcare workers during covid-19 pandemic', BMJ, 368. https://doi.org/10.1136/bmj.m1211 

Howe, A., Smajdor, A. and Stöckl, A. (2012) 'Towards an understanding of resilience and its relevance to medical training', Medical Education, 46(4), pp. 349-356. https://doi.org/10.1111/j.1365-2923.2011.04188.x

Humphrey-Murto, S., Wood, T. J., Ross, S., Tavares, W., et al. (2017) 'Assessment pearls for competency-based medical education', Journal of Graduate Medical Education, 9(6), pp. 688-691. https://doi.org/10.4300/JGME-D-17-00365.1

Leite, H., Hodgkinson, I. R. and Gruber, T. (2020) 'New development: ‘Healing at a distance’—telemedicine and COVID-19', Public Money & Management, pp. 1-3. https://doi.org/10.1080/09540962.2020.1748855

 

Mortaz Hejri, S., Jalili, M., Masoomi, R., Shirazi, M., et al. (2020) 'The utility of mini-Clinical Evaluation Exercise in undergraduate and postgraduate medical education: A BEME review: BEME Guide No. 59', Medical Teacher, 42(2), pp. 125-142. https://doi.org/10.1080/0142159X.2019.1652732

 

Poulton, T. and Balasubramaniam, C. (2011) 'Virtual patients: a year of change', Medical Teacher, 33(11), pp. 933-937. https://doi.org/10.3109/0142159X.2011.613501

 

Sohrmann, M., Berendonk, C., Nendaz, M. and Bonvin, R. (2020) 'Nationwide introduction of a new competency framework for undergraduate medical curricula: a collaborative approach', Swiss Medical Weekly, 150(1516). 

Tsai, T. C., Harasym, P., Nijssen‐Jordan, C., Jennett, P. et al. (2003) 'The quality of a simulation examination using a high‐fidelity child manikin', Medical Education, 37, pp. 72-78. https://doi.org/10.1046/j.1365-2923.37.s1.3.x

Worldometer. Available at: https://www.worldometers.info/coronavirus/ (Accessed: 14 April 2020).

Appendices

None.

Declarations

There are no conflicts of interest.
This has been published under Creative Commons "CC BY-SA 4.0" (https://creativecommons.org/licenses/by-sa/4.0/)

Ethics Statement

This is a personal opinion article and does not require Ethics approval.

External Funding

This article has not had any External Funding

Reviews

Please Login or Register an Account before submitting a Review

Hosam Eldeen Gasmalla - (01/08/2020)
/
It is very important that we share our prospectives and ideas about assessment solutions in the time of COVID-19.
I think the COVID-19 pandemic will shift the practice of assessment into the formative assessment as it was mentioned several times in your article, I suggest that the emphasis on formative assessment and multiple assessments should be part of take home message.
Kulsoom Ghias - (12/05/2020)
/
An important read in the current crisis with universal appeal covering a spectrum of assessment activities that impact all med ed.
Sadia Fatima - (12/05/2020)
/
Interesting prospective and practical solutions for conducting assessments. With COVID-19 influenced online shift of T&L; it is imperative to share ideas, protocols, success and failure stories while offering online/ alternate assessment plans. Specially for assessing practical skills.
Elpida Artemiou - (04/05/2020) Panel Member Icon
/
Much appreciated the thoughts and questions raised in the manuscript on issues that all medical educators are currently facing and exploring options. It would have been quite beneficial to also share the specific changes implemented at the author's institution and how these met/or not the required objectives of their curriculum. I echo the importance of attitudes surrounding resilience, problem solving, self-efficacy and social responsibility in medical education. In lieu of emphasis on competency curricula; does such require an even greater leap in utilising virtual and augmented technologies? And how to equally translate competency assessments remotely?
Possible Conflict of Interest:

No conflict of interest

PATRICIA CURY - (04/05/2020)
/
Assessment in COVID era is a very important task to discuss. How can we evaluate competences in these students? Do we have to wait a couple of years and evaluate them again in order know if they will be good doctors in the future? I also would like to know how the author is doing in her medical school, with some examples.
Sarmishtha Ghosh - (02/05/2020)
/
COVID-19 has changed our perception of teaching-learning as well as assessment. This personal viewpoint by the author brings out important points that most of us would agree and some would disagree. But on the whole, it needs to be understood that alternative authentic methods of assessment in health professions students are to be worked out so as to maintain the integrity and standards of assessment, ensuring quality of the products.

I appreciate that the paper has pointed out such factors and hinted towards the issues.
It is timely and can be referred.
Possible Conflict of Interest:

No Conflict of Interest

James Gray - (29/04/2020)
/
This is timely in its publication given the current imperatives to deliver remote education during the Covid19 pandemic however is somewhat limited to being very much a summation of existing literature rather than offering solutions from the authors experience which may then have made this work that could offer more to the reader in search of insight and assistance regarding their own situation.
The emphasis of how crisis can drive innovation is clear in the latter part of the work but again I would have liked to see more regarding how the authors have specifically made alterations that they will continue not just stating what I am sure we would all agree that there will be changes in both clinical and academic medicine that will arise from this that will no doubt have some permanence.
An interesting opinion piece.
David Bruce - (28/04/2020) Panel Member Icon
/
In this timely opinion piece, the author considers the effects of the COVID pandemic on how medical school assessment programmes may need to change and suggests that the crisis may be an opportunity to re-think the focus of the assessments towards a competency-based design. As a reader I felt the paper was really discussing two separate issues:
• what needs to change in assessments as a result of the COVID pandemic and how this might be achieved
• the case for a move from a traditional curriculum to a competency based one and the resulting changes in the assessment programme.
In respect of the first issue, two papers referenced helpfully highlight changes directly related to the COVID pandemic. The author states that standard assessment formats will need to change – and while the paper does suggest how many of the assessments might be modified, the practicalities and difficulties of such changes are not explored. This would be an area that merits further discussion in the paper and hearing views and experience of the author would be important.
The second issue is interesting and follows the “never waste a crisis” line of thinking. The introduction of assessments for learning and the idea of a programme of multiple assessments is well established in many medical schools, and to understand the authors perspective I felt I really needed to know more about the curriculum and assessment programme that is currently operating in the authors medical school. While consideration is given to the COVID pandemic and how new assessments may need to change, I felt as a reader, that the thrust of the paper was more about the value in changing the curriculum and assessment programme.
I appreciate that how medical education responds to the COVID pandemic is a new and evolving area with little evidence to guide us forward, and feel that this paper is a helpful start to the conversation.
Trevor Gibbs - (28/04/2020) Panel Member Icon
/
An interesting and timely overview paper of assessment during the the Covid-19 pandemic. Whilst few readers would disagree with the authors points, which are very relevant, many faculty are now looking at practical applications and how thoughts are converted into actions , often within the context that they are based. Like my co-reviewers I think more could be learned from hearing how the author's school is coping within the present pandemic; what is working and what is not, can any of the assessment methods transfer and work in other contexts.
I congratulate the author however in bringing these important points to our notice.
Possible Conflict of Interest:

For transparency, I am one of the Associate Editors of MedEdPublish.

Samar Ahmed - (28/04/2020)
/
Thank you for the work. It shows a lot of effort. Since the work is basically an opinion article I would have preferred to see some reflection on the experience of the author and an in depth analysis of a case study yo justify the opinion.
Dujeepa D. Samarasekera - (27/04/2020) Panel Member Icon
/
Interesting opinion piece. Generic views and citations from literature. It would have been more useful and I would have preferred if the author gives examples from her own experience or how these guidelines could be applied in specific situations or contexts. Limitations of the tools or the guidelines specified are also missing. A summary table would have added more clarity to the manuscript.