Research article
Open Access

Adapting online education resources to meet the needs of an emergency medicine training programme in a pilot feasibility study

Sierra Beck[1], Emma Carlin[2], Kristen Grabow Moore[3], Megan Anakin[1][a]

Institution: 1. University of Otago, 2. Southern District Health Board, 3. Emory University School of Medicine
Corresponding Author: Dr Megan Anakin ([email protected])
Categories: Educational Strategies, Postgraduate (including Speciality Training), Curriculum Evaluation/Quality Assurance/Accreditation, Undergraduate/Graduate
Published Date: 26/03/2021

Abstract

Introduction: Online resources are available to enhance emergency medicine training programmes. The aim of this study was to evaluate the feasibility of using online case-based resources created in the United States in a New Zealand emergency medicine training programme.

Methods: Evaluation data were collected from junior doctors and educators after they participated in the programme. Data sources included research notes and questionnaire responses. The data were analysed qualitatively using a general inductive approach.

Results: Evaluation feedback from 19 junior doctors and 14 educators was interpreted to suggest that the online resource, with minor adaptations, was feasible to use in a New Zealand emergency medicine training programme. Findings indicated that educators were able to modify the materials to modify to meet local requirements, however, the opportunity to include a cultural component was missed. Participants appreciated the case-based format and felt that they established a safe and encouraging learning environment with each other. Participants were able to develop a systematic approach to emergency situations and identify red flags related to deteriorating patients.

Discussion and Conclusion: Evaluation findings indicate that adapting an online-sourced curriculum is feasible to educators and acceptable to junior doctors and educators participating in a single New Zealand based Emergency Medicine training program. Educators in other international training settings may find the lessons learned helpful when adapting online resources to address the learning needs of their junior doctors. Next steps are to evaluate the impact of this resource on the knowledge and skills learned by junior doctors and any changes in their care for patients in emergency medicine situations.

Keywords: online resources; curriculum; emergency medicine; graduate medical education; feasibility study

Introduction

Emergency medicine training programs are employing a variety of learning strategies beyond traditional lecture-based delivery of education to enhance the engagement of the trainees (Brazil, 2014; Tan, Brainard and Larkin, 2015). Case-based learning is one learning strategy that requires learners to recall previously covered material and use it to solve cases based on clinical practice (Williams, 2005). This strategy provides opportunities for teamwork, hypothesis generation, as well as testing and self-evaluation of prior knowledge. Case-based learning is generally acceptable to learners and their instructors, in a variety of settings; however, it can be instructor and resource intensive (Srinivasan et al., 2007; Garvey, O'Sullivan and Blake, 2000). Instructors may have limited resources for curriculum development, and consequently, they may find it challenging to develop and evaluate resources that meet the needs of a diverse group of postgraduate learners. One way to address this challenge comes from the Free and Open Access Medical Education (FOAM) movement (Nickson and Cadogan, 2014). The FOAM movement has produced a wealth of online educational resources which instructors can use to augment local training programs. These resources can be freely used with acknowledgement. These resources appear to be acceptable and useful according to junior doctors and their educators, but how to adapt resources to local learning needs is yet to be explored (Goldman, Barthman and Merritt, 2019). To explore this need, the aim of this study was to evaluate the feasibility of using online case-based resources created in the United States in a New Zealand emergency medicine training programme.

Methods

An evaluation approach was used in this study (Creswell and Creswell, 2018). A survey-based study design was approved by departmental review and ethical approval was granted by the University of Otago Human Ethics Committee (D18/280). The implications of this study for Māori were discussed with the Ngāi Tahu Research Consultation Committee. Locality authorization was obtained from the Southern District Health Board to conduct this study from September to December 2018 at the Dunedin Public Hospital, Dunedin, New Zealand.

 

Participants in this study included junior doctors and their educators. The junior doctors were registrars employed in the Emergency Department at the Dunedin Public Hospital. We use the term junior doctor to refer to someone who has completed their undergraduate medical training such as an MB ChB degree and who has not yet enrolled in a postgraduate training programme. Educators who participated in this study included registrars in the advanced stages of the Fellowship Australasian College of Emergency Medicine (FACEM) training programme and senior emergency medicine doctors who had completed FACEM training and were employed by the Dunedin Public Hospital and/or the University of Otago.

 

FOAM resource used was the Foundations of Emergency Medicine (FoEM) curriculum (https://foundationsem.com/). FoEM is a free, open-access, case-based, and is used to supplement the American Board of Emergency Medicine curriculum that is used in some emergency medicine training programs in the United States (Goldman, Barthman and Merritt, 2019). FoEM resources are designed to be used in a flipped classroom manner and targeted to early stage trainees within a postgraduate system. FoEM resources provide a systems‐based review of fundamental knowledge and core diagnoses for emergency medicine. The case-based content was reviewed by instructors prior to delivery to ensure consistency with Dunedin Public Hospital practice such as medication availability, common referral pathways, and laboratory units. During weekly one-hour sessions, junior doctors worked in small groups rotating though three case discussions facilitated by an educator. Notes were kept by the research team about how the educators used and modified the FoEM resources.

 

Junior doctors were invited to participate when they attended their first structured education session that included the FoEM resources in September 2018. If junior doctors declined to participate in the study, they were informed that there would be no difference in their training. They completed a questionnaire after participating in the programme for three months. Educators were invited to participate before they facilitated a case discussion session during the study period. They completed the educator version of the questionnaire after they facilitated one or more session in the programme. Questionnaire items are shown in Table 1. All participants gave written informed consent prior to participating in the study.

 

Table 1. Questionnaire Instructions and Items Provided to Participants.

Instructions: Please write brief responses to the following questions/statements.

For junior doctors

  • What aspects of decision-making do you think you learned during the case discussions?
  • Please suggest an improvement for your teacher to consider before their next case discussion with you.
  • Did you prefer these interactive cases or would you prefer to return to lecture based curriculum?
  • Any other comments?

For educators

  • What aspects of decision-making did you impart to learners during the case discussions?
  • How did you help learners get the most of out of these case-based discussions?
  • Do you think trainees learned medical decision making better from lectures or interactive cases? Please explain why?
  • Any other comments?

 

A general inductive approach (Thomas, 2006) was used to analyse two data sets separately. First, research notes were analysed to identify how the educators used and modified the FoEM resources. Second, questionnaire responses were analysed to identify how all participants described their reactions of the FoEM resources in the training programme. The goal of the coding procedure was to identify a limited number of curriculum features in common in each set of data. Coding was derived from the words or phrases written in the research notes and reflection responses. Only one code was applied to each note or response. Similar codes were grouped until a limited number of themes were identified to represent the curriculum features and reactions in each data set.

Results/Analysis

Twenty-nine staff members participated in this study. In general, the 15 participating junior doctors were younger, held more junior job types, and had less training than the educators than the 14 educators. More females volunteered to participate (17 female, 12 male) and no participant self-identified their ethnicity as Māori or Pacific Island (15 New Zealand European, 4 Asian, 10 not New Zealand European, Māori, Pacific Island, or Asian).

Findings from the research notes indicated that educators made six modifications to the FoEM curriculum to adapt it to the local context. First, since the FoEM curriculum was created for a postgraduate training programme in the United States, each case required approximately 15 minutes for an educator to review it prior to a session to ensure local relevance and appropriateness. For example, a case on Rocky Mountain Spotted Fever was skipped entirely. Second, laboratory values such as glucose and creatinine required conversion to standard international units. Third, terminology such as ‘primary care provider (PCP)’ required translation to ‘general practitioner’ to be familiar to doctors in New Zealand. Fourth, pharmaceutical recommendations, including antibiotics usage, were reviewed to ensure medications were appropriate to the local context. Fifth, practice differences between our local hospital and the plan of care outlined in the cases were identified and addressed by incorporating discussions of local practice variation. Sixth, there were no Māori or Pacific Island patients in the FoEM cases and this omission was not addressed by the educators.

 

Findings from the questionnaire responses indicated that educators and junior doctors’ reactions to the modified FoEM curriculum were positive and could be characterised by four themes shown in Table 2. First, participants expressed that the case-based format was acceptable. Second, participants noted the importance of developing a systematic approach to patient care in an emergency medicine context. Third, they noted that learning to identify ‘red flags’ or signs that must not be missed so that they could manage deteriorating patients was also an important feature of the curriculum. Fourth, participants expressed that a safe and challenging learning environment encouraged them to ask and answer questions and be willing to make and accept mistakes.

 

Table 2. Curriculum Features Identified by Participants (n = 29) and Supported with Representative Reflective Responses.

Curriculum Feature

Representative Reflective Response

Case-Based Format

Prefer interactive (P11JD)

Interactive cases, they typically fall asleep in lectures (P26E)

Developing a Systematic Approach

Systematic approach even when uncertain (P16JD)

Prioritisation, systematic approach, when to seek help (P17E)

Identifying Red Flags and Managing Deteriorating Patients

Interventions – timing, acute life threatening care (P13JD)

Prioritisation and recognition of red flags, and how this can influence decisions (P29E)

Encourage a Safe and Challenging Learning Environment

All the registrars and house officers to extract the history rather than [educators] telling us (P10JD)

Encourage interaction in nonthreatening small groups, encourage all to take part (P28E)

Note. Abbreviations used: P = participant, JD = junior doctor, E = educators.

Discussion

Adapting internationally sourced FOAM resources for use in a New Zealand emergency medicine training programme appears to be feasible. Adapting the FoEM resources required several modifications that should be considered by others interested in using internationally sourced FOAM resources. The modifications included skipping cases that are not relevant to the local health care context, converting laboratory values to standard international units, translating terminology, review pharmaceutical recommendations, researching and discussing patient management to align with local hospital practices. While these differences may be viewed as a drawback, educators used them as opportunities to further explore factors that may impact a doctor’s ability to prioritise and to make decisions effectively, efficiently, and safely. One element noticed but not addressed by educators in this study was the omission of cultural competency and cultural representation because there were no Māori or Pacific Island patients in the FoEM cases. The case-based learning format may be well suited to educational goals set out in Manaaki Mana (Australasian Association for Emergency Medicine, 2019) and to address dimensions health valued by Pacific Island peoples (Pulotu-Endermann, 2001). Incorporating a cultural perspective should be part of the modifications of the FoEM resources for future sessions to ensure the health equity needs of Māori and Pacific Island patients are met by doctors in the FACEM training program.

 

A curriculum feature identified by all participants involved developing a systematic approach that includes attending to red flags when managing a deteriorating patient. This finding underscores the importance of learning prioritisation and decision-making skills in local training programmes to standards established by the Australasian Association for Emergency Medicine (2015; 2020). Another curriculum feature appreciated by all participants was the need to establish a safe and challenging learning environment. This finding is well-supported by evidence and pedagogy developed to maintain psychological safety while encouraging learners to set challenging goals for themselves (Dumont, Istance and Benavides, 2010).

 

Since a focus of this study was to identify challenges faced by educators when implementing an external training curriculum, the design of this study focused the reactions of the participants. This focus on participant reactions resulted in limiting the data collection and evidence interpretion to the lowest level of Kirkpatrick’s framework for programme evaluation (Praslova, 2010). Next steps will be to design a study that gathers higher quality data to evaluate program outcomes such as a change in knowledge, skills, behavior, and impact on clinical outcomes.

Conclusion

This study’s findings were interpreted to suggest that adapting an online sourced curriculum to an international training environment was feasible and acceptable to participating junior doctors and educators in a single New Zealand based emergency medicine training program. Educators were able to modify the online resources to meet local requirements, however, the opportunity to include a cultural component was missed. Features of the curriculum appreciated by junior doctors and educators in this training programme included the case-based format, the opportunity to develop a systematic approach that includes attending to red flags when managing a deteriorating patient, and the need to establish a safe and challenging learning environment.

Take Home Messages

  • Online resources are available to enhance emergency medicine training programmes but they may need to be adapted to meet local needs.
  • Modifications to online resources were feasible and acceptable to junior doctors and educators in a New Zealand emergency medicine training context.
  • When adapting online resources to meet local expectations, educators in other contexts may need to skip cases that are not relevant to the local health care context, convert laboratory values to standard international units, translate terminology, review pharmaceutical recommendations, and research patient management so it aligns with local hospital practices.

Notes On Contributors

Sierra Beck, FACEM, Senior Lecturer, Department of Medicine, Otago Medical School – Dunedin Campus, University of Otago, Dunedin, New Zealand.

 

Emma Carlin, Registrar, Dunedin Hospital, Southern District Health Board, Dunedin, New Zealand.

 

Kristen Grabow Moore, MD, MEd, Assistant Professor, Emory University School of Medicine, Atlanta, GA, USA.

 

Dr Megan Anakin, BSc (McGill), BFA (NSCAD), BEd (Dalhousie), MEd (Simon Fraser), PhD (Otago), AFANZAHPE, Lecturer and Education Adviser, Education Unit, Otago Medical School – Dunedin Campus, University of Otago, Dunedin, New Zealand. ORCID ID: https://orcid.org/0000-0002-6499-7802

Acknowledgements

Editorial assistance was received from Rosemary Stamm, Department of Medicine, University of Otago, New Zealand.

Bibliography/References

Australasian Association for Emergency Medicine. (2015) Curriculum Framework AFF440. Melbourne, Australia: Australasian Association for Emergency Medicine. Available at: https://acem.org.au/getmedia/fae9de05-e9c2-40d3-bee9-be1e66b3b84f/ACF440_0-5.aspx (Accessed: 6 December 2020).

 

Australasian Association for Emergency Medicine. (2019) Te Rautaki Manaaki Mana Excellence in Emergency Care for Māori May 2019-April 2022. Melbourne, Australia: Australasian Association for Emergency Medicine. Available at: https://acem.org.au/getmedia/e26f63e8-09e8-4bc4-be74-a3cea69d1131/Manaaki-Mana-strategy150419_FINAL (Accessed: 6 December 2020).

 

Australasian Association for Emergency Medicine. (2020) FACEM Training Program Site Accreditation AC549. Melbourne, Australia: Australasian Association for Emergency Medicine. Available at: https://acem.org.au/getmedia/003f59d2-b5c9-433f-9ff5-bb0843b2f950/AC549_1-6.aspx (Accessed: 6 December 2020).

 

Brazil, V. (2014) ‘Past and future of emergency medicine education and training’, Emergency Medicine Australasia. 26(1), pp. 69-71. https://doi.org/10.1111/1742-6723.12188

 

Creswell, J. W. and Creswell, J. D. (2018) Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. 5th edn. Thousand Oaks, CA: Sage.

 

Dumont, H., Istance, D., and Benavides, F. (Eds.) (2010) The Nature of learning: Using research to inspire practice. Paris, France: OECD. Available at: https://dx.doi.org/10.1787/9789264086487-en (Accessed: 6 December 2020).

 

Garvey, M. T., O'Sullivan, M. and Blake, M. (2000) ‘Multidisciplinary case-based learning for undergraduate students’, European Journal of Dental Education. 4(4), pp. 165-168. https://doi.org/10.1034/j.1600-0579.2000.040404.x

 

Goldman, S., Barthman, M. and Merritt, R. (2019) ‘Reversing the script: Peer-based teaching of foundational concepts in emergency medicine using a FOAMed curriculum’, Western Journal of Emergency Medicine. 20(4.1), pp. S23. https://escholarship.org/uc/item/3158300c

 

Nickson, C. P. and Cadogan, M. D. (2014) ‘Free open access medical education (FOAM) for the emergency physician’, Emergency Medicine Australasia. 26(1), pp. 76-83. https://doi.org/10.1111/1742-6723.12191

 

Praslova, L. (2010) ‘Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in higher education’, Educational Assessment, Evaluation and Accountability. 22(3), pp. 215-225. https://doi.org/10.1007/s11092-010-9098-7

 

Pulotu-Endermann, F. K. (2001). The Fonofale model of health: A Pacific Island model for health promotion. Wellington, New Zealand. Retrieved from: https://whanauoraresearch.co.nz/ (Accessed: 6 December 2020).

 

Srinivasan, M., Wilkes, M., Stevenson, F., Nguyen, T. et al. (2007) ‘Comparing problem-based learning with case-based learning: effects of a major curricular shift at two institutions’, Academic Medicine. 82(1), pp. 74-82. https://doi.org/10.1097/01.ACM.0000249963.93776.aa

 

Tan, E., Brainard, A. and Larkin, G. L. (2015) ‘Acceptability of the flipped classroom approach for in-house teaching in emergency medicine’, Emergency Medicine Australasia. 27(5), pp. 453-459. https://doi.org/10.1111/1742-6723.12454

 

Thomas, D. A. (2006) ‘A general inductive approach for analyzing qualitative evaluation data’, Americal Journal of Evaluation. 27(2), pp.237-246. https://doi.org/10.1177/1098214005283748

 

Williams, B. (2005) ‘Case based learning--a review of the literature: is there scope for this educational paradigm in prehospital education?’, Emergency Medicine Journal. 22(8), pp. 577-581. http://dx.doi.org/10.1136/emj.2004.022707

Appendices

None.

Declarations

There are some conflicts of interest:
For transparency, Megan Anakin is a member of the MedEdPublish Editorial Board.
This has been published under Creative Commons "CC BY-SA 4.0" (https://creativecommons.org/licenses/by-sa/4.0/)

Ethics Statement

Ethical approval was granted by the University of Otago Human Ethics Committee (D18/280).

External Funding

This article has not had any External Funding

Reviews

Please Login or Register an Account before submitting a Review

Barbara Jennings - (26/03/2021) Panel Member Icon
/
Thank you to the authors for this well written paper about participant (educator and trainee) reaction to an adapted online curriculum for emergency medicine training. It raises the important issue of free & open access medical education (FOAM) and is a timely reminder that not all teaching materials have to be created from scratch, there is a wealth of excellent resources that can be shared and reused. Although I am in a different field, it strikes me from #mededtwitter that emergency medicine is a leader in FOAM.
There are many important points to recommend the structure and content of the paper: a title and abstract that succinctly and accurately describe the study and its findings; the research governance including the REC approval details and the explanation of consenting in the methods section; the explicit description of the research method used. In the results section, curriculum modifications were noted and challenges e.g. full consideration of cultural competence as well as the benefits of a case based and clinically systematic curriculum.
The article is very clearly written with a logical flow of information between the methods-results- discussion and conclusion, so I only had one unanswered question by the time I had finished reading it. This was about the items selected from the Foundations of Emergency Medicine (FoEM) curriculum (https://foundationsem.com/) – do the authors think it could be helpful to readers if they itemise the particular resources selected? Or perhaps the resources are best matched to the host needs (e.g. ensuring consistency with Dunedin Public Hospital practice in this instance) and so a precise list may be unlikely to be generalisable to other centres?

Possible Conflict of Interest:

I am an Associate Editor of MedEdPublish. I have posted this review as a member of the review panel and so this review represents a personal, not institutional, opinion.