Improving Procedural Skills, Teamwork and Confidence in just One Day: A NICU Fellow Boot Camp

This article was migrated. The article was marked as recommended. Objective: To develop a compact, simulation-based orientation session aimed at improving confidence and teamwork amongst new trainees. Methods: Two cohorts of five fellows participated in the one-day boot camp. Confidence in team leading and neonatal procedures was assessed using a pre and post survey administered on the day of boot camp as well as a delayed post-test after 6 months. Teamwork and communication were assessed using the TEAM scale. Results: We found that fellows confidence was significantly improved in 5 out of 6 categories (team leader in code (p < 0.001), team leader in delivery room (p < 0.001), decision making (p < 0.001), intubation (p < 0.001), chest tube placement (p < 0.001) and umbilical catheter placement (p = 0.09)). There was an overall improvement in teamwork, team leadership and communication throughout the day (p < 0.001). There was no significant change in self-reported confidence scores in all categories 6 months following boot camp. Conclusions: We demonstrated a significant improvement in confidence in first year neonatal-perinatal medicine fellows following a one-day simulation-based boot camp. The format for this boot camp could be emulated in institutions across the country to improve the confidence and skills of incoming fellows.


Introduction
The first month of fellowship training can be an overwhelming transition.Unfamiliarity with a new medical institution, uncertainty regarding complex medical diagnoses and inexperience with procedural and communication skills can contribute to the challenge.(Nishisakiet al., 2009;Younget al., 2011;Cohenet al., 2013;Wenet al., 2015) In the neonatal intensive care unit (NICU), incoming fellows have had increasingly limited experiences due to work hour restrictions and may be further limited by the level of acuity at their previous institution.Beyond this, the role of a fellow is different than that of a resident, with increased clinical and leadership responsibilities, further adding to the stress.An orientation experience to onboard fellows to the institution and the position may help smooth this transition and reduce their stress.
Across the country, neonatal fellowship programs do not have a standardized process for orientation leaving individual fellowships to develop an onboarding process.To help fill this need, an orientation curriculum, based on specific institutional needs assessments should be developed.A structured "boot camp" during the onboarding process could help bridge this gap while improving confidence and performance.Boot camps utilize various modes of teaching, including: simulation-based exercises, focused procedural skills (task training) sessions and resuscitation leadership opportunities in an arena similar to the patient care environment.(Blackmoreet al., 2014) High fidelity simulation-based boot camps lasting days to months are emerging throughout many procedural focused specialties as a way to teach leadership, communication and procedural skills to a group of trainees with various levels of experience and training.This type of simulation training has the potential to decrease medical errors secondary to lack of practice.(Nishisakiet al., 2009;Bismuthet al., 2012;Cohenet al., 2013;Atayaet al., 2015;Heskinet al., 2015) In addition, simulation training provides a unique opportunity for training that will not cause patient harm yet has been shown to induce stress and increase cortisol levels in neonatal fellows, similar to a real-life emergency.(Finanet al., 2012) Despite these potential advantages, a NICU fellow boot camp has not yet been described in the literature.Furthermore, literature describing the possible benefits of a boot camp lasting only one day, in any specialty, is lacking.
The goal of this study was to develop a compact, one-day simulation-based orientation session utilizing didactics, simulation scenarios and debriefing sessions.The sessions were focused on critical neonatal diagnoses and teamwork, and deliberate practice focused on life-saving neonatal procedures.We hypothesized that this type of orientation session would (1) improve first-year fellow confidence in their ability to perform critical procedures and (2) improve first-year fellow teamwork in resuscitative care.

Setting and Subjects
We conducted a prospective study at Cincinnati Children's Hospital Medical Center (CCHMC) of two cohorts of incoming clinical fellows in 2016 and 2017 with repeated measures before and after a one-day boot camp.Training was delivered in an on-campus, simulation lab that included a debriefing room, two simulation rooms and a control room.Supplies, including different non-invasive ventilation, intubation and chest tube equipment, from three hospitals were used to simulate the environment of all NICUs where fellows work.Simulations were video recorded to allow application of measures of outcome.Participation in boot camp was mandatory for all fellows but participation in the study was optional.This study was approved by our institutional review board prior to enrollment.

Study Aim and Outcomes
The aim of this study was to develop and implement a simulation and deliberate practice-based orientation session or boot camp for new fellows focusing on teamwork, leadership and procedural skills.In doing this, our primary outcome was to determineif a one day NICU fellow boot camp improved the participant confidence in the ability to perform key procedures (intubation, chest tube placement and UVC placement) and lead resuscitation scenarios.Our secondary outcome was to determineif the boot camp was able to improve teamwork amongst new fellows measured using the Team Emergency Assessment Measure (TEAM) scale.Our tertiary outcome was to determine whether participants maintained confidence six months following completion of boot camp.

Simulation and Debriefing
A total of five simulation scenarios were developed and piloted for feasibility, conceptual accuracy, and reliability after performing an online needs assessment of current neonatology fellows and faculty.A description of the scenarios is included in Appendix 1.Each fellow was required to act as team leader for one simulated resuscitation.Debriefing occurred following every scenario and focused on the non-technical behaviors (teamwork, leadership and communication skills) as well as cognitive/knowledge objectives (assessment, medical decision making, and reassessment).This was led by a facilitator formally trained in simulation debriefing, a faculty-level neonatologist and education specialist from the Center for Simulation.To augment the cognitive/knowledge objectives of each scenario, a 15 minute didactic segment by a faculty-level neonatologist was included after the debriefing.

Deliberate Skills Practice
Following three of the core simulation scenarios, there were deliberate practice training sessions focusing on endotracheal intubation, chest tube placement/needle thoracostomy, and umbilical catheter placement.This training was performed under the direction of senior fellows, attending physicians and/or neonatal nurse practitioners.Task trainers and simulators, which were selected based on previous experiences of the educators, are listed in Appendix 1.

Data Collection and Outcome Measures
Participant confidence was measured using self-reported pre and post-surveys (Appendix 2) with Likert scale style questions using a 1 to 10 scale, administered prior to the first scenario, following completion of all five scenarios and 6 months after boot camp.The post-tests were identical to the pre-test in content.Team-level performance was assessed using the TEAM scale via video review of the first and last scenario.The TEAM scale is a 12-item tool that has been validated in simulation-based training to evaluate the non-technical domains of team leadership (items 1-2), teamwork (items 3-9) and task management (items 10-11).The tool had a high Total Content Validity Index (0.96), internal consistency (0.89) and concurrent validity demonstrated by high correlation of the items with the global rating item.Intraclass correlation was included with inter-rater and retest rating of 0.60 and 0.80 respectively.(Cooperet al., 2010)This review was performed by an expert in simulation who has been trained in application of this tool.

Statistical Analysis
A two sided t-test was used to compare the continuous variables on pre and post-tests.For our secondary outcome, the mean differences between the first and last scenario were analyzed using one-way between scenario ANOVA.A two sided t-test was again used to compare the continuous variables between the pre, post and delayed post-tests to compare retention of confidence.

Study population demographics
We enrolled ten incoming neonatology fellows, five in 2016 and five in 2017.All ten fellows completed their three year pediatric residency through an accredited program in the United States.Three of the participants completed an additional year as chief resident.One of the trainees had completed residency at CCHMC prior to enrollment in boot camp.Selfreported previous experiences are listed in Table 1.All trainees were certified by the Neonatal Resuscitation Program (NRP) prior to enrollment in the boot camp.

Confidence Scores
Confidence scores were significantly improved between pre and post-test assessment in five of the six categories assessed (Table 2).

Team Level Performance
The first-year fellows' non-technical performance was analyzed using the TEAM scale, comparing performance in scenarios 1 and 5.For line items 1-11 (scale range 0-4 for each item) performance improved in scenario 5 for all line items, with exception of line 2, which remained the same between scenario 1 and 5 in year 2. For the overall global assessment rating (scale range 1-10), the reported scores were 3 and 4 for the first scenario in years 1 and 2 respectively and 8 in the fifth scenario in year 1 and 2. One way ANOVA was performed for the combined 2 year TEAM scale analysis and revealed a statistically significant difference between scenario 1 and scenario 5 with a p value of Maintenance of Confidence at Six Months Over the first 6 months of training, all ten fellows reported acting as a team leader, performing intubations and umbilical catheter placement.Nine of the ten fellows had the opportunity to place a chest tube.There was no statistical difference between the post-test confidence ratings and the delayed post-test confidence ratings.(Table 3).

Discussion
Following the implementation of boot camp, we saw a significant improvement in the mean self-reported confidence scores for fellows in making medical decisions, acting as a team leader during a code and neonatal resuscitation, chest tube insertion and intubation.Our findings support the use of high fidelity simulation orientation sessions to improve participant confidence in the ability to perform key procedures and lead resuscitation scenarios and highlights the possibility of limiting the sessions to one day.
As a measure of teamwork and communication, the TEAM scale was employed to evaluate the changes in non-technical behavior between the first and last scenario.Fellows demonstrated an overall improvement in teamwork, team leadership and communication throughout the day.This parallels the participants' improved confidence in their ability to perform as team leaders in resuscitation scenarios.It is not surprising that non-technical behaviors improved, as these behaviors were the focus of debriefing after scenario 1 and were part of the discussion in debriefings after scenarios 2-4.However, this improvement speaks to the effectiveness of simulation-based training for non-technical skills when emphasized during simulations and debriefings.(Burtonet al., 2011;Pattersonet al., 2013) We found no significant change in confidence scores six months following the training session compared to scores at completion of the training session.This could be interpreted as a maintenance of confidence throughout the first six months of fellowship.However, this may also indicate that new fellows gain confidence in the session that would ordinarily be obtained in the first six months of fellowship.This would demonstrate that the boot camp allows fellows to start fellowship with a level of confidence that is typically developed several months into fellowship and that they maintain this confidence as they continue to hone their skills.The biggest limitation to this study is the small sample size.The sample included only ten neonatal-perinatal fellows.
Fortunately, each of the fellows participating in boot camp came with a different background and training experience, allowing us to assess the effect of the boot camp on a small but diverse sample.Second, this boot camp was performed with incoming fellows at only one institution, limiting generalization of our findings to additional institutions.In the future we hope to dampen these limitations by continuing to offer this boot camp each July, prospectively collecting data on the pre and post confidence ratings of new fellows here and at other institutions initiating similar boot camp type simulation sessions increasing our sample size and offering better generalizability to findings at locations different from ours.
As shown by self-report at 6 months, the fellows had to team lead in the delivery room and at codes, as well as perform all three procedures clinically.

Conclusion
In a time when incoming fellows are potentially arriving with less overall procedural, leadership and resuscitation skill experience, we demonstrated a significant improvement in self-perceived skill confidence, teamwork, and team leadership in first year neonatal-perinatal medicine fellows following a one-day high fidelity simulation-based boot camp.The format for this boot camp is feasible and could be emulated in institutions across the country as a first step to improve the confidence and skill set of incoming fellows.

Take Home Messages
A one-day simulation-based boot camp experience is a feasible and efficient strategy to introduce teamwork and procedural skills to incoming trainees.
This one-day experience improves teamwork, team leadership and communication measured over the course of the day, while also increasing confidence in team leadership prior to the onset of clinical care.
This one-day experience increases confidence in performing critical procedural skills prior to the onset of clinical care.
Confidence levels reported after the boot camp were mirror confidence levels reported by trainees six months into fellowship.
This one day boot camp would be easily reproduced in a variety of training centers.-UVC Placement Unstable Supraventricular Tachycardia (SVT) You are called by a community pediatrician regarding a former 29 week infant who is now 42 weeks corrected.His hospital course was uncomplicated with exception of intermittent SVT, treated with propranolol.His episodes typically self-resolved prior to intervention while in the NICU.The infant presented to the office today for follow up and was found to have heart rate of 220 but was been hemodynamically stable.You instruct the pediatrician to have EMS bring the baby from the clinic across the street for admission to the NICU.EMS places an IV in the foot in route but they do not administer medications.On arrival: T 39, HR 220, RR 79, MAP 45, SpO2 95% on room air.Infant is awake, all pulses are palpable but extremities are mottled.Capillary refill initially Throughout the scenario, despite vagal maneuvers and administration of Adenosine, infant remains in SVT

Reference Source
Wen, T., Attenello, F. J., Wu, B., Ng, A., et al. (2015) The july effect: an analysis of never events in the nationwide inpatient sample.J Hosp Med.10(7), pp.432-438.Reference Source Young, J. Q., Ranji, S. R., Wachter, R. M., Lee, C. M., et al. (2011) "July effect": Impact of the academic year-end changeover on patient outcomes: a systematic review.Ann Intern Med.155(5), pp.309-315.Reference Source performance and confidence.Confidence was sustained after a six month duration.The teaching component included clinical skills and non-technical skills for five scenarios.Although not a new idea, this is well-written example of how such a program should be developed and assessed and the authors should be congratulated on a successful program.I'd be interested to hear more about how the five scenarios were determined.The authors say a 'needs analysis' was undertaken of fellows and faculty but it's unclear whether they were identifying the high risk scenarios, where previous gaps had been noted, or something else.It's also not currently clear if the scenarios were randomised.If they were presented in the same order (and the last scenario was easier than the first) it could introduce a systematic bias into the assessment.The TEAM scores are not fully reported.It would be useful to see those statistical tests.My concern is that the ANOVA test might not be appropriate due to non-normality of data arising from the granularity of the scale.Nevertheless, the results superficially look robust.I'd be interested in seeing a trial with a control of 'traditional' teaching compared to the boot camp method.

Ken Masters
Sultan Qaboos University This review has been migrated.The reviewer awarded 4 stars out of 5 A very well written and detailed paper, measuring and discussing the impact of a procedure and teamwork boot camp on both perceptions of, and actual, abilities.Although the authors have correctly noted the limitations of one institution and small sample size, the result are certainly encouraging, and indicate that this process could be repeated, both by the authors and by other institutions.The six-month follow-up is a useful rounding to the study.Although external to the paper, it would be of value (copyright permitting) if the authors could put links to their teaching and other materials, so that other institutions may implement a similar boot camp.Some small issues:• Early in the Introduction, the authors write "Across the country, neonatal…" with no indication of which country.Based on the authors' affiliations and the later mention of the study location, one senses that it is the USA, but this needs to be stated explicitly.•A few minor punctuation and spacing errors, especially lack of commas.In most cases, it is just an irritation; in some cases, it affects meaning.An example is the sentence containing "patient harm yet has been shown" which definitely needs a comma before the yet.(As it stands, it means that it will not yet cause patient harm).Also "determineif" and others.
Competing Interests: No conflicts of interest were disclosed.Reviewer Report 21 November 2018 https://doi.org/10.21956/mep.19700.r29459© 2018 Masters K.This is an open access peer review report distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Table 1 .
Summary of self-reported clinical experiences prior to boot camp

Table 2 .
Pre-simulation training mean confidence ratings vs post-simulation training mean confidence ratings While improving confidence in trainees is valuable, it is important to determine if this type of training has an impact on clinical performance.Previous studies have demonstrated an improvement in technical skill, teamwork, and communication in simulated patient scenarios as well as improvement in time to recognition of disease process and escalation of patient care following implementation of longitudinal staff simulation training programs.(Bangetal., 2016; Murphyet al., 2016; Tairaet al., 2016; Rabagoet al., 2017; Theilenet al., 2017)However, research demonstrating clear impacts on patient outcomes is lacking.(Neylanetal., 2017)Future studies should aim to evaluate the impact of this type of training on not only resident and fellow self-perceived confidence of skill but also impact on performance and outcomes in the clinical realm.
Sadie Williams recently completed fellowship in neonatal-perinatal medicine at Cincinnati Children's Hospital Medical Center where she worked to improve the education of trainees within the department of neonatology.As a member of the Center for Neonatal Care at Florida Hospital, she plans to continue working to improve the education of trainees in the NICU.
Gary Geis, an Associate Professor in the Division of Emergency Medicine and Medical Director of the Center for Simulation and Research, has 18 years of experience in medical simulation including institutional recognition as an educator and federal funding from AHRQ to implement and evaluate simulation-based training for medical trainees.