The Eﬀectiveness of an Online Training to Disseminate an Evidence-Based Intervention to Address Pediatric Secondhand Smoke Exposure

Background: There is a signiﬁcant lag between research on evidence-based practices (EBP) and adoption into clinical practice. A critical step is to train clinics on EBPs. There is limited research on the eﬃcacy of online trainings on EBP implementation in clinic settings. Methods: A cluster randomized control trial of 13 pediatric practices compared the acceptability and impact of an online versus in-person EBP training to reduce secondhand smoke exposure (SHSE) in children. A retrospective pre-post design was used to assess practice performance on speciﬁc metrics of the SHSE intervention. Results: 13 clinics were randomized to receive the online (n=5) or in-person training (n=8). Both groups reported high knowledge and self-eﬃcacy after the trainings, though these were signiﬁcantly higher for the in-person training group. Both groups showed signiﬁcant pre-post improvements in implementing key components of the intervention. When analyzed by training group, there were signiﬁcant improvements in some but not all of the metrics. Conclusion: Despite preferences for the in-person training, the online module is acceptable to providers, and improves self-eﬃcacy and implementation of some aspects of a SHSE intervention. Given the challenges and high costs of in-person trainings, carefully designed online modules represent a promising approach to disseminate EBPs.

generally a trusted source of health information (Glanz, 2013) and have access to many parents, especially medically underserved adults, who may lack a primary care provider for themselves but contact the health care system through their child's doctor (Heberlein, 2012, Winickoff, 2003. Despite the importance of screening pediatric patients for SHSE, recent estimates indicate that only 3.5% of pediatricians report providing smoking cessation assistance to parents. Thus, there is a significant missed opportunity to address SHSE in the pediatric setting (Winickoff 2013).
The Clinical Effort Against Secondhand Smoke Exposure (CEASE) is an evidence-based intervention that aims to improve pediatricians' ability to assess SHSE and provide cessation support through a three-step approach: ask, assist and connect (Winickoff, 2013;Winickoff, 2008). In this model, pediatricians are trained to ask each family about their child's SHSE, assist families by providing brief motivational interviewing and offering nicotine replacement therapy (NRT) prescriptions to smokers, and connect the smoker to the California Smokers' Helpline (Helpline) for additional support. In national clinical trials, CEASE increased pediatric providers' rates of screening and counseling to parents/caregivers who smoke (Winickoff 2013). Based on this evidence, CEASE has been recognized as a Research Tested Intervention Program by the National Cancer Institute (National Cancer Institute 2014). The success of the CEASE intervention, coupled with high rates of SHSE among children living in poverty, led the American Academy of Pediatrics California Chapter 1 to secure funding to disseminate the intervention model via in-person trainings and clinic support tools to 19 pediatric practices in Northern California. Over a one year period (January 2013-2014), 24 sites received in-person trainings on how to implement CEASE. These training sites generated over 200 referrals to the California Smokers' Helpline and of those referred, 40% were contacted and 75% of those contacted agreed to counseling and/or other smoking cessation support services (Marbin 2016). Independent, anonymous evaluations of the CEASE trainings found that 100% of participants rated the in-person CEASE training and the CEASE trainers as excellent or above average. In addition, 98% of participants reported the training increased both their knowledge of SHSE and their ability to screen patients for SHSE and provide smoking cessation support including counseling, NRT prescriptions and referrals to the Helpline. There were also significant pre-post increases in rates of SHSE screening, NRT provision, and referrals to state Helpline (Marbin 2016).
Findings from this initial effort were encouraging; however, additional dissemination efforts were limited due to the high costs associated with in-person trainings to reach pediatric practices across California's diverse geography. In addition, a major barrier to effective implementation of CEASE at some clinics was high staff turnover (Marbin 2016); there were not enough resources to return to these sites and provide in-person trainings to new staff.
To overcome these limitations, we developed an interactive online module to improve clinical uptake of CEASE, both by increasing the numbers of clinics able to participate in the training, and by allowing unlimited access to training course at a relatively low cost. However, the efficacy of an online training has not been rigorously studied for this EBP. Thus, the purpose of the current study was to translate the evidence-based CEASE intervention into an online training course using principles of instructional design, and to evaluate the acceptability and effectiveness of the online training module compared to the in-person CEASE training using a cluster randomized control trial design.

I. Development of the Online CEASE training
The online module was developed by the principal investigator (PI) who had previously developed and conducted a number of in-person trainings on CEASE throughout California. The PI partnered with E-Learning Mind, a learning development company with a track record of producing high quality and effective online training modules. As noted Marbin J, Purdy C, Tebb K MedEdPublish https://doi.org/10.15694/mep.2017.000013 Page | 4 previously, the CEASE online training incorporated characteristics associated with optimal learning outcomes including include interactivity, repetition, practice exercises and feedback. To maximize interactivity, the module incorporated multiple choice and "drag and drop" questions about key content areas that need to be answered before moving to the next section. The user was given feedback based on whether their answer was correct or incorrect; incorrect answers also provided the correct answer with an explanation. Interaction with peers was also incorporated into the online CEASE trainings through a training model which allowed learners to complete the online module individually, and then encouraged the individuals to come together as a group to discuss implementation of the CEASE in practice. The online CEASE training module was completed in the spring of 2015 and was awarded the Brandon Hall Group Silver Medal for "Best Advance in Custom Content" in 2016 (Brandon Hall Group 2016).

II. Evaluation Design
A cluster randomized control trial design was used to evaluate the acceptability and effectiveness of the online CEASE training. A total of 13 pediatric practice sites in Southern California (Riverside and San Bernardino Counties) who had not yet received the CEASE training were randomized to receive either the online or in-person training. This region was selected because of the similarity across the practice sites in terms of populations served and high rates of SHSE in children (see Table 1). In addition, these counties are among those with the highest rates of smoking and SHSE in California (County and Statewide Archive of Tobacco Statistics C-STATS 2011).

III. Procedures
To the greatest extent possible, the procedures for the online and in-person trainings were the same. Clinics were recruited through outreach to the American Academy of Pediatrics chapters, as well as through the Departments of Public Health in each county. This study received approval from the Institutional Review Board at Children's Hospital of Oakland Research Institute.
Once clinic leadership expressed interest in receiving the CEASE training and agreed to participate and be randomized to either online or in-person trainings, a physician champion at each clinic was identified. In both groups, the physician champion was asked to set a one hour time block (either lunch or breakfast) for the training.
The in person trainings were delivered by an experienced CEASE trainer and consisted of a 40 minute power point slide presentation followed by a 20 minute group discussion on how CEASE could be implemented in clinic. Similarly, the online training module consisted of a 25 minute interactive course and immediately following the module, participants also met as a team for 20-30 minutes to discuss the implementation plan for their clinic.
Both the online and in person trainings reviewed the prevalence and consequences of SHS exposure among children; the rationale for screening in the pediatric setting; strategies to screen pediatric patients for SHSE; motivational interviewing; how to prescribe NRT; and how to connect smokers to the Helpline for smoking cessation support counseling and services.
In a unique blended learning model, participants in the online training were able to call the CEASE PI during this meeting to share innovative ideas, get help solving challenges and ask any questions that might arise. Both groups were provided with worksheets to help them plan implementation of CEASE.
All clinics were provided with a variety of additional support tools to support the implementation of practice change strategies including pre-printed NRT prescription forms; Helpline referral forms; county specific NRT prescribing information, postcards with the Helpline's toll free number, and posters encouraging parents to seek help quitting smoking.

IV. Measures
Immediate post-training survey: Immediately after each training (in-person and online), attendees were asked to complete a confidential training evaluation survey. The survey consisted of 23 items. Participants were asked how often they screened for SHSE prior to the training, and if they had heard of CEASE prior to the training. Using a 5pt Likert rating scale participants' rated acceptability of the training (6-items), the extent to which the training improved their ability to implement various components of the CEASE intervention (6-items), and the extent to which the training was convenient, an effective use of time and engaging (3-items). Participants were also asked which training modality they prefer (online, in person, or no preference). The online group was asked five additional questions to further assess acceptability and quality of the online training.
Six-month post training provider champion survey: To assess the impact of the CEASE training on clinical practice, we used a retrospective pre-post design. A confidential online evaluation was sent to each physician champion six months after the CEASE training. Champions were asked to rate their site's performance on each of the CEASE specific metrics of Ask (SHSE screening), Assist (prescribing NRT) and Connect (referral to the smokers' helpline) prior to the CEASE training and after they had been implementing CEASE for the past six months. Participants used a 4 pt Likert Scale where 4=Always, 3= Usually, 2=Sometimes, and 1=Never. Pre-post scores were compared using the Wilcoxon Rank Sign non-parametric test.

Results
A total of 21 clinics were invited to participate. Of these 13 clinics agreed to participate and were randomized to receive the online training (5 clinics) and in-person training (8 clinics). Table 1 provides data for the two counties which shows a similar population on key variables such as smoking rates, and number of children exposed to secondhand smoke. Table 2 shows the distribution of participants by role at the clinic for the sites that received the in-person group. The in-person trainings were noted to have a higher attendance of medical providers, while medical assistants, nurse managers, and other support staff were more highly represented in the online training group.
anticipated. There are additional study limitations. The first pertains to difficulty in validating provider reports regarding referrals to the Helpline. Although champions from both groups reported an increase in referrals to the Helpline, the Helpline did not report receiving many direct referrals from either group. Champions informed us that providers often give parents the phone number for the Helpline because that is faster than entering the referral into the web-based Helpline referral system -which is considered a direct referral. Second, our data suggests that posttraining NRT provision was higher in the in-person group compared to the online group. This may be due to the larger number of providers who attended the in person training relative to the online training and since prescribing NRT is limited to providers (due to licensing issues), this difference in attendance may in part explain this discrepancy. Finally, California is generally knows as a state that adopts technology quickly; it is unknown how communities that are less open to new technology would respond to such an online training modality.

Conculsion
Despite preferences for the in-person training, the online module is both acceptable to providers, improves participants' knowledge and self-efficacy in their ability to implement CEASE, and improves implementation of key elements of the CEASE intervention. Given the challenges and high costs associated with in person trainings, this study suggests that carefully designed interactive online modules represent a promising approach to disseminate evidence based practices in clinical settings.

Take Home Messages
There is often a lag between development and clinical implementation of evidence based practices. Interactive online training modules are acceptable to health care providers and staff, and result increased knowledge and selfefficacy in implementing evidence-based practice intervention models (EBP). Carefully designed interactive online modules can be an effective alternative to in person trainings for dissemination of EBP.