Despite meeting the high academic standards required to study undergraduate medicine, a proportion of students struggle to perform consistently to a satisfactory standard (Yates and James, 2006). Furthermore, struggling medical students appear to be at an enhanced risk of later-career underperformance within postgraduate licensing examinations, clinical competency ratings, and meeting expected professionalism standards (Hojat et al. 1993; Gonnella, Erdmann and Hojat, 2004; Papadakis et al. 2005; Hemann et al. 2015). Therefore the timely identification and effective remediation of struggling students, are recognised as important steps to support acute and help avoid longer-term practice underperformance (Challis, Flett and Batstone 1999; Cleland et al. 2013; Winston, Van der Vleuten and Scherpbier, 2014).

Factors associated with chronic medical student underperformance are often multiple, complex and interlinked, with more targeted holistic remediation interventions have been advocated (Cleland, Cilliers and van Schalkwyk, 2018). The remediation intervention described in the present study is a holistic approach to support chronically struggling students. It explores any underlying pastoral and poor motivational issues, cognitive and organisational aspects of personal study skills and examination technique. The remediation intervention is underpinned by core principles taken from Self-Regulated Learner (SRL) (Zimmerman, 2008) and Spaced-Retrieval Practice theories (Roediger and Butler, 2011) and Motivational Interviewing (Wells and Jones, 2016).

SRL practice has been defined as the ability to initiate and direct one’s learning beyond formal learning settings, via the proactive control of one’s own learning environment and processes (Zimmerman, 2008). Remediation strategies that harness SRL principles have been promoted within medical education (Sandars and Cleary 2011; Cleland et al. 2013). Available evidence suggests that students possessing an effective SRL practice correlates with higher academic performance (Patel et al. 2015; Gandomkar et al. 2016). Effective SRLs possess metacognitive, motivational and behavioural strategies that help optimise the learning process. The systematic SRL process follows a dynamic, interrelated three phase sequential cycle: Forethought, Performance and Self-Reflection phases (Zimmerman, 2008). Forethought phase processes include: setting effective goals, strategic planning and self-motivational beliefs (eg self-efficacy and task interest perceptions). Performance phase processes, occurring during the learning task, involve the metacognitive learning monitoring, built on the preceding Forethought learning strategies. Finally, the Self-Reflection processes that occur after the learning task, include self-judgement and adaptive self-reaction elements, that self-assess and evolve learning practice (Zimmerman, 2008).

Spaced-Retrieval Practice theory revolves around the premise that long-term information retention is enhanced (versus massed rote learning and test “cramming”) by the effective, temporal spacing of independent study sessions, combined with structured, repetitive and spaced information retrieval attempts and testing (Larsen, Butler and Roediger, 2008; Roediger and Butler, 2011). Motivation Interviewing, developed within clinical settings to target unhealthy behaviours, harness a non-judgemental, empathic and collaborative approach (Miller and Rollnick, 2012). Its application to university-level teaching has been advocated, as a tool to build student self-motivation, autonomy, confidence, resilience and to overcome resistance to required change (Wells and Jones, 2016).  

This longitudinal study aimed to explore the possible effectiveness of our holistic remediation intervention, offered to struggling medical students, by analysing summative Progress Test performances.

Participants and setting

This study was undertaken within the Peninsula Medical School, located in South West England, between 2008-2017. From a total population of 2,088 students, the applied medical knowledge performance of identified chronically struggling Years 1-5 undergraduate medical students that experienced the holistic remediation intervention were analysed.

Our assessment principles are based on a “frequent assessment, rapid remediation” paradigm (Ricketts and Bligh, 2011). A key part of our detection of struggling students, is via monitoring ongoing student performances within our frequent, summative medical knowledge-assessing Progress Tests (Coombes et al., 2010). This practice enables the relatively early identification of underperforming students and offering of resultant timely remediation packages. An Academic Review Group, convenes frequently to monitor all individual students’ summative assessment performances within the medical programme. By following set remediation criteria, a range of tiered and timely remediation packages are then offered by this group to identified struggling students.

For any single Progress Test failures, students are required to meet initially with their Academic Tutor to discuss academic study skills, receive appropriate pastoral tutor signposting (“Level One” Remediation), and agree a learning contract. Any students whose academic performance does not then follow a sustained recovery following this lower level of remediation, are then offered a more in-depth and more intensive “Level Two” remediation holistic package, the focus of the current study. 

Holistic Level Two Remediation

The Level Two remediation pathway builds upon a previously described remediation process: assessment-based referral, a diagnostic meeting, an agreed management plan and follow-up appraisal (Sayer et al., 2002; Kalet, Guerrasio and Chou, 2016). The principal element is a semi-structured diagnostic individual interview with a remediation team tutor. Our remediation team are experienced medical school educators and clinical teachers, with extensive training and experience in remediation.

During the diagnostic interview, the student’s approach to study is discussed. Organisational (e.g  time management and planning) and cognitive aspects are explored and aligned to SRL (Zimmerman, 2008) and Spaced-Retrieval Practice (Roediger and Butler, 2011) principles. Motivational Interviewing techniques are also employed to help students adjust inappropriate, and focus on successful, behaviours (Wells and Jones, 2016). Students are encouraged follow sequences of short study periods (up to 30min), interspersed with brief rest periods and spaced opportunities for information recall and self-testing. This model is reported to enhance learning, versus a practice of devoting multiple continuous hours of constant studying (Larsen, Butler and Roediger, 2008; Roediger and Butler, 2011). Students are recommended to set a limited amount of SMART learning goals, limit environmental study distractions (eg noise and social media access) and engage in peer study groups to help optimise available cognitive load (Young et al., 2014).

Additionally, medical knowledge exam technique, screening for any undiagnosed specific learning difficulties and any underlying pastoral issues that may influence both motivation and academic performance are explored. Following the remediation interview, a resultant remediation report is agreed comprising of: meeting notes, study adjustment recommendations, agreed goals and any further appropriate specialist support referrals (Sayer et al., 2002). Examples of referral to supplemental specialist services include: university learning support centres, student counselling, their own General Practitioner and Occupational Health services. The student’s ongoing engagement with the remediation report and subsequent academic performance are then reviewed by both the student’s Academic Tutor and by the Academic Review Group throughout the rest of the academic year.

Performance in Progress Test multiple choice questions

The Peninsula Medical School employs Progress Testing, a MCQ-based testing model, to assess longitudinal growth in applied medical knowledge. Assessment of medical students’ medical knowledge by Progress Testing has been described previously (Ricketts, Brice and Coombes, 2010). Briefly, each Progress Test consists of 125 single best answer questions, each question comprising a clinical vignette, a choice of five possible answers and a “don’t know” option. In Years 1-4, students sit four tests per academic year, whereas in Year 5 students are required to pass two tests. Each question is assessed with +1 mark for a correct answer, -0.25 for an incorrect answer and 0 for a “don’t know” response (Wade et al., 2011). In Years 1-4, scores are norm-referenced, in which the deviation of an individual student score from their cohort is expressed as a z-score. In Year 5, test cut scores are criterion referenced to the standard of a newly qualified F₁ doctor.  Test data are displayed within an online assessment database, enabling graphical test performance representation of individual student performance and academic performance trajectories, relative to their cohort.

Statistical analysis

Students’ Progress Test performances were examined in Year 1-5 students between 2008-2017. First attempt year Progress Test student performances were analysed pre- and post-Level Two remediation intervention. Progression was compared in response to Level Two remediation using a Bonferroni Multiple comparison test and paired t-test, where appropriate.

Between 2008-2017, a total of 2088 students were enrolled onto the BMBS programme. During this period, mean Years 1-5 incidence of Level Two remediation interviews offered to struggling students was 5.9% ± 0.6% total students per year. Following the Level Two remediation interview offer from the Academic Review Group, the majority of students then opted to receive remediation (93.1 ± 3.7% uptake per year).

Longitudinal Analysis of Medical Knowledge Performance

Figure 1 compares the incidence of Level Two Remediation per year of study. During Year 2, Level Two remediation referral incidence peaked (8.6 ± 1.4% interviews per year, n=145), then declined significantly versus later student cohorts (p<0.05). The lowest Level Two remediation incidence occurred in Year 5 (1.7 ± 0.9%, n=16), versus other years of study (p<0.01).

Figure 1: Incidence of Level Two remediation referral by year of study

Data displays frequency of Level Two remediation by study year. Year Two (n=145, 8.6 ± 1.4%) displayed a significantly higher Level Two remediation incidence vs. years three or four (p<0.05, Bonferroni Multiple comparison test). Year 5 (1.7± 0.9%, n=16) displayed the significantly lower Level Two remediation incidence versus Years 1-4 (p<0.01, Bonferroni Multiple comparison test)

Prior to Level Two remediation, student test performances in preceding tests declined significantly (p<0.001),resulting in student referrals to our Level Two remediation pathway. Figure 2A depicts struggling students mean medical knowledge performances pre- and post-Level Two Remediation intervention for study Years 1-4. Across Years 1-4 of the programme, a significant enhancement in mean medical knowledge assessment performance immediately followed Level Two Remediation (p<0.0001, n=403). Of particular note, was the significantly higher acute mean performance recovery in Year 1 cohorts versus Years 2-4 (p< 0.05, n=87; Figure 2B). This enhancement in Progress Test performance was sustained over the four following tests following remediation. Additionally, this enhancement displayed either no significant gender or ethinicity bias (data not shown).

This effect was also observed in Year 5 (Figure 2C), although any comparable sustainability was not possible to identify, as Year 5 students are only required to pass two medical knowledge assessments in Year 5 prior to graduation.

Figure 2: Progress Test performance is enhanced following Level Two remediation

Test performance pre- (-1) and post (+1) - Level Two Remediation were analysed in struggling students offered Level Two remediation. Medical Knowledge performance data is displayed as SD from cohort mean (z-score).  A) Years 1-4 (combined), B) Years 1-4 and C) Year 5 only. The arrow indicates the first test after students received Level Two remediation.  Prior to Level Two remediation, student performances in preceding tests significantly declined across all year levels (p<0.001, paired t-test). Level Two remediation afforded significant enhancement of medical knowledge performance across all years of study (p<0.0001; paired t-test, n=403). Year 1 displayed a higher recovered performance in the first two tests following  Level Two remediation, significantly different versus to Years 1-4 (n=87, p<0.05, Bonferroni Multiple comparison test).

The relative effectiveness of the Level Two remediation intervention appears to be consistent, with no significant difference found in its effectiveness over the 2008-2017 evaluation period (Figure 3).

Figure 3: Consistent remediation package effectiveness

Years 1-4 mean Progress Test performances pre- (-1) and post (+1) Level Two remediation between 2008-2017. Progress Test performance data is displayed as SD from cohort mean (z-score). The arrow indicates the first test after students received Level Two remediation. There was no significant difference in Level Two remediation effectiveness over the 2008-2017 period.

Struggling undergraduate medical students often present with interlinked underpinning issues, that may benefit from a holistic supportive approach and environment to promote a sustained academic performance recovery (Cleland, Cilliers and van Schalkwyk, 2018).  The present study provides evidence that our holistic Level Two remediation intervention is associated with a resultant both significant and sustained enhancement of students’ medical knowledge performance.

Our “frequent assessment, rapid remediation” assessment practice enables the timely identification of any sustained decline in medical knowledge test performance relative to previous test scores and student peers, and reveals sudden declines in performance. The significant peak of Level Two remediation incidence earlier in our programme appears to be is consistent with previous reports of academically-struggling students manifesting at early programme stages (Yates and James, 2006; Kebaetse et al., 2018). In our experience, a sudden divergence from previous sustained satisfactory performance, may often be explained by an acute personal adverse event affecting performance. Such acute underperformance instances are explored initially by our lower (Level One) remediation packages, during a 1-2-1 meeting with the student’s Academic Tutor.

Where academic underperformance is sustained, it is often linked to multifactorial associated issues, including: poor SRL practice (Artino, Hemmer and Durning, 2011; Gandomkar et al., 2016), adverse relationship, financial and/or mental health issues (Sayer et al., 2002; Yates, 2011; Heikkila et al., 2012; Patel et al., 2015).  Chronic struggling students can often present with maladaptive failure-coping and superficial learning strategies, reported to correlate with poor support system engagement and elevated risk of entering a cyclical failure pattern (Patel et al., 2015).

Our non-punitive, non-judgmental and collaborative remediation intervention, avoids a “more of the same” type intervention, such as supplementary knowledge teaching, reported to be relatively ineffective (Cleland et al., 2013). Instead, our holistic support offering focusses on both affording pastoral support (including clinical referrals, as appropriate), motivational coaching and the developing of reflective SRL practice, that includes spaced-practice testing and evolving exam technique. Available evidence suggests that those students who engage successfully with SRL, are more likely to achieve higher successful academic performance, via a process that includes setting appropriate learning goals (Sandars and Cleary, 2011; Patel et al., 2015).

Motivational Interviewing attempts to disrupt the possible causative destructive cyclical pattern of poor SRL-derived failure. Motivational Interviewing, and the parallel collaborative Appreciate Coaching approach (White and Barnett, 2014), have been reported to result in remediation-based changes in poor behaviours in both clinical and higher education settings (Wells and Jones, 2016). Our motivational coaching remediation element, focusses on recognising and enhancing personal strengths and confidence, previously reported to help adjust poor SRL behaviours and boost self-efficacy (White and Barnett, 2014). It aims to support the learner’s adverse failure-causal attributions, often perceived as being uncontrollable or externally-derived and afford support packages targeting any associated adverse emotional impact, and resultant sequelae, of their failure experience (Patel et al., 2015).

Our holistic and voluntary-attendance remediation offering, has also moved away from a “one size fits all” support or further teaching support packages. Our use of clinically-trained Level Two remediation members, not directly involved in the students’ assessments or academic progression decisions, avoids the reported adverse “judge and jury” perceptions that struggling students’ may have of their remediators (Yates, 2011; Patel et al., 2015). Additionally, a previous study recommended that remediation packages should be made compulsory, as struggling students may often lack appropriate insight and awareness of their developmental needs (Winston, Van der Vleuten and Scherpbier, 2010). In the present study, the vast majority of chronically struggling students who were offered Level Two remediation accepted the invitation.

Although it would be desirable to identify specific Level Two remediation components that may consistently deliver an enhancement in a student’s performance, this is not a straightforward task to isolate such effective factors. Indeed, the efficacy of any individual remediation element tested will also likely depend on an individual’s complex needs (Patel et al., 2015). The in-depth student interview referral and resultant advisory report employed in this study, enables both timely identification and resultant bespoke recommendations, based on an individual's particular recognised needs. Exploration of students’ perceptions of their Level Two remediation experience is the subject of our ongoing study.

The early detection and subsequent successful remediation (ie sustained performance recovery) are identified key attributes in an effective student support programme. The present study, describes a holistic support package that has high levels of student engagement and correlates with a consequent sustained, enhanced recovery in Progress Test performance.