The USMLE STEP 1 exam is arguably the most critical test of a medical student’s career. This medical school milestone requires efficient study methods, motivation and concentration for a student to reach their full potential. It is thus the responsibility of educators to identify students who would benefit from early intervention and provide guidance with sound learning strategies.

One widely used educational tool is the Learning and Study Strategies Inventory (LASSI), which is a 60 question, 10 subscale self-assessment of a student’s study habits and attitudes using a 5-point Likert scale (Table 1) (Weinstein, Palmer and Acee, 2016). It is designed to provide students with insight on their “skill, will, and self-regulation” pertaining to academic success. This allows for the improvement of learning strategies that will enrich their education. Furthermore, this educational survey has been applied in a variety of studies across different fields to enhance learning. 

For example, a cohort study by Flowers et al. investigated the internal validity of LASSI while applied to a cohort of African American high school students in association with their ACT performance (Flowers, Bridges and Moore, 2011). The investigators only concluded a significant relationship between Anxiety and Test Strategies subscales. Similarly, a comparison of LASSI results between high and low GPA chiropractic students was performed and demonstrated significant differences of LASSI results in the respective subscales of Anxiety, Attitude, Concentration, Motivation, Test Strategies, and Selecting Main Ideas (Schutz, Gallagher and Tepe, 2011). Studies across these respective academic fields serve to illustrate that LASSI is easily generalizable.

Many investigators have also used LASSI to assess different intervention strategies to improve student academic performance. One such intervention investigated was the formulation of a “Study Strategy” class for medical students. In this study, a LASSI was administered to a cohort of 32 participants prior to and after completing the strategy class. The investigators subsequently found a significant increase in all of the LASSI subscales with exception of Anxiety, Test Strategies, Motivation, and Concentration for student participants (Haghania and Sadeghizadeh, 2011). Similarly, Sera et al. created a study course and assessed its effects on LASSI results for 312 pharmacy students (Sera and McPherson, 2019). The findings also demonstrated significant increases in mean LASSI scores in all sub-categories except Anxiety. These studies on academic intervention help define LASSI as a powerful measure used to assess real change in student study strategies.

Relationships between LASSI data and STEP 1 performance have also been previously described in the literature. West et al. studied a cohort of 79 medical students and created linear regression models that illustrated correlational relationships between STEP 1 score and the predictor variables of Year 1 and Year 2 grade averages, MCAT scores, CBSE and CBSSA scores, and LASSI results (West et al., 2016). In the study, Concentration was the only LASSI sub-category found to be a significant predictor of STEP 1 performance. Likewise, Khalil et al. performed a study on 180 medical students assessing the relationship between LASSI data and Anatomical Sciences in addition to STEP 1 performance. This study found 5 subscales of LASSI which held statistically significant relationships with STEP 1 performance: Anxiety, Motivation, Selecting Main Ideas, Test Strategies, and Information Processing specifically (Khalil, Williams and Hawkins, 2016).

However, despite the documented statistical relationships between STEP 1 performance and LASSI results in these respective studies, significant differences in LASSI results amongst students stratified into STEP 1 class quartiles have not been described. Additionally, due to the wide variation seen in LASSI data, more studies with larger cohorts are needed to further assess the limitations of LASSI and demonstrate which contexts it is most applicable. Accurate interpretation of LASSI is especially important since it has the potential to help struggling students identify their strengths and weaknesses in order to achieve success in examinations that shape their lives.

This current retrospective study employed an entire graduating class cohort of 220 medical students, the largest cohort described in the “LASSI and STEP 1” literature, and evaluated for any statistical differences in LASSI results between the quartiles of STEP 1 scores. It was designed to investigate the potential learning gaps underperforming students may have, and would benefit clinical educators to allow tailoring of academic guidance of future classes. It also uses the most current version of LASSI which is more efficient, statistically sound, user-friendly, and updated based on recent educational research (Weinstein, Palmer and Acee, 2016). Upon literature review, it was hypothesized that successful students would have statistically significant mean scores in different LASSI subscales. Furthermore, this study would serve to increasingly validate LASSI as an effective education tool and add more significant, generalizable data to the didactic community at large.

Participants

The study cohort consisted of 220 medical students from the graduating UT Southwestern class of 2022. 108 students were female and 112 were male. The mean STEP 1 score for this sample (n = 220) was 238 with a standard deviation of 17.

Students were stratified into four quartiles based on UTSW STEP 1 score data (Table 2).

Instrumentation

The 3^rd Online Edition Learning and Study Strategies Inventory (LASSI) is a 60-item index that contains 6 questions for each of the 10 LASSI subscales (Weinstein, Palmer and Acee, 2016) (Table 1). Notably, half of the questions are asked in a positive fashion while half in a negative fashion to reduce response bias. The responses are assessed by a 5-point Likert scale with positive questions being coded as 5 being “very much typical of me” and 1 being “ not at all typical of me” whereas negative questions are coded as 5 being “not at all typical of me” and 1 being “very much typical of me”. Each student is then assigned a percentile (0-100) per subscale identifying its respective strength.

Data Collection

This Retrospective Cohort study was approved by the UT Southwestern Institutional Review Board. LASSI surveys were administered once to students in the first pre-clinical semester, specifically in September 2018 after their first exam. Data responses were then collected by the Student Academic Support Services (SASS) office and stored in the respective student academic profiles. Pre-clinical grade averages and gender data were provided by the Office of Medical Education at UT Southwestern. USMLE STEP 1 scores were then collected after students took the examination anywhere between January and July, 2020.

Data Analysis

Descriptive statistics with means and standard deviations of LASSI subscales, STEP 1 scores, and final exam grades were determined for all quartiles. Given a sample of 220 students, the central limit theorem was applied to assume the sample was normally distributed. One-way Analysis of Variance (ANOVA) with post-hoc Tukey-Kramer test analysis was performed comparing the quartile groups within individual LASSI subscales.

Descriptive Statistics

Descriptive statistics of mean LASSI subscale results and their respective standard deviations are shown below in Table 3. The distribution of mean LASSI scores per quartile is illustrated in Figure 1. The mean STEP 1 score for the 1^st, 2^nd, 3^rd, and 4^th quartiles were 257 (SD = 4.6), 246 (SD = 2.5), 236 (SD = 3.2), and 215 (SD = 13) respectively. The mean grade averages of compiled final exam performance for the 3 pre-clinical curriculum semesters were 91.3 (SD = 2.7), 88.9 (SD = 2.6), 86.3 (SD = 2.4), and 81.6 (SD = 2.1) for the individual quartiles.

Figure 1: LASSI Quartile Distributions

LASSI Subscale Analysis

Data analysis with one-way ANOVA revealed statistically significant differences in 8 LASSI subscales (Table 4). These subscales were determined to be Anxiety, Concentration, Information Processing, Motivation, Selecting Main Ideas, Self-Testing, Test Taking, and Time Management. The Attitude and Utilizing Academic Resources subscales were not found to have any significant differences between quartiles.

Post-Hoc Tukey-Kramer analysis of the eight significant LASSI subscales, defined above (Table 4), illustrate the statistically significant relationships between the individual quartiles (Table 5).

LASSI is an invaluable educational tool that serves to help academic administrators identify struggling students and provide help where it is needed most. Additionally, successful students benefit from the LASSI survey by identifying and polishing their minor flaws. This is especially important in the context of assisting medical students who wish to perform at their best for the high stakes USMLE STEP 1 exam. However, despite its convenience, a wealth of information on LASSI is not available which questions the accuracy of its interpretation. This study however, with one of the largest student cohorts, combats the need for more generalizable investigations of LASSI in medical education. 

Recognizing the importance of STEP 1, our study aimed to demonstrate significant differences in LASSI subscales between STEP 1 underperformers vs those who excelled. Comparing class quartiles by STEP 1 score, the null hypothesis was rejected since higher performing students were determined to have significant increases in almost all mean LASSI subscale scores. These findings are similar to prior studies assessing medical student STEP 1 performance in correlation with LASSI data (Khalil, Williams and Hawkins, 2016; West et al., 2016). However, our investigation was the first to stratify STEP 1 performance into quartiles before performing data analysis. This difference was able to provide a more focused view for educational administrators by illustrating which characteristics align with STEP 1 underperformance versus success.

Notably, the ANOVA and post-hoc analysis illustrated the most differences between the 1^st and 4^th  STEP 1 quartiles with significant p values in all 8 significant subscales. In contrast, there were no significant differences found between students in the 2^nd vs 3^rd quartiles or 3^rd vs 4^th quartiles in any of the 8 subscales. This sheds light on the characteristics that differentiate high performing students versus those who struggle.

Additionally, it is important to not ignore the subscales that did not yield any significant differences between quartiles: Attitude (ATT) and Utilizing Academic Resources (UAR). Regarding the ATT subscale, it was also not found to have significant relationships in other prior studies assessing medical student STEP 1 performance and LASSI data (Khalil, Williams and Hawkins, 2016; West et al., 2016). This serves to further the point that a students’ general attitude towards school or academic tasks may not be the best predictor or key factor driving success. Secondly, the UAR subscale may not have had any significant differences due to internal institutional factors. UT Southwestern has developed a Student Academic Support Services (SASS) department that is available to all students and well publicized during the first week of orientation and at several mandatory academic lectures throughout the first and second years. Recently, SASS has developed a required “STEP 1 prep course” for all students which inherently standardizes academic resource utilization between students (Sachs et al., 2019). The popular reputation of SASS and class wide “STEP 1 prep course” may explain the lack of variation between quartiles for utilizing academic resources. However, despite the lack of significant difference among quartiles, there still remains a marginal difference in resource utilization. As the distributions in Figure 1 demonstrate, the 4^th quartile students score higher in UAR likely due to the fact that SASS identifies struggling students and promptly addresses their learning gaps. Students in the 1^st-3^rd quartiles who are not failing or deemed “at-risk” are thus not “utilizing” SASS as much which may explain the slight differences between groups.

Given the findings discussed, there need to be interventions focused towards advocating for student wellness to combat stress and anxiety, creating “study strategy” classes to improve test taking, advising on time management skills to build student efficiency, and others targeted to the significant subscales shown in this cohort. Additionally, recent literature on medical education stress many analogous intervention strategies which strike close resemblance to the LASSI subscales (Sein, Dathatri and Bates, 2020). Notably, our institution’s “STEP 1 prep course”, mentioned earlier, serves as an example of a multi-faceted approach for improving STEP 1 performance by targeting some of the highest yield LASSI subscales. Through the 6-week course during dedicated STEP 1 study time, students have opportunities to attend workshops on time management, overall wellness, testing strategies, and development of effective study plans. Furthermore, students must submit weekly progress reports to SASS allowing the administrators to identify struggling students in need of more focused help. This intervention served to significantly increase UTSW STEP 1 performance with a 97% pass rate in 2017 (mean = 235) and a 100% pass rate (mean = 239) in 2018 (Sachs et al., 2019). Similarly, many academic medical institutions have also documented analogous interventions demonstrating the beneficial impacts on academic success (Haghania and Sadeghizadeh, 2011; McConville, McAleer and Hahne, 2017; Sera and McPherson, 2019). Accordingly, future interventions should focus on important aspects of student learning and utilize tools like the LASSI to define the specific traits of their student populations.

Limitations

In our study we recognize several limitations. Firstly, the LASSI is an introspective survey in which students self-report their study habits and attitudes. This is susceptible to subjectivity and may not capture the reality of their capabilities. Additionally, the LASSI was administered at the beginning of the first year and study strategies may have significantly changed to adjust for the rigor of medical school (Zhou, Graham and West, 2016).

This investigation assessed the relationship between LASSI subscale results and STEP 1 performance between STEP 1 quartiles of a class cohort. Significant differences between Anxiety, Concentration, Information Processing, Motivation, Selecting Main Ideas, Self-Testing, Test Taking, and Time Management were found between the groups. These findings demonstrate that LASSI could be an effective tool in assessing the characteristics of students in the context of STEP 1 performance. It thus allows medical educators to identify at-risk students and intervene by targeting the highest yield subscales to maximize academic success. This is imperative due to the significance of STEP 1 and should be applied to other important board exams across the academic spectrum.