This study explores the correlations between a student’s academic performance, socioeconomic background (i.e. their gadget ownership, internet availability, monthly expenses, commuting time to school), internet habits, technology familiarity & comfortability level, and online-learning engagement in blended-learning environment in Indonesian medical school through surveys, learning management system (LMS) analytics, and class assessments.
Ethics statement
This study was reviewed by the Institutional Review Board at Harvard University on 11/19/2018 and was deemed exempt.
Sample
In this project, we measured the academic performance of the students based on their test scores. With various assumptions of the outcome's average, outcome's standard deviation, type I error rate, and that the linear regression would be used to assess associations, we calculate that the sample size will be at least 40 students to have the 97-99% power to detect mean differences of both groups.
The intervention’s target was a dermatology clinical rotation at the Faculty of Medicine, Universitas Indonesia (FKUI). The research was conducted with 46 international medical students in FKUI in Jakarta who are enrolled in a dermatology clinical rotation. We studied 2 cohorts (January 2019 and February 2019) which consist of 46 students. This study was aimed to be generalizable to the international medical student population.
Our inclusion criteria were 5th-year medical students who are in a dermatology clinical rotation during the research time period and have not previously taken the dermatology rotation. Our exclusion criteria were students who are not engaged in at least one part of the study plan: online learning, face-to-face session, survey, and RAT.
Operational Steps
Participating students studied in a blended-learning environment. There were three sources of data: surveys, analytics, and examinations. The analytics was extracted from a native LMS of FKUI, the Student Centered e-Learning Environment (SCeLE). Then, the data was analyzed through multiple regression using STATA.
There were two cohorts of students. Each student cohort attended a dermatology clinical rotation for four weeks. On day 1, informed consent and survey were collected from the samples. Online-learning analytics were collected throughout the online part of blended-learning. On Day 3. The RAT result was collected.
The survey
A survey (See Supplementary File 1) was given to the participating students measuring their level of comfort, digital skills, information about their accessibility to the internet, backgrounds, etc. The survey was completed in a classroom of FKUI and administered by paper.
There were 23 survey questions completed by each participant which were turned into 7 variables; (1) average monthly spending; (2) daily commuting time; (3) home internet availability; (4) number of gadgets owned (4 questions); (5) level of technology familiarity (4 questions, the results ranged between 0 and 4); (6) level of technology comfortability (6 questions, Likert scale 1 to 5, the results range between 6 and 30); (7) level of internet habit (5 questions, Likert scale 1 to 5, the result is ranged between 5 and 25).
The average monthly spending variable was obtained in thousands of IDR and then converted into millions of IDR in the dataset. The daily commuting time variable was written in the time unit of hours in the dataset. The home internet availability variable was written in the dataset as ‘0’ for those students who do not have home internet and ‘1’ for those students who do have home internet.
The LMS analytics
The analytics of the online-learning platform informed the participants’ learning engagement variables. From the analytics, there are five variables: (1) page views, (2) access duration, (3) login times, (4) time of first login, and (5) time of last login. For the data collection method, please refer to Supplementary File 2.
The online-learning component contained 5 videos, including introduction video; 800-word introduction guides; 4 compressed downloadable learning materials and assignments, 3 reading materials were compressed into 1 of those files; and 1 podcast. If a student views and downloads every piece of material once, we anticipated that it will take at least 10 'page views' and around 30 minutes 'duration of access.’ We anticipated that a student would take about 2 hours total to complete all of the assignment and read the supplemental materials; however, the exact duration is very hard to capture precisely in the online-learning analytics.
The LMS was based on Moodle technology. One of the variables collected is home internet availability. This specific platform does not have a mobile version, so students must access the platform on a PC or laptop. Therefore, most students needed to have an internet connection to access the learning platform.
The academic performance measurement
The RAT measures a student’s first performance after the online part of blended-learning. The examinations’ question sets were obtained from FKUI and were based on the student’s regular coursework.
The scores were gathered from the readiness assessment test (RAT) using multimodality questions: a combination of multiple choices, short answer, and matching questions. In order to get the best results, students were encouraged to effortfully participate on the test and we incentivized the best performer on the test with a gift. Students were also informed that the formative assessment provides a chance for them to calibrate their knowledge. Students were required to sign an honor code or statement of academic honesty before each test.
The analysis plans
Some of the data was categorized into new variables. For example, we formed two students’ groups: 1. Students who spend more than 5 million IDR a month; 2. Students who spend under 5 million IDR a month. Using Microsoft Excel, we calculated the number of samples from each group and included its percentage. Using STATA, we ran 12 linear regression analysis for all 12 predictors (7 variables from group A and 5 variables from group B) and one outcome (academic performance). Afterwards, those variables that have associations with the outcome were taken into a bivariate model and a multivariate model of regression analysis.