Case study
Open Access

No Relation Between Social Media Use During Teacher-Supervised Activities and Exam Results in Students Medicine and Biomedical Sciences

Dick Thijssen[1], Maria Hopman[2], Andre Verbeek[2], Thijs Eijsvogels[2]

Institution: 1. Radboud university medical center, 2. Radboudumc
Corresponding Author: Prof Dick Thijssen ([email protected])
Categories: Assessment, Students/Trainees, Technology
Published Date: 03/09/2018

Abstract

In addition to the potential benefits of mobile technology in (bio)medical education, mobile technology can also be used for personal use and access to social media. In this study, we explored the relation between (study-related) use of social media and exam results in a cohort of 259 Dutch students enrolled in Medicine (n=211) and Biomedical Sciences (n=48) programs. Self-reported social media use was scored for the use of internet, Facebook, Twitter, Whatsapp and SMS during teacher-supervised activities (e.g. lectures, interactive lectures and group assignments). Subsequently, students scored whether use was study-related on a continuous scale. We found a highly prevalent use of social media during teacher-supervised activities (97%), with a relatively small portion being study-related (30±24%). We divided students into: 1.Low user+low study-related, 2.Low user+high study-related, 3.High user+low study-related, and 4.High user+high study-related. Exam results did not differ between groups (ANOVA; P=0.45). Moreover, we found no relation between exam result versus social media use (r2=0.002, P=0.49) or study-related social media use (r2=0.007, P=0.18). In summary, we found no evidence that the amount and/or type of social media use during teacher-supervised activities significantly relate to exam results in a cohort of Dutch (bio)medical students.

Keywords: social media; internet; exam results; medical education; medicine

Case study description

We are living, learning, teaching and practicing medicine in the digital era. Consequently, the use of mobile technology, including smartphones, tablets and laptops, in (bio)medical education is common practicel (Ellaway et al., 2014). Previous work has demonstrated the potential benefits of these devices for educational and clinical purposes, as social media use may improve knowledge (e.g. exam scores), attitude (e.g. empathy, learner engagement) and skills (e.g. reflective writing) (Cheston, Flickinger and Chisolm, 2013). However, potential negative aspects relate to distraction from learning (Khan et al., 2009) and the risk of inappropriate use (Wallace, Clark and White, 2012). Currently, little is known about the impact of social media use on exam results. For this purpose, we explored whether the use of social media during teacher-supervised activities (i.e. lectures, interactive lectures and group assignments) affected exam results in Dutch students (bio)medical science. We also explored if the type of social media use affects the relation between social media use and exam results.

 

In our institution, we run an annual, 4-week programme for students Medicine (1st study year) and Biomedical Science (2nd study year). This programme does not contain social media-based learning, whilst students are free to use mobile technology during teacher-supervised activities. This created an opportunity to examine students’ use of social media in relation to exam results, relatively free of other confounding factors. After their exam, students were invited to fill in an online questionnaire about their social media use during teacher-supervised activities (lectures, interactive lectures and group assignments). This research was conducted upon approval from the local ethics committee and followed the principles of the Declaration of Helsinki. Participants scored the social media use (quantified as ‘none’ (0), ‘sometimes’ (1), ‘often’ (2), or ‘always’ (3)) for separate domains: internet, Facebook, Twitter, Whatsapp and SMS. This resulted in a “social media”-score that ranged from 0 (i.e. no use of social media) to 15 (i.e. always use of all 5 domains of social media). An average was taken from the 3 separate teacher-supervised activities (lectures, interactive lectures and group assignments) for the “social media”-score. In addition, students indicated how much of their social media-use was study-related (i.e. 0%=no study-related activities, 100%=only study-related activities). Finally, we calculated social media use for (home-based) learning activities, including self-study assignments, self-study activities and exam preparation.

 

Out of the 429 students eligible for participation, 259 students (60% respondent rate, 28.2% men, 19.3±1.1 years, 81.5% medicine, 18.5% biomedical science) responded to our online questionnaire (Table 1). Students adopted a range of (multiple) technological devices, including mobile phone (7.8%), smartphone (91.0%), tablet (18.0%), laptop (81.6%) and desktop (12.1%). The use of (study-related use) of social media was lower during teacher-supervised activities compared to (home-based) learning-related activities (Table 1). Whatsapp was the dominant domain of social media during teacher-supervised activities (89%, 67% and 77% for lectures, interactive lectures and group assignments, resp.), followed by internet (78%, 48% and 79%, resp.), Facebook (73%, 39%, and 54%, resp.), SMS (34%, 18%, and 26%, resp.) and Twitter (13%, 8%, and 10%, resp.). During (home-based) learning activities, internet was most commonly used (98%, 96% and 95% during self-study assignments, self-study and exam preparation, resp.), followed by Whatsapp (92%, 89% and 84%, resp.), Facebook (93%, 86%, and 82%, resp.), SMS (44%, 39% and 34%, resp.) and Twitter (14%, 14% and 11%, resp.). These data demonstrate the high frequency of social media use, which depend on the types of (learning) activities, with only 30% of its use being study-related.

 

In subsequent analysis we explored whether social media use during teacher-supervised activities was related to exam results. For this purpose, we divided the students into 4 groups based on high versus low social media use (i.e. above/below the median “social media”-score) and made further distinction in high versus low study-related use of social media (using the median score again). This resulted in 4 distinct social media groups: 1) low user + low study-related, 2) low user + high study-related use, 3) high user + low study-related, and 4) high user + high study-related (Table 1). Interestingly, no significant differences were found in exam results between groups (Table 1, P=0.45). Moreover, we found no significant correlation between the (study-related) “social media”-score and exam result (r2=0.002 and 0.007, P=0.49 and 0.18, respectively). These results suggest that social media use during teacher-supervised activities does not relate to exam results.

 

Using the median-score to divide our groups may have resulted in relatively mild differences in (study-related) use of social media between groups. For this purpose, we performed additional analyses in which we compared the highest 10% versus lowest 10% in (study-related) social media use. Despite the presence of more pronounced differences in the social media score between groups (lowest 10%: 0.3±0.3 vs highest 10%: 7.3±1.0, P<0.001), we found no differences in exam result (7.2±1.0 vs 7.1±0.8, respectively, P=0.87). Similarly, no differences in exam result were found (7.5±1.1 vs 7.2±1.0, respectively, P=0.21) between students with the highest study-related use of social media (83±12%) compared to students with the lowest use (0.9±1.1%, P<0.001). These data reinforce our main observation that social media use does not relate to exam outcomes, a finding independent of the study-related use of social media.  

 

One may question whether social media use during (home-based) learning activities relates to exam results. Interestingly, the use of social media during (home-based) learning activities is markedly higher than during teacher-supervised activities, with a larger component being study-related (Table 1). Furthermore, moderate correlations were present between (study-related) use of social media in teacher-supervised versus (home-based) learning (r=0.39 and r=0.66, respectively, both P<0.05). Subsequently, we have repeated the original analysis, but now used the (study-related) “use of social media”-score during (home-based) learning activities. This analysis confirmed our original analysis, in that no significant differences were found between the exam results between groups (ANOVA P=0.67).

 

Some limitations of our study must be considered. The use of online questionnaires may have introduced selection bias of students with stronger interest around this topic. In addition, the self-reported component may have caused social desirability bias, leading to an underestimation of the true use of social media (and overestimation of the study-related component of it). Nonetheless, we expect that such bias similarly affected high and low-responders to the questionnaires, thereby unlikely altering the main outcomes of our study.

 

Studies have reported both negative impact (e.g., attention, (superficial) learning, memory) (Khan et al., 2009; Wallace, Clark and White, 2012; Uncapher, M and Wagner, 2016) and positive effects (e.g., collaborative approach) (Bullock and Webb, 2015). Nonetheless, results from our observational study among 259 Dutch (bio)medical students found no relation between social media use during teacher-supervised activities and exam results, a finding independent of whether the use of social media was study-related. Acknowledging the positive and negative effects of social media use, other factors than social media use may be more relevant in influencing academic performance, such as (intrinsic and extrinsic) student characteristics (Woolf, Potts and McManus, 2011; Adam et al., 2015; Hayden, Jeong and Norton, 2016). The practical implication of our finding is it is supportive for the various initiatives integrating mobile technology in teacher-supervised activities to improve learning experiences.

 

Table 1: Impact of (study-related) social media use on exam results in students (bio)medical science (n=259), divided into groups based on high/low “social media”-use and high/low study-related social media use. Comparisons were made using a one-way ANOVA (for continuous data) or Chi-square (for categorical data). Post-hoc significantly different for *low vs high social media use or #low study-related vs high study-related (P<0.05, including correction for multiple comparisons using Bonferroni).

 

 

 

Pooled

Low user

High user

 

Subject characteristics

Low study-related

High study-related

Low study-related

High study-related

P-value

Number of students (n)

259

63

80

65

51

 

Age (years)

19.3±1.1

19.4±1.3

19.5±1.4

18.8±0.9*

19.4±1.3

0.01

Sex (% female)

71.8

77.8

68.8

73.8

66.7

0.52

Medicine / Biomedical Science (%)

81.5 / 18.5

75 / 25

82.5 / 17.5

89.2 / 10.8

78.4 / 21.6

0.18

Times participated (% first time)

99.2

100

97.5

100

100

0.26

Exam

7.2±1.0

7.1±0.9

7.4±1.0

7.1±1.0

7.2±0.9

0.45

Social media use (teacher-supervised)

 

 

 

 

 

Social media-score (0-15)

3.4±2.0

2.1±0.9

1.8±1.0

5.5±1.4*

4.9±1.4*

<0.001

 Lecture (0-15)

4.5±2.6

3.1±1.4

2.4±1.6

7.0±1.8*

6.2±1.9*

<0.001

 IL (0-15)

2.2±1.9

1.2±1.0

0.8±1.0

3.8±1.7*

3.6±1.7*

<0.001

 GA (0-15)

3.6±2.5

2.1±1.4

2.3±1.5

5.7±2.4*

5.0±2.2*

<0.001

Study-related social media use (%)

30±24

11±8

54±22#

12±7

43±15#*

<0.001

 Lecture – study-related (%)

19±21

8±10

32±27#

8±7

29±20#

<0.001

 IL – study-related (%)

20±26

6±7

37±33#

8±9

39±28#

<0.001

 GA – study-related (%)

46±31

22±18

74±24#

22±15

62±20#*

<0.001

Social media use (learning activities)

 

 

 

 

 

Social media-score (0-15)

6.1±2.2

5.1±1.8

5.0±1.8

7.5±2.2*

7.2±2.0*

<0.001

 SSA (0-15)

6.7±2.2

5.8±1.5

5.6±2.0

8.2±2.1*

8.0±1.9*

<0.001

 Self study (0-15)

6.0±2.4

5.0±1.7

4.8±2.0

7.4±2.4*

7.2±2.4*

<0.001

 Exam preparation (0-15)

5.6±2.6

4.6±2.0

4.6±2.0

7.0±2.9*

6.7±2.5*

<0.001

Study-related social media use (%)

56±22

50±24

67±18#

44±22

62±18#

<0.001

 SSA – study-related (%)

50±23

44±22

60±20#

40±20

53±23#

<0.001

 Learning– study-related (%)

55±26

50±28

65±22#

42±24

60±21#

<0.001

 Exam preparation – study-related (%)

  65±25

59±29

77±20#

53±26

73±19#

<0.001

IL: interactive lecture, SSA: self study assignment, GA: group assignment 

Take Home Messages

We found no evidence that the amount and/or type of social media use during teacher-supervised activities relate to exam results in a cohort of Dutch (bio)medical students.

Notes On Contributors

MTEH, ALMV and TMHE designed the study. All authors contributed to the collection of the data. DHJT and TMHE performed the statistical analyses. All authors contributed to data interpretation. DHJT and TMHE drafted the manuscript. MTEH and ALMV critically revised the manuscript and contributed to the scientific discussion of the data.

Acknowledgements

The authors did not receive funding for the present study.

Bibliography/References

Adam, J., Bore, M., Childs, R., Dunn, J., et al. (2015) 'Predictors of professional behaviour and academic outcomes in a UK medical school: A longitudinal cohort study', Med Teach, 37(9), pp. 868-80. https://doi.org/10.3109/0142159X.2015.1009023

 

Bullock, A. and Webb, K. (2015) 'Technology in postgraduate medical education: a dynamic influence on learning?', Postgrad Med J, 91(1081), pp. 646-50. https://doi.org/10.1136/postgradmedj-2014-132809

 

Cheston, C. C., Flickinger, T. E. and Chisolm, M. S. (2013) 'Social media use in medical education: a systematic review', Acad Med, 88(6), pp. 893-901. https://doi.org/10.1097/ACM.0b013e31828ffc23

 

Ellaway, R. H., Fink, P., Graves, L. and Campbell, A. (2014) 'Left to their own devices: medical learners' use of mobile technologies', Med Teach, 36(2), pp. 130-8. https://doi.org/10.3109/0142159X.2013.849800

 

Hayden, L. J., Jeong, S. Y. and Norton, C. A. (2016) 'An Analysis of Factors Affecting Mature Age Students' Academic Success in Undergraduate Nursing Programs: A Critical Literature Review', Int J Nurs Educ Scholarsh, 13(1), pp. 127-138. https://doi.org/10.1515/ijnes-2015-0086

 

Khan, N., Coppola, W., Rayne, T. and Epstein, O. (2009) 'Medical student access to multimedia devices: most have it, some don't and what's next?', Inform Health Soc Care, 34(2), pp. 100-5. https://doi.org/10.1080/17538150902779550

 

Uncapher, M. R., M, K. T. and Wagner, A. D. (2016) 'Media multitasking and memory: Differences in working memory and long-term memory', Psychon Bull Rev, 23(2), pp. 483-90. https://doi.org/10.3758/s13423-015-0907-3

 

Wallace, S., Clark, M. and White, J. (2012) ''It's on my iPhone': attitudes to the use of mobile computing devices in medical education, a mixed-methods study', BMJ Open, 2(4). https://doi.org/10.1136/bmjopen-2012-001099

 

Woolf, K., Potts, H. W. and McManus, I. C. (2011) 'Ethnicity and academic performance in UK trained doctors and medical students: systematic review and meta-analysis', BMJ, 342, p. d901. https://doi.org/10.1136/bmj.d901

Appendices

None.

Declarations

There are no conflicts of interest.
This has been published under Creative Commons "CC BY-SA 4.0" (https://creativecommons.org/licenses/by-sa/4.0/)

Ethics Statement

Ethical approval was waived by the local Institutional Review Board of the Radboud University Medical Center due to low burden of the study proposal (questionnaires). Nevertheless, the ethical principles of the Declaration of Helsinki were taken into account during the study design, data collection and data analysis phases, whereas all students provided written informed consent upon enrolment in the study.

External Funding

This paper has not had any External Funding

Reviews

Please Login or Register an Account before submitting a Review

Ken Masters - (10/03/2019) Panel Member Icon
/
This assessment of the relationship between social media usage and exam results is well-timed on a topic that is of interest. The paper does, however, suffer from some serious flaws.

Some issues
• Perhaps the authors can give some details of the questionnaire and explanations given to the students, because, without these, their results are really confusing: “Whatsapp was the dominant ... (89%, 67% and 77% for lectures, interactive lectures and group assignments, resp.)… followed by internet (78%, 48% and 79%, resp.)” Given that accessing WhatsApp, Facebook, etc. all require accessing the internet (sometimes through apps, sometimes through browsers), how can internet be less than WhatsApp? This may be so if the researchers are using “Internet” in a specific sense, but no specific sense has been defined in the paper. (Earlier, the internet is described as a domain of social media, and that really does need explanation).
• Although the authors are careful to speak of relation rather than causation, they do slip into implying that their study is one of causation. The sentence “Studies have reported both negative impact (e.g., attention, (superficial) learning, memory) (Khan et al., 2009; Wallace, Clark and White, 2012; Uncapher, M and Wagner, 2016) and positive effects (e.g., collaborative approach) (Bullock and Webb, 2015).” This sentence speaks only of negative impact, but does not says of what? The reader is left to interpret, perhaps incorrectly, of social media usage, and perhaps on exam results. Table 1’s description further undermines the authors use of “relation” with “Impact of (study-related) social media use on exam results in students (bio)medical science” This rather clearly indicates that the authors believe that they are studying the impact of study-related social media use on exam results, when, in fact they are not. They are studying the relationship only.
• The final comment “The practical implication of our finding is it is supportive for the various initiatives integrating mobile technology in teacher-supervised activities to improve learning experiences” is highly problematic. I’m afraid that statement simply cannot be made. Given that there is no positive or negative relationship between social media use and exam results, how would that support “initiatives integrating mobile technology in teacher-supervised activities to improve learning experiences”? If there had been a positive relationship, then the next stage would be to determine whether or not causation exists, and in what direction. Only then, would the researchers know whether their study supported “initiatives integrating mobile technology in teacher-supervised activities to improve learning experiences”.

Two other points:
• The statistics in the text should be reported in raw numbers followed by percentages, not percentages only.
• In several places, the authors have placed phrases in brackets, although it is not clear why (e.g. “The use of (study-related use) of social media was lower during teacher supervised activities compared to (home-based) learning-related activities”). And in numerous other places. What is the purpose of these brackets? This bracketing is repeated several times during the paper but does not appear to have a purpose, only serves as an irritation.

So, the study is an interesting read, but does need to be tidied, and the emphatic conclusion needs to be removed.
Gulshat Kemelova - (19/10/2018) Panel Member Icon
/
This case study is very interesting and unexpected. In fact, that nowadays we can read a lot of papers about using social media and mobile technology to improve education overall. Some people believe that mobile technology can help learners in the study of Medicine or other subjects. I would like to involve in this study and we could compare Dutch medical students with Kazakhstan medical students. Because this way interested me and my colleagues, who actively use social media in teaching and learning.
This article is quite clear and well described the study results and it may be of interest to a wide range of readers.