In the age of digital technology, Wikipedia has emerged as a widely-used resource for health information (Heilman et al., 2011). After receiving a diagnosis, patients who search online for additional medical information often use Wikipedia for a concise summary of their symptoms or conditions (Heilman and West, 2015). Like these patients, many physicians and nearly all medical students report using Wikipedia to look up medical information (Hughes et al., 2009; Allahwala, Nadkarni and Sebaratnam, 2013).

Wikipedia is often presumed to be unreliable because of its open source collaborative editing structure. Many medical educators discourage the use of Wikipedia, and instead direct students to use resources that require paid subscriptions. Since students are consulting Wikipedia despite these admonitions, there is a discontinuity between medical student practices and the prevailing medical education paradigm. As patients frequently cannot access paid medical databases, there is a similar disparity between “physician-facing” and “patient-facing” information sources.

A novel alternative to teaching medical student Wikipedia abstinence is to teach students how to evaluate and improve the quality of articles under an expert facilitator. This confers at least two benefits: 1) students develop critical thinking and reasoning skills when using online resources, and 2) patients, laypeople and future students find more accurate information on Wikipedia. A few early-adopting schools offer courses on medical Wikipedia editing, though the pace of adoption in other institutions has been slow (Joshi et al., 2019). Limitations include a lack of faculty enthusiasm and insufficient time in the curriculum for an additional course. 

In this report, we detail an alternative that bypasses such limitations via a decentralized course. Specifically, we developed an online course, run by a medical school faculty member and a medical librarian at two different institutions, that instructed students on how to 1) evaluate the reliability of a Wikipedia article and the referenced material, 2) assess and incorporate new references, 3) improve poor quality medical articles, and 4) interact through a peer review process. This approach may reduce barriers to training larger numbers of medical students to evaluate and improve health information on Wikipedia, addressing a critical need as the relatively small number of core medical editors continues to decline (Heilman and West, 2015).

Osmosis is a medical and health sciences education technology platform used by 800,000 active users. In January 2017, Osmosis received grants to promote open educational pedagogy—a curricular strategy where students learn by producing open educational resources (Hewlett Foundation, 2018). Leveraging the large global network of Osmosis-subscribing medical students, Osmosis’ Director of Open Learning Initiatives created a novel Wiki-editing course structure. The course lasted 4 weeks in the summer of 2019. Students were invited to apply for the course by filling out a GoogleForm, stating their motivation to take part in such a course, as well as prior academic or teaching experience.

The course was run by a faculty member as well as a librarian, and began with an entirely videoconference-based opening session where students were introduced to becoming a Wikipedian in medical school, as well as their peers. Students were then expected to complete nine training modules on the Wiki Education platform (categorized broadly as “the basics of Wikipedia” and “editing Wikipedia as a health-professional”) and identify a single Wikipedia page to edit for the duration of the course by the second session. Students could work on a page individually, or in groups with clearly defined roles. During the second session, work plan strategies were discussed as well as expectations for weekly work-in-progress (WIP) videoconference meetings and a peer-review in the penultimate week. From this discussion a list of expectations for WIP meetings and the peer-review were drawn up and posted on the WikiEdu platform for guidance and accountability. Additionally, JN, the librarian, had a presentation on “Library resources, searching strategies and techniques” to inform students on how to assess as well as find reliable sources to edit their Wikipedia articles. The librarian remained available throughout the course for consultation on information seeking strategies, as well as credibility of online resources. Students were expected to develop a workplan and to post it on the “Talk” page of their chosen Wikipedia article, where the Wikipedia community could interact with and provide feedback on their proposed changes throughout the month. Students working in groups had to detail which individual was responsible for which section for a transparent and accountable collaboration. There were 4 weekly work-in-progress meetings as well as a peer-review in the penultimate week, which was then followed up by a response to the peer-review in the final week, allowing students to learn about the peer-review process.

To have the highest potential impact, students were advised to select pages that were rated top or high importance by WikiProject Medicine, or other health related WikiProjects, as well as relatively low on the Wikipedia’s quality rating (C or Start quality). All details of the course structure are available on Wikipedia itself (Azzam, 2019). To assess our course, we invited students to complete a 19-item post course survey about their experiences. The University of California, San Francisco internal review board approved this study of our novel curricular structure.

A total of 29 students submitted the first part of a 2-part application. Of these, 19 completed part 2 and 18 were accepted.  One individual who completed the application was not accepted and one individual who did not formally complete the application process was enrolled in the course. A total of 12 students attended orientation, of which 11 completed all training modules and posted workplans on Wikipedia. Though only 10 articles were assigned, students enrolled in the course voluntarily edited an additional 18 articles. Extracurricular editing typically involved small improvements in grammar and punctuation. To the assigned articles, a total of 8,775 words and 119 higher-quality references were added whilst 35 lower-quality references were removed. The 11 assigned Wikipedia pages were viewed 175,022 times during the course. A total of 10 students submitted peer reviews of an article edited by a fellow student and completed the course in its entirety. Three of the 11 students continued to edit Wikipedia after the course was completed. Table 1 provides a detailed breakdown of student impact on Wikipedia’s content.

We queried all 10 students who completed the course and 9 completed our survey (90% response rate). Student responses were overwhelmingly positive--all agreed that courses such as these are a good investment into the future of healthcare. One student's responses to the quantative section of the survey were discounted due to suspected human error, as they were in clear contradiction to what the student filled out in the short-answer form.   Figure 1 summarizes key quantitative survey results.

Figure 1- Summary of student’s views on Wikipedia and the course from post-course Survey

Students felt a sense of ownership towards the work they were producing on Wikipedia and reported a growing trust in health-related content on Wikipedia. To evaluate free text survey items, we consolidated students’ responses into themes. Table 2 summarizes these themes with representative exemplar quotes.

The post-course survey provided valuable feedback on how students felt the course could be improved. Numerous student responses recommended that extending the course would improve student experience. One student recommended more time specifically for researching which article to select. A potential benefit for allowing a longer selection period could be increased motivation towards their chosen topic.

A couple students suggested more collaborative editing efforts (e.g. multiple students working together on a single article). It’s worth noting that students were allowed to work in groups during this course, but most opted not to do so. This may be due to a limited time getting to know each other before starting the editing process. Future iterations of this course could consider how to implement more groupwork into the course structure.

Students were encouraged on to work on lower-rated articles. Low quality articles typically have few contributing editors, which diminishes opportunities for discussion during coursework. Another drawback was less engagement from the Wikipedia community.

Enrolled students hailed from 3 different continents, leading to challenges associated with time-zone coordination (e.g. what time to hold video-conference sessions). We also limited eligibility to Osmosis-subscribing medical students, which may not accurately represent the global pool of medical students.

While learning how to improve Wikipedia, students developed critical thinking through evaluating referenced material and participating in peer review. These future health practitioners also learned how to navigate Wikipedia’s quality ratings. This skill can be shared with future patients to enable independent evaluation of health information on Wikipedia. The outcome of greatest potential public health impact is the direct improvements to health-related Wikipedia content, which may be read by patients, laypeople and future students. By opening a course to a global cohort, students from underserved regions and countries will be able to network and learn with peers from wealthier areas. Together, these outcomes serve to bridge a gap in the flow of information between the medical profession and the communities in which we serve, in both the privileged and underdeveloped world.