Research article
Open Access

Wikipedia Editing Courses at Three US Medical Schools in the 2017-2018 Academic Year

Mihir Joshi[1], Rafael Verduzco[1], Sara Yogi[1], Maite Garcia[1], Sanskriti Saxena[1], Sean Tackett[2], Nadine Dexter[3], Angela Dempsey[4], Evans Whitaker[5], Amin Azzam[6][a]

Institution: 1. University of California, Berkeley; School of Public Health, 2. Johns Hopkins Bayview Medical Center, 3. Harriet F Ginsburg Health Sciences Library, University of Central Florida, 4. Department of Obstetrics and Gynecology, Medical University of South Carolina, 5. Library, University of California San Francisco, San Francisco, 6. University of California, San Francisco School of Medicine
Corresponding Author: Prof Amin Azzam ([email protected])
Categories: Educational Strategies, Scholarship/Publishing, Students/Trainees, Teaching and Learning, Technology
Published Date: 03/07/2019

Abstract

Introduction: Most medical students use Wikipedia, yet most medical schools do not train students to improve Wikipedia or use it critically.

 

Methods: During academic year 2017 - 2018, the University of California San Francisco School of Medicine, Medical University of South Carolina & University of Central Florida College of Medicine offered credit-bearing Wikipedia-editing courses to 4th year medical students. We sought to assess student experiences across these schools and courses.

 

Results: 78 students participated in the courses, making 3,368 edits, adding 155,100 words to 71 health-related Wikipedia pages. Collectively student contributions were viewed 2,688,500 times (37,866/page) during active dates of each course. Feedback from the 18 students who completed our study survey (23% response rate) reflected that students 1) increased their perception on Wikipedia reliability, 2) increased their ability to practice evidence based medicine, 3) were positively inclined to encourage peers to also edit Wikipedia, and 4) desired to contribute to public good. All students felt they improved their respective articles, and all but one felt that editing Wikipedia was a good investment in the future of healthcare. Despite our low response rate, qualitative comments suggest several students felt their courses impacted their professional identity as future physicians.

 

Conclusions: We encourage the implementation of similar Wikipedia-editing courses across all health professional schools. Study replication will help clarify generalizability of these findings. Standardizing outcome metrics (e.g. impact on students, impact on Wikipedia quality, and impact on readers of Wikipedia) will allow more robust assessment of the impact of Wikipedia-editing courses and assignments across health professional schools globally.

 

Keywords: Wikipedia; Open Educational Resources; Open Educational Practice; Crowdsourcing

Introduction

The public commonly uses Wikipedia for general information. In addition, 50-70% of physicians and over 90% of medical students use Wikipedia (Heilman and West, 2015). As of March 2017, Wikipedia had nearly 30,000 articles on medical topics in English, and 164,000 in other language Wikipedias (Shafee et al., 2017). Collectively these articles are viewed more than 10 million times per day (Shafee, Mietchen and Su, 2017). However, the website carries a negative stigma in content integrity and is criticized for having a lack of traditional editorial controls (Metcalfe and Powell, 2011). To protect content from inaccuracies (e.g. vandalism, self-promotion, factually incorrect content), Wikipedia’s quality control processes include the use of watchlists, recent change tracking, bots, special content page protection, filters, blocking, and banning problematic editors (Metcalfe and Powell, 2011). Above all, it is fundamentally the work of human volunteer editors that has kept Wikipedia’s content reliable to date. 

 

Extensive collaboration is necessary for a Wikipedia article to become accurate and comprehensive. Wikipedians aggregate around WikiProjects-- defined as collections of individuals interested in particular topics on Wikipedia. WikiProject Medicine is a group of about 300 active experienced editors, many of whom are health professionals (Wikiproject Medicine, 2018). Individual volunteers within WikiProject Medicine work to improve the currently 43,333 English Wikipedia health content pages through collaborations with libraries, scholarly journals, and organizations such as Cochrane, and Cancer Research UK (Wikiproject Medicine, 2018a). Translators Without Borders translates these health-related English language Wikipedia pages to other language Wikipedias (Wikiproject Medicine, 2019a).  “Internet-in-a-box”—a low-cost digital library designed to provide access to the internet offline—works with WikiProject Medicine to provide access to Wikipedia’s health-related content in places where access to the internet is limited or controlled (Wikimedia, 2019). Lastly, WikiProject Medicine has designed an offline Medical Wikipedia app available in 10 languages (Wikiproject Medicine, 2019b). When aligned with open-access publishing and indexing, this “collaboration chain” provides communities around the world access to high quality health information in their preferred languages and for free (Heilman et al., 2011). Consequently, Wikipedia has the capacity to be more than a resource for general information-- it is a “public health platform” that makes it possible for established and health-professionals-in-training to contribute to population-level health (Heilman et al., 2011).The current rate-limiting step is the limited number of active volunteer editors participating in WikiProject Medicine.

 

The Wiki Education Foundation (Wiki Ed) is a non-profit organization promoting Wikipedia integration into higher education coursework (Wiki Education Foundation, 2018a). Wiki Ed has a dashboard through which instructors can designate and customize learning modules for students, track student progress and organize other course content (such as work-in-progress sessions and assignment deadlines) (Wiki Education Foundation, 2018b). Since 2010, over 63,000 students across over 500 universities in North America have contributed over 30 million words to over 68,000 Wikipedia articles  (Blumenthal, 2017; Wiki Education Foundation, 2018b). Editing Wikipedia within higher education provides opportunities to improve open access health content used for professional and personal reasons alike. Health professional students who edit Wikipedia apply their newly acquired knowledge and information-seeking and appraisal skills. And their effort goes onto a platform that is public-facing, which challenges these emerging professionals to present technical content in comprehensible terms. Furthermore, faculty and librarians can customize Wikipedia-editing courses and/or assignments to local contexts.

 

We published an earlier innovation report summarizing early experiences of the first US medical school course dedicated entirely to Wikipedia-editing (Azzam et al., 2017). In writing for Wikipedia, those medical students found it challenging to balance the needs of the general population with health professional audiences. Those results were limited, as it was a single institution study with early adopting faculty and medical students.  But by November 2018, at least 10 health professional schools across 5 countries and 6 health professions had developed Wikipedia editing courses or assignments, most using the Wiki Ed course dashboard.  We sought to evaluate the impact of this emerging curricular approach by querying students who completed some of these courses.

 

Course Structures

The Wiki Education Foundation restricts its supportive function to universities in North America.  Consequently, for our study we limited inclusion criteria to US medical schools that offered credit-bearing courses for fourth year medical students dedicated primarily or exclusively to editing health content on Wikipedia.  In the 2017 - 2018 academic year, the entire eligible pool included courses at University of California San Francisco (UCSF), Medical University of South Carolina (MUSC) and University of Central Florida (UCF). The MUSC elective was offered once, the UCSF elective twice, and the UCF elective offered 3 times. These courses were all co-led by faculty and librarians at each medical school. Each month-long course began with an in-person orientation and each instructor assigned Wiki Education Foundation-produced learning modules (e.g. training on plagiarism, WikiProject Medicine style guidelines, how to add citations, finding open-access images). Each course additionally held synchronous work-in-progress sessions weekly (though students were allowed to participate remotely via video conference). During these work-in-progress sessions, faculty and medical librarians provided task guidance and answered general questions (e.g. formatting, what content to include and where, what content to deliberately exclude, how to engage with Wikipedians, etc.). 

 

Each course was designed and executed via the Wiki Education Foundation course dashboard. A screenshot of one of these courses’ dashboard is provided as Figure 1. 

 

Figure 1: UCSF Spring 2018 WikiEdu Dashboard Example.

 

Student editors created Wikipedia editor logins so all of their efforts could be appropriately attributed. A screenshot example of a course dashboard demonstrating student author attribution is provided as Figure 2. 

 

Figure 2: Tracking of editor contributions to Wikipedia page on WikiEdu Dashboard

 

In some courses, students could only select articles from a restricted list provided by their faculty instructors. In all courses, students were encouraged (but not required) to select high-importance but lower quality articles-- as cataloged by WikiProject Medicine (WikiProject Medicine, 2018a). Students were also permitted to work in teams. Through the Wiki Education Foundation course dashboards, faculty tracked the progress of individual student editors as well as efforts of the course en masse regarding aggregate effort. All courses required students to peer review at least one classmate’s work. The final day of each of the courses required students to present their revised article to their classmates and reflect on their accomplishments and lessons learned. Detailed course structures are publicly available through Wikipedia itself (Wiki Education Foundation, 2017a; Wiki Education Foundation, 2017b; Wiki Education Foundation, 2017c; Wiki Education Foundation, 2018c; Wiki Education Foundation, 2018d; Wiki Education Foundation, 2018e).

Methods

The UCSF Institutional Review Board approved this data collection protocol across the 3 participating medical schools. While each school conducted its own internal course evaluations, we sought to assess variation and similarities across all courses. Post-course student feedback was therefore evaluated via a Qualtrics survey designed by the authors in partnership with Hanover Research. Hanover Research is a research and analytics firm that partners with higher education institutions to provide custom analysis for academic solutions (Hanover Research, 2018). Our research was conducted with the explicit permission of the local medical school faculty. But because our eligible student pool was quite small, we chose to use Hanover Research to assist with data collection to maximize student candor and minimize any perception of obligation to participate in our study. The 16-item mixed-methods survey included quantitative and qualitative inquiries through Likert-style and free-text items respectively. We did not pilot test or validate the survey items, which are provided as Appendix 1. The survey was launched on May 8th, 2018 and results were collected electronically. Non-responding students received intermittent email reminders to complete the survey. Data collection was completed by June 8th, 2018.

Results/Analysis

Student Impact on Wikipedia

A total of 78 fourth year medical students participated across the six courses (range 4 - 37). Collectively the students contributed to 71 articles (range 3 - 36/course), made 3,368 edits (average 43/student), and added 155,100 new words to their articles (average 1,988 words/student). Additionally students integrated 27 new images to help illustrate their articles. In total these 71 Wikipedia pages were viewed over 2,688,500 times during the active course dates (34,468 views/article; range: 6,320 - 61,200/course). Detailed breakdown of each course is provided in Table 1.

 

Table 1: Summary of US Fourth-Year Medical School Contributions to Wikipedia During Academic Year 2017-2018

School

Cycle

Enrolled Students

Wikipedia articles edited

Number of edits made (range/student)

Words added* (range/student)

Images added (range/student)

Views during course

(Range across articles)

MUSC

Fall 2017

12

12

868

(40 – 121)

35,000

(1090 – 5505)

2

(0 – 1)

612,000

(18,555 – 165,819)

UCF

Fall 2017

6

3

128

(7 – 52)

6,320

(22 – 2580)

1

(0 – 1)

44,300

(4,713 – 18,5780)

UCF

Winter 2018

4

3

94

(0 – 45)

7,380

(0 – 3174)

0

86,100

(4,549 – 42,638)

UCF

Spring 2018

6

5

157

(13 – 64)

10,900

(795 – 2493)

0

79,100

(2,174 – 30,455)

UCSF

Fall 2017

37

36

1,420

(2 – 129)

61,200

(299 – 5916)

14

(0 – 4)

1,380,000

(721 – 177,791)

UCSF

Spring 2018

13

12

701

(30 – 106)

34,300

(1552 – 4627)

10

(0 – 2)

487,000

(7,005 – 209,900)

TOTAL

 

78

71

3,368

155,100

27

2,688,500

*Words added ranges approximated through bytes added, with a conversion ratio in English of 5.175 bytes per word. This ratio is used across all Wiki Education Foundation courses.

 

Survey Results

Eighteen students completed our survey (response rate = 23%), including students from all 3 schools (3 from MUSC, 7 from UCF, and 8 from UCSF). All eighteen students felt confident they improved the content of their Wikipedia article and all would recommend their course to other students. All but one agreed that improving Wikipedia is a good investment in the future of healthcare. The cohort believed participation in their courses led to an increase in likelihood of editing Wikipedia in the future (n=15, 83%), ability to practice Evidence-Based Medicine (n=15, 83%), capacity to write in clear terms (n=15, 83%), perception of Wikipedia quality (n=14, 78%), and likelihood of recommending Wikipedia as a learning resource (n=14, 78%). Students ranked their reasons for selecting their respective articles. Figure 3 summarizes key highlights of the survey results.

 

Figure 3: General Student Perspectives on Wikipedia and Course.

 

When asked to describe the reliability of Wikipedia for patients, many students felt Wikipedia was only moderately to slightly reliable. One student wrote “It's moderately reliable for health information, but I would only use it as a starting point to talk with your doctor rather than using it to guide decisions.” Reasons for future editing varied, but the most recurring motivators were 1) to help the general public and 2) if the Wikipedia article was related to their field of interest. The most valuable lessons students learned were 1) the use of Wikipedia for quality/relevant information and 2) how the Wikipedia editing community operates. Five of the eighteen felt the class was so impactful that it influenced their professional identity as future physicians. Lastly, the two most frequently recommended improvements were 1) a more structured learning experience and 2) more peer/instructor communication. Exemplar quotes from students’ qualitative responses are provided in Table 2.

 

Table 2: Medical Student Representative Responses to Open-Response Survey Questions

Questions Posted:

Representative Responses:

What do you feel is the most valuable thing you learned from this course?

  • “I appreciated learning about the quality ratings on Wikipedia articles. Since this course, I have reviewed the quality ratings of articles I have used for medical purposes before referencing them to get a sense of the quality of the information provided.”
  • “Learning the inner workings of Wikipedia and the rigor with which the community edits the available content. The course provided much clarity into both the strengths and weaknesses of Wikipedia, and helped me reframe how I perceived Wikipedia as a distributor of medical content.”
  • “Understanding the concept of health knowledge accessibility as a major issue in the realm of medical care -- practicing writing concisely and in lay-language to make medical knowledge more digestible.”
  • “Delving deep into a focused topic as well as learning how to synthesize medical topics into a way understandable to the general public.”

What would have improved your learning experiences throughout this course?

  • “More personal connection to medical students and/or healthcare providers who rely on this resource. Do they have any particular requests for articles that they would want updated or improved?”
  • “A factor that could not be controlled, but [hearing from] previous participants in the course…could have helped improve the advice given by instructors.”
  • “Actually editing content and spending time engaging with the Wikipedia community was the most valuable aspect of the course. The course did an excellent job exposing us to both, but anything that can increase master Wikipedian exposure would improve the experience further.”
  • “I found it somewhat difficult to work with other editors in the Wikipedia community. Often times, I would spend extensive time editing my article only to have all of my edits undone within minutes with very minimal explanations. This became pretty frustrating really quickly.”

What would motivate you to continue contributing to Wikipedia as an editor beyond this course?

  • “If there was a topic I was particularly passionate about, or quite frankly if I'm working for an organization in the future I would absolutely want to think about that org's wiki presence.”
  • “The only reason I would consider contributing to Wikipedia in the future is if it was a component of my job. I likely will not volunteer my time in the future to this cause.”
  • “Knowing that patients frequently read articles to learn more about their own health and to contribute to the efforts of Wikipedia to provide access to educational materials to underserved communities and third world countries.”
  • “Some form of continuing education credits or recognition; professional societies with improving Wikipedia as an objective; CV publication”

In the language you would use speaking to your patients, describe the reliability of Wikipedia's health-related content.

  • “The articles on Wikipedia do contain a good deal of accurate information. This information comes from reliable sources which doctors use to remain up to date on information. Those sources are the best place to look for information, and Wikipedia is a great jumping off point. You are always invited to consult with me as your physician to explain anything you don’t understand or want to verify.”
  • “When it comes to common, well published, medical conditions/topics I would trust what you read on Wikipedia. The rare conditions, where editors may not regularly follow the page, I would suggest taking the information you read on Wikipedia with a grain of salt.”
  • “Wikipedia articles, more often than not, contain accurate, reliable and appropriately-referenced information. However, anyone can theoretically edit or change Wikipedia articles at any time. Therefore with Wikipedia, it is important to pay attention to the sources that are referenced in the articles”

How have your opinions about Wikipedia changed since completing the course?

  • “I believe Wikipedia is an excellent resource to quickly find information from well-organized, succinct articles and that those in the medical profession can greatly contribute to public health by dedicating time to editing articles.”
  • “There is more that goes into editing than I thought.”
  • “Honestly, my opinions about the quality of Wikipedia probably decreased through this course. After seeing some of the poorer quality articles on Wikipedia, it just made me more aware of the fact that you can't believe everything you read on Wikipedia.”

Please describe how this course influenced your professional identity

  • “I hadn't really thought about a responsibility to use my training, knowledge, expertise, etc. to cultivate a media environment with more accurate, actionable information for the general population.”
  • “I feel that contributing to the availability and reliability of medical information is a part of my role as a physician.”

 

Discussion

Impact on Medical Students

Based on both quantitative and qualitative responses, these Wikipedia editing courses appear to have demonstrated substantive impact on medical students, as well as their perceptions of the quality of Wikipedia as an open-access resource for health information. Students are challenged to translate their technical medical knowledge into language that is comprehensible, succinct and thorough. By interfacing with Wikipedia, students expand their physician identities in the context of a developing, digital workspace, as well as a collaborative platform involving health professionals and other editors. Participation in these courses may be the beginning of extensive careers involving improvement of Wikipedia’s health content.

 

Impact on Wikipedia

We did not systematically assess the impact of students’ work on Wikipedia quality. However, based on personal communication with the course faculty of record at each institution, during the time students were actively editing, their selected pages appeared to be significantly improved. WikiProject Medicine’s community of volunteer editors have consistently agreed with the edits and additions being made by students enrolled in these courses. Acknowledging the relatively recent course end dates, the contributions made by students do not appear to be actively vandalized or deleted.

 

Impact on Medical Education

These courses are part of several larger movements in higher education. Open Educational Resources (OER) are defined as free and openly licensed educational materials that can be used for teaching, learning, research, and other purposes (Hewlett Foundation, 2018; Creative Commons, 2018). A relatively newer term is Open Educational Practices (OEP).  One definition of OEP is the use of OER for teaching and learning in order to innovate the learning process (Wikipedia, 2019). Lastly, Free Open Access Medical Education (FOAM) can be defined as a community of open access resources for learning in medicine (Scott et al., 2014). All of these movements are influenced by larger forces such as crowdsourcing, social media, and the penetration of technologies in developed (and increasingly developing) countries. As these movements grow, there are calls for consortia of medical schools to pool resources (Le and Prober, 2018) as well as examples of networked communities of health professional students using free videos and adaptive learning platforms to make learning more efficient (Menon et al., 2017; Tackett et al., 2018). We believe these are all positive developments in the longitudinal trajectory of medical education.

 

Limitations

There are several limitations to our work. These results represent only three US medical schools, and we had a very low response rate to our survey.  This may have been due to the unexpected source (Hanover Research), but also due to the timing in students’ academic calendars (all were imminently graduating from medical school). Despite having student responses across all three participating schools, the low response rate may not represent an accurate distribution of opinions of the students on their respective courses. Furthermore, variations in these responses may arise from the inconsistencies between courses across different instructors. Each instructor likely implemented different teaching methods, causing even more variation between individual student experiences. We also did not independently review the content of students’ work for accuracy (though we presume all local faculty did). We did not grade the Wikipedia articles for quality, either before or after the students’ contributions. Despite these limitations, we believe that the qualitative data provides ample and important insight into student perspectives on contributing to Wikipedia within formal medical school curricula across several medical schools.

 

Areas for future study

Through the creation of the Wiki Education Foundation dashboard, Wikipedia editing classes and coursework and have become more accessible to students and faculty alike. Resources such as instructional videos and interactive learning modules in the Wiki Education dashboard, and collaborative student spaces within the Wikipedia editing interface can appeal to Millennial learners who place a premium on flexibility and active educational experiences (Eckleberry-Hunt, Lick and Hunt, 2018). These types of collaborations create a model environment for students across various health-related fields to work together to improve Wikipedia’s large, free information library without geographic barriers.  

 

Wikipedia editing as a teaching and learning strategy has expanded into other Health Professional schools, as has centralized tracking of progress. The Wiki Education Foundation has created an aggregate campaign called “Students in Health Professions." At the time of this writing (May 2019), the campaign included 857 students across 27 courses. Collectively those students added 613,000 words and 84 images to 406 health-related Wikipedia pages that had been viewed 16.1 million times since students began contributing (Blumenthal, 2017). Future multi-institutional and even global collaborative studies might systematically evaluate these courses’ impacts on their students, their faculty, and on the quality of Wikipedia itself.  

 

Currently, the Wikipedia editing courses use the dashboard feature to track individual student contributions and assess metrics such as number of edits, words, images and citations added. In order to evaluate each article, WikiProjects have previously created an assessment rubric of article quality. The grading scale ranges from “stub” to “featured article” and includes explicit criteria for the promotion of an article from one grade level to the next. Despite the presence of this standardized Wikipedia article quality rubric, students’ work in each medical school course is graded based on subjective criteria by individual instructors.  In most cases the courses were pass-fail (personal communication with course instructors). The Wiki Ed course dashboard provides a suggested mechanism to grade student’s efforts on Wikipedia assignments, but none of the faculty of these courses elected to use that grading mechanism (personal communication).

 

Standardizing both the assessment of students and their Wikipedia-editing efforts would provide more accurate assessments of student capacity to meaningfully contribute to Wikipedia’s health related content.  Yet standardization would also carry some risk, as the amount of effort necessary to improve articles varies based on multiple factors, such as the breadth of the topic, the baseline completeness prior to course start, and the student’s year in his/her training program. This again represents a rich area for potential future studies.

Conclusion

By March 2017 Wikipedia’s health content had grown to over 194,000 articles across 295 languages.  Collectively read >10 million times/day, this is the most viewed health and medical resource globally.  Health professions educators have begun to broaden their curricula by embracing Wikipedia-editing as a teaching & learning strategy. In the 2017 – 18 academic year, three US medical schools offered credit-bearing courses focused primarily on Wikipedia-editing.  Our survey of students participating in those courses suggests that these efforts had substantive impacts on students and Wikipedia quality. We invite other health professional schools to join this movement, as we believe these efforts advance Wikipedia’s mission-- to act as a comprehensive written compendium that contains information on all branches of knowledge (Wikipedia, 2018).

Take Home Messages

  • Wikipedia is a relied upon and increasingly reliable resource for medical information
  • There is a growth of infrastructure to execute and evaluate Wikipedia-editing courses and course assignments
  • Several medical schools have incorporated Wikipedia-editing as a formal part of their curricular offerings
  • Medical students editing health content on Wikipedia is a learning and collaborative practice
  • Wikipedia-editing courses within formal medical school curricula broaden students’ professional identities as future physicians

Notes On Contributors

Mihir Joshi, Rafael Verduzco, Sara Yogi, Maite Garcia, and Sanskriti Saxena were all undergraduate research assistants with Dr. Azzam at the University of California, Berkeley at the time of their participation in this research.

 

Sean Tackett is a General Internist at Johns Hopkins Bayview Medical Center and Research Director for Osmosis. His career aspiration is to contribute to improvements in the quality of health professions education internationally. Sean currently teaches curriculum development to Johns Hopkins faculty, has authored online modules on curriculum development and learning theories for Johns Hopkins School of Medicine, and has taught educational skills to faculty from China, Oman, Saudi Arabia, Vietnam, Syria, and Brazil. He also practices hospital medicine and teaches medical students and residents on the medical wards.

 

Nadine Dexter is the Director of the Health Sciences Library and Director of Medical Informatics at the University of Central Florida College of Medicine (UCF). She also is a core faculty in the Wikipedia-editing courses at UCF.

 

Angela Dempsey is an Associate Professor of Obstetrics and Gynecology and the Associate Dean for Curriculum, Clinical Sciences at the Medical University of South Carolina (MUSC).  She also is a core faculty in the Wikipedia-editing courses at MUSC.

 

Evans Whitaker is a Research Librarian at the Library of the University of California, San Francisco (UCSF). His scholarly interests include information literacy, sytematic reviews, referenced management, and clinical decision support. He also is a core faculty in the Wikipedia-editing courses at UCSF.

 

Amin Azzam is a full professor at 3 San Francisco bay area health professional schools: 1) University of California, San Francisco School of Medicine; 2) University of California, Berkeley School of Public Health; and 3) Samuel Merritt University. He is also a pedagogical consultant on two grants from the Hewlett Foundation to Knowledge Diffusion, Inc (DBA Osmosis). The grants are to promote open educational pedagogy. His role is to help other health professional schools (that are part of the network using Osmosis tools) design and launch Wikipedia-editing assignments and/or courses at their schools. 

Acknowledgements

This work is sponsored in part through grants from the William and Flora Hewlett Foundation. The authors wish to acknowledge Aparna Manocha and Anthony Sanabria for their contributions to the survey design. The authors thank all the student editors for their hard work in making Wikipedia better.

Bibliography/References

Azzam, A., Bresler, D., Leon, A., Maggio, L., et al. (2017) 'Why Medical Schools Should Embrace Wikipedia: Final-Year Medical Student Contributions to Wikipedia Articles for Academic Credit at One School', Acad Med, 92(2), pp. 194-200. https://doi.org/10.1097/ACM.0000000000001381

 

Blumenthal, H. (2017) 'What students learn from contributing to Wikipedia', WikiEdu. Available at: https://wikiedu.org/blog/2017/06/19/what-students-learn-from-contributing-to-wikipedia/   (Accessed: 13 October 2018)

 

Creative Commons. (2018) What is OER?. Available at: https://wiki.creativecommons.org/wiki/What_is_OER%3F (Accessed 26 November 2018).

 

Eckleberry-Hunt, J., Lick, D. and Hunt, R. (2018) 'Is Medical Education Ready for Generation Z?', J Grad Med Educ, 10(4), pp. 378-381. https://doi.org/10.4300/JGME-D-18-00466.1

 

Hanover Research. (2018) Higher Education. Available at: https://www.hanoverresearch.com/education-solutions/higher-education/ (Accessed: 13 October 2018)

 

Heilman, J. M., Kemmann, E., Bonert, M., Chatterjee, A., et al. (2011) 'Wikipedia: a key tool for global public health promotion', J Med Internet Res, 13(1), p. e14. https://doi.org/10.2196/jmir.1589

 

Heilman, J. M. and West, A. G. (2015) 'Wikipedia and medicine: quantifying readership, editors, and the significance of natural language', J Med Internet Res, 17(3), p. e62. https://doi.org/10.2196/jmir.4069

 

Hewlett Foundation. (2018) Open Educational Resources. Available at: https://hewlett.org/strategy/open-educational-resources/ (Accessed: 13 October 2018).

 

Le, T. T. and Prober, C. G. (2018) 'A Proposal for a Shared Medical School Curricular Ecosystem', Acad Med, 93(8), pp. 1125-1128. https://doi.org/10.1097/ACM.0000000000002194

 

Menon, A., Gaglani, S., Haynes, M. R. and Tackett, S. (2017) 'Using "big data" to guide implementation of a web and mobile adaptive learning platform for medical students', Med Teach, 39(9), pp. 975-980. https://doi.org/10.1080/0142159X.2017.1324949

 

Metcalfe, D. and Powell, J. (2011) 'Should doctors spurn Wikipedia?', J R Soc Med, 104(12), pp. 488-9. https://doi.org/10.1258/jrsm.2011.110227

 

Scott, K. R., Hsu, C. H., Johnson, N. J., Mamtani, M., et al. (2014) 'Integration of social media in emergency medicine residency curriculum', Ann Emerg Med, 64(4), pp. 396-404. https://doi.org/10.1016/j.annemergmed.2014.05.030

 

Shafee, T., Masukume, G., Kipersztok, L., Das, D., et al. (2017) 'Evolution of Wikipedia's medical content: past, present and future', J Epidemiol Community Health, 71(11), pp. 1122-1129. https://doi.org/10.1136/jech-2016-208601

 

Shafee, T., Mietchen, D. and Su, A. I. (2017) 'Academics can help shape Wikipedia', Science, 357(6351), pp. 557-558. https://doi.org/10.1126/science.aao0462

 

Tackett, S., Slinn, K., Marshall, T., Gaglani, S., et al. (2018) 'Medical Education Videos for the World: An Analysis of Viewing Patterns for a YouTube Channel', Acad Med, 93(8), pp. 1150-1156. https://doi.org/10.1097/ACM.0000000000002118

 

Wiki Education Foundation. (2017a) Expanding WikiProject Medicine Fall 2017. Available at: https://en.wikipedia.org/wiki/Wikipedia:Wiki_Ed/University_of_California,_San_Francisco/Expanding_WikiProject_Medicine_Fall_2017_(Fall_2017) (Accessed: 10 May 2019).

 

Wiki Education Foundation. (2017b) Expanding WikiProject Medicine MUSC Fall 2017. Available at: https://en.wikipedia.org/wiki/Wikipedia:Wiki_Ed/Medical_University_of_South_Carolina/Expanding_WikiProject_Medicine_MUSC_Fall_2017_(Block_5) (Accessed: 10 May 2019).

 

Wiki Education Foundation. (2017c) WikiProject Medicine Fall 2017 UCF COM. Available at: https://en.wikipedia.org/wiki/Wikipedia:Wiki_Ed/University_of_Central_Florida/WikiProject_Medicine_Fall_2017_UCF_COM_(Fall) (Accessed 10 May 2019).

 

Wiki Education Foundation. (2018a) Does editing Wikipedia change a student’s life?.  Available at: https://wikiedu.org/changing/students/ (Accessed: 10 May 2019).

 

Wiki Education Foundation. (2018b) Wiki Education Dashboard. Available at: https://dashboard.wikiedu.org/ (Accessed: 4 October 2018).

 

Wiki Education Foundation. (2018c) WikiMed Spring 2018 USCF SOM. Available at: https://en.wikipedia.org/wiki/Wikipedia:Wiki_Ed/University_of_California,_San_Francisco/WikiMed_Spring_2018_UCSF_SOM_(Spring_2018) (Accessed 10 May 2019).
 

Wiki Education Foundation. (2018d) WikiProject Medicine Spring 2018 UCF COM. Available at: https://en.wikipedia.org/wiki/Wikipedia:Wiki_Ed/University_of_Central_Florida/WikiProject_Medicine_Spring_2018_UCF_COM_(Spring) (Accessed 10 May 2019).

 

Wiki Education Foundation. (2018e) WikiProject Medicine Winter 2018 UCF COM. Available at: https://en.wikipedia.org/wiki/Wikipedia:Wiki_Ed/University_of_Central_Florida/WikiProject_Medicine_Winter_2018_UCF_COM_(Winter) (Accessed 10 May 2019).

 

Wikimedia. (2019). Internet-in-a-Box. Available at: https://meta.wikimedia.org/wiki/Internet-in-a-Box (Accessed: 10 May 2019).

 

Wikipedia. (2018) Purpose. Available at: https://en.wikipedia.org/wiki/Wikipedia:Purpose (Accessed 13 October 2018).

 

Wikipedia. (2019) Open Educational Practices. Available at: https://en.wikipedia.org/wiki/Open_educational_practices (Accessed: 10 May 2019)

 

Wikiproject Medicine. (2018a) Wikiproject Medicine. Available at: https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Medicine (Accessed 4 October 2018).

 

Wikiproject Medicine. (2018b) WikiProject Medicine/Stats/Top medical editors 2017 (all). Available at: https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Medicine/Stats/Top_medical_editors_2017_(all) (Accessed 16 November 2018).

 

Wikiproject Medicine. (2019a) Translation Task Force. Available at: https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Medicine/Translation_task_force (Accessed 10 May 2019).

 

Wikiproject Medicine. (2019b) Wiki Project Med App. Available at: https://meta.wikimedia.org/wiki/Wiki_Project_Med/App (Accessed: 10 May 2019).

Appendices

Appendix 1: Survey Items

  1. Please rate how strongly you agree or disagree with the following statements (Likert Scale: Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree)
    1. I feel confident that I improved the content of the Wikipedia page(s) I edited.
    2. I would recommend this course to other students in my program/school.
    3. Courses/assignments focused on improving Wikipedia are a good investment in the future of healthcare.
    4. Courses/assignments focused on improving Wikipedia should be a required part of health professional schools.
    5. I would recommend other students use Wikipedia as a reliable source for health-related information.
  2. Please indicate how important each of the following were in your decision to enroll in the course (Likert Scale: Significantly Unimportant, Unimportant, Neutral, Important, Significantly Important)
    1. Enjoyment of writing
    2. Wanting to learn more about a medical topic
    3. Wanting to learn how to edit Wikipedia
    4. Wanting to contribute to public good
    5. Need for flexible schedule
    6. Reputation of course
    7. Reputation of faculty instructor(s)
    8. Recommendation from other students
  3. Please rate how much each of the following items changed as a result of your participation in the course (Likert Scale: Strongly Decreased, Decreased, Neutral, Increased, Strongly Increased)
    1. Perception of Wikipedia quality
    2. Likelihood of recommending Wikipedia as a learning resource
    3. Likelihood of editing Wikipedia in the future
    4. Likelihood of encouraging peers to edit Wikipedia
    5. Ability to find high quality references
    6. Ability to practice Evidence-Based Medicine
    7. Capacity to write in clear terms
  4. Please indicate how important each of the following were in selecting the article(s) you edited (Likert Scale: Significantly unimportant; Unimportant; Neutral; Important; Significantly Important)
    1. Wikiproject Medicine importance & quality scales
    2. Relevance to my chosen specialty
    3. Desire to learn something new
    4. Desire to capitalize on my existing expertise
    5. Course Requirements
  5. To what extent did each of the following contribute to your learning in the course? (Likert Scale: Insignificantly, Very little, Neutral, Somewhat, Significantly)
    1. Faculty Instructor
    2. University librarian
    3. Peer students
    4. Wikipedia Community
  6. I had experience editing Wikipedia articles prior to this course. (Yes/No)
  7. People outside of the class have expressed interest in the work I’ve done with this course assignment. (Yes/No)
  8. What do you feel is the most valuable thing you learned from this course? (Open-ended)
  9. What would have improved you learning experiences throughout this course? (Open-ended)
  10. What would motivate you to continue contributing to Wikipedia as an editor beyond this course? (Open-ended)
  11. In the language you would use speaking to your patients, describe the reliability of Wikipedia’s health-related content. (Open-ended)
  12. Have your opinions on Wikipedia changed since completing this course? (Yes/No)
  13. How have your opinions about Wikipedia changed since completing this course? (Open-ended, if yes to 12)
  14. Did this course influence your professional identity as a future physician? (Yes/No)
  15. Please describe how this course influenced your professional identity/ (Open-ended, if yes to 13)
  16. We could like to conduct focus groups to learn more about your experiences across your host medical schools. Would you be interested in participating? (Yes/No)
  17. If we have questions about your answers may we contact you to clarify? (Yes/No)
  18. Please provide us with your preferred email address (optional)
  19. How old are you? (optional)
  20. What is your intended specialty (optional)

Declarations

There are some conflicts of interest:
Amin Azzam is a pedagogical consultant on two grants from the Hewlett Foundation to Knowledge Diffusion, Inc (DBA Osmosis). The grants are to promote open educational pedagogy. His role is to help other health professional schools (that are part of the network using Osmosis tools) design and launch Wikipedia-editing assignments and/or courses at their schools.
This has been published under Creative Commons "CC BY-SA 4.0" (https://creativecommons.org/licenses/by-sa/4.0/)

Ethics Statement

This study was reviewed by the University of California, San Francisco Human Subjects Protection Program Institutional Review Board (IRB) as study #18-24516 with reference #212707. It was approved on 25-March-2018 under expedited review.

External Funding

Amin Azzam is a pedagogical consultant on two grants from the Hewlett Foundation to Knowledge Diffusion, Inc (DBA Osmosis). The grants are to promote open educational pedagogy. His role is to help other health professional schools (that are part of the network using Osmosis tools) design and launch Wikipedia-editing assignments and/or courses at their schools. The grants are itemized here: 1. https://hewlett.org/grants/knowledge-diffusion-inc-dba-osmosis-for-promoting-open-educational-practice-through-scalable-technology/ 2. https://hewlett.org/grants/knowledge-diffusion-inc-dba-osmosis-for-promoting-open-educational-practice-through-scalable-technology-0/

Reviews

Please Login or Register an Account before submitting a Review

Mary Dankbaar - (17/08/2019) Panel Member Icon
/
This article offers an interesting perspective on open access information, linking it to medical education. It is a new and good idea to consider to have medical students examine and elaborate health information on Wikipedia. However, if other medical schools should also consider to implement this in their medical curricula, as the authors are suggesting, a number of questions on the educational value which remain open after this study, should be answered in a follow-up study.
1. What are the learning goals of training students to improve online Wikipedia content? Are these mainly focused on developing scientific skills, or developing written communicating skills to inform patients? It would have been interesting to elaborate on the possible educational benefits, and compare these with other (credit-bearing) learning activities which could also be effective to reach those goals. A comparison study, including assessment of learning effects among students, would have been valuable and enable decisions by other medical schools. Now it remains unclear what the comparative educational value is.

2. 23% of the students gave their opinion on the Wikipedia initiative. While it certainly is interesting to read about their views, we don’t know what the majority of students feel and whether they actually learned from it. This is an important limitation which the authors mention, but they maintain the conclusion that these data provide insight in (the majority of the) students’ perspective on this, and formulate generic conclusions that ‘Wikipedia-editing courses … broaden students’ professional identities as future physicians’. There seems little evidence in the study data for that.

3. The results are based on students’ opinions regarding their ability to practice EBM and write clearly. These competences were not assessed by teachers, nor did students receive formative feedback from teachers on their written products, only by peers. It is quite hard for students to develop a good idea on how adequate their contributions were and what they learned from it without this kind of expert feedback.
Barbara Jennings - (02/08/2019) Panel Member Icon
/
I strongly recommend this article to colleagues interested in open educational resources; to those of us who use Wikipedia regularly, and have wondered about the creation and curation of its pages; and to medical educators who are interested in creating engaging and meaningful assignments for their learners. In short, I will be recommending it to pretty much everyone.
I learned a whole new language when I read the introduction to this article: e.g. Wikipedians, Wikiprojects, WikiProject Medicine, and it made me much more aware of the complex collaborative networks that feed into the production of the visible pages on medical topics, which are “viewed 10 million times per day”.
Curation, quality control, and editing are important considerations and the authors comment on the stigma about Wikipedia and the inaccuracy that can arise from vandalism, self-promotion, and low quality entries.
The educational interventions that were created and studied by these authors involved the editing of Wikipedia entries, particularly those identified to be important but low-quality. These could be team-based activities.
I found several aspects of the lesson design interesting, e.g. the use of the Wiki Ed (virtual learning) platform, that is for the integration of the Wikipedia Project into higher education (but which is unfortunately restricted for use in North America.) Also, the sessions were co-led by the course faculty member and the librarian – thus highlighting the central importance of expertise in information curation to this endeavour.
Aspects of the intervention were then studied, including editorial output from the whole cohort, and an evaluation of student experience using a carefully designed mixed methods survey.
The editing output by the students was impressive. The results of the student evaluation must have been both interesting and very encouraging to the authors. However, the rich survey was only completed by 18 (23 %) of learners. I wonder if the limited time students were given to respond affected this (only one calendar month?)
I am keen to see how this project develops and want to know from the authors if this article has provoked other US and global collaborations to extend this project?
Possible Conflict of Interest:

For Transparency: I am an Associate Editor of MedEdPublish.

Irine Sakhelashvili - (31/07/2019) Panel Member Icon
/
I have read this article with great interest because of my personal attitude to Wikipedia as a source of medical education. My colleagues and I are always against using Wikipedia as a scientific source. Yes, you can find almost everything on Wikipedia, especially if we are talking about content in English, but there are two or more “But”:
First, there are far fewer articles in other languages than in English; secondly, there are many doubts about the accuracy and reliability of the content. Thirdly, given the accuracy of the content, as far as we know, there is no board a reviewers or a committee of professionals responsible for verifying the materials, etc. The fact that everyone, even a 6th grade student, can upload or edit content on Wikipedia makes this source not so suitable to use as a scientific source. After this article, I tried to find up-to-date information about this particular issue and came to the conclusion that yet there is no agreement on the use of Wikipedia for educational purposes, or for finding reliable medical information; the main limitations, according to most authors, are the same arguments, mentioned above.
Perhaps in the future as a source in medical education it will be useful to allow medical students to edit the content of WikiMedicine and after checking it by the professionals it will become one of the most useful sources for medical education, but still we should think about the accuracy and correctness of the information provided .... Now I strongly believe that further research is needed in this direction and this article could be considered as a valuable contribution in investigation of this issue.
Ann Blair Kennedy - (14/07/2019)
/
This is a novel and interesting approach in medical education. This type of course supports needed skills for future physicians including evaluation of public/patient consumed media, creating/editing content for public consumption through dissemination of knowledge, and integrating technology into practice.

The small response rate to the survey is limiting yet the inclusion of the qualitative comments provides additional insight. It would be interesting to possibly include course evaluation data gathered by each school. That data might provide additional insight even if the even though the course evaluation forms are most likely very different between the schools.

I believe this study provides an interesting starting point for a more robust study in the future and provides health educators with new activities to add to their curricula.
Tripti Srivastava - (04/07/2019) Panel Member Icon
/
The manuscript can serve as a stepping stone for more robust research on impact of Wiki on Learning outcomes of health professionals. The authors have comprehensively described the attributes and scope of wiki as an editing platform and it’s utility as a learning tool. Low response rate is a limitation besides inclusion of just the qualitative data. Overall a must read for the novelty of it.
Michael Rowland - (03/07/2019)
/
This article provides unique perspective on information that is widely used by the public as an information site that many of us do not often associate with providing accurate health information. It is a virtuous idea to have medical students critically examine health information on website to determine its accuracy and influence of helping the general public.
It would have been helpful to have a larger response rate from the participants but the low response rate of 23% offers some useful information. This is a novel approach to helping medical students understand and know the importance of providing critical health information to the public.