Research article
Open Access

Online Test Application during COVID-19 Pandemic: Academic Impact on Medical Students of the biggest School of Medicine in Mexico

Maria de los Angeles Fernandez-Altuna[1], Diego Gutierrez Rayon[1], Patricia Cruz Mendez[1], Mariela Ramirez Resendiz[1], Fabiola Berenice Angeles Diaz[1], Karla Alejandra Tovar Lopez[1], Carlos Alberto Pantoja-Melendez[1]

Institution: 1. National Autonomous University of Mexico, School of Medicine
Corresponding Author: Dr Maria de los Angeles Fernandez-Altuna ([email protected])
Categories: Assessment, Education Management and Leadership, Technology, Undergraduate/Graduate
Published Date: 11/01/2021

Abstract

Introduction: Online assessment is an educational alternative in this health emergency generated by COVID-19. To meet the educational demands of the School of Medicine of the UNAM, we implemented the evaluation of their subjects of the Medical Bachelor online, avoiding the delay of the school career of their students.

 

Methods: A quantitative analysis was conducted to evaluate the academic performance of students enrolled in the first and second year of the degree of Medical Bachelor, being the school years with the highest percentage of failure. Two cohorts of students (2019 vs 2020) in the first and second year of the degree (4,737 students) were analyzed. We compared their school performance in the exams taken in those years (2019 on School premises and 2020 online due to the COVID-19 pandemic) to observe if there was a significant difference through statistical tests according to the distribution of the cohorts.

 

Results: A significant difference was obtained in subjects of both school years according to their school performance. Besides, the percentage of the passing of the 16 subjects in their two evaluations was evaluated, observing that the students of the year 2020 had a higher rate of passing compared to the year 2019, even in those subjects with a higher percentage of failing.

 

Discussion: Our study allowed us to compare the academic performance of our students in two types of examinations, in the School of Medicine and online (because of COVID-19), a situation that will enable us to assess the advantages of this adaptation and a new form of evaluation as a result of the health emergency.

 

Keywords: School development; Active learning; Medicine; Change; Online evaluation; COVID-19

Introduction

Recently, humankind has experienced one of the most critical pandemics in contemporary history. Since its first notification, at the end of 2019 in Wuhan (China), the coronavirus disease (COVID-19) caused a borderless health emergency, which evidenced the existence of endless economic, financial, social, and educational deficiencies worldwide. 

 

It was not until the end of February 2020 that the first case of COVID-19 was reported in Mexico City, which led to the creation of health and distance measures in the country's productive, business, and educational sectors. Thus, as of the third week of March 2020, the National Autonomous University of Mexico (UNAM) suspended all its academic and administrative activities in all its campuses (Fernandez et al., 2020); this led to the search for educational strategies with distance modalities that would guarantee the exercise of teaching-learning in an organized and systematic way for its 360,883 students enrolled in the period 2019-2020 (UNAM, 2020).

 

The UNAM School of Medicine, since it is in the season of annual academic closure, developed actions aimed at continuing with the provision of education and educational evaluation in the face of this pandemic. Like many other medical education institutions, the School of Medicine sought to ensure that this temporary interruption of its activities would have minimal repercussions on the school career of each of its students; however, the duration and impact of this new pandemic is unknown.

 

The assessment of students during the COVID-19 pandemic has been an international challenge (Hall et al., 2020), leading to a change even in the methods of implementation of the tests. It should be noted that school evaluation through examinations has represented one of the most complete tools, in which the teacher can assess the preparation of the student, having a significant impact on teaching and learning (Al-Qdah and Ababneh, 2017), under this principle, distance evaluations would have to meet this objective. However, this modality still has disadvantages and seems not to have conclusive studies that evaluate the matching of evaluation techniques and analysis of learning processes and academic performance (Kühbeck et al., 2019).

 

Imperial College London is an example of an institution that adopted an online examination approach in this pandemic, to ensure the educational development of its sixth-year medical students (Tapper, Batty and Savage, 2020), with the dilemma that the application of exams from home does not have the necessary conditions for a quiet application environment for students, compared to what has been the norm in the institutions.

 

Other universities have introduced open book exams, and although it is a sudden change, it has been shown that this examination modality reduces student anxiety, being an area of opportunity during the health emergency, minimizing stress symptoms for some students (Sandhu and De Wolf, 2020).

 

The Yong Loo Li School of Medicine also implemented the online exams with three modalities in which social distancing was the objective that regulated the evaluation aspects. The first two actions were aimed at the application of remote, digital and teleconference exams; the third action consisted in exams involving the student, patient, and evaluator where the use of simulators was chosen to minimize contact; in cases where the presence of a patient was necessary, the evaluators were trained in the appropriate use of personal protective equipment to reduce and control health risks (Ashokka et al., 2020).

 

Although this modality provides notable advantages such as the accessibility of evaluating different places and at different times, greater randomization of reagents, instantaneous scores and the possibility of continuous feedback, it also has several areas of opportunity such as the inevitable need for personal computer equipment for students, and efficient navigation network for the interrupted performance of the evaluations as well as the search for a controlled environment that prevents the student from searching for the answer on the web, or communicating with a third party to obtain answers (Gehringer and Peddycord, 2013).

 

Although educational institutions were forced to transform their teaching and assessment methods at a fast pace due to the pandemic, we cannot fail to highlight the emerging and essential challenge of innovation in the educational field that the health emergency highlighted.

 

Acknowledging this process' importance, we conducted a quantitative study to estimate the statistical difference between the face-to-face assessments in 2019 and the assessments conducted online in 2020 in response to the health emergency by COVID-19, of students enrolled in the first and second year of the UNAM School of Medicine.

Methods

Our medical curriculum structure

In the School of Medicine of the UNAM, to complete the Medical Bachelor (MB), students must complete 4 phases of academic training under their 2010 Plan of Studies (UNAM, School of Medicine, 2009):

  • Phase one: organized in two years, consisting of 16 subjects, where the student is required to acquire theoretical biomedical and sociomedical knowledge.
  • Phase two: organized in five semesters, made up of 38 subjects where students will acquire the skills, abilities, attitudes, and aptitudes necessary for the practice of general medicine in real environments.
  • Phase three: corresponding to the undergraduate medical internship, distributed in 6 clinical rotations, practice phase, and incorporation to the health care team of the health care institutions.
  • Phase four: corresponding to the social service year, where the student provides comprehensive care as a complementary social clinical activity.

Phase one - Academic status

According to the 2010 Plan of Studies, medical students during phase one take fourteen annual and two-semester courses (Biomedical Informatics I and II) within the facilities of the School of Medicine in a theoretical-practical manner. According to a 2010 study plan's monitoring of student trajectory, we observed that in this phase of knowledge the subjects with a high rate of failure are found, highlighting Anatomy, Biochemistry and Molecular Biology, Cell Biology and Medical Histology and Human Embryology in the first year; while in the second year we observe Physiology and Immunology (Fernandez et al., 2018).

 

Because of the high failure rate in the subjects mentioned above (40% in the first year and 27% in the second year during the 2010-2015 school year), it is crucial to monitor them to evaluate the curriculum. However, with the pandemic conflict caused by COVID-19, the form of evaluation of the final assessment exams of the 16 subjects in phase one was readjusted.

 

Before the pandemic, these subjects were evaluated employing an online exam in computers at the School of Medicine, monitored by professors at university facilities, distributed at different times of the day according to the quota of the facilities and the number of applicants per subject; however, after the closure of the university facilities due to the social distancing carried out in March 2020 by COVID-19, the application of these exams were carried out from the home of each of the students.

 

Phase one - Online evaluation

With the support of each of the academic departments responsible for the 16 subjects in this theoretical phase, it was agreed to hold the exams on the dates established by the 2019-2020 school calendar, through the online and distance modality. The exams would have to be applied in half the usual time (around one hour); they would be made up of 40 to 60 multiple choice reagents, of an explanatory nature, including clinical scenarios and reducing the contents of memory.

 

Two examinations per subject known as first and second ordinary exams were carried out to accredit them with a score equal to or higher than six to continue with their school career through the MB.

 

Knowledge evaluation - Outcome

Academic performance in the two school years was evaluated with the score obtained through the application of the exams according to the school year. In the case of 2019, it was through monitoring by the Faculty in person at its facilities, through a computer system where it is prevented from opening other web pages and having interaction during the exam; while in 2020, after the social distancing, students were asked to enter a link where it was verified that no two users registered with the same account, the movements of the exam system were monitored and recorded. As an aid to students, a communication channel (Telegram) was implemented to address technical issues relevant to the exam.

Results/Analysis

A total of 4,737 MB students were included: 2,864 freshmen (1,518 in 2019 and 1,346 in 2020); as well as 1,873 sophomores (935 in 2019 and 938 in 2020). The majority of the population studied was between the ages of 18 and 21, with a predominance of females (68%).

 

Evaluation of school performance

Because the student population that took regular exams was different in each subject and school year, Table 1 shows the number of candidates who took the exams in the first and second year of the degree.

 

Table 1: Students who took ordinary examinations in Medicine Bachelor's phase one

SCHOOL YEAR                                                                                                                                                                             FIRST YEAR'S SUBJECTS

 

ANATOMY

HISTOLOGY AND CELL BIOLOGY

BIOCHEMISTRY AND MOLECULAR BIOLOGY

 

HUMAN EMBRIOLOGY

BASIC CLINICAL INTEGRATION I

INTRODUCTION TO MENTAL HEALTH

PUBLIC HEALTH AND COMMUNITY

BIOMEDICAL INFORMATICS I

FIRST ORDINARY

2019

1,095

627

948

988

370

62

68

691

2020

529

604

952

929

47

35

529

83

SECOND ORDINARY

2019

770

327

676

526

95

20

9

275

2020

337

329

537

168

30

5

3

23

SCHOOL YEAR                                                                                                                                                                             SECOND YEAR'S SUBJECTS

 

PHARMACOLOGY

PHYSIOLOGY

BASIC CLINICAL INTEGRATION II

SURGERY INTRODUCTION

HEALTH PROMOTION IN THE LIFE CYCLE

IMMUNOLOGY

 

BIOMEDICAL INFORMATICS II

MICROBIOLOGY AND PARASITOLOGY

FIRST ORDINARY

2019

297

563

194

64

65

540

230

351

2020

175

473

33

37

66

284

396

367

SECOND ORDINARY

2019

287

282

43

11

18

221

12

157

2020

31

329

17

10

2

106

45

83

 

To verify whether there was a significant difference between the 2019 and 2020 school year grades in phase one of the MB, we obtained the statistical normality distribution of grades.

 

According to the distribution, two statistical tests were performed, with Mann Whitney's U being used for the population that did not have a normal distribution (most subjects). While in the case of Introduction to Mental Health and Biomedical Informatics II, the Student T-test was applied with two independent samples since they had a normal distribution (Table 2).

 

Table 2: Asymptotic significance according to Mann Whitney's U Test and Student T-test

SCHOOL YEAR                                                                                                                                                                                             FIRST YEAR'S SUBJECTS

 

 

ANATOMY

HISTOLOGY AND CELL BIOLOGY

 

BIOCHEMISTRY AND MOLECULAR BIOLOGY

 

HUMAN EMBRIOLOGY

 

BASIC CLINICAL INTEGRATION I

 

INTRODUCTION TO MENTAL HEALTH

 

PUBLIC HEALTH AND COMMUNITY

 

BIOMEDICAL INFORMATICS I

 

FIRST ORDINARY

2019

.000

.000

.000

.000

.036

.004

.000

.000

2020

.000

.000

.000

.000

.179

.000

.000

.000

SECOND ORDINARY

2019

.000

.000

.000

.002

.052

.609

.002

.000

2020

.023

.000

.000

.000

.007

.691

.000

.665

SCHOOL YEAR                                                                                                                                                                                         SECOND YEAR'S SUBJECTS

 

PHARMACOLOGY

 

PHYSIOLOGY

BASIC CLINICAL INTEGRATION II

 

SURGERY INTRODUCTION

 

HEALTH PROMOTION IN THE LIFE CYCLE

 

IMMUNOLOGY

 

BIOMEDICAL INFORMATICS II

 

MICROBIOLOGY AND PARASITOLOGY

 

FIRST ORDINARY

2019

.003

.000

.000

.000

.000

.000

.000

.000

2020

.000

.001

.000

.000

.000

.000

.000

.000

SECOND ORDINARY

2019

.021

.089

.200

.486

.043

.048

.308

.200

2020

.200

.001

.020

.000

--

.079

.200

.187

 

When assessing academic performance between the 2019 and 2020 school years, it was observed that the subjects that performed similarly in the first year were:

  • First grade: Anatomy, Cell Biology, and Medical Histology and Biomedical Informatics.
  • Second grade: Basic Clinical Integration I and Public Health and Community

In contrast, in the second-year group, a similar behavior was observed in:

  • First ordinary: Introduction to Surgery, Health Promotion in the Life Cycle, and Immunology.
  • Second ordinary: Microbiology and Parasitology, Basic Clinical Integration II, Introduction to Surgery, and Immunology.

The rest of the subjects have a significant difference in school performance in these school years.

 

Percentage of approval of phase one

Table 3 shows the percentage of approval in each of the evaluations carried out in the first and second year of the MB during 2019 and 2020, observing a notable increase in the percentage of approval in 2020 referring to 2019 in almost all subjects. Overall, there was an increase in total approval in 2020 by nearly 9% for the first year and 1% for the second year compared to 2019.

 

Table 3: Approval rate in phase one subjects in the school years 2019 and 2020

SCHOOL YEAR                                                                                                                                                                                          FIRST YEAR'S SUBJECTS

 

 

ANATOMY

HISTOLOGY AND CELL BIOLOGY

 

BIOCHEMISTRY AND MOLECULAR BIOLOGY

 

HUMAN EMBRIOLOGY

 

BASIC CLINICAL INTEGRATION I

INTRODUCTION TO MENTAL HEALTH

PUBLIC HEALTH AND COMMUNITY

BIOMEDICAL INFORMATICS I

 

FIRST ORDINARY

2019

27.21%

40.98%

19.40%

43.72%

72.97%

59.67%

86.76%

61.07%

2020

34.59%

43.21%

41.59%

81.37%

23.40%

88.57%

34.59%

73.44%

SECOND ORDINARY

2019

43.24%

21.10%

20.11%

50.19%

68.42%

50%

44.44%

98.18%

2020

18.39%

36.47%

46.18%

58.92%

73.33%

100%

100%

95.65%

SCHOOL YEAR                                                                                                                                                                                          SECOND YEAR'S SUBJECTS

 

PHARMACOLOGY

 

PHYSIOLOGY

BASIC CLINICAL INTEGRATION II

 

SURGERY INTRODUCTION

 

HEALTH PROMOTION IN THE LIFE CYCLE

 

IMMUNOLOGY

 

BIOMEDICAL INFORMATICS II

 

MICROBIOLOGY AND PARASITOLOGY

 

FIRST ORDINARY

2019

8.08%

49.73%

81.44%

92.18%

83.07%

58.70%

96.08%

57.83%

2020

81.14%

30.02%

36.36%

75.67%

96.96%

62.67%

94.19%

79.01%

SECOND ORDINARY

2019

31.70%

28.36%

81.39%

100%

94.44%

43.89%

25%

78.34%

2020

48.38%

41.94%

70.58%

80%

50%

54.71%

55.55%

72.28%

Discussion

This study compared school performance and pass rates of regular examinations in phase one of the 2010 MB curriculum in the 2019 and 2020 school years. It was observed that, following the restructuring of assessment methods in the face of the pandemic by COVID- 19, there was an 8.89% increase in student pass rates in 2020 in the first year, while in the second year it was 1.21%; it should be noted that each country has implemented educational tools in response to its possibilities and needs for this type of modality used by the health emergency (Herrera and Toro, 2020).

 

According to UNESCO data, many countries including Angola, India, Indonesia, Malta, Mexico, Micronesia, Mongolia, Morocco, Palau, Thailand, Tonga, and Venezuela decided to introduce alternative approaches in specific contexts for testing and validating learning by reducing the number of tests, modifying the test format and assessing the student's learning portfolio without considering test results (Gwang et al., 2020).

 

The adaptation of the School of Medicine, as well as other universities to this global crisis (Watson et al., 2020; João and Carvalho, 2020), allowed technological innovation and the provision of distance education to become indispensable educational elements to continue the training process of thousands of students.

 

Online evaluation, therefore, requires a redesign of the evaluation system of the subjects in the current curriculum, allowing the establishment of methodological and technological competencies, to know the functionalities and limitations of the computer tools, but being aware that technology is a simple facilitator of evaluation (Garcia et al., 2020); therefore, after this transition to online and distance formats, a post-evaluation is required (Rose, 2020).

 

In our case, because of the pandemic, this “emergency” online evaluation resulted in a lesser degree of failure for our students. This has been an initial quantitative approach that deserves to be complemented by other studies to be followed. Possible initial explanations could be better performance of students in remote evaluations, an unusual response due to the pandemic conditions, not allowed collaborative work among students, or exams with less knowledge to explore (due to the sudden end of courses).

 

For better performance of students at home we do not have yet a plausible explanation since the time considered for online evaluations hardly would have let them access to consult information during the exam (cheating) or could be secondary to lesser stress at home when answering exams. An unusual response during this pandemic could be for students to focus on their exams as a form to avoid the devastating reality due to the health situation.

 

More evidence should be analyzed, and qualitative studies should be performed to better explain and characterize this situation.

Conclusion

The educational intervention of the School of Medicine of the UNAM, as well as of several universities in the face of this health emergency, represented an organizational challenge.

 

Nevertheless, the educational evaluation was carried out within the established time frame to avoid a delay in the students' school career. According to our study, there was more approval in the subjects that generally have a higher percentage of failure. However, it is necessary to complement this study with a qualitative analysis evaluating those variables that led to this situation.

 

The online evaluation is a handy tool that has been implemented in several educational institutions worldwide; this has allowed us to visualize several areas of opportunity that educational entities should consider, and not only use it as a tool of immediate need in the face of health emergencies.

 

Medical schools face an enormous challenge at the organizational and institutional level, which will undoubtedly leave significant learning good enough to undertake successful actions at the educational level.

Take Home Messages

  • The educational processes have to continue despite the health emergency, so the UNAM School of Medicine sought to adapt in the best possible way to this situation, implementing the final evaluation of phase 1 with online examinations of an explanatory nature and special features, such as clinical scenarios and the reduction of memory content.
  • The student population of the degree in Medicine, despite the contingency of the COVID-19, had a higher percentage of approval in subjects that generally have the characteristic of having a high rate of non-approval, showing a statistically significant difference in most of their subjects in comparison with the 2019 school year.

Notes On Contributors

Maria de los Angeles Fernandez-Altuna. General Physician and PhD in Sociomedical Sciences. Currently, Public Health and Basic -Clinical Sciences Integration teacher. Vice Dean for Academic Management Affairs at National Autonomous University of Mexico's School of Medicine. ORCiD: https://orcid.org/0000-0002-7990-3856

 

Diego Gutierrez Rayon. General Physician, Public Health and Basic -Clinical Sciences Integration teacher. Head of Integration, Information and Data Analysis Unit at the National Autonomous University of Mexico's School of Medicine Academic Management Affairs Secretariat. ORCiD: https://orcid.org/0000-0003-1063-7075

 

Patricia Cruz Mendez. General Physician. Currently works at the Re-engineering and Substantive Processes Improvement Area at the National Autonomous University of Mexico's School of Medicine Academic Management Affairs Secretariat. ORCiD: https://orcid.org/0000-0001-6024-4136

 

Mariela Ramirez Resendiz. General Physician. Head of the Undergraduate Unit at the National Autonomous University of Mexico's School of Medicine Academic Management Affairs Secretariat. ORCiD: https://orcid.org/0000-0003-2418-6455

 

Fabiola Berenice Angeles Diaz. General Physician. Currently contributes to the Re-engineering and Substantive Processes Improvement Area at the National Autonomous University of Mexico's School of Medicine Academic Management Affairs Secretariat. ORCiD: https://orcid.org/0000-0001-8935-9869

 

Karla Alejandra Tovar Lopez. General Physician. Currently contributes to the Re-engineering and Substantive Processes Improvement Area at the National Autonomous University of Mexico's School of Medicine Academic Management Affairs Secretariat. ORCiD: https://orcid.org/0000-0002-2212-7884

 

Carlos Alberto Pantoja-Melendez. General Physician. PhD in Epidemiology. Teacher of  the  Public Health Department at the National Autonomous University of Mexico's School of Medicine. ORCiD: https://orcid.org/0000-0001-5244-2538

Acknowledgements

To face the multiple challenges of this pandemic regarding evaluation, it is very important to acknowledge the titanic and creative work of many people of every Academic Department of UNAM´s School of Medicine, Informatics and Telecommunications Unit, Basic Sciences Coordination, General Secretary (Dr. Irene Durante) and the Dean of the School, Dr. German Fajardo-Dolci. 

 

All those involved in evaluation had to change in a very short time from traditional evaluation to many other forms that could be suitable or possible during the COVID-19 pandemic and being faire for the students.

 

Our academic community is proud of their efforts and marvelous work.

Bibliography/References

Al-Qdah, M. and Ababneh, I. (2017) 'Comparing Online and Paper Exams: Performances and Perceptions of Saudi Students', International Journal of Information and Education Technology, 7 (12), pp. 106-109. https://doi.org/10.18178/ijiet.2017.7.2.850

 

Ashokka B., Ong S. Y., Tay K. H., Loh, N. H. W., et al. (2020) 'Coordinated responses of academic medical centres to pandemics: Sustaining medical education during COVID-19', Medical Teacher, 42(7), pp. 762-771. https://doi.org/10.1080/0142159X.2020.1757634

 

Chol, G., Huong, L. T., Moumne, R., Bianchi, S., et al. (2020) 'COVID-19 Panorama of the strategies of response for the tests and evaluations of high impact or high impact', UNESCO. Available at: https://en.unesco.org/sites/default/files/unesco-covid-19-ed-webinar-4-working-document-es.pdf (Accessed: 20 August 2020).

 

Fernandez, M. A., Gutierrez, D., Ramirez, M., Cruz, P., et al. (2020) 'Experience of the biggest Med School in Mexico during the COVID-19 pandemic ', MedEdPublish, pp. 1-12. https://doi.org/10.15694/mep.2020.000101.1

 

Fernandez, M. A., Gutierrez, D., Ramirez, M., Tellez, R., et al. (2018) 'The most difficult subjects at the biggest Med School in Mexico', School of Medicine UNAM. Available at: https://escolares.facmed.unam.mx/publicaciones/Poster_AMEE_2018_%20Basilea.pdf (Accessed: 20 August 2020).

 

Garcia, F. J., Corell, A., Abella, V. and Grande, M. (2020) 'Online Assessment in Higher Education in the Time of COVID-19', Education in the Knowledge Society, 21, pp.12.1-12.26. https://doi.org/10.14201/eks.23013

 

Gehringer, E. F. and Peddycord III, B. W. (2013) 'Experience with online and open-web exams', Journal of Instructional Research, 2, pp. 10-18. https://doi.org/10.9743/JIR.2013.2.12

 

Hall, A. K., Nousiainen, M. T., Campisi, P., Dagnone, J. D., et al. (2020) 'Training disrupted: Practical tips for supporting competency-based medical education during the COVID-19 pandemic', Medical Teacher, 42(7), pp.1-6. https://doi.org/10.1080/0142159X.2020.1766669

 

Herrera, P. and Toro C. J. (2020) 'Medical education during the COVID -19 pandemic: global initiatives for the undergraduate, internship, and medical residency', Acta Med Peru, 37(2), pp. 169-75. https://doi.org/10.35663/amp.2020.372.999

 

João. M. and Carvalho, M. (2020) 'A new era for medical education after COVID-19', Journal of the Medical Education Foundation, 23 (2), pp. 55-57. Available at: http://scielo.isciii.es/scielo.php?script=sci_arttext&pid=S2014-98322020000200001 (Accessed: 20 August 2020).

 

Kühbeck, F., Berberat P. O., Engelhardt, S. and Sarikas A. (2019) 'Correlation of online assessment parameters with summative exam performance in undergraduate medical education of pharmacology: a prospective cohort study', BMC Medical Education, 19(1), pp.412. https://doi.org/10.1186/s12909-019-1814-5

 

Rose, S. (2020) 'Medical Student Education in the Time of COVID-19', JAMA Network, 323 (21), pp. 2131-2132. https://doi.org/10.1001/jama.2020.5227

 

Sandhu, P. and De Wolf, M. (2020) 'The impact of COVID-19 on the undergraduate medical curriculum', Medical Education Online, 25, pp. 1. https://doi.org/10.1080/10872981.2020.1764740

 

Tapper, J., Batty, D. and Savage, M. (2020) 'Medical students take final exams online for first time, despite student concern', The Guardian. Available at: https://www.theguardian.com/education/2020/mar/22/coronavirus-forces-medical-students-sit-final-exams-online (Accessed: 20 August 2020).

 

UNAM. (2020) University Statistics Portal. Available at: http://www.estadistica.unam.mx/numeralia/ (Accessed: 20 August 2020).

 

UNAM, School of Medicine. (2009) Curriculum 2010 and Academic Programs of the Bachelor of Medicine. Available at: http://www.facmed.unam.mx/plan/PEFMUNAM.pdf (Accessed: 20 August 2020).

 

Walsh, K. (2015) 'Online assessment in medical education current trends and future directions', Malawi Medical Journal, 27 (2), pp. 71-72. https://doi.org/10.4314/mmj.v27i2.8

 

Watson, A., Mckinnon, T., Prior, S. D., Richards, L., et al. (2020) 'COVID-19: time for a bold new strategy for medical education', Medical Education Online, 25, pp. 1. https://doi.org/10.1080/10872981.2020.1764741

Appendices

None.

Declarations

There are no conflicts of interest.
This has been published under Creative Commons "CC BY-SA 4.0" (https://creativecommons.org/licenses/by-sa/4.0/)

Ethics Statement

This paper did not require Ethics Approval as there were no human or animal subjects involved in the research. No confidential subject's information was used or published. This paper describes academic impact based only on grades. Nevertheless, we have sent a letter of ethics exemption from our School of Medicine Ethics Review Board as required by the Editor.

External Funding

This article has not had any External Funding

Reviews

Please Login or Register an Account before submitting a Review

Ken Masters - (26/04/2021) Panel Member Icon
/
An interesting paper dealing with the academic impact of online assessment on medical students in a large school of medicine in Mexico.

Although the paper does not necessarily have broader applicability, its value lies in the fact that a world picture of these assessments is necessary, and this is a useful contribution to that picture. I am also pleased that, while the authors did consider that cheating was a possible cause of the increase, they did not jump to that conclusion based on the available data.

As the authors note, an area that they may wish to explore in future (which may give some clarity on this issue), might involve a deeper investigation of the online system’s logs and student interviews. (This is not required for Version 2 of this paper, but would be a useful follow-up research paper).

Two small issues:
• When statistical packages report a p-value of .000, it is best to report them as <.001
• In the results, the word “approval” is used, but, in the Discussion the word “pass” is used. If these are two different things, then they need to be explained. If they are the same thing, then please use only one term (I suggest “pass”), as this is more widely used.

I look forward to Version 2 of the paper in which these issues (and those raised by the previous reviewer) are addressed.


Possible Conflict of Interest:

For transparency, I am an Associate Editor of MedEdPublish.

Barbara Jennings - (20/04/2021) Panel Member Icon
/
I was very interested to read about the positive experience described by the authors for using online assessments for large cohorts of learners in this medical school. The authors from UNAM School of Medicine describe the huge challenge of a rapid adaptation for their phase 1 (undergraduate years 1 and 2) knowledge exams in response to the pandemic from March 2020.
In a useful introduction, the authors outline the global response for competency-exams in medical curricula and touch on options like open book exams to reduce stress and the use of proctoring methods to ensure robustness.
Unfortunately, the methods section presented several problems for me because the protocol followed was not clear with respect to the comparisons made, the study design, and the statistical analysis.
e.g. terms that may be specific to the curriculum such as assessment/ evaluation/ and approval may have been used interchangeably but were not precisely defined. So, I found it difficult to understand the longitudinal follow-up and whether the results and conclusions were generalisable beyond an evaluation for UNAM. Also, did statistical analyses correct for multiple testing with 16 subjects in this theoretical phase?
Despite the letter from the school’s review board described in the ethics statement I do think there are research governance questions. The journal’s guidelines for authors go beyond respect for the privacy by publishing only anonymised findings from the student cohorts. In the methods section it would be good to know how data was handled and stored anonymously for the article development and if the cohorts were informed about the study. Research board review of protocols are recommended because they can also ensure that research questions and methods are clear and valid.


Possible Conflict of Interest:

I am an Associate Editor of MedEdPublish. I have posted this review as a member of the review panel and so this review represents a personal, not institutional, opinion.