The results showed that classes, specifically those based on the Sophistic method (oral exposure of contents by the teacher), are effective in learning, independent of knowledge improvement evaluation method, since they revealed a similar increment in global knowledge acquisition on the subject at study (approximately 60%), relatively to the respective baseline. These are results that reassure the task of the teacher is unquestionably important and effective and should encourage students to attend more frequently theoretical classes.
However, of the entire population of 4th year medical students enlisted in Lisbon Medical University, the attendance to these optional lectures was only of 34%, which unfortunately supports the premise that, in present days, facultative academic lectures have low attendance rates (Kelly, 2012).
As can be seen in the results, there was a higher participation in the classes using paper than those with Sli.do (100% and 63% answer rate, respectively). This can be explained since in the classes that answered in paper, there was an inevitably face-to-face rapport between interviewer and interviewee, which in some way originates a personal responsibility making it less likely for the interviewee to actively deny participation. While inherently more time-consuming, the advantage of paper-based questionnaires seems to be a higher rate of responders, which decreases the risk of participation bias, thus making the results more generalizable to the intended population.
On the other hand, web-based questionnaires, such as Sli.do, imply a masked approach where the faceless invisibility provided by technology can create a selection bias, since it is more likely that only students more interested and keen to participate would reply. Furthermore, technical issues inevitably associated with any such internet-based option, such as requiring a mobile phone or computer with battery connected to the university wireless or with data, could have decreased response rates.
The baseline difference of knowledge between the two groups was not very different, being slightly bigger with Sli.do (46% vs. 52,4%), which doesn´t precisely mean that the classes in which were given the questionnaires in paper had less overall knowledge of glaucoma than the classes using Sli.do. More probably might indicate that the facultative utilization of digital technologies, as previously referred, selected a more motivated group of students and therefore most probably to have prepared the class beforehand. However, it is important to point out that being motivated does not directly correspond with being more connoisseur of the subject (Soper, 2017).
Interestingly, despite non-significant differences in results, this study supports the use of digital-based tools to assess and increase knowledge transfer during classes.
Though a selection bias with the web-based method is possible, the almost complete overlap of results between the two approaches suggest that the loss in the number of participants is not associated with different outcomes. Considering this, there are important factors that support the change for a more digital way of evaluation, while on the meantime not jeopardizing the final outcome. These factors are: ecologic - using digital tools allows us to save paper (reducing deforestation), being more time-efficient - it´s a faster method of collecting and analysing data, and being logistically easier in the long term - although it involves an initial effort in creating the questionnaire in the platform, it can be reproduced every semester with no need of creating it again. Accordingly, interpretation and eventually fine tune of the pedagogical approach, if necessary, would be the same in both types of assessment.
Another important aspect that supports it, although not as expressive, is that although relative values of improvement in knowledge were similar, it is necessary to have in consideration the starting point of both populations, and since basal scores were better with Sli.do, and thus a population more difficult to improve, it shows that people who are integrated in the class through the use of digital tools benefit in terms of knowledge improvement.
There are various possibilities to be considered by teachers and universities to not disregard the non-participating students associated with web-based methods, involving all class in this beneficial teaching. For instance, if before a compulsory lesson, the presence list that traditionally is made by a signature in a paper could be changed to the record that the student answered the questionnaire in the platform, or in order for the student to have access to its grade he needs to have answered the questionnaire. These are just two examples on how to encourage student’s participation, and each professor or academic committee should analyse and discuss the best way to implement it on its specific student population.
Nevertheless, we encourage investigators to design and implement new studies to more accurately understand what the real value of improvement in knowledge is when all the class is involved.
On another subject, using a web-based tool like Sli.do to assess the information that is being effectively taught to the students by the teachers has several benefits for the teachers themselves, the university and ultimately the students, since it allows the professors to find gaps in teaching in order to amend them and improve their own skills, and for universities to carry out an internal evaluation of the performance of the employed teachers, which consequently benefit the students.
A specific example of this benefit was observed in question 10, as both methods revealed a regression in knowledge. In this question, students showed a regression in their confidence that glaucoma when effectively treated can prevent blindness, questioning themselves if their basal knowledge (which was elevated - 69% in paper vs. 84% in Sli.do) was wrong. Although being a very small regression in the correctness of the answers (69% to 68% in paper, a relative decline of 3% vs. 84% to 82% in Sli.do, a relative decline of 3%), it demonstrates that this specific information was not effectively passed on to the students, not due to the evaluation method, because the variation was equal, but due to some failure in communication or not being referred in class.
Another example of how these in-class questions can help detect miscommunications during the teaching process are the replies to question number 6. Regarding this question, in both set of questionnaires, students vastly selected an incorrect option after the lesson (incorrectly replying hyperopia to be a risk factor for open angle glaucoma - 7% up to 42% in paper vs. 3% up to 37% in Sli.do).
In both cases, having performed questionnaires would have allowed the teacher to detect what was not being properly understood by the audience.
In the opposite spectrum of the scale, this in-class questions also allow teachers to perceive the good results during the teaching process, as it can be seen in question number 3. In this question, in both set of questionnaires, students revealed a great increase in the correctness of the answers after class, comparatively to the results before class (44% to 95% in paper, a relative improvement of 116% vs. 47% to 97% in Sli.do, a relative improvement of 106%).
Since the introduction of new technologies seems to stimulate learning, we questioned what possible alternatives there were, besides interactive questionnaires, that could also be implemented in classrooms and benefit learning. We considered many hypotheses but the most inclusive and comprehensive was that a change in teaching panorama, from a Sophistic method to Flipped Classrooms, could be positive to knowledge improvement (Tune, Sturek and Basile, 2013).
Flipped Classrooms invert the usual organizational structure of the classrooms by providing educational tools and contents, such as recorded multimedia lectures, PowerPoints or other digital documents, before class, so students can view and study them outside of it and at their own pace. This asynchronous approach allows for more in class time for student centred learning activities, encouraging their participation and motivation through debates, presentations, questionnaires and other dynamics (Flaherty and Phillips, 2015).
We hope that, with this study, we further opened a door and encouraged other investigators to give more importance to methods in teaching and in finding better ways to reach the students, so in the future we can benefit from their excellence.
On a different matter, in the literature research we had access to various scientifically validated questionnaires. However, most presented some limitation to the objective, such as they were not in the mother tongue of the population at study (Portuguese) and they were not designed to it, being more appropriate for patients and their knowledge of this disease. Therefore, we choose the validated questionnaire that better accomplishes our specifics.
Although we did our best to minimize the limitations of this study by executing it the most impartial, professional and correct way, it is important to point out some of the limitations that we encountered in our study, which are an incentive for further studies. First, although we gathered a reasonable number of students (a total of 89), we think that with a larger sample it would be more representative of the population at study. Second, although we had the same professor lecturing all four classes and he prepared himself to give the best lecture possible and the most equal between them, it is still impossible to recreate exactly the same 4 classes of 50 minutes each, separated over a year, which might explain some of the result already discussed. Finally, as previously referred, we were not able to perform more specific analysis tests, such as an analysis of variance (ANOVA), because Sli.do does not provide the results in a discriminatory way person by person. Therefore, it was not possible to evaluate the statistical significance of the results obtained, being the results analysed by means of descriptive statistics.
Acknowledging these limitations is a needed step for, in the future, designing new studies that amend these aspects and consequently develop the knowledge on this subject, so we can all benefit.