Review of the literature
Open Access

Pros and Cons of Audience Response Systems in the Education of Health Professionals

Ryan Denkewicz[1]

Institution: 1. Frank Netter School of Medicine at Quinnipiac University
Corresponding Author: Mr Ryan Denkewicz ([email protected])
Categories: Educational Strategies, Teaching and Learning
Published Date: 24/09/2019

Abstract

Well over 1000 articles have been written on the subject of Audience Response Systems in the education of health professionals. These studies have involved the students at the undergraduate, graduate and continuing professional education levels. From the articles found, a subset of over 120 articles of ARS use in health professional education was reviewed. A holistic view of the results in these studies showed that there are clear benefits and drawbacks to ARS use from the perspective of the students and teachers. Benefits were found to fit ten (10) distinct categories while drawbacks fit into six (6) discrete categories. This paper summarizes and discusses the benefits (i.e. pros) and drawbacks (i.e. cons) of the use of ARS in the educational process of health professionals.

Keywords: Audience response systems; ARS; clickers; learning outcomes; interactive learning; active learning

Introduction

Audience Response System (ARS) technology has been used over the last several decades as another modality of teaching to enhance learning outcomes (Boyle, 2006; Keengwe, 2007).  The technology was first developed by IBM in the 1960s.  One of its first uses was in the area of market research, most notably to collect anonymous survey data from consumers previewing unreleased Hollywood films. Academically, ARS was first seen in the 1960s at Cornell and Stanford universities (Kay and LeSage, 2009; Nelson et al., 2012). More recently, ARS technology has been gaining popularity in the classroom (Han, 2014) including its more frequent use by medical and dental schools, medical residency programs and studies in the biological sciences and health professions. The collective body of research on ARS has highlighted both the attendant benefits and drawbacks associated with this new pedagogical tool. While some studies have demonstrated a positive statistical correlation between ARS use and knowledge retention and exams scores (Crouch and Mazur, 2001; Bunce, VandenPlas and Havanki, 2006; Mains et al., 2015; Subramanian et al., 2011; Benson et al., 2017; Stoddard and Piquette, 2010; Nelson et al., 2012; Berry, 2009; Alexander et al., 2009; Hettinger et al., 2014; Nosek et al., 2006;  Slain et al., 2004), others have found the use of ARS to be disruptive to the classical didactic teaching approach (Clausen, Alkhateeb and Singh-Franco, 2012; Connor, 2009) and too cumbersome on the instructor (Abate, Gomes and Linton, 2011; Arneja et al., 2009; Cain and Robinson, 2008; De Gagne, 2011; DeBourgh, 2008; Zurmehly and Leadingham, 2008).

This paper summarizes the pros and cons of ARS use as cited by various studies, particularly those performed in the area of health professional education, including continuing education and conferences for physicians along with some undergraduate studies in biological and health sciences (see Table 1). It is hoped that the review and analysis performed here accomplishes at least two objectives:  1) provides a more complete picture of the potential for ARS technology, and, 2) provides insights to teaching professionals deciding if ARS technology should be incorporated into their specific teaching regimen.

Table 1: Articles in the Health Profession on Audience Response Systems

Student Classification

# of Papers Reviewed

Medical Students

48

Medical Residents

10

Physicians

13

Nursing & Pharmacy

27

Veterinary & Dental Medicine

15

Biological & Health Sciences

15

 

What is ARS Technology?

ARS technology, in its most simplistic form, consists of handheld devices (called clickers) that are remotely connected to a hardware device that contains software which enables the collection and display of student responses in the classroom via the instructor’s computer (Collins, 2008; Thampy and Ahmad, 2014; Vana et al., 2011). ARS goes by other names including personal response system, interactive voting system, electronic voting system, student response system, interactive student response systems, group response system, classroom response system, immediate response system, classroom performance system and clickers (Hunsu, Adesope and Bayly, 2016). A very common implementation of the technology is for the teacher to display a multiple choice question in the classroom (e.g. using a projector and a program such as Microsoft Powerpoint), with the students using their clicker devices to select what they believe is the correct answer. The ARS system collects the responses and displays them in a tabular or graphic format. The handheld clickers can be registered online such that each device is tied to a specific student. This is desirable if the ARS system is being used to confirm class attendance and/or record student performance. Alternatively, systems can be configured whereby the student submits responses using any wireless network-enabled device (e.g. laptop, cellphone). The use of ARS can replace the more conventional teacher-student interaction whereby the teacher poses a question and then calls upon a single student for an answer. Here, the entire student body is involved with the preservation of anonymity.

ARS can be used to achieve a number of educational objectives. These include: (Dufresne et al., 1996):

  • Aiding the instructor in identifying the students’ understanding
  • Aiding the instructor in evaluating his/her own teaching performance
  • Increasing the students’ knowledge of difficult concepts
  • Increasing the students’ knowledge of the instructors expectations
  • Enhancing the students’ awareness of their own understanding
  • Providing a ‘springboard’ for class debate and discussion
    Each of these objectives is valuable in its own right, but when taken together, they potentially represent a significant new tool in education. For this reason, ARS is worthy of critical review and analysis.

Methodology

Hundreds of research papers were located on PubMed, EMBASE, SCOPUS, Cochrane Library, CINAHL, ERIC using the keywords, audience response systems, personal response stations, interactive voting systems, electronic voting systems, student response systems, interactive student response systems, group response systems, and clickers. The research studies located were then narrowed to the field of health professional education. This included studies performed in undergraduate college level biology and anatomy classes, nursing, dental, veterinary and medical school curricula, as well as residency programs and continuing educational classes for physicians and other health professionals. Inclusion of research articles found beyond the scope of the health profession, would have led to studies numbering well over 1000.

The pros and cons of the use of ARS technology, from both the perspective of the students and teachers, were captured and then compiled into distinct categories.

Discussion

A review of over 120 research articles on the use of ARS technology in the education of health professionals led to a myriad of opinions, attitudes and facts regarding the benefits and drawbacks to ARS in the educational process. When viewed holistically, the perceived and real benefits (i.e. pros) and drawbacks (i.e. cons) cited in the research were found to fit a distinct set of categories as shown in Table 2.

 

The Pros

Ten distinct categories of benefits were noted in the compilation of the research studies (see Table 2). These are discussed, in turn, below.

Table 2: Pros and Cons of Audience Response Systems

Pros

References

Increases student Engagement/

Attention

Abate et al., 2011; Addison et al., 2009; Arneja et al., 2009; Baker et al., 2017; Barbour 2008; Beaumont et al., 2017; Benson et al., 2016; Berry 2009; Caldwell 2007; Chaudhry 2011; Clauson et al., 2012; Connor 2009; Copeland et al., 2000; De Gagne 2011; Dhaliwal et al., 2015; Doucet et al., 2009; Fies 2005; Fitzpatrick et al., 2011; Freeman and Dobbie 2008; Gauci et al., 2009; Graeff et al., 2011; Hatch et al., 2005; Ioannou and Artino 2007; Johnson 2005; Kay and Lesage 2009; Kung et al., 2012; Latessa and Mouw 2005; Llena et al., 2015; Mains et al., 2015; Mastoridis and Kladidis 2010; Meedzan and Fisher 2009; Miller et al., 2003; Mostyn et al., 2012; Petit et al., 2014; Pradhan et al., 2005; Premkumar and Coupal 2008; Preszler et al., 2007; Uhari et al., 2003

Improves Learning outcomes

Abate et al., 2011; Alexander et al., 2009; Arneja et al., 2009; Baker et al., 2017; Brady and Forest 2018; Brady et al., 2013; Cain and Robinson 2008; Caldwell 2007; Chaudhry 2011; Devitt 2012; Dhaliwal et al., 2015; Doucet et al., 2009; Duggan et al., 2007; Elashvili et al., 2008; Freeman and Dobbie 2008; Goldberg et al., 2006; Grimes et al., 2010; Guse and Zobitz 2010; Hettinger et al., 2014; Hoyt et al., 2010; Hunsu et al., 2015; Jelsing et al., 2007; Johnson 2005; Kay and Lesage 2009; Latessa and Mouw 2005; Leidner and Jarvenpaa 1995; Liu et al., 2010; Lymn and Mostyn 2010; Mahon et al., 2018; Mains et al., 2015; Mastoridis and Kladidis 2010; Meedzan and Fisher 2009; Mostyn et al., 2012; Palmer et al., 2005; Pradhan et al., 2005; Premkumar and Coupal 2008; Preszler et al., 2007; Rubio et al., 2008; Schackow et al., 2004; Schlegel and Selfridge 2014; Uhari et al., 2003

Provides focus for Study

Abate et al., 2011; Alexander et al., 2009; Arneja et al., 2009; Benson et al., 2016; Cain and Robinson 2008; Caldwell 2007; Copeland et al., 2000; Crouch and Mazur 2001; De Gagne 2011; Draper et al., 2002; Egelandsdal and Krumsvik 2015; Fitch 2004; Gauci et al., 2009; Halloran 1995; Hatch et al., 2005; Hudson and Bristow 2006; Ioannou and Artino 2007; Johnson 2005; Leung et al., 2013; Mostyn et al., 2012; Pradhan et al., 2005; Premkumar and Coupal 2008; Knight and Wood 2005;  Lymn and Mostyn 2010; Robertson 2000; Schlegel and Selfridge 2014; Simpson and Oliver 2007; Stuart et al., 2004; Wait et al., 2009

Realtime Feedback for Teacher

Abate et al., 2011; Akl 2010; Alexander et al., 2009; Arneja et al., 2009; Beatty et al., 2006; Beaumont et al., 2017; Benson et al., 2016; Brady and Forest 2018; Bruff 2015; Cain and Robinson 2008; Caldwell 2007; Chaudhry 2011; Connor 2009; De Gagne 2011; Draper et al., 2002;  Duggan et al., 2007; Efstathiou and Bailey 2012; Egelandsdal and Krumsvik 2015; Fissell 2016; Fitch 2004; Fitzpatrick et al., 2011; Fujikura et al., 2013; Gauci et al., 2009; Gousseau et al., 2016; Guse and Zobitz 2010; Harvey 2015; Hoyt et al., 2010; Hudson and Bristow 2006; Ioannou and Artino 2007; Kay and Lesage 2009; Kennedy and Cutts 2005; Kung et al., 2012; Leeds et al., 2018;  Llena et al., 2015; Lymn and Mostyn 2010;  Mastoridis and Kladidis 2010; Menon et al., 2004; Mostyn et al., 2012; Nicol and MacFarlane-Dick 2006; Petit et al., 2014; Revell and McCurry 2010; Schlegel and Selfridge 2014; Simpson and Oliver 2007; Stuart et al., 2004

Increases Peer Interactivity & Class Participation

Alexander et al., 2009;  Arneja et al., 2009; Baker et al., 2017; Barbour 2008; Beatty et al., 2006; Beaumont et al., 2017; Beekes 2006; Brady and Forest 2018; Caldwell 2007; Dhaliwal et al., 2015; Doucet et al., 2009; Draper et al., 2002; Efstathiou and Bailey 2012; Fissell 2016; Fitzpatrick et al., 2011; Gousseau et al., 2016; Horowitz 1988; Hudson and Bristow 2005; Jackson and Trees 2003; Knight and Wood 2005; Mahon et al., 2018; Naismith and Steinert 2001; Streeter and Rybicki 2006; Wood 2004

Provides Student Anonymity

Beekes 2006; Cain and Robinson 2008; Chaudhry 2011; Clauson et al., 2012; Connor 2009; De Gagne 2011; Draper and Brown 2004; Fies 2005; Gauci et al., 2009; Guse and Zobitz 2010; Ioannou and Artino 2007; Jones et al., 2001; Kung et al., 2012; Mahon et al., 2018; Mostyn et al., 2012; Stuart et al., 2004

Easy to use

Alexander et al., 2009; Beaumont et al., 2017; Benson et al., 2016; Cain and Robinson 2008; Caldwell 2007; Llena et al., 2015; Miller et al., 2003; Parsons 2005; Pradhan et al., 2005

Enjoyable to use

Addison et al., 2009; Akl 2010; Arneja et al., 2009; Barbour 2008; Doucet 2009; Cain and Robinson 2008; De Gagne 2011; Devitt 2011; Fitzpatrick et al., 2011; Freeman and Dobbie 2005; Graeff et al., 2011; Hudson and Bristow 2006; Ioannou and Artino 2007; Latessa and Mouw 2005; Llena et al., 2015; Lymn and Mostyn 2010; Mains et al., 2015; Mastoridis and Kladidis 2010; Petit et al., 2014

Encourages Active Learning

Addison et al., 2009; Alexander et al., 2009; Arneja et al., 2009; Caldwell 2007; Clauson et al., 2012; De Gagne 2011; Draper et al., 2002; Goldberg et al., 2006; Hoyt et al., 2010; Johnson 2005; Kung et al., 2012; Meedzan and Fisher 2009; Meyers and Jones 1993; Premkumar and Coupal 2008; Revell and McCurry 2010; Roschelle 2003; Rubio et al., 2008; Simpson and Oliver 2007

Improves Student Attendance

Connor 2009; Gauci et al., 2009; Kay and Lesage 2009; Kung et al., 2012

Cons

References

Teacher Prep Time

Abate et al., 2011; Arneja et al., 2009; Collins 2008; De Gagne 2011; Duggan et al., 2007; Gauci et al., 2009; Kay and Lesage 2009; Stuart et al., 2004

Cost

Arneja et al., 2009; Barber and Njus 2007; Cain and Robinson 2008; Collins 2008; De Gagne 2011; Mastoridis and Kladidis 2010; Menon et al., 2004

Learning Curve for use

Arneja et al., 2009; Cain and Robinson 2008; Collins 2008; Gauci et al., 2009; Gousseau et al., 2016; Kay and Lesage 2009; Kung et al., 2012

Technical Malfunctions

Benson et al., 2016; Cain and Robinson 2008; Collins 2008; De Gagne 2011; Gauci et al 2009

Reduced Classtime

Clauson et al., 2012; Collins 2008; Connor 2009; Ioannou and Artino 2007; Kay and Lesage 2009

Students Object to being Monitored

Fujikura et al., 2013; Kay and Lesage 2009; Meedzan and Fisher 2009

 

Increased Student Engagement & Attention. ARS technology was found in a number of studies to increase student engagement in the classroom (Abate, Gomes and Linton, 2011; Cain and Robinson, 2008; Hatch, Jenson and Moore, 2005). This result was typically gauged through a post-ARS use survey of students and teachers. Unlike conventional didactic lectures where student involvement may or may not be solicited or required, the use of ARS demands some attention by the student. This is particularly true in those applications of its use when the student is graded based on their participation or when the teacher uses the ARS question format to then stimulate a class discussion. Generally, students willingly engaged with the system, citing the desire to challenge their own knowledge against that of their peers and the enjoyment of the trivia game-like environment ARS provides (Schlegal and Selfridge, 2014). The anonymity offered by ARS also makes it easier for students to participate without the fear of public speaking or ridicule for not knowing the correct answer (Cain and Robinson, 2008; Draper and Brown, 2004; Beekes, 2006). Teachers cited a pronounced level of student attention during ARS lectures that was not observed in the absence of ARS (Arneja et al., 2009; Freeman and Dobbie, 2005; Johnson, 2005; Pradhan, Sparano and Ananth, 2005).

Improved Learning OutcomesMultiple studies (see Table 3) reported a distinct improvement in learning outcomes as measured by test scores, including those of National exams. Subramanian et al., (2011) observed a significant improvement in student learning outcome with ARS compared to the traditional didactic lecture format. Specifically, in a course in Cardiology at Baylor Medical School, the mean score for the control group was 61.7 +/- 2% whereas the mean score for the ARS led group was 86.7+/- 2%. After a 22 day waiting period, the students were reassessed on the same information and the control group scored 55.8% while the ARS led group scored 70.1%. Similarly, Arneja et al (2009), observed statistically higher scores (p = 0.01) with the use ARS in a Plastic Surgery Residence program. However, Arneja et al., (2009) and Tregonning et al., (2012) found the improvement in knowledge retention to be short-lived. Some researchers (Pradhan, Sparano and Ananth, 2005) have speculated that improvements in learning outcomes are simply a result of the Hawthorne effect. That is, the novelty of the Audience Response System, when used for the first time, creates excitement, increased student attention and enhanced learning outcomes, but such benefits diminish over time.

Table 3: Studies Reporting the Improvement in Learning Outcomes

Study

Cited evidence of Improved Learning Outcome

Alexander et al., 2009

Statistically significant (p<0.0001) positive correlation between ARS use and final exam scores over 3 year period with 1st year Medical students in all didactic blocks

Arneja et al., 2009

Cumulative mean test scores were 85% (with ARS) and 75% (without ARS) in Plastic Surgery Resident program

Berry, 2009

Student t-test exam and final exam course grades were significantly higher in a Pediatric nursing course

Cain and Robinson, 2008

Significantly higher test and final exams scores (with ARS) over 2 years in Pharmacy Resident program

Elashvili et al., 2008

Statistically significant (p= 0.002) written test scores on post-lecture quiz for Dental students

Gauci et al., 2007

Significant improvement in all 3 years (p<0.01) in undergraduate Physiology course when ARS was used

Hettinger et al., 2014

In 2 year study, overall performance improved significantly (p= 0.0068) on PRITE testing in Psychiatry Resident program

Hoyt et al., 2010

Students in lower quartile had better exam performance after ARS was implemented in Human Anatomy course over 2 year study period

Hunsu, Adesope and Bayly, 2016

Meta-analysis of 53 articles involving 26,095 students showed ARS use produced small but significant effects on measures of knowledge transfer and final achievement across a myriad of subject matters

Mains et al., 2015

ARS use increased knowledge both immediately after lecture as well as 2 weeks later for 1st year Medical School students (p=0.001)

Nosek et al., 2006

Mean test score progressively improved with higher use of ARS in a Hematology/Oncology course for 2nd year Medical School students

Palmer et al., 2005

ARS improved the long term retention of material and provided better understanding of topics discussed for 1st year Medical school students

Pradhan, Sparano and Ananth, 2005

21% improvement in test scores (p = 0.018) for Residency students in OB-GYN

Preszler et al., 2007

The more ARS was utilized the better the students performed across 6 separate Biology classes

Rubio et al., 2008

ARS group had significantly higher learning (p=0.02) and long term retention (p=0.001) scores on post-lecture quizzes and quizzes 3 months later in a Radiology Resident program

Schackow et al., 2004

96% (ARS) versus 61% (non-ARS)  post-lecture quiz score for Family Medicine Residents

Schlegal and Selfridge, 2014

Improvement in academic performance was observed in Microbiology, Pharmacology, Pathology and Clinical Medicine

Slain et al., 2004

Students using ARS in Clinical Pharmacokinetics, Medical Literature Evaluation, and Pathophysiology and Therapeutics had better final exam scores (p<0.001, p=0.016 and p= 0.0002, respectively)

Subramanian et al., 2012

ARS modality demonstrated a significant improvement (p < 0.001) in student learning retention compared to traditional didactic lecture in Cardiology students at Baylor Medical School, both immediately after lecture and after 22 days

Tregonning et al., 2012

Significantly higher immediate post-lecture quiz scores (p<0.001) in an Ob-Gyn course.

 

A few studies (Alexander et al., 2009; Schackow et al., 2004; Premkumar and Coupal, 2008; Rubio et al., 2008; Devitt, 2012; Leidner and Jarvenpaa, 1995; Mastoridis and Kladidis, 2010; Meedzan and Kelly, 2009; Palmer et al., 2005) claim that longer term knowledge retention did occur when ARS was employed.

In a dental studies course (Wenz et al., 2014), it was observed that the greatest performance improvement with ARS benefited the ‘below-average” students.

Other studies showed no measurable improvement in testing scores (Patterson, Kilpatrick and Woebkenberg, 2010; Stoddard and Piquet, 2010; Addison, 2008; Graeff et al., 2011; Plant, 2007; Welch, 2013). Stoddard and Piquet (2010), for example, observed no measureable difference in exams scores in a medical school Pulmonology class when ARS was used. Similarly, Addison (2008) saw no improvement in class mean composite exam scores in an undergraduate biochemistry course between students taught with clickers than those taught in traditional lectures. Graeff et al., (2011), in a Physician Assistant program, observed no statistical difference in knowledge retention between groups that had traditional lecture formats and those that had used clicker response questions.

Provides Focus for Study.  Students perceive that the material highlighted by the teacher using ARS, provides a clear indication of the most important areas of study (Benson et al., 2016; Abate, Gomes and Linton, 2011; Ionone and Aretino, 2007). As such, students did report a belief that ARS helps guide their preparation for class discussion and exams by identifying gaps in their understanding of the presented material (Goose and Obits, 2010; Mostyn, Meade and Lymn, 2012; Schlagle and Selfridge, 2014). In one study at a Urological Conference (Leung et al., 2013), it was noted that the use of ARS may be able to identify high-yield topics for medical education.

Real-time Feedback for Teacher. Audience Response Systems have the unique benefit of providing immediate feedback to the teacher as to the knowledge gaps that exist among students. Well posed questions help illuminate areas of student conceptual misunderstanding (Duggan, Palmer and Devitt, 2007; Fujikura et al., 2013). ARS immediately provides teachers with the data to know whether or not they are adequately conveying the information. Are more example problems needed? Is a class discussion/debate warranted?  Such formative feedback is useful as it affords the teacher the ability to adjust the lecture ‘on the fly’ or reconfigure future lectures, or homework assignments, to address knowledge gaps. This ‘contingent teaching’ or ‘agile teaching’ is responsive to the needs of the students and not driven by a preset download of information determined by the teacher (Draper & Brown, 2004; Beatty et al., 2006).

Additionally, formative feedback from the ARS approach enables teacher to gauge their own teaching style and performance (Draper, Cargill and Cutts, 2002). Am I moving too fast through the material? Should I try an alternative means of conveying the concept? Might a specific homework assignment be valuable to solidify the students’ understanding? Through the instructor’s self-reflection catalyzed by the use of ARS, the teaching dynamic changes in that it “moves [the instructor] out from behind the relative safety of the lecture podium to adopt active strategies that shift classroom emphasis away from teachers’ teaching towards students’ participation and learning” (Allen and Tanner, 2005).

Increased Peer Interactivity and Class participation.  Inherent in the design of Audience Response Systems is the engagement of all the students in the subject matter being taught through the answering of questions followed by a review of the responses. Some teachers have used the results of the student responses to spark classroom debate over why one answer was better than another (Alexander et al., 2009; Horowitz, 1988; Cain and Robinson, 2008; Beekes, 2006; Fissell, 2016). Encouraging peer-to-peer interaction is considered an invaluable technique in education (Mazur, 1997). Harvard professor Eric Mazur first found that “Peer Instruction” is one of the best modes for learning; that is, students were better at clearing up each other’s misconceptions and confusions than the instructor themselves. Wood (2004) demonstrated this peer-to-peer benefit directly in an undergraduate biology class studying molecular genetics. Towards this end, ARS provides a ready platform for classroom discussion and debate over the best answer and alternative viewpoints.

Additionally, Fies (2005) notes that the “level of intrinsic motivation [for students] increases with an engaging learning environment in which mastery goals are emphasized and frequent feedback is given.”

Provides Student Anonymity. Perhaps one of the greatest barriers to student learning is certain fears that a student may have in a public classroom environment (Clausen, Alkhateeb and Singh-Franco, 2012; De Gagne, 2011; Fies, 2005; Ioannou and Artino, 2007).  Many students readily admit that they choose not to engage in dialogue during class time for fear of:

  • Saying something that makes them look ‘stupid’ in front of their peers
  • Looking like they are showing off (if they happen to have the correct response)
  • Public speaking

These student fears are well documented and are real barriers for many (Caldwell, 2007; Chaudhry, 2011; Connor, 2009; De Gagne, 2011; Fies, 2005; Gauci et al., 2009; Guse and Zobitz, 2010; Ioannou and Artino, 2007; Kung et al., 2012). However, ARS, by virtue of the anonymity it provides, helps remove these fears. With ARS, students are able to actively participate without the possibility of public embarrassment or scrutiny. This may be particularly helpful to those students who are struggling with the conceptual understanding of the lecture material, as this group might be more inclined not to otherwise participate (Mostyn, Meade and Lymn, 2012; De Gagne, 2011; Ioannou and Artino, 2007; Connor, 2009).

Easy to Use.  Clickers have been repeatedly found to be intuitively easy to use (Beaumont et al., 2017) and no studies were found in which the students found them complicated or confusing.

Enjoyable to Use. A nice benefit of ARS is the enhanced student enjoyment (Halloran, 1995; Mastoridis and Kladidis, 2010; Devitt, 2012; Cain and Robinson, 2008; Arneja et al., 2009; Freeman and Dobbie, 2005; Baker, Francis and Cathcart, 2017; Torbeck, 2007). In fact, some studies reported that student’s class attendance improved with the use of ARS (Gauci et al., 2009; Kay and Lesage, 2009; Kung et al., 2012).  Those students cited the enjoyment of using ARS as the primary reason for increased class attendance. The trivia game-like atmosphere that is created by ARS, makes the classroom dynamic more exciting for many students (Schlegal and Selfridge, 2014).

Encourages Active Learning. A common criticism of didactic lecture formats is the tendency for the learning process to devolve into a monotonous process of transferring facts, followed by a reliance on the rote memorization of those facts (White, 2011). Many educators have found that learning outcomes are improved when students are engaged in ‘active learning’ (Prince, 2004; Handelsman et al., 2004; Miller and Metz, 2014). Active learning is the process of ongoing and adaptive dialogue between teacher and student (Laurillard, 2002). In such an environment, active learning gives students time to reflect on their understanding of recent material, practice a skill or highlight gaps in their knowledge before engaging in class discussion (Prince, 2004). Active learning encourages students to accomplish higher order objectives on Bloom’s taxonomy, such as analysis, synthesis and evaluation. (Bloom, 1956) Many studies have shown that active learning processes can improve student’s comprehension, problem-solving abilities and critical thinking skills (Michael, 2006; Bonwell and Eison, 1991; McKeachie, 1994; Palmer and Devitt, 2007).
Improves Student Attendance. Many studies on ARS have shown that there is an increase in class attendance when the use of ARS is tied to class grading (Kay and Lesage, 2009). Such motivation to attend class has been observed even when the attendance represents as little as 5% of the grade (Caldwell, 2007). In three studies, improved class attendance was also observed when grades were not associated with ARS use (El-Rady, 2006; Preszler et al., 2007), including that of a Pediatric Residency training program at the Mayo College of Medicine (Homme, Asay and Morgenstern, 2004).

The Cons
Six distinct categories of drawbacks were noted in the compilation of the research studies (see Table 2). These are discussed, in turn, below.

Teacher Prep Time. The use of ARS in the classroom requires a concerted effort on the part of the teacher to prepare this part of the lecture (Collins, 2008). Failure in this area by the teacher can have the resulting effect of student disenchantment and demotivation (Abate, Gomes and Linton, 2011; Ioannou and Artino, 2007; Kay and Lesage, 2009; Duggan, Palmer and Devitt, 2007). Some teachers found the preparation for the questions time-consuming, particularly because the questions to be posed must be well formulated (Abate, Gomes and Linton, 2011; Arneja et al., 2009; Connor, 2009; Duggan, Palmer and Devitt, 2007; Gauci et al., 2009; Bruff, 2018). Well formulated questions elucidate the student’s true level of understanding, are not too simple and boring, and are not too complex or confusing. Allen and Tanner (2005) noted that “multiple choice questions are a difficult format in which to pose questions that nudge students to the realm of higher order thinking.” The answer records, using a multiple choice questioning approach, “do not provide explanations of the reasons or reasoning patterns for students’ selection.”

Cost. Audience Response Systems are not free. System cost varies considerably but, in the end, the costs must be borne either by the institution, the student or both and this can be a deterrent (Cain and Robinson, 2008; Mastoridis and Kladidas, 2010; Menon et al., 2004). Wood (2004) reported that clickers cost between $5 - $30 while the audience response system itself costs about $1000 for each 100 students. Barber and Njus (2007) reported a net cost to the University bookstore of $16 for the clicker and a per term charge of between $13 - $15 per student.

Learning to Use. Utilizing the clickers as a student essentially requires no training. The institution, on the other hand, has to install the hardware and software and ensure proper network connectivity to all the devices (Barber and Njus, 2007). The teacher, for their part, needs to learn how to integrate the system into the lecture format of the computer system (Arneja et al., 2009; Cain and Robinson, 2008; Kay and Lesage, 2009).

Technical Malfunctions. ARS are wireless electronic devices working on a software platform that can be subject to periodic malfunctions (Benson et al., 2016). Interruptions in wireless signals, glitches in software, or mechanical breakage in the clickers can render the system inoperable. Such events, however rare, require the attention of a staff technician.

Reduced Lecture Time.  In several studies (Abate, Gomes and Linton, 2011; Connor, 2009; Ioannou and Artino, 2007; Kay and Lesage, 2009), it was noted that the reduced time for lecture is a negative attribute of the ARS approach. At the very least, the use of ARS requires some adaptation to the lecture format to cover less lecture material and make room for questions. This is not always a welcomed feature by all instructors.

Students Do Not Want to be MonitoredTwo of the features of ARS are the ability to use the technology to track student attendance and record answers to the multiple choice questions asked during class. One study (Kay and LeSage, 2009) noted that students were displeased about being forced to attend lectures in order to earn academic credit for a class.  In other studies (Caldwell, 2007; Fujikura et al., 2013) it was found that students disapproved of grades being attached to ARS participation. Caldwell (2007) noted that “Attaching a grade to ARS may undermine the goal of developing an effective learning environment.” Meedzan and Fisher (2009) also found that there was lower satisfaction to class participation when ARS was used as a tool for summative assessment.

Conclusion

A review of the literature has demonstrated that Audience Response Systems (ARS) have been widely employed at all levels in the education of health professionals and that such systems have clear benefits and drawbacks from both the perspective of the students and the teachers. Benefits were found to fit ten (10) distinct categories while drawbacks fit into six (6) discrete categories. ARS can provide a complementary pedagogical approach to the traditional didactic lecture format and provide exciting potential for improving the education of health professionals in areas such as improved learning outcomes, greater class attendance, and encouraged active learning.

Take Home Messages

  • Audience Response Systems (ARS) have been widely employed at all levels in the education of health professionals
  • ARS can provide a complementary pedagogical approach to traditional didactic lecture formats
  • Experiences with the use of ARS, by both students and teachers, has led to the identification of a distinct set of benefits and drawbacks
  • ARS has exciting potential for improving the education of health professionals in areas such as improved learning outcomes, greater class attendance, and encouraged active learning

Notes On Contributors

Ryan Denkewicz BS BioMedical Engineer is a 4th year Medical student at the Frank H. Netter School of Medicine at Quinnipiac University, North Haven, Connecticut. His academic interests include understanding the means by which students are motivated to learn. ORCID ID https://orcid.org/0000-0002-5683-5803

Acknowledgements

The author gratefully acknowledges the input and guidance of Professor Robert Bona throughout the completion of this paper.

Bibliography/References

Abate, L. E., Gomes, A. and Linton, A. (2011) ‘Engaging Students in Active Learning: Use of a Blog and Audience Response System’, Medical Reference Services Quarterly, 30(1), 12-18. https://doi.org/10.1080/02763869.2011.540206

Addison, Stephen, Wright, A. and Milner, R. (2009) ‘Using Clickers to Improve Student Engagement and Performance in an Introductory Biochemistry Class’, Biochemistry and Molecular Biology Education, 37(2), 84–91. https://doi.org/10.1002/bmb.20264

Akl, E. A., Pretorius, R. W., Sackett, K., Erdley, W. S., et al., (2010) ‘The effect of educational games on medical students’ learning outcomes: A systematic review’, BEME Guide No 14. Medical Teacher, 32(1), 16-27. https://doi.org/10.3109/01421590903473969

Alexander, C. J., Crescini, W. M., Juskewitch, J. E., Lachman, N., et al., (2009) ‘Assessing the integration of audience response system technology in teaching of anatomical sciences’, Anatomical Sciences Education, 2(4), 160-166. https://doi.org/10.1002/ase.99

Allen, D. and Tanner, K. (2005) ‘Infusing Active Learning into the Large-enrollment Biology Class: Seven Strategies, from the Simple to Complex’, Cell Biology Education, 4(4), 262-268. https://doi.org/10.1187/cbe.05-08-0113

Arneja, J. S., Narasimhan, K., Bouwman, D. and Bridge, P. D. (2009) ‘Qualitative and Quantitative Outcomes of Audience Response Systems as an Educational Tool in a Plastic Surgery Residency Program’, Plastic and Reconstructive Surgery, 124(6), 2179-2184. https://doi.org/10.1097/prs.0b013e3181bcf11f

Baker, P. R., Francis, D. P. and Cathcart, A. (2017) ‘A Mock Randomized Controlled Trial With Audience Response Technology for Teaching and Learning Epidemiology’, Asia Pacific Journal of Public Health, 29(3), 229-240. https://doi.org/10.1177/1010539517700473

Barber, M. and Njus, D. (2007) ‘Clicker Evolution: Seeking Intelligent Design’, CBE—Life Sciences Education, 6(1), 1-8. https://doi.org/10.1187/cbe.06-12-0206

Barbour, M. E. (2008) ‘Electronic Voting in Dental Materials Education: the Impact on Students’ Attitudes and Exam Performance’, Journal of Dental Education, 72(9), 1042–1047.

Beatty, I. D., Leonard, W. J., Gerace, W. J. and Dufresne, R. J. (2006) ‘Question Driven Instructions: Teaching Science (Well) with an Audience Response System’, Information Science Publishing, 96–115. https://doi.org/10.4018/978-1-59140-947-2.ch007

Beaumont, A., Gousseau, M., Sommerfeld, C., Leitao, D. et al., (2017) ‘Mobile Audience Response Systems at a Continuing Medical Education Conference’, IOS Press.

Beekes, W. (2006) ‘The 'Millionaire' Method for Encouraging Participation’, Active Learning Higher Education, 7, 25–36. https://doi.org/10.1177/1469787406061143

Benson, J. D., Szucs, K. A., DeIuliis, E. D. and Leri, A. (2017) ‘Impact of Student Response Systems on Initial Learning and Retention of Course Content in Health Sciences Students’, Journal of Allied Health, 46(3), 158–163.

Benson, J. D., Szucs, K. A. and Taylor, M. (2016) ‘Student Response Systems and Learning: Perceptions of the Student’, Occupational Therapy In Health Care, 30(4), 406–414. https://doi.org/10.1080/07380577.2016.1222644

Berry, J. (2009) ‘Technology Support in Nursing Education: Clickers in the Classroom’, Nursing Education Perspective, 5, 295–298.

Bloom, B. S. (1956) Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain. New York: David McKay Co Inc.

Bonwell, C. and Eison J. (1991) ‘Active Learning: Creating Excitement in the Classroom’, Washington, DC: George Washington University.

Boyle, J. (2006) ‘Eight Years of Asking Questions’, Audience Response Systems in Higher Education, 289–304. https://doi.org/10.4018/978-1-59140-947-2.ch019

Brady, M. and Forest, C. P. (2018) ‘Metacognition, Formative Assessment, and Student Perspective’, The Journal of Physician Assistant Education, 29(2), 104–108. https://doi.org/10.1097/jpa.0000000000000203

Brady, M., Seli, H. and Rosenthal, J. (2013) ‘Clickers’ and Metacognition: A Quasi-Experimental Comparative Study about Metacognitive Self-Regulation and Use of Electronic Feedback Devices’, Computers & Education, 65, 56–63. https://doi.org/10.1016/j.compedu.2013.02.001

Bruff, D. (2018) Classroom Response Systems (‘Clickers’), Vanderbilt University. Available at: https://wp0.vanderbilt.edu/cft/guides-sub-pages/clickers (Accessed 15 May 2019).

Bruff, D. (2015) Clickers and Classroom Dynamics, NEA, Vanderbilt University. Available at: www.nea.org/home/34690.htm (Accessed 15 May 2019).

Bunce, D. M., VandenPlas, J. R. and Havanki, K. L. (2006) ‘Comparing the Effectiveness on Student Achievement of a Student Response System versus Online WebCT Quizzes’, Journal of Chemical Education, 83(3), 488. https://doi.org/10.1021/ed083p488

Cain, J. and Robinson, E. (2008) ‘A Primer on Audience Response Systems: Current Applications and Future Considerations’, American Journal of Pharmaceutical Education, 72(4), 77. https://doi.org/10.5688/aj720477

Caldwell, J. E. (2007) ‘Clickers in the Large Classroom: Current Research and Best-Practice Tips’, CBE—Life Sciences Education, 6(1), 9–20. https://doi.org/10.1187/cbe.06-12-0205

Chaudhry, M. A. (2011) ‘Assessment of Microbiology Students’ Progress With an Audience Response System’, Journal of Microbiology & Biology Education, 12(2), 200–201. https://doi.org/10.1128/jmbe.v12i2.306

Clauson, Kevin A., Alkhateeb, F. M. and Singh-Franco, D. (2012) ‘Concurrent Use of an Audience Response System at a Multi-Campus College of Pharmacy’, American Journal of Pharmaceutical Education, 76(1), 6. https://doi.org/10.5688/ajpe7616

Collins, J. (2008) ‘Audience Response Systems: Technology to Engage Learners’, Journal of the American College of Radiology, 5(9), 993–1000. https://doi.org/10.1016/j.jacr.2008.04.008

Connor, E. (2009) ‘Perceptions and Uses of Clicker Technology’, Journal of Electronic Resources in Medical Libraries, 6(1), 19–32. https://doi.org/10.1080/15424060802705145

Copeland, H. L., Longworth, D. L., Hewson, M. G. and Stoller, J. K. (2000) ‘Successful Lecturing’, Journal of General Internal Medicine, 15(6), 366–371. https://doi.org/10.1046/j.1525-1497.2000.06439.x

Crouch, C. and Mazur, E. (2001) ‘Peer Instruction: Ten Years of Experience and Results’, American Journal of Physics, 69(9), 70-977. https://doi.org/10.1119/1.1374249

DeBourgh, G. (2008) ‘Use of Classroom ‘‘Clickers’’ to Promote Acquisition of Advanced Reasoning Skills’, Nurse Education in Practice, 8, 76–87. https://doi.org/10.1016/j.nepr.2007.02.002

DeGagne, J. C. (2011) ‘The Impact of Clickers in Nursing Education: A Review of Literature’, Nurse Education Today, 31(8). https://doi.org/10.1016/j.nedt.2010.12.007

Devitt, P. (2012) Assessment of an Electronic Voting System within the Tutorial Setting: a Randomised Controlled Trial, ISRCTN Registry. Available at: http://Isrctn.org/ (Accessed 18 May 2019).

Dhaliwal, H. K., Allen, M., Kang, J., Bates, C. et al., (2015) ‘The Effect of Using an Audience Response System on Learning, Motivation and Information Retention in the Orthodontic Teaching of Undergraduate Dental Students: a Cross-over Trial’, Journal of Orthodontics, 42(2), 123–135. https://doi.org/10.1179/1465313314y.0000000129

Doucet, M., Vrins, A. and Harvey, D. (2009) ‘Effect of Using an Audience Response System on Learning Environment, Motivation and Long-Term Retention, during Case-Discussions in a Large Group of Undergraduate Veterinary Clinical Pharmacology Students’, Medical Teacher, 31(12). https://doi.org/10.3109/01421590903193539

Draper, S. W. and Brown, M. I. (2004) ‘Increasing Interactivity in Lectures Using an Electronic Voting System’, Journal of Computer Assisted Learning, 20(2), 2004, 81–94. https://doi.org/10.1111/j.1365-2729.2004.00074.x

Draper, S. W., Cargill, J. and Cutts, Q. (2002) ‘Electronically Enhanced Classroom Interaction’,  Australasian Journal of Educational Technology, 18(1). https://doi.org/10.14742/ajet.1744

Dufresne, R. J., Gerace,W. J., Leonard, W. J., Mestre, J. P. and Wenk, L. (1996) ‘Classtalk: A Classroom Communication System for Active Learning’, J of Computing in Higher Education, 7, 3-47.

Duggan, P. M., Palmer, E. and Devitt, P. (2007) ‘Electronic Voting to Encourage Interactive Lectures: a Randomised Trial’, BMC Medical Education, 7(1). https://doi.org/10.1186/1472-6920-7-25

Efstathiou, N. and Bailey, C. (2012) ‘Promoting Active Learning Using Audience Response System in Large Bioscience Classes’, Nurse Education Today, 32(1), 91–95. https://doi.org/10.1016/j.nedt.2011.01.017

Egelandsdal, K. and Krumsvik., R. J. (2015) ‘Clickers and Formative Feedback at University Lectures’, Education and Information Technologies, 22(1), 55–74. https://doi.org/10.1007/s10639-015-9437-x

Elashvili, A., Denehy, G. E., Dawson, D. V. and Cunningham, M. A. (2008) ‘Evaluation of an Audience Response System in a Preclinical Operative Dentistry Course’, Journal of Dental Education, 72(11), 1296–1303.

El-Rady, J. (2006) ‘To click or not to click: That’s the question’, Innovate Journal of Online Education, 2(4).

Fies, C. H. (2005) Classroom Response Systems: What Do They Add to an Active Learning Environment?, Austin: PhD thesis. University of Texas at Austin.

Fissell, M. E. (2016) ‘How I Learned to Love Clickers’, Bulletin of the History of Medicine, 90(3), 540–542. https://doi.org/10.1353/bhm.2016.0076

Fitch, J. L. (2004) ‘Student Feedback in the College Classroom: A Technology Solution’, Educational Technology Research and Development, 52(1), 71–77. https://doi.org/10.1007/bf02504773

Fitzpatrick, K. A., Finn, K. E. and Campisi, J. (2011) ‘Effect of Personal Response Systems on Student Perception and Academic Performance in Courses in a Health Sciences Curriculum’, Advances in Physiology Education, 35(3), 280–289. https://doi.org/10.1152/advan.00036.2011

Freeman, J. and Dobbie, A. (2005) ‘Use of an Audience Response System to Augment Interactive Learning’, Family Medicine, 37, 12–14.

Fujikura, T., Takeshita, T., Homma, H., Adachi, K., et al., (2013) ‘Team-based Learning Using an Audience Response System: A Possible New Strategy for Interactive Medical Education’, Journal of Nippon Medical School, 80(1), 63-69. https://doi.org/10.1272/jnms.80.63

Gauci, S. A., Dantas, A. M., Williams, D. A. and Kemm, R. E. (2009) ‘Promoting student-centered active learning in lectures with a personal response system’, Advances in Physiology Education, 33(1), 60-71. https://doi.org/10.1152/advan.00109.2007

Goldberg, H. R., Haase, E., Shoukas, A. and Schramm, L. (2006) ‘Redefining Classroom Instruction’, Advances in Physiology Education, 30(3), 124–127. https://doi.org/10.1152/advan.00017.2006

Gousseau, M., Sommerfeld, C. and Gooi, A. (2016) ‘Tips for Using Mobile Audience Response Systems in Medical Education’, Advances in Medical Education and Practice, 7, 647–652. https://doi.org/10.2147/amep.s96320

Graeff, E. C., Vail, M., Maldonado, A., Lund, M., et al., (2011) ‘Click It: Assessment of Classroom Response Systems in Physician Assistant Education’, Journal of Allied Health, 40(1), 1–5.

Grimes, C., Joiner-Rodgers, G., Volker, D. and Ramberg, E. (2010) ‘Classroom Performance System Use in an Accelerated Graduate Nursing Program’, CIN: Computers, Informatics, Nursing, 28(2), 86–87. https://doi.org/10.1097/ncn.0b013e3181cd822c

Guse, D. M. and  Zobitz, P. M. (2010) ‘Validation of the Audience Response System’, British Journal of Educational Technology, 42(6), 985–991. https://doi.org/10.1111/j.1467-8535.2010.01120.x

Halloran, L. (1995) ‘A Comparison of Two Methods of Teaching: Computer Managed Instruction and Keypad Questions versus Traditional Classroom Lecture’, Comput. Nursing, 13(6), 285–288.

Han, J. H. (2014) ‘Closing the Missing Links and Opening the Relationships among the Factors: A Literature Review on the Use of Clicker Technology Using the 3P Model’, Educ Tech Soc, 17(4), 150-168.

Handelsman, J., Ebert-May, D., Beichner, R., Bruns, P., et al., (2004) ‘EDUCATION: Scientific Teaching’, Science, 304, 521-522. https://doi.org/10.1126/science.1096022

Harvey, G. (2015) ‘Connecting Theory to Practice: Using Guided Questions to Standardize Clinical Post-conference’, Journal of Nursing Education, 54(11), 655–658. https://doi.org/10.3928/01484834-20151016-08

Hatch, J., Jenson, M. and Moore, R. (2005) ‘Manna from Heaven or ‘Clickers’ from Hell’,  Journal of College Science Teaching, 24, 36–39.

Hettinger, A., Spurgeon, J., El-Mallakh, R. and Fitzgerald, B. (2014) ‘Using Audience Response System Technology and PRITE Questions to Improve Psychiatric Residents’ Medical Knowledge’, Academic Psychiatry, 38(2), 205–208. https://doi.org/10.1007/s40596-014-0058-2

Homme, J., Asay, G. and Morgenstern, B. (2004) ‘Utilization of an Audience Response System’, Medical Education, 38, 575. https://doi.org/10.1111/j.1365-2929.2004.01888.x

Horowitz, H. M. (1988) Student Response Systems: Interactivity in a Classroom Environment. IBM Corporation, Available at: www.einstruction.com/News/index.cfm? (Accessed 20 May 2019).

Hoyt, A., Mcnulty, J. A., Gruener, G., Chandrasekhar, A., et al., (2010) ‘An audience response system may influence student performance on anatomy examination questions’, Anatomical Sciences Education, 3(6), 295-299. https://doi.org/10.1002/ase.184

Hudson, J. N. and Bristow, D. R. (2006) ‘Formative Assessment Can Be Fun as Well as Educational’, Advances in Physiology Education, 30(1), 33–37. https://doi.org/10.1152/advan.00040.2005

Hunsu, N. J., Adesope, O. and Bayly, D. J. (2016) ‘A Meta-Analysis of the Effects of Audience Response Systems (Clicker-Based Technologies) on Cognition and Affect’, Computers & Education, 94, 102–119. https://doi.org/10.1016/j.compedu.2015.11.013

Hunter-Revell, S. M. and McCurry, M. K. (2010) ‘Engaging Millennial Learners: Effectiveness of Personal Response System Technology with Nursing Students in Small and Large Classrooms’, Journal of Nursing Education, 49(5), 272-275. https://doi.org/10.3928/01484834-20091217-07

Ioannou, A. and Artino, Jr., A. R. (2007) ‘Using a Classroom Response System to Support Active Learning in an Educational Psychology Course: A Case Study’, University of Connecticut.

Jackson, M.H. and Trees, A. R. (2003) Clicker Implementation and Assessment.  Colorado University. Available at: comm.colorado.edu/mjackson/clickreport (Accessed 20 May 2019).

Jelsing, E. J., Lachman, N., O’Neil, A. E. and Pawlina, W. (2007) ‘Can a Flexible Medical Curriculum Promote Student Learning and Satisfaction?’, Ann Academy of Medicine Singapore, 36, 713–718.

Johnson, J. T. (2005) ‘Creating Learner-Centered Classrooms: Use of an Audience Response System in Pediatric Dentistry Education’, Journal of Dental Education, 69(3), 378–381.

Jones, C., Connolly, M., Gear, A. and Read, M. (2001) ‘Group Interactive Learning with Group Process Support Technology’, British Journal of Educational Technology, 32(5), 571–586. https://doi.org/10.1111/1467-8535.00226

Kay, R. H. and Lesage, A. (2009) ‘Examining the Benefits and Challenges of Using Audience Response Systems: A Review of the Literature’, Computers & Education, 53(3), 819–827. https://doi.org/10.1016/j.compedu.2009.05.001

Keengwe, J. (2007) ‘Faculty Integration of Technology into Instruction and Students’ Perceptions of Computer Technology to Improve Student Learning’, Journal of Information Technology Education: Research, 6, 169–180. https://doi.org/10.28945/208

Kennedy, G. E. and Cutts, Q. I. (2005) ‘The Association between Students' Use of an Electronic Voting System and Their Learning Outcomes’, Journal of Computer Assisted Learning, 21(4), 260–268. https://doi.org/10.1111/j.1365-2729.2005.00133.x

Knight, J. K. and Wood, W. B. (2005) ‘Teaching More by Lecturing Less’, Cell Biology Education, 4(4), 298–310. https://doi.org/10.1187/05-06-0082

Kung, J. W., Slanetz, P. J., Chen, P., Lee, K. S., et al., (2012) ‘Resident and Attending Physician Attitudes Regarding an Audience Response System’, Journal of the American College of Radiology, 9(11), 828-831. https://doi.org/10.1016/j.jacr.2012.06.004

Latessa, R. and Mouw, D. (2005) ‘Use of an Audience Response System to Augment Interactive Learning’, Family Medicine, 37, 12–14.

Laurillard, D.  (2002) Rethinking university teaching: A conversational framework for the effective use of learning technologies. London: RoutledgeFalmer, 2nd Editionhttps://doi.org/10.4324/9780203304846

Leeds, I. L., Dibrito, S. R., Jones, C. D., Higgins, R. S., et al., (2018) ‘Using Audience Response Systems for Real-Time Learning Assessments During Surgical Morbidity and Mortality Conference’, Journal of Surgical Education, 75(6), 1535-1543. https://doi.org/10.1016/j.jsurg.2018.05.010

Leidner, D. E. and Jarvenpaa, S. L. (1995) ‘The Use of Information Technology to Enhance Management School Education: A Theoretical View’, MIS Quarterly, 19(3), 265. https://doi.org/10.2307/249596

Leung, C. P., Klausner, A. P., Habiba, J. R., King, A. B., et al., (2013) ‘Audience response system: a new learning tool for urologic conferences’, Canadian Journal of Urology, 20(6), 7042-7045.

Liu, F. C., Gettig, J. P. and Fjortoft, N. (2010) ‘Impact of a Student Response System on Short- and Long-Term Learning in a Drug Literature Evaluation Course’, American Journal of Pharmaceutical Education, 74(1), 6. https://doi.org/10.5688/aj740106

Llena, C., Forner, L. and Cueva, R. (2015) ‘Student Evaluation of Clickers in a Dental Pathology Course’, Journal of Clinical Experimental Dentistry, 7(3), 369–373. https://doi.org/10.4317/jced.52299

Lymn, J. S. and Mostyn, A. (2010) ‘Audience Response Technology: Engaging and Empowering Non-Medical Prescribing Students in Pharmacology Learning’, BMC Medical Education, 10, 73. https://doi.org/10.1186/1472-6920-10-73

Mahon, P., Lyng, C., Crotty, Y. and Farren, M. (2018) ‘Transforming classroom questioning using emerging technology’, British Journal of Nursing, 27(7), 389-394. https://doi.org/10.12968/bjon.2018.27.7.389

Mains, T. E., Cofrancesco, J., Milner, S. M., Shah, N. G., et al., (2015) ‘Do questions help? The impact of audience response systems on medical student learning: A randomised controlled trial’, Postgraduate Medical Journal, 91(1077), 361-367. https://doi.org/10.1136/postgradmedj-2014-132987

Mareno, N., Bremner, M. and Emerdson, C. (2010) ‘The Use of Audience Response Systems in Nursing Education: Best Practice Guidelines’, International Journal of Nursing Education Scholarship, 7(1), 32. https://doi.org/10.2202/1548-923x.2049

Mastoridis, S. and Kladidis, S. (2010) ‘Coming Soon to a Lecture Theatre near You: the ‘Clicker’, The Clinical Teacher, 7(2), 97–101. https://doi.org/10.1111/j.1743-498x.2010.00355.x

Mazur, E. (1997) ‘Peer instruction: Getting students to think in class’, AIP Conference Proceedings, pp. 981–988. Available at: https://aip.scitation.org/doi/abs/10.1063/1.53199.

McKeachie, W. (1994) ‘Learning and cognition in the college classroom’, in McKeachie, W. J. (ed.) Teaching Tips. 9th edn. Lexington, MA: DC Health, pp. 279-295.

Meedzan, N. and Fisher, K. (2009) ‘Clickers in nursing education: An active learning tool in the classroom’, Online Journal of Nursing Informatics, 13, 1-19.

Menon, A. S., Moffett, S., Enriquez, M., Martinez, M. M., et al., (2004) ‘Audience Response Made Easy: Using Personal Digital Assistants as a Classroom Polling Tool’, Journal of the American Medical Informatics Association, 11(3), 217-220. https://doi.org/10.1197/jamia.m1468

Meyers, C. and Jones, T. B. (1993). Promoting Active Learning. Strategies for the College Classroom. San Francisco: Jossey-Bass Inc.

Michael, K. (2006) ‘Where’s the evidence that active learning works?’, Adv Physiol Educ. 30, 159-167. https://doi.org/10.1152/advan.00053.2006

Miller, C. J. and Metz, M. J. (2014) ‘A Comparison of Professional-Level Faculty and Student Perceptions of Active Learning: Its Current Use, Effectiveness, and Barriers’, Advances in Physiology Education, 38(3), 246–252. https://doi.org/10.1152/advan.00014.2014

Miller, R. G., Ashar, B. H. and Getz, K. J. (2003) ‘Evaluation of an Audience Response System for the Continuing Education of Health Professionals’, Journal of Continuing Education in the Health Professions, 23(2), 109–115. https://doi.org/10.1002/chp.1340230208

Morrell, L. J. and Joyce, D. A. (2015) ‘Interactive Lectures: Clickers or Personal Devices?’, F1000 Research, 4(64). https://doi.org/10.12688/f1000research.6207.1

Mostyn, A., Meade, O. and Lymn, J. S. (2012) ‘Using Audience Response Technology to Provide Formative Feedback on Pharmacology Performance for Non-Medical Prescribing Students - a Preliminary Evaluation’, BMC Medical Education, 12(1), 2012. https://doi.org/10.1186/1472-6920-12-113

Nasmith, L. and Steinert, Y. (2001) ‘The Evaluation of a Workshop to Promote Interactive Lecturing’, Teaching and Learning in Medicine, 13(1), 43–48. https://doi.org/10.1207/s15328015tlm1301_8

Nayak, L. and Erinjeri, J. P. (2008) ‘Audience Response Systems in Medical Student Education Benefit Learners and Presenters’, Academic Radiology, 15(3), 383–389. https://doi.org/10.1016/j.acra.2007.09.021

Nelson, C., Hartling, L., Campbell, S. and Oswald, A. E. (2012) ‘The Effects of Audience Response Systems on Learning Outcomes in Health Professions Education. A BEME Systematic’, Review: BEME Guide No. 21,Medical Teacher, 34(6). https://doi.org/10.3109/0142159x.2012.680938

Nosek, T., Wang, W., Medvedev, I., Wile, L., et al., (2006) ‘Use of a Computerized Audience Response System in Medical Student Teaching: Its Effect on Exam Performance’, Medical Teacher, 28(8), 736–738.

Palmer, E. J., Devitt, P. G., De Young, N. J. and Morris, D. (2005) ‘Assessment of an Electronic Voting System within the Tutorial Setting: A Randomised Controlled Trial’, BMC Medical Education, 5(24). https://doi.org/10.1186/1472-6920-5-24

Parsons, C.V. (2005) ‘Decision Making in the Process of Differentiation’, Learning & Leading with Technology, 33(1), 8–10.

Patterson, B., Kilpatrick, J. and Woebkenberg, E. (2010) ‘Evidence for Teaching Practice: The Impact of Clickers in a Large Classroom Environment’, Nurse Education Today, 30(7), 603–607. https://doi.org/10.1016/j.nedt.2009.12.008

Pettit, R. K., McCoy, L., Kinney, M. and Schwartz, F. N. (2014) ‘A Multimedia Audience Response Game Show for Medical Education’, Medical Science Educator, 24(2), 181–187. https://doi.org/10.1007/s40670-014-0038-x

Plant, J. D. (2007) ‘Incorporating an Audience Response System into Veterinary Dermatology Lectures: Effect on Student Knowledge Retention and Satisfaction’, Journal of Veterinary Medical Education, 34(5), 674–677. https://doi.org/10.3138/jvme.34.5.674

Pradhan, A., Sparano, D. and Ananth, C. V. (2005) ‘The Influence of an Audience Response System on Knowledge Retention: An Application to Resident Education’, American Journal of Obstetrics and Gynecology, 193(5), 1827–1830. https://doi.org/10.1016/j.ajog.2005.07.075

Premkumar, K. and Coupal, C. (2008) ‘Rules of Engagement – 12 Tips for Successful Use of ‘‘Clickers’’ in the Classroom’, Medical Teacher, 30, 146-149. https://doi.org/10.1080/01421590801965111

Preszler, R. W., Dawe, A., Shuster, C. B. and Shuster, M. (2007) ‘Assessment of the Effects of Student Response Systems on Student Learning and Attitudes over a Broad Range of Biology Courses’, CBE—Life Sciences Education, 6(1), 29–41. https://doi.org/10.1187/cbe.06-09-0190

Prince, M. (2004) ‘Does active learning work? A review of the research’, J Engin Educ., 93(3), 223–231. https://doi.org/10.1002/j.2168-9830.2004.tb00809.x

Robertson, L. J.  (2000) ‘Twelve Tips for Using a Computerized Interactive Audience Response System’, Medical Teacher, 22(3), 237–239. https://doi.org/10.1080/01421590050006179

Roschelle, J. (2003) ‘Keynote Paper: Unlocking the Learning Value of Wireless Mobile Devices’, Journal of Computer Assisted Learning, 19(3), 260–272. https://doi.org/10.1046/j.0266-4909.2003.00028.x

Rubio, E. I., Bassignani, M. J., White, M. A. and Brant, W. E. (2008) ‘Effect of an Audience Response System on Resident Learning and Retention of Lecture Material’, American Journal of Roentgenology, 190(6). https://doi.org/10.2214/ajr.07.3038

Rush, B. R., Hafen, M., Biller, D. S., Davis, E. G., et al., (2010) ‘The Effect of Differing Audience Response System Question Types on Student Attention in the Veterinary Medical Classroom’, Journal of Veterinary Medical Education, 37(2), 145-153. https://doi.org/10.3138/jvme.37.2.145

Satheesh, K. M., Saylor-Boles, C. D., Rapley, J. W., Liu, Y., et al., (2013) ‘Student Evaluation of Clickers in a Combined Dental and Dental Hygiene Periodontology Course’, Journal of Dental Education, 77(10), 1321–1329.

Schackow, T. E., Chavez, M., Loya, L. and Friedman, M. (2004) ‘Audience Response System: Effect on Learning in Family Medicine Residents’, Family Medicine, 3(6), 496–504.

Schlegel, E. F. M. and Selfridge, N. J. (2014) ‘Fun, Collaboration and Formative Assessment: Skinquizition, a Class Wide Gaming Competition in a Medical School with a Large Class’, Medical Teacher, 36(5), 447–449. https://doi.org/10.3109/0142159x.2014.888409

Siddiqui, F., Iqbal, I. and Azizi, S. (2017) ‘Using Mobile Audience Response Systems to Enhance Medical Education: a Medical Student Perspective’, Advances in Medical Education and Practice, 8, 325–327. https://doi.org/10.2147/amep.s138025

Simpson, V. and Oliver, M. (2007) ‘Electronic Voting Systems for Lectures Then and Now: A Comparison of Research and Practice’, Australasian Journal of Educational Technology, 23(2), 2007. https://doi.org/10.14742/ajet.1264

Skiba, D. (2006) ‘Got Large Lecture Hall Classes? Use Clickers’, Nursing Education Perspective, 27, 278–280.

Slain, D., Abate, M., Hodges, B. M., Stamatakis, M. K., et al., (2004) ‘An Interactive Response System to Promote Active Learning in the Doctor of Pharmacy Curriculum’, American Journal of Pharmaceutical Education, 68(5), 117. https://doi.org/10.5688/aj6805117

Stevens, N. T., Mcdermott, H., Boland, F., Pawlikowska, T., et al., (2017) ‘A comparative study: Do “clickers” increase student engagement in multidisciplinary clinical microbiology teaching?’, BMC Medical Education, 17(1). https://doi.org/10.1186/s12909-017-0906-3

Stoddard, H. A. and Piquette, C. A. (2010) ‘A Controlled Study of Improvements in Student Exam Performance With the Use of an Audience Response System During Medical School Lectures’, Academic Medicine, 85. https://doi.org/10.1097/acm.0b013e3181ed3b40

Streeter, J. L. and Rybicki, F. J. (2006) ‘A Novel Standard-Compliant Audience Response System for Medical Education’, RadioGraphics, 26(4), 1243–1249. https://doi.org/10.1148/rg.264055212

Stuart, S. A. J., Brown, M. I. and Draper, S. W. (2004) ‘Using an Electronic Voting System in Logic Lectures: One Practitioner's Application’, Journal of Computer Assisted Learning, 20(2), 95–102. https://doi.org/10.1111/j.1365-2729.2004.00075.x

Subramanian, A., Timberlake, M., Mittakanti, H., Lara, M., et al., (2012) ‘Novel Educational Approach for Medical Students: Improved Retention Rates Using Interactive Medical Software Compared with Traditional Lecture-Based Format’, Journal of Surgical Education, 69(4), 449-452. https://doi.org/10.1016/j.jsurg.2012.05.013

Thampy, H. and Ahmad, Z. (2014) ‘How to Use Audience Response Systems’, Education for Primary Care, 25(5), 294–296. https://doi.org/10.1080/14739879.2014.11494296

Toothaker, R. (2018) ‘Millennial's Perspective of Clicker Technology in a Nursing Classroom: A Mixed Methods Research Study’, Nurse Education Today, 62, 80–84. https://doi.org/10.1016/j.nedt.2017.12.027

Torbeck, L. (2007) ‘Enhancing Programme Evaluation Using the Audience Response System’, Medical Education, 41(11), 1088–1089. https://doi.org/10.1111/j.1365-2923.2007.02864.x

Trapskin, P. J., Smith, K. M., Armistead, J. A. and Davis, G. A. (2005) ‘Use of an Audience Response System to Introduce an Anticoagulation Guide to Physicians, Pharmacists, and Pharmacy Students’, American Journal of Pharmaceutical Education, 69(2), 28. https://doi.org/10.5688/aj690228

Tregonning, A. M., Doherty, D. A., Hornbuckle, J. and Dickinson, J. E. (2012) ‘The Audience Response System and Knowledge Gain: A Prospective Study’, Medical Teacher, 34(4). https://doi.org/10.3109/0142159x.2012.660218

Turpin, D L. (2003) ‘Enhance Learning with an Audience Response System’, American Journal of Orthodonic Dentofacial Orthopedics, 124, 607. https://doi.org/10.1016/j.ajodo.2003.10.009

Uhari, M., Renko, M. and Soini, H. (2003) ‘Experiences of Using an Interactive Audience Response System in Lectures’, BMC Medical Education, 3(1). https://doi.org/10.1186/1472-6920-3-12

Vana, K. D., Silva, G. E., Muzyka, D. and Hirani, L. M. (2011) ‘Effectiveness of an Audience Response System in Teaching Pharmacology to Baccalaureate Nursing Students’, CIN: Computers, Informatics, Nursing, 29. https://doi.org/10.1097/ncn.0b013e3182285d71

Wait, K. R., Cloud, B. A., Forster, L. A., Jones, T. M., et al., (2009) ‘Use of an Audience Response System during Peer Teaching among Physical Therapy Students in Human Gross Anatomy: Perceptions of Peer Teachers and Students’, Anatomical Sciences Education, 2(6), 286–293. https://doi.org/10.1002/ase.107

Welch, S. (2013) ‘Effectiveness of Classroom Response Systems Within an Active Learning Environment’, Journal of Nursing Education, 52(11), 653–656. https://doi.org/10.3928/01484834-20131014-01

Wenz, H. J., Zupanic, M., Klosa, K., Schneider, B., et al., (2014) ‘Using an Audience Response System to Improve Learning Success in Practical Skills Training Courses in Dental Studies - a Randomised, Controlled Cross-over Study’, European Journal of Dental Education, 18(3), 147–153. https://doi.org/10.1111/eje.12071

White, G. (2011) ‘Interactive Lecturing’, Clinical Teaching, 8, 230-235. https://doi.org/10.1111/j.1743-498x.2011.00457.x

Wood, W. (2004) ‘Clickers A Teaching Gimmick That Works’, Developmental Cell, 7(6), 796–798. https://doi.org/10.1016/s1534-5807(04)00414-9

Zurmehly, J. and Leadingham, C. (2008) ‘Exploring Student Response Systems in Nursing Education’, CIN: Computers, Informatics, Nursing, 26(5), 271–272. https://doi.org/10.1097/01.ncn.0000304840.36960.b5

Appendices

None.

Declarations

There are no conflicts of interest.
This has been published under Creative Commons "CC BY-SA 4.0" (https://creativecommons.org/licenses/by-sa/4.0/)

Ethics Statement

An ethics review was not required as this is a literature review only.

External Funding

This article has not had any External Funding

Reviews

Please Login or Register an Account before submitting a Review

Lubna Jahanzeb - (05/10/2019)
/
A very detailed and informative review. It explains the pros and cos in a very deep, insightful and interesting manner. Each point in this article is supported by a strong reference which increases the worth of paper. Moreover, it is great to see students working in this important area.
BALAJI ARUMUGAM - (27/09/2019) Panel Member Icon
/
A review of 120 articles on audience response system and narrating the pros and cons of the ARS was very clearly mentioned and the study looks comprehensive. Actually its a great effort by the authors to publish such a narrative review on ARS. The various names of the ARS was mentioned in the introduction which was quoted with references was amazing. These studies are still a tough to conduct and get the feedback from the students in the resource poor settings. The advanced technology and the methods of getting ARS was nicely narrated in the study. The listed pros and cons with various studies demonstrating with the statistical significance. The cited evidence of improved learning outcomes presented from the recent studies of 2005 to 2015. Overall this study will be very useful for the faculty those who are working on this area of interest.
Possible Conflict of Interest:

NONE

Julie Williamson - (24/09/2019) Panel Member Icon
/
This review paper summarizes the basic pros/cons of using audience response devices, or "clickers", in lecture-based pre-medical and medical education. The author does a good job of assembling diverse literature into themes and correctly identifies many of the benefits and the relatively few drawbacks of using these devices in medical education. I think the article is helpful for educators who may be considering implementing the use of ARS devices in their lectures.

The article could have benefited from having more details on the literature search criteria used, including the dates of articles searched and how articles were determined to be included or excluded. Also, although the article described the use of multiple choice questions embedded in lectures--the most common use of ARS devices in education---it missed mentioning the many other uses of these devices, which include uses such as providing free text answers or survey data to guide how instructors cover topics, or using the self-paced poling feature to allow students to submit answers to quizzes or tests with instant data capture.
Possible Conflict of Interest:

For transparency, I am an Associate Editor of MedEdPublish. However I have posted this review as a member of the review panel with relevant expertise and so this review represents a personal, not institutional, opinion.