ORIGINAL RESEARCH

A Multimedia Audience Response Game Showfor Medical Education

Robin K. Pettit & Lise McCoy & Marjorie Kinney &

Frederic N. Schwartz

Published online: 3 April 2014# International Association of Medical Science Educators 2014

Abstract Games are increasingly popular in medical educa-tion. However, there is a need for games that target today’slearners, including their preferences for active participation,social interaction, immediate feedback, and multimedia for-mats. With these preferences in mind, a commercially avail-able game show template was used to develop a game showfor review of medical microbiology. The game show wascombined with an audience response system (“clickers”) toenable participation of all students in a large group setting. A19-item questionnaire was used to measure students’ percep-tions of the game. The questionnaire was administered afterparticipants had played the games on three separate occasionsduring their first year of medical school. The response ofmedical students to the game shows was overwhelminglypositive. Students valued the ability of the game shows toengage them, to provide a positive learning environment, toclarify concepts, and to develop clinical thinking. The gamesoftware combined with an audience response system pro-vides a visually rich, engaging format that could be used for areview of any basic science discipline.

Keywords Game show . Clicker .Medical microbiology .

Bravo C3 Softworks™

Introduction

Undergraduate medical school classes are now largely madeup of net generation students, or students born between 1982and 1991 [1]. Net generation students have a high degree of

technological literacy and a preference for active versus pas-sive learning [1].

The effectiveness of active learning on a broad range oflearning outcomes is widely accepted [2–4]. Students whoengage interactively with each other and with the instructorlearn concepts better, retain them longer, and can apply themmore effectively in other contexts than students who sit pas-sively listening [2, 5]. Two core elements of active learning,student activity and engagement, are central to games.

A survey of family medicine and internal medicine resi-dency program directors in the USA indicated that 80 % usedgames as an educational strategy in their residency trainingprograms [6]. Educational games for medical students includeboard games [7–9], card games [10], video games [11], andgame shows [12–18]. Some of the games inmedical educationare directed toward teaching new concepts [7, 8, 10, 12, 14,16–18], while the purpose of others is review [9, 13, 15, 19].While there is evidence that students find games more enjoy-able and stimulating than standard lectures [16, 18], evidencefor their utility in increasing knowledge is conflicting, perhapsin part due to the limited number of rigorous studies [20].

Several published game shows for medical education arePowerPoint-based. Jirasevijinda and Brown [14] and Shiromaet al. [18] describe a PowerPoint-based Jeopardy game, andMoy et al. [15] describe a PowerPoint-basedWhoWants to bea Millionaire game. Other medical school programs have usednon-electronic game show formats [13, 16, 17]. In 2008, Aklet al. [12] published a description of Guide-O-Game, aninteractive Jeopardy game developed by IT specialists at theiruniversity. The authors report that it was expensive to imple-ment and time-consuming to develop.

Given the popularity of game shows in medical education[12–18], a template for creating multimedia games for largegroup settings that are easy to build and implement is needed.We developed a review tool using a commercially availableelectronic game show template, Bravo C3 Softworks™. To

R. K. Pettit (*) : L. McCoy :M. Kinney : F. N. SchwartzSchool of Osteopathic Medicine in Arizona, A. T. Still University,5850 E. Still Circle, Mesa, AZ 85206, USAe-mail: rpettit@atsu.edu

Med.Sci.Educ. (2014) 24:181–187DOI 10.1007/s40670-014-0038-x

our knowledge, the application of Bravo C3 Softworks™training software to educational settings has not beenpublished.

The software uses a customizable gameboard template thatcan be enhanced by adding video, audio, and graphics. Whilethe software can be used to create teaching games or reviewgames, the games described here targeted the review of med-ical microbiology knowledge. The games were combinedwith an audience response system to enable participation ofall students in a large group setting.

The game shows were developed and then implemented inthree different organ system courses during year 1 of anosteopathic medical curriculum. With the goal of improvinginstruction in our program, particularly with regard to increas-ing the number and types of interactive sessions, we posed thefollowing research questions: What are students’ perceptionsof the game shows regarding the ability of the games toengage, provide a positive learning environment, clarify con-cepts and develop clinical thinking?

Materials and Methods

Game Show Software

Game show software was purchased from C3 SoftWorks(Minneapolis, MN). Wireless keypads (clickers) and a receiv-er were purchased from Turning Technologies (Youngstown,OH). Bravo C3 Softworks™ has several game show tem-plates, for example, Quiz Show, Billionaire, and Spin-Off.The Spin-Off game was selected for this study.

Construction of the Game Shows

A menu for customizing various features of the game appearswhen the game builder interface opens (Fig. 1a, left side).Under the question tab (Fig. 1b, left side), questions andanswers were copied/pasted from Microsoft Word into thegame template (one can also create and edit questions withinthe question interface). The Spin-Off game allows a maximumof 50 questions. To improve readability in large group set-tings, the question preview option was selected for questionswith long stems. The question builder interface also has anoption for adding a review screen that displays once studentsanswer a question. The review screen can facilitate teaching,clarification, or reinforcement. Images (jpg, gif, png) andvideo clips (swf, flv) were imported into the correspondingquestion template. Audio clips (mp3) can also be imported.Using the teams tab, team avatars were customized prior to thegame using a large selection of attributes available in thesoftware. For games in the current study, we used the attri-butes to create male and female doctor avatars (Fig. 1a, mid-dle). If there is time prior to game play, students can select or

customize their own avatars, which stimulates investment inthe game at the outset. Under the settings tab, we set the gameto automatically go to the next question within a category(instead of spinning again, which saves time in class), and setthe time to answer each question to 40 s. Keypads wereselected as the response type using the response settings tab.Sounds and graphics can also be customized, but for thesegames, default settings were used.

Building and implementing these games is quite straight-forward. The game software is very user-friendly, and if gamequestions are prebuilt and copied/pasted into the game tem-plate, a game can be created in 1 h to a few hours (dependingon the number of questions).

Implementation of the Game Shows

The game is controlled by the instructor at the podium using amouse. Students view the game on eight, 50-in. plasma flatscreens in our large group classroom. At the outset of thegame, each student joins their team (male or female doctors)by simply pressing any button on their clickers. The instructorspins the wheel to select question categories, students individ-ually respond to questions in each category with their clickers,and the instructor clicks through correct answers, team scores,and review slides. At the end of the game, a detailed report ofall of the results is displayed and can be transferred to aMicrosoft Excel file. More details of game play, includingthe entertaining and competitive features that distinguish thisactivity from other types of games, can be found in the“Discussion” section.

Construction of the Survey Instrument

A 19-item questionnaire was used to measure students’ per-ceptions of the games. A literature search facilitated the designof key constructs for this survey [1, 7, 8, 10, 15, 16, 18].Students’ perceptions of the games were measured in fourcategories: the ability of the games to engage (four questions),provide a positive learning environment (four questions),clarify concepts (four questions), and develop clinical think-ing (four questions). Three questions at the beginning of thesurvey addressed age range (22–25, 26–30, 31–35, 36–40,41+), gender, and number of times the students had partici-pated in these games (0, 1, 2, 3). The A. T. Still UniversityInstitutional Review Board deemed the study exempt.

Data Collection

Survey data collection involved an email solicitation contain-ing a clickable link to an online survey. A reminder email wassent 1 week later. Participation was voluntary and anonymous.There were no rewards offered for completing the survey.Students were asked to evaluate the extent to which they

182 Med.Sci.Educ. (2014) 24:181–187

agreed with the statements using a Likert five-point ratingscale (1, strongly agree; 2, agree; 3, neutral; 4, disagree; 5,strongly disagree). A general comment section was included(other comments) at the end of the survey. The questionnairewas administered after participants had played the games onthree separate occasions in three different organ systemcourses (summer 2012, fall 2012, and spring 2013) duringtheir first year of medical school. The student population was107 first-year medical students.

Data Analysis

All statistical analyses were completed using the statisticalanalysis software IBM SPSS Statistics 21™. Responses foreach item were compared to determine if item rankings werethe same across the number of games played (independentsamples, Kruskal-Wallis test). Differences found in itemsusing Kruskal-Wallis tests were then compared individuallyusing repeatedMann-WhitneyU tests. Finally, responses weregrouped according to the four categories: engagement, crea-tion of a positive learning environment, clarification of con-cepts, and practice with clinical thinking. The Likert ratingswere categorized into either positive or neutral/negative

response, combining “strongly agree” and “agree” into thepositive category and “neutral,” “disagree,” and “stronglydisagree” into the neutral/negative response. Responses toeach of the four categories were compared using a chi-square test.

Results

Students’ perceptions of the game shows were queried in fourareas: engagement, creation of a positive learning environ-ment, clarification of concepts, and practice with clinicalthinking. A total of 73 students (68.2 %) in the class of 2016completed the perception survey. Four respondents were omit-ted from analysis because these students selected zero for thenumber of times they had participated in the games. Onehundred and seven students are enrolled in the class of 2016,but attendance for large group sessions, where the games wereoffered, is optional (lecture capture technology is used in ourprogram). Of the remaining 69 respondents, 36 were male, 32were female, and one student did not specify gender. Therewere 37 respondents aged 22–25, 20 aged 26–30, 7 aged 31–35, 4 aged 36–40, and 1 student over age 41. Forty students

Fig. 1 Screenshots of the game builder menu and customized avatars (a), question builder interface (b), student’s response feedback during a game (c),and team results feedback during a game (d)

Med.Sci.Educ. (2014) 24:181–187 183

had played three games, 17 students had played two games,and 12 students had played one game.

Figure 2 summarizes student responses to Likert-scalestatements about their perceptions of the games. The majorityof the respondents agreed or strongly agreed that the gamesoffered an engaging format (Fig. 2(a)), provided a positivelearning environment (Fig. 2(b)), clarified concepts(Fig. 2(c)), and developed clinical thinking (Fig. 2(d)). Therewere no statistically significant differences in rankings for anyof the questions when grouped according to number of gamesplayed (independent samples, Kruskal-Wallis tests, p valuesrange from 0.128 to 0.939).

To determine whether students valued any single categorymore than the others, a chi-square test on the proportion ofpositive responses in each of the four categories (Fig. 2) wasperformed. Likert ratings were categorized into either positiveor neutral/negative response, combining strongly agree andagree into the positive category and neutral, disagree, andstrongly disagree into the neutral/negative category. Therewas no statistically significant difference in the responses tothe four sets of questions (chi-squared=0.507, p=0.917).

Nine students wrote comments in response to the optionalprompt “other comments.” These specific survey commentswere categorized by theme using open coding [21] (Table 1).Five of the statements were extremely positive, providinginsights as to why students felt the games were valuable orfun; students appreciated the competition, unconscious

learning, and the practice exam questions. One student wishedevery teacher would provide similar activities. Six statementsdelivered specific suggestions for improving the games. Keythemes included pace, the length of the stem or case vignette,access to the game show questions outside of game play, andthe timing of the review game with respect to the materialbeing learned. Two responses mentioned speed or pace withinthe games; for one student, the pace was too fast, and for theother student, the pace was acceptable (a positive comment,but grouped here for simplicity). Two responses suggestedshortening the question stems or vignettes. Another two re-sponses requested access to the questions before or after theclass session, and one requested more time to digest coursecontent material prior to playing the game. Two commentswere omitted because they were unclear.

Discussion

We combined the engaging qualities of a multimedia gameshow with the power of an audience response system to createa review tool that would appeal to first year medical students.Net generation students are visual and kinesthetic learners;they gravitate toward activities that promote and reinforcesocial interaction (prefer to learn and work in teams), andexpect immediate responses [1]. Features of the describedgame show that address net generation student preferences

Fig. 2 Student’s perceptions of the game shows in four categories: the ability of the games to engage them (a), provide a positive learning environment(b), clarify concepts (c), and develop clinical thinking (d)

184 Med.Sci.Educ. (2014) 24:181–187

include a multimedia format, physical action (clickers), teamplay, and immediate feedback.

Akl et al. [20] suggest that accessing the potential benefit ofeducational games is best accomplished when three criticalfactors are addressed in game development: active learning,integration of fun and excitement, and feedback mechanisms.The game shows incorporate all of these features. At least inpart because of the clickers, all students in the class areactively involved, as opposed to an instructor-led reviewwhere students sit passively and respond to questions individ-ually. Fun and excitement are generated by the entertainingand competitive features of the game. The game opens withgame show music and colorful graphics, and students log onto their teams (males against females for the games inthis study) with their clickers. As individual studentsclick their answer choices during the game, the count-down timer ticks and then buzzes when the time is up.Other game-generated sounds include the spinningwheel, clapping, and booing. Team results are tabulatedand displayed immediately after each question round, sothe competition heats up regularly. The students contrib-ute to the noise as well, as they send vocal encourage-ment to teammates around the classroom after eachresult display. When the winning team is announcedon-screen, fireworks explode (and the students roar!),and the winning team receives a candy reward.

The game houses multiple feedback mechanisms, the thirdcritical factor in game development. Feedback from formativeassessment, for example, this type of review game, can moti-vate students and redirect their learning towards areas ofdeficiency, particularly if done under conditions that arenon-judgmental and conducive to learning [22]. After thecountdown clock buzzes in the Spin-Off game, the instructorselects the correct answer and the percent of studentsresponding to each answer choice is displayed (Fig. 1c).Misconceptions about concepts can be addressed immediate-ly, with or without a review screen. As described in the“Materials andMethods” section, review screens for teaching,reinforcement, or clarification are easily built into the game.Team results are tabulated immediately and displayed graph-ically on the subsequent screen (Fig. 1d, right side). Teamavatars appear to the left of the results graph, either clapping orgroaning (Fig. 1d, left side). A detailed report of all of theresults is displayed at the end of the game, which allows theinstructor to quickly identify problem areas for furtherinstruction.

In addition to providing feedback, formative assessmentactivities should be an opportunity for students to developfamiliarity with summative instruments [22]. Approximatelyhalf of the questions in the game shows were strict recall,while the other half matched the style used in our summativeassessments, board style with clinical vignette stems and

Table 1 Responses to othercomments prompt, categorized bytheme using open coding [21]

Theme Specific statement

Positive comments

Competition Making it competitive also made it fun and collaborative withineach group

Battle of the sexes was the most fun format

Unconscious learning Very fun activity where you don’t even realize you’re learning

Practice exam questions The in-class games have been a great way to get somemuch-neededpractice questions on difficult information

Request for more games These are amazing!!! I wish every teacher would do this

Suggestions for improving the activities

Pace The pace on these can often be too fast for me to critically thinkabout the material before moving on

I like that it is not a fastest answer response set-up so that everyonehas an opportunity to answer the questions

Length of question stem or vignette I wish that vignettes were a little shorter (maybe bullet pointsinstead of paragraphs)/font a little larger to read on the screens

Sometimes the questions stems are too long for the pace of thegame. If they are long, the font is small and difficult to read

Access to quiz game show questions It would be helpful to have the game accessible at a later time forreview

It would be helpful if a list of the questions given were in a worddocument or sent out after the event so I could review thequestions in preparation for the test

Game schedule Holding themwithin or near to the lecture in which the material wastaught is not helpful, I need time to integrate and learn theinformation before being tested on it

Med.Sci.Educ. (2014) 24:181–187 185

multiple choice answers. The multimedia game show de-scribed is an extremely useful tool for difficult, content-dense subjects like medical microbiology, where games havebeen shown to help counter feelings of despair related toassimilating the large volume of facts and terminology [7].

Student’s responses to the game shows, both net generationstudents and older students (>31 years old), were overwhelm-ingly positive (Fig. 2). The majority of the students agreed orstrongly agreed that the games were engaging (Fig. 2(a)), pro-vided a positive learning environment (Fig. 2(b)), clarifiedconcepts (Fig. 2(c)), and developed clinical thinking(Fig. 2(d)). Statistical analysis indicated that students did notplace more value in any one of the four categories. The positiveresponse to the categories that addressed clarification of con-cepts (Fig. 2(c)) and development of clinical thinking(Fig. 2(d)) indicates that these games were perceived to be avaluable educational tool. It is noteworthy that 87.7 % of thestudents agreed or strongly agreed with the statement I lookforward to playing more games in the classroom. One studentstrongly disagreed with all statements in all categories but didnot submit an optional comment. As such, the student’s dissat-isfaction with the games (or perhaps the instructor!) cannot beaddressed.

While the majority of the students did not provide optionalcomments (Table 1), useful information was gained. In ourlarge group classroom, students sit at round tables, and thereare eight, 50-in. plasma flat screens. As such, smaller fonts canbe difficult to read. For future games in our large groupclassroom, question stems will be shortened as much as pos-sible to improve visibility. Two 80-in. projector screens wererecently added to the classroom, which should also help. Thecomment Holding them within or near to the lecture in whichthe material was taught is not helpful, I need time to integrateand learn the information before being tested on it is under-standable; however, it is extremely difficult to schedule addi-tional time for a particular topic in our integrated clinicalpresentation curriculum [23–25]. A possible solution wouldbe to schedule lunchtime review games several days aftermaterial is delivered.

One limitation of this study is that survey participantsrepresented 64.4 % of the class, and thus, caution is recom-mended in extrapolating results to non-participating studentsand to other medical student populations.

Conclusion

Net generation students have a high degree of technologicalliteracy. They prefer active, first-person experiential learning,interactivity, and image-rich formats versus text. The gameshow described here was developed with these preferences inmind, and it received overwhelmingly positive responses fromfirst-year medical students. The game software combined with

an audience response system provides a visually rich, engag-ing format that should be applicable to any basic sciencediscipline.

Acknowledgments This research was supported by HRSA grant no.D54HP20674.

Notes on Contributors Robin K. Pettit, Ph.D., is a Professor ofMicrobiology at the School of Osteopathic Medicine, A. T. StillUniversity, Mesa, AZ, USA.

Lise McCoy, MTESL, is an Assistant Director, Faculty Developmentat the School of Osteopathic Medicine, A. T. Still University, Mesa, AZ,USA.

Marjorie Kinney, M.Ed., is a Curriculum Assessment Analyst at theSchool of Osteopathic Medicine, A. T. Still University, Mesa, AZ, USA.

Frederic N. Schwartz, D.O., FACOFP, is the Associate Dean, ClinicalEducation and Services, and Professor and Chair, Family and CommunityMedicine at the School of Osteopathic Medicine, A. T. Still University,Mesa, AZ, USA.

References

1. Oblinger DG, Oblinger JL. Educating the net generation. 2005. www.educause.edu/educatingthenetgen. Accessed 2 Jul 2013.

2. Handelsman J, Ebert-May D, Beichner R, Bruns P, Chang A,DeHaan R, Gentile J, Lauffer S, Stewart J, Tilghman SM, WoodWB (2004) Scientific teaching. Science 304:521–522

3. Michael J (2006) Where’s the evidence that active learning works?Adv Physiol Educ 30:159–167

4. Prince M (2004) Does active learning work? A review of the re-search. J Engr Educ 93:223–231

5. Wood W (2004) Clickers: A teaching gimmick that works. Dev Cell7:796–798

6. Akl EA, Gunukula S, Mustafa R,WilsonMC, Symons A,Moheet A,Schunemann HJ (2010) Support for and aspects of use of educationalgames in family medicine and internal medicine residency programsin the US: a survey. BMC Med Educ 10:1–5

7. Beylefeld AA, Struwig MC (2007) A gaming approach to learningmedical microbiology: student’s experiences of flow. Med Teach 29:933–940

8. Valente P, Lora PS, Landell MF, Schiefelbein CS, Girardi FM, SouzaLR, Zonanto A, Scroferneker ML (2009) A game for teachingantimicrobial mechanisms of action. Med Teach 31:e383–e392

9. Zakaryan V, Bliss R, Sarvazyan N (2005) Non-trivial pursuit ofphysiology. Adv Physiol Educ 1:11–14

10. Da Rosa AC,Moreno Fde L,MezzomoKM, ScrofernekerML (2006)Viral hepatitis: an alternative teaching method. Educ Health 19:14–21

11. Graafland M, Schraagen JM, Schijven MP (2012) Systematic reviewof serious games for medical education and surgical skills training. BrJ Surg 99:1322–1330

12. Akl EA, Mustafa R, Slomka T, Alawneh A, Vedavalli A,Schunemann HJ (2008) An educational game for teaching clinicalpractice guidelines to internal medicine residents: development, fea-sibility and acceptability. BMC Med Educ 8:1–9

13. Hudson JN, Bristow DR (2006) Formative assessment can be fun aswell as educational. Adv Physiol Educ 30:33–37

14. Jirasevijinda T, Brown LC (2010) Jeopardy! An innovative approachto teach psychosocial aspects of pediatrics. Patient Educ Couns 80:333–336

15. Moy JR, Rodenbaugh DW, Collins HL, DiCarlo SE (2000) Whowants to be a physician? An educational tool for reviewing pulmo-nary physiology. Adv Physiol Educ 24:30–37

186 Med.Sci.Educ. (2014) 24:181–187

16. O’Leary S, Diepenhorst L, Churley-Strom R, Magrane D (2005)Educational games in an obstetrics and gynecology core curriculum.Am J Obst Gynec 193:1848–1851

17. Schuh L, Burdette DE, Schultz L, Silver B (2008) Learning clinicalneurophysiology: gaming is better than lectures. J Clin Neurophysiol25:167–169

18. Shiroma PR,MassaAA,AlarconRD (2011)Using game format to teachpsychopharmacology to medical students. Med Teach 33:156–160

19. Rajasekaran SK, Senthilkumar U, Gowda V (2008) A PowerPointgame format to teach prescription writing. Med Teach 30:717–718

20. Akl EA, Pretorius RW, Sackett K, Erdley S, Bhoopathi PS, Alfarah Z,Schunemann HJ (2010) The effect of educational games on medicalstudents’ learning outcomes: a systemic review. BEME Guide No.14. Med Teach 32:16–27

21. Glaser BG, Strauss AL (1967) The discovery of grounded theory:strategies for qualitative research. Aldine Transaction, NewBrunswick, USA

22. Rolfe I, McPherson J (1995) Formative assessment: how am I doing?Lancet 345:837–839

23. Pettit RK, Kuo YP (2013) Mapping of medical microbiologycontent in a clinical presentation curriculum. Med Sci Educ23:201–211

24. Schwartz FN, Hover ML, Kinney M, McCoy L (2012) Studentassessment of an innovative approach to medical education. MedSci Educ 22:102–107

25. Schwartz FN, Hover ML, Kinney M, McCoy L (2012) Facultyassessment of an innovative approach to medical education. MedSci Educ 22:108–116

Med.Sci.Educ. (2014) 24:181–187 187