reflecting on quiz placement in online tutorials: creating a more active learning experience
TRANSCRIPT
This article was downloaded by: [University of Louisville]On: 08 October 2013, At: 11:27Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Medical Reference Services QuarterlyPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/wmrs20
Reflecting on Quiz Placement in OnlineTutorials: Creating a More ActiveLearning ExperienceAmy Blevins a & Megan E. Besaw ba University of Iowa , Iowa City, Iowa, USAb East Carolina University , Greenville, North Carolina, USAPublished online: 29 Jul 2011.
To cite this article: Amy Blevins & Megan E. Besaw (2011) Reflecting on Quiz Placement in OnlineTutorials: Creating a More Active Learning Experience, Medical Reference Services Quarterly, 30:3,316-324, DOI: 10.1080/02763869.2011.590427
To link to this article: http://dx.doi.org/10.1080/02763869.2011.590427
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.
This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
INFORMATICS EDUCATION
Linda Hasman and Susan LaValley, Column Editors
Reflecting on Quiz Placement in OnlineTutorials: Creating a More Active
Learning Experience
AMY BLEVINSUniversity of Iowa, Iowa City, Iowa, USA
MEGAN E. BESAWEast Carolina University, Greenville, North Carolina, USA
As the number of online library tutorials increases, so does theneed to create active learning experiences and options for self-assessment. This article looks at embedding short Flash quizzesinto tutorials created with Camtasia as a way to address this need.It also attempts to determine how quiz placement affects desire toview online tutorials and information retention.
KEYWORDS Active learning, online instruction, quizzes, self-assessment, tutorials
INTRODUCTION
Web-based tutorials offer many benefits to both users and libraries. Althoughthe initial creation of tutorials can take time, in the long run they providelibrary users with access to information anytime. They can supplementlibrary instruction by allowing users to review material covered early on inthe semester or as a substitute for face-to-face instruction for distance users.Web-based tutorials can also assist library users who have not received
Comments and suggestions should be sent to the Column Editors: Linda Hasman([email protected]) and Susan LaValley ([email protected]).
Address correspondence to Amy Blevins, Hardin Library for the Health Sciences, Univer-sity of Iowa, 600 Newton Road, Iowa City, IA 52242-1098. E-mail: [email protected]
Medical Reference Services Quarterly, 30(3):316–324, 2011Copyright # Taylor & Francis Group, LLCISSN: 0276-3869 print=1540-9597 onlineDOI: 10.1080/02763869.2011.590427
316
Dow
nloa
ded
by [
Uni
vers
ity o
f L
ouis
ville
] at
11:
27 0
8 O
ctob
er 2
013
library instruction or who prefer to learn on their own. Numerous librarieshave begun to take advantage of the benefits provided by web-based tutor-ials. As of 2007, 66% of academic medical libraries had created their owntutorials, and 79% linked to third-party tutorials.1
Often library instruction tutorials are provided in a passive format inwhich users simply click through a sequence of instructional materials.Another option is to offer a more active web tutorial experience, where userparticipation is required to progress through the instructional material.Currently, it appears that the majority of web-based tutorials are in a passiveformat. Out of 111 Association of Research Libraries reviewed websites, only39 contained active learning components.2 According to a study examiningthe features of academic medical library tutorials, merely 11% of all tutorialsexamined contained some form of interactivity, with 8% containing a quiz orreview question.1 However, when given the option, library users frequentlyprefer the experience provided by an active tutorial. Anderson and Wilsonevaluated preferences between active and passive web-based tutorials onthe topic of RSS feeds and discovered that 78% of participants preferredthe active method.3
In 2006, librarians at the William E. Laupus Health Sciences Library atEast Carolina decided to create video tutorials for the most commonly useddatabases with the software Camtasia. At first, due to time constraints, thevideos were simple videos produced in Flash that library users could watchpassively. One was created for CINAHL via EBSCO and another for MEDLINEvia Ovid. Each tutorial was broken down into sections that covered only afew concepts at a time so that none of the sections of the tutorials were overfive minutes in length. In 2007, a new librarian position was created, entitledthe Education and Instructional Technology Librarian. One of the librarian’sresponsibilities was to develop and improve online instructional materials, aswell as to create more competency-driven online tutorials.
In an effort to provide an opportunity for tutorial users to test theirknowledge, short quizzes were added to the end of each section of both aMEDLINE via Ovid tutorial and a CINAHL via EBSCO tutorial. The ideawas to give the user a way to complete a self-assessment after watchingthe tutorial. The quizzes contained one question that covered the mostimportant concept presented. The quiz was embedded into the Flash filefor the tutorial and provided solely for the benefit of the user. No informationwas collected regarding whether the quizzes were being utilized or if thequizzes were being answered correctly. If the quiz was answered correctly,the user saw a textual response that said ‘‘Correct.’’ If the user answeredthe question incorrectly, the screen would say ‘‘Incorrect.’’
With the arrival of a more recent version of Camtasia, a new option forquiz feedback was discovered. Rather than textual feedback for an incorrectresponse, the quiz could be set up in such a way that an incorrect responsewould trigger the tutorial to take the user back to the point in the tutorial
Informatics Education 317
Dow
nloa
ded
by [
Uni
vers
ity o
f L
ouis
ville
] at
11:
27 0
8 O
ctob
er 2
013
where the answer to the question was given. The quiz would then presentitself again so that the person could pick the right answer. If the useranswered the quiz correctly, a text response showed that said ‘‘correct,’’and the user could then click ‘‘done’’ to continue watching the tutorial.The user also had the ability to fast forward the video past the quiz ratherthan answering.
This new method of taking the user back in the tutorial was thought tobe beneficial since the textual response for an incorrect answer had to berelatively short and did not allow for explaining the correct response. Inaddition, it was theorized that having the user answer a question duringthe tutorial would allow for self-assessment, reinforce the information pre-sented, and provide a more active learning experience. However, using thisnew method with a quiz at the end of the tutorial was problematic as therewas not a way to set the system up to play only a section of the tutorial. Theuser would be taken back to the part of the tutorial where the answer wasexplained and then have to watch the whole tutorial again before getting achance to try the question once more.
The Education and Instructional Technology Librarian felt that this newfunctionality would encourage a more active learning experience and wouldbe more useful than the short textual response in the current tutorials. Inaddition, it was thought that users would retain more information if the ques-tion was asked directly after the information was presented rather than at theend of the tutorial. In order to test this theory, a pilot study was conducted todetermine if there was a preference for having the quiz at the end of thetutorial or in the middle of the tutorial.
METHODS
Two video tutorials on utilizing subject headings in the CINAHL databasewere created using Camtasia. Each tutorial contained the same one-questionquiz (see Figure 1), but the question was located in a different place in thetwo versions of the tutorial. Specifically, the first tutorial contained a quizplaced in the middle of the video, immediately following the informationneeded to correctly respond, and the second tutorial contained the exactsame quiz located at the end of the tutorial. In both tutorials, if the questionwas answered incorrectly, the user was taken back to the point in the tutorialwhere the information was explained. In both versions of the tutorial thequiz showed up again allowing the user to answer once more, and both ver-sions had the option to fast forward past the quiz if desired. Although thepublished library tutorials have always had sound, both tutorials were cre-ated with captions since the classroom setup where the pilot study took placewas not conducive to sound. The library classroom allows for each student to
318 Informatics Education
Dow
nloa
ded
by [
Uni
vers
ity o
f L
ouis
ville
] at
11:
27 0
8 O
ctob
er 2
013
have a personal laptop but not headphones. The captions reflected the samespeed and content of the audio for the tutorial.
In order to collect the participant’s feedback, two surveys were createdvia Survey Monkey. Both surveys were exactly the same, but one was createdfor use with the tutorial with the quiz in the middle and the other for thetutorial with the quiz at the end; this was to ensure that results were not con-fused. The survey consisted of the following question and statements:
1. Have you received database instruction before?2. The question within the tutorial reinforced the information presented.3. The question in the tutorial made me less likely to finish watching the
tutorial.4. I was able to answer the question correctly.5. Additional comments.
Questions 2 and 3 were evaluated on a Likert scale of 1–5, with 1 being‘‘not at all’’ and 5 being ‘‘very much,’’ questions 1 and 4 were ‘‘yes or no’’
FIGURE 1 Screen capture of one-question quiz within CINAHL tutorial. (Figure is provided incolor online.)
Informatics Education 319
Dow
nloa
ded
by [
Uni
vers
ity o
f L
ouis
ville
] at
11:
27 0
8 O
ctob
er 2
013
format, and the final question was open ended. Separate web pages werecreated for each version of the tutorial. The web pages contained one versionof the tutorial and a link to the survey previously mentioned.
For this pilot study, 41 graduate students in East Carolina University’scommunication sciences and disorders program were asked to viewthe instructional tutorials during four library orientation classes. Two of theclasses viewed the CINAHL tutorial with the question in the middle ofthe tutorial, and the two remaining classes viewed the tutorial with the quizquestion at the end of the tutorial. Following completion of the video tutorial,participants were asked to complete a five-question online survey assessingtheir opinion of the quiz in the tutorial.
RESULTS
Of 41 students, only 30 completed the survey due to technical complicationswith the survey and the computers that were utilized. Twelve studentsviewed the tutorial with the quiz in the middle, and 18 students viewedthe tutorial with the quiz at the end.
Of the students in the group with the quiz in the middle, 41.7% hadreceived database instruction before with 53.8% not having received anydatabase instruction. For the participants who watched the tutorial with thequiz at the end, 55.6% had received database instruction before (seeFigure 2).
The majority (41.7%) of students that received the quiz in the middle ofthe tutorial reported that the question had reinforced the material presented
FIGURE 2 Have you received database instruction before?
320 Informatics Education
Dow
nloa
ded
by [
Uni
vers
ity o
f L
ouis
ville
] at
11:
27 0
8 O
ctob
er 2
013
(the average rating for the Likert scale was 4.08). When the group with thequiz at the end of the tutorial was asked if the quiz reinforced the informationpresented, the majority (61.1%) responded ‘‘somewhat,’’ and the averagescore on the Likert scale was 4.28 (see Figure 3).
When the group with the quiz in the middle was asked if the question inthe tutorial made the students less likely to finish watching the tutorial, 33.3%responded not at all, 41.7% were neutral, and 25% said somewhat for anaverage Likert scale rating of 2.58. For the group with the quiz at the end,the majority (52.9%) said ‘‘not at all,’’ 17.6% were neutral, 5.9% said ‘‘some-what,’’ and 5.9% responded ‘‘very much’’ with an average Likert scale ratingof 1.94 (see Figure 4).
All of the students with the quiz in the middle reported that theyanswered the question correctly, and most of the respondents (89%) withthe quiz at the end also stated that they were able to answer the question cor-rectly (see Figure 5).
For the open ended question, one response from the group with thequiz in the middle said, ‘‘It was a little difficult to read, probably easier whensound is involved.’’ For the group with the quiz at the end, seven studentsresponded to the open-ended question, with six of those responses com-menting that they would have preferred sound over captions and one notingthat the tutorial made the person feel more comfortable with using thedatabases.
The majority of respondents within both groups felt that the quiz rein-forced the information presented while also stating that having a quiz withinthe tutorial did not make them less likely to finish watching the tutorial.Based on the results of this small study, the librarians decided to create
FIGURE 3 The question within the tutorial reinforced the information presented.
Informatics Education 321
Dow
nloa
ded
by [
Uni
vers
ity o
f L
ouis
ville
] at
11:
27 0
8 O
ctob
er 2
013
tutorials that had quizzes located near the major concepts utilizing the ability tosend people back to the point in the tutorial where the information was pre-sented if the question was answered incorrectly. The results of the study didsupport the hypothesis that students would find the quizzes helpful and thatthe majority would not be compelled to stop using the tutorials.
DISCUSSION
There are several ways this study could be improved. One would be toenable the students to view the tutorials with sound if desired. The other
FIGURE 4 The question in the tutorial made me less likely to finish watching the tutorial.
FIGURE 5 I was able to answer the question correctly.
322 Informatics Education
Dow
nloa
ded
by [
Uni
vers
ity o
f L
ouis
ville
] at
11:
27 0
8 O
ctob
er 2
013
would be to break the students into three groups at the beginning of a se-mester. Two groups would be quizzed, and the third group would simplybe asked to view the tutorial without a quiz. For the two groups being quizzed,results would be collected to see if the students actually were able to answerthe question correctly immediately following the tutorial. Later on in the sem-ester, all three groups would be given the quiz to see if the information wasretained. Then a comparison between the three groups could be conducted,one with no quiz in the tutorial, one with the quiz in the middle, and one withthe quiz at the end. It would have been interesting to see results of this studywith a larger and more diverse group of respondents. The tutorials are meantfor faculty, staff, or students within the Schools of Medicine and Dentistry andthe Colleges of Nursing and Allied Health Sciences at East Carolina University,but time and resources only allowed for a short case study with students in theCollege of Allied Health Sciences.
CONCLUSION
Despite the limitations of this study, it appears that adding short Flashquizzes into online video tutorials is a simple way to provide active learningactivities and possibly improve the retention of information presented. Themajority of students in both groups felt that having a question withinthe tutorial reinforced the information and that having the questions in thetutorial did not affect their desire to finish watching the videos. As of June2010, Laupus Library no longer has an Education and Instructional Tech-nology Librarian. Currently, one of the co-authors of this piece, the liaisonto the College of Allied Health at Laupus Health Sciences Library, is nowin charge of the Medline via Ovid tutorials and is using the techniquedescribed in this article to provide opportunities for active learning andself-assessment. There are plans to attempt a multi-institutional study inthe future with regards to online instructional techniques and studentpreferences.
REFERENCES
1. Anderson, R.P.; Wilson, S.P.; Yeh, F.; Phillips, B.; and Livingston, M. ‘‘Topicsand Features of Academic Medical Library Tutorials.’’ Medical Reference ServicesQuarterly 27, no. 4 (Winter 2008): 406–418.
2. Hrycaj, P. ‘‘Elements of Active Learning in the Online Tutorials of ARLMembers.’’ Reference Services Review 33, no. 2 (2005): 210–218.
3. Anderson, R., and Wilson, S. ‘‘Quantifying the Effectiveness of InteractiveTutorials in Medical Library Instruction.’’ Medical Reference Services Quarterly 28,no. 2 (April–June 2009): 10–21.
Informatics Education 323
Dow
nloa
ded
by [
Uni
vers
ity o
f L
ouis
ville
] at
11:
27 0
8 O
ctob
er 2
013
ABOUT THE AUTHORS
Amy Blevins, MALS ([email protected]) is the Clinical EducationLibrarian, Hardin Library for the Health Sciences, University of Iowa, 600Newton Road, Iowa City, IA 52242-1098. She was previously the Educationand Instructional Technologies Librarian, William E. Laupus Health SciencesLibrary, East Carolina University, Greenville, NC 27834. Megan E. Besaw,MLIS ([email protected]) is the Liaison to the College of Allied HealthSciences and Liaison to the College of Health and Human Performance,William E. Laupus Health Sciences Library, East Carolina University,Greenville, NC 27834.
324 Informatics Education
Dow
nloa
ded
by [
Uni
vers
ity o
f L
ouis
ville
] at
11:
27 0
8 O
ctob
er 2
013