The Eighth International Conference on eLearning for Knowledge-Based Society, 23-24 February 2012, Thailand

1

Evaluating Usability of Pre-recorded VDOsin eLearning

Dr. Kuldeep NagiCollege of Internet Distance Education

Assumption UniversityBangkok, Thailand

DrKuldeep@Live.Com

Abstract - Since the inception of T.V.in1940 there has been a phenomenalincrease in the creation as well as in theuse of pre-recorded media forentertainment, education and training.The last two decades have been marked byanother revolution. Use of digital camerasand VDO recorders has led to easyproduction of digital content. In addition,advances in computer hardware andsoftware have further increased theproduction of high quality digital mediafor its use in education, training andentertainment. The expansion of Internetin the last ten years has also led toexplosion in pre-recorded digital media foron-line learning and training. This paperexplores the extent of usefulness of themost important digital components ineLearning- the pre-recorded VDOs.Usability of this key component isevaluated through data obtained via thesystem reports and logs provided in aLearning Management System (LMS).The results indicate that eLecture VDOsare not adequately used by students. Studyonline VDOS are provided with anintention to augment the learning processbut there is no indication that they areused as preferred resource for on-linelearning.Keywords - Digital Media, eLearning,LMS, Podcasts, Usability, VDOss

I. INTRODUCTIONMultimedia is defined as “a method for

VDO and narration (audio content)” andemphasizes that multimedia material can

reduce extraneous cognitive load on thelearner while studying.

Today the Internet VDO is something weuse to call "the television" and the people inthe entertainment sector are obsessed withhow the internet has changed the way weconsume VDO- downloads instead of DVDs,on-demand rather than on a schedule. VDOshave a special place in eLearning andeTraining programs. They are being usedextensively by academicians andpractitioners to teach concepts and skills to ageneration of visually oriented learners [2, 3and 4]. There have been lots of studiesindicating harmful effects of television.Jokingly the T.V. is also called as an “idiotBox’. Actually T.V. is not the idiot box but,the people watching it are sometimes calledidiots. Research indicates that peoplewatching become idiots because the TVmakes their intelligence and creativity subnormal. It is passivity and long exposure tojunk content the media brings. There isplenty of evidence about harmful effects oftelevision [1]. But so far research effortscomparing on-line pre-recorded VDO andprint media as teaching tools have notfocused on their effectiveness.

People also argue that in many waysthe internet has changed the way we watchwhat used to be the T.V. These days on anyinternet VDO site, the vendors provide statusbar (Fig. 3) at the bottom of the VDO thattells you how long you have been watching,how much is in the buffer, and how muchlonger until the end of the VDO. Sometimeswith the commercials there's a numbercounting down the seconds left in theadvertisement. Among so many other distinct

Dr. Kuldeep Nagi

Special Issue of the International Journal of the Computer, the Internet and Management, Vol. 19 No. SP2, February, 2012

2

Web 2.0 technologies the You-Tube haschanged the world of on-line VDOs and theirrole in knowledge acquisition, education andtraining.

Learning is mostly an active processconducted in a self-directed fashion.eLearning provides many opportunities forconstructivist learning by supportingresource-rich, student-centered, andinteractive learning. Most on-line coursespresent synchronized multimedia materialson the Internet in an interactive and cohesivemanner. Among other things the studymaterials include (Fig.1- Fig.4) lecture notes,graphics, diagrams, podcasts and VDOs.eLecture VDOs are recorded in posh studiosspending lots of time and money. Since eachVDO is likely to contain many subtopics,VDOs are logically segmented into smallerindividual clips based on the content so thateach clip focuses on a specific subtopic. Thenthese clips are compressed and stored on aVDO streaming server. An integratedknowledge repository is created on a Webserver, containing metadata of VDOs andother multimedia instructions.

Clark (1983) laid down the gauntlet foreducational media researchers by stating thatafter reviewing research comparing theeffectiveness of various media he learned that"... media do not influence learning under anyconditions, media are mere vehicles thatdeliver instructions that do not influencestudent achievement anymore than the truckthat delivers our groceries causes changes inour nutrition". In short, in Clark's view, it isthe selection of information and how it isorganized that will determine studentachievement, not whether this information ispresented in books, VDOs, or computers [5].

Those who support the other side of thedebate on the effects of media on learningemphasize the complexity of experimentallyseparating the effects of instructionalmethods and media [6, 7 and 8]. Reiser(1994) summarizes the controversyeloquently by stating that while "Clark iscorrect in stating that methods are whatcauses learning to occur, he fails toacknowledge that certain media attributes

make certain methods possible". Referring toClark's delivery truck analogy, Reiser statesthat "the successful delivery of frozen foodsrequires the use of a vehicle withrefrigeration"[7].

A typical eLearning program such asMaster of Science in Information Technology(MSIT) used here as a case study wouldconsists of several components. Some keycomponents are listed below:

1. Text files such as articles for reading,Assignments, Reports, Term Papers,Power-Points and PDF files.

Fig.1 Text Material

2. Still images such as pictures, figures anddiagrams with or without labels andaccompanying text.

Fig.2 Images with labels

3. VDO is shown in Fig. 3 is very commoncomponent of a eLearning course and itsoperations are described by animation orVDO material synchronized withnarration. In most cases textcorresponding to the narration isdisplayed synchronously with the VDOcontent in the manner of subtitles. Thesedays on-line VDOs can also showanimation and special effects. 3-D VDOsare also becoming very common.

Evaluating Usability of Pre-recorded VDOs in eLearning

The Eighth International Conference on eLearning for Knowledge-Based Society, 23-24 February 2012, Thailand

3

Fig.3 Typical VDO Lecture

4. Audio as shown below in Fig.4 isdelivered as information in the form of anarration with no visual content. Usuallyaudio lectures are available as podcasts inMP/3 and MP/4 format.

Fig.4 Audio Lecture

II. OBJECTIVES ANDMETHODOLOGY

The main objective of this paper is toanalyze the usability of pre-recorded VDOmaterials and their role in learning in on-linecourses. In the context of this paper, thisauthor prefers to use the definition ofusability as specified by ISO 9241-11 [2]:“The extent to which a product can be usedby specified users to achieve specified goalswith effectiveness, efficiency and satisfactionin a specified context of use”.

VDO and audio components are integralpart of an on-line course. Therefore, thispaper would survey, analyze and reporteffectiveness of only one component- StudyOnline VDOs with regard to:

1. Learner Experiences: Determiningusability level of VDO eLectures usingsystem logs provided through the reportof an LMS.

2. Total Duration: Examining total timespent on watching on-line VDOs in avirtual learning environment (VLE).

As mentioned in (1) the LMS reportswere used as main instrument to surveyanalyze and report usage of VDO eLecturesin a VLE. For the purposes of this study 8eLearning courses were chosen. Thesecourses are part of the MS-TCT program andwere offered at CIDE in 2011.

For the purposes of this work system“Reports” a tool provided in the LMS wereused as instruments. Reports contain activitylogs and other pertinent data sets that can beused for examining events in a virtualsession. Table-1 given below contains veryimportant piece of information about over allusage of the chapters or units in a giveneLearning courseware. Logs contain detailinformation about a student session andrecord every event or activity including usageof Study Online VDOs. To keep the focus onthe main topic only data about “StudyOnline” or VDOs was selected. Details aboutthe 8 courses are given in Table-1. Three setsof data about a course are listed in the Table-1: Units Total, Units Viewed Minimum andPercentage (%) of Units Viewed. The tableshows the total number of chapters or units inevery course. They range from a minimum of10 to maximum of 14. Column 2 lists thenumber of chapters or units actually viewedby students. The last column showscalculations about % of units used on-lineVDOs for learning. A closer look into thetable indicates that Course 3 shows themaximum usage of on-line VDOs. Thesecond on the list is Course 2 with 81.80% ofusage. In all other courses the usage was 50%or less, lowest being 28.57% % for Course 8.

Dr. Kuldeep Nagi

Special Issue of the International Journal of the Computer, the Internet and Management, Vol. 19 No. SP2, February, 2012

4

TABLE 1COURSE UNIT INFORMATION

Courses Students Units Units %

Total Viewed Unit

Mini Viewed

Course1 2 14 6 42.80%

Course2 6 11 9 81.80%

Course3 5 13 13 100%

Course4 4 10 5 50%

Course5 4 12 11 91%

Course6 3 14 6 42.80%

Course7 2 13 5 38.46%

Course8 3 14 4 28.57%

III. RESULTS & DISCUSSIONTo continue our discussion on

usability of VDOs let us look at the Tale-1again. Table-1 Column 1 shows thenumber of chapters or units in everycourse. Presuming that every student willwatch the VDO of a unit at least one time(at least once), the actual usage is shownin Column 3. It is very much possible thatin a given circumstance a student maywatch the same VDO more than once. Ifthat happens, the logs will show all thedetails such as instances and duration ofusage of every course componentincluding the VDOs. VDO and audio filesare supposed to constitute the corecomponents of a course. To further clarifythe issue of usability of VDOs let usbriefly examine the significance of Table-2. Table-2 shows that due to repeat usageof a VDO for a particular unit the % usageas shown in the last column may rangefrom a minimum (25%) to a maximum(170%). For full determination the dataabout total instances of a VDO session foreach unit by each student will have to becalculated. Such calculations wereavoided because of insignificance ofrepeat instances of a VDO session. Thisstudy focuses on the minimum instances.

Data given in Table-1 clearly indicatesthat 75% of courses which is 6 out of 10

showed usage of VDO in the range of50% and bel0w. Naturally, this should be

TABLE 2ACTUAL USAGE WITH REPEAT INSTANCES

Courses Expected Total %

Mini Views Viewed Usages

Max

Course1 28 19 67

Course2 66 73 110.6

Course3 65 98 150

Course4 40 38 95

Course5 48 82 170

Course6 42 17 40.5

Course7 26 28 107

Course8 42 10 25

% Usage includes repeat sesion

a cause for concern because lots of money andtime is spent in recording VDO eLectures. Theinformation given in Table-1 is graphed below inFig. 5. A comparison of the total chapters or unitsversus actual number of units used by students isshown in brown color.

A clearer picture about percentage ofchapters used for study using on-line VDO isshown in Fig.6. Graphical representation ofthe data provides a very clear understandingof percentage of on-line VDOs being used inan eLearning courseware.

Fig. 5 Total number units vs. Units viewed

Evaluating Usability of Pre-recorded VDOs in eLearning

The Eighth International Conference on eLearning for Knowledge-Based Society, 23-24 February 2012, Thailand

5

It is remarkable that even though onlycomponent (Study online VDO) is selectedfor this study the results are convincinglyclear that in 6 out of 8 courses the VDOswere not used as expected. From Fig. 7 it isquite clear that the total time spent during thesemester watching VDOs was also very low.The maximum time spent on accessingStudy-Online VDOs was 105 minutes duringthe whole semester. The minimum time was10 minutes which is almost negligible whenyou calculate the number of hours availablefor the whole semester.

One key attribute of quality is “quantity”.In other words, how many students wereactually using a component for study is oneof the most critical elements of quality? Suchquantitative analysis done by the industries inthe private sector relies on huge amount ofdata collected over time.

Fig. 6 Percentage (%) of Units Viewed

Although this study may be limited inmany ways the result of this work indicatesthat the new learning model should featuresshort and tightly focused VDO sessionsdelivered to individuals or groups that needthem with the right type of content toaugment the understanding of importantconcepts.

Fig.7 Total time spend in a semester on VDOs

Overall success of a eLearning programis measured not by the number of coursesdelivered, but by the achievement of keyperformance benchmarks and their bottomline usage. It is nothing less than a paradigmshift from rote learning to performance, madepossible largely by advances in Web 2.0technologies and their implementation forteaching and learning. Study online VDOalone cannot help in meeting thesebenchmarks [9].

IV. CONCLUSIONSThe greatest challenge for on-line

educators who are trying to use VDO andaudio to their best advantage seems to beharnessing the motivating impact of VDOwithout falling prey to its failings, shallowcomprehension, trivialization, passivity andlowered mental effort. More specifically,addition of books, articles, activities andprojects can bring learners from basic tomoderate levels of comprehension, whereVDOs can cognitively engage them. Becauseof VDO's capabilities to generate interest in asubject matter, motivation to explore ingreater depth can be expected. One approachto meshing VDO and other components foroptimal educational outcomes is thedevelopment of a set of learning activities,called the VDO learning sequence (VLS).Consistent with research findings andsuggestions put forth below, the VLS offers apromising approach to raising studentinterest, effort, duration of usage and contentcomprehension.

Dr. Kuldeep Nagi

Special Issue of the International Journal of the Computer, the Internet and Management, Vol. 19 No. SP2, February, 2012

6

So in a nutshell, the issue is how can wemake on-line learning engaging for student?Many instructors struggle to engage studentsin a face-to-face (F2F) environment. Can theVDO and other resources provided in acourseware engage students in a VLE? Theresults indicate that in the given situationVDOs were not fully utilized. The reasonsfor low usage of VDOs range for their lowquality, long duration, lack of appeal andother defects such language errors andlacking relevance to the overall learningprocess [10].

Based on these preliminary findings thisauthor proposes the following twosuggestions:

Suggestion 1: VDO should be used optimallyfor highlighting key concepts and increasingresponsiveness to the main content.Suggestion 2: The transient quality of theVDO requires that the contentcomprehension should be followed by Q &A. Comprehension from VDO can beincreased when learners are instructed toview a VDO clip with an objective in mind.A follow up set of questions in the form ofaudio inputs or review questions is alwayshelpful in meeting the objectives of theVDOs [11].

There is an ongoing surge in eLearning,but only if we put the right components withadequate quality in place, which issomething, most educational institutions donot do a very good job with. Success of You-Tube indicates that quality of VDOs can bevery instrumental in enhancing knowledgeacquisition, learning and training.

REFERENCES

[1] Villani. S. (2001),”Impact of Media on Childrenand Adolescents: A 10-Year Review of theResearch”, Journal of the American Academy ofChild and Adolescent Psychiatry 1 April 2001(volume 40 issue 4 Pages 392-401

[2] Lacho, K.J., Herring, R.A. and Hartman, S.J.(1991) "The VDO age: an analysis of classroomuse of VDO technology by managementprofessors", paper submitted to the SouthernManagement Association, Management EducationTrack

[3] Lee, C. (1987) "Where the training dollars go",Training, Vol. 24 No. 10, pp. 51-65.

[4] Gioia, D. and Brass, D. (1985-1986), "Teachingthe TV generation: the case for observationallearning", Organizational Behavior TeachingReview, Vol. 10 No. 2, pp. 11-18.

[5] Clark, R.E. (1983), "Reconsidering research onlearning from media", Review of EducationalResearch, Vol. 53 No. 4, pp. 445-59.

[6] Kozma, R.B. (1994, "Will media influencelearning: reframing the debate", EducationalTechnology Research and Development, Vol. 42No. 2, pp. -19.

[7] Reiser, R.A. (1994, "Clark's invitation to thedance: an instructional designer's response",Educational Technology Research andDevelopment, Vol. 42 No. 2, pp. 45-48.

[8] Ross, S.M. (1994, "From ingredients to recipes ...and back: it's the taste that counts", EducationalTechnology Research and Development, Vol. 42No. 3, pp. 5-6

[9] Phillips, Jack, Ph.D. “The Business Value of e-Learning,” Dec. 2007-Jan. 2008 “ELearning!”(with Holly Burkett). http://media.roiinstitute.net/articles/pdf/2008/01/18/el1207_ROI.pdf

[10]Solomon, G. and Leigh, T. (1984,"Predispositions about learning from print andtelevision", Journal of Communication, Vol. 34,pp. 119-35.

[11]Nagi, K . (2011). “Quality Assurance (QA) in On-Line Learning Courses: Prospects andPossibilities-A Case Study”, NIDA Journal ofCase Studies, Bangkok, Thailand