mis - computer based training

7/31/2019 MIS - Computer Based Training

1/41

[ A project report by Supratik Bhattacharya ]

Management Information Systems08Fall

[C.B.T. COMPUTER BASED TRAINING]

U D A I P A R E E K H R L A B S

E M P I B U S I N E S S S C H O O L

N E W D E L H I - 1 1 0 0 7 4


2/41

Q u ic k T im e ad e c o m p r e s s

a re n e e d e d to s e

A

Report

On

Computer Based Training

Submitted to: Submitted by:

Mr. B. G. Gupta Supratik Bhattacharya

UDAI PAREEK HR LABSEMPI BUSINESS SCHOOL

NEW DELHI(2008 2010)

2


3/41

Contents

Topic Page No.

Overview 4

Introduction 6

Propositions for the design of CBL systems 7

In -house or outsource CBT development 8

Pedagogical approaches or perspectives 9

Description 10

Modify the business process 13

Standard output 15

Computer-Based Training and Assessments:

An Exploratory Study of Social Factors (Abstract) 16

Introduction 18

The literature review 20

The research model 24

Research methodology 26

Methodology 29

Results and implications 30

Conclusions 37

References 38

3


4/41

Overview

Computer based training (CBT) or Electronic learning is a type ofTechnology supported education/learning (TSL) where the medium ofinstruction is computer technology. In some instances, no in-personinteraction takes place. E learningis used interchangeably in a wide variety of

contexts. Computer-based training (CBT) services are where a student learnsby executing special training programs on a computerrelating to their

occupation. CBT is especially effective for training people to usecomputer applications because the CBT program can be integrated with theapplications so that students can practice using the application as they learn.CBT nevertheless allows teachers to exercise their skills, although the skillsneeded are in some respects different from pre-CBT teaching. CBT is used todefine a specific mode to attend a course or program of study where thestudents rarely, if ever, attend face-to-face for on-campus access toeducational facilities, because they study online. In companies, it refers to thestrategies that use the company network to deliver training courses toemployees. Computer Based Training or Learning refers to the use ofcomputers as a key component of the educational environment. While this

can refer to the use of computers in a classroom, the term more broadly refersto a structured environment in which computers are used for teachingpurposes.

Computer Based Training is highly effective. People get theinformation they need, when they need it, no matter where they are located,and they can study at their own pace on their own computer.

But creating a Computer Based Training program presents a problem.The material to be taught has to be converted into a computer program. Andin many cases the person who knows the material to be taught is not familiar

with computer programming. And that's where the Easy CBT Authoring Toolscome in.

4

QuickTime and a

decompressorare needed to see this picture.
http://en.wikipedia.org/wiki/Computerhttp://en.wikipedia.org/wiki/Computer_applicationshttp://en.wikipedia.org/wiki/Computershttp://en.wikipedia.org/wiki/Classroomhttp://en.wikipedia.org/wiki/Computer_applicationshttp://en.wikipedia.org/wiki/Computershttp://en.wikipedia.org/wiki/Classroomhttp://en.wikipedia.org/wiki/Computer


5/41

These Authoring Tools - Easy Tutor, Easy Test, Easy Quiz, Easy Bookand Easy Study - were written specifically to allow non-programmers to createComputer Based Training programs. They are user-friendly and can belearned in a very short time.

E-learning services have evolved since computers were first used in

education. There is a trend to move toward blended learning services, wherecomputer-based activities are integrated with practical or classroom-basedsituations. It is defined as a planned teaching/learning experience that uses awide spectrum of technologies, mainly Internet or computer-based, to reachlearners. Lately in most Universities, E-learning is used to define a specificmode to attend a course or programs of study where the students rarely, ifever, attend face-to-face for on-campus access to educational facilities,because they study online.

5


6/41

Introduction

As early as 1993, Graziadei, W. D. described an online computer-delivered lecture, tutorial and assessment project using electronic Mail, twoVAX Notes conferences and Gopher/Lynx together with several softwareprograms that allowed students and instructor to create a Virtual InstructionalClassroom Environment in Science (VICES) in Research, Education, Service& Teaching (REST). In 1997 Graziadei, W.D., et al., published an articleentitled "Building Asynchronous and Synchronous Teaching-LearningEnvironments: Exploring a Course/Classroom Management System Solution".They described a process at the State University of New York (SUNY) ofevaluating products and developing an overall strategy for technology-basedcourse development and management in teaching-learning. The product(s)had to be easy to use and maintain, portable, replicable, scalable, andimmediately affordable, and they had to have a high probability of success

with long-term cost-effectiveness. Today many technologies can be, and are,used in e - Learning, from blogs to collaborative software, e - Portfolios, andvirtual classrooms. Most e - Learning situations use combinations of thesetechniques.

By 2006, nearly 3.5 million students were participating in on-linelearning at institutions of higher education in the United States. Many highereducation, for-profit institutions, now offer on-line classes. By contrast, onlyabout half of private, non-profit schools offer them. The Sloan report, basedon a poll of academic leaders, says that students generally appear to be at

least as satisfied with their on-line classes as they are with traditional ones.Private institutions may become more involved with on-line presentations asthe cost of instituting such a system decreases. Properly trained staff mustalso be hired to work with students on-line. These staff members need tounderstand the content area, and also be highly trained in the use of thecomputer and Internet. Online education is rapidly increasing, and onlinedoctoral programs have even developed at leading research universities.

6
http://en.wikipedia.org/wiki/Blogshttp://en.wikipedia.org/wiki/Collaborative_softwarehttp://en.wikipedia.org/wiki/EPortfoliohttp://en.wikipedia.org/wiki/Virtual_classroomshttp://en.wikipedia.org/wiki/United_Stateshttp://en.wikipedia.org/wiki/United_Stateshttp://en.wikipedia.org/wiki/Blogshttp://en.wikipedia.org/wiki/Collaborative_softwarehttp://en.wikipedia.org/wiki/EPortfoliohttp://en.wikipedia.org/wiki/Virtual_classroomshttp://en.wikipedia.org/wiki/United_States


7/41

Propositions for the design of CBL systems

In this section the training propositions derived from the aforementionedresearch efforts will be described. The overall objective of this collectiveresearch was to develop and test cognitive- based principles for designinglearner support tools for distributed training environments that would enabletrainees to develop the knowledge necessary for operating complex systems.Within these efforts there were two main initiatives that are directly relevant tothe instructional design characteristics of distributed CBL systems.

The first initiative involve a basic level cognitive initiative investigatingways in which knowledge can be integrated to facilitate knowledge structureorganization and mental model development.

The second initiative involves an investigation in individual differences

and meta-cognitive process within multimedia CBL programs. Furthermore,these investigations will provide us methodologies in the ways that multimediaCBL instruction can influence knowledge acquisition, and also how thistechnology can best be used for assessment of training effectiveness. Thefollowing propositions for optimizing distributed learning effectiveness incomplex CBL environments will be evaluated.

Proposition 1:comparative study between the various CBL platforms

Proposition 2: detect the factors that influence the success of CBL.

Proposition 3:findings of surveyed students results will be matched withindustry findingsProposition 4: based on these findings the hypothesis will be proposed

Based on the hypotheses, develop a design model for development ofinteractive multimedia to effectively support E-learning in Bahrain. Theresearch will also attempt to design and construct prototype CBL environmentto verify the suitability of the proposed design model in e-learningdevelopment in Bahrain.

The anticipated outcomes of the investigation of the above propositions are asfollows:

1. Improved understanding of the current issues facing e-learning in Bahrain,and potential for the use of in-house CBL in Bahrain.

2. Design an effective model based on field study bearing in mind theconventional CBL development methods used in Bahrain.

3. New design model for the development of effective interactive multimedialearning environment.

7


8/41

IN-HOUSE OR OUTSOURCECBT DEVELOPMENT

Influencing Factors:

Some of the factors that influencing the outsource of CBT development -

Staff

Money

Time

Commitment

STAFF

Existing or new hires

CBT software experienced

Subject matter experienced

Targeted audience experiencedMONEY

What does it cost

Do you have the funds

TIME

Does the developing staff have time

Do the developers have technical expertise

Do the Technical People have time

COMMITMENT

Is the group committed

Who is taking Ownership

8


9/41

Pedagogical approaches or perspectives

It is possible to use various pedagogical approaches for computer basedtraining which include:

Instructional design - the traditional pedagogy of instruction which iscurriculum focused, and is developed by a centralized educating group or asingle teacher.

Social-constructivist - this pedagogy is particularly well afforded by the useof discussion forums, blogs, wikipedia and on-line collaborative activities. It isa collaborative approach that opens educational content creation to a wider

group including the students themselves.

Laurillard's Conversational Model - is also particularly relevant toeLearning, and Gilly Salmon's Five-Stage Model is a pedagogical approach tothe use of discussion boards.

Cognitive perspective - focuses on the cognitive processes involved inlearning as well as how the brain works.

Emotional perspective - focuses on the emotional aspects of learning, likemotivation, engagement, fun, etc.

Behavioral perspective - focuses on the skills and behavioral outcomes ofthe learning process. Role-playing and application to on-the-job settings.

Contextual perspective - focuses on the environmental and social aspects,which can stimulate learning. Interaction with other people, collaborativediscovery and the importance of peer support as well as pressure.

9
http://en.wikipedia.org/wiki/Pedagogicalhttp://en.wikipedia.org/wiki/Instructional_designhttp://en.wikipedia.org/wiki/Constructivism_(learning_theory)http://en.wikipedia.org/wiki/Constructivism_(learning_theory)http://en.wikipedia.org/wiki/Gilly_Salmonhttp://en.wikipedia.org/wiki/Gilly_Salmonhttp://en.wikipedia.org/wiki/Pedagogicalhttp://en.wikipedia.org/wiki/Instructional_designhttp://en.wikipedia.org/wiki/Constructivism_(learning_theory)http://en.wikipedia.org/wiki/Gilly_Salmon


10/41

Description

Training is a powerful tool that successful organizations use to respond toevolving missions, changing audiences, and the increasing need for a diverseworkforce. Training is essential for organizations wanting to take advantage ofshifting market circumstances or to extend their impact by creatingcommunity-based coalitions for action. Macro has decades of global trainingexperience. Our highly trained trainers and facilitators have trained thousandsof public and private employees to be more efficient and effective and to beleaders and team players. We have conducted trainings in all 50 States andU.S. territories and in 70 countries around the world.

Macro emphasizes training that builds an organizations capacity to achieveits mission. As an example, Macro conducts demographic and health surveysfor the U.S. Agency for International Development in support of better publichealth in developing nations. As part of this effort, we conduct on-the-jobtraining for government and nongovernmental organizations to incorporatebetter research technologies and methodologies and to use their data toimprove public information and education. When implementing surveysinternationally, we often supply and install computers and other infrastructureand train users on software used to collect and analyze data and to control itsquality. We dont just dispense training informationwe make sure that theorganization can understand it, can apply it, and can benefit fully from its use.

We have developed hundreds of curricula specific to clients and projects. Ourtraining curriculum can focus on train-the-trainer sessions or skillsdevelopment of employees organization-wide. Macro uses a results-orientedtraining that can begin at any stage: from a training needs assessmentthrough post-training evaluation and reinforcement. We offer flexible trainingoptions, such as partial day or multiday onsite training or training at one of ourfacilities.

We offer customized computer-based trainings, including interactive CD-ROMs, online participation indistance-based learning, and

video conferencing. Ourmultimedia specialists cantape training sessions asvideos for sale or distribution.We also can tape sessionsfor live or streamed Web siteuse, both of which are highlycost-effective methods ofexpanding the number ofpeople trained.

10
http://www.macrointernational.com/ProgramAreas/Health/research.aspxhttp://www.macrointernational.com/Management/Training/tna.aspxhttp://www.macrointernational.com/Management/Training/ost.aspxhttp://www.macrointernational.com/Management/Training/cb.aspxhttp://www.macrointernational.com/ProgramAreas/Health/research.aspxhttp://www.macrointernational.com/Management/Training/tna.aspxhttp://www.macrointernational.com/Management/Training/ost.aspxhttp://www.macrointernational.com/Management/Training/cb.aspx


11/41

Macro also provides logistical support for training meetings. We trackregistration and billing online. We arrange for shipping of training materials,which we often print in-house. We develop and design agendas, meeting

signage, badges, and other support materials.

Macro can provide training wherever and in whatever form best meets yourneeds.

Computer- based trainingis a highly effectivemethod of maximizingworkforce exposure totraining while containingtraining costs. When Trainingis computer- based, it canbe made accessible toanyone with computeraccess, can accommodatediverse schedules,and does not require anonsite trainer, travel time, orroom reservations.Macro has combined itstraining and technological expertise to develop computer-based training that

fits the specific needs of many clients. We design self-paced training modules,interactive trainings, distance learning, and trainings linked to person-to-person technical support. We integrate training evaluations into each Web-based training module, providing instant feedback on training effectiveness.We can develop training that qualifies for continuing education units.Computer-based training is just one of many ways Macro can help youremployees and your organization move forward.e

Total Training Solutions develops cutting edge computer based training (CBT)products to enhance, and supplement your training initiatives. Our computer-delivered training includes CD-ROM and the World Wide Web.

Computer based training courses allow your employees to learn newconcepts, applications, services, skills, and products as needed at their pace.CBT puts training at the fingertips of your employees both at the workplaceand at home. It provides interactive learning from remote work locations andconvenience of employees. CBT lends itself to the concepts of self-pacedlearning learning that occurs at a time the learner determines and at a paceestablished by the learner.

We custom design CD-ROM delivered training that can be used as a standalone training course or one that can be a supplement to an Instructor-Led

course. In many cases, Computer Based Training can be used to teach the"basics" of a given course and then supplemented with hands-on Instructor-Led classroom training that addresses specific learner needs.

11

QuickTime and adecompressor

are needed to see this picture.


12/41

This staged method of training provides your employees with a jump start onlearning the basic concepts and procedures of a given course prior to the

hands-on instruction to develop skill competency.

CBT can help to reduce your training expenses by reducing your classroomtraining time, eliminating some hotel and travel expenses, and reducing thetime employees are away from their job. By reducing your classroom trainingtime, you can also shorten your overall implementation training time frame.

Our Computer Based Training courses help your company build a confidentwork team that has gained the knowledge power to succeed in your business.

12


13/41

Modify the business process

The term e-Learning 2.0 is used to refer to new ways of thinking about e-

learning inspired by the emergence of Web 2.0. From an e-Learning 2.0

perspective, conventional e-learning systems were based on instructional

packets that were delivered to students using Internet technologies. The role

of the student consisted in learning from the readings and preparingassignments. Assignments were evaluated by the teacher. In contrast, the

new e-learning places increased emphasis on social learning and use of

social software such as blogs, wikis, pod casts and virtual worlds such as

Second Life. This phenomenon has also been referred to as Long Tail

Learning. The first 10 years of e-learning (e-learning 1.0) was focused on

using the internet to replicate the instructor-led experience. Content was

designed to lead a learner through the content, providing a wide and ever-

increasing set of interactions, experiences, assessments, and simulations. E-

learning 2.0, by contrast (patterned after Web 2.0) is built around

collaboration. E-learning 2.0 assumes that knowledge (as meaning and

13

QuickTime and adecompressor

are needed to see this
http://en.wikipedia.org/wiki/Web_2.0http://en.wikipedia.org/wiki/Social_softwarehttp://en.wikipedia.org/wiki/Second_Lifehttp://en.wikipedia.org/wiki/Web_2.0http://en.wikipedia.org/wiki/Social_softwarehttp://en.wikipedia.org/wiki/Second_Life


14/41

understanding) is socially constructed. Learning takes place through

conversations about content and grounded interaction about problems and

actions. Advocates of social learning claim that one of the best ways to learn

something is to teach it to others.

The entwining portal offered by European school net is one of Europe's

largest eLearning projects, comprising 50,000 registered teachers from across

Europe. It is funded by the European Commission's Directorate General for

Education and Culture, and has a network of 22 National Support Services,

mostly operated by the national Ministries for Education in the EU.

As another example, Second Life has recently become one of the virtual

classroom environments used in colleges and universities, including

University of Edinburgh (UK) Princeton University (USA), Rice University

(USA), University of Derby (UK), Vassar College (USA), the University of

Plymouth (UK) and the Open University (UK),[20]. In 2007 Second Life started

to be used for foreign language tuition [21]. Both Second Life and real life

language educators have begun to use the virtual world for language tuition.

English (as a foreign language) has gained a presence through several

schools, including British Council projects which have focused on the Teen

Grid. Germany's cultural institute Goethe-Institut started an island in 2008[22],

Spains language and cultural institute Instituto Cervantes has an island on

Second Life. A list of educational projects (including some language schools)

in Second Life can be found on the SimTeach site. SLanguages 2008 was the

2nd annual conference of language education using virtual worlds such as

Second Life.

The event took place in Second Life at the EduNation islands. Additionally,

Mobile Assisted Language Learning (MALL) is a term used to describe using

handheld computers or cell phones to assist in language learning.

There is also an increased use of virtual classrooms (online presentations

delivered live) as an online learning platform and classroom for a diverse set

of education providers such as Fox School of Business for Templer

University, Grades Grow, Minnesota State Colleges and Universities,

BenAstrum Center of Regulatory eTraining, and Sachem[23][24][25][26].

Webex is a Cisco Web Meetings and Collaboration Solution. The platform has

worked for educational institutions because of real time collaboration using an

interactive whiteboard, chat, and VOIP technology that allows audio and video

sharing. In distance learning situations, while replacing the classroom with

features, institutions have also looked for security features which are

inherently strong in a Cisco powered collaboration environment. The

downside is that Webex is not a free platform like WiZiQ or Module, and fees

are paid per 'host' of a classroom or a meeting.

14
http://en.wikipedia.org/wiki/Social_constructionismhttp://en.wikipedia.org/wiki/Social_constructionismhttp://en.wikipedia.org/wiki/Grounded_theoryhttp://www.etwinning.net/http://en.wikipedia.org/wiki/European_schoolnethttp://en.wikipedia.org/wiki/Second_Lifehttp://en.wikipedia.org/wiki/University_of_Edinburghhttp://en.wikipedia.org/wiki/Princeton_Universityhttp://en.wikipedia.org/wiki/Princeton_Universityhttp://en.wikipedia.org/wiki/Rice_Universityhttp://en.wikipedia.org/wiki/University_of_Derbyhttp://en.wikipedia.org/wiki/Vassar_Collegehttp://en.wikipedia.org/wiki/University_of_Plymouthhttp://en.wikipedia.org/wiki/University_of_Plymouthhttp://en.wikipedia.org/wiki/Open_Universityhttp://en.wikipedia.org/wiki/Computer_based_training#cite_note-19http://en.wikipedia.org/wiki/Computer_based_training#cite_note-20http://en.wikipedia.org/wiki/Teaching_English_as_a_foreign_languagehttp://en.wikipedia.org/wiki/British_Councilhttp://en.wikipedia.org/wiki/Teen_Second_Lifehttp://en.wikipedia.org/wiki/Teen_Second_Lifehttp://en.wikipedia.org/wiki/Goethe-Instituthttp://en.wikipedia.org/wiki/Computer_based_training#cite_note-21http://www.cervantes.es/http://secondlife.cervantes.es/http://www.simteach.com/wiki/index.php?title=Institutions_and_Organizations_in_SLhttp://www.slanguages.net/http://www.slanguages.net/venue.phphttp://en.wikipedia.org/wiki/Mobile_Assisted_Language_Learninghttp://www.sbm.temple.edu/http://www.sbm.temple.edu/http://www.gradesgrowtutors.com/http://www.mnscu.edu/http://www.benastrum.com/http://www.sachem.edu/http://en.wikipedia.org/wiki/Computer_based_training#cite_note-22http://en.wikipedia.org/wiki/Computer_based_training#cite_note-22http://en.wikipedia.org/wiki/Computer_based_training#cite_note-23http://en.wikipedia.org/wiki/Computer_based_training#cite_note-23http://en.wikipedia.org/wiki/Computer_based_training#cite_note-24http://en.wikipedia.org/wiki/Computer_based_training#cite_note-24http://en.wikipedia.org/wiki/Computer_based_training#cite_note-25http://en.wikipedia.org/wiki/Computer_based_training#cite_note-25http://en.wikipedia.org/wiki/Social_constructionismhttp://en.wikipedia.org/wiki/Grounded_theoryhttp://www.etwinning.net/http://en.wikipedia.org/wiki/European_schoolnethttp://en.wikipedia.org/wiki/Second_Lifehttp://en.wikipedia.org/wiki/University_of_Edinburghhttp://en.wikipedia.org/wiki/Princeton_Universityhttp://en.wikipedia.org/wiki/Rice_Universityhttp://en.wikipedia.org/wiki/University_of_Derbyhttp://en.wikipedia.org/wiki/Vassar_Collegehttp://en.wikipedia.org/wiki/University_of_Plymouthhttp://en.wikipedia.org/wiki/University_of_Plymouthhttp://en.wikipedia.org/wiki/Open_Universityhttp://en.wikipedia.org/wiki/Computer_based_training#cite_note-19http://en.wikipedia.org/wiki/Computer_based_training#cite_note-20http://en.wikipedia.org/wiki/Teaching_English_as_a_foreign_languagehttp://en.wikipedia.org/wiki/British_Councilhttp://en.wikipedia.org/wiki/Teen_Second_Lifehttp://en.wikipedia.org/wiki/Teen_Second_Lifehttp://en.wikipedia.org/wiki/Goethe-Instituthttp://en.wikipedia.org/wiki/Computer_based_training#cite_note-21http://www.cervantes.es/http://secondlife.cervantes.es/http://www.simteach.com/wiki/index.php?title=Institutions_and_Organizations_in_SLhttp://www.slanguages.net/http://www.slanguages.net/venue.phphttp://en.wikipedia.org/wiki/Mobile_Assisted_Language_Learninghttp://www.sbm.temple.edu/http://www.sbm.temple.edu/http://www.gradesgrowtutors.com/http://www.mnscu.edu/http://www.benastrum.com/http://www.sachem.edu/http://en.wikipedia.org/wiki/Computer_based_training#cite_note-22http://en.wikipedia.org/wiki/Computer_based_training#cite_note-23http://en.wikipedia.org/wiki/Computer_based_training#cite_note-24http://en.wikipedia.org/wiki/Computer_based_training#cite_note-25


15/41

Another real time collaboration provider making headway is Web Train. The

Technology in Business Schools Roundtable, a group representing over 70

Canadian and US business schools[27], announced a program for their

AACSB accredited members and affiliated colleges and universities to use

Web Train for faculty meetings, student association meetings, virtual live

classes, homework assistance, tutoring, teacher aid assistance, student

technical support and remote control, lecture broadcasting, board meetings,virtual labs and anonymous drug, rape and depression counseling.[28] The

announcement stated TBS will assist member business schools and their

associated colleges and universities with implementation planning and rollout

assistance to help increase the success of e-learning initiatives. The

announcement also stated Web Train will provide their hosted services free to

the business school and their associated college or university for a fiscal

school year to reduce financial risk related to using an enterprise class hosted

service.

Standard output

Much effort has been put into the technical reuse of electronically-basedteaching materials and in particular creating or re-using Learning Objects.These are self contained units that are properly tagged with keywords, or

other metadata, and often stored in an XML file format. Creating a courserequires putting together a sequence of learning objects. There are bothproprietary and open, non-commercial and commercial, peer-reviewedrepositories of learning objects such as the Merlotrepository.

A common standard format for e-learning content is SCORM whilst otherspecifications allow for the transporting of "learning objects" (SchoolsInteroperability Framework) or categorizing meta-data (LOM).

These standards themselves are early in the maturity process with the oldestbeing 8 years old. They are also relatively vertical specific: SIF is primarily pK-12, LOM is primarily Corp, Military and Higher Ed, and SCORM is primarilyMilitary and Corp with some Higher Ed. PESC- the Post-Secondary EducationStandards Council- is also making headway in developing standards andlearning objects for the Higher Ed space, while SIF is beginning to seriouslyturn towards Instructional and Curriculum learning objects.

In the US pK12 space there are a host of content standards that are critical aswell - the NCES data standards are a prime example. Each stategovernment's content standards and achievement benchmarks are criticalmetadata for linking e-learning objects in that space.

An excellent example of e-learning that relates to knowledge managementand reusability is Navy E-Learning, which is available to Active Duty, Retired,or Disable Military members. This on-line tool provides certificate courses toenrich the user in various subjects related to military training and civilian skillsets. The e-learning system not only provides learning objectives, but also

15
http://en.wikipedia.org/wiki/WebTrainhttp://en.wikipedia.org/wiki/Computer_based_training#cite_note-26http://en.wikipedia.org/wiki/Computer_based_training#cite_note-26http://en.wikipedia.org/wiki/WebTrainhttp://en.wikipedia.org/wiki/Online_lecturehttp://en.wikipedia.org/wiki/Online_lecturehttp://en.wikipedia.org/wiki/Computer_based_training#cite_note-27http://en.wikipedia.org/wiki/WebTrainhttp://en.wikipedia.org/wiki/Learning_Objectshttp://en.wikipedia.org/wiki/Metadata_(computing)http://en.wikipedia.org/wiki/XMLhttp://www.merlot.org/http://www.merlot.org/http://en.wikipedia.org/wiki/SCORMhttp://en.wikipedia.org/wiki/Learning_objecthttp://en.wikipedia.org/wiki/Schools_Interoperability_Frameworkhttp://en.wikipedia.org/wiki/Schools_Interoperability_Frameworkhttp://en.wikipedia.org/wiki/Learning_object_metadatahttp://en.wikipedia.org/wiki/WebTrainhttp://en.wikipedia.org/wiki/Computer_based_training#cite_note-26http://en.wikipedia.org/wiki/WebTrainhttp://en.wikipedia.org/wiki/Online_lecturehttp://en.wikipedia.org/wiki/Computer_based_training#cite_note-27http://en.wikipedia.org/wiki/WebTrainhttp://en.wikipedia.org/wiki/Learning_Objectshttp://en.wikipedia.org/wiki/Metadata_(computing)http://en.wikipedia.org/wiki/XMLhttp://www.merlot.org/http://en.wikipedia.org/wiki/SCORMhttp://en.wikipedia.org/wiki/Learning_objecthttp://en.wikipedia.org/wiki/Schools_Interoperability_Frameworkhttp://en.wikipedia.org/wiki/Schools_Interoperability_Frameworkhttp://en.wikipedia.org/wiki/Learning_object_metadata


16/41

evaluates the progress of the student and credit can be earned toward higherlearning institutions. This reuse is an excellent example of knowledgeretention and the cyclical process of knowledge transfer and use of data andrecords.

Computer-Based Training andAssessments:

An Exploratory Study of SocialFactors

16


17/41

Abstract

An exploratory research program on different types of hybrid classes toanswer those and other questions around its efficacy and applicability for

training and education. Our objective is to learn the outcomes through theireffect on in-class and computer training phases of knowledge and skillsacquisition and testing. The overall research question is: Which and howmuch do CBT, individual student, class, instructor, and CBA factors affect

17

Q u i c k T i m e d e c o m p r e s

a r e n e e d e d t o


18/41

student learning outcomes? Leidner and Jarvenpaa (2001) work introduced aresearch model that helps instructors determine the best teaching methoddepending on course content, available technology, an individual instructor,and student factors. Thirty-six questions were posed to over 400 students withdirect and current experience using CBT and CBA for course credit. Thefindings show that there is a strong potential for student as well as corporatebenefits in training using online assessment tools. Online assessmenteffectiveness should be given further research study given the explosive jump

in reported learning.

1. Introduction

A quick scan of educational institutions and their programs show they areincreasingly turning to computer-based training (CBT) and computer-basedassessment (CBA) tools, especially for entry-level courses like introductorycomputing or for administering computer literacy or proficiency exams.

Electronic, online, or computer-based training can provide a number ofadvantagessuch as time and place convenience for students andinstructors, standardized delivery, self-paced learning, economies of scale interms of classrooms and instructors, automated feedback to students and

18


19/41

instructors, and a variety of available content (Strother, 2002). IT can assistan instructor in extending availability beyond class time and office hours,establish links between classmates, and accomplishing administrativeactivities (Benbunan-Fich, 2002). One of the leading CBA/CBT providers,Course Technology, boasts on its web site how many millions of exams havebeen taken using their Skill Assessment Manager software since its inceptionin 1998. Additionally, data from CBA results can be used to conduct itemanalysis and strengthen course personalization, content, and delivery. There

is a potential for computer-based teaching methods to improve classroomlearning that remains untapped by the inability to use them effectively (Leidnerand Jarvenpaa, 2001). And it's not just used in academia. In 1999, companiesin the United States spent $62.5 billion on training and educating theiremployees, with more than $3 billion spent on technology-delivered trainingestimated to then to grow to $11billion by 2003with some retailers doing20% of their training online (Khirallah, 2000). Companies are using it toscreen job applicants, train employees, and test them on the training. Majorvendors like IBM and Hewlett-Packard have established large CBT/CBAprograms; IBM alone offers hundreds of subjects in 26 curriculum areas. Butdoes it work well? There is a growing body of literature in and out of the

academic community on its pedagogical efficiencies and effectiveness. One ofthe issues with CBT and CBA is using "hybrid" coursescombining traditionallecture pedagogy with computer-based technology to reap the best of bothworlds. Moreover, after many lectures of June 16, 2005 introductory universitycourses are conducted in largeclasses to more easily absorb enrollmentvariations, provide economies of scale in terms of classrooms, and moreefficiently use the skills of a professor especially during times of shrinkingfinancial support. Using CBT and CBA with large classes can reportedlyimprove personalization, content tailoring, increased flexibility, and decreasedadministrative overhead. But what is the best mix of traditional and computer-based learning and assessment? What are the issues that need study and

addressing? What are the factors that affect learning outcomes? How are theyrelated? This paper introduces an exploratory research program on differenttypes of hybrid classes to answer those and other questions around itsefficacy and applicability for training and education. Our objective is todevelop and perform an initial test of a new model designed to trace theinfluence of individual and technical characteristics on learning outcomesthrough their effect on in-class and computer training phases of knowledgeand skills acquisition and testing.The overall research question is:

Which and how much do CBT, individual student, class, instructor, andCBA factors affect student learning outcomes?

19


20/41

20

Q u i c k T i m e a n d ad e c o m p r e s s o r

a r e n e e d e d t o s e e t h i s p i c t u r


21/41

2. Literature Review

Previous literature relative to this study's research question is in two majorareas: using CBA and online tools in a hybrid course, and the relationshipsbetween various technical and individual characteristics and academicperformance.

Two types of classroom information technologies are reviewed here: usingtechnology to improve student learning, and using technology to improvestudent performance evaluation.

A hybrid course provides teachers and students with face-to-face lectures andtechnology-enabled interaction for explanations, small group discussions,presentations, and individual assistance. This instructional format has beenfound to have many advantages over traditional lectures (Christopher, 2002).First, interaction between the professor and the students is regulated by the

professor and occurs one-by-one; interaction via technology is controlled bythe students and can occur in parallel. Second, students often receive anadvance copy of the lecture slides and some prefer to study "at home" ratherthan attend class; studying via technology can always be done at home.Third, lectureseven while attendedmay not have student attentionnecessary for learning; training provided through technology may be morelikely to keep student attention. Finally, people may learn more by doing thanby watching and listening. At the same time, online training may be a viablealternative to those from rural areas and students with nontraditionalschedules. Research has shown that the hybrid format can couple onlinehomework with in-class, active learning exercises to improve attendance (Van

Blerkom, 1992). Cameron (2003) used simulation in a hybrid course onnetworking, and found that it improved conceptual understanding and raisedperformance. Willett (2002) proposed to use online discussions to provide agood substitute for in-class discussion, and Haggerty, Schneberger, and Carr(2001) found that online discussions lead to better cognition due to theincrease in available time to reflect and respond.

21


22/41

Cywood and Duckett (2003) discovered no significant differences betweenquantitative measures of online versus on-campus learning and suggest thatthere is no actual difference regarding learning. McGray (2000) demonstratedthe potential of IT to enable an instructor to be more efficient and effective inbroadening and deepening the learning process for business students in MIS.It has been shown that technology allows individuals to share tacit knowledge

22


a r e n e e d e d t o s e e t h i s p


23/41

in a manner uninhibited by the time and location (Leidner and Jarvenpaa,1993). Another study, by Caywood and Duckett (2003), looked at theperformance of students on campus and online during one specific course;the results showed no significant differences in learning across environmentsand concurred with the previous studies. But what are the factors involved inhybrid teaching? Bostrom, et. al., (1988) argued that individual differences areimportant for end-user training. Two studies in particular examined factors thatinfluence

computer training and skill gaining (Leidner, D. and Jarvenpaa, S. (2001); Yiand Davis, 2003). This exploratory study is aimed at identifying and testingspecific variables that could predict learning outcomes for CBA. Leidner andJarvenpaa (1995) described using technology to support an objectivist modelof learning in hybrid courses by facilitating information delivery via atechnology-enhanced instructor console and by using CBA.

CBA allowed students to learn more effectively and efficiently because theywere in control of the pace, time, and location. CBA feedback can be a criticalpart of learning; active involvement can lead to more effective learning thanpassive involvement. CBA enables instructors to collect, analyze, and use

information about student learning as feedback to improve their teaching, andthey enable students to demonstrate what they know (Ebert-May, Baltzli, andLim, 2005). According to Riffell and Sibley (2003), surveyed studentsresponded that the most effective way to learn material was through onlinehomework and email with instructors. Ricketts and Wilks (2001) suggestedthat well-designed CBA can benefit students by improving their performancesin assessments in the introduction of statistics in biology. Noyes, Garland, andRobbins (2004) studied paper-based and computer-based assessments,comparing the test performances of undergraduate students taking each testtype. Given the identical multiple choice questions, students who used CBAachieved better results than those taking paper-based tests, and students with

higher scores were found to benefit the most from CBA. Finally, CBA helpedto improve long term recall of key concepts and resulted in higher scores thanconventional exams, and students with computer experience had noadditional advantages versus less experienced students (Bocij and Greasley,1999).

Many researchers have studied the relationships among student individualcharacteristics and academic performance. Previous academic history andgrades, as well as propensity to study, are the most popular dimensions. Ariasand Walker (2004) found strong negative relationships between classsize and student performance calculated as aggregate exam points whileteaching economics. The results suggested to them that student ethics andproximity to an instructor in small classes help studentsunderstand economic concepts better. They included several measures ofstudent academic abilities, i.e., SAT, SAT verbal and SAT math, GPA, anddemographic data (such as year of study, age, and gender) as explanatoryvariables and class size as the control variable. A number of noncognitiveindividual dimensions not measured by academic outcomes relate toacademic performance as well. Gender, family size, and income have beenused as academic performance predictors. Availability of support systems andpreference of long term goals over short term needs were proposed by

Northcote (2002). On the other hand, external collaborations (i.e., cheating)on online assessments have been shown to be problematicas they are withtraditional paper testing (Kozma, 2003). Compeau and Higgins (1995)concentrated on studying self efficacythe conviction that one can control

23


24/41

his/her outcomes and do what is necessary to produce a certain resultandits importance in user acceptance and use of information technology.Learning style defined through demographic variables were found to have aneffect on teaching and learning processes (Bostrom, et. al., 1990). Studentmajor as a predictor was mentioned in McGray (2000). There is also literatureon the effectiveness of technical support for computer assisted learning. Bocijand Greasley (1999) concluded that students with computer experience hadno additional advantages versus less-experienced students. Another reported

factor that affects academic performance and CBA is class size. Hancock(1996) found no significant difference in the performance among students inthree large and six small classes on statistics. Tuckman (2002) compared theacademic performance and learning in terms of GPA in a hybrid course (189students in two academic quarters) and a traditional course (74 students intwo academic quarters) with the knowledge of the control group (189 studentswho did not take the course) using the same textbook, content, andperformance activities. His results suggested that student skills using thecombined classroom and computermediated model improved significantlymore in academic performance than the students taught the same material bythe conventional classroom approach. Siegfried and Kennedy, Durfee, et. al.,

(2005), and Amoroso (2004) found no evidence to support that teachingstrategy should depend on class size. Hill (1998) investigated the effect oflarge sections of 120 students in accounting on their performance andperceptions in the introductory courses. Data was collected from studentsurveys, instructor and university records, and student course evaluations.She used student interest toward accounting, course organization andplanning, instructor-student interaction, student course evaluations, GPA andSAT scores, attendance, age, academic hours earned, hours worked, hoursstudied, and course completion as independent variables; final exampercentage scores and the overall course percentage points were thedependent variables. The study did not find statistically significant differences

between student performance and section size. When attendance anduniversity GPA were controlled, the large sections actually outperformed thesmall sections.

In summary, there are many factors based on previous research that mayaffect student outcomes in hybrid courses using CBT and CBA.

3. The Research Model

Leidner and Jarvenpaa (2001) proposed a research model that helpsinstructors determine the best teaching method depending on course content,available technology, an individual instructor, and student factors. They

24


25/41

suggested that the amount of class versus in-class learning should depend onthe chosen teaching method and impact of out-of-class learning (i.e.,computer based training and assessments in our case) and in-classperformance (i.e., paper exams). They used graduate students in a smallclass to investigate the proposed relationship and suggested that thisexperimental setting can be changed to reveal other interesting relationships.

At the same time, Yi and Davis (2003) presented the conceptual framework ofthe relationships between modeling-based training interventions, pre-training

individual differences, learning processes, and training outcomes. They hadtested the model with 95 students engaged in computer spreadsheet training.Based on this literature, our observations, and our desire to extrapolate fromprevious studies on larger undergraduate student bodies, we propose theoverall research model shown in Figure 1.

25


26/41

26


a r e n e e d e d t o s e e t h


27/41

Figure 1. Proposed Research Model

The model incorporates three main groups of factors leading to the dependentvariable, the learning outcome. The first group consists of the characteristicsof the technology and the characteristics of the individual. The technologycharacteristics include variables such as ease of use, understandability, easynavigation, and Internet connection speed. The individual studentcharacteristics include variables like previous computer experience, currentgrade point average, and self-efficacy. The second group is about thetraining, consisting of computer-based training and traditional classroomlectures. Some variables for computer-based training are technical support,how often it's used, and where it's used (at home or in a school computerlaboratory). Classroom lecture variables include the size of the class, theinstructor, and when the class meets. It should also be noted that we believethe technology characteristics would directly influence the computer-basedtraining, while the individual characteristics would affect the computer-basedtraining and the classroom lecture training. The third group of variables in the

model concern computer-based assessments, with variables such as helpfrom other sources during the assessment, where the assessment isaccomplished, and technical support during the assessment. Finally,outcomes are generally measurable variables showing the results of individualtraining and assessment, such as the difference between pre- and post-tests.

4. Research Methodology

This paper describes the initial, exploratory study of a longer-range researchprogram on computer-based training (CBT) and computer-based

assessments (CBA). As such, we purposely chose to use a wide range ofvariables for the factors identified in our research model shown in Figure 1.The variables were from the aforementioned literature search, our ownexperiences using CBT and CBA, and reasoned postulation. Even thoughsome variables appeared to overlap, they were still used for subsequentrefinement as the program progresses. The key variable driver was to beinclusive, not exclusive, in variable selection. To be exploratory, our datacollection methodology used the survey approach and direct performancemeasurement. Thirty-six questions

Technology Characteristics

Individual Characteristics

Computer-based Training

Classroom Lectures

Training

Computer-based assessments

Outcomes

27


28/41

Proceedings of the 39th Hawaii International Conference on System Sciences

2006 were posed to over 400 students with direct and current experienceusing CBT and CBA for course credit. Additionally, the subjective orperceptive survey data was matched with measured, objective courseperformance scores. On top of that, performance data from about 150students using the CBT/CBA software was added. The combination of twocollection approaches allowed us to search for relationships among thesubjective data, among the objective data, and between the subjective andobjective data. Moreover, the measured data gave us measured learningoutcomes which could be used as dependent variables during data analysis.The survey instrument consisted of 36 questions on the responder'sdemographics, computing experiences and skills, and opinions about theCBT/CBA software or its use (see Appendix 1 for a list of the surveyquestions). Some were open-ended, some checked their perception withknown data, and most asked for subjective answers using a Likert scale. Thesurvey was offered online for respondent convenience (i.e., they didn't have tocome to class, they could take it anywhere there was Internet access,and they could take it any time of the day) and for automated surveymanagement and data compilation. Students volunteered to take the onlinesurvey, but were given token credit toward their final coursegrade for doing so. The survey was available for two weeks, but studentswere not allowed to take it more than once. In opening up the survey,

students used a personal and unique non-descriptive campus ID codewhichcould then be correlated with their course performance scores. When thesurvey was closed, the data was downloaded in spreadsheet format for inputto statistical analysis programs. The measured data consisted of performance

28

QuickTimedecompres

are needed to s


29/41

scores on CBA exams and traditional paper exams. The CBA exams testedstudent skills performing specific tasks using Microsoft Office 2003applications. The software presented a screen exactly duplicating the Officeapplication with a canned document, and asked students to perform anoperation (e.g., globally replace all instances of "bought" with "purchased").To reduce cheating, each CBA exam had a time limit for minimizing the use ofnotes or textbooks, and the tasks were presented in random order to eachstudent. Most importantly, the CBA software did not score solely the task

result; it scored the steps taken to achieve the result and the order in whichthey were done. In other words, a student had to perform the right task stepsin the right orderit was very difficult to "experiment" in achieving the taskresult. Realizing that some steps may have been done in error (e.g., mouse-clicking twice instead of once), each student was given two opportunities tocorrectly perform each task. Each task was scored as right or wrong, and theresults were automatically scored for reporting. There were six CBT exams; acomprehensive pre-test exam on the entire course, four exams on specificOffice applications, and one comprehensive post-test. Each exam was openonline for one 15hr window to minimize conflicts with other courses, outsidework, etc. Students could take the CBA exams in the school laboratories, or

on their own or anyone else's computer provided the CBT software wasloaded on the machine. The traditional paper exams were administeredin class using paper exam booklets and opticalreadable scan sheets that wereautomatically scored, analyzed, and reported to the professor. Each examtested a student's knowledge about Office 2003 applications; the combinationof the paper exams and the computer-based exams measured studentknowledge and performance, respectively. There were two paper-basedexams; one on each half of the course. All students used the same CBT/CBAsoftware linked to the same course textbook. Office applications were coveredin class using traditional lectures supplemented a large screen projectionsystem for lecture slides, and a podium computer used to demonstrate

application tasks onto the large screen display. Six course sections weresmall (100 students). Allsections were taught individually by two professors; one had five smallsections one semester then three large sections the following semester, theother had one large and one small section the same semester. Bothprofessors followed identical syllabi over both semesterseach taught thesame topics in the same sequence using the same textbook, lecture slides,and exams. The students were university undergraduate students of a widerange of ages and of all academic years from all schools across campus. Itwas a required course for some students, but not for all.

29


30/41

5. Methodology

The initial step in data analysis was to pair CBA performance scores withsurvey results. The data and the matches were not totally complete; not allstudents took all CBA and paper exams, not all students took the survey, afew students gave random identification codes when taking the survey (andtherefore could not be matched), and some simply skipped to the last surveyquestion. But most analysis didn't require a complete set of all variables fromall students, especially with almost 500 students the entire sample set(n=489). There were 226 students with complete, matched surveys.Proceedings of the 39th Hawaii International Conference on System Sciences

2006 shows the student demographics of the set of students who completedthe survey and were matched with performance scores.

Mean Range

GenderMale: 145Female: 81

Age 19.91 18-32Academic YearFreshman: 49Sophomore: 119Junior: 37Senior: 21Grade Point Average 2.95 1.50-4.00Internet Connection Type Dial-up: 7

Cable/DSL: 140T1 or better: 58Don't know: 21Years Experience on the Internet 8.27 3-11

Student Demographics (Matched Students)

The first set of exploratory data analysis was achieved using multivariateanalysis on all the variables to find potential relationships that maywarrant further analysis. The second set of exploratory data analyses wasdone using the general linear model. The univariate analysis was made withthe difference between the CBA comprehensive pretest and post-test(variable A1A6) as the dependent variable. The initial set of independentvariables was all the independent variables to look for potentialcorrelations and those deserving closer examination. The next analysisfocused in on the independent variables class section, class size, instructor,written exam scores, SAT scores, GPA, the demographic data, andcomputing skills and experience. Then various groups and combinations ofthe independent variables were analyzed to focus in on specific factorsrelative to the dependent variable as suggested by previous literature and ourresearch model. The independent variables were also examined with the

paper exam scores as individual dependent variables.

30


31/41

6. Results and Implications

Some of the key results are shown in subsequent tables. Not all thestatistically significant correlative results are shownonly the correlationswith 1- tailed significance of .005 or less given the initial exploratory nature ofthe data analysis. Only the very strongest correlations are shown. Table 2shows the correlations among the pre-test/post-test variable A1A6 and somestudent characteristics (again, only statistically significant correlations at .005are shown). The regression model had a significance of .003.

Significant Variable Correlation Coefficients,

First GroupA1A6 Sec Inst GPA A1 Q3 Q4 Q7 Q9 Q10 Q11 Q12 Q13A1A6Sec .238

Inst .162GPA .256Q3 .239Q4Q7 -.250Q9Q10Q11 -.326 .222 .348 .586Q12 -.347 .296 -.249 .398 .410 .772Q13 -.291 .292 -.286 .418 .273 .554 .717Q14 -.283 -.276 .194 -.309 .441 .467 .540 .677

Note: A1A6: Difference between pre-test and post-test SEC: classsectionInst: instructorGPA: university cumulative grade point average

A1: the comprehensive CBA pre-testQ3: genderQ4: ageQ7: Internet connection typeQ9: level of Internet experience

Q10: computer skill level IQ11: computer skill level IIQ12: computer skill level IIIQ13: computer skill level IVQ14: computer skill level VSignificant Variable Correlation Coefficients,

Second GroupQ18 Q19 Q21 Q22 Q23 Q24 Q25 Q26 Q27 Q28 Q32 Q33 Q34Q18Q19Q21Q22Q23 .245 .245

31


32/41

Q24 .253Q25 .255 .243 .287 .276Q26 .270Q27 .286 .306Q28 .231 .267Q32 .226 .227Q33 .250 .252 .350 .258Q34 .226

Q35 .287Q18: CBT/CBA software is easy to useQ19: CBT software helps prepare for CBAQ21: CBT software reduced time to learn OfficeQ22: CBT improved ability to use OfficeQ23: using technical support oftenQ24: technical support is timely and effectiveQ25: technical support is accessible and knowledgeableQ26: work hard in courseQ27: certain can master the courseQ28: receiving help during CBA

Q32: CBT prepares me well for CBAQ33: use CBT oftenQ34: often discuss course with friends and familyQ35: often receive emotional support from others

CBA performance improvement (A1A6) is correlated to class section andinstructor. This suggests that even though students performed computer-based skills training and took computerbased skills assessments, theaggregate group of students and the individual instructor teaching themwith traditional lectures still had a significant effect on their computer-basedassessments. Follow-on research comparing this data with students who

relied solely on CBT and CBA without any classroom lecture sessions mightclarify the importance of combining traditional lectures with CBT/CBA.Further research could also investigate whether the section and instructorbenefits are due to cognitive, social, procedural, or explanatory factors(Haggerty, Schneberger, and Carr, 2001). We found that the higher thecumulative GPA, the higher the CBA improvement (A1A6). This implies thathigh performing students in general also perform well in CBAnot just inclassroom environments. This suggests that at least some of the beneficiallearning skills used in traditional classroom settings can be beneficially usedin computer-based settings. We also found that the higher the cumulativeGPA, the less initially known about computers and the Internet. Theimplications could be numerous and beg further study. Is academicperformance degraded by high levels of student attention to computers? Arecomputer activities including gaming, Internet surfing, etc., supplantingattention to coursework? Do lesser computer-savvy students realize theirshortfall and work harder on CBT and CBA as a result? Are high performingstudents less inclined or interested in computers, and vice-versa? Oneimplication for CBT and CBA, however, is that high performing students mayneed special attention in terms of basic computer skills, but their ultimate

32


33/41

CBA performance will not suffer as a result of low computer skills (seeparagraph 2 immediately above). Females had higher GPAs than males, and

males were initially more computer savvy. This follows on the heels ofparagraph 3 immediately above. The more important implication for CBA,however, is in the links with paragraphs 2 and 3 above; that gender may be afactor in how much a student improves but not directly because of gender

33


a r e n e e d e d t o s e e t h i s


34/41

per se but because of the gender imbalance in initial computer knowledge.The higher the initial student computer skills, the higher the initial skillsassessment scores. While this may seem inescapable, the two skill sets arenot necessarily identical; one involves skills about basic computer operations,while the other concerns specific Microsoft Office application skills. But thispositive correlation implies measuring one may be a useful indicator to theother, especially given the widespread use of Microsoft Office applications.The higher the initial student computer skills, the less they improve overall in

computer-based assessments. While this may seem trivial since students whostart out with higher skills assessment scores have less room to improve, thedata also showed that students with higher initial computer skills did not scorecommensurately higher in the post-test computer-based assessment. Thisalso begs further investigation. Do computer-savvy students ease off in theirefforts during the course because they don't feel they need to study andpractice, while students with less computer knowledge work harder tocompensate for their sense of computer inadequacy? The more accessibletechnical support is during computer-based training and assessment, theeasier it is to use CBA, the more they take advantage of computer-basedtraining, and the greater their perceived benefit from CBA. The happier a

student is about technical support, the greater the student's belief in beingable to master the CBT/CBA course. These relationships imply that technicalsupport plays an important role in successful CBT and CBA. Computer-basedtraining and assessments require well functioning computers and software.When there are problemsespecially to students with lower initial computerknowledge and skillstechnical support can make or break a CBT or CBAsession. These relationships suggest that good technical support may beessential to good CBT and CBA. The higher the perceived benefits of CBTand CBA, the more often it's used. This may also appear insignificant, but theimplication for CBT/CBA is that the more the potential benefits are explainedand understood, the more they can be realized through increased utilization.

The more other students and friends help, the less time it takes to learn CBAand the CBA material.

This suggests that social networking is important to CBA, just as the classsection and instructor are. This implies that face-to-face interaction with otherscan improve individual interaction with computerbased training andassessment software. While isolated CBT and CBA may be the mostconvenient for students, it may not be optimum for learning efficiency. Thehigher students' perceptions about how hard they work, the more they use CBtraining, and the more they discuss the course with friends and family. Causalrelationships are not readily evident here and suggest further areas of study.But these correlations again suggest that hard work and networking withfriends and family may have a role in CBT/CBA effects. If so, it may suggestthe importance of a holistic approach to CBT/CBA courses well beyond justthe underlying technology.

34


35/41

Survey Questions

Demographics

1. English is my primary language (1=Yes, 2=No)

2. My SAM username is ____________

3. My gender (1=Male, 2=Female)

4. My age is __________

5. My academic year (1=Freshman, 2=Sophomore, 3=Junior, 4=Senior)

6. My current GPA: __________

7. My Internet connection type (1=Dial-up, 2=Cable/DSL, 3=T1 or better,4=Don't know)

8. Number of years using the Internet (9)

9. Experience with Internet (1=No, 2=Little, 3=Some, 4=Much, 5=Extensive)

Computing Skills

10. Basic skills like typing a document, etc. (1=strongly disagree, 2=slightlydisagree, 3=indifferent, 4=slightlyagree, 5=strongly agree)

11. Install programs, etc. (same 5pt. scale)

12. Set up virus checkers, etc. (same 5pt. scale)

13. Install networks, etc. (same 5pt. scale)

14. Install new hardware (same 5pt. scale)

SAM Expertise

15. Learning SAM was easy (same 5pt. scale)

16. Navigating and accomplishing SAM tasks is easy (same 5pt. scale)

17. What SAM tells me is clear and understandable (same 5pt. scale)

18. Overall, I find SAM easy to use (same 5pt. scale)

19. SAM helps me prepare for assessments (same 5pt. scale)

20. SAM easily trains me on MS Office basic functions (same 5pt. scale)

21. SAM decreased the time to learn MS Office functions (same 5pt. scale)

22. SAM improved my ability to use MS Office (same 5pt. scale)

35


36/41

23. I use SAM technical support often (same 5pt. scale)

24. SAM tech support helps me well and timely with SAM problems (same5pt. scale)

25. SAM tech support is very accessible and knowledgeable (same 5pt. scale)

28. I receive help from other students while doing assessments (same 5pt.

scale)

29. I receive help from SAM/IT tech support while doing assessments (same5pt. scale)

30. I receive help from my instructor while doing assessments (same 5pt.scale)

31. SAM training reflects what is covered in assessments (same 5pt. scale)

32. SAM training prepares me well for assessments (same 5pt. scale)

33. I use SAM training often (same 5pt. scale)

36. I do assessments (1=on my own computer, 2=in the lab, 3=at a friend'shouse, 4=elsewhere).

Self-Efficacy

26. I work very hard and persistently in CIS1025 (same 5pt. scale)

27. I am certain I can master the skills in CIS1025 (same 5pt. scale)

34. I often discuss CIS1025 content with friends/family/etc. (same 5pt. scale)

35. I often receive general emotional support from others (same 5pt. scale)

36


37/41

Q u i c k T i m e a nd e c o m p r e s s o r

a r e n e e d e d t o s e

37


38/41

7. Conclusions

Organizations that use computer-based training and assessment tools canpotentially reap significant rewards in improved employee knowledge and skilllevels. As stated earlier, this is the initial, exploratory stage in a series ofstudies on efficacy of computerbased training and assessment. The findingsin this study may be limited to some degree to students at one school in one

American regionalthough the mix of students at this one university appearsgood for generalization. The motivational factors of students, however, maynot correspond with corporative employee motivations; corporate CBT andCBA users need to be studied as well. Given the convenience factor of CBT,CBA, and online surveys that are available to use on any computer with theright software and Internet access, the students who took CBT, CBA, and thesurveys were not all in controlled physical environments. It cannot be saidwith certainty that the person taking CBT, CBA, and the survey was the actualperson supposed to be doing the work, or did not receive undue help from

others while online. But if most students would not engage in this deception,we believe that the large number of students involved mitigates this effect inthe results. There appear to be large numbers of opportunities to extend thispreliminary research by looking at all correlations with 1-tailed significanceless than .05 rather than .005, refining the variables, adding new variables,exploring the cumulative effects of the variables on the proposed researchmodel in Figure 1, and expanding data analysis efforts. Numerousopportunities for searching for causal relationships in the model may beparticularly beneficial to CBT/CBA developers, educators, and users. The roletechnical support plays may be of particular interest, including its educationalvalue in addition to textbooks, lectures, and computer-based training.

38


39/41

References

Amoroso, D. (2004), Use of Online Assessment Tools to Enhance StudentPerformance in Large Classes.In the Proceedings of ISECON- 2004, 21: 8.

Arias, J. J. and Walker, D. M. (2004), Additional Evidence on the Relationshipbetween Class Size and Student Performance. Journal of EconomicEducation 35(4): 311-326.

Benbunan-Fich, R. and Hiltz, S. (2002), Correlates of Effectiveness ofLearning Networks. Proceedings of the 35th HICSS.

Bocij, P. and Greasley, A. (1999). Can computer-based testing achieve qualityand efficiency in assessment? International Journal of EducationalTechnology, Vol. 1, No. 1.

Booms, B. H. and D. L. Kaltreier (1974), Computer-Aided Instruction for LargeElementary Courses. Economic Education 64(2): 408-413.

Bostrom, et. al. (1990), The Importance of Learning Style in End- UserTraining, MIS Quarterly, March 1990, pp 101-120 Cameron, B. (2003), Theeffectiveness of simulation in a Hybrid and Online Networking Course,TechTrends: Vol. 47, No. 5, pp 18-21.

Caywood, K. and Duckett, J. (2003), Online vs. on-campus learning in teachereducation. Teacher Education and Special Education, Vol. 26, No. 2, p. 98-105.

Christopher, D. (2002), Interactive Large Lecture Classes and the Dynamicsof Teacher/Student Interaction. Journal of Instruction Delivery Systems 17(1):13-18.

Compeau, D. and Higgins, C, Computer Self-Efficacy: Development of aMeasure and Initial test. MIS Quarterly, Vol. Proceedings of the 39th HawaiiInternational Conference on System Sciences - 2006, No. 2, 1995. pp 145 -158.

Durfee, A., Schneberger, S., and Amoroso, D., Computer-Based

Assessments of Student Performance in Hybrid Classes: Does Class SizeReally Matter?Americas Conference on Information Systems (AMCIS) 2005.

Driscoll, M. (2001), Building Better E-Assessments, ASTDs Source for E-Learning:Learning Circuits, http://www.learningcircuits.org/2001/jun2001/driscoll.html.

Ebert-May, D., Batzli, J., Lim, H. (2003), Disciplinary research strategies forassessment of learning. Bioscience Vol. 53 No. 12: pp. 1221-1228.

Haggerty, N., Schneberger, S. and Carr, P. (2001), Exploring MediaInfluences on Individual Learning: Implications for Organizational Learning.Proceedings of the 22nd Annual International Conference on InformationSystems (ICIS), New Orleans, LA, December 16-19, 2001.

39
http://www.learningcircuits.org/2001/jun2001/driscoll.htmlhttp://www.learningcircuits.org/2001/jun2001/driscoll.html


40/41

Hill, M. (1998), Class size and Student Performance in IntroductoryAccounting Courses: Further Evidence. Issues in Accounting Education 13(1):47-64.

Hogan, D and Kwiatkowski, R. (1998), Emotional Aspects of Large GroupTeaching. Human Relations 51(11): 1403-1417. Khirallah, D. (2000), A newway to learn? InformationWeek, May 22, 2000.

Kozma, R. (2003), Technology and Classroom Practice: An internationalStudy. Journal of Research on Technology in Education, v. 31, no. 1, p. 13.

Leidner, D. and S. Jarvenpaa (1995), The Use of Information Technology toEnhance Management School Education: A Theoretical View. MIS Quarterly:pp. 265-292.

Leidner, D. and Jarvenpaa, S. (2001), The Information Age ConfrontsEducation: Case Studies on Electronic Classrooms. Information SystemsResearch 4:1, 24-54

Mason, P., Problems in the handling of large sections in accounting by thelecture method. The Accounting Review: 179-182.

McCray, C. (2000), The hybrid course: merging on-line instructionand the traditional classroom. Information Technology and Management, Vol.1, No. 4, pp. 307-327.

Northcote, M. (2002), Online assessment: friend, foe, or fix? British Journal ofEducational TechnologyVol. 33, No. 5, pp. 623-625.Noyes, J., Garland, K. and Robbins, L. (2004), Paper-based versus computer-

based assessment: is workload another test mode effect? British Journal ofEducational Technology, Vol. 35, No. 1, pp. 111-113.

Ricketts, C., Wilks, S. J. (2002), Improving Students Performance ThroughComputer-based Assessment: Insights From Recent Research. Assessment& Education in Higher Education, v. 27, no. 5, p. 476-479.

Riffell, S.K. and Sibley, D.H. (2003), Learning Online Student Perceptions of aHybrid Learning Format. Journal of College Science Teaching, Vol. 32, No. 6.

Siegfried, J. and P. Kennedy, Does Pedagogy Vary with Class Size inIntroductory Economics?AEA Papers and Proceedings, Better Learning fromBetter Management85(2): 347-351.

Strother, J. (2002), An Assessment of the Effectiveness of e- Learning inCorporate Training Programs, International Review of Research in Open andDistance Learning, Vol. 3, No. 1, April 2002.

Tuckman, B.W. (2002), Evaluating ADAPT: a hybrid instructional modelcombining Web-based and classroom components. Computers & EducationVol. 39, No. 3: pp. 261-269.

Van Blerkom, M. L. (1992), Class attendance in undergraduate courses.Journal of Psychology, 126, 487-494.

40


41/41

Willett, H. (2002), Not one or the other but both: hybrid course delivery usingWebCT. The Electronic LibraryVol. 20, No. 5: pp. 413-419.

Yi, M. and Davis, F. (2003), Developing and Validating an ObservationalLearning Model of Computer Software Training and Skill Acquisition,Information Systems Research, vol. 14 no 2, June 2003, pp146-169.

mis - computer based training

Documents