computer-based assessment and the effects it has on … uses their computer-based learning (cbl) to...

1

Computer-based Assessment and the Effects it has on Middle School Aged Students

by

John Mulvaney A Research Paper

Submitted in Partial Fulfillment of the Requirements for the

Masters of Science Degree in

Education

Approved: 2 semester Credits

_______________________ Dr. Diane Klemme

The Graduate School University of Wisconsin-Stout

December, 2011

Diane KlemmeDigitally signed by Diane Klemme DN: cn=Diane Klemme, o=UW-Stout, ou=School of Education, [email protected], c=US Date: 2011.12.22 15:45:55 -06'00'

3

Abstract

The purpose of this study is to examine computer-based assessment and their effect on

middle school age student learning. The primary purpose of the study is to gain understanding of

computer-based assessment accurately measures what students are learning and retaining.

A researcher designed survey was distributed to faculty of students in grade six, seven,

and eight at a rural upper Midwest school. A research designed survey was also distributed to students in

Family and Consumers Sciences classes at the same school. Findings from the study concluded that

among educators at Tomah Middle School the results were indifferent about computer-based

assessment. A few areas they strongly agreed were that students could easily log in and use

assessments, that computer-based assessments are the future of how students will be assessed,

and that they save educators time. Students were more favorable toward CBAs and were more

positive. One Surprise was student’s thoughts about being able to browse during computer-based

assessments.

4

Acknowledgements

I would like to take this opportunity to thank the students and staff of Tomah Middle school who were a tremendous help in conducting research into their thoughts and feelings about the use of Computer-based and online assessments. I would also like to thank my advisor Diane Klemme PhD. who without her help I would have never finished this paper. Finally I would like to thank my wife and family who if not for them I could never have pushed myself to be where I am today.

5

Table of Contents

……………….……………………………………….………………………………………Page

Abstract………………………………………………..……………………………………….…3

List of Tables………...……………………………………………………………………………...…….7

Chapter: I Introduction…………………………………………………..………………………...8

Statement of Problem………………………………….………….…………………...…..9

Purpose of Study…………………………………………………….………...………....10

Assumption of Study…………………………………………………….………………10

Limitations……………………………………………………………………..……...…10

Methodology……………………………………………………………………………..10

Definition of terms…………………………………………………………………....….11

Chapter II: Literature Review………………………………………………………………..….13

Computer-based Assessment, Past, Present and Future………………………………….13

Transition to Computer-Based Assessment…..………………………………………….16

Benefits and Risks of Computer-based Assessments…………………...……………….17

Computer-based Assessment Linked to Performance………………………………...…19

Chapter III: Methodology………………………………………………………………….……21

Subject Selection and Description………………………………………...……………..21

Data Collection Procedures……………………………………………………..………..21

Data Analysis………………………………………………………………….…………22

Limitations…………………………………….…………………………………………23

Chapter IV: Results………………………………………………………………………….…...24

Staff Results…………………………………………………….………………………..24

Student Results……………………………………………….………………………….37

6

Chapter V: Discussion………………………………………………………………...…………45

Conclusions………………………………………………………………………………46

Recommendations………………………………………….…………………………….47

References………………………………………………………………………………………..48

Appendix A: Teacher Survey……………………………….……………………………………53

Appendix B: Student Survey……………………………….……………………………………55

Appendix C: Consent Form…………………………………….………………………………..57

Appendix D: IRB Approval Form.………………………………………………………………59

7

List of Tables …………………………………………………………………………………………………Page

Table 1: 6th grade Staff Results……………………………………..……………………………24

Table 2: 7th Grade Staff Results……………………………………..…………………………...28

Table 3: 8th Grade Staff Results…………………………………………….……………………30

Table 4: Special Education Staff Results…………………………………..…………………….33

Table 5: Total Staff Results……………………………………………………………………...35

Table 6: 6th Grade Student Results…………………………………………….………………...38

Table 7: 7th Grade Student Results……………………………………………….……………...39

Table 8: 8th Grade Student Results………………………………………………………………41

Table 9: Total Student Results…………………………………………………………………...43

8

Chapter I: Introduction

Despite almost three decades of advances in information and communications technology

(ICT) and a generation of research on cognition and new pedagogical strategies, the field of

assessment has not progressed much beyond century-old, paper-and-pencil item-based tests

whose fundamental model was developed a century ago (Clarke-Midura & Dede 2010)

.Computer-based assessments and online assessments may be wave of the future. Because

mobile learning in its infancy it is an opportune time to look at the impact that this will have on

the education of children and adult learners alike. Tools like wikispaces, Twitter, Xam and

TCExam give educators the option of going paperless and assessing learning outside of time

bound classes. However, what is the cost “jumping on” the technology bandwagon? Will we

have a generation of learners who have the concept of writing as typing? Will students be

required to have a Smart phone or tablet instead of paper or pencils? These are questions that

need to be asked when looking at adopting of new technology.

The widespread and increasing use of computer-based and online testing for state and district

assessments requires educational professionals to examine the considerations for students with

disabilities (Thurlow,, Lazarus, Albus, Hodgson 2010).Governments and schools are already

eliminating paper and pencils and moving toward a screen and webcam. E-Learning continues

to grow as districts across the country create virtual schools that allow learners to complete high

school requirements from the comfort of their home. And universities now offer completely

online degree options where professors may never meet their students face to face. This study

will seek to examine what effects computer-based assessment has on learning in a small school

district in Wisconsin Perspectives from both students and faculty will be reviews to understand

how computer assessments are being used, what students and faculty think about the results and

9

how these tools may benefit or not benefit students we can see the urgent push not to be left

behind in the push for technology use in the classroom and expand to the impact studies have

shown to have on students and their thoughts and feelings about the impact that computer-based

assessment has on their education as well as the educators thoughts who are being pushed to use

these tools for the supposed benefit of their students and children.

Statement of Problem

The higher score that a student receives on a test does not mean that they are necessarily

a better learner the student may have already known the material. Nor does a lower score mean

that a student does not grasp the concepts that were delivered in class.

Computer-based assessments most often focus on lower level cognitive skills such as

recalling information that requires students to give, correct or incorrect response that can be

graded effectively and efficiently. Typically a computer cannot do is give partial credit for nearly

correct answers or account for spelling errors.

The U.S. Federal government, state departments of local education and local school

districts are moving from paper and pencil standardized testing to the electronic format eliminate

materials and provide more timely feedback. However, test scores may not accurately reflect

student test scores regardless of the impact it may have on students. But, there is more that needs

to be taken into account other than just test scores. There is no getting around that this is the

direction education is heading. The playing field is not level however; special needs, poverty and

ethnic background are all aspects that need to be explored before the push can go any further.

10

Purpose of Study

The purpose of this study is to explore computer-based assessments and identify areas of

concern that students and staff might have about the implementation of computer-based

assessments.

Assumptions of the Study

It is assumed that all participants have either participated or administered a computer-

based assessment. It is also assumed that each participant has at a basic knowledge of what

Computer-based assessments are and a general knowledge of computer use. It can also be

assumed that participants will answer all questions in a honest manner befitting a person of

character.

Limitations

Limitations of this study were limited to the students of the Family Consumer Science

program and the staff of Tomah Middle school. No other students will be studied at this time. It

is not the writer objective to limit this study but rather form a conclusion from a sampling of the

student body of nearly 700 students.

Methodology

After UW-IRB approval of surveys I will have students(parents that have signed a

consent form) in two sixth grade classes, two seventh grade classes, and two eighth grade classes

complete the 11 question survey by themselves at their seats, quietly. Once the survey has been

completed by the student it will be turned into the researcher and it will be stored in a secure

location. The 21 question survey and letter of consent relevant to staff members will be emailed

11

to each of the staff members asking them to participate in this survey. The researcher will ask

that all surveys are completed and turned in within a four day period. The researcher will send a

reminder that I would like them to complete the survey print off and return the survey to the

researchers mailbox in a timely fashion. The survey consists of 21 statements to which

respondents can reply, strongly agree, agree; neither agree or disagree, disagree and strongly

disagree. This survey’s questions will broad a ranges of answers in hope to receive an accurate

portrayal of the staffs perceptions about computer-based assessments.

Definition of Terms

Computer-based assessment (CBA)- describes one potential source of (diagnostic)

assessment input. It also refers to the conceptualization and administration of tests using

computers with software applications either commercial or open source.(Scheuerman, Periera,

2007).

Information and Communications Technology (ICT) - ICT is the term that is currently

favored by most businesses and educational institutions. The "C" reflects the important role that

computers now play in communications, e.g. by email, the Web, by satellite and cellphone

(mobile phone). We always insist on the "s" at the end of communications, which is a term that

predates computer technology and was originally associated with Morse code, radio, etc. and

often abbreviated to comms. (Davies 2011).

Virtual Environment - a computer-generated, three-dimensional representation of a

setting in which the user of the technology perceives themselves to be and within which

interaction takes place; also called virtual landscape, virtual space, virtual world

12

Electronic assessment (e-assessment) - e-Assessment is the end-to-end electronic

assessment processes where ICT is used for the presentation for assessment activity, and the

recording of responses. This includes the end-to-end assessment process from the perspective of

learners, tutors, learning establishments, awarding bodies and regulators, and the general public.

(Joint Information Systems Committee 2011).

Multi user virtual environment-a computer-generated, three-dimensional representation

of a setting in which the user of the technology perceives themselves to be and within which

interaction takes place; also called virtual landscape, virtual space, virtual world where more

than one user is liked and can interact with other users

13

Chapter II: Literature Review

This chapter the paper will review literature that focuses on computer-based assessment.

The first section will define Computer-based assessment, explore its past and what might be the

future use. The following sections will focus on implementation and how student achievements

are being affected by this form of assessment. The final section will identify best practices in

implementing electronic assessments

Computer-based Assessment, Past, Present and Future

According to Thurlow Lazarus, Albus, and Hodgson, (2010), authors of Computer-based

Testing: Practices and Considerations computer-based testing (CBT) has emerged as one of the

recent “innovative” approaches to assessments most pursued by states. CBT is lauded as the

answer to having cheaper and speedier test delivery for state and district-wide assessments

(2010). Budgetary issues have forced states and districts to look at cost effective assessments.

Computer labs and laptop programs are not being used for just word processing and Power Point

presentations anymore. In an effort to use resources more effectively school districts such as

Tomah Area School District require their teachers create common assessments for electronic

delivery to students via an in house software program called Xam. CBT has been used by

business and industries to deliver electronic training for years. Wal-Mart uses their Computer-

based Learning (CBL) to deliver training and testing to its employees on an annually basis for

the last 12 years or more. Thurlow, Lazarus, Albus, and Hodgson (2010) identify that since the

early 2000s CBT seems to have advantages over paper and pencil testing, both for states that run

the assessment programs and for the students who participate in them. More efficient delivery

and grading, being cost effective and using less resources are all advantages of computer-based

14

testing. These advantages are recognized by the U.S. Department of Education, which in one of

its major initiatives (Race to the Top Assessment Program), encouraged the development of

CBT.

Currently the major limitation to CBT is financial. Schools are unable to provide the

latest version of tablets and smart phones(Education Commission of the States 2010). Becker

(2006) questioned “digital equity” in computer access, computer use, and state-level technology

policies. Data from the National Assessment of Educational Progress to examine digital equity,

and reported that “students in rural schools or schools with high percentages of African

American students were likely to have less access to computers” (Becker, 2006 p. 1). The

availability of the technology needed greatly changes when you move away from wealthier areas

and move towards rural poverty stricken areas where there is not a computer or internet access in

every home

What does the future holds for Computer-based Assessments? Thirty years ago if

someone said that you would be able to sit on your couch at home and get a college degree

people would have laughed. If someone said that it was going to be a common occurrence for

people to have a video conference call with anyone anywhere in the world they might be

considered to be strange. And finally if someone suggested to educators that keyboarding would

force penmanship to take a backseat they would scoff at the idea. Wells and Lewis (2006)

reported that in 2005, 94 % of the nation’s public schools had internet access with 97% reporting

broadband high speed internet. In 2005, the ratio of students to instructional computers with

Internet access in public schools was 3.8 to 1, a decrease from the 12.1 to 1 ratio in 1998, when it

was first measured (Wells & Lewis, 2006 p. 6) Ipads, smartphones, and cellular carriers are

making information increasingly easier to access. Sophisticated educational media, such as

15

single-user and multi-user virtual environments, extend the nature of the performance challenges

presented and the knowledge and cognitive processes assessed. Single-user and multi-user virtual

environments (MUVEs) provide students with an “Alice in Wonderland” experience: Students

have a virtual representation of themselves in the world, called an “avatar,” that one may think of

as a digital puppet. This avatar enters the “looking glass” (monitor screen) to access a 3-D virtual

world. These simulated contexts provide rich environments in which participants interact with

digital objects and tools, such as historical photographs or virtual microscopes. Moreover, this

interface facilitates new ways of communication between students and computer-based agents

using media such as text chat and virtual gestures (Clarke, Dede, & Dieterle, 2008) The

improvement in technology allows itself to be more usable to the students of today. This allows

education to become non-place bound and students can tour to places that some may never have

the opportunity to go or share a science experiment with someone who happens to be on the

other side of the world.

Online student enrollment in the last few years has been growing at around 20% or more

and schools expect this growth to accelerate In fact, at the time, for-profit institutions expected

growth rates greater than 40% (Allen & Seaman, 2004, p. 1, 18). Certainly, the present culture of

convenience has encouraged the development of this medium that commonly promises “anytime,

anywhere” instruction. Given these factors, it appears that the increasing demand for college

level education may have found its match in asynchronous online instruction (Miner & Topping,

2001, p. 7).

16

Transition to Computer-Based Assessments

According to Scheuermann and Björnsson (2009) technological innovation and new

requirements posed by the global economy are affecting the performance of educational systems.

Societal and structural changes urge educational reforms in many countries where given

traditional education seems to not meet the needs of educational institutions and individuals.

Advances achieved with integration of educational technology into teaching and learning and

new pedagogical approaches enhance the capacities to update to new The Challenge educational

policy makers is to ensure a good match of increasing potentials with skills needed by modern

society. Research findings revealed the necessity to incorporate different learning styles into the

educational assessment in order for a more accurate and meaningful outcome to be achieved

(Ayre & Nafalski, 2000).

There are test items (“semi-innovative”) that could in principle be conducted on paper.

They test content that could be tested on using paper and pencil but use techniques that are only

possible on screen such as examining a real life picture of a cell or examining a piece of artwork

that is in a museum. (Gomersall, 2004) Huang, He, and Davidson-Shivers(2011)state that recent

progresses which ones in computer and Web-based technology have advanced the state of the art

and has transformed from traditional paper-and-pencil format to computerized and web-based

format. For instance, Baker and Mayer (1999) analyzed the essential components of the

computer-based assessment for students’ problem solving capability, which included problem

translation and integration as well as solution planning and execution. Web-based systems offer

many novel features that cannot be implemented in the paper-based systems, such as real-time

data collection, management and analysis capability; individualized learning and teaching

assessment; and distributed and interactive assessment. Finally, increased student numbers and

17

decreased staff resources expedite the need to develop electronic learning and assessment

methods for students and instructors (2011). Few people will object to the conclusion that Randy

Bennett from Educational Testing Service formulated in his May 2002 paper ‘Inexorable and

Inevitable: The continuing Story of Technology and Assessment’:” the incorporation of

technology into assessment is inevitable because as technology becomes intertwined with what

and how students learn.”(p.14)

Benefits and Risks of Computer-based Assessments

There are many benefits to Computer-based testing such as immediate results, the ability

to build in accommodations for those students who have special needs and efficiency. This lends

us to think, why haven’t we exploited this resource earlier? What is it that researchers are

missing when they examine the world of CBA? Shavelson et al.(1999) compared computer-

simulated performance assessments to paper-based performance assessments Findings suggested

that hands-on and virtual investigations were not tapping the same knowledge as paper-based

assessments: students were using lower function parts of their brains to complete tasks than with

paper-based assessments, that prior knowledge and experience influence how students solve the

problem (Shavelson, Baxter, & Pine, 1991), and that the volatility of student performance limits

the exchangeability of any methods used for delivering tasks because students are unpredictable

and it is difficult to identify how students will perform on different forms of assessments (direct

observation, notebook, computer simulation, paper-and-pencil methods, etc.) (Shavelson et al.,

1999).

In England the Qualifications and Curriculum Development Agency (2005) published

their strategic decision to promote e-assessment by stating: “E-assessment can provide timely

18

feedback to inform future teaching and learning and ‘when ready’ assessments give learners

greater ownership of their learning (p.19). With e-assessments it is possible to test areas of

knowledge, skills and understanding that would be impossible using pen-and-paper-based

testing. van Lent(2007) identified that the educational benefits anticipate the closing of the gap

between assessment and learning. More specifically, timely feedback, when-ready testing

facilitated by increased access and more realistic/appropriate tasks. van Lent (2007) goes on to

write that from the Educational Testing Institute, Reykjavik, Iceland Julius Björnsson (2007)

identified that the advantages for Iceland compulsory education to transition their national tests

to a computer adaptive format included shorter testing through adaptive tests allow for better

test-student fit, quick results, better measurement of the extremes on both ends of the scale, and

less stress are advantages as well. Computer-based assessments can be more easily designed to

meet the needs of special populations, including students with disabilities and those from diverse

linguistic backgrounds (Gamire & Pearson, 2006).

Electronic delivery is also less expensive than printing and mailing large quantities of

testing materials. In addition, errors found in test booklets or answer sheets can be quickly and

easily corrected, instead of reprinting and reshipping testing materials at considerable expense

(Bridgeman, 2009; van Lent, 2009; Bennett, 2003; Choi & Tinkler, 2002).

The information Capsule a research service for the Miami-Dade County Public Schools

(2010) noted efforts to computerize assessments have been hindered by a number of

methodological and technological challenges. There are significant start-up costs for assessment

systems that have previously been implemented only in paper and pencil format, including

hardware, software, and network purchases, connectivity, item banking, staff training, and

technical support (Education Commission of the States, 2010; Kikis-Papadakis & Kollias, 2009;

19

Kozma, 2009; Kyllonen, 2009; Lee, 2009; Gamire & Pearson, 2006; Bennett, 2003). Computer-

based assessments can lead to equity issues if some students have more access to computers and

greater computer literacy skills than others. Research suggests that students with more computer

skills perform at higher levels on computer-based tests than students with lower levels of

computer skills (Csapó et al., 2010; Education Commission of the States, 2010; Thompson &

Weiss, 2009; Gamire & Pearson, 2006; Paek, 2005; Poggio et al., 2005).

Computer-based Assessment Linked to Performance

Learning style is one of the important factors that affect personal academic competence

(Kolb 1984). Butler (1987) argued that learning style shows a natural method, which is the

easiest and most effective, and is used by the learner to realize the self, the environment, and

relation between self and environment. Not every student has the ability to succeed when placed

in front of a computer and asked to perform a task. Ricketts and Wilkes (2001) suggest that the

performance of students needs to be carefully monitored to ensure that any changes in delivery

are seen to be fair and consistent and that the use of computer-based assessment is generally

acceptable to students, who particularly appreciate the speed of marking and feedback. However,

the acceptability to students is highly influenced by the way in which questions are presented.

Thus, presentations which require scrolling are less acceptable than those in which questions are

presented one at a time.

Wang et al. (2006) states that formative assessment refers to those activities that are used

to help students learn. These types of activities include short tests and quizzes, question and

answer in the lesson, assignments, homework, and so on. Bransford et al. (2000) suggested that

the learning environment must consider centralization in assessment, in particular to value

20

formative assessment. The authors further noted that formative assessment designs should be

able to engage student attention and engender student commitment to self-evaluation and

enhancing learning effectiveness and that many Web-based formative tests provide useful

strategies for enhancing student learning by providing a final score and immediate feedback to

students(Wang et al.2010). This feedback includes the correct answer of each test item, and a

‘more information’ link to a further discussion of the question and related learning materials to

the topic. Higgins, Russell, and Hoffman (2005) found that on average, students who were

categorized as high home computer users scored lower on a reading comprehension test while

students who were categorized as medium level home computer users scored highest.

Higgins, Russell, and Hoffman (2005) concluded that scrolling negatively affects

students with low computer skills and knowledge, but that more research would need to be done

to get a more vivid picture of the impact of different types of computer-based assessments.

It is not just that computer-based assessments may affect the way that students learn but it

is also the type used that also has an effect. Whether students are able to see whole passages or

the assessment requires scrolling will affect their performance. This information is not however

readily available and the situation needs to be carefully monitored so that it is possible to identify

any changes in performance due to changes in delivery.

21

Chapter III: Methodology

The purpose of this study is to examine computer-based assessments and how it affects

student learning by surveying middle school students. And examine the perceptions that teachers

have about computer-based assessments. Surveys were given to a sampling of students in grades

six, seven and eight enrolled in a Family Consumer Sciences class in addition a more in-depth

survey was given to the staff members of Tomah Middle School. This chapter presents the

method used for collecting data, a description of the subjects.

Subject Selection and Description

All of the participants are students enrolled at Tomah Middle School and have completed

at least one summative assessment using the on-line Xam testing program. All participants are

currently in grade six, seven, or eight and presently enrolled in a Family Consumer Sciences

class. Participants must have returned a signed permission slip that was sent home explaining

the details of the survey and the research project to complete the survey. Adult participants all

currently employed by the Tomah Area School district and presently employed at Tomah Middle

School as educators or administration.

Data Collection Procedures

Qualitative data will be collected through the use of two surveys, one for students, and

one for faculty members. The groups will consist of 33 sixth grade students, 34 seventh grade

students, 28 eighth grade students, and 50 faculty members.

Start student subjects received a verbal description of the study before receiving a written

letter and the survey. Students will be asked to take the eleven question survey with the written

consent form home and have it signed by their parents and completed by the student at home.

22

They will be asked to return this the following day. This survey will then be collected and

processed by the author.

Adult subjects will be given a 21 question survey asking their beliefs about computer-

based assessments. They will be asked to complete and return the survey within a two day time

period to the author for processing. Staff members were reminded via email to complete survey

and return.

Data Analysis

Data was compiled and analyzed according to grade level and overall participation of the

two surveys. Of the fifty surveys given to staff members of Tomah Middle School 31 were

returned and of the 95 handed out to students, 34 were returned. The quantitative data was

inputed in a Microsoft Excel documentby the researcher. Data was collected by processing each

of the returned surveys and tallying their response under the appropriate area. Findings were then

placed within a Microsoft word document in table form for ease of use. Entries into both the

Microsoft Excel and Word Document were meticulously checked to maintain the integrity of this

research.

The data was then reviewed to find any areas of concern or missed questions that may

have occurred. To accurately capture responses the author compiled responses to create a

summary document by grade level and as a total overview of data collected. Results of collected

data are presented in detail in Chapter Four.

23

Limitations

Using a survey to sample the population has its limitation. Some survey may not be

returned because the survey was lost or misplaced. For one there is not a 100 percent return rate

when working with minors. Another limitation of this study may be that sampling of the

population may not reflect the views of the total population. Finally, not everyone may fill out

the survey and write down responses without reading the question.

24

Chapter IV: Results

This chapter will examine the results from both staff and student survey administered to

both staff and students in detail. The researcher gathered data through two written surveys one

was distributed to 50 staff members and the other was given to 95middle school students at

Tomah Middle School. Of the student surveys 35% were returned and 64% of the staff surveys

were returned

Staff Results

Results from the survey found the staff indifferent. Staff members have been requested

by school administration that students take at least one computer-based assessment in their

classrooms. The results(table 1) were broken into four categories, 6th grade teachers, 7th grade

teachers, 8th grade teachers, and special education teachers, and the overall results of the staff

members who participated. The results are displayed by grade level, special education, and

overall results. Numbers shown are the number of respondents that either strongly agreed,

agreed, neither agreed or disagreed, disagreed or strongly disagreed.

Table 1

6th Grade Teacher Computer-Based Assessment Survey Results

Grade/Position/ Title: 6th grade teacher totals

Strongly

Agree

Agree Neither

Agree or

Disagree

Disagree Strongly

Disagree

I give/take more than six different computer 2 1 0 3 2

25

based assessments throughout the year

Computer based testing improves assessment

scores

0 2 3 2 1

Computer based assessments accurately

measure what I have learned/taught

0 4 1 2 1

Computer based assessments save time 3 3 1 0 1

I have given/taken a computer based

assessment from home

0 0 2 1 5

Computer based assessments are easier to

complete than traditional paper based

assessments

0 4 2 1 1

Computer based assessments are the future of

how students will be assessed

2 3 2 1 0

Computer based testing would be improved by

allowing students to complete from home

0 0 4 2 2

Web navigation should be allowed during

computer based assessments

0 1 2 2 3

It is easy to log in and begin computer based

assessment

1 4 1 1 0

Computer based assessments fairly measure

what is learned i.e. spelling errors, incomplete

0 0 1 5 2

26

responses, etc.

Computer based assessments is relevant to all

content areas

0 0 2 2 4

Instructional time is lost because of lack of

technological resources and computer space

2 3 3 0 0

Computer based assessments save school

districts money

0 1 5 2 0

The type of questions available( multiple

choice, true/false, etc.) fit the needs of the

content area

0 2 0 4 2

The format of the testing program used is user

friendly

0 4 2 2 0

The data collected from computer based

assessments clear and easy to understand and

use

0 7 1 0 0

The assessment program being used shares

results with grading program easily

1 6 1 0 0

The information is easily modified and

understood by students with special needs

0 1 1 4 2

27

The sixth grade teachers had a higher tendency to disagree with the statements being

made. One reasons staff might disagree with the statements for this is because students coming

from the elementary level are not as familiar with the program that the district uses to assess

students electronically. Students must take time to acclimate themselves with logging in and

getting their accounts up and running. Once students understand the expectations set they begin

to be able to use the software with ease. One interesting item was that most teachers believe that

computer based assessment saved time by eliminating the need to make paper copies and the

ability to have instant feedback given to students the time spent correcting papers is eliminated

as stated in chapter two this is a benefit viewed by other researchers. With the program being

used there is a question bank where you can potentially store literally thousands of questions and

change them in and out with the click of a mouse.

Do sixth grade teachers feel that the ease of taking a CBA improves test scores? There is

little agreement, two felt that it helped improve scores, three had no opinion either way and three

believed that computer-based testing did not help to improve scores.

I have a computer at home 6 1 0 0 1

I have internet access at home 6 1 0 0 1

Computer based assessments favors those

students with computers at home

0 1 5 2 0

28

Table 2 identifies the results of seventh grade teachers who participated in the survey.

Eight of ten seventh grade teachers completed the survey

Table 2

7th Grade Teacher Computer-Based Assessment Survey Results

Grade/Position/ Title: 7th grade teacher totals

Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree

I give/take more than six different computer based assessments throughout the year

2 2 0 2 4

Computer based testing improves assessment scores

0 1 7 2 0

Computer based assessments accurately measure what I have learned/taught

0 4 4 2 0


I have given/taken a computer based assessment from home

3 1 2 2 2

Computer based assessments are easier to complete than traditional paper based assessments

2 1 5 1 1

Computer based assessments are the future of how students will be assessed

3 5 2 0 0

Computer based testing would be improved by allowing students to complete from home

0 1 1 6 3

Web navigation should be allowed during computer based assessments

0 1 2 2 4

It is easy to log in and begin computer based assessment

3 4 2 0 1

Computer based assessments fairly measure what is learned i.e. spelling errors, incomplete

0 2 4 3 1

29

responses, etc.

Computer based assessments is relevant to all content areas

0 1 0 8 1

Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree

Instructional time is lost because of lack of technological resources and computer space

1 4 1 2 0

Computer based assessments save school districts money

1 4 3 0 0

The type of questions available( multiple choice, true/false, etc.) fit the needs of the content area

0 4 0 4 0

The format of the testing program used is user friendly

0 5 1 2 0

The data collected from computer based assessments clear and easy to understand and use

1 3 3 1 0

The assessment program being used shares results with grading program easily

1 4 1 2 0

The information is easily modified and understood by students with special needs

0 2 3 4 0



Computer based assessments favors those students with computers at home

3 4 2 0

Seventh grade teachers were more likely to agree with the statement. Most seventh grade

teachers agreed that computer-based assessments were a way to save time as well. One area of

30

disagreements focused on the future of assessmen.t Seventh grade teachers felt slightly strongly

that there may one day be a possibility of doing away with the paper and pencil.

The largest topic of disagreement was whether or not computer-based testing saved

school districts money. The majority at both grade levels were indifferent but more seventh

grade teachers agreed that yes the ability to use an electronic assessment did in fact save money.

Table 3 identifies results from the eighth grade staff members who participated in the

survey.

Table 3

8th Grade Teacher Computer-based Assessment Results

Grade/Position/Title: 8th grade teacher totals

Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree


2 1 1 2 0


0 0 4 1 1


0 5 1 0 0



0 0 1 2 3


1 0 3 1 1

31


1 4 1 0 0


0 2 1 3


1 0 2 2 1


1 3 1 1 0

Computer based assessments fairly measure what is learned i.e. spelling errors, incomplete responses, etc.

0 3 2 1 0


0 1 2 2 1

Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree


1 2 1 1 1


1 1 2 2 0


0 3 2 0 1


0 5 0 1 0


2 2 1 0 0


2 2 2 0 0

The information is easily modified and 0 2 2 2 0

32

understood by students with special needs




0 3 0 2 1

Eighth grade teachers total only six in their responses the eighth grade was the smallest

group that reported in. Their responses are also more in agreement with the statements given.

Like with the rest of the grade level teachers. The eighth grade teachers also agree that computer-

based assessments save time. However, teachers split on the issue of whether or not they save

money. By the time that students reach eighth grade they should have a firm grasp on what is

expected of them and what to expect when it comes to computer-based assessments. Eight grade

teachers agree that they accurately measure what has been taught and believe that they fairly

measure what it is that is learned regardless of incomplete responses or spelling errors. Although

information Capsule(2010) disagrees and state that scoring interactive design problems with

open-ended responses is much more difficult than developing an answer key for multiple-choice

questions.

In order for many special needs students to succeed in this format there are many

variables that need to be examined. Often times a student with special needs must have

something to manipulate otherwise that may not be able to focus. Other times students get

intimidated by computers and get nervous, their response is to get through the test as quick as

possible and just click responses until they are finished. Technologies have the potential to

overwhelm those students whose problems cause them difficulty in organizing information

(Hasselberg & Williams Glasser 2000). Other factors that need to be taken into account is that

33

what if a student needs the test read to them or if there is too much stimulation in the

environment and often times programs can be difficult to work with if a student need an

extended period of time beyond class time. One area of disagreement is whether or not tests

created for the computer can be easily modified for students who require modifications five out

of six special education teachers who responded disagreed that they could be easily modified and

understood by students with special needs. Six agreed though that they are easier to complete

than traditional paper based tests. Special education teachers disagreed that computer-based tests

accurately measured what they had taught to students.

Table 4

Special Education Computer-Based Assessment Survey Results

Grade/Position/ Title: Special Education

Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree


0 1 2 1 1


1 0 3 2 0


0 1 2 2 1



0 1 1 1 2


1 2 2 1 0

Computer based assessments are the future of 1 4 1 1

34

how students will be assessed


0 1 1 3 1


0 1 1 3 1


0 4 1 1 0


0 0 1 3 2


0 3 2 0 1

Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree


1 2 1 2 0


0 1 3 1 1


0 3 0 3 0


0 0 4 2 0


0 4 2 0 0


1 1 4 0 0


0 0 1 1 4

35




2 2 1 1 0

To gain a better understanding of the school climate the total staff participants have been tallied

(table 5).

Table 5

Total Staff Computer-Based Assessment Survey Results

Totals

Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree


6 5 3 8 7


1 3 14 7 2


2 14 8 6 2



3 2 6 6 12


4 7 12 4 3


7 16 5 3 1

36


0 5 9 12 8


1 3 7 9 8


5 15 5 5 1


0 5 8 12 5


0 5 6 12 7

Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree


5 11 6 5 1


2 7 13 5 0


0 12 2 11 3


0 14 7 7 0


3 16 7 1 0


5 13 5 2 0


0 5 7 11 6


37



2 9 10 6 1

Table 5 shows that most of the staff is in agreement about most aspects computer-based

assessments. Most agree that computer-based assessments are the future of assessment and that it

is easy to have students log on and use. Most also disagree that it is easy to modify computer-

based assessments for students with special needs.

Student Results

This section will focus on student beliefs about computer-based testing. Student were

given the letter of consent and survey to take it home They were then asked to have their parents

sign the consent form and complete the survey and return it the following Monday. Of the 95

surveys distributed 34 were returned for a return rate of 35%. Although the lower participation

rate than expected the researcher believes that the results (table 6) represent a fairly accurate

portrayal of thoughts and beliefs.

38

Table 6

6th Grade Student Computer-Based Assessments Survey Results

Grade/Position/ Title: 6th grade student

Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree


1 5 2 0 0


1 2 4 1 0


3 3 2 0 0


4 3 1 0 0


1 3 1 3 0


5 2 0 1 0


2 1 4 1 0


4 2 2 0 0


39

Though sixth grade students have limited experience with computer-based assessments the results

suggest students feel comfortable accessing computer-based assessments, and that they find the format

to be user friendly. Students also believe that computer-based assessments accurately measure what they

have learned through instruction. Sixth grade students also believe that they are easier to complete than

the traditional paper based tests.

Table 7 identifies results of seventh grade students who have completed the survey.

Table 7

7th Grade Student Computer-Based Assessment Survey Results


Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree


5 5 5 0 0


5 2 6 1 1


4 5 3 3


3 5 1 5 1


4 4 2 3 2



1 2 2 2 1

40


5 9 1 0 0


1 7 7 0 0


4 10 1 0 0




1 3 10 1 0

Seventh grade students reflect results mirror sixth grade student’s results. Although it is

interesting that there is a higher number of students in this group that believes that web navigation should

not be allowed. Web navigation would allow students to use the internet to locate answers to questions

they may not know the answers to. Web navigation during an assessment would open the doors to

dishonesty and cheating. Trenholm (2007) states that culturally speaking we are in the midst of a time

where, increasingly, the product is valued more than the process of education. Students place more value

in getting the right answer instead of the process of coming to that conclusion. Web navigation creates a

shortcut that does exactly that, gives the answer without doing the work to get the answer.

Seventh grade students also have a higher propensity to believe that those who are computer

owners have an advantage over those who do not have a computer. This contradicts the thoughts of both

the sixth and the eighth grade students. Though there was a large portion who neither agree nor disagree

the majority of those who were not stuck in the middle thought it gives an unfair advantage. This possible

reflects that those who have more experience using technology are more apt to succeed with computer-

based assessments. This is confirmed by Higgins, Russell, and Hoffman (2005)who state that in general,

41

students with higher computer literacy scores tended to score higher on the reading comprehension

assessment

Table 8 identifies results from eighth grade Family Consumer Sciences students who participated in the

survey.

Table 8

8th Grade Student Computer-Based Assessment Survey Results


Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree


6 4 1 0 0


1 5 4 1 0


1 4 4 2 0


2 2 6 1 0


2 1 5 2 1


7 2 2 0

42


0 6 2 1 2


3 3 4 1 0




0 3 2 4 2

Eighth grade students appear to consider themselves quite savvy when it comes to computer-

based assessments. Technology in nothing new to them, they find using computers to complete tasks

second nature and it is reflected in the data from Table 8. The majority of eighth grade students find that

logging in is easy and that the format is user friendly. All eighth grade participants had a computer at

home and the majority believes that having a computer does not give you an unfair advantage when it

comes to computer-based assessments.

Table 9 offers a complete look at the participants in the students group. The information shows

that students are comfortable using technology. The majority of students found that computer-based

assessments save time as well as improves the scores they receive. The data shows that a high percentage

of students who participated take more than six computer-based assessments only helping to build their

confidence in using the technology available to them.

43

Table 9

Total Student Computer-Based Assessment Results

Grade/Position/ Title: Total Student

Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree


12 14 8 0 0


7 9 14 3 1


8 12 9 5 0


9 10 8 6 1


7 8 8 8 3


17 13 1 2 1


3 14 13 2 2


11 15 7 1 0




2 8 14 7 3

44

Limitations

The study was limited by the difficulty to rely on sixth, seventh, and eighth grade students to

respond honestly and accurately. Finally, another difficulty was relying on participants to respond to each

statement in ones haste it is easy to overlook that each statements did not receive a response.

Summary

Most students were in agreement with most responses. All students agreed that they take six or

more computer-based assessments throughout the course of a school year. Most students also agree that

computer-based assessments accurately measure what it is that they have learned. Overall, students

appear to be in favor of computer based assessments. Table 9 shows that most find it easy to log in and

that they feel comfortable using computers to be assessed.

45

Chapter V: Discussion

There are many aspects of computer-based assessments that really do need to be examined before

we can make any conclusions whether or not computer-based assessments are step forward for education.

The different views from students and educators one can determine the impact that these types of

assessments have on education and data. According to the data collected from this survey 71% of teachers

surveyed believe that computer-based assessments do not raise test scores; 47% of students felt that it

does improve scores. When examining results between students at different grade levels 54% of eight

grade students believed that computer-based assessments improved test scores compared to 46% of

seventh graders and 37% of sixth graders. These results may be due to eighth grade students having more

experience with completing assessments on the computer. Computer ownership was high among students

84% of responding students responded that they have a computer at home and of that 96% reported that

they have internet access comparable to the 81% of staff members who stated they owned a computer

96% stated that they had internet access at home.

Of the sixth grade teachers 37% stated that they give more than six different CBAs per year while

seventh grade teachers reported 40% gave more than six CBAs per year. Eighth grade reported the

highest total with 50% of those who participated reported that they give six or more per year. This may be

due to the teachers feeling that student have a better grasp of computer-based assessments and what

expectations are.

When examining whether computer-based assessments are easier to complete that traditional

paper-based tests 50% of sixth grade teacher believed that computer-based tests were in fact easier to

complete than their paper-based counterpart. 25% were found to be indifferent when it came to assessing

whether or not computer-based were less difficult and 25% did not agree that they were less difficult. 7th

grade teachers replied that 30% found computer-based to be easier where half were indifferent on the

46

topic and 20% did not agree. 16% of eighth grade teacher believed that they were less difficult to

complete where half were found to be indifferent to computer based assessments being easier to complete

and 33% disagreed and did not find computer-based assessments to be easier than traditional assessments.

Lastly, half of special education surveyed believe that computer based assessments are less difficult to

complete.

Finally, how did participants find computer based assessments to affect the education of students?

50 % of sixth grade teacher believe that computer-based assessments accurately measure what they are

teaching in class where 40% of seventh grade teachers felt that computer-based assessments were an

accurate measurement and 83% of eight grade teachers felt they are an accurate measurement of what the

teacher had taught, and just 16% of special education teacher responded that they agreed that computer-

based assessments accurately measured what they had taught to their students.

Conclusions

Students are looking toward the future when it comes to assessment. 59% feel comfortable with

being able to log in and begin taking an assessment on the computer that compared to the 62% of teachers

who think it is a simple task. Both teachers and students agree that the opportunity is there to be able take

advantage of the resources that are available to us. Students do not have a problem with using a computer

to take a test most feel that it is easier to take a test on the computer where only 33% of teachers thought

it were easier. Students are moving along at a much faster rate than many teachers. As educators they tend

to get stuck in their ways and do not necessarily resist change but, are not the quickest to embrace it

either. Students can tell you how they learn and what they believe is the best way for them to be assessed

on what is being taught in class. 58% of students felt that CBA’s accurately measure what they have

learned in class. When you compare this to the 50% of teachers who feel the same way there is not a

tremendous gap to bridge. Students obviously believe that their grades are being impacted in a positive

way by using technology to assess their learning. Teachers believe that this is the direction that education

47

is heading; it is up to educators to push the envelope and strive to challenge students and provide them

with the experience they need to succeed.

Recommendations a larger sampling

It is the recommendation of this researcher that further research is needed to determine the extent

of the impact that computer-based assessment has on student learning. A study that examines assessment

scores as compared to paper-based scores should be completed on a larger scale to determine whether

there is a positive impact on student performance or not. Data should be collected and students should be

broken down into more specific categories to determine other factors that may impact assessment scores.

Statistical data should include but not limited to age, gender, race, socioeconomic status, and special

needs. Data should then be compared to similar student groups in a comparable area.

48

References

apa. (n.d.). The American Heritage® Dictionary of the English Language, Fourth Edition.

Retrieved November 20, 2011, from Dictionary.com website:

http://dictionary.reference.com/browse/apa

Allen, E., & Seaman, J. (2005, November). Growing by degrees: Online education in the

United States, 2005. A report by the Sloan Consortium.

Ayre, M., & Nafalski, A. (2000). Recognizing diverse learning styles in teaching and assessment

of electronic engineering. Proceeding of IEEE 30th Frontiers in Education Conference

(pp.18-23), October 2000, Kansas City, M.O..

Baker, E. L., & Mayer, R. E. (1999). Computer-based assessment of problem solving. Computers

in Human Behavior, 15(9), 269-282.

Bennett, Randy Elliot (2002) Inexorable and Inevitable: The Continuing Story of Technology and

Assessment Report Number: RM-02-03, available from

http://www.ets.org/research/researcher/RM-02-03.html

Bennett, R.E. (2003). Online Assessment and the Comparability of Score Meaning. Educational

Testing Service, Princeton, NJ. Retrieved from

http://www.ets.org/Media/Research/pdf/RM-03-05-Bennett.pdf.

Bransford J., Brown A. & Cocking R. (2000) How People Learn: Mind, Brain, Experience and

School, Expanded Edition. National Academy Press, Washington, DC.

Butler K. (1987) Learning and Teaching style in Theory and Practice. The Learner’s Dimension,

Columbia, CT.

Clarke, J., Dede, C., & Dieterle, E. (2008). Emerging technologies for collaborative, mediated,

http://dictionary.reference.com/browse/apa

http://www.ets.org/research/researcher/RM-02-03.html

http://www.ets.org/Media/Research/pdf/RM-03-05-Bennett.pdf

49

immersive learning. In J. Voogt & G. Knezek (Eds.), The international handbook of

technology in primary and secondary education (pp. 901–910). New York: Springer-

Verlag.

Clarke-Midura, J., & Dede, C. (2010). Assessment, technology, and change. Journal of Research

on Technology in Education, 42(3), 309–328.

Csapó, B., Ainley, J., Bennett, R., Latour, T., & Law, N. (2010). Draft White Paper 3:

Technological Issues for Computer-Based Assessment. Assessment and Teaching of 21st

Century Skills, The University of Melbourne, Australia. Retrieved from

http://www.atc21s.org/GetAssets.axd?FilePath=/Assets/Files/dc7c5be7-0b3a-4b7d-8408-

cc610800cc76.pdf

Education Commission of the States. (2010). Assessment: Computer-Based. Retrieved from

http://www.ecs.org/html/issue.asp?issueID=12&subIssueID=76

Gamire, E., & Pearson, G. (Eds.). (2006). Tech Tally: Approaches to Assessing Technological

Literacy.Washington, DC: The National Academies Press.

Davies, G. (undefined). Glossery of ITC Terminology. In ICT4LT. Retrieved November 20,

2011, from http://www.ict4lt.org/en/en_glossary.htm#GlossI.

Hasselbring, T.S., & Glaser, C.H. (2000). Use of computer technology to help students with

special needs. Future of Children, 10(2), 102-22.

Higgins, J., Russell, M., & Hoffmann, T. (2005). Examining the effect of computer-based

passage presentation on reading test performance. Journal of Technology, Learning,

and Assessment, 3(4). Available from http://www.jtla.org

Hingel, A. (2009). The Transition to Computer-Based Assessment. Luxembourg: Luxembourg:

Office for Official Publications of the European Communities. Retrieved from

www.eurosfaire.prd.fr/7pc/bibliotheque/consulter.php?id=1213

http://www.atc21s.org/GetAssets.axd?FilePath=/Assets/Files/dc7c5be7-0b3a-4b7d-8408-cc610800cc76.pdf

http://www.atc21s.org/GetAssets.axd?FilePath=/Assets/Files/dc7c5be7-0b3a-4b7d-8408-cc610800cc76.pdf

http://www.ecs.org/html/issue.asp?issueID=12&subIssueID=76

http://www.jtla.org/

http://www.eurosfaire.prd.fr/7pc/bibliotheque/consulter.php?id=1213

50

Huang, J., He, L., & Davidson-Shivers, G. V. (2011). Education assessment via a web-based

intelligent system. (5 ed., Vol. 8, pp. 666-674). Retrieved from

http://eric.ed.gov/PDFS/ED520693.pdf

Joint Information Systems Committee (2007) Student Expectations Study,

www.jisc.ac.uk/search.aspx?keywords=student%20expectations%20study&filter=s.

Kolb D. (1984) Experiential Learning: Experience as the source of learning and development.

Prentice-Hall Inc., New Jersey.

Kikis-Papadakis, K., & Kollias, A. (2009). Reflections on Paper-and-Pencil Tests to

eAssessments: Narrow and Broadband Paths to 21st Century Challenges. In F.

Scheuermann & J. Bjórnsson (Eds.). The Transition to Computer-Based Assessment: New

Approaches to Skills Assessment and Implications for Large-Scale Testing. Luxembourg:

Kozma, R. (2009). Transforming Education: Assessing and Teaching 21st Century Skills.

Assessment Call to Action. In F. Scheuermann & J. Bjórnsson (Eds.). The Transition to

Computer-Based Assessment: New Approaches to Skills Assessment and Implications for

Large-Scale Testing. Luxembourg: Office for Official Publications of the European

Communities.Office for Official Publications of the European Communities.

Lee, M. (2009). CBAS in Korea: Experiences, Results and Challenges. In F. Scheuermann & J.

Bjórnsson (Eds.). The Transition to Computer-Based Assessment: New Approaches to

Skills Assessment and Implications for Large-Scale Testing. Luxembourg: Office for

Official Publications of the European Communities.

Paek, P. (2005). Recent Trends in Comparability Studies. Pearson Educational Measurement

Research Report 05-05. Retrieved from

http://www.pemsolutions.com/downloads/research/TrendsCompStudiesrr0505.pdf.

Poggio, J., Glasnapp, D.R., Yang, X., & Poggio, A.J. (2005). A Comparative Evaluation of Score

Results from Computerized and Paper & Pencil Mathematics Testing in a Large Scale

http://eric.ed.gov/PDFS/ED520693.pdf

http://www.pemsolutions.com/downloads/research/TrendsCompStudiesrr0505.pdf

51

State Assessment Program. The Journal of Technology, Learning, and Assessment, 3(6).

Retrieved from

http://escholarship.bc.edu/cgi/viewcontent.cgi?article=1057&context=jtla.

Gomersall, B. (2005) Practical implementation of e-testing on a large scale, and implications for

future e-assessment and e-learning [PDF document]. Retrieved from

http://www.btl.com/pdfs/Educa%202005%20-%20Bob%20Gomersall.pdf

QCA Factsheet (2005) QCA leading the way in e-assessment, Retrieved from

http://www.qca.org.uk/libraryAssets/media/6929_factsheet_e-assessment.pdf

Rickets, C. & Wilks, S. J. (2001) Is computer-based assessment good for students? , in: M.

DANSON & C. EABRY (Eds) Proceedings of the Fifth International Computer

Assisted Assessment Conference 2001 (Loughborough , Loughboroug h University) .

Saadé, R. G., & Kira, D. (2009). Computer anxiety in e-learning: The effect of computer, self-

efficacy. Journal of Information Technology Education, 8, 177-191. Retrieved from

http://www.jite.org/documents/Vol8/JITEv8p177-191Saade724.pdf

Scalise, K. and Gifford, B. (2006). Computer-based assessment in e-learning: a framework for

constructing ‘intermediate constraint’ questions and tasks for technology platforms. J.

Technol. Learn. Assess., 4(6) (available from http://www.jtla. org).

Shavelson, R. J., Ruiz-Primo, M. A., & Wiley, E. W. (1999). Note on sources of sampling

variability in science performance assessments. Journal of Educational Measurement, 36,

61–71.

Thompson, N.A., & Weiss, D.J. (2009). Computerized and adaptive testing in educational

assessment. In F. Scheuermann & J. Bjórnsson (Eds.). The Transition to Computer-Based

Assessment: New Approaches to Skills Assessment and Implications for Large-Scale

Testing. Luxembourg: Office for Official Publications of the European Communities.

http://escholarship.bc.edu/cgi/viewcontent.cgi?article=1057&context=jtla

http://www.btl.com/pdfs/Educa%202005%20-%20Bob%20Gomersall.pdf

http://www.qca.org.uk/libraryAssets/media/6929_factsheet_e-assessment.pdf

http://www.jite.org/documents/Vol8/JITEv8p177-191Saade724.pdf

52

Thurlow, M., Lazarus, S. S., Albus, D., & Hodgson, J. (2010). Computer-based testing:

Practices and considerations (Synthesis Report No. 78). Minneapolis: University of

Minnesota, National Center on Educational Outcomes.

Trenholm, S. (2007). A review of cheating in fully asynchronous online courses: A math

or fact-based course perspective. Journal of Educational Technology Systems, 35(3),

281–300.

Van Lent, G. (2009). Risks and benefits of CBT versus PBT in high-Stakes testing: Introducing

key concerns and decision making aspects for educational authorities. In F. Scheuermann

& J. Bjórnsson (Eds.). The Transition to Computer-Based Assessment: New Approaches

to Skills Assessment and Implications for Large-Scale Testing.

Wang, K.H., Wang, T.H., Wang, W.L. and Huang, S.C. (2006). Learning styles and format

assessment strategy: enhancing student achievement in Web-based learning: Journal of

Computer Assisted Learning, 22, 207-217.

Wells, J., and Lewis, L. (2006). Internet Access in U.S. Public Schools and Classrooms: 1994–

2005 (NCES 2007–020). National Center for Education Statistics, Institute of Education

Sciences, U.S. Department of Education. Washington, DC.

53

Appendix A

Staff Member Survey

Grade/Position/ Title_____________________

Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree




Computer based assessments save time









54

Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree








I have a computer at home

I have internet access at home


.

55

Appendix B

Name_________________________________

Grade/Position/ Title_____________________

Strongly

Agree

Agree Neither

Agree or Disagree

Disagree Strongly

Disagree









I have a computer at home

I have internet access at home

56


I I

57

Appendix C

Consent to Participate in UW-Stout Approved Research

Title: Computer Based Assessment and Its Effects on Students Learning

Investigator: Research Sponsor: John Mulvaney Dianne Klemme Tomah Middle School 120 Heritage Hall 612 Hollister Ave Menomonie WI, 54751 Tomah WI, 54660 715-232-2546 608-374-7896 Description: Each person is asked to complete a short survey of either eleven questions for students or 22 questions for staff members that asks their feelings about computer based assessments and how it relates to their education or curriculum. The information gathered will be used to get a general idea of the thoughts about computer based assessments and how it affects student learning. Risks and Benefits: There are no risks associated with this survey all information will be kept anonymous and there will be absolutely no repercussions for ones responses. The benefits however are that by having more respondents I will have a larger pool of information that will allow me to make better conclusions based on the higher amount of data collected. Special Populations: This survey is being distributed and asked to be completed by all Mr. Mulvaney’ s classes at Tomah Middle School as well as the faculty. Time Commitment and Payment: The time commitment for completing this survey by staff and students should not exceed fifteen minutes and there will be no compensation awarded to those who complete this survey. Confidentiality: Your name or the name of your child will not appear anywhere on any of the documents We do not believe that you can be identified from any of this information. This informed consent will not be kept with any of the other documents completed with this project. Right to Withdraw: Your participation in this study is entirely voluntary. You may choose not to participate without any adverse consequences to you. Should you choose to participate and later wish to withdraw from the study, you may discontinue your participation at this time without incurring adverse consequences. IRB Approval:

58

This study has been reviewed and approved by The University of Wisconsin-Stout's Institutional Review Board (IRB). The IRB has determined that this study meets the ethical obligations required by federal law and University policies. If you have questions or concerns regarding this study please contact the Investigator or Advisor. If you have any questions, concerns, or reports regarding your rights as a research subject, please contact the IRB Administrator. Investigator: John Mulvaney, IRB Administrator 608-374-7896 Sue Foxwell, Director, Research Services [email protected] 152 Vocational Rehabilitation Bldg. Advisor: Diane Klemme, UW-Stout 715-232-2546 Menomonie, WI 54751 [email protected] 715-232-2477 [email protected] Statement of Consent: By signing this consent form you agree to participate in the project entitled, Computer Based Assessment and Its Effects on Students Learning. _________________________________________________ Signature Date _________________________________________________ Signature of parent or guardian Date (If minors are involved)

mailto:[email protected]

59

Appendix D John Mulvaney Family and Consumer Science Department UW-Stout RE: A Grant proposal to study the Benefits of computer-based Assessments Dear John, The IRB has determined your project, "A Grant proposal to study the Benefits of computer-based Assessments” is Exempt from review by the Institutional Review Board for the Protection of Human Subjects. The project is exempt under Category 1 of the Federal Exempt Guidelines and holds for 5 years. Your project is approved from December 15, 2011, through December 14, 2016. Should you need to make modifications to your protocol or informed consent forms that do not fall within the exemption categories, you will need to reapply to the IRB for review of your modified study. If your project involved administration of a survey, please copy and paste the following message to the top of your survey form before dissemination:

If you are conducting an online survey/interview, please copy and paste the following message to the top of the form: “This research has been reviewed by the UW-Stout IRB as required by the Code of Federal Regulations Title 45 Part 46.” Informed Consent: All UW-Stout faculty, staff, and students conducting human subjects research under an approved “exempt” category are still ethically bound to follow the basic ethical principles of the Belmont Report: 1) respect for persons; 2) beneficence; and 3) justice. These three principles are best reflected in the practice of obtaining informed consent from participants. If you have questions, please contact Research Services at 715-232-1126, or [email protected], and your question will be directed to the appropriate person. I wish you well in completing your study. Sincerely,

Susan Foxwell Research Administrator and Human Protections Administrator, UW-Stout Institutional Review Board for the Protection of Human Subjects in Research (IRB)

mailto:[email protected]

computer-based assessment and the effects it has on … uses their computer-based learning (cbl) to...

Documents