thesis malyn

EFFECTIVENESS OF RATIONAL CLOZE – TEST AS INTEGRATIVE ASSESSMENT IN HISTORYAT

LAGUNA STATE POLYTECHNIC UNIVERSITYSAN PABLO CITY CAMPUS

ACADEMIC YEAR, 2012-2013

A Research PaperPresented to the

Faculty of the College of Teacher EducationLaguna State Polytechnic University

San Pablo City CampusSan Pablo City

In Partial of FulfillmentOf the Requirements in

Methods of Research for the DegreeBachelor of Secondary Education

Major in Social Studies

MARILYN ATIENZA MENDOZA

October 2012Chapter I

The Problem and Its Background

Introduction

The cloze procedure is considered by many as an 'integrative' method of

assessment, in contrast with 'discrete point' methods, The Cloze procedure deals with

several linguistic components at once, focusing more on language use, and typically

requiring the examinees to read and comprehend a substantial amount of discourse. Much

research has been devoted to the validity and the reliability of Cloze tests. Cloze tests are

reported to have moderate to high correlations with standardized tests and their subtests

such as listening comprehension and reading comprehension. The report on high

correlations between Cloze tests and International Journal of Human and Social Sciences

tests of listening comprehension, writing, reading comprehension, and oral interview. The

findings of these researchers reveal a relation between the scores on Cloze and global

language ability tests.

As for C-test, since its inception in 1981, its principles have been applied to more

than 20 languages. C-tests have been used in numerous contexts and for various purposes

.C-tests, then, figure prominently in the fields of language testing and assessment. C-

tests, like the classic Cloze test, are an operationalization principle of reduced test.

Although there are weaknesses in the reliability of cloze procedure, it is widely used in

English teaching. As Legenza and Elijah (2001) point out, many studies have been

conducted on cloze procedure concerning the construction of cloze tests, the relationship

between cloze and reading comprehension, and the usefulness of cloze as a teaching

technique. Cloze procedure has been used in English teaching in accordance to History

subject for decades since 1953. It was firstly used as an instrument for assessing the

readability of written materials for school children in the United States (Brown, 2002).

Then it was used in teaching for different purposes.

Cloze procedure is not only valuable in providing information as to how readable

the particular texts are for the students who are going to have to use them, but also

invaluable as a means of comparing different text. Cloze procedure is also used for

testing purposes. It seems that a wide range of skills like vocabulary, grammar, structure,

and reading skills are involved in the process of completing a cloze procedure. Many

researchers indicate that cloze procedure is a good test of overall English language

proficiency and in History. For example, Ahluwalia (2005) claims that cloze procedure is

an integrative, global measure of language competence. She explains that cloze tests

measure the grammar of expectancy underlying the skills of thinking, understanding,

speaking, reading, understanding and writing. For Cohen (2001 cited in Ahluwalia, 2005)

cloze procedure measures global language competence consisting of linguistic

knowledge, textual knowledge, and knowledge of the world. As it calls on examinee to

use knowledge such as vocabulary, grammar, sentence construction, text structure,

cohesion and the reader’s prior background knowledge.

Askes (2002) regards cloze procedure as one of the integrative tests (global tests)

that integrate language components into a total language event, which requires an

integrated performance from the learner in a meaningful context. Thus for Steinman

(2002), a cloze can replace the sections on test structure, written expression, vocabulary,

and reading comprehension not only in English subject as well as different areas of

learning.

The reason behind giving a Cloze test and obtaining a test score is interpreting

that score as an indicator of what a test taker knows or what he/she can do with that

knowledge. Furthermore, our interpretation of that test score forms the basis for decision

making. As such, when using a test score, we make an implicit link between test

performance and a domain of language knowledge the test taker has or something the test

taker can do with language in some language use domain beyond the test itself. In other

words, when we use test scores, we are essentially reasoning from evidence, using the

test score as the evidence for inferences or interpretations and decisions we want to make.

Yet, we cannot simply draw on test score to make inferences and decisions without

efficient justification. If we want to use a test score for a particular purpose, we must

justify it through a rationale and supporting evidence. As Bachman (2005) puts it, "We

need to demonstrate, with logical argumentation and empirical evidence, that the

intended interpretations and uses are valid." Validity in testing and assessment has

traditionally been understood to mean "discovering whether a test measures accurately

what it is intended to measure", or uncovering the "appropriateness of a given test or any

of its component parts as a measure of what it is purposed to measure".

Background of the Study

Cheng (1999) claims that the term ‘cloze procedure’ was first developed by

Wilson Taylor in 1953. He explains that the term cloze derived from the Gestalt

psychology concept of ‘closure’. It describes a tendency that humans have to complete a

familiar but not-quite-finished pattern. Ellis (2004) further explain that it refers to the

tendency of individuals to complete a pattern once they have grasped its overall

significance. Rye (1982) explains that Cloze procedure is essentially a cognitive task. The

reader has to reason and construct suggestions to fill the gap on the basis of the evidence

derived from the context the completion of meaning, based on understanding and

reasoning is a cognitive task. Brown (2002) claims that it is not difficult to get people to

take a cloze test because of the compulsive human need to fill gaps.

Clarke (2001) was the first to study cloze procedure for its effectiveness as an

instrument for determining the readability of materials in the reader’s native language in

1953. After that initial study, it was investigated for its appropriateness as a measure of

readability of L1 and L2 materials. In the 1960s, studies focused on cloze tests as a

measurement of reading comprehension in L1 and L2. During the 1970s, cloze tests

began to be used as a measurement of overall L2 proficiency (Ahluwalia, 1992:82).

Today, cloze tests are widely used in some places (such as China) and as part of some

large-scale language. What is a cloze test? A standard cloze test is a passage with blanks

of standard length replacing certain deleted words which students are required to

complete by filling in the correct words or their equivalents. During traditional cloze

testing, every fifth word is removed from a 250-500 word reading passage, and is

replaced by a standard-length blank space (Helfeldt et al, 1986:216). Usually, no word is

omitted either in the first or the last sentence of the passage. Students are required to

supply either the original word of the author or an appropriate equivalent in the blank

space. Many studies show that the reliability and the validity of cloze tests are affected by

factors like the rate of deletion, nature of the text and scoring systems, etc. ‘Deletion rate’

refers to the frequency of deleting words. As Steinman (2002:293-294) explains, there are

two options in designing a cloze test according to its deletion rate: a random cloze or a

rational cloze. A random cloze deletes every nth word consistently, so that all classes and

types of words have an equal chance of being deleted. A rational cloze is the one in

which a specific type of word is deleted according to a linguistic principle, such as nouns,

verbs, adjectives, etc.

Alderson (2000) cited by Yamashita (2003:269) clearly differentiates between

these two types of format by calling the rational cloze ‘gap-filling tests’ and confining the

term ‘cloze’ only to the random cloze. Ahluwalia (1992:83-84) states that different

deletion rates would result in either increasing the difficulty of the text or in

unpredictable differences in the difficulty. Another factor that would affect the reliability

and the validity of the cloze is the nature of the text. The nature of the text such as its

familiarity level and difficulty level would improve or hamper cloze performance

(Ahluwalia, 1992:86-88).

However, Alderson (1980, 1983 cited in Ahluwalia, 1992:86) and Yamashita

(2003:286-287) argue that cloze test performance is not directly related to the difficulty

level of the text; it involves other factors such as scoring procedures, and content

familiarity for the readers. Different scoring methods include: exact word method,

multiple-choice scoring method and contextually acceptable word method. 16‘Exact

word method’ requires the examinee to provide the original word deleted from the text

while ‘contextually acceptable word method’ allows for the words that fit the gap either

to be synonyms of the deleted.

Statement of the Problem

This study aims to know the effectiveness of cloze – test as integrative

assessment in history at Laguna State Polytechnic University, San Pablo City Campus,

Academic Year, 2012-2013.

1. What is the profile of the respondents in terms of:

1.1 Age;

1.2 Gender;

1.3 Grade point in history subject?

2. What is the level of perception of the following assessment – related variables as

perceive by the respondents:

2.1 Controllability;

2.2 Reliability and validity; and

2.3 Usability?

3. What is the mean score of the respondents in Cloze – test?

4. Is there a significant relationship between the person – related factors and their

perception of the following variables:

4.1 Controllability;

4.2 Reliability and validity; and

4.3 Usability?

5. Is there a significant relationship between the level of perception of the

respondents and their mean score in Cloze – test?

6. Is the grade point average in History is significantly related to the result of Cloze

– test?

Scope and Limitations

This study focused on the effectiveness of Cloze test in measuring performance

related to History subject.

It is limited only to twenty - nine (29) respondents of the third year students who

are enrolled in Academic Year 2011-2012 at Laguna State Polytechnic University – San

Pablo City Campus, they were randomly selected.

In this study, the researcher used instruments such as survey questionnaire

intended for students and 25 items for measuring their performance using Cloze type of

test which served as the main instrument of the study.

Significance of the Study

The study will be significant to the following:

Teachers and instructors especially for those specializing History, for it will

benefit them in a way of identifying the factors underlying proper assessment;

Students, for it will help them to improve and be competent in studying History

and provoke their interest.

Parents, to furnish and give aid for their children an interest to learn the

complexities of motivating a child in History;

And finally, for researchers, for it will help them gain satisfaction for knowing

some insights about the given problem and develop research skills to be more productive

in the field that they had chosen to belong in the future.

Theoretical framework

Cloze technique, developed by Taylor (1952), is firstly designed as a tool for

measuring readability of texts, and now it has been used widely as an exercise or test

method in foreign language teaching and testing. “It works on the principle of closure and

anticipation, in that the reader is required to reconstruct systematically mutilated passage”

(Hofman, J. E. & Habib-Allah, M., 1982:276). Actually, as early as 1971, John Oller

advocated that cloze testing was an excellent integrative test. Joseph Boyle and Peter

Falvey (1994) claims that cloze test is one variant of integrative tests, too. And they

believe that “for cloze test, it is pragmatic expectancy grammar that constitutes the trait

that reflects linguistic competence”. In addition, many researches from 1970s to 1980s on

cloze test supply people with an objective viewpoint that cloze test is a reliable and valid

test method to investigate the difficulty level of the text and the test-takers’ capability in

reading comprehension. It can be used as a part of standard integrative language

proficiency testing and a subsidiary tool for language teaching.

Cloze is originally called “cloze procedure”, whose theoretical foundation is Gestalt

psychology. Those psychologists believe that when people observe the shape of an object,

they will unconsciously fill in the broken part. Such is the case in reading process. A

sentence or a passage, deleted some words, is just like a broken picture. People will

complete it unconsciously according to their comprehensive language knowledge. The

Gestalt theory also claims that the more familiar with the picture the easier for people to

recognize it. Expounded in language learning, the case is that the higher language

proficiency the higher ability to complete the broken passage.

Besides the Gestalt theory, researchers try to use Redundancy Information theory

and Expectancy Grammar theory to explain the cloze test. In Longman Dictionary of

Language Teaching and Applied Linguistics, redundancy is defined as “the degree to

which a message contains more information than is needed for it to be understood.

Languages have built-in redundancy, which means that utterances harbor more

information than is necessary for comprehension. And 50% of normal language is said to

be redundant”. Li Xiaoju (2001) also approves that in cloze test, people use the

redundancy information of natural language to complete the broken language. Actually,

this theory is crucial for Chinese learners when learning English. The process of

accumulating language knowledge is the process of recognizing the redundancy

information in English language. However, the information is necessary in that it can

make the meaning of the text much more accurate, although it is regarded as more than is

needed. For Chinese students, the recognition of the redundancy information is helpful

for them to improve their sensitivity to the English language environment and their

ability to understand the inner relationship in language itself and the consistency of its

usage, which enhances the students’ language proficiency. Therefore, the cloze test

removes some redundancy information from the completed text to examine whether test-

takers obtain the capability to comprehend the broken passage and to fill in the blanks or

not.

Expectancy Grammar was firstly advocated by Oller in 1976. Later Carter (1998)

believes that it refers to language user’s comprehensive language ability to use syntax,

semantic and discourse knowledge. In reading process, the reader can use the ability to

predict in what way the sentence will end and to what trend the text will develop. Cloze

test investigates the very ability acquired by language learners through long-term learning

and perception, and proficient mastering of language structure as well as basic use

principle of the target language. In a broken passage, according to the context and their

language ability, students can predict the syntactic and semantic function of the deleted

word, along with the relationship between the word and the other part within the

sentence.

Conceptual Framework

The conceptual framework of this study is presented to give view of the work to

be done. The inputs or the independent variables are the profile of the respondents mainly

3rd year high school students and their perception regarding the effectiveness of a test

which composed of Controllability, Reliability and Validity and Usability.

The process is the strategy to be used such as generating data through the use of

questionnaire. The possible output or the dependent variable is the result or their

performance in History by means of using Cloze Test. It is assumed to be affected by the

independent variable.

Paradigm

Independent Variables Dependent Variable

I. Profile of the respondentsAgeGender Grade in history

II. Assessment – related factorsControllabilityReliability & ValidityUsability

Figure1. Shows the relationship between the independent variables (profile of the

respondents and the assessment – related factors) and the dependent variable or the Cloze

– test in History.

Hypotheses

1. There is no significant relationship between the person – related factors and their

perception to the assessment related variables.

2. There is no significant relationship between the level of perception of the

respondents and their mean score in Cloze – test

3. There is no significant relationship between grade point average in History and

their result of Cloze – test

Cloze – Test

Definition of Terms

For a better understanding of this study, the following terms are

operationally defined.

Cloze Test. Refers to integrative assessment in measuring higher cognitive task, deals

with several linguistic components at once, focusing more on language use, and typically

requiring the examinees to read and comprehend a substantial amount of discourse.

Age. Refers to how old or young a particular person was.

Gender. It refers to a particular person whether s/he is a male or a female.

Grade Point Average. Refers to the recent average grade of 3rd year high school students

during 2012 – 2013.

Controllability. It refers to the level of manipulation of effectiveness of Cloze type of

test.

Validity. Refers to what purports to measure in a given context.

Reliability. Refers to the consistency of Cloze test in assessing History subject.

Chapter II

Review of Related Literature and Studies

The chapters presents a selection of literature and studies that has a

bearing on present study. Most of the literature gathered comprises Cloze Test

procedure.

Related Literature

Cloze test has been changing and developing. According to different deletion and

fill-in-blank methods, it can be mainly classified to fixed-ratio cloze, rational cloze,

multiple choice cloze, C-test and banked cloze.

Fixed-ratio cloze was proposed by Oller in 2005. It is constructed by deleting words

according to a fixed pattern, either at even numbers (4, 6, 8, 10, etc.) or at odd numbers

(5, 7, 9, 11, etc.). “This procedure is intended to sample regularly various variants of

words, some of which are governed by local grammatical constraints and wide discipline

vocabularies in sorted areas of History, Mathematics, Statistics, others of which are

governed by long-range textual constraints”. The advantages of this cloze variant are

convenience and high consistency, while there are also a lot of problems such as some of

the blanks being too easy or too difficult, the broken passage sometimes arousing test-

takers’ dislike and resulting in a dilemma for testers to choose subjective or objective

scoring method, and being lack of control which possibly leads to examine the language

points the designer doesn’t want to test actually. Rational cloze can also be called open

cloze or gap-filling. The test developer control over the types of the words deleted such

as functional words, verbs or nouns, and thus the language traits measured. This selected

deletion changes the random of cloze testing into objectivity and practicality. Just like in

fixed-ratio cloze, test-takers have to fill in the blanks in rational cloze according to their

comprehension capability. “Rational cloze research and practice rests on the assumption

that different cloze items can be explicitly chosen to measure different language traits”

(Chapelle & Roberta, 1990:122). The understanding advantage of rational cloze is that

the test designer can select particular item to determine test-point. However, it is

confronted with the problem that subjective or objective scoring method, either.

In order to solve the scoring problem of fixed-ratio and rational cloze, test

developers design multiple-choice cloze. This cloze test format requires test-takers to

select the correct answer from the provided options. It offers four or five options to each

blank, while only one is the correct answer and the others are distracters. There are no

accepted answers which get rid of scoring controversies. Generally speaking, high

reliability is the specialty of multiple-choice cloze. However, the inappropriate options or

explanation of new words will give test-takers some hints, which possibly reduce the

difficulty level and the validity of test. Therefore, multiple-choice cloze demands test

developers’ enough attention and obeying some strict rules.

A cloze test (also cloze deletion test) is an exercise, test, or assessment consisting

of a portion of text with certain words removed (cloze text), where the participant is

asked to replace the missing words. Cloze tests require the ability to understand context

and vocabulary in order to identify the correct words or type of words that belong in the

deleted passages of a text. This exercise is commonly administered for the assessment of

native and second language learning and instruction.

The word cloze is derived from closure in Gestalt theory . The exercise was first

described by W.L. Taylor in 1953.

Words may be deleted from the text in question either mechanically (every nth

word) or selectively, depending on exactly what aspect it is intended to test for. The

methodology is the subject of an extensive academic literature; nonetheless, teachers

commonly devise ad hoc tests.

A Cloze test removes certain words from a sample of your text and asks users to

fill in the missing words. Your test participants must rely on the context as well as their

prior knowledge of the subject to identify the deleted words. It’s based on the Gestalt

theory of closure—where the brain tries to fill in missing pieces—and applies it to written

text.

It looks something like this:

If you want to __________ out whether your site __________ understand your content,

you __________ test it with them.

It looks a lot like a Mad Lib, doesn’t it? Instead of coming up with a sentence that

sounds funny or strange or interesting, participants must guess the exact word the author

used. While Cloze tests are uncommon in the user experience field, educators have used

http://www.madlibs.com/

http://www.andyrutledge.com/closure.php

http://www.andyrutledge.com/closure.php

http://en.wikipedia.org/wiki/Cloze_test

them for decades to assess whether a text is appropriate for their students, particularly in

English-as-an-additional-language instruction.

Here’s how to do it:

Take a sample of text—about 125-250 words or so.

Remove every fifth word, replacing it with a blank space.

Ask participants to fill in each space with the word they think was removed.

Score the answers by counting the number of correct answers and dividing that by the

total number of blanks.

A score of 60% or better indicates the text is appropriate for the audience.

Participants who score 40-60%, will have some difficulty understanding the original text.

It’s not a deal breaker, but it does mean that the audience may need some additional help

to understand your content. A score of less than 40% means that the text will frustrate

readers and should be rewritten.

It might sound farfetched, but give this method a try before you dismiss it. In a

government study on healthcare information readability, an expert panel categorized

health articles as either easy or difficult. We ran a Cloze test using those articles with

participants—who had low to average literacy skills—and found that the results reflected

the expert panel’s findings. The average score for the “easy” version was 60, indicating

the article was written at an appropriate level for these readers. The average score for the

“difficult” version was 39: too hard for this audience.

Cloze tests are simple to create, administer, and score. They give you a good idea

as to whether the content is right for the intended audience. If you use Cloze tests—either

on their own or with more traditional usability testing methods—know that it takes a lot

of cognitive effort to figure out those missing words. Aim for at least 25 blanks to get

good feedback on your text; more than 50 can be very tiring.

When to test

Test your content at any point in your site development process. As long as you

have content to test, you can test it. Need to convince your boss to budget for content

testing? Run it through a readability formula. Got content but no wireframes or visual

design? Run a Cloze test to evaluate content appropriateness. Understands the content

key to a task or workflow? Display it in context during usability testing.

What to test

You can’t test every sentence on your site, nor do you need to. Focus on tasks that

are critical to your users and your business. For example, does your help desk get calls

about things the site should communicate? Test the content to find out if and where the

site falls short.

So get to it

While usability testing watches what users do, not what they say they do, content

testing determines what users understand, not what they say they understand.

Whatever your budget, timeline, and access to users, there’s a method to test

whether your content is appropriate for the people reading it. So test! And then, either

rest assured that your content works, or get cracking on that rewrite.

A language teacher may give the following passage to students:

“ Today, I went to the ________ and bought some milk and eggs. I knew it was

going to rain, but I forgot to take my ________, and ended up getting wet on the

way ________. ”

Students would then be required to fill in the blanks with words that would best

complete the passage. Context in language and content terms is essential in most, if not

all, cloze tests. The first blank is preceded by "the"; therefore, a noun, an adjective or

an adverb must follow. However, a conjunction follows the blank; the sentence would not

be grammatically correct if anything other than a noun were in the blank. The words

"milk and eggs" are important for deciding which noun to put in the blank; "supermarket"

is a possible answer; depending on the student, however, the first blank could either

be store, supermarket, shop or market while umbrella or raincoat fit the second.

The definition of success in a given cloze test varies, depending on the broader

goals behind the exercise. Assessment may depend on whether the exercise is objective

(i.e. students are given a list of words to use in a cloze) or subjective (i.e. students are to

fill in a cloze with words that would make a given sentence grammatically correct).

“ I saw a man lay his jacket on a puddle for a woman crossing the street. I thought ”

that was very ______.

Given the above passage, students' answers may then vary depending on their

vocabulary skills and their personal opinions. However, the placement of the blank at the

end of the sentence restricts the possible words that may complete the sentence;

following an adverb and finishing the sentence, the word is most likely an adjective.

Romantic, chivalrous or gallant may, for example, occupy the blank, as well

as foolish or cheesy. Using those answers, a teacher may ask students to reflect on the

opinions drawn from the given cloze.

In addition to use in testing, cloze deletion can be used in learning, particularly

language learning, but also learning facts. This may be done manually – for example, by

covering sections of a text with paper, or highlighting sections of text with a highlighter,

than covering the line with a colored ruler in the complementary color (say, red ruler for

green highlighter), so the highlighted text disappears; this is popular in Japan, for

instance. Cloze deletion can also be used as part of spaced repetition software, and

the Super Memo application features semi-automated creation of cloze tests, particularly

as part of its incremental reading feature.

Related Studies

The Cloze test is used primarily to determine a student’s level of reading ability.

A student is first tested for reading level; in this case the Slosson oral has previously been

administered. After a student’s reading level has been determined, a reading selection is

taken from a book of like grade level. As is standard, every fifth word is removed and

replaced by a blank space, in this assignment there were 20 blanks in the selection. In this

assignment, as was suggested, a paragraph of text preceded the actual test material.

Evaluation of the test determines a student’s reading level; an independent reading level

indicates that the student could read the selected book on their own, instructional level

means that the student can read the book with assistance and frustration level means that

the book is difficult for the student to read and is likely above the student’s reading grade

level. For the purposes of this assignment, the following scale was assigned for

evaluation: the independent reading level is based as 50% or more of the blanks filled in

correctly, instructional level is 30% - 50% and frustration level is rated as 30% or less of

the blanks correctly filled in.

On Friday, October 6, 1995, I administered a Cloze test to a student

named Natalie at Stivers Middle School in Dayton, who I had administered a Slosson oral

test to the previous week. On the Slosson oral, Natalie scored in the 8th grade reading

level (8.95 to be exact). Taking her Slosson score, I set out in search of a text book

involving African American studies, an area she is very interested in. After much

searching, I secured the book African American History at the Wright State ERC and

submitted it to Fry Graph analysis (attachment 1). Taking three 100 word samples

randomly from the book I computed the approximate reading grade level of the book.

Sample 1 consisted of 7.3 sentences and 149 syllables, sample 2 had 9.2 sentences and

155 syllables and sample 3 had 8.5 sentences and 158 syllables. The average of the

samples was 8.33 sentences and 154 syllables. I plotted this point on the graph, and

fortunately it fell within the 8th grade reading level. I then constructed a Cloze test to be

administered.

The test was administered to Natalie during the 4th period in the same conference room

Mrs. King had selected for me to give Natalie the Slosson oral test the previous week. I

explained the test to Natalie and gave her the test to fill out (attachment 2). While Natalie

was taking the test, I either sat down at a nearby table or walked to a nearby

window. Natalie finished the test in about eight minutes, and I quickly compared it to my

master copy (attachment 3). I then sat down with Natalie and asked her why she had

chosen each of her selections, the correct and incorrect ones alike. I recorded her

responses, and thanked her for helping me and she returned to class.

Natalie correctly filled in 11 of the 20 blanks (55%) and placed in the independent

reading level. Below is a chart of error, indicating the correct response, the response

given, the reason the answer was given, if there was a change in grammar and if there

was a change in the meaning of the statement. Following the chart will be a more detailed

account of why each incorrect answer was given.

I asked Natalie why she chose each of her selections, and asked her if

other words would make sense in the context of the sentence. I did this for all 20

selections; including the ones she got right as well as the ones she got wrong. For

example on selection 1, which she filled in correctly, I asked her if ‘is’ would have

worked. She promptly told me that everything was in the past, so ‘is’ would be wrong.

In her first mistake on the test, Natalie substituted ‘food’ for the word

‘literature’. I asked Natalie if the word ‘literature’ would fit better, and Natalie told me

that most likely books would be together with art. On the second mistake, Natalie used

the phrase ‘brought on’ instead of ‘stimulated’. I asked her about using ‘stimulated’ and I

was promptly asked what the word meant. The third mistake made was the substitution of

the phrase ‘in many’ instead of ‘of’. When I asked Natalie about this, she read the

sentence back with ‘of’ in its place and said it didn’t sound right. On this I really had to

agree.

The fourth mistake made was the usage of ‘racism’ instead of

‘oppression’. When I asked Natalie about this I was met with the same response I got for

‘stimulated’: what does it mean? Natalie’s next mistake came when she used ‘art’ instead

of ‘society’ in the selection. Again I asked her about using society in the sentence, and

Natalie said it seemed right to her since art had been used before in the paragraph.

Natalie’s sixth error was in putting ‘something’ in the selection instead

of ‘things’. When I asked her about the possibility of using ‘things’, she simply told me

that nobody used that word that way. Again, I would have to agree with her since I have

heard few people ever say ‘things new’. The seventh mistake was the use of ‘movement’

instead of ‘Renaissance’. As I had done several times before, I asked her about her choice

and asked if ‘Renaissance’ would have worked. Natalie’s response was that it wouldn’t

work because a renaissance is something 'really old'. I could understand this reasoning,

since the only time students really hear this term is in early European history.

The eighth mistake made on the Cloze test was the use of the word

‘racist’ instead of ‘this’. I really didn’t understand this substitution, and when I asked

about it Natalie said that she couldn’t think of anything else, that she had used ‘racist’

before in the selection and ‘racist’ was the only thing she could associate with New York

in the context of the sentence. The ninth, and final, mistake Natalie made was the use of

the phrase ‘period for’ instead of ‘by’. By this time Natalie seemed tired of all this and

simply said she couldn’t think of any other word to use. She wanted to return to her class,

so I thanked her for putting up with me and my tests and let her return to the room.

From my perspective, most of Natalie’s errors on the Cloze test could

be attributed to two things: she was unfamiliar with some of the vocabulary used and the

text in some places was written in a way in which few students (or adults for that matter)

speak. So, what can be done about it? If this had been an actual textbook assignment, one

thing that could be done is to provide the students with a list of vocabulary words they do

not understand completely.

Chapter III

Research Methodology

This chapter describes how the study was conducted and the researcher used in gathering

the necessary data for the completion of the study. This includes the research design, the

population and sampling, sampling procedure, data gathering procedure, research instrument and

the statistical treatment of data.

Research Design

This study used the quasi - experimental type of research in determining the

effectiveness of Cloze Test in measuring History as a subject in secondary level since the

absence of a control group. The possible solutions obtained through a questionnaire as

well as interviews and observation made by the researcher. This study used of the third

year students in Laguna State Polytechnic University as its respondents, the main source

of data. They were selected by random order among the total number of junior level. The

instrument used was a collaboration of information based on Crystal,

David (2004). Contextual Constraints in Cloze Test. Cambridge: Cambridge University

Press. ISBN 0-521-53032-6. Some of its modifications were undertaken to adapt on the

research problem and scope of the study.

Population and Sampling

This study was conducted at Laguna State Polytechnic University, San Pablo City

Campus Academic Year 2012-2013. The researcher gets a representative for 3 rd year high

school department to suit the twenty - nine (29) respondents needed for the study.

A purposive sampling as a sampling technique was utilized.

Sampling Procedure

To determine the sample size needed for the study, the researcher used random

sampling technique. This sampling technique is unbiased since the respondent has the

same chance of being chosen and the selection of respondent has no influence of the

selection of other respondents.

Research Instrument

The main instrument used in the study is a questionnaire which was used to know

person related factors such as age, gender, and grade point in history during second

semester A.Y. 2012- 2013. And an integrative assessment in History comprises of twenty

– five (25) items using Rational Cloze Test procedure.

Statistical Treatment of Data

The data were gathered from the respondents treated and interpreted using the

statistical tools. The following statistical analysis was coded in the study:

Survey Questionnaire

Age

Legend:

14y/o below – 1

15y/o – 2

16y/o and above – 3

Gender

Legend:

Male – 1

Female – 2

Current Grade Point in History

Legend:

93 and above – 1

90 – 92 – 2

87 – 89 – 3

84 – 86 – 4

81 – 83 – 5

80 and below – 6

Cloze test

Legend:

correct – 1

incorrect - 0

Percentage distribution was utilized to describe the person related variables in

terms of, age, gender, and grade point in history.

For the assessment – related factors. The researcher utilized the mean and

standard deviation using a Likert scale, 5 being the highest, denotes ‘strongly agree’ and

1 being the lowest, denotes ‘strongly disagree’.

To find that if there is a relationship existing between the independent and

dependent variables, Pearson Product Moment of Correlation Coefficient was used to tell

how well two sets of continuous data correlate to each other at 0.05 level of significance.

Chapter IV

Presentation, Analysis, and Interpretation of Data

This chapter presents the findings of the study in illustrative tables and analysis as

well as the interpretation based from the treatment of the data.

Table 1Distribution of Respondents According to Age

Age Frequency Percent14 y/o below 8 28

15y/o 20 6916y/o and above 1 3

TOTAL 29 100

Table 1 shows the distribution of respondents according to their age, the majority of the respondents are 15y/o having twenty (20) or 69% of the total respondents.

This implies though the inability to randomize the selection because the research has lack of features of the true experiment and based on the experimental values the respondents are equipped subject for experimental condition since they undergone History as subject. As Datta (2002) mentioned Cloze tests require the ability to understand context and vocabulary in order to identify the correct words or type of words that belong in the deleted passages of a text. This exercise is commonly administered for the assessment of native and second language complex learning and instruction.

Table 2Distribution of Respondents According to Gender

Gender Frequency PercentMale 12 41.38Female 17 56.62TOTAL 29 100

Table 2 clearly shows that female outnumbered the male having a frequency of 17 or 56.62% and the male has 12 frequency or 41.38%.

This implies that more female are subjected in the experimental condition.

Table 3

Distribution of Respondents According to Grade Point in HistoryGrade point Frequency Percent93 and above 0 0

90 – 92 6 2187 – 89 10 3484 – 86 8 2881 – 83 5 17

80 and below 0 0TOTAL 29 100

Table 3 shows the grade point average in History of the respondents. The highest grade point average that the respondents attained ranged from 87 – 89 having a frequency of ten (10) or 34% second ranging from 84 – 86 having a frequency of 8 (eight) or 28% followed by 90 – 92 having six (6) or 21% subsequent to that is ranged from 81 – 83 and the grade point average of 93 and above and 80 below has zero frequency.

Table 4Respondents’ Perception on Controllability as a Factor of Integrative Assessment

Indicators MeanStandardDeviation

Remarks

1. The Cloze test procedure can easily adjust the instructional planning.

4.55 0.56 Strongly Agree

2. It can limit the learners the extension of different vocabularies.

4.72 0.58Strongly Agree

3. Can easily manipulate by the teachers and learners as well.


4. Placement of the blank at the end of the sentence in the passage restricts the possible words that may complete the sentence.


5. The instructors can readily assume the fairness in the interpretation results.

4.0 0.74 Agree

TOTAL 4.32 0.65 Strongly Agree

Legend:4.21 – 5.0 : Strongly Agree3.41 – 4.20 : Agree2.61 – 3.40 : Fair1.81 – 2.60 : Disagree1.0 – 1.80 : Strongly Disagree

However, Alderson (1980, 1983 cited in Ahluwalia, 1992:86) and Yamashita (2003:286-287) argue that cloze test performance is not directly related to the difficulty level of the text; it involves other factors such as scoring procedures, and content familiarity for the readers.

Table 5

Respondents’ Perception on Reliability and Validity as a Factor of Integrative Assessment


Remarks

1. Cloze test have adequacy and representativeness of learning outcomes.

4.55 0.56 Strongly Agree

2. The passage measures what it purports to measure.

4.14 0.59 Agree

3. Cloze test must first consider the errors of measurement.


4. It can easily interpret the test results to improve teaching and learning.


5. It has a value of testing as a tool to improve instruction and learning performance.


TOTAL 4.53 0.63Strongly Agree


. Much research has been devoted to the validity and the reliability of Cloze tests.

Cloze tests are reported to have moderate to high correlations with standardized tests and

their subtests such as listening comprehension and reading comprehension. The report on

high correlations between Cloze tests and International Journal of Human and Social

Sciences tests of listening comprehension, writing, reading comprehension, and oral

interview. The findings of these researchers reveal a relation between the scores on Cloze

and global language ability tests (Greene, 2001).

Table 6Respondents’ Perception on Usability as a Factor of Integrative Assessment


Remarks

1. It manifests strong retention constructed – 3.48 1.04 Agree

response type.2. Creates critical reasoning for learners on

how to arrive on precise conclusions. 4.62 0.89

Strongly Agree

3. Learners can employ to assess vocabulary, understanding facts, seeing, relationships, drawing inferences, detecting author’s literary style, and approximating the patterns of language structure.


4. The test provides communicative skills in understanding such areas of learning such as in the history subject.


5. Contributes logical argumentation and empirical evidence for learners such as resourceful in resolving authentic problems in studying history.


TOTAL 4.32 0.82Strongly Agree


A sentence or a passage, deleted some words, is just like a broken picture. People

will complete it unconsciously according to their comprehensive knowledge. The Gestalt

theory also claims that the more familiar with the picture the easier for people to

recognize it. Expounded in language learning, the case is that the higher language

proficiency the higher ability to complete the broken passage and promotes higher order

thinking skills but also more on life – relating tendencies which could be applicable to

further level of a child.

Table 7Distribution of Result in Rational Cloze Test

Item no. Frequency Percent Mean Interpretation1 13 52 0.45 Satisfactory2 13 52 0.45 Satisfactory

3 8 32 0.27 Fair4 9 36 0.31 Fair5 13 52 0.45 Satisfactory6 12 48 0.44 Fair7 2 8 0.06 Needs improvement8 12 48 0.17 Fair9 5 20 0.51 Needs improvement10 15 60 0.51 Satisfactory11 15 60 0.37 Satisfactory12 11 44 0.06 Fair13 2 8 0.34 Needs improvement14 10 40 0.04 Fair15 12 48 0.27 Fair16 8 32 0.41 Needs improvement17 12 48 0.17 Fair18 0 0 0 Needs improvement19 5 20 0.51 Needs improvement20 0 0 0 Needs improvement21 8 32 0.27 Fair22 11 44 0.06 Fair23 6 24 0.22 Needs improvement24 6 24 0.22 Needs improvement25 13 52 0.45 Satisfactory

TOTAL 221 100 0.28 FairLegend:24 – 25 : Outstanding20 – 23 : Very Satisfactory13 – 19 : Satisfactory8 – 12 :Fair0 – 7 : Needs Improvement

This table shows the result of the cloze test of 3rd year high school, as we observed item number 10 and 11 has a high frequency which interpreted as ‘Satisfactory’ based on the scale values above having a 60% of the total correct answer. As we look generally there are more results ranged from 8 - 12 thus the mean score of the respondents based on the experiment which is interpreted as ‘Fair’.

This implies that the respondents performed the test even-handedly since the total interpretation is said to be ‘Fair’ with regards to this Legenza (2001) reasoned behind giving a Cloze test and obtaining a test score is interpreting that score as an indicator of what a test taker knows or what he/she can do with that knowledge. Furthermore, our interpretation of that test score forms the basis for decision making. As such, when using a test score, we make an implicit link between test performance and a domain of knowledge the test taker has or something the test taker can do with schema in some area use domain beyond the test itself.

Table 8Correlation between the Person – related factors and Assessment Variables

Variables r - value p - value Interpretation

Age .209 .0320 Not Significant

Gender .227 .0411 Not Significant

Grade Point Ave. .124 .0309 Not Significant

Legend:p >.05 : not significantp <.05 : significant

The advantages of this cloze variant are convenience and high consistency, while there are also a lot of problems such as some of the blanks being too easy or too difficult, the broken passage sometimes arousing test-takers’ dislike and resulting in a dilemma for testers to choose subjective or objective scoring method, and being lack of control which possibly leads to examine the language points the designer doesn’t want to test actually. Rational cloze can also be called open cloze or gap-filling.

Table 9Correlation between Assessment – related variables and Cloze Test

Variables r - value p - value Interpretation

Controllability -.789 .0776 Not Significant

Reliability and Validity

.076 .0467 Not Significant

Usability .098 .0202 Not Significant


This procedure is intended to sample regularly various variants of words, some of which are governed by local grammatical constraints and wide discipline vocabularies in sorted areas of History, Mathematics, Statistics, others of which are governed by long-range textual constraints (Datta, 2002).

Table 9Correlation between Grade Point Average in History and Cloze Test

Variable r - value p - value Interpretation

Grade Point Ave. -.565 .0676 Not Significant


Askes (2002) regards cloze procedure as one of the integrative tests (global tests)

that integrate language components into a total language event, which requires an

integrated performance from the learner in a meaningful context. Thus for Steinman

(2002), a cloze can replace the sections on test structure, written expression, vocabulary,

and reading comprehension not only in English subject as well as different areas of

learning.

The reason behind giving a Cloze test and obtaining a test score is interpreting that score as an indicator of what a test taker knows or what he/she can do with that knowledge. Furthermore, our interpretation of that test score forms the basis for decision making. As such, when using a test score, we make an implicit link between test performance and a domain of language knowledge the test taker has or something the test taker can do with language in some language use domain beyond the test itself. In other words, when we use test scores, we are essentially reasoning from evidence, using the test score as the evidence for inferences or interpretations and decisions we want to make. Yet, we cannot simply draw on test score to make inferences and decisions without efficient justification. If we want to use a test score for a particular purpose, we must justify it through a rationale and supporting evidence.

Chapter V

Summary, Conclusions, And Recommendations

This chapter presents the summary of findings, conclusions, and

recommendations related to the effectiveness of Cloze test in assessing History subject.

Appendix A

Laguna State Polytechnic University

San Pablo City Campus

San Pablo City

LETTER TO THE DIRECTOR OF LABORATORY HIGH SCHOOL

Date:

Nelia T. Salvador, Ed.D.

Laguna State Polytechnic University

San Pablo City Campus, San Pablo City

Greetings of the day!

I am fourth year student of Laguna State Polytechnic University San Pablo City Campus, taking up Bachelor of Secondary Education, major in Social Studies. I am presently conducting a study entitled “Effectiveness of Cloze Test in History at Laguna State Polytechnic University San Pablo City Campus A.Y 2012-2013.

I would like to ask permission from your good office to evaluate teacher’s performance by administering a researcher-made test to twenty - nine (29) third year high school students of LSPU-SPCC.

It is in this juncture that I express my warmest gratitude. Should my request be granted. Thank you very much.

Truly yours,

MARILYN A. MENDOZA

Researcher

Appendix B

Republic of the PhilippinesLAGUNA STATE POLYTECHNIC UNIVERSITY

San Pablo City CampusSan Pablo City

COLLEGE OF TEACHER EDUCATION

LETTER TO THE RESPONDENT

Date:

Dear Respondents,

This is in reference to my study of the requirements for the course Methods of Research

entitled “Effectiveness of Cloze Test in History at Laguna State Polytechnic University”.

The data that will be gathered from you will be extremely useful for the successful

realization of my research.

It is in this juncture that I express my warmest gratitude. And will be treated with strictest

confidentiality. Thank you very much.

Very truly yours,

Marilyn A. Mendoza

Researcher

Appendix C

Republic of the Philippines LAGUNA STATE POLYTECHNIC UNIVERSITY

San Pablo CampusDel Remedio, San Pablo City

COLLEGE OF TEACHER EDUCATION

BACHELOR OF SECONDARY EDUCATION

Survey Questionnaire

I. Profile of the Respondents Direction: Please fill in the blanks with your personal information

Name (optional):____________

Age:__________

Gender: male ( ) female ( )

Grade point in History subject 1st semester 2012 – 2013:____________

II. Perception of the respondents ABOUT EFFECTIVENESS OF CLOZE – TEST AS INTEGRATIVE ASSESSMENT IN HISTORY.

Direction: Please indicate your responses on the following statements by putting a check (√) using the scale below as follows:

5 – Strongly Agree

4 – Agree

3 – Fair

2 – Disagree

1 – Strongly Disagree

A. Controllability

indicators 1 2 3 4 5

1. The Cloze test procedure can easily adjust the instructional planning.

2. It can limit the learners the extension of different vocabularies.

3. Can easily manipulate by the teachers and learners as well.

4. Placement of the blank at the end of the sentence in the passage restricts the possible words that may complete the sentence.

5. The instructors can readily assume the fairness in the interpretation results.

B. Reliability & Validity


1. Cloze test have adequacy and representativeness of learning outcomes.

2. The passage measures what it purports to measure.

3. Cloze test must first consider the errors of measurement.

4. It can easily interpret the test results to improve teaching and learning.

5. It has a value of testing as a tool to improve instruction and learning performance.

C. Usability


1. It manifests strong retention constructed – response type.

2. Creates critical reasoning for learners on how to arrive on precise conclusions.

3. Learners can employ to assess vocabulary, understanding facts, seeing, relationships, drawing inferences, detecting author’s literary style, and approximating the patterns of language structure.

4. The test provides communicative skills in understanding such areas of learning such as in the history subject.

5. Contributes logical argumentation and empirical evidence for learners such as resourceful in resolving authentic problems in studying history.

III. Cloze – test (adopted from Crystal David’s educational research: Cambridge 2004)

Directions: Read the passage and hint or predict the word that was removed and replace it with the word that best fits the intended meaning in a passage. (Time allotment is 25minutes).

The cat has a 1.__________ as fascinating and mysterious as the creature

itself. The true beginnings of the domestic cat are unknown, but the cat may have first

appeared around3000 B.C. in a 2.__________ called Nubia, which bordered Egypt. By

2500 B.C., the cat was domesticated in Egypt. The cat's first 3.__________ in Egypt was

Mau. The Mau’s 4.__________ in Egypt grew rapidly; she was eventually considered

guardian of the temple and was worshipped as a goddess. Besides being worshipped as

goddesses, cats also had a practical 5.__________: they kept 6.__________ from

overrunning the Egyptian grain store-houses.

The Greeks were probably the first 7.__________ to recognize cats for their

mouse- catching talents. When International Journal of Human and Social Sciences 5:9

2010556Egyptians refused to sell or trade any of their cats, the Greeks 8.__________

several of the Egyptian cats and sold the 9.__________ of these stolen cats to Romans.

The cat became the 10.__________ of liberty in ancient Rome. By the end of the

eleventh 11.__________ cats were popular among sailors because of their rat-catching

skills. Sailors admired cats because they 12__________ disease-infested rats which lived

on ships. Many sailors believed that cats possessed special powers that could

13.__________ them at sea.

Although the cat was held in high regard and fancied during 14.__________

times, the cat didn't fare will in Europe in the Middle Ages. Cats were associated with

evil, witchcraft, and black 15.__________. Many people believed that 16.__________

regularly transformed themselves into cats. Men and women were killed for helping a

17.__________ or injured cat. During the witch-hunts in Europe many innocent people

were accused of witchcraft simply because they owned cats. Black cats were especially

feared. Some legends and 18.__________ about cats exist today, like that about the nine

lives of cats. Another legend that survived from Europe's Middle Ages into the present

states that a black cat crossing one's path brings bad 19.__________.

Today the elegant, graceful cat has become a popular house 20.__________

throughout the 21.__________. The cat is one of the smartest of tame animals, but they

are independent and harder to train. Cats are valued for their gentle, affectionate natures.

They have 22.__________ memories; they 23.__________ who treats them well and who

treats them badly. A cat's loyalty is earned; a cat won't stay where it is 24.__________.

They respond to loving owners with loyalty, affection, and respect. Cats are noted for

their keen senses: their sharp hearing, sense of smell, and ability to 25.__________ in

near darkness. Perhaps Leonardo Da Vinci summed it up best when he referred to the cat

as “Nature’s Masterpiece.”

Appendix D

The Cloze test answer key:

1. history2. country3. name4. status5. function6. mice7. Europeans8. stole9. kittens10. symbol11. century12. destroyed13. protect14. ancient15. magic16. witches17. sick18. superstitions19. luck20. pet21. world22. good23. remember24. mistreated25. see

BIBLIOGRAPHY

Ahluwalia, N. (1992). Major Issues in the Cloze Procedure. Indian Journal of Applied Linguistics. (Volume No is not available):81-96.

Askes, H. (1991). Some Techniques in Communicative Second Language Testing. Journal for Language Teaching. Vol.25.1:36-63.

Bai, J. (2004). An Analysis of Methods for Facilitating the Completion of Cloze (Chinese). English Coaching Paper (for Senior Grade Three).Vol.2004/01/26.

Carrell, P. L. & Eisterhold, J. C. (1988). Schema Theory and ESL Reading Pedagogy. In Carrell, P. L., Devine, J. & Eskey, D. (eds.) Interactive Approaches to Second Language Reading. Cambridge: Cambridge University Press. 73-89. 92CET public website. (2004). College English Test. [Online]. Available: http://www. cet.edu.cn [Accessed on 2004, August 19].

Chan, D. & Mok, K. (2001). Educational Reforms and Coping Strategies under the Tidal Wave of Marketisation: a Comparative Study of Hong Kong and the Mainland. Comparative Education. Vol.37.1: 21-41.

Cheng, K. (1999). Young Adults in a Changing Socialist Society: Post-compulsory Education in China. Comparative Education. Vol.30.1: 63-73.

Clarke, M. A. & Silberstein, S. (2001). Toward a Realization of Psycholinguistic Principles in the ESL Reading Class. In Long, M. H. & Richards, J. C. (eds.) Methodology in TESOL. New York: Newbury House. 233-247.

Datta, S. & Macdonald-Ross, M. (2002). Reading Skills and Reading Habits: a Study of New Open University Undergraduate Reservees. Open Learning. Vol.7.1: 69-88.

Dreyer, C. & Nel, C. (2003). Teaching Reading Strategies and Reading Comprehen- sion within a Technology-enhanced Learning Environment. System. Vol.31. 3: 349-366. 93Du, H. (2001). The globalisation of the English language: Reflections on the Teaching of English in China. International Education Journal Educational Research Conference 2001 Special Issue. Vol.2.4: 126-133.

Ellis, R. (1994). The Study of Second Language Acquisition. Oxford: Oxford University Press.

Greene, B. B. (2001). Testing Reading Comprehension of Theoretical Discourse with Cloze. Journal of Research in Reading. Vol.24.1: 82-98.

Groebel, L. (1981). Reading: The Students’ Approach as Compared to Their Teachers’ Recommended Method. English Language Teaching Journal. Vol. XXXV.3: 282-287.

Helfeldt, J. P., Henk, W. A. & Fotos, A. (1986). A Test of Alternative Cloze Test Formats at the Six-Grade Level. Journal of Educational Research. Vol.79.4: 216-221.

Legenza, A. & Elijiah, D. (2001). The cloze procedure: Some New Applications. The Journal of Educational Research. Vol.72.6: 351-355.

thesis malyn

Education

reliability of cloze

relationshipbetween

cloze testsmeasure

usefulness of cloze

reliability of cloze

construction of cloze

classic cloze test

standardized tests