4.01

4.02

4.03

4.04

4.05

4.06

4.07

4.08

4.09

4.10

4. 11

4.12

CHAPTER IV

PLANNING AND PROCEDURE FOR CONSTRUCTING THE TEST

Introduction

Designing the test

Description of the population

Sampling the behavior to be tested

Review of literature

Choice of behaviours

Determination of weightage

Selection of listening materials

Preparation of test items

Determination of rate of presentation of listening material

Determining the length of the test

Arrangement of the test items

46

46

47

49

50

52

53

55

56

57

58

59

46

CHAPTER· IV

PLANNING AND PROCEDURE FOR CONSTRUCTING THE TEST :

4.01 INTRODUCTION:

chapter contained a detailed account

listening com prehension carried out in

The previous

researches on

and in recent

problem , these

times. Apart from giving an insight

of the

the past

into the

planning had also helped the investigator in

the present research work.

Planning is a necessary adjunct even for an ordinary day to

day work. And when this work is spread over a number of

days and many months, planning must be at a core if the work

is to be completed without any confusion or disarray. The

present research is a multi-dimentional study. Hence every

care and caution was exercised in preparing and executing the

plan to arrive at definite and reliable conclusion.

4.02 DESIGNING THE TEST:

While measurement is neassary at every stage of life, planning

is essential for every project - small or otherwise - whether

it is a project undertaken by a government or that by a

peasant. As K. W. Vughn points out, planning is an essential

activity in all stages of test construction project. Inattention

to planning may result in failure to meet production deadlines

or may necessiate use of uneconomical procedure or of below

standard materials in order to meet this deadlines. (Lindquist,

j. p. 139).

Economy that planning ensures can hardly be exaggerated efore

actual work of designing the tests was started ,the ' investigator

directed his efforts towards a careful planning which enabled

him to anticipate the difficulties and also to avoid them or

to minimise them. Different phases of coining the test, one

of the essential stage of the standardisation process, form the

main content of the present chapter.

47

The present test is aimed to measure an individual's ability

of listening comprehension. The first requisite of a good test

is that it must measure what it is meant to measure it means

that its content validity and reliability should be high.

This is not possible unless the test designer (investigator) is

quite clear' about the term listening comprehension. The meaning

of the term discussed at length in the previous chapter has

been accepted and consistently kept in mind throughout the work.

This alone does not suffice. One who attempts a practical

application of a theory must also be clear about other aspects

as under.

1. The uses of test-score.

2. The description of population that is range of appli­

cability.

3. The aspects to be measured that is the sample of the

behaviour to be tested.

4. Material to te used for listening comprehension.

5c The way it is to be measured that is the principle for

writing the test items.

4.03 DESCRIPTION OF THE POPULATION:

The population to be tested needs to be precisely defined befor'e

commencing to design the actual test itGms. It is essential

because the related phases of the test designing

bearing on the population to be tested.

direct

The nature of the mental measurement inevitably requires to

restrict the population to be tested. In fixing the range of

applicability the following four factors arc consi·dered.

1 • Schooling

2. Age

3. Cultural background

4. Sex

SCHOOLING:

The present

boys and

48

test is intended to be made applicable to pupils

girls studying in primary schools of Ahmedabad

city. For the purpose of the present study, a primary school

will mean a school imparting instruction to grades V, VI and

VII grades. VIII, IX and X are treated as part of the

Secondary School and are, therefore, excluded from the

selected population. Here primary schools are of two types,

primary schools conducted by private agencies are private

primary schools and those conducted by the j\hmedabad Muncipal

Corporation are called Muncipal Primary Schools. Pupils from

both the types of schools are taken as population.

CULTURAL BACKGROUND: Another problem which demands consideration is that of the

differences found in the cultural background. The present study

has been delimited to urban area only ,so this factor needP.d no

extra consideration.

AGE: Age norms are not found suitable for use as researches in the

area do not yet provide a curve of growth similar to the one

usually obtained while measuring intelligence. It is enough

to state, at present, that the problem has been considered

theoretically as well as practically.

SEX: As in the case of differences found in different subcultures,

the problem of sex differences also demand attention. Boys

and girls differ significantly in certain mental operations.

Sometimes the differences are large enough to be accounted

for. This issue was to be checked statistically· rather than

on arbitrary judgement, and decision regarding the establishment

of norms are to be taken on the basis of the statistical treatment

of the data. The factor is discussed fully in the chapter

dealing with the establishment of norms.

49

One of the crucial aspects of mental measurement namely

restricting the range of individuals to be tested, having fully

considered the other one, i.e. selecting the activities which

are to be tested, is considered next.

4.04 SAMPLING THE BEHAVIOUR TO BE TESTED:

A psychological test is defined as, " •••••• essentially an

objective and standardized measure of a sample of a behaviour."

(Anastasi, 2, p. 22).

The mental ability is to be measured and is bound to manifest

through various behaviours. The psychologist cannot and need

not measure them all while discussing the sampling of the traits

and functions, as Freeman suggests.

In developing a psychological test it is impossible, in fact

unnecessary, to use an unlimited number of items. It is not

necessary to attempt to present the individual being tested

(called the 1 subject 1 or 1 testee 1) with problems that will

ascertain his responses for every conceivable situations involving

a given trait or function. It is sufficient to get an adequate

sampling of responses in a particular area or range of behaviour

the assumption being that the sampling is representative of the

whole

( Freeman. P. 4)

Even then the task of psychologist remains complicated. The

sampling of human behaviour is obviously not a simple process.

Cron bach remarks,

50

We have come a long way from Wissle' s trying to correlate

college success with speed of cancelling 'a' s and Binet's

wondering whether psychologist had anything to contri b..Jte to

the identification of mentally inferior children.

( Cran bach, P. 189).

Yet the present test designer would like to agree with Bhatt,

when she says," It is not pessimism to believe that

measurement in psychology can never hope to reach the precision

and accuracy found in physical science".

(Bhatt. P.38).

It is not rationalization tut a caution. A test maker has to

bear in mind that it deals with a complex creature and hence

do his utmost to make his measurement accurate and precise,

taking due precautions at every stage of test development.

4.05 REVIEW OF LITERATURE:

The knowledge about mental test only can hardly suffice,

obviously the procedure to be followed in constructing different

types of mental tests cannot be the same. Moreover, language

is a fairly well defined ability, or group of abilities

relatively independent. It was, therefore, necessary t~ study

language tests in general and listening comprehension test in

particular.

A good deal of work has been done in these types of tests in

Western countries, while similar work in India is rather limited.

Even a few tests on language and very" few on listening

com prehension cover only one or two aspects of the subject.

The investigator had to rely on listening comprehension tests

51

constructed in foreign countries. He did not hesitate to do

so, because of the fact that though every language has its unique

characteristics, all languages have certain common ones also.

The test maker should take into account the distinctive nature

of the four language skills and measure them all separately

as well as in various combinations. With this in mind, the

investigator attempted to know how others had viewed it. The

following types of relevant literature on test construction was,

therefore, studied:

1 • The original

Section •• 2.

test as enlisted in the bibliography.

2. Review of tests in the Mental ~asurement Year book.

3. Description of tests in the books on eduational and

psychological measurement.

While studying the aforesaid literature, the investigator aimed

to study the following aspects :-

1. The skills measured.

2. Content of Tests

3. The variety of ways in which the same content is

evaluated.

The inference from the study is presented subsequently • The

first two aspects are discussed here. The third one is

discussed later on with the discussion on types of tests.

Various language tests are designed to cover different areas

of different skills. They measure some of the specific language

skills. Although speaking and listening are considered essential

parts of the language arts, few standard tests are available

for general use. Compared with reading and writing, research

and experimentation in the measurement of speaking and listening

are relatively limited. The reason for the limited number of

tests in these areas is the complexity of these skills and that

of difficulty in getting suitable practical media for appraising

them.

52

The study of the content of the tests revealed the fact that

the test measuring language ability covered a vast area. Various

topics are selected for the content of listening comprehension

tests. The study and the critical review of the available

literature on the listening comprehension test indicated that

listening comprehension is an understanding of fact and concepts

through listening and is composed of different documents. The

various components of listening comprehension so far described

had revealed the following characteristics:

* ability to express main idea of · the content listened to,

* ability to remember the requence of events or ideas.

* ability to give the significant details of what is listened

to.

* ability to find out the relationship between ideas presented

by the speaker.

* ability ~0 find out the meaning of words and phra!:ies in

the context of the material listed.

* ability to draw inferences.

* ability to draw generalization.

* ability to summarise what is listened to.

* ability to criticise the things listened to.

Considering the aforesaid topics some of them were found to

be rather frequent.

4.06 CHOICE OF BEHAVIOURS:

The decision for the number of components to be covered by

the test was based on the insight gained by review of literature.

The outline of the objectives discussed in previous chapter

and appended in appendix was helpful at this stage. Hence,

it was decided to include the following topics on which items

could be constructed to measure .listening comprehension.

1. The significant details of what is listened to.

2. The sequence of ideas or events described by the speaker.

53

3. The main ideas of the things listened to.

4. Meaning of the words in context of things listened to.

5. Relationship between ideas and generalisation.

The list thus finalized was given to experts and after their

confirmation only the final decision of construction of items

was taken. It can be concluded that a fair and honest attempt

was made to select suitable and purposeful components with

a view to making the instrument valid and useful.

Here it has been decided to plan and construct a listening

comprehension test in Gujarati for the pupils of V, VI and VII

standards. The test could measure the following components

of listening:

1. The significant details of what is listened to.

2. The sequence of ideas or events described by the speaker.

3. The main idea or the caption of the things listened to.

4. Meaning of the words in the context of things listened

to.

5. Infering the relationship between ideas and drawing of

the generalisation.

4.07 DETERMINATION OF WEIGHT AGE:

After fixing the components to be tested the next step is to

determine the weightage to be given to these components. For

this, the sheets containing all the above mentions five components

were given to experienced teachers with a request to assign

marks out of 100 to these five components. This was done

keeping in view the age and the grade of pupils. The average

weightage for each class was then calculated. The overall

average weightage given to the components in the test for

measuring listening comprehension was worked out.

data are given in table below :

The relevant

TABLE 4.01

Class wise average weightage & the overall

to different components :

54

\veightage given

NO. COMPONENTS STD. AVERAGE WEIGHTAGEOVERALL AVERAGE WEIGHT AGE

1. Signi ticent details of what is listened • 20 • 00 20.00 21.00 20.33

2. The sequence of ideas events. 20 .oo 20.00 21.00 20.33

3. The main ideas of caption of the things listened 22 .00 20 .00 20.00 20.66

4. i context. 20 .00 20 .00 20 .oo 20 .00

5. Relationship between ideas and drawing generalization. 18 .00 20 .00 18.00 18 0 66,

TOTAL 100 .00 100.00 100 .00 100.00

TABLE 4.02

Weightage given to content

NO. PASSAGE NO. MARKS % MARKS

12 24

2 2 11 22

3 3 9 18

4 4 9 18

5 5 9 18

TOTAL 50

55

4.08 SELECTION OF LISTENING MATERIAL

The listening skills were cited earlier in the body of the

report. The listening materials which were to be used in the

final test must at their face value measure one or other of the

listening skills enumerated earlier. This was the talisman before

the investigator in selection of the material.

In order to select various listening materials, a group discussion

of experienced teachers teaching Gujarati in grades V, VI and

VII was organised. Individual meetings were also arranged and

certain criteria for selection of different materials was framed.

The listening material included paragraphs, stories, narratives

etc. The following criteria for the selection of the listening

material were given to the teachers participating in the

discussion.

1. Language, because of

comprehension, must

for whom the test is

of

the listening

the pupils

its important role in

be within the reach

being constructed. Here 1 Language 1

means the mode of presentation, i.e. its direction or style.

2. Vocarulary makes the passage difficult inspite of its

dictation being simple. The words must be familiar, and

of appropriate length and. t hose words which fall outside

the ken of pupil 1 s comprehension should be substituted

by easier ones.

3. Phrases and sentence construction

Certain phrases, idioms and otrer dialectical words which

were creating difficulties in the comprehension of the

average pupils were to be deleted from the passage and

substituted by easier words or phrases. Whenever possible

the long and clumzy sentence construction such as complex

sentences were made shorter and simple looking to the

level of understanding of the average mass of the pupils

for whom the test was meant.

56

4. Interest· is a potent factor in listening though it is

enigmatic to measurement. The paragraphs, stories,

scientific nirratives were to be selected in such a way -that· the in-jerest of the pupils must be maximLim when

they listen to them. :.-. ..

The teachers in the group discussion selected materials

according to the criteria given to them. After selecting

the sample materials, the next step was to coin the test

items for which the help was taken from various quarters

such as linguist its research fellows and experienced

teachers and method masters. See appendix No.-1.

The above selection of materials was from the printed

books. The materials could be read or listened to when

read alound •

4.09 PREPARA-:-ION OF TEST ITEMS:

It was decided to prepare multiple choice type of items for

testing the components of listening comprehension. The teachers

participating in the discussion were given illustrations of the

nature of multiple choise type of question and the points to

be kept in mind while coining the items. A greater number ..

of items were prepared th. n what would be required for. the

final form of the test. The items thus prepared by teachers

teaching Gujarati in classes V, VI and VII were screened by a

committee of experts consisting of Principal investigator, research

fellow and method masters in Gujarati who are well conversant

with the preparation of framing objective type of questions.

The items were prepared and screened keeping in view the

components of listening comprehension mentioned earlier in this

chapter. In all 100 items were finalized and edited for including

them in the experimental form of tests. The items were arranged

according to the sequence of the points presented in the listening

material. The experimental form of the test, thus prepared,

was tried out on a sample of students with a view to finding

out the gross mistakes in wording of the questions, and to see

4.10

57

whether the instructions communicate well. Prior to this, there

was a question aoout the rate of presentation of listening

material, for which the following method was adopted.

DETERMINA liON OF RATE OF PRESENT A liON OF LISTENING

MATERIAL:

In case of administering the listening comprehension test, the

rate of presenting material before the subjects is of paramount

importance. For deciding the rate of presentation, it was

thought to tape-record the speech of the teachers teaching

different subjects in classes v, VI and VII. Accordingly, the

speeches of teachers for five minutes from different schools

situated in Ahmedabad city areas were tape-recorded. The

sample includes teachers teaching various school subjects. An

analysis was carried out and the average rate of speech and

the delivery for each class was computed. The average rates

of deli very of words per minute in the class-room by teachers

are given in the table below.

TABLE 4.03

Average no.of words spoken per minute by teachers during

class room teaching.

Class No.of Teachers •. Mean Medtum S.D.

v 25 67.11 63.50 24.60

VI 25 65.80 81.15 22.00

VII 25 70.50 78.50 18.06

Average of

the whole

Group 75 67.80 71.70 21.55

58

The above table 4.03 reveals that the general mean rate delivery

of words per minute is 67.80 and the general medium rate is

71. 70. Besides this, the comparison of the average rate of

deli very of words for classes V, VI and VII could be considered

as by and large the same, for the differences in the means

are insignificant. Therefore, the average total rate of delivery

of words per minute was worked out and the figure obtained

was converted into whole. It was found to be 67. This small

study helped to decide that the listening materials be presented

at the rate of 67 words per minute.

4.11 DETERMINING THE LENGTH OF THE TEST:

The number of items to be included in the final for·m of the

test is generally

of the test and

determined by keeping in

important decisions to be

view the purpose

taken about the

individuals for whom it is prepared. For this the test must

be reliable and hence must contain more items. The construction

of the present test was undertaken with a view to providing

schools with a valid and reliable tool for measuring

comprehension, and to make a comparative study. Therefore,

it was decided that the test should be of an adequate length

so that it could be administered to a class within usual length

of two class periods. The time was fixed keeping in view

the avoidance of an element of fatigue while taking the test.

This practical situation had to be paid due attention, otherwise

the test would remain unused. Therefore, the number of items

constructed for tryout was considerably larger than what would

be required for the final form of the test.

59

The number of items constructed before try out was 100 and

after pre-try, ten items were rejected. So, the no. of items

now remains 90. In the final try out, 40 items were rejected,

so only 50 items are remained in the final format of the test.

Now the final test of listening comprehension has only 50 items.

See table No.4.04, 4.05 and 4.06. and appendex No.3.

4.12 ARRANGEMENT OF THE TEST ITEMS:

After the

the items

items

from

had been constructed, the

amongst. those that survived

problem selecting

the review and

screening process, and then try-out those which constituted

the test and arranging them into some order for each sub-test

While compiling the test items, the following points were born

in mind.

Overlapping of items be avoided.

Selected items be arranged according to the difficulty

level.

Selected items testing one of the components of listening

comprehension be included in the test.

Items of the same type be grouped together.

Accordingly, in every sub-test, the multiple choice type of

items were put together and they were first (before trying out)

arranged according to the sequence of the content of the passage.

The items testing the pupils ability to follow the sequence of

events or ideas presented were kept in the middle. This item

was followed by the items testing vocabulary.

DESCRIPTION OF SUB-TESTS:

In all five sub-tests were constructed. Initially these five

sub-tests had altogether 100 items. After pre-try-out their

no. was only 90 and after experimental try-out, the no. of items

was 50. So in the final test format, there were 5 sub-tests

and the no. of items was 50. See table below :-

TABLE 4.04

showing the orignal form of the test.

TE:ST NO.OF ITEMS

1 19

2 21

3 16

4 21

5 23

TOTAL 100

TABLE 4.05

Showing the no. of test and items in each test after

pre try-out on 30 pupils.

Test No. of Item

17

2 19

3 14

4 19

5 21

TOTAL 90

TABLE 4.06

Showing the no. of test & item in each test after the

experimental try-out.

Test

2

3

4

5

TOTAL

No. of Item

12

11

9

9

9

50

60

61

First sub-test is based on the story of the sparrow and its

kinds. The main idea of the story is regarding mother-love.

Finally, the sub-test based on this has 12 items.

Second sub-test is constructed on the passage 1 Ibrahim Kaka 1

•

The passage deals with the good relation between two families.

One is of Hindu and the other is of Muslim family. The

passage narrates the unity and love for each other. This second

sub-test has finally 11 items in all.

Third sub-test is constructed on the

chhun 1 • ( 1 Cooking the hatch patch 1 ) •

story

The

1 Khichadi pakavu

story deals with

- We come across

The third sub-test an incident Showing the brain of - Birbal.

many stories based on Aka bar and Bir bal.

is based on this 1 Khichadi Pakav chhun 1 story. It has nine

items. The fourth sub-test is constructed on the content of

Rastra-Premi Margaret. (Patriotic Margaret). This passage deals

with the love towards Nation at the time of war. At the time

of second world war, England was on the defit. How the old

lady Margaret was useful in creating love towards Nation! main

factors for the victory of England. The fourth sub test is based

on the contect of this passafe. It has finally nine items.

The fifth and last sub-test is based on the content of the

passage "A letter from Ajanta". This is a narration of a

journey. (trip from the school). Ajanta is known for its

classical art paintings and old pictures of Bhagwan Buddha and

his followers. The ancient paintings give inspiration to our

young artists. The last sub-test is based on this content.

It has finally nine Items. The selected passages for listing

are given in Appendix No.4.