document resume mccall, chester h.; walters, lauren e ... · document resume. ed 419 839 tm 028...

DOCUMENT RESUME

ED 419 839 TM 028 415

AUTHOR McCall, Chester H.; Walters, Lauren E.TITLE Words and Their Value to the Survey Researcher.PUB DATE 1998-04-00NOTE 27p.; Paper presented at the Annual Meeting of the American

Educational Research Association (San Diego, CA, April13-17, 1998).

PUB TYPE Reports - Research (143) -- Speeches/Meeting Papers (150)EDRS PRICE MF01/PCO2 Plus Postage.DESCRIPTORS Administrators; *Definitions; Engineers; *Graduate Students;

Higher Education; *Research Methodology; Responses;*Surveys; Tables (Data); *Test Construction; Test Items;*Undergraduate Students

IDENTIFIERS *Quantifiers

ABSTRACTIn May 1972, 2,900 copies of a survey were distributed

eliciting opinions about the meaning of some qualifying adjectives andphrases commonly used in "verbal communications." These same adjectivephrases are commonly used in survey research. They include phrases such as "afew," "a majority," "nearly all," and other quantifiers. The 1972 survey hada 40% response rate, indicative of interest in the study. This study updatesthat 1972 study, reporting findings from different groups of subjects. Ninegroups (graduate students in several disciplines, including education,undergraduates, and engineers and administrators) ranging in size from 3 to20 members were surveyed. Results made it clear that individuals withindifferent groups give different interpretations to quantifying phrases, andthat differences also exist among groups. It is suggested that there is aneed to study item reliability in surveys in more detail if words andphrases, such as the 22 identified in this study, are to be included inresponse options, narrative instructions in a survey, or reporting on thesurvey findings. Appendix A is the report from the 1972 survey, and AppendixB contains a copy of the original survey form. (Contains five tables, eightfigures, and three references.) (SLD)

********************************************************************************* Reproductions supplied by EDRS are the best that can be made *

* from the original document. *

********************************************************************************

00

WORDS AND THEIR VALUE

TO THE

SURVEY RESEARCHER

Chester H. McCallGraduate School of Education and Psychology

Pepperdine UniversityCulver City, CA 90230

Lauren E. WaltersTeacher

Los Angeles Unified School District

U.S. DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement

ED ATIONAL RESOURCES INFORMATIONCENTER (ERIC)

This document has been reproduced asreceived from the person or organizationoriginating it.

Minor changes have been made toimprove reproduction quality.

Points of view or opinions stated in thisdocument do not necessarily representofficial OERI position or policy. 1

American Educational Research AssociationLO 1998 Annual Meetingd. San Diego, CACO

April 13-17, 1998

I

AERA\1998.doc

PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL HAS

BEEN GRANTED BY

C. mcCQ I I

TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC)

[email protected] 4/3/98

WORDS AND THEIR VALUE TOTHE SURVEY RESEARCHER

DESCRIPTORS:Survey research, research methodology, validity/reliability, instruments

Background

In May of 1972, 2,900 copies of a survey were distributed eliciting opinions regarding the meaning ofsome qualifying adjectives/phrases commonly used in "verbal communications." These sameadjectives/phrases are commonly used in survey research. John Hoyt (1972) in his report emphasizes theproblems which arise when using some of these words and/or phrases, particularly when decisions mightbe made based upon an interpretation of what these qualifying adjectives/phrases might mean. In the 1972paper, the author does not identify how the 2,900 initial sample was selected. He indicates that withinthree weeks a 40% response was obtained, "demonstrating that there was 'rather considerable' interest inthe subject." Stanley Payne (1951) in his provocative book points out how important it is in developinganinstrument with questions that the meanings of the allowable responses be as clear as possible. PaulScipione (1995) in his timely article includes the following introductory comments:

"Those of us who design and direct research studies spend so much time with numbers that weoften forget that words also represent values (in the minds of the writer as well as the readers;authors comments). Yet both the questionnaires we use to elicit information from respondentsand the reports we use to communicate research findings and recommendations to clients dependsignificantly on language."

He goes on to make several additional points which, independently, have been raised after more than 15years of survey research involvement by the author of this paper:

We speak of a substantial change rather than indicating the numerical amount of the change;We convert descriptive words to numerical values and develop descriptive statistics, such asaverages, to discuss the findings; and,We make considerable use of descriptive words and phrases, such as a lot, many, orfrequently, in providing a report on the findings of a survey.

The descriptive results of the 1972 study appear as Appendix A to this paper. A copy of the initial surveyform appears as Appendix B. It is important to note that these two appendices are exactly as reported byJohn Hoyt in 1972.

Objectives/Purposes

Believing strongly that the 1972 results have probably not changed much and, because of the author'sinterest in using surveys to collect useful information in educational research and for makingmanagementdecisions both within education and elsewhere, the author received release time from his educationalinstitution to collect additional data using the same survey and the same set of instructions. Had the authorbeen aware of the Scipione (1995) study at the time of his research, the items used on the instrument mighthave been slightly modified; although, from a research perspective, a replication of the exact instrumentand instructions (Hoyt, 1972) seemed most appropriate.

The purpose of this study was to update the 1972 study and to report findings from groups of differentsubjects. It was hypothesized that individuals who might become subjects for surveys would continue toattach different meanings or interpretations to the words and phrases in the original study, creating a needfor an awareness on the part of those using the survey results about the possible differences ininterpretation of the words and phrases, not only as instruments are developed but also as the results arereported.

AERA\PAPER-98.DOC PAGE - 1 3 4/6/98

Perspectives/Theoretical Framework

Of considerable concern in survey research is assessing the reliability and validity ofan instrument andalso items on the instrument. In the context of this paper, reliability of an item is the extent to which theitem means the same thing to all respondents at a given time and the same thing to an individual respondentat different points in time. It is generally agreed that survey results may be, essentially, worthless if theindividual items on the instrument are unreliable and invalid. This paper addresses the reliability issue interms of the way in which words or phrases used in assessing opinions or perceptions may be interpretedby the respondent and, hence, interpreted and reported by the researcher. The instrument used in the 1972study and again in this study contains 22 items.

Methods. Techniques. Modes of Inquiry

Unlike the 1972 study in which the sampling procedure was not clearly identified, the intent of thisreplication was to utilize individuals from nine different groups to address two questions:

Do those responding today produce similar results (in terms of averages, medians, modes,ranges, and distributions) to those obtained in 1972?Is there a difference in the response patterns among nine different groups?

The instrument was administered in a group setting by the author and by three other faculty members at asingle institution. Because of the ambiguity of the 1972 instrument (Appendix B), faculty explained whatwas expected in assigning numerical values to the verbal descriptions. Data were coded, entered onto anEXCEL spread sheet and imported into NCSS 97© 6.0 for Windows, a statistical analysis softwarepackage.

Data Sources or Evidence

The nine groups from which data were collected were:

1. A research class at the doctoral level, fail 1995.2. A special class of doctoral students studying "organization change," fall 1995.3. A research class at the doctoral level, winter 1996.4. Undergraduate students in a communications major, fall 1995.5. An education faculty associated with graduate programs, winter 1996.6. A third research class at the doctoral level, winter 1996.7. A masters class majoring in educational administration, winter 1996.8. A masters class in research with students primarily in teacher training education, spring 1996.9. A group of engineers and administrators in the management construction business during a

seminar on survey methods, summer 1996.

The nine groups above ranged in size from nine to twenty three individuals. After data were coded andsummary statistics obtained, individual responses were reviewed for logic and consistency. In three casesthe results were discarded because the respondent apparently did not understand the directions.

AERA\PAPER-98.DOC PAGE - 2 4/6/98

4

Results and/or Conclusions/Point of View

Summary statistics for the 1972 data appear as Appendix C. Appendix D (Tables 1 through 5) indicate foreach of the nine groups and each of the 22 quantifying phrases, the following statistics:

Arithmetic mean (average)MedianModeThe range of reported valuesThe minimum and maximum reported values

It is interesting to note, for selected quantifying phrases and the arithmetic mean, median, and minimumand maximum values, comparisons among the nine groups and the Hoyt (1972) study:

ARITHMETIC MEANHoyt 1 2 3 4 5 6 82 9

A few 8.31 7.4 17.1 9.8 3.9 17.4 8.2 10.1 6.4Many 31-40' 54.3 45.7 51.7 30.4 50.3 49.9 45.8 44.9A majority 51-55 59.8 56.7 56.9 57.9 56.6 66.4 68.7 56.8Nearly all 86-90 95.4 87.8 94.7 85.8 84 95.5 88.1 85.6

MODE'A few 10 3 3 3 3 5 3 3Many 40 50 80 60 40 25A majority 51 51 51 51 51 51 51 51 51Nearly all 90 98 90 95 95 90

MAXIMUM AND MINIMUMA few N /A' 0-25 3-80 3-50 2-12 3-80 2-25 3-60 0-30Many N/A 20-100 15-80 10-100 4-80 20-90 20-99 6-90 5-100A majority N/A 51-90 51-90 10-90 10-99 51-90 51-100 40-98 25-100Nearly all N/A 90-99 20-98 85-99 20-99 0-99 89-99 5-99 25-99

1 - For some reason, Hoyt reported, for some phrases, the arithmetic mean as a range of values.2 - Data from Group 7 in Appendix D appear to be entered in error and have not been included in this table.3 - Due to small samples the mode did not exist in some of the groups.4 - Hoyt did not report ranges.

Detail for the other phrases and groups are reported in Appendix D.

To give a flavor of the specific relative frequency of responses, Appendix E contains histograms for thefollowing phrases:

A coupleA fewA majorityA substantial majorityNumerousLotsA large proportion ofVirtually all

AERA\PAPER-98.DOC PAGE - 3 4/6/98

5

Even if we assume that some of the respondents may not have understood the directions or were "jokingaround," the results still are unnerving and suggest the need for great caution in using quantifying phrasesin both the body of surveys and reporting the findings of surveys.

It is quite clear that individuals within groups give different interpretations to quantifying phrases andamong groups differences also exist. It is suggested, although not necessarily supported by the data in thisstudy, that different professions might exhibit differences in not only location statistics but also ranges.

Educational or Scientific Importance of the Study

It is clear to the author that the diverse opinions with respect to the meaning of many of the "quantifying"words and phrases are pause for reflection on how we are analyzing and reporting survey results. It issuggested that there is a need to pursue further the issue of item reliability if words and phrases, such as the22 identified in this study, are to be included in response options, in narrative instructions in a survey, or inreporting on the survey findings.

REFERENCES

Hoyt, J. S. (1972). Do quantifying adjectives mean the same thing to all people? AgriculturalExtension Services: University of Minnesota.

Scipione, P. A. (1995). The value of words: numerical perceptions associated with descriptive wordsand phrases in market research reports. Journal of Advertising Research, May/June 1995.

Payne, S. L. (1951). The art of asking questions. Princeton, NJ: Princeton University Press.

NOTE: Scipione's paper includes 17 excellent references related to both verbal and visual presentations ofitems associated with questionnaires and their analyses.

ARRA\PAPER-98.DOC PAGE - 4 4/6/98

RESULTS OF THE SURVEY

General

APPENDIX A

The first, and probably strongest, point to be made in looking at the surveyresults is that, despite the large number of valid responses (1180) and thelarge rate of return (40%) to the survey, no claim is made for the precisionof the results. Indeed, the most valid single conclusion that can be drawnis that the quantifying adjectives used in the survey represent to the sampleof respondents surveyed a broad range of numbers. It is also clear that thecontext in which the adjective is used may be as relevant to the quantity impliedas is the quantifying phrase itself. In addition; the results suggest to all whoverbalize quantitative data that they ought to be as precise as possible and, tothose of us who listen to verbalized data, to insist that the speaker injectprecision into his verbalizations.

The meanings of the quantifying adjectives are summarized briefly in figure 1and graphically in figures 2 through 25. In addition, for those who are con-cerned with additional statistical analysis a printout of the 1180 valid responsesfor each of the quantifying adjectives is available from MAPS.

The graphical representations, in addition to presenting the distribution of theresponses, also show three statistics for each response. First, the "mode"which is simply the discrete number that appeared most often for each adjective.Second, the "median" which is the number in the series of responses that representsa point such that half of the numerical responses fall on one side of it and halfon the other side. Third, the "mean" which is the sum of all of the responsesdivided by the number of responses. The mean is also frequently referred to as

. the "average" response. To illustrate, if a group of 9 numerical responses were:1,1,2,2,3,4,4,4,6 --then the mode would be 4 (the most frequent response); themedian would be 3 (there are as many responses below 3 as there are above 3);and the mean would be 3 (the sum of all the responses, 29, divided by the numberof responses, 9).

Another general observation about the responses is that, with the exception ofquantifying adjectives which represent numbers less than 10 or numbers greaterthan 90, the most frequent responses fell on numbers divisible by 5. There are

. two other exceptions to this observation; the phrase 'a minority' and the phrasea majority' are subject to rather precise statistical definition and the responses

were, as one would expect, most frequently given as these two single numbers.For responses where the quantifying adjective has a mode of 90 or above therewere relatively frequent usages of the numbers 95, 98, and 99 and a relativelyinfrequent choice of the other discrete numbers between 90 and 100.

APPENDIX B

Out of a total of 100 items (votes, apples, dollars, opinions, or what haveyou) place a number in the right hand column that represents your under-standing of each of the following descriptors. (Please use only one wholenumber as your "best" answer; no decimals, fractions, or ranges). -Remember,enter the whole number out of 100(%) for each description.

Descriptor Number it Represents

"A few"

"Damn few"

"A couple"

"Lots"

"A majority"

"Several"

"Many"

"Most"

"Almost all"

"Hardly-any"

"A substantial majority"

"A significant number of . . ."

"Virtually all"

"Almost none"

"A small number of"

"Numerous"

"Nearly all"

"A minority of"

"A consensus of"

"Not very man of"

"A large proportion of"

"A clear mandate" (i.e. votes outof the 100)

Quantifying Phrase

SUMMARY OF SURVEY RESPONSES APPENDIX

Statistic

C

Mean*(Average)

ResponseMode

(Most Frequent)Median*

(Mid-Point)

Almost None 1 2.19. 2.84

A Couple 2 2.04 2.21

Damn Few 3 3.17 3.81

Several 3 7.47 11.94

Hardly Any 5 4.52 4.84

Few 10 7.08 8.31

Small Number of 10 8.16 9.02

Not Very Many 10 6 - 10 6 - 10

Lots 40 41 - 50 41 - 50

Many 40 31 - 40 31 7 40

Significant Number of 40. 31 - 40 31 - 40

Considerable Number of 40 31 - 40 31 - 40

Numerous 40 31 - 40 31 - 40

Minority 49 21 - 30 21 - 30

Majority 51. 51 - 55 51 - 55

Consensus 60 51- 60 51 - 60

Substantial Majority 75 66 - 70 66-- 70

Large Proportion 75 61 - 70 51 - 60

Clear Mandate 75 61 - 70 61 - 70

Most 90 71 - 80 61 - 70

Nearly All 90 86 - 90 86 - 90

Almost All 95 86 - 90 86 - 90

Virtually All 95 95.25 94.04

.

* In order to make the statistics of general usefulness medians and means arereported as ranges in those cases when appropriate.

9

TA

BL

E 1

. Ari

thm

etic

Mea

n fo

r N

ine

Gro

ups

on T

wen

ty T

wo

Item

s

ME

AN

Ave

rage

17,

Qua

ntify

ing

Phr

ase

Gro

up 1

Gro

up 2

Gro

up 3

Gro

up 4

Gro

up 5

'G

roup

6 -

Gro

up 7

',

Gro

up 8

',..

Gro

up 9

Age

4142

21.6

5340

.329

.329

.345

.2A

Few

7.4

17.1

9.8

3.9

17.4

8.2

10.1

10.1

6.4

A D

amn

Few

3.3

12.7

4.5

3.9

4.3

4.7

3.2

3.2

8.4

A C

oupl

e2

142.

92

10.5

10.9

6.4

6.4

7.1

Lots

60.2

63.7

58.7

34.9

46.6

62.6

46.1

46.1

48.9

A M

ajor

ity59

.856

.756

.957

.956

.666

.468

.768

.756

.8S

ever

al23

.317

.315

13.1

21.1

16.5

22.9

22.9

13.9

Man

y54

.345

.751

.7-,

30.4

50.3

49.9

45.8

45.8

44.9

Mos

t81

76.7

85.4

67.9

7881

.281

.781

.775

.6A

lmos

t All

95.8

93.3

94.4

81.4

92.4

85.5

86.3

86.3

88.2

Har

dly

Any

4.6

14.1

9.1

6.2

2.8

8.6

8.5

8.5

10.2

A S

ubst

antia

l Maj

ori

76.5

69.5

77.2

72.8

61.5

74.4

71.5

71.5

68.7

A S

igni

fican

t Num

b(57

.446

.760

.649

.354

.354

.258

.358

.350

.2V

irtua

lly A

ll87

.9_

94.8

9590

.785

.795

.889

.489

.487

.9A

lmos

t Non

e2.

410

.45.

92.

82.

21.

86.

96.

88.

3A

Sm

all N

umbe

r of

6.4

16.6

11.1

9.6

7.7

98.

78.

711

.6N

umer

ous

44.6

50.7

51.2

42.3

39.5

39.4

41.6

41.6

39.3

Nea

rly A

ll95

.487

.894

.785

.884

95.5

55.1

88.1

85.6

A M

inor

ity o

f24

.442

.333

.921

.141

.649

.126

.626

.630

.4A

Con

sens

us o

f73

.574

.773

.159

.484

.581

.568

.368

.370

.9N

ot V

ery

Man

y of

12.9

1519

10.5

8.3

15.5

12.5

12.5

13

A L

arge

Pro

port

ion

174

7071

.570

.570

.568

.466

.266

.256

.9A

Cle

ar M

anda

te84

.468

.777

.381

.969

.182

59.5

59.5

67.7

AE

RA

\ M

EA

N.X

LS

10P

age

1

113/

31 /9

8

TA

BL

E 2

. Med

ian

for

Nin

eG

roup

s on

Tw

enty

Tw

o It

ems

ME

DIA

N

Gro

up 5

:56

Gro

up 6

41.5

Gro

up 7

27G

roup

827

.. G

roup

944

Qua

ntify

ing

Phr

ase

Gro

up 1

,G

roup

.2G

roup

3G

roup

4A

ge41

4120

A F

ew6

76

312

55

53

A D

amn

Few

35

33

2.5

33

33

A C

oupl

e2

22

22

22

22

Lots

7075

6120

55.5

7540

4050

A M

ajor

ity51

5151

5151

6070

7051

Sev

eral

9.5

79

9.5

1210

1010

6.5

Man

y50

4052

.525

47.5

4540

4035

Mos

t85

8090

82.5

8085

9090

80A

lmos

t All

9695

9590

9095

9595

90H

ardl

y A

ny3

53

4.5

2.5

33

34.

5A

Sub

stan

tial M

ajor

'79

7575

7570

7577

7775

A S

igni

fican

t Num

b(60

4065

5065

5160

6055

Virt

ually

All

9898

9895

9598

9898

96A

lmos

t Non

e1

21.

51.

82

21

13

A S

mal

l Num

ber

of5

107.

510

97

66

7N

umer

ous

4850

47.5

3035

3735

3530

Nea

rly A

ll97

9095

9495

9895

9590

A M

inor

ity o

f20

4036

.522

.547

4025

2531

.5A

Con

sens

us o

f71

8075

5588

8570

7066

Not

Ver

y M

any

of10

1212

.510

415

1010

10A

Lar

ge P

ropo

rtio

n72

.570

7578

7570

7575

66A

Cle

ar M

anda

te82

.570

8085

68.5

8575

7570

AE

RA

WE

DIA

N.X

LS

12

Pag

e 1

J3/

31/9

8

TA

BL

E 3

. Mod

e fo

r N

ine

Gro

ups

on T

wen

ty T

wo

Item

s

MO

DE

-Q

uant

ifyin

g P

hrci

se..

Gro

Up:

1..

.Gro

up 2

,;G

roup

3G

roup

4G

roU

p 5

.,G

roU

p 6

Gro

up 7

Gro

up 8

Gro

up 9

Age

4135

2057

2323

A F

ew3

33

35

33

3A

Dam

n F

ew3

33

32

33

33

A C

oupl

e2

22

22

22

22

Lots

7030

2075

50A

Maj

ority

5151

5151

5151

5151

51S

ever

al3

155

55

Man

y50

8060

4040

25M

ost

9090

9090

9090

Alm

ost A

ll99

9599

9095

9590

Har

dly

Any

21

22

1

A S

ubst

antia

l Maj

orl

8075

7575

7085

8575

A S

igni

fican

t Num

b(60

4075

5060

6070

Virt

ually

All

9998

9995

9599

9999

98A

lmos

t Non

e1

11

11

11

A S

mal

l Num

ber

of10

610

1010

55

5N

umer

ous

6035

3015

3030

30N

early

All

9890

9595

;90

90A

Min

ority

of

2049

.49

3049

4010

10A

Con

sens

us o

f10

010

010

010

010

010

0N

ot V

ery

Man

y of

1010

10-3

1010

10A

Lar

ge P

ropo

rtio

n of

7075

8070

70A

Cle

ar M

anda

teI

7575

8010

067

8080

75

AE

RA

\ M

OD

E.X

LS

1 4

Pag

e 1

3/31/98

15

TA

BL

E 4

. Ran

ge V

alue

for

Nin

e G

roup

s on

Tw

enty

Tw

o It

ems

RA

NG

E V

ALU

E(T

he D

iffer

ence

Bet

wee

n th

e M

inim

um a

nd M

axim

um V

alue

s)Q

uant

ifyin

g P

hras

e,

Gro

Llp

1G

roup

:21G

toup

3G

roup

4G

roup

5G

roup

6G

roup

7G

roup

8G

roup

9A

ge19

2210

3529

2121

16A

Few

2577

4710

7723

5757

30A

Dam

n F

ew10

8822

815

119

974

A C

oupl

e0

8818

078

9888

8878

Lots

100

9085

8370

8595

9695

A M

ajor

ity39

3980

8939

4958

5875

Sev

eral

8877

9746

8768

8686

67M

any

8065

9076

7079

8484

95M

ost

4844

4069

2549

9090

94A

lmos

t All

,.9

1920

949

9098

9874

Har

dly

Any

2088

9929

568

7474

79A

Sub

stan

tial M

ajor

i48

7038

9075

5390

9075

A S

igni

fican

t Num

b(95

6575

5555

8693

9385

Virt

ually

All

9919

2479

100

990

9074

Alm

ost N

one

1069

749

54

9090

80A

Sm

all N

umbe

r of

1268

7918

1224

2323

27N

umer

ous

8157

7066

8068

7979

99N

early

All

978

1479

9910

9494

74A

Min

ority

of

4965

7946

7790

7373

45A

Con

sens

us o

f49

6595

8049

4999

9970

Not

Ver

y M

any

of30

3388

2130

3228

4725

A L

arge

Pro

port

ion

4040

6070

6576

8150

80A

Cle

ar M

anda

te33

6052

5010

040

9949

60

16A

ER

A \

RA

NG

E.X

LSP

age

13/

31/9

8

TA

BL

E 5

. Ran

ge o

f V

alue

s fo

r N

ine

Gro

ups

on T

wen

ty T

wo

Item

s

AB

CD

El

FG

I-I

IJ

K1 2

RA

NG

E O

F V

ALU

ES

3,

4-:

CIL

iarif

ifyin

g P

hras

e-,

:Gro

Up

1'.

Gro

iip 2

-.%

.:`G

roup

3'G

roU

P 4

,.G

roup

5 '

' Gro

Up

6". G

roup

7 .

Gro

Up

8G

roup

95

Age

30 to

49

35 to

57

20 to

30

31 to

66

25 to

54

22 to

43

22 to

43

38 to

54

6A

Few

0 to

25

3 to

80

3 to

50

2 to

12

3 to

80

2 to

25

3 to

60

3 to

60

0 to

30

7A

Dam

n F

ew0

to 1

02

to 9

02

to 2

42

to 1

00

to 1

52

to 1

31

to 1

01

to 1

01

to 7

58

A C

oupl

e2

to 2

2 to

90

2 to

20

2 to

22

to 8

02

to 1

002

to 9

02

to 9

02

to 8

09

Lots

0 to

100

10 to

100

15 to

100

4 to

87

0 to

70

15 to

100

4 to

100

4 to

100

5 to

100

10A

Maj

ority

51 to

90

51 to

90

10 to

90

10 to

99

51 to

90

51 b

100

40 to

98

40 to

98

25 to

100

11S

ever

al2

to 9

03

to 8

03

to 1

004

to 5

03

to 9

02

to 7

04

to 9

04

to 9

03

to 7

012

Man

y20

to 1

0015

to 8

010

to 1

004

to 8

020

to 9

020

b 9

96

to 9

06

to 9

05

to 1

0013

Mos

t51

to 9

951

to 9

560

to 1

0010

to 9

965

to 9

051

to 1

0010

to 1

0010

to 1

005

to 9

914

Alm

ost A

ll90

to 9

980

to 9

980

to 1

005

to 9

990

to 9

99

to 9

91

to 9

91

to 9

925

to 9

915

Har

dly

Any

0 to

20

2 to

90

1to

100

1to

30

0 to

51

to 6

91

to 7

51

to 7

51

to 8

016

A S

ubst

antia

l Maj

ority

51 to

99

20 to

90

60 to

98

10 to

100

0 to

75

45 to

98

8 to

98

8 to

98

20 to

95

17A

Sig

nific

ant N

umbe

r of

0 to

95

15 to

80

20 to

95

20 to

75

25 to

80

10 to

96

5 to

98

5 to

98

10 to

95

18V

irtua

lly A

ll0

to 9

980

to 9

975

to 9

920

to 9

90

to 1

0090

to 9

910

to 1

0010

to 1

0025

to 9

919

Alm

ost N

one

0 to

10

1to

70

1to

75

1to

10

0 to

51

to 5

0 to

90

0 to

90

0 to

80

20A

Sm

all N

umbe

r of

0 to

12

2 to

70

1to

80

2 to

20

0 to

12

2 to

26

2 to

25

2 to

25

3 to

30

21N

umer

ous

0 to

81

23 to

80

20 to

90

20 to

86

0 to

80

7 to

75

10 to

89

10 to

89

0 to

99

2 2

Nea

rly A

ll90

to 9

920

to 9

885

to 9

920

to 9

90

to 9

989

b 9

95

to 9

95

to 9

925

to 9

92

3A

Min

ority

of

0 to

49

15 to

80

1to

80

3 to

49

3 to

90

10 to

100

2 to

75

2 to

75

4 to

49

24A

Con

sens

us o

f51

to 1

0035

to 1

005

to 1

0020

to 1

0051

to 1

0051

to 1

001

to 1

001

to 1

0030

to 1

0025

Not

Ver

y M

any

of0

to 3

07

to 4

02

to 9

02

to 2

30

to 3

03

to 3

52

to 3

03

to 5

05

to 3

026

A L

arge

Pro

port

ion

of60

to 1

0045

to 8

530

to 9

020

to 9

025

to 9

020

to 9

63

to 9

045

to 9

55

to 8

527

A C

lear

Man

date

67 to

100

30 to

90

48 to

100

50 to

100

0 to

100

60 to

100

1to

100

51 to

100

30 to

90

9/4/

961

8P

age

111

:46

AM

19

Page/Date/Time 1 03-31-1998 12:37:31Database DAAERA\QualDB.S0

Histogram Section

150.0-

100.0-

50.0

APPENDIX E

0.0 r0 0 40.0 80.0

A_CoupleFIGURE 1. HISTOGRAM FOR "A COUPLE"

120.0

20

Page /Date/Time 1 03-31-1998 13:29:41Database D:\AERA \QuaIDB.S0

Histogram Section

100.0-

66.7-

Percent

33.3-

0.0-20.0

I 1 1

20.0 60.0 100.0

A_Few

FIGURE 2. HISTOGRAM FOR "A FEW"

Page/Date/Time 1 03-31-1998 12:54:55Database DAAERA\OualDB.S0

Histogram Section

6.wV

60.0-

40.0-

20.0-

0.00.0 40.0.0 80.0 120.0

A_MajorityFIGURE 3. HISTOGRAM FOR "A MAJORITY"

Page/Date/Time 1 03-31-1998 12:57:33Database D:\AERA \QuaIDB.S0

Histogram Section

"E'a)

12a)a.

60.0-

40.0-

20.0-

d----, t--.710.0-20.0

, I ,I I

73.3 120.026.7 7d.3

A_Substantial_Majority

FIGURE 4. HISTOGRAM FOR"A SUBSTANTIAL MAJORITY"

Page /Date/Time 1 03-31-1998 13:02:35Database D:\AERA \QuaIDB.S0

Histogram Section

30.0-

20.0-

Percent

10.0-

0.0-20.0 26.7 73.3

I 2

Numerous

FIGURE 5. HISTOGRAM FOR "NUMEROUS"

I 1

120.0


Histogram Section

30.0,

20.0-

Percent

10.0-

0.0 ,

-20.0 26.7 73.31 I 1

120.0Lots

FIGURE 6. HISTOGRAM FOR "LOTS"

25

Page/Date/TimeDatabase DAAERA

Histogram Section

50.0-

33.3-Za)Ua)o.

16.7-

0.0

1

\QuaIDB.S003-31-1998 13:04:42

00 40.0 80.0 120.0

A_Large_Proportion_of

FIGURE 7. HISTOGRAM FOR"A LARGE PROPORTION OF"

26


Histogram Section

120.0-

80.0-

Percent

40.0_

0.0 ,

-20.0

.. 1,'

26.7 73.3

Virtually_All

FIGURE 8. HISTOGRAM FOR"VIRTUALLY ALL"

1

120.0

U.S. Department of EducationOffice of Educational Research and Improvement (OERI)

National Library of Education (NLE)Educational Resources Information Center (ERIC)

REPRODUCTION RELEASE(Specific Document)

I. DOCUMENT IDENTIFICATION:

ERICTM028415

Title: W ORAS 4-A/ L T1, 6zR V A e- To r H 6- Sol? viryRESEVIRO#7?

Author(s): c'eAu i,i9v-REiti E. WALTel?SCorporate Source:prppE-Roive O., C.SEP. Aos / iv GEC ES ibt)IF-1-Er>

III:ST*RZC

Publication Date:

04 /6 -5'8

II. REPRODUCTION RELEASE:In order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announced in the

monthly abstract journal of the ERIC system, Resources in Education (RIE), are usually made available to users in microfiche, reproduced paper copy,and electronic media, and sold through the ERIC Document Reproduction Service (EDRS). Credit is given to the source of each document, and, ifreproduction release is granted, one of the following notices Is affixed to the document.

If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following three options and sign at the bottomof the page.

The sample slicker shown below will beaffixed to all Level 1 documents

1

PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL HAS

BEEN GRANTED BY

Sa


Level 1

Check here for Level 1 release, perrnitdng reproductionand dissemination in microfiche or other ERIC archival

media (e.g., electronic) endpaper copy.

The sample sticker shown below will beaffixed to all Level 2A documents

PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL IN

MICROFICHE, AND IN ELECTRONIC MEDIAFOR ERIC COLLECTION SUBSCRIBERS ONLY,

HAS BEEN GRANTED BY

2A


Level 2A

Check here for Level 2A release, permltdng reproductionand dissemination in microfiche and in electronic media

for ERIC archival collection subscribers only

The sample sticker shown below will beaffixed to all Level 28 documents

PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL IN

MICROFICHE ONLY HAS BEEN GRANTED BY

2B


Level 2B

Check here for Level 2B release, permittingreproduction and dissemination in microfiche only

Documents will be processed as Indicated provided reproduction quality permits.If permission to reproduce Is granted, but no box Is checked, documents will be processed at Level 1.

I hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminate this documentas indicated above. Reproduction from the ERIC microfiche or electronic media by persons other than ERIC employees and its systemcontractors requires permission from the copyright holder. Exception is made for non-profit reproduction by libraries and other service agenciesto satisfy information needs of educators in response to discrete inquiries.

here,-) 131- 4-Sign Sig

Organizatplease

d

cseT/F7tl7on

74. RecIAl L:ao

Chet McCall31 Pepperdine Univ., GSEP400 Corporate PointeCulver City, CA 90230

Telephone:3/0-'568* 2323 Fi'xio -54 8-5 9Ss-

E-Mail Address: 085iti-1i-ifiri ('cti 076.PI) NF. EDU (over)

III. DOCUMENT AVAILABILITY INFORMArTION (FROM NON-ERIC SOURCE):

If permission to reproduce is not granted to ERIC, or; if you wish ERIC to cite the availability of the document from another source, pleaseprovide the following information regarding the availability of the document. (ERIC will not announce a document unless it is publiclyavailable, and a dependable source can be specified. Contributors should also be aware that ERIC selection criteria are significantly morestringent for documents that cannot be made available through EDRS.)

Publisher/Distributor:

Address:

Price:

IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER:

If the right to grant this reproduction release is held by someone other than the addressee, please provide the appropriate name andaddress:

Name:

Address:

V. WHERE TO SEND THIS FORM:

Send this form to the following ERIC Clearinghouse:THE UNIVERSITY OF MARYLAND

ERIC CLEARINGHOUSE ON ASSESSMENT AND EVALUATION1129 SHRIVER LAB, CAMPUS DRIVE

COLLEGE PARK, MD 20742-5701Attn: Acquisitions

However, if solicited by the ERIC Facility, or if making an unsolicited contribution to ERIC, return this form (and the document beingcontributed) to:

ERIC Processing and Reference Facility1100 West Street, 2" Floor

Laurel, Maryland 20707-3598

Telephone: 301-497-4080Toll Free: 800-799-3742

FAX: 301-953-0263e -mail: [email protected]

WWW: http://ericfac.piccard.csc.com

EFF-088 (Rev. 9/97)PREVIOUS VERSIONS OF THIS FORM ARE OBSOLETE.