masurement and assessment in education

7/30/2019 Masurement and Assessment in Education

1/9

M easurement and Assessmentin EducationCecil R. ReynoldsTexas A&M University

Ronald B. LivingstonUniversity of Texas at Tyler

Victor W illsonTexas A&M University

Boston New York San FranciscoMexico City M ontreal Toronto London Madrid Munich Paris

Hong Kong Singapore Tokyo Cape Town Sydney


2/9

CONTENTS

Preface xiii

Introduction to Educational Assessment 1The Language of Assessment 2

Tests, Measurement, and Assessment 3 / Types of Tests 4 / Typesof Score Interpretations 8Assumptions ofEducational Assessmen t 9

Psychological and Educational Constructs Exist 9 / Psychological andEducational Constructs Can Be Measured 10 / Although We Can M easureConstructs, Our Measurement ls Not Perfect 10 / There Are Different W aysto Measure Any Given Construct 10 / All Assessment Procedures HaveStrengths and Limitations 10 / Multiple Sources of Information ShouldBe Part of the Assessment Process 11 / Performance on Tests Can BeGeneralized to Nontest Behaviors 11 / Assessment Can Provide InformationThat Helps E ducators Make Better Educational De cisions 11 / AssessmentsCan Be Conducted in a Fair Mann er 11 / Testing and Assessm ent Can BenefitOur Educational Institutions and Society as a Whole 12Participants in the Assessment Process 13

People W ho Develop Tests 13 / People Who Use Tests 14 / PeopleW ho Take Tests 14 / Other People Involved in the Assessment P rocess 15Common Applications ofEducational A ssessments 15

Student E valuations 15 / Instructional Decisions 16 / Selection,Placement, and Classification Decisions 16 / Policy Decisions 17 /Counseling and Guidance Decisions 18What Teachers Need to Know about Assessment 18

Teachers Should Be Proficient inSelecting Professionally Developed A ssessmentProcedures A ppropriate for Making Instructional Decisions 19 / TeachersShould Be Proficient inDeveloping Assessment Procedures Approp riate for MakingInstructional Decisions 19 / Teachers Should Be Proficient in Adm inistering,Scoring, and Interpreting Professionally Developed and Teacher-Made Assessm entProcedures 20 / Teachers Should Be Proficient in Using AssessmentResults When Mak ing Educational Decisions 20 / Teachers Should BeProficient in Developing Valid Grading Procedures That Incorporate Assessm entInformation 20 / Teachers Should Be Proficient inCommunicating AssessmentResults 20 / Teachers Should Be Proficient in Recognizing U nethical, Illegal,and Other Inappropriate Uses of Assessment P rocedures or Information 21Educational Assessment in the Tw enty-First Century 21

Com puterized Adaptive Testing (CAT) and Other Technological Advances 22 /"Authentic" or Complex-Performance Assessments 22 / Educational


3/9

V i C O N T E N T SAccoun tability and High-Stakes Assessm ent 24 / Trends in the Assessm entof Stude nts with Disab ilities 24

Summary 26

TL The Basic Mathematics of Measurement 31The Role of M athematics in Assessment 31Scales of Measurem ent 32

W hat Is M easurement? 32 / Nom inal Scales 33 / Ordinal Scales 33Interval Sca les 34 / Ratio Scales 34

Th e Des cription of Test Scor es 36Distributions 36 / Measures of Central Tendency 40 /M easures of Variability 43

Cor relation Coefficients 49Scatterplots 50 / Correlation and Prediction 52 / Types of Co rrelationCoefficients 52 / Correlation versus Causa lity 52

Sum m ary 55

3 The Meaning of Test Scores 58Norm -Referenced and Cri terion-Referenced Score Interpretat ions 59

Norm -Referenced Interpretations 60 / Criterion-Referenc edInterpretations 76

Norm -Referenced, Cri terion-Referenced, or Both? 79Qua litative Des cription of Score s 81Sum m ary 82

4 Reliability for Teachers 85Errors of Me asurem ent 86

Sources of Me asurement Error 87Me thods of Est imating Rel iabi li ty 90

Test-Retest Reliability 92 / Alternate-Form Reliability 93 / Internal-Consistency Reliability 93 / Inter-rater Reliability 97 / Reliabilityof Com posite Scores 99 / Selecting a Reliability Coefficient 99 /Evaluating Reliability Coefficients 101 / How to Improve Reliability 103 /Special Prob lems in Estimating Reliability 105The Standard Error of M easure men t 106

Evaluating the Standard Error of Me asureme nt 108


4/9

CONTENTS VIlReliab ility: Practical Strateg ies for Teachers 111Summary 114

Validity for Teachers 117Threats to Validity 118Reliab ility and Validity 119"Types of Validity" versus "Types of Validity Evid ence" 120Types of Validity Eviden ce 123

Evidenc e Based on Test Content 123 / Evidence Based on Relations to OtherVariables 126 / Evidence Based on Internal Structure 133 / EvidenceBased on Response Processes 134 / Evidence Based on Conseq uences ofTesting 134 / Integrating Evid ence of Validity 135Validity: Practical Strategies for Teachers 137Summary 138

Item An alysis for Teachers 141Item Difficulty Index (or Item Difficulty Level) 142

Spec ial Asse ssme nt Situation s and Item Difficulty 144Item Discrimination 144

Discrim ination Index 145 / Item-Total Correlation Coefficients 147 /Item Discrimination on Mastery Tests 149 / Item Analysisof Speed Tests 150

Distracter Analysis 151How Distra cters Influence Item Difficulty and Discrim ination 152

Item An alys is: Practical Strategies for Teachers 153Using Item Ana lysis to Improve Items 155Item Analysis of Performance Assessments 157Qualitative Item Analysis 158Using Item Ana lysis to Improve Classroom Instruction 159Summary 161

7 The Initial Steps in Develop ing a Classroom Test 163Characteristics of Educational Objectives 165

Scope 165


5/9

viii CONTENTSTaxonomy of Educational Objectives 166

Cognitive Domain 166 / Affective Domain 169 /Psychomotor Domain 170

Behavioral versus Nonbehavioral Educational Objectives 171Writing Educational Objectives 172Developing a Table of Specifications (or Test Blueprint) 173Implementing the Table of Specificationsand Developing an Assessment 175

Norm-Referenced versus Criterion-Referenced Score Interpretations 176Selecting Which Types of Items to Use 176 / Putting the AssessmentTogether 180

Preparing Your Students and Administering the Assessment 183Summary 185

O The Development and Use of Selected-Response Items 188Multiple-Choice Items 189

Guidelines for Developing Multiple-Choice Items 190 / Strengths andWeaknesses ofMultiple-Choice Items 199

True-False Items 204Guidelines for Developing True-False Items 205 / Strengths andWeaknesses ofTrue-False Items 206

Matching Items 208Guidelines for Developing Matching Items 209 / Strengths andWeaknesses of Matching Items 211Summary 213

The Development and Useof Constructed-Response Items 215

Oral Testing: The Oral Essay as a Precursorof Constructed-Response Items 216Essay Items 217

Purposes of Essay Items 217 / Essay Items at Different Levelsof Complexity 219 / Restricted-Response versus Extended-ResponseEssays 221 / Guidelines for Developing Essay Items 222 /Strengths and Weaknesses of Essay Items 223 / Guidelines forScoring Essay Items 226Short-Answer Items 230

Guidelines forDeveloping Short-Answer Items 232 / Strengths andWeaknesses of Short-Answer Items 234


6/9

CONTENTS IX

A Final Note: Constructed-Response versus Selected-Response Items 235Summary 236

l i ) Performance Assessments and Portfolios 238What Are Performance Assessments? 239Guidelines for Developing Effective Performance Assessments 245

Selecting Appropriate Performance Tasks 245 / Developing Instructions 249Developing Procedures for Evaluating Responses 249 / ImplementingProcedures to Minimize Errors in Rating 254Strengths and Weaknesses of Performance Assessments 258Portfolios 262

Guidelines for Developing Portfolio Assessments 262 / Strengths andWeaknesses of Portfolio Assessments 264

Summary 266

11 Assigning Grades on the Basisof Classroom Assessments 270

Feedback and Evaluation 271Formal and Informal Evaluation 274 / The Use of Formative Evaluationin Summative Evaluation 274

Reporting Student Progress: Which Symbols to Use? 275The Basis for Assigning Grades 277Frame of Reference 278

Norm-Referenced Grading (Relative Grading) 278 / Criterion-ReferencedGrading (Absolute Grading) 280 / Achievement in Relation to Improvement orEffort 281 / Achievem ent Relative to Ability 282 / Recommendation 282Combining Grades into a Composite 283Informing Students of Grading System 288Parent Conferences 288Summary 289

Standardized Achievement Tests inthe Era of High-Stakes Assessment 291

Group-Administered Achievement Tests 294Com mercially Developed G roup Achievement Tests 295 / State-DevelopedAchievement Tests 304 / Best Practices in Using Standardized AchievementTests in Schools 306


7/9

CONTENTS

Individual Achievement Tests 315Selecting an Achievement Battery 317Summary 318

1 3 The Use of Aptitude Tests in the Schools 320A Brief History of Intelligence Tests 323The Use of Aptitude and Intelligence Tests in Schools 326

Aptitude-Achievement Discrepancies 327Major Aptitude/Intelligence Tests 329

Group A ptitude/ Intelligence T ests 329 / Individual Aptitude/IntelligenceTests 334 / Selecting Aptitude/Intelligence Tests 342

College Admission Tests 343Summary 344

1 4 Assessment of Behavior and Personality 347Assessing Behavior and Personality 349

Response Sets 349 / Assessment of Behavior and Personalityin the Schools 350Behavior Rating Scales 352

Behavior Assessment System for ChildrenTeacher Rating Scale and Parent RatinScale (TRS and PRS) 353 / Conners Rating ScalesRevised (CRS-R) 357Child Behavior Checklist and Teacher Report Form (C BCL and TRF ) 358

Self-Report Measures 360Behavior Assessment System for ChildrenSelf-Report of Personality(SRP) 360 / Youth Self-Report (YSR) 364

Projective Techniques 364Projective Drawings 366 / Sentence Com pletion Tests 367 / ApperceptTests 367 / Inkblot Techniques 367

Summary 369

15 Assessment Accommodationsfor Students with Disabilities 371

Major Legislation That Impacts the Assessmentof Students with Disabilities 373


8/9

C O N T E N T S X I

Individuals with Disabilities Education Act (IDEA) 373IDEA Categories of Disabilities 375

Section 504 378The Rationale for Assessment Accomm odations 379Wh en Are Accomm odations Not Appropriate or Necessary? 380Strategies for Accomm odations 380

Modifications of Presentation Format 381 / Modifications of ResponseForm at 381 / Mo difications of Timing 383 / Mo dification of Setting 383Adap tive Devices and Supp orts 383 / Using Only a Portion of a Test 384Using Alternate Assessments 385Determining What Accomm odations to Provide 385Reporting Results of Modified Assessments 387Summary 390

l u The Problem of Bias in Educational Assessment 395What Do We Mean by Bias? 398Past and Present Concerns: A Brief Look 399The Controversy over Bias in Testing: Its Orig in,What It Is, and What It Is Not 399Cultural Bias and the Nature of Psychological Testing 405Objections to the Use of Educational and PsychologicalTests with Minority Students 406

Inappropriate Content 406 / Inappropriate Standardization Samples 407Exam iner and Language Bias 407 / Inequitable Social Consequences 407M easu reme nt of Different Con structs 407 / Differential PredictiveValidity 407 / Qualitatively Distinct Aptitude and Personality 407The Problem of Definition in Test Bias Research: Differential Validity 408Cultural Loa ding, Cultural Bias, and Culture-Free Tests 408Inappropriate Indicators of Bias: Mean Differencesand Equivalent Distributions 409Bias in Test Content 410Bias in Other Internal Features of Tests 413Bias in Prediction and in Relation to Variables External to the Test 415Summary 420


9/9

xii CONTENTS

1 7 Best Practices in Educ ational Assessment 422Guidelines for Developing Assessments 424Guidelines for Selecting Published Assessm ents 425Guidelines for Adm inistering Assessments 429Guidelines for Scoring Assessments 432Guidelines for Interpreting, Using, andCom municating Assessment Results 434Responsibilities of Test Takers 435Summary 437

APPENDIX A: Sum m ary Statem ents of The StudentEvaluation Standards (JC SE E, 2003) 441APPENDIX B: Code of Professional Responsibilitiesin Educational Measurement ( N C M E , 1995) 444APPENDIX C: Code of Fair Testing Practicesin Education (JCTP , 1988) 452APPENDIX D: Rights and Responsibilities of Test Takers:Guidelines and Expectations (JCTP, 1998) 456APPENDIX E: Standards for Teacher Com petence in EducationalAssessment of Students (AFT, NC M E, and NEA , 1990) 465APPENDIX F: Prop ortions of Area un der the Nor m al Cu rve 471APPENDIX G: An swers to Practice Problem s 475

References 477Index 483

masurement and assessment in education

Documents