Build Bright UniversityBuild Bright UniversityLanguage Testing and AssessmentLanguage Testing and Assessment

Chapter- 5Chapter- 5

Standards-based AssessmentStandards-based AssessmentPrepared by Kheang Sokheng

Ph.D Candidate and MEd in TESOL

Mid 20th CenturyMid 20th Century

Standardized tests had unchallenged Standardized tests had unchallenged popularity and growth.popularity and growth.

Standardized tests brought convenience, Standardized tests brought convenience, efficiency, air of empirical science.efficiency, air of empirical science.

Tests were considered to be a way of Tests were considered to be a way of making reforms in educationmaking reforms in education

Quickly and cheaply assessing students Quickly and cheaply assessing students became a political issue.became a political issue.

Late 20th CenturyLate 20th Century

*There was possible inequity and disparity *There was possible inequity and disparity between the tests in such tests and the ones between the tests in such tests and the ones they teach in classes.they teach in classes.**The claims in mid-20th century began to be The claims in mid-20th century began to be questioned/criticised in all areas.questioned/criticised in all areas.*Teachers were in the leading position of those *Teachers were in the leading position of those challenges.challenges.

The Last 20 YearsThe Last 20 Years*Educators become aware of weaknesses in *Educators become aware of weaknesses in standardized testing: standardized testing: TheyThey were not accurate were not accurate measures of achievement and success and measures of achievement and success and they were not based on carefully framed, they were not based on carefully framed, comprehensive and validated standards of comprehensive and validated standards of achievement.achievement.*A movement has started to establish *A movement has started to establish standards to assess students of all ages and standards to assess students of all ages and subject-matter areas.subject-matter areas.*There have been efforts on basing the *There have been efforts on basing the standardised tests on clearly specified criteria standardised tests on clearly specified criteria for each content area being measured.for each content area being measured.

Crit icismCrit icism Some teachers claimed that those tests were Some teachers claimed that those tests were

unfair there were dissimilarity between the unfair there were dissimilarity between the content & task of the tests & what they were content & task of the tests & what they were teaching in their classes.teaching in their classes.

SolutionsSolutions By becoming aware of these weaknesses, By becoming aware of these weaknesses,

educators started to establish some educators started to establish some standards on which students of all ages & standards on which students of all ages & subject matter areas might be assessed. subject matter areas might be assessed.

SolutionsSolutions most departments of education at all state most departments of education at all state

level in the US have specified the level in the US have specified the appropriate standards (criteria, objectives) appropriate standards (criteria, objectives) for each grade level(pre-school to grade 12) for each grade level(pre-school to grade 12) and each content area (math, sciences, and each content area (math, sciences, arts…)arts…)

The construction of standards makes The construction of standards makes possible concordance between possible concordance between standardized test specification and the standardized test specification and the goals and objectives (ESL, ESOL, goals and objectives (ESL, ESOL, ELD,ELLs). LEP has now been discarded ELD,ELLs). LEP has now been discarded because of the negative connotation of the because of the negative connotation of the word limited.)word limited.)

ELD STANDARDSELD STANDARDS

The process of designing and conducting The process of designing and conducting appropriate periodic reviews of appropriate periodic reviews of ELDELD standards involves dozens of curriculum and standards involves dozens of curriculum and assessment specialists, teachers, and assessment specialists, teachers, and researchers (researchers (Fields, 2000; Kuhlman, 2001Fields, 2000; Kuhlman, 2001))

In creating such “ benchmarks for In creating such “ benchmarks for accountability” (accountability” (O’Malley & Valdez PierceO’Malley & Valdez Pierce, , 19961996), there is a tremendous responsibility to ), there is a tremendous responsibility to carry out a comprehensive study of a carry out a comprehensive study of a number of domains: number of domains:

ELD STANDARDSELD STANDARDS

literally thousands of categories of language literally thousands of categories of language ranging from phonology at one end of a ranging from phonology at one end of a continuum to discourse, pragmatic, continuum to discourse, pragmatic, functional and sociolinguistic elements at the functional and sociolinguistic elements at the other end;other end;

specification of ELD students’ needs are, at specification of ELD students’ needs are, at thirteen different grade levels, for thirteen different grade levels, for succeeding in their academic and social succeeding in their academic and social development;development;

a consideration of what is a realistic number a consideration of what is a realistic number and scope of standards to be included withinand scope of standards to be included within

ELD STANDARDSELD STANDARDS

a given curriculum;a given curriculum;a separate set of standards(qualifications, a separate set of standards(qualifications, expertise, training) for teachers to teach ELD expertise, training) for teachers to teach ELD students successfully in their classrooms; and students successfully in their classrooms; and a thorough analysis of the means available to a thorough analysis of the means available to assess student attainment of those standards.assess student attainment of those standards.Standards-setting is a global challenge. In Standards-setting is a global challenge. In many countries, English is now a required many countries, English is now a required subject starting as early as the first grade in subject starting as early as the first grade in some countries and by seventh grade in virtually some countries and by seventh grade in virtually every country worldwide. every country worldwide.

ELD ASSESSMENTELD ASSESSMENT

The development of standards obviously The development of standards obviously implies the responsibility for correctly implies the responsibility for correctly assessing their attainment.assessing their attainment.

It is found that the standardized tests of the It is found that the standardized tests of the past decades were not in line with newly past decades were not in line with newly developed standardsdeveloped standards the interactive process the interactive process not only of developing standards but also of not only of developing standards but also of creating standards-based assessment creating standards-based assessment started. started.

Specialists design, revise and validate many Specialists design, revise and validate many tests.tests.

ELD ASSESSMENTELD ASSESSMENT The California English Language Development The California English Language Development

Test (CELDT) is a battery of instruments Test (CELDT) is a battery of instruments designed to assess attainment of ELD standards designed to assess attainment of ELD standards across grade level. (not publicly available)across grade level. (not publicly available)

LLanguage and Literacy Assessment Rubric, in anguage and Literacy Assessment Rubric, in which multiple forms of evidence of students’ which multiple forms of evidence of students’ work are collected. work are collected.

Teachers observe students year-round and Teachers observe students year-round and record their observations on scannable forms. record their observations on scannable forms.

CASAS AND SCANSCASAS AND SCANS CASAS:CASAS: (Comprehensive Adult Student (Comprehensive Adult Student Assessment System): Assessment System): Designed to provide broadly based Designed to provide broadly based assessments of ESL curricula across assessments of ESL curricula across US.US.It includes more than 80 standardized It includes more than 80 standardized assessment instruments used to;assessment instruments used to; *place Ss in programs*place Ss in programs *diagnose learners’ *diagnose learners’ needsneeds *monitor progress*monitor progress *certify mastery of *certify mastery of functional skillsfunctional skills

CASAS AND SCANSCASAS AND SCANS At higher level of education (colleges, adult At higher level of education (colleges, adult

and language schools, workplace)and language schools, workplace) SCANS: (Secretary’s Commissions in SCANS: (Secretary’s Commissions in

Achieving Necessary Skills): Achieving Necessary Skills): outlines competencies necessary for outlines competencies necessary for

language in the workplacelanguage in the workplace the competencies are acquired and the competencies are acquired and

maintained through training in basic skills(4 maintained through training in basic skills(4 skills); skills);

thinking skills (reasoning & problem solving); thinking skills (reasoning & problem solving);

CASAS AND SCANSCASAS AND SCANS personal qualities (self-esteem & sociability)personal qualities (self-esteem & sociability) Resources (allocating time, materials, staff Resources (allocating time, materials, staff

etc.)etc.) Interpersonal skills, teamwork, customer Interpersonal skills, teamwork, customer

service etc.service etc. Information processing, evaluating data, Information processing, evaluating data,

organising files etc,organising files etc, Systems, understanding social and Systems, understanding social and

organizational system,organizational system, Technology use and applicationTechnology use and application

TEACHER STANDARDS TEACHER STANDARDS KuhlmanKuhlman (2001) emphasized the (2001) emphasized the

importance of importance of teacher standards in three teacher standards in three domainsdomains::

1.1. Linguistic and language developmentLinguistic and language development2.2. Culture and interrelationship between Culture and interrelationship between

language and culturelanguage and culture3.3. Planning and managing instructionsPlanning and managing instructions Professional teaching standards have also Professional teaching standards have also

been the focus of several committees in the been the focus of several committees in the international association of Teachers of international association of Teachers of English to Speakers of Other Languages English to Speakers of Other Languages (TESOL).(TESOL).

4. Make appropriate evaluations of 4. Make appropriate evaluations of

different kinds of itemsdifferent kinds of items.. Item facility (IF) – % of people who give the Item facility (IF) – % of people who give the

right answerright answer Item Discrimination (IDis) – indicates the Item Discrimination (IDis) – indicates the

extent to which success on an item extent to which success on an item corresponds to success on the whole test.corresponds to success on the whole test.

Item Difficulty (ID) - finding out the % of Item Difficulty (ID) - finding out the % of people who get the item right in the try-out people who get the item right in the try-out group.group.

Consequences of standards-based Consequences of standards-based and standardized testing and standardized testing

Positive Positive High level of practicality and reliabilityHigh level of practicality and reliabilityProvides insights into academic performanceProvides insights into academic performanceAccuracy in placing a number of test takers Accuracy in placing a number of test takers on to a norm referenced scaleon to a norm referenced scaleOngoing construct validation studiesOngoing construct validation studies

Negative Negative They involve a number of test biasesThey involve a number of test biases

Negative Negative A small but significant number of test A small but significant number of test

takers are not assessed fairly nor theytakers are not assessed fairly nor they are are assessed accuratelyassessed accurately

Fostering extinct motivationFostering extinct motivation Multiple intelligence are not consideredMultiple intelligence are not considered There is danger of test driven learning and There is danger of test driven learning and

teachingteaching In general performance is not directly In general performance is not directly

assessed.assessed.

Test BiasTest Bias Standardized tests involve many test bias Standardized tests involve many test bias

(language, culture, race, gender,(language, culture, race, gender, learning learning styles)styles)

National Centre for Fair and Open Testing National Centre for Fair and Open Testing claims of tests bias from; teachers, parents, claims of tests bias from; teachers, parents, students, and legal consultants. (reading students, and legal consultants. (reading texts, listening stimulus) texts, listening stimulus)

Standardized tests do not promote logical-Standardized tests do not promote logical-mathematical and verbal linguistic to the mathematical and verbal linguistic to the virtual exclusions of the other virtual exclusions of the other contextualized, integrative intelligence. contextualized, integrative intelligence.

Test BiasTest Bias

(some learners may need to be (some learners may need to be assessed with interviews, portfolios, assessed with interviews, portfolios, samples of work, demonstrations, samples of work, demonstrations, observation reports) more formative observation reports) more formative assessment rather than summative.assessment rather than summative.That would solve test bias problems but That would solve test bias problems but it is difficult to control it in standardized it is difficult to control it in standardized items.items.

Test-driven learning and teachingTest-driven learning and teaching It is another consequence of standardized It is another consequence of standardized

testing. When students know that one single testing. When students know that one single measure of performance will determine their measure of performance will determine their lives they are less likely to take positive lives they are less likely to take positive attitudes towards learning. Extrinsic motivation attitudes towards learning. Extrinsic motivation is not intrinsic.is not intrinsic.

Teacher are also affected from test-driven Teacher are also affected from test-driven policies. They are under pressure to make sure policies. They are under pressure to make sure their Ss excelled in the exam, at the risk of their Ss excelled in the exam, at the risk of ignoring other objectives in the curriculum. A ignoring other objectives in the curriculum. A more serious effect was to punish schools with more serious effect was to punish schools with lower-socioeconomic neighbourhood.lower-socioeconomic neighbourhood.

ETHICAL ISSUES: CRITICAL ETHICAL ISSUES: CRITICAL LANGUAGE TESTINGLANGUAGE TESTING

One of by-products of rapid growing testing One of by-products of rapid growing testing industry is the danger of an abuse of power. industry is the danger of an abuse of power.

Shohamy (1997, p.2) further defines the Shohamy (1997, p.2) further defines the issue: “Tests represent a social technology issue: “Tests represent a social technology deeply embedded in education, government deeply embedded in education, government and business; tests are most powerful as and business; tests are most powerful as they are often the single indicators for they are often the single indicators for determining the future of individuals.”determining the future of individuals.”

Teachers can demonstrate standards in Teachers can demonstrate standards in their teaching. their teaching.

ETHICAL ISSUES: CRITICAL ETHICAL ISSUES: CRITICAL LANGUAGE TESTINGLANGUAGE TESTING

Teachers can be assessed through their Teachers can be assessed through their classroom performanceclassroom performance

Performance can be detailed with ‘indicators’: Performance can be detailed with ‘indicators’: examples of evidence that the teacher can examples of evidence that the teacher can meet a part of a standard.meet a part of a standard.

Student learning is at the heart of the Student learning is at the heart of the teacher’s performanceteacher’s performance..

The issues of critical language testing are The issues of critical language testing are numerousnumerous