expanding educational opportunities worldwide - ets … · expanding educational ... practise these...

12
1 Listening. Learning. Leading. ETS Expanding Educational Opportunities Worldwide

Upload: vuonglien

Post on 03-Aug-2018

253 views

Category:

Documents


0 download

TRANSCRIPT

1

Listening. Learning. Leading.

ETS Expanding Educational

Opportunities Worldwide

2

On December 19, 1947, Henry Chauncey brought to life a concept first proposed a decade earlier by Harvard University President James Conant — that a single organisation devoted to education research and assessment could make fundamental contributions to the progress of education. Under Chauncey’s leadership — and through significant contributions from the American Council on Education, the Carnegie Foundation for the Advancement of Teaching and the College Entrance Examination Board — the organisation now known as ETS was formed.

Today, non-profit ETS is the largest private educational testing and measurement organisation in the world. We are at the forefront of educational research, the development of performance assessments, and the application of new methods and technologies to advance the quality and effectiveness of educational assessments and computer-based learning tools. ETS leads the way in advancing educational measurement through:

65 YEARS ETS: Leading the way for more than

Our Mission: Our Goals: Our Values:

To advance quality and equity in education by providing fair and valid assessments, research and related services. Our products and services measure knowledge and skills, promote learning and performance, and support education and professional development for everyone.

To be the global leader in providing fair and valid assessments, research and related products and services that help individuals, parents, teachers, educational institutions, businesses, governments, countries, states, school districts, measurement specialists and researchers.

Social responsibility, equity, opportunity and quality. We practise these values by listening to employers, governments, educators, parents and critics. We understand what employers and their current or potential employees need to be successful in the global economy. We provide solutions to countries that will help in the advancement of learning. We learn what students and the institutions they attend need. And we lead in the development of products and services to help teachers teach, students learn and parents measure the educational and intellectual progress of their children.

Henry Chauncey

3

👥 More than 3,000 employees work at ETS’s offices throughout the world. Of these, more than 1,100 of our professional staff have training and

expertise in education, psychology, statistics, psychometrics, computer sciences, sociology and humanities. More than 665 employees have advanced degrees, and over 610 hold doctorates. In addition, another 2,500 employees support ETS’s wholly owned subsidiary, Prometric™.

In 2001 ETS Global, a wholly owned subsidiary of ETS, was established to promote ETS tests, build and strengthen relations with educational and assessment leaders in Europe and offer customised assessments. A few years later ETS Global’s scope widened to make the TOEIC® tests available in more than 90 countries around the world and in building a network of offices and partners to support test takers and test users locally.

Today ETS Global is active in offering a variety of ETS’s assessments and tools that support English-language learning. It also continues to perform its outreach and client relationship functions for higher education and governments in Europe, Middle East, and Africa. Finally it continues to function as a bridge between the countries in which it is active and all of ETS’s assessment expertise true to the mission to advance quality and equity in education worldwide.

B Test developers

B Statisticians and psychometricians

B Leaders in developing performance assessments

B Experts in global assessment

B Researchers

B Education policy specialists

Our experts:

WHO we are

4

ETS’s activities and products fall into five broad areas of expertise:Assessment Development and Research: Our core capabilities in this area include designing assessments, developing test content, analysing and managing data for large-scale test programmes, and promoting the fair and valid use of test scores for all test takers, including those who have disabilities and those whose first language is not English.

Assessment Innovation: ETS is committed to ongoing research, not only to support and analyse existing assessments, but also to generate ideas for future assessment products and services and to contribute to the field of educational measurement. We have numerous patents for innovations in computerised essay scoring technology, computer-based test creation, answer- sheet scanning and Internet-based testing.

Policy Research: ETS is one of the foremost organisations in conducting research on, and evaluations of, public policy with the goals of improving the quality of instruction in schools, closing achievement gaps among under-represented student populations and investigating factors that adversely affect student populations at all levels.

Test Scoring: ETS has state-of-the-art capabilities for scoring multiple-choice questions and has a variety of methods for scoring “constructed-response” questions — test items that elicit open-ended responses, such as essays, speech and short written answers. These methods utilise:

B subject-matter experts to evaluate responses in live, face-to-face scoring sessions;

B our proprietary Online Scoring Network system to enable expert scorers to evaluate responses from remote locations, such as their homes or offices; and

B advanced, automated scoring systems that reduce the need for human scoring of constructed-response questions.

Instructional Products and Services: We support academic institutions, government agencies, corporations and individuals in more than 180 countries. Through our global subsidiaries and the ETS Preferred Network, we are positioned to serve the current and future needs of the global community.

ETS serves educators, employers, policy makers and learners of all levels, from school children to working professionals.

We develop, administer and score more than 50 million tests annually around the world, including the TOEFL®, TOEIC® and GRE® tests and The Praxis Series™ assessments.

WHAT we do

5

Assessments

B The TOEIC® Listening and Reading Test

B The TOEIC® Speaking and Writing Tests

B The TOEIC Bridge™ Test

Professional Development

B The Propell® Teacher Workshop for the TOEIC® Listening and Reading Test

B The Propell® Teacher Workshop for the TOEIC® Speaking and Writing Tests

B The Propell® Teacher Workshop for the TOEIC Bridge™ Test

Test Preparation

B The TOEIC® Official Learning and Preparation Course (for the TOEIC® Listening and Reading test)

B TOEIC® Speaking and Writing Online Preparation Tool

B Tactics for TOEIC® Listening and Reading Test

B Tactics for TOEIC® Speaking and Writing Tests

The TOEIC® Programme

PRODUCTS and SERVICESGlobal Workplace Assessments: The TOEIC® Programme Developed in 1979, the TOEIC® test measures the proficiency of test takers’ everyday English skills used in the workplace. With more than seven million tests administered in 2013, the TOEIC® tests are the global standard of measurement of workplace English-language

proficiency. Today, the TOEIC® Programme — used by more than 14,000 companies, organisations, government agencies and English language learning programmes worldwide — helps assess how well individuals comprehend and use English effectively in the workplace.

6

Assessments

B The TOEFL iBT® test

B The TOEFL® ITP Assessment Series

B The TOEFL Junior® Tests

B The TOEFL® Primary™ Tests

Professional Development

B The Propell® Workshop for the TOEFL iBT® Test

The TOEFL® Family of Assessments

Global Academic Assessments: The TOEFL® ProgrammeSupported by more than four decades of world renowned research, the TOEFL test is accepted by more than 9,000 institutions in 130 countries, including the top 100 universities in the United States, the top 20 universities in Canada, all of the Russell Group universities in the U.K.,

and all of the Group of Eight universities in Australia. With 100% academic content, the TOEFL test gives an accurate measure of a student’s ability to communicate in English in a university setting.

Free TOEFL iBT® Prep:

B TOEFL® Go Anywhere website

B The TOEFL® Test Prep Planner

B TOEFL iBT® Test Sample Questions

B TOEFL iBT® Quick Prep

B The TOEFL Journey® programme

Priced TOEFL iBT® Prep:

B TOEFL® Practice Online

B The Official Guide to the TOEFL® Test

B Official TOEFL iBT® Tests with Audio

B TOEFL iBT® Online Prep Course

B TOEFL® Value Packs

Test Preparation

ETS offers official test preparation resources to help improve students’ English skills and prepare for test success, including:

A growing range of preparation materials for other tests of the TOEFL® family are also available, including sample test questions and practice tests.

7

Global Programmes and ServicesETS provides a wide range of resources that help evaluate and build the skills of students, individuals and employees. There are also valuable learning tools that can be integrated into a comprehensive curriculum.

B The TFI™ Test (French proficiency)

B Criterion® Online Writing Evaluation Service

B Engaging English™ Service, featuring Lexile® measures, from MetaMetrics®

B Workforce Assessment

B EL Teach College and Graduate Programme Resources ETS offers a full range of academic assessments and services for higher education that enable administrators and faculty members to make the most of their resources and enhance student learning outcomes.

B The GRE® revised General Test and the GRE® Subject Tests

B GRE® Search Service

B ETS® Personal Potential Index (ETS® PPI)

B Criterion® Online Writing Evaluation Service

Resources for K–12 Education ETS develops programmes and services for U.S.-based K–12 educators, stakeholders and parents, including custom assessment programmes that help measure student learning and the content knowledge and teaching skills of teachers.

B Large-scale customised assessments for clients

B National Assessment of Educational Progress (NAEP)

B The Praxis Series™ Assessments

B The School Leadership Series

B The ParaPro Assessment

B Washington ProTeach Portfolio

Global Education SolutionsETS’s Global Institute and Centre for Global Assessments build on ETS’s long tradition of promoting learning and opportunity around the world. The ETS Global Institute serves as a central resource for educators and researchers by providing test-related training, consultation and educational solutions to international clients. The Centre for Global Assessment develops, manages and delivers large-scale assessment projects throughout the world.

8

Purpose of Standardised TestsOur goal at ETS and the purpose of standardised tests are the same — to provide fair, valid and reliable assessments that produce meaningful results. Standardised testing, if done carefully and with a high degree of quality assurance, can eliminate bias and prevent unfair advantages by testing the same or similar information under the same testing conditions.

Standardised tests allow the comparison of test takers from different areas of the state, the country and the world. What a test can do depends on whether it is well-designed for a particular purpose. Well-designed tests can provide results that can be used in a variety of meaningful ways, such as:

Purpose: Tests results can be used to:

Licensure or Certification Verify whether someone has the necessary knowledge and skills to be a qualified practitioner or to be given advanced standing in an occupation or profession

Admissions Inform decisions about which people should be selected for entrance to an educational institution

Placement Determine which courses or level of a course a student should take

Employment Inform decisions on the hiring, placement and promotion of potential and current employees

Curriculum-based End of Course Testing Determine whether students have mastered the objectives of the course taken

Exit Testing Find out whether students have learned the amount necessary to graduate from a level of education

Policy Tools Provide data to policymakers that helps them make decisions regarding funding, class size, curriculum adjustments, teacher development and more

Course Credit Indicate whether a student should receive credit for a course he or she didn’t take through demonstration of course content knowledge

Accountability Hold various levels of the education system responsible for test results that indicate if students learn what they should have learned

TESTINGHow ETS approaches

9

ETS develops assessments that are of the highest quality, accurately measure the necessary knowledge and skills, and are fair to all test takers. Creating a fair, valid and reliable test is a complex process that involves multiple checks and balances.

Professionals including test specialists, test reviewers, editors, teachers and specialists in the subject or skill being tested are involved in developing every test question. All questions are put through multiple, rigorous reviews and meet the highest standards for quality and fairness in the testing industry.

To help you further understand our process, here is an overview of the key steps ETS takes when developing a new test:

Step 1: Defining Objectives

Educators, licensing boards or professional associations identify a need to measure certain skills or knowledge. Once a decision is made to develop a test to accommodate this need, test developers ask some fundamental questions:

B Who will take the test and for what purpose?

B What skills and/or areas of knowledge should be tested?

B How should test takers be able to use their knowledge?

B What kinds of questions should be included? How many of each kind?

B How long should the test be?

B How difficult should the test be?

Step 2: Item Development Committees

The answers for the questions in Step 1 are usually completed with the help of item development committees, which typically consist of educators and/or other professionals appointed by ETS with the guidance of the sponsoring agency or association. Responsibilities of these item development committees may include:

B defining test objectives and specifications,

B helping ensure test questions are unbiased,

B determining test format (e.g., multiple-choice, essay, constructed-response, etc.),

B considering supplementary test materials,

B reviewing test questions, or test items, written by ETS staff,

B writing test questions.

How Tests and Test Questions are DEVELOPED

10

Step 3: Writing and Reviewing Questions

Each test question — written by ETS staff or item development committees — undergoes numerous reviews and revisions to ensure it is as clear as possible, that it has only one correct answer among the options provided on the test and that it conforms to the style rules used throughout the test. Scoring guides for open- ended responses, such as short written answers, essays and oral responses, go through similar reviews.

Step 4: The Pretest

After the questions have been written and reviewed, many are pretested with a sample group similar to the population to be tested. The results enable test developers to determine:

B the difficulty of each question

B if questions are ambiguous or misleading

B if questions should be revised or eliminated

B if incorrect alternative answers should be revised or replaced

Step 5: Detecting and Removing Unfair Questions

To meet the stringent ETS Standards for Quality and Fairness guidelines, trained reviewers must carefully inspect each individual test question, the test as a whole and any descriptive or preparatory materials to ensure that language, symbols, words, phrases and content generally regarded as sexist, racist or otherwise inappropriate or offensive to any subgroup of the test-taking population are eliminated.

ETS statisticians also can identify questions on which two groups of test takers who have demonstrated similar knowledge or skills perform differently on the test through

a process called differential item functioning (DIF). If one group performs consistently better than another on a particular question, that question receives additional scrutiny and may be deemed biased or unsatisfactory. Note: If people in different groups actually differ in their average levels of relevant knowledge or skills, a fair test question will reflect those differences.

Step 6: Assembling the Test

After the test is assembled, it is reviewed by other specialists, committee members and sometimes other outside experts. Each reviewer answers all questions independently and submits a list of correct answers to the test developers. The lists are compared with the ETS answer keys to verify that the intended answer is, indeed, the correct answer. Any discrepancies are resolved before the test is published.

Step 7: Making Sure - Even After the Test Is Administered - That the Test Questions are Functioning Properly

Even after the test has been administered, statisticians and test developers review to make sure that test questions are working as intended. Before final scoring takes place, each question undergoes preliminary statistical analysis and results are reviewed question by question. If a problem is detected, such as the identification of a misleading answer to a question, corrective action, such as not scoring the question, is taken before final scoring and score reporting takes place.

Tests are also reviewed for reliability. Performance on one version of the test should reasonably predict performance on any other version of the test. If reliability is high, results will be similar no matter which version a test taker completes.

11

ETS uses both human scoring and automated scoring to score standardised tests.

How Standardised Tests are SCORED

Human Scoring

Human-scored tests are scored manually rather than by machine and require human judgment. At ETS, test scorers are carefully selected and go through rigorous training to ensure the accuracy of their work.

Automated Scoring

There are two types of automated scoring used at ETS:

B machine scoring of multiple-choice test questions

B automated scoring of open-ended responses, such as short written answers, essays and recorded speech

We have developed a number of automated scoring technologies through extensive research in natural language processing (NLP) that spans more than a decade. They include:

B the e-rater® engine

B the c-rater™ system

B the m-rater engine

B the SpeechRaterSM engine

B the Text Adaptor tool

We are innovators in the field of automated scoring and have incorporated these technologies into many of our testing programmes, products and services, including the GRE® General Test, the TOEFL iBT® test and the Criterion® Online Writing Evaluation service.

Copyright © 2014 by Educational Testing Service. All rights reserved. ETS, the ETS logo, CRITERION, GRE, PROPELL, TOEFL, TOEFL IBT, TOEFL JUNIOR, TOEFL PRIMARY and TOEIC are registered trademarks of Educational Testing Service (ETS) in the United States and other countries and used under license. THE PRAXIS SERIES, TOEFL JOURNEY are trademarks of ETS. Lexile and MetaMetrics are registered trademarks of MetaMetrics, Inc. All other trademarks are property of their respective owners. MAR169

www.etsglobal.org

Educational Testing Service

Listening to policy makers, educators, parents and criticsLearning what companies, institutions, government agencies and students need

Leading the development of new and innovative products and services

@ETSGlobalETS Global