1997 edition - list-english.rulist-english.ru/pdf/tests/testandscoremanual.pdf · 1997 edition ®;...

TEST&SCORE

MANUAL

1997 Edition

®

Больше ресурсов по изучению английского на List-English.ru

http://list-english.ru/

T he TOEFL Test and Score Manual has been prepared for deans,admissions officers and graduate department faculty, adminis-

trators of scholarship programs, ESL teachers, and foreign studentadvisers. The Manual provides information about the interpretationof TOEFL® scores, describes the test format, explains the operationof the testing program, and discusses program research activities andnew testing developments.

This edition of the Test and Score Manual updates material inthe 1995-96 edition, providing a description of revisions to the testintroduced in July 1995, and other information of interest to scoreusers. With the exception of “Program Developments” on page 10,information in this Manual refers specifically to the paper-and-pencilTOEFL test.

As this edition goes to press (summer 1997), a computer-based TOEFL test is under development and planned forintroduction in the second half of 1998 (see page 11). Moreinformation about the computer-based test and score interpreta-tion will appear on the TOEFL website at http://www.toefl.organd through new publications as it becomes available.

Add your name to the TOEFL web list (on “Educators”directory page) and receive e-mail announcements as they arereleased.

TOEFL Programs and ServicesInternational Language ProgramsEducational Testing Service



Educational Testing Service is an Equal Opportunity/Affirmative Action Employer.

Copyright © 1997 by Educational Testing Service. All rights reserved.

EDUCATIONAL TESTING SERVICE, ETS, the ETS logo, GRADUATE RECORD EXAMINA-TIONS, GRE, SLEP, SPEAK, THE PRAXIS SERIES: PROFESSIONAL ASSESSMENTS FORBEGINNING TEACHERS, TOEFL, the TOEFL logo, TSE, and TWE are registered trademarksof Educational Testing Service.

COLLEGE BOARD and SAT are registered trademarks of the College Entrance Examination Board.

GMAT and GRADUATE MANAGEMENT ADMISSION TEST are registered trademarks of theGraduate Management Admission Council.

SECONDARY SCHOOL ADMISSION TEST and SSAT are registered trademarks of the SecondarySchool Admission Test Board.

No part of this publication may be reproduced or transmitted in any form or by any means,electronic or mechanical, including photocopy, recording, or any information storage and retrievalsystem, without permission in writing from the publisher. Violators will be prosecuted in accordancewith both United States and international copyright and trademark laws.

Permissions requests may be made on-line at http://www.toefl.org/copyrigh.html or sent to:Proprietary Rights OfficeEducational Testing ServiceRosedale RoadPrinceton, NJ 08541-0001USA

Phone: 609-734-5032



(continued)

CONTENTS ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7TOEFL Policy Council . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Committee of Examiners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Finance Committee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Research Committee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Outreach and Services Committee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9TWE Test (Test of Written English) Committee . . . . . . . . . . . . . . . . . . . . 9TSE Test (Test of Spoken English) Committee . . . . . . . . . . . . . . . . . . . . . 9

Program Developments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10TOEFL 2000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10The Computer-Based TOEFL Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Test of English as a Foreign Language . . . . . . . . . . . . . . . . . . . . . . . 11Use of Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Description of the Paper-Based TOEFL Test . . . . . . . . . . . . . . . . . . . . . . . 11Development of TOEFL Test Questions . . . . . . . . . . . . . . . . . . . . . . . . . . 12

TOEFL Testing Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Friday and Saturday Testing Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . 13TWE Test (Test of Written English) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13TSE Test (Test of Spoken English) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Institutional Testing Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Procedures at Test Centers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Measures to Protect Test Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Identification Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Photo File Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Photo Score Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Checking Names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Supervision of Examinees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Preventing Access to Test Materials . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

TOEFL Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Release of Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Test Score Data Retention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Image Score Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Official Score Reports from ETS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Information Printed on the Official Score Report . . . . . . . . . . . . . . . . . . . 20Examinee Score Records . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

Acceptance of Test Results Not Received from ETS . . . . . . . . . . . . . . . 21Additional Score Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Confidentiality of TOEFL Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Calculation of TOEFL Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Hand-Scoring Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23Scores of Questionable Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23Examinees with Disabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24Use of TOEFL Test Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3



4

Statistical Characteristics of the Test . . . . . . . . . . . . . . . . . . . . . . . . . 29Level of Difficulty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29Test Equating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29Adequacy of Time Allowed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29Reliabilities and the Standard Error of Measurement . . . . . . . . . . . . . . . . 29Reliability of Gain Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32Intercorrelations Among Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Content Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Criterion-Related Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Construct Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Other TOEFL Programs and Services . . . . . . . . . . . . . . . . . . . . . . . . . 39TWE Test (Test of Written English) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39TSE Test (Test of Spoken English) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39SPEAK Kit (Speaking Proficiency English Assessment Kit) . . . . . . . . . . 40SLEP Test (Secondary Level English Proficiency Test) . . . . . . . . . . . . . . 40Fee Voucher Service for TOEFL and TSE Score Users . . . . . . . . . . . . . . . 40TOEFL Fee Certificate Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41TOEFL Magnetic Score-Reporting Service . . . . . . . . . . . . . . . . . . . . . . . . 41Examinee Identification Service for TOEFL and TSE Score Users . . . . . 41Support for External Research Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Research Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43TOEFL Research Report Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43TOEFL Technical Report Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45TOEFL Monograph Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47TOEFL Products and Services Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Bulletin of Information for TOEFL, TWE, and TSE . . . . . . . . . . . . . . . . 47Test Center Reference List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Test Forms Available to TOEFL Examinees . . . . . . . . . . . . . . . . . . . . . . . 47Guidelines for TOEFL Institutional Validity Studies . . . . . . . . . . . . . . . . 47TOEFL Test and Score Data Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Institutional Testing Program Brochure . . . . . . . . . . . . . . . . . . . . . . . . . . . 48TOEFL Test of Written English Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . 48TSE Score User’s Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48Secondary Level English Proficiency Test Brochure . . . . . . . . . . . . . . . . . 48The Researcher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

TOEFL Study Materials for the Paper-Based Testing Program . 49

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

ETS Offices Serving TOEFL Candidates and Score Users . . . . . . 54

TOEFL Representatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55



5

TABLES ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Table 1. Minimum and Maximum Observed Section and Total Scores . . 23

Table 2. Reliabilities and Standard Errors of Measurement (SEM) . . . . . 30

Table 3. Intercorrelations Among the Scores . . . . . . . . . . . . . . . . . . . . . . . 33

Table 4. Correlations of Total TOEFL Scores with University Ratings . . 35

Table 5. Correlations of TOEFL Subscores with Interview andEssay Ratings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Table 6. TOEFL/GRE Verbal Score Comparisons . . . . . . . . . . . . . . . . . . . 37

Table 7. TOEFL/SAT and TSWE Score Comparisons . . . . . . . . . . . . . . . 37

Table 8. TOEFL, GRE, and GMAT Score Comparisons . . . . . . . . . . . . . . 37

Table 9. Correlations Between GMAT and TOEFL Scores . . . . . . . . . . . . 38



6



7

OVERVIEW ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

The purpose of the Test of English as a ForeignLanguage (TOEFL®) is to evaluate the Englishproficiency of people whose native language is notEnglish. The test was initially developed to measurethe English proficiency of international studentswishing to study at colleges and universities in theUnited States and Canada, and this continues to beits primary function. However, a number of academicinstitutions in other countries, as well as certainindependent organizations, agencies, and foreigngovernments, have also found the test scores useful.The TOEFL test is recommended for students at theeleventh-grade level or above; the test content isconsidered too difficult for younger students.

The TOEFL test was developed for use starting in1963-64 through the cooperative effort of more than30 organizations, public and private. A NationalCouncil on the Testing of English as a ForeignLanguage was formed, composed of representativesof private organizations and government agenciesconcerned with testing the English proficiency offoreign nonnative speakers of English who wished tostudy at colleges and universities in the United States.The program was financed by grants from the Fordand Danforth Foundations and was, at first, attachedadministratively to the Modern Language Associa-tion. In 1965, the College Board® and EducationalTesting Service (ETS®) assumed joint responsibilityfor the program.

In recognition of the fact that many who take theTOEFL test are potential graduate students, a coop-erative arrangement for the operation of the programwas entered into by Educational Testing Service,the College Entrance Examination Board, and theGraduate Record Examinations® (GRE®) Board in1973. Under this arrangement, ETS is responsiblefor administering the TOEFL program accordingto policies determined by the TOEFL Policy Council.

Educational Testing Service. ETS is a non-profit organization committed to the developmentand administration of responsible testing programs,the creation of advisory and instructional services,and research on techniques and uses of measurement,human learning and behavior, and educationaldevelopment and policy formation. It develops andadministers tests, registers examinees, and operatestest centers for various sponsors. ETS also suppliesrelated services; e.g., it scores tests; records, stores,

and reports test results; performs validity studiesand other statistical studies; and undertakes programresearch. All ETS activities are governed by a16-member board of trustees composed of personsfrom the fields of education and public service.

In addition to the Test of English as a ForeignLanguage and the Graduate Record Examinations,ETS develops and administers a number of othertests, including the Graduate Management AdmissionTest®, and The Praxis Series: Professional Assess-ments for Beginning Teachers® tests, as well asthe College Board testing programs.

The Chauncey Group International Ltd., awholly-owned subsidiary of ETS, provides assess-ment, training, and guidance products and servicesin the workplace, military, professional, and adulteducational environments.

College Board. The College Board is a nonprofit,educational organization with a membership of morethan 2,800 colleges and universities, schools, andeducational associations and agencies. The CollegeBoard’s board of trustees is elected from the member-ship, and institutional representatives serve onadvisory councils and committees that review theprograms of the College Board and participate inthe determination of its policies and activities.

The College Board sponsors tests, publications,software, and professional conferences and trainingin the areas of guidance, admissions, financial aid,credit by examination, and curriculum improvementin order to increase student access to higher educa-tion. It also supports and publishes research studiesabout tests and measurement and conducts studieson education policy developments, financial aid needassessment, admissions planning, and related educa-tion management topics.

One major College Board service, the SAT® Pro-gram, includes the SAT I: Reasoning Test, and SAT II:Subject Tests. Subject Tests are available in suchdiverse content areas as writing, literature, languages,math, sciences, and history. The College Board con-tracts with ETS to develop these tests, operate testcenters in the United States and other countries, scorethe answer sheets, and send score reports to examineesand to the institutions they designate as recipients.



8

Graduate Record Examinations Board.The GRE Board is an independent board affiliatedwith the Association of Graduate Schools and theCouncil of Graduate Schools in the United Statesand Canada. It is composed of 18 representativesof the graduate community. Standing committeesof the board include the Research Committee, theServices Committee, and the Minority GraduateEducation Committee.

ETS carries out the policies of the GRE Boardand, under the auspices of the board, administersand operates the GRE program. Two types of testsare offered: a General Test and Subject Tests in 16disciplines. ETS develops the tests, maintains testcenters in the United States and other countries,scores the answer sheets, and sends score reports tothe examinees and to the accredited institutions andapproved fellowship sponsors the examinees designateas recipients. ETS also provides information, technicaladvice, and professional counsel, and develops propos-als to achieve the goals formulated by the board.

In addition to its tests, the GRE program offersmany services to graduate institutions and to prospec-tive graduate students. Services to institutions includeresearch, publications, and advisory services to assistgraduate schools and departments in admissions,guidance, placement, and the selection of fellowshiprecipients. Services to students include test familiar-ization materials and services related to informingstudents about graduate education.

TOEFL Policy CouncilPolicies governing the TOEFL program are formu-

lated by the 15-member TOEFL Policy Council. TheCollege Board and the GRE Board each appoint threemembers to the Council. These six members comprisethe Executive Committee and elect the remainingnine members. Some of these members-at-large areaffiliated with such institutions and agencies asgraduate schools, junior and community colleges,nonprofit educational exchange organizations, andother public and private agencies with interest ininternational education. Others are specialists in thefield of English as a foreign or second language.

There are six standing committees of the Council,each responsible for specific areas of program activity.

Committee of Examiners

The TOEFL Committee of Examiners is composedof seven specialists in linguistics, language testing, orthe teaching of English as a foreign or second language.Members are rotated on a regular basis to ensure thecontinued introduction of new ideas and philosophiesrelated to second language teaching and testing.

The primary responsibility of this committee is toestablish overall guidelines for the test content, thusassuring that the TOEFL test is a valid measureof English language proficiency reflecting currenttrends and methodologies in the field. The committeedetermines the skills to be tested, the kinds of ques-tions to be asked, and the appropriateness of thetest in terms of subject matter and cultural content.Committee members review and approve the policiesand specifications that govern the test content.

The Committee of Examiners not only lends itsown expertise to the test and the test developmentprocess but also makes suggestions for research and,on occasion, invites the collaboration of other authori-ties in the field, through invitational conferences andother activities, to contribute to the improvement ofthe test. The committee works with ETS test develop-ment specialists in the actual developmentand review of test materials.

Finance Committee

The TOEFL Finance Committee consists of at leastfour members and is responsible to the TOEFLExecutive Committee. The members develop fiscalguidelines, monitor and review budgets, and providefinancial analysis for the program.



9

Research Committee

An ongoing program of research related to theTOEFL program of tests is carried out under thedirection of the Research Committee. Its six membersinclude representatives of the Policy Council and theCommittee of Examiners, as well as specialists fromthe academic community. The committee reviewsand approves proposals for test-related research andsets guidelines for the entire scope of the TOEFLresearch program.

Because the studies involved are specific to theTOEFL testing programs, most of the actual researchwork is conducted by ETS staff members rather thanby outside researchers. However, many projectsrequire the cooperation of consultants and otherinstitutions, particularly those with programs in theteaching of English as a foreign or second language.Representatives of such programs who are interestedin participating in or conducting TOEFL-relatedresearch are invited to contact the TOEFL office.

As research studies are completed, reports arepublished and made available to anyone interested inthe TOEFL tests. A list of those in print at the timethis Manual was published appears on pages 38-40.

Outreach and Services Committee

This six-member committee is responsible forreviewing and making recommendations to improveand modify existing program outreach activities andservices, especially as they relate to access and equityconcerns; initiating proposals for the development ofnew program products and services; monitoring theCouncil bylaws; and carrying out additional tasksrequested by the Executive Committee or the Council.

TWE® Test (Test of Written English)Committee

This seven-member group consists of writing andESL composition specialists with expertise in writingassessment and pedagogy.

The TWE Committee, with ETS test developmentspecialists, is responsible for developing, reviewing,and approving test items for the TWE test. Thecommittee also prepares item writer guidelines andmay suggest research or make recommendations forimproving the TWE test to ensure that the test is avalid measure of English writing proficiency.

TSE® Test (Test of Spoken English)Committee

This committee has six members who have exper-tise in oral proficiency assessment and represent theTSE constituency.

The TSE Committee, with ETS test developmentspecialists and program staff, oversees the TSE testcontent and scoring specifications, reviews test itemsand scoring procedures, and may make recommenda-tions for research or test revisions to assure that thetest is a valid measure of general speaking proficiency.



10

PROGRAM DEVELOPMENTS ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

TOEFL 2000The TOEFL 2000 project is a broad effort underwhich language testing at ETS will evolve into thetwenty-first century. The impetus for TOEFL 2000came from the various constituencies, includingTOEFL committees and score users. These groupshave called for a new TOEFL test that (1) is morereflective of models of communicative competence;(2) includes more constructed-response itemsand direct measures of writing and speaking;(3) includes test tasks integrated across modalities;and (4) provides more information than currentTOEFL scores about international students’ abilityto use English in an academic environment.

Changes to TOEFL introduced in 1995 (i.e.,eliminating single-statement listening comprehen-sion items, expanding the number of academiclectures and longer dialogs, and embedding vocabu-lary in reading comprehension passages) repre-sented the first step toward a more integrativeapproach to language testing. The next major stepwill be the introduction of a computer-basedTOEFL test in 1998. (See next column.)

TOEFL 2000 now continues with efforts thatwill lead to the next generation of computerizedTOEFL tests. These include:

n the development of a conceptual frameworkthat— takes into account models of communicative

competence— identifies various task characteristics

and how these will be used in theconstruction of language tasks

— specifies a set of variables associated witheach of these task components

n a research agenda that informs and supportsthis emerging framework

n a better understanding of the kinds ofinformation test users need and want from theTOEFL test

n a better understanding of the technologicalcapabilities for delivery of the TOEFL test intothe next century

A series of TOEFL 2000 reports that are part ofthe foundation of the project are now available (seepage 44). As future projects are completed, mono-graphs will be released to the public in this newresearch publication series.

The Computer-Based TOEFL TestTesting on computer is an important advancement thatenables the TOEFL program to take advantage of newforms of assessment made possible by the computerplatform. This reflects ETS’s commitment to create animproved English-language proficiency test that will

n better reflect the way in which peoplecommunicate effectively

n include more performance-based tasksn provide more information than the current TOEFL

test about the ability of international students touse English in an academic setting

The computer-based test is not just the paper testreformatted for the computer. While some questionswill be similar to those on the current test, others willbe quite different. For example, the Listening Compre-hension and Reading Comprehension sections willinclude new question types designed specifically forthe computer. In addition, the test will include an essaythat can be handwritten or typed on the computer. Theessay will measure an examinee’s ability to generateand organize ideas and support those ideas using theconventions of standard written English.

Some sections of the test will be computer-adaptive.In computer-adaptive testing (CAT), the computerselects a unique set of test questions based on the testdesign and the test taker’s ability level. Questions arechosen from a very large pool categorized by itemcontent and difficulty. The test design ensures fairnessbecause all examinees receive the same

n number of test questionsn amount of time (if they need it)n directionsn question typesn distribution of content

The CAT begins with a question of medium diffi-culty. The next question is one that best fits theexaminee’s performance and the design of the test. Thecomputer is programmed to make continuous adjust-ments in order to present questions of appropriatedifficulty to test takers of all ability levels.

The TOEFL program has taken steps to assure thatan individual’s test performance is not influenced by alack of computer experience. A computerized tutorial,designed especially for nonnative speakers of English,has been developed to teach the skills needed to takeTOEFL on computer.

For periodic updates on the computer-based TOEFLtest, visit TOEFL OnLine at http://www.toefl.org.



11

Use of ScoresThe TOEFL program encourages use of the test

scores by an institution or organization to help makevalid decisions concerning English language profi-ciency in terms of its own requirements. However,the institution or organization itself must determinewhether the TOEFL test is appropriate, with respectto both the language skills it measures and its level ofdifficulty, and must establish its own levels of accept-able performance on the test. General guidelines forusing TOEFL scores are given on pages 26-28.

TOEFL score users are invited to consult with theTOEFL program staff about their current or intendeduses of the test results. The TOEFL office will assistinstitutions and organizations contemplating use ofthe test by providing information about its applicabil-ity and validity in particular situations. It also willinvestigate complaints or information obtained aboutquestionable interpretation or use of reported TOEFLtest scores.

Description of the Paper-BasedTOEFL Test

The TOEFL test originally contained five sections.As a result of extensive research (Pike, 1979; Pitcherand Ra, 1967; Swineford, 1971; Test of English as aForeign Language: Interpretive Information, 1970), athree-section test was developed and introduced in1976. In July 1995, the test item format was modifiedsomewhat within the same three-section structureof the test.

Each form of the current (1997) TOEFL testconsists of three separately timed sections deliveredin a paper-and-pencil format; the questions in eachsection are multiple-choice, with four possibleanswers or options per question. All responses aregridded on answer sheets that are computer scored.

The total test time is approximately two and one-half hours; however, approximately three and one-halfhours are needed for a test administration to admitexaminees to the testing room, to allow them to enteridentifying information on their answer sheets, andto distribute and collect the test materials. Briefdescriptions of the three sections of the test follow.

n Section 1, Listening Comprehension

Section 1 measures the ability to understand Englishas it is spoken in North America. The oral featuresof the language are stressed, and the problems testedinclude vocabulary and idiomatic expression aswell as special grammatical constructions that arefrequently used in spoken English. The stimulusmaterial and oral questions are recorded in standardNorth American English; the response options areprinted in the test books.

There are three parts in the Listening Comprehen-sion section, each of which contains a specific typeof comprehension task. The first part consists of anumber of short conversations between two speakers,each followed by a single spoken question. Theexaminee must choose the best response to thequestion about the conversation from the four optionsprinted in the test book. In the second and third partsof this section, the examinee hears conversations andshort talks of up to two minutes in length. Theconversations and talks are about a variety of sub-jects, and the factual content is general in nature.After each conversation or talk the examinee is askedseveral questions about what was heard and, for each,must choose the one best answer from the choicesin the test book. Questions for all parts are spokenonly one time.

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

TEST OF ENGLISH ASA FOREIGN LANGUAGE:The Paper-Based Testing Program

11



12

n Section 2, Structure and WrittenExpression

Section 2 measures recognition of selected structuraland grammatical points in standard written English.The language tested is formal, rather than conversa-tional. The topics of the sentences are of a generalacademic nature so that individuals in specific fieldsof study or from specific national or linguistic groupshave no particular advantage. When topics havea national context, they refer to United States orCanadian history, culture, art, or literature. However,knowledge of these contexts is not needed to answerthe structural or grammatical points being tested.

This section is divided into two parts. The firstpart tests an examinee’s ability to identify the correctstructure needed to complete a given sentence. Theexaminee reads incomplete sentences printed in thetest book. From the four responses provided for eachincomplete sentence, the examinee must choose theword or phrase that best completes the given sentence.Only one of the choices fits correctly into the particularsentence. The second part tests an examinee’s abilityto recognize correct grammar and to detect errors instandard written English. Here the examinee readssentences in which some words or phrases are under-lined. The examinee must identify the one underlinedword or phrase in each sentence that would not beaccepted in standard written English.

n Section 3, Reading Comprehension

Section 3 measures the ability to read and understandshort passages that are similar in topic and style tothose that students are likely to encounter in NorthAmerican colleges and universities. The examineereads a variety of short passages on academic subjectsand answers several questions about each passage.The questions test information that is stated in orimplied by the passage, as well as knowledge of someof the specific words as they are used in the passage.To avoid creating an advantage to individuals in anyone field of study, sufficient context is provided sothat no subject-specific familiarity with the subjectmatter is required to answer the questions. Questionsare asked about factual information presented in thepassages, and examinees may also be asked to makeinferences or recognize analogies. In all cases, thequestions can be answered by reading and under-standing the passages.

Development ofTOEFL Test Questions

Material for the TOEFL test is prepared by lan-guage specialists who are trained in writing questionsfor the test before they undertake actual item-writingassignments. Additional material is prepared byETS test development specialists. The members ofthe TOEFL Committee of Examiners establish overallguidelines for the test content and specifications. Allitem specifications, questions, and final test forms arereviewed internally at ETS for cultural and racial biasand content appropriateness, according to establishedETS procedures.

These reviews ensure that each final form of thetest is free of any language, symbols, references, orcontent that might be considered potentially offensiveor inappropriate for subgroups of the TOEFL testpopulation, or that might serve to perpetuate negativestereotypes.

All questions are pretested on representativegroups of international students who are not nativespeakers of English. Only after the results of thepretest questions have been analyzed for statisticaland content appropriateness are questions selectedfor the final test forms.

Following the administration of each new formof the test, a statistical analysis of the responses toquestions is conducted. On rare occasions, whena question does not function as expected, it will bereviewed again by test specialists. After this review,the question may be deleted from the final scoringof the test. The statistical analyses also providecontinuous monitoring of the level of difficulty ofthe test, the reliability of the entire test and of eachsection, intercorrelations among the sections, andthe adequacy of the time allowed for each section.(See “Statistical Characteristics of the Test,” page 29.)



13

TOEFL TESTING PROGRAMS ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

The TOEFL test is administered internationally onregularly scheduled test dates through the Fridayand Saturday testing programs. It is also administeredat local institutions around the world throughthe Institutional Testing Program (ITP). The ITPprogram does not provide official TOEFL scorereports; scores are for use by the administeringinstitution only.

Friday and SaturdayTesting ProgramsThe official TOEFL test is given at centers aroundthe world one day each month – five Fridays andseven Saturdays.

The TOEFL office diligently attempts to make thetest available to all individuals who require TOEFLscores. In 1996-97, more than 1,275 centers locatedin 180 countries and areas were established for theSaturday testing program to accommodate the morethan 703,000 persons registered to take the test; 350centers in more than 60 countries and areas wereestablished for the more than 248,000 personsregistered to take TOEFL under the Friday program.

Registration and administration procedures areidentical for the Friday and Saturday programs. Thetest itself is also identical in terms of format andcontent. Score reports for administrations under bothprograms provide the same data. More informationabout these testing programs can be found in theBulletin of Information for TOEFL, TWE, and TSE.(See page 47.)

As noted above, the TOEFL program provides12 test dates a year. However, the actual number ofadministrations at any one center in a given countryor area is scheduled according to demand and theavailability of space and supervisory staff.

There are sometimes local scheduling conflictswith national or religious holidays. Although theTOEFL office makes every effort to avoid schedulingadministrations of the test on such dates, it may beunavoidable in some cases.

Registration must be closed well in advance of eachtest date to ensure the delivery of test materials to thetest centers. Registration deadline dates are aboutseven weeks before the test dates for centers outsidethe United States and Canada and five weeks beforethe test dates for centers within these two countries.

Almost all administrations are held as scheduled.On occasion, however, shipments of test materials maybe impounded by customs officials or delayed by mailembargoes or transportation strikes. Other problems,ranging from political disturbances within countries,to power failures, to the last-minute illness of a testsupervisor, may also force postponement of a TOEFLtest administration.

If an administration must be postponed, a makeupadministration is scheduled, usually on the nextregularly scheduled test date. Occasionally it isnecessary to arrange a makeup administration onanother date.

Different forms of the test may be used at a singleadministration. Following each administration, theanswer sheets are returned to ETS for scoring; testresults are mailed to score recipients about one monthafter the answer sheets are received at ETS.

TWE Test (Test of Written English)In 1986, the TOEFL program introduced the Test ofWritten English. This direct assessment of writingproficiency was developed in response to requestsfrom many colleges, universities, and agencies thatuse TOEFL scores. The TWE test is currently(1997) a required section of the TOEFL test at fiveadministrations per year. For more informationabout the Test of Written English, see page 39.

TSE Test (Test of Spoken English)The Test of Spoken English measures the ability ofnonnative speakers of English to communicate orallyin English. It requires examinees to tape recordspoken answers to a variety of questions. The TSEtest is administered on all 12 Friday and SaturdayTOEFL test dates. For more information about theTest of Spoken English, see page 39.

Institutional Testing ProgramThe Institutional Testing Program permits approvedinstitutions throughout the world to administer theTOEFL test to their own students on dates conven-ient for them (except for regularly scheduled TOEFLadministration dates), using their own facilities andstaff. Each year a number of forms of the TOEFL testpreviously used in the Friday and Saturday testingprograms are made available for the InstitutionalTesting Program.



14

In addition to the regular TOEFL test, which isespecially appropriate for use with students at theintermediate and higher levels of English languageproficiency, ITP offers the Preliminary Test of Englishas a Foreign Language (Pre-TOEFL) for individuals atthe beginning level. Pre-TOEFL measures the samecomponents of English language skills as the TOEFLtest. However, Pre-TOEFL is less difficult and shorter.Pre-TOEFL test results are based on a restricted scalethat provides more discriminating measurement atthe lower end of the TOEFL scale.

Note: There are minor differences in the numberof questions and question types between the ITPTOEFL test and the Pre-TOEFL test.

How Institutional TOEFL Can Be Used

The Institutional Testing Program is offered primarilyto assist institutions in placing students in Englishcourses at the appropriate level of difficulty, fordetermining whether additional work in Englishis necessary before an individual can undertakeacademic studies, or as preparation for an officialFriday or Saturday TOEFL administration.

Institutional TOEFL Test Scores

Scores earned under the Institutional TestingProgram are comparable to scores earned under theworldwide Friday and Saturday testing programs.However, ITP scores are for use by the administeringinstitution only.

ETS reports test results to the administeringinstitution in roster form, listing the names andscores (section and total) of all students who took thetest at that administration. Two copies of the scorerecord for each student are provided to the administer-ing institution: a file copy for the institution and apersonal copy for the individual. Both copies indicatethat the scores were obtained at an InstitutionalTesting Program administration.

ETS does not report scores obtained under thisprogram to other institutions as it does for officialscores obtained under the Friday and Saturday testingprograms. To ensure score validity, scores obtainedunder the Institutional Testing Program shouldnot be accepted by other institutions to evaluate anindividual’s readiness to begin academic studiesin English.



15

PROCEDURES AT TEST CENTERS ○ ○ ○ ○ ○

Standard, uniform procedures are important in anytesting program, but are essential for an examinationthat is given worldwide. Therefore, the TOEFLprogram provides detailed guidelines for test centersupervisors to ensure uniform administrations.Preparing for a TOEFL/TWE or TSE Administrationis mailed to test supervisors well in advance of the testdate. This publication describes the arrangements thesupervisor must make to prepare for the test adminis-tration, including selecting testing rooms and theassociate supervisors and proctors who will be neededon the day of the test.

The Manual for Administering TOEFL, includedwith every shipment of test materials, describesappropriate seating plans, the kind of equipment thatshould be used for the Listening Comprehensionsection, identification requirements, the priorities foradmitting examinees to the testing room, and instruc-tions for distributing and collecting test materials.It also contains detailed instructions for the actualadministration of the test.

TOEFL program staff work with test center super-visors to ensure that the same practices are followed atall centers, and they conduct workshops during whichsupervisors can discuss procedures for administering thetest. TOEFL staff respond to all inquiries from supervi-sors and examinees regarding circumstances or condi-tions associated with test administrations, and theyinvestigate all complaints received about specificadministrations.

Measures to Protect Test SecurityIn administering a worldwide testing program at morethan 1,275 test centers in 180 countries, the TOEFLprogram considers the maintenance of security attesting sites to be of paramount importance. Theelimination of problems at test centers, including test-taker impersonations, is a continuing goal. To offerscore users the most valid, reliable, and secure mea-surements of English language proficiency available,the TOEFL office continuously reviews and refinesprocedures to increase the security of the test before,during, and after its administration.

Because of the importance of TOEFL test scores toexaminees and institutions, it is inevitable that someindividuals will engage in practices designed to increasetheir reported scores. The careful selection of supervi-sors, a high proctor-to-examinee ratio, and carefullydeveloped procedures for the administration of the test

(explained in the Manual for Administering TOEFL)are measures designed to prevent or discourage exam-inee attempts at impersonation, copying, theft of testmaterials, and the like, and thus to protect the integrityof the test for all examinees and score recipients.

Identification Requirements

Strict admission procedures are followed at all testcenters to prevent attempts by some examinees tohave others with greater proficiency in Englishimpersonate them at a TOEFL administration. Tobe admitted to a test center, every examinee mustpresent an official document with a recognizablephotograph and a completed photo file record with arecent photo attached. Although the passport is thebasic document that is acceptable at all test centers,other specific photobearing documents may be accept-able for individuals who may not be expected to havepassports or who are taking the test in their owncountries.

Through embassies in the United States andTOEFL representatives and supervisors in othercountries, the TOEFL office continually verifies thenames of official, secure, photobearing identificationdocuments used in each country, such as nationalidentity cards, work permits, and registration certifi-cates. In the Friday and Saturday testing programs,each admission ticket contains a statement specifyingthe documents that will be accepted at TOEFL testcenters in the country in which the examinee isregistered to take the test. This information is com-puter-printed on a red field to ensure that it will beseen. (The same information is printed on theattendance roster prepared for each center.) Followingis a sample of the statement that appears on admis-sion tickets for Venezuela.

YOUR VALID PASSPORT. CITIZENS OF VEN-EZUELA MAY USE NATIONAL IDENTITY CARDOR LETTER AS DESCRIBED IN THE BULLETIN.

Complete information about identification require-ments is included in all editions of the Bulletin ofInformation for TOEFL, TWE, and TSE.



16

Photo File Records

Every TOEFL examinee must present a completedphoto file record to the test center supervisor beforebeing admitted to the testing room. The photo filerecord contains the examinee’s name, registrationnumber, test center code, and signature, as well asa recent photo that clearly identifies the examinee(that is, the photo must look exactly like the exam-inee, with the same hairstyle, with or without a beard,and so forth). The photo file records are collected atthe test center and returned to ETS, where the photosand identifying information are electronically cap-tured and included on the examinee’s score data file.

Photo Score Reporting

As an additional procedure to help eliminate thepossibility of impersonation at test centers, theofficial score reports that are routinely sent to institu-tions designated by the test taker, and the examinee’sown copy of the score report, bear an electronicallyreproduced photo image of the examinee and his orher signature. (The score report also includes the

number of the passport or other identificationdocument used to gain admission to the testing centerand the name of the country issuing the document.)Examinees are advised in the Bulletin of Informationthat the score reports will contain these photo images.In addition to strengthening security through thisdeterrent to impersonation, the report form providesscore users with the immediate information they mayneed to resolve any issues of examinee identity. Keyfeatures of the image score reports are highlighted onpage 19.

Checking Names

To prevent examinee attempts to exchange answersheets or to grid another person’s name (for whom heor she is taking the test) on the answer sheet, supervi-sors are asked to compare names on the identificationdocument and the answer sheet and also to check thegridding of names on the answer sheet before examin-ees leave the room.

16



17

Supervision of Examinees

Supervisors and proctors are instructed to exerciseextreme vigilance during a test administration toprevent examinees from giving or receiving assistancein any way.

In addition, the Manual for Administering TOEFLadvises supervisors about assigning seats to examin-ees. To prevent copying from notes or other aids,examinees may not have anything on their desks buttheir test books, answer sheets, pencils, and erasers.They are not permitted to make notes or marks of anykind in their test books. (Warning/Dismissal Noticeforms are used to report examinees who violateprocedures. An examinee is asked to sign the notice todocument the violation and to indicate he or sheunderstands that a violation of procedures hasoccurred and that the answer sheet may not bescored.)

If a supervisor is certain that someone has given orreceived assistance, the supervisor has the authorityto dismiss the examinee from the testing room; scoresfor dismissed examinees will not be reported. If asupervisor suspects someone of cheating, the exam-inee is warned about the violation, is asked to sign aWarning/Dismissal Notice, and must move to anotherseat selected by the supervisor. A description of theincident is written on the Supervisor’s IrregularityReport, which is returned to ETS with the answersheet. Both suspected and confirmed cases of cheatingare investigated by the Test Security Office at ETS.(See “Scores of Questionable Validity,” page 23.)

Turning back to another section of the test,working on a section in advance, or continuing towork on a section after time is called are not permit-ted and are considered cheating. (To assist thesupervisor, a large number identifying the sectionbeing worked on is printed at the top of each page ofthe test book.) Supervisors are instructed to warnanyone found working on the wrong section and toask the examinee to sign a Warning/Dismissal Notice.

Preventing Access toTest Materials

To ensure that examinees have not seen the testmaterial in advance, a new form of the test is devel-oped for each Friday and Saturday administration.

To prevent the theft of test materials, procedureshave been devised for the distribution and handlingof these materials. Test books are individually sealed,then packed and sealed in plastic bags. Test books,answer sheets, and Listening Comprehension record-ings are sent to test centers in sealed boxes and areplaced in secure, locked storage that is inaccessible tounauthorized persons. Supervisors are directed tocount the test books several times — upon receipt,during the test administration, and after the test isover. No one is permitted to leave the testing roomuntil the supervisor has accounted for all test materi-als. Except for “disclosed” administrations, whenexaminees may obtain the test book (see “Test FormsAvailable to TOEFL Examinees,” page 47), supervi-sors must follow detailed directions for returning thetest materials. Materials are counted upon receipt atETS, and its Test Security Office investigates all casesof missing test materials.



18

TOEFL TEST RESULTS ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Release of Test ResultsAbout one month after a Friday or Saturday TOEFLadministration, test results are mailed to the examin-ees and to the official score recipients they havespecified, provided that the answer sheets are receivedat ETS promptly after the administration. Test resultsfor examinees whose answer sheets are incomplete orwhose answer sheets arrive late are usually sent twoor three weeks later. All test results are mailed by thefinal deadline — 12 weeks after the test.

For the basic TOEFL test fee, each examinee isentitled to four copies of the test results: one copyis sent to the examinee, and up to three official scorereports are sent directly by ETS to the institutionswhose assigned code numbers the examinee hasmarked on the answer sheet.* The institution codedesignates the recipient college, university, or agency.A list of the most frequently used institution andagency codes is printed in the Bulletin of Information.An institution whose code number is not listed shouldgive applicants its code number before they take thetest. (See page 20 for more information.)

The most common reason that institutions do notreceive score reports following an administration isthat examinees do not properly specify the institu-tions as score report recipients by marking the correctcodes on the test answer sheet. (Examinees cannotwrite the names of recipients on the answer sheet.)An examinee who wants scores sent to an institutionwhose code number was not marked on the answersheet must submit a Score Report Request Formnaming the institution that is to receive the scores.There is a fee for this service.

Test Score Data RetentionLanguage proficiency can change considerably in arelatively short period. Therefore, the TOEFL officewill not report scores that are more than two yearsold. Individually identifiable TOEFL scores areretained on the TOEFL database for only two yearsfrom the date of the test. Individuals who took theTOEFL test more than two years ago must take itagain if they want scores sent to an institution.*After two years, all information that could be used toidentify an individual is removed from the database.Score data and other information that may be used forresearch or statistical purposes do not include indi-vidual examinee identification information and areretained indefinitely.

Image Score ReportsThe image-processing technology used to producethe photo score reports allows ETS to electronicallycapture the image from the examinee’s photograph,as well as the signature and other identifying datasubmitted by the examinee at the testing site, andto reproduce these with the examinee’s test resultsdirectly on the score reports. The computerizedelectronic transfer of photo images permits a high-quality reproduction of the original photo on the scorereport. (If a photograph is too damaged or for otherreasons cannot be accepted by the image-processingsystem, “Photo Not Available” will be printed on thescore report.)

Steps have been taken to reduce the opportunitiesfor tampering with examinee score records thatinstitutions may receive directly from applicants.However, to ensure that institutions receivevalid score records, we urge that admissionsofficers and others responsible for the admis-sions process accept only official score reportssent directly by ETS.

* An institution or agency that is sponsoring an examinee and has madeprior arrangements with the TOEFL office will also receive a copy of theexaminee’s official score report if the examinee has given permission tothe TOEFL office.

* A TOEFL score is measurement information and is subject to all therestrictions noted in this Manual. (These restrictions are also noted in theBulletin of Information.) The test score is not the property of the examinee.



19

YOUR SIGNATURE

NAME OF COUNTRY ISSUING PASSPORT OR IDENTIFICATION

NUMBER ON IDENTI- FICATION DOCUMENT

®

TEST OF ENGLISH AS A FOREIGN LANGUAGE

Test of English as a Foreign Language P.O. Box 6151 • Princeton, NJ 08541-6151 • USA

TEST OF WRITTEN ENGLISH

TOEFL SCALED SCORES

NAME (Family or Surname, Given, Middle)

EXAMINEE'S ADDRESS:

SECTION 1 SECTION 2 SECTION 3 TOTAL SCORE

TWE SCORE

REGISTRATION NUMBER

Month Year TEST DATE

CENTER NUMBER

INTERPRETIVE INFORMATION

The face of this document has a multicolored background — not a white background.

NATIVE LANGUAGE

NATIVE COUNTRY

DEPARTMENT

Month/Day/Year DATE OF BIRTH SEX

INST. CODE

DEPT. CODE

DEGREE

REASON FOR

TAKING TOEFL

TOEFL TAKEN

BEFORE

Facsimile reduced. Actual size of entire form, 81/2( x 11(; score report section 81/2( x 35/8(.

1

5

42 3

Official Score Reports from ETSTOEFL score reports give the score for each of thethree sections of the test and the total score. Examin-ees who take the TOEFL test during an administra-tion at which the Test of Written English is given alsoreceive a TWE score printed in a separate field on theTOEFL score report. See page 20 for informationabout the score report codes.

Features of the Image Reports:

a The blue background color quickly identifies thereport as being an official copy sent from ETS.

s The examinee’s name and scores are printed inred fields.

d Reverse type is used for printing the name andscores.

f The examinee’s photo is taken from the photo filerecord given to the test center supervisor on theday of the test and reproduced on the scorereport.

g The examinee’s signature and ID number andthe name of the country issuing identificationare reproduced from the photo file record.

h The word “copy” appears in the backgroundcolor of score reports that are photocopied usingeither a black or color image copier.

Score reports are valid only if received directly fromEducational Testing Service. TOEFL test scores areconfidential and should not be released by therecipient without written permission from the ex-aminee. All staff with access to score records shouldbe advised of their confidential nature.

If you have any reason to believe that someonehas tampered with a score report or would liketo verify test scores, please call the following toll-free number between 8:30 AM and 4:30 PM NewYork time.

800-257-9547TOEFL/TSE Services will verify the accuracy of

the scores.

6



20

Information Printed on theOfficial Score ReportIn addition to test scores, native country, nativelanguage, and birth date, the score report includesother pertinent data about the examinee and informa-tion about the test.

INSTITUTION CODE. The institution code designates the recipient college,university, or agency. A list of the most frequently used institution and agencycodes is printed in the Bulletin of Information. An institution whose code numberis not listed should give applicants its code number before they take the test.(This information should be included in application materials prepared forinternational students.)

Note: An institution that does not know its TOEFL code number or wishes toobtain one should call 609-771-7975 or write to ETS Code Control, P.O. Box6666, Princeton, NJ 08541-6666, USA.

DEPARTMENT CODE. The department code number identifies the profes-sional school, division, department, or field of study in which the graduateapplicant plans to enroll. The department code list shown below is also includedin the Bulletin of Information. The department code for all business schools is(02), for law schools (03), and for unlisted departments (99).

Fields of Graduate Study Other Than Business or LawBIOLOGICAL SCIENCES31 Agriculture32 Anatomy05 Audiology33 Bacteriology34 Biochemistry35 Biology45 Biomedical Sciences36 Biophysics37 Botany38 Dentistry39 Entomology46 Environmental Science40 Forestry06 Genetics41 Home Economics25 Hospital and Health Services

Administration42 Medicine07 Microbiology74 Molecular and Cellular Biology43 Nursing77 Nutrition44 Occupational Therapy56 Pathology47 Pharmacy48 Physical Therapy49 Physiology55 Speech-Language Pathology51 Veterinary Medicine52 Zoology30 Other biological sciences

PHYSICAL SCIENCES54 Applied Mathematics61 Astronomy62 Chemistry78 Computer Sciences63 Engineering, Aeronautical64 Engineering, Chemical65 Engineering, Civil66 Engineering, Electrical67 Engineering, Industrial68 Engineering, Mechanical69 Engineering, other71 Geology72 Mathematics73 Metallurgy75 Oceanography76 Physics59 Statistics60 Other physical sciences

Use 99 for any departmentnot listed.

HUMANITIES11 Archaeology12 Architecture26 Art History13 Classical Languages28 Comparative Literature53 Dramatic Arts14 English29 Far Eastern Languages and Literature15 Fine Arts, Art, Design16 French17 German04 Linguistics19 Music57 Near Eastern Languages and Literature20 Philosophy21 Religious Studies or Religion22 Russian/Slavic Studies23 Spanish24 Speech10 Other foreign languages98 Other humanities

SOCIAL SCIENCES27 American Studies81 Anthropology82 Business and Commerce83 Communications84 Economics85 Education (including M.A. in Teaching)01 Educational Administration70 Geography92 Government86 History87 Industrial Relations and Personnel88 International Relations18 Journalism90 Library Science91 Physical Education97 Planning (City, Community,

Regional, Urban)92 Political Science93 Psychology, Clinical09 Psychology, Educational58 Psychology, Experimental/

Developmental79 Psychology, Social08 Psychology, other94 Public Administration50 Public Health95 Social Work96 Sociology80 Other social sciences

TOEFL SCORES: Three section scores and a total score are reported for theTOEFL test. The three sections are:

Section 1 — Listening ComprehensionSection 2 — Structure and Written ExpressionSection 3 — Reading Comprehension

TEST OF WRITTEN ENGLISH (TWE): Effective July 1995, the TWE test isadministered in August, October, December, February, and May.

Scores Explanations of TWE Scores

6.0 Demonstrates clear competence in writing on both the rhetoricaland syntactic levels, though the essay may have occasional errors.

5.55.0 Demonstrates competence in writing on both the rhetorical and

syntactic levels, though the essay will probably have occasionalerrors.

4.54.0 Demonstrates minimal competence in writing on both the

rhetorical and syntactic levels.3.53.0 Demonstrates some developing competence in writing, but the

essay remains flawed on either the rhetorical or syntactic level, orboth.

2.52.0 Suggests incompetence in writing.1.51.0 Demonstrates incompetence in writing.1NR Examinee did not write an essay.OFF Examinee did not write on the assigned topic.

INTERPRETIVE INFORMATION: The date of the most current edition of theTOEFL Test and Score Manual is printed here. (This date is printed only onthe official score report.)

TEST DATE: Because English proficiency can change considerably in arelatively short period, please note the date on which the test was taken.Scores more than two years old cannot be reported, nor can they be verified.

PLANS TO WORK FOR DEGREE:1 = Yes 2 = No 0 = Not answered

REASON FOR TAKING TOEFL:1 = To enter a college or university as an undergraduate student2 = To enter a college or university as a graduate student3 = To enter a school other than a college or university4 = To become licensed to practice a profession5 = To demonstrate proficiency in English to the company for which

the examinee works or expects to work6 = Other than above0 = Not answered

NUMBER OF TIMES TOEFL TAKEN BEFORE:1 = One 3 = Three 0 =None or not2 = Two 4 = Four or more answered



21

Facsimile reduced

YOUR SIGNATURE

NAME OF COUNTRY ISSUING PASSPORT OR IDENTIFICATION

NUMBER ON IDENTI- FICATION DOCUMENT

®

TEST OF ENGLISH AS A FOREIGN LANGUAGE

TEST OF WRITTEN ENGLISH

TOEFL SCALED SCORES

NAME (Family or Surname, Given, Middle)

EXAMINEE'S ADDRESS:

Test of English as a Foreign Language • P. O. Box 6151 • Princeton, NJ 08541-6151 • USA

SECTION 1 SECTION 2 SECTION 3 TOTAL SCORE

TWE SCORE

REGISTRATION NUMBER

Month Year TEST DATE

CENTER NUMBER SPONSOR CODE

EXAMINEE'S ORIGINAL SCORE RECORD

NATIVE LANGUAGE

NATIVE COUNTRY

Month/Day/Year DATE OF BIRTH SEX

INST. CODE

DEPT. CODE

DEGREE

REASON FOR

TAKING TOEFL

TOEFL TAKEN

BEFORE

4 5

2

1

3

Examinee Score RecordsExaminees receive their test results on a form titledExaminee’s Score Record. These are NOT officialTOEFL score reports and should not be acceptedby institutions.

Acceptance of Test Results Not Receivedfrom ETS

Bear in mind that examinees may attempt to alterscore records. Institution and agency officials areurged to verify all TOEFL scores supplied by examin-ees. TOEFL/TSE Services will either confirm or denythe accuracy of the scores submitted by examinees.

If there is a discrepancy between the official scoresrecorded at ETS and those submitted in any form byan examinee, the institution will be requested to sendETS a copy of the score record supplied by theexaminee. At the written request of an official of theinstitution, ETS will report the official scores, as wellas all previous scores recorded for the examineewithin the last two years. Examinees are advised ofthis policy in the Bulletin, and, in signing theircompleted registration forms, they accept theseconditions. (Also see “Test Score Data Retention”on page 18.)

How to Recognize an Unofficial ScoreReport:

a ✭✭✭Examinee’s Original Score Record✭✭✭ isprinted at the bottom of the score record.

s The Examinee’s Score Record is printed on whitepaper.

How to Recognize If a Score ReportHas Been Altered:

d The last digit of the total score should end in “0,”“3,” or “7.”

f There should be no erasures. Do the shaded areasseem lighter than others, or are any of these areasblurred?

g The typeface should be the same in all areas.



22

DOs and DON’Ts

Do verify the information on an examinee’s scorerecord by calling TOEFL/TSE Services:

800-257-9547Don’t accept scores that are more than two

years old.

Don’t accept score reports from another institu-tion that were obtained under the TOEFLInstitutional Testing Program.

Don’t accept photocopies of score reports.

Additional Score ReportsIndividuals who have taken the TOEFL test atscheduled Friday or Saturday test administrationsmay request that official score reports be sent toadditional institutions at any time up to two yearsafter the date on which they took the test.

There are two score reporting services: (1) regularand (2) rush reporting. The regular service mailsadditional score reports within two weeks afterreceipt of an examinee’s Score Report Request Form.The rush reporting service mails score reports toinstitutions within four working days after a requestform has been received. There is an additional fee forthe rush service.

Confidentiality of TOEFL ScoresInformation retained in TOEFL test files about anexaminee’s native country, native language, and theinstitutions to which the test scores have been sent,as well as the actual scores, is the same as the infor-mation printed on the examinee’s score record andon the official score reports. An official score reportwill be sent only at the written consent of the exam-inee to those institutions or agencies designated onthe answer sheet by the examinee on the day of thetest, on a Score Report Request Form submitted ata later date, or otherwise specifically authorized bythe examinee.*

* See footnote on page 18.

To ensure the authenticity of scores, the TOEFLprogram office urges that institutions accept onlyofficial copies of TOEFL scores received directlyfrom ETS.

Score users are responsible for maintaining theconfidentiality of an individual’s score information.Scores are not to be released by the institutionalrecipient without the explicit permission of theexaminee. Dissemination of score records should bekept to a minimum, and all staff with access to themshould be informed of their confidential nature.

The TOEFL program recognizes the right ofinstitutions as well as individuals to privacy withregard to information supplied by and about themthat is stored in data or research files held by ETSand the concomitant responsibility to safeguardinformation in its files from unauthorized disclosure.As a consequence, information about an institution(identified by name) will be released only in a mannerconsistent with a prior agreement, or with the explicitconsent of the institution.

Calculation of TOEFL ScoresThe raw scores for the three sections of the TOEFLtest are the number of questions answered correctly.No penalty points are subtracted for wrong answers.Although each new form of the test is constructed tomatch previous forms in terms of content and diffi-culty, the level of difficulty may vary slightly from oneform to another. Raw scores from each new TOEFLtest are statistically adjusted, or equated, to accountfor relatively minor differences in difficulty acrossforms, thereby allowing scores from different formsof the test to be used interchangeably.

At the time of the first administration of the three-section TOEFL test (1976), the scale for reporting thetotal score was linked to the scale that was then in usefor the original five-section test. Since April 1996 thescale has been maintained by linking current tests tothe scale of the July 1995 initial revised TOEFL test.

The three separate sections are scaled so the meanscaled score for each section equals one-tenth of thetotal scaled score mean (the standard deviations of thescaled scores for the three sections are equal) and thetotal score equals ten-thirds times the sum of the threesection scaled scores.



23

Example:

Section 1 Section 2 Section 3 Sum46 + 54 + 50 = 150

(150 x 10) ÷ 3 = 500

TOEFL scores for Sections 1 and 2 are reported ona scale that can range from 20 to 68. Section 3 scoresrange from 20 to 67. TOEFL total scores are reportedon a scale that can range from 200 to 677.

Scores for each new test form are converted to thesame scale by a statistical equating procedure knownas item response theory (IRT) true score equating,which determines equivalent scaled scores for personsof equal ability regardless of the difficulty level of theparticular form of the test and the average ability levelof the group taking the test.*

The reported scores are not based on either thenumber or the percentage of questions answeredcorrectly. Nor are they related to the distributionof scores on any other test, such as the SAT or theGRE tests.

Actual ranges of observed scores for the periodfrom July 1995 through June 1996 are shown inTable 1. Note that for the section and total scores,all minimum observed section and total scores arehigher than the lowest possible scores.

Hand-Scoring ServiceExaminees are responsible for properly completingtheir answer sheets to ensure accurate scoring. Theyare instructed to use a medium-soft black lead pencil,to mark only one answer to each question, to fill inthe answer space completely so the letter inside thespace cannot be seen, and to erase all extra marksthoroughly. Failure to follow any of these instructionsmay result in the reporting of an inaccurate score.

Examinees who question whether their reportedscores are accurate may request that their answersheets be hand scored. There is a fee for this service.A request for hand scoring must be received withinsix months of the test date; later requests cannotbe honored.

The TOEFL office has established the followinghand-scoring procedures: the answer sheet to be handscored is first confirmed as being the one completedby the person requesting the service; the answer sheetis then hand scored twice by trained ETS staffworking independently. If there is a discrepancybetween the hand-scored and computer-scored results,the hand-scored results, which may be higher or lowerthan those originally reported, will be reported to allrecipients of the earlier scores, and the hand-scoringfee will be refunded to the examinee. The results ofthe hand scoring are available about three weeks afterreceipt of the examinee’s request. Experience hasshown that very few score changes result from hand-scoring requests.

Scores of Questionable ValidityImproved scores over time are to be expected if aperson is studying English; they may not indicateirregularities. However, institutions and other TOEFLscore recipients that note inconsistencies between testscores and English performance, especially in caseswhere there is reason to suspect an inconsistencybetween a high TOEFL score and relatively weakEnglish proficiency, are encouraged to refer to theofficial photo score report for the possibility ofimpersonation. Institutions should notify the TOEFLoffice if they find any evidence of impersonation. ETSreports TOEFL scores for a period of two years afterthe date the test was administered.

Table 1. Minimum and Maximum ObservedSection and Total Scores, July 1995 - June 1996

Section Min. Max.

1. Listening Comprehension 25 682. Structure and Written Expression 21 683. Reading Comprehension 22 67

Total Score 263 677

* See Cook and Eignor (1991) for further information about IRT truescore equating.

This method of scaling results in rounded scoresfor which the last digit can take on only three values:zero, three, or seven.



24

Irregularities uncovered by institutions andreported to ETS, as well as those brought to theattention of the TOEFL office by examinees orsupervisors who believe that misconduct may havetaken place, are investigated.

Misconduct irregularities are reviewed, statisticalanalyses are conducted, and scores may be canceledby ETS. For other irregularities, the ETS Test Secu-rity Office assembles relevant documents, such asprevious score reports, registration forms, and answersheets. When handwriting differences or evidenceof possible copying or exchange of answer sheets isfound, the case is referred to the ETS Board ofReview, a group of senior professional staff members.Based on its independent examination of the evi-dence, the Board of Review directs appropriate action.

ETS policy and procedures are designed to providereasonable assurance of fairness to examinees in boththe identification of suspect scores and the weighingof information leading to possible score cancellation.These procedures are intended to protect both scoreusers and examinees from inequities that could resultfrom decisions based on fraudulent scores and tomaintain the integrity of the test.

Examinees with DisabilitiesNonstandard testing arrangements may include specialeditions of the test, the use of a reader and/or amanuen-sis, a separate testing room, and extended time and/orrest breaks during the test administration.

Nonstandard administrations are given on regularlyscheduled test dates, and security procedures are thesame as those followed for standard administrations.

The TOEFL office advises institutions that the testmay not provide a valid measure of the examinee’sproficiency, even though the conditions were designedto minimize any adverse effects of the examinee’sdisability upon test performance. The TOEFL officecontinues to recommend that alternative methods ofevaluating English proficiency be used for individualswho cannot take the test under standard conditions.Criteria such as past academic record (especially ifEnglish has been the language of instruction), recom-mendations from language teachers or others familiarwith the applicant’s English proficiency, and/or apersonal interview or evaluation are suggested in lieuof TOEFL scores. Because the individual circum-stances of nonstandard administrations vary sowidely and the number of examinees tested undernonstandard conditions is still quite small, theTOEFL program cannot provide normative data forinterpreting scores obtained in such administrations.

A statement that the scores were obtained undernonstandard conditions is printed on the official scorereport (and on the Examinee’s Score Record) of anexaminee for whom special arrangements were made.Each score recipient is also sent an explanatory noticeemphasizing that there are no normative data forscores obtained under nonstandard testing conditionsand, therefore, that such scores should be used withinthese parameters.



25

USE OF TOEFL TEST SCORES ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

The TOEFL test is a measure of general Englishproficiency. It is not a test of academic aptitudeor of subject matter competence, nor is it a directtest of English speaking or writing ability.TOEFL test scores can assist in determining whetheran applicant has attained sufficient proficiency inEnglish to study at a college or university. However,even though an applicant may achieve a high TOEFLscore, the student who is not academically preparedmay not easily succeed in a given program of study.Therefore, determination of academic admissibilityof nonnative English speakers is dependent uponnumerous additional factors, such as previous aca-demic record, other institution(s) attended, level andfield of study, and motivation.

If a nonnative English speaker meets academicrequirements, official TOEFL test scores may be usedin making the following kinds of decisions:

n The applicant may begin academic work with norestrictions.

n The applicant may begin academic work with somerestrictions on academic load and in combinationwith concurrent work in English language classes.(This implies that the institution can provide theappropriate English courses to complement theapplicant’s part-time academic schedule.)

n The applicant is declared eligible to begin anacademic program within a stipulated period oftime but is assigned to a full-time program ofEnglish study. (Normally, such a decision is madewhen an institution has its own intensive English-as-a-second-language program.)

n The applicant’s official status will not be deter-mined until he or she reaches a satisfactory level ofEnglish proficiency. (Such a decision will requirethat the applicant pursue full-time English train-ing, at the same institution or elsewhere.)

All of the above decisions require the institutionto judge whether the applicant has sufficientcommand of English to meet the demands of aregular or modified program of study. Suchdecisions should never be based on TOEFLscores alone; they should be based on all rel-evant information available.

Who Should Take the TOEFL Test?All international applicants who are nonnativespeakers of English should provide evidence of theirlevel of English proficiency prior to beginning aca-demic work at an institution where English is thelanguage of instruction. TOEFL scores are frequentlyrequired for the following categories of applicants:

n Individuals from countries in which English is oneof the official languages, but not necessarily thefirst language of the majority of the population orthe language of instruction at all levels of school-ing. Such countries may include, but are notlimited to, the British Commonwealth countriesand US territories and possessions.

n Persons from countries where English is not thenative language, even though there may be schoolsor universities in which English is the language ofinstruction.

Many institutions report that they frequently donot require TOEFL test scores of certain kinds ofinternational applicants. These include:

n Nonnative speakers who hold degrees or diplomasfrom postsecondary institutions in English-speaking countries (e.g., the United States, Canada,England, Ireland, Australia, New Zealand),provided they have spent a specified minimumperiod of time in successful full-time study(generally two years) with English as the languageof instruction.

n Transfer students from other institutions in theUnited States or Canada after favorable evaluation ofprevious academic course work and course load andlength of time at the previous institution.

n Nonnative speakers who have taken the TOEFLtest within the past two years and who havesuccessfully pursued academic work in an English-speaking country for a specified minimum periodof time (generally two years) with English as thelanguage of instruction.



26

Guidelines for UsingTOEFL Test ScoresAs part of its general responsibility for the tests itproduces, the TOEFL program is concerned about theuse of TOEFL test scores by recipient institutions.The program office makes every effort to ensurethat institutions use TOEFL scores properly — forexample, by providing this Manual to all institutionsthat are interested in using the scores and by regularlyadvising institutions of any program changes that mayaffect the interpretation of TOEFL test scores. TheTOEFL office encourages individual institutions torequest assistance of TOEFL professional staff relatingto the proper use of scores.

An institution that uses TOEFL test scores shouldconsider certain factors to evaluate an individual’sperformance on the test and to determine appropriatescore requirements. The following guidelines arepresented to assist institutions in arriving at reason-able decisions.

n Base the evaluation of an applicant’sreadiness to begin academic work on allavailable relevant information, not solelyon TOEFL test scores.

The TOEFL test measures an individual’s ability inseveral areas of English language proficiency. It isnot designed to provide information about scholasticaptitude, motivation, language-learning aptitude,or cultural adaptability. The eligibility of a foreignapplicant should be fully established on the basisof all relevant academic and other criteria, includingsufficient proficiency in English to undertake theacademic program at that institution.

n Do not use rigid cut-off scores to evaluate anapplicant’s performance on the TOEFL test.

Because test scores are not perfect measures of ability,the use of rigid cut-off scores should be avoided. Thestandard error of measurement should be understoodand taken into consideration in making decisionsabout an individual’s test performance or in establish-ing appropriate critical score ranges for theinstitution’s academic demands (see “Reliabilities andthe Standard Error of Measurement,” page 29).

* See page 39 for information about the Test of Spoken English andoral proficiency.

n Consider TOEFL section scores as well astotal scores.

The total score on the multiple-choice TOEFL testis based on the scores of the three sections of the test.Although a number of applicants may achieve thesame total score, they may have different section scoreprofiles, which could significantly affect subsequentacademic performance. For example, an applicant witha low score on the Listening Comprehension sectionbut relatively high scores on the other sections mighthave greater initial difficulty in lecture courses.*This information could be used in advising andplacing applicants.

If an applicant’s score on the Structure and WrittenExpression section is considerably lower than thescores on the other sections or if the applicant’s scoreon the TWE test is low, it may be that the individualshould take a reduced academic load or be placed ina course designed to improve composition skills andknowledge of English grammar. An applicant whosescore on the Reading Comprehension section is muchlower than the scores on the other two sections mightbe advised to take a reduced academic load or topostpone enrollment in courses that involve a signifi-cant amount of reading.*

n Consider the kinds and levels of Englishproficiency required in different fields andlevels of study and the resources available atthe institution for improving the Englishlanguage skills of nonnative speakers.

An applicant’s field of study can affect the kind andlevel of language proficiency that are appropriate.Students pursuing studies in fields requiring highverbal ability (such as journalism) will need a greatercommand of English, particularly structure andwritten expression and writing, than will those infields that are not so dependent upon reading andwriting abilities. Many institutions require a higherrange of TOEFL test scores for graduate applicantsthan for undergraduates.

Institutions offering courses in English for nonna-tive speakers of English can modify academic courseloads to allow for additional concurrent languagetraining, and thus may be able to consider applicantswith a lower range of scores than can institutions thatdo not offer additional language training.

* See page 39 for information about TSE.



27

* Chase and Stallings, 1966; Heil and Aleamoni, 1974; Homburg, 1979;Hwang and Dizney, 1970; Odunze, 1980; Schrader and Pitcher, 1970;Sharon, 1972.

** A separate publication, “Guidelines for TOEFL Institutional ValidityStudies,” provides information to assist institutions in the planning oflocal validity studies. This publication is available without charge fromthe TOEFL program office upon request.

n Consider TOEFL test scores to helpinterpret an applicant’s performance onother standardized tests.

International applicants are frequently required totake standardized admission tests in addition to theTOEFL test. In such cases, TOEFL scores may proveuseful in interpreting the scores obtained on the othertests. For example, if an applicant’s TOEFL scoresare low and the scores on another test are also low(particularly one that is primarily a measure ofaptitude or achievement in verbal areas), one canlegitimately infer that the applicant’s performanceon the other test was impaired because of deficienciesin English. On the other hand, application recordsof students with high verbal aptitude scores but lowTOEFL scores should be reviewed carefully. Thescores may not be valid.

Interpreting the relationship between the TOEFLtest and aptitude and achievement tests in verbalareas can be complex. Few of even the most qualifiedforeign applicants approach native proficiency inEnglish. Factors such as cultural differences ineducational programs may also affect performanceon tests of verbal ability.

The TOEFL program has published four researchreports that can assist in evaluating the effect oflanguage proficiency on an applicant’s performanceon specific standardized tests.

The Performance of Nonnative Speakers of English onTOEFL and Verbal Aptitude Tests (Angelis, Swinton,and Cowell, 1979) gives comparative data aboutforeign student performance on TOEFL and eitherthe GRE verbal or the SAT verbal and the Test ofStandard Written English (TSWE). It providesinterpretive information about how combined testresults might best be evaluated by institutions that areconsidering foreign students. The Relationship BetweenScores on the Graduate Management Admission Testand the Test of English as a Foreign Language (Powers,1980) provides a similar comparison of performanceon the GMAT and TOEFL tests. Finally, LanguageProficiency as a Moderator Variable in Testing Aca-demic Aptitude (Alderman, 1981) and GMAT andGRE Aptitude Test Performance in Relation to PrimaryLanguage and Scores on TOEFL (Wilson, 1982)contain information supplementing that providedin the other two studies. (See “Validity,” page 34.)

n Do not use TOEFL test scores to predictacademic performance.

The TOEFL test is designed to be a measure ofEnglish language proficiency, not of academic apti-tude. Although there may be some unintended overlapbetween language proficiency and academic aptitude,other tests have been designed to measure academicaptitude more precisely and are available for thatpurpose. Use of TOEFL scores to predict academicperformance is inappropriate. Numerous predictivevalidity studies,* using grade-point averages ascriteria, have been conducted in the past. Thesestudies have shown that correlations between TOEFLtest scores and grade-point averages are often too lowto be of any practical significance. Moreover, lowcorrelations are to be expected when TOEFL scoresare used properly. If an institution admits thoseinternational applicants who have demonstrated ahigh level of language competence, one would expectthat English proficiency would no longer be highlycorrelated with academic success.

The English proficiency of an internationalapplicant is not as stable a characteristic as verbal ormathematical aptitude. Proficiency in a language issubject to change over relatively short periods of time.If considerable time has passed between the date onwhich an applicant took the TOEFL test and the dateon which he or she actually begins academic studies,there may be a greater impact on academic perfor-mance due to language loss than had been anticipated.On the other hand, a student who might be disadvan-taged because of language problems during the firstterm of study might not be disadvantaged in subse-quent terms.

n Assemble information about the validityof TOEFL test score requirements at theinstitution.

The TOEFL program strongly encourages users todesign and carry out institutional validity studies.**Because it is important to establish appropriate stan-dards of language proficiency, validity evidence mayprovide support for raising or lowering a particularstandard as necessary. It may also be used to defend thestandard should its legitimacy be challenged.



28

An important source of validity evidence forTOEFL scores is contained in information aboutsubsequent performance by applicants who areadmitted. Student scores may be compared to a varietyof criterion measures, such as teacher (or adviser)ratings of English proficiency, graded written presenta-tions, grades in ESL courses, and self-ratings of Englishproficiency. However, when evaluating a standard withdata obtained solely from individuals who have met thestandard (that is, only students who have been admit-ted), an interesting phenomenon may occur. If thecurrent standard is set at a high level, so that onlythose with a high degree of language proficiency areadmitted, there may be no relationship between theTOEFL scores and any of the criterion measures.Because there will be no important variability inEnglish proficiency among the group members, varia-tions in success on the criterion variable will likely bedue to other causes, such as knowledge of the subjectmatter, academic aptitude, study skills, culturaladaptability, and financial security.

On the other hand, if the language proficiencystandard is set at a low level, a large number ofapplicants selected with TOEFL scores may beunsuccessful in the academic program because ofinadequate command of English, and there will bea relatively high correlation between their TOEFLscores and its criterion measure. Also, with a standardthat is neither too high nor too low, the correlationbetween TOEFL scores and subsequent success willbe only moderate. The magnitude of the correlationwill depend on other factors as well. These factorsmay include variability in scores on the criterionmeasure and/or the reliability of the raters, if ratersare used. Expectancy tables can be used to show thedistribution of performance on the criterion variablesfor students with given TOEFL scores. Thus, it maybe possible to depict the number or percentage ofstudents at each score level who attain a certainlanguage proficiency rating as assigned by an instruc-tor, or who rate themselves as not being hamperedby lack of English skills while pursuing college-level studies.

Another approach is to use a regression equationto support a score standard. Additional informationabout the setting and validation of test score standardsis available in a manual by Livingston and Zieky(1982).

Several other methodological issues should beconsidered when conducting a standard-setting orvalidation study. Because language proficiency canchange within a relatively short time, student perfor-mance on a criterion variable should be assessedduring the first term of enrollment. However, ifTOEFL scores are not obtained immediately priorto admission, gains or losses in language skills mayreduce the relationship between the TOEFL test andthe criterion.

Another issue that should be addressed is therelationship between subject matter or level of studyand language proficiency. All subjects may not requirethe same level of language proficiency for the studentto perform acceptably. For instance, the study ofmathematics normally requires a lesser degreeof English language proficiency than the studyof philosophy. Similarly, first-year undergraduateswho are required to take courses in a wide range ofsubjects may require a level of language proficiencydifferent from that of graduate students who areenrolled in a specialized field of study.

Section scores may also be taken into considerationin the setting and validating of score standards. Forfields that require a substantial amount of reading,the Reading Comprehension score may be particularlyimportant. In fields that require little writing, theStructure and Written Expression or TWE score maybe less important. Assessment of the relationship ofsection scores to the criterion variables can furtherrefine the process of interpreting TOEFL scores.

To be useful, data about subsequent performancemust be collected for relatively large numbers ofstudents over an extended period of time. Institutionsthat have only a small number of foreign applicantseach year or that have only recently begun to requireTOEFL scores may not find it feasible to conduct therecommended studies. Such institutions might find ithelpful to seek information and advice from collegesand universities that have had more extensive experi-ence with the TOEFL test. The TOEFL office suggeststhat institutions evaluate their TOEFL requirementsregularly to ensure that they are consistent with theinstitutions’ own academic requirements and thelanguage training resources they can provide nonna-tive speakers of English.



29

Level of DifficultyIt is generally agreed by measurement specialists thatthe TOEFL test will provide the best measurement inthe critical score range of about 450 to 600 when thetest is of moderate difficulty. One indicator of testdifficulty is provided by the percentage of correctitems. The mean percent correct for the sections forthe 13 different forms administered between July1995 and June 1996 falls within 58.3 percent and 81.6percent of the maximum possible score. For ListeningComprehension, the average percent correct rangesfrom 58.3 to 75.8 percent, with a mean percentcorrect of 67.3. For Structure and Written Expression,the values range from 63.7 to 81.1 percent, with amean percent correct of 69.7. For Reading Compre-hension, the values range from 59.1 to 78.7 percent,with a mean percent correct of 69.1.

Percent correct, as a measure of difficulty, dependsboth on the inherent difficulty of the test and on theability level of the group of examinees that took thetest. Both factors are of concern in determiningwhether the test is properly matched to the abilitylevel of the examinees. However, for the scaled scoresthat are reported to examinees and institutions, theeffect of the differences in difficulty level among thevarious forms of the test is removed, or adjusted for,by a statistical process called score equating. (See“Calculation of TOEFL Scores,” page 22.)

Test EquatingTOEFL test equating has two major purposes: (1) toadjust minor differences in difficulty among differentTOEFL forms to ensure that examinees having equallevels of English proficiency will receive equivalentscaled scores and (2) to ensure that scores fromdifferent TOEFL forms are on a common scale sothat they are comparable. To equate scores, theTOEFL program employs a “true score” equatingmethod based on item response theory (Cook andEignor, 1991; Hambleton and Swaminathan, 1985;Lord, 1980). All new TOEFL forms are equated tothe TOEFL base form administered in July 1995.The equating procedure consists of establishing whatscores on the new TOEFL form and on the TOEFLbase form correspond to the same level of Englishproficiency. Scores for the new TOEFL form and thebase form corresponding to the same level of Englishproficiency are considered to be equivalent. An

STATISTICAL CHARACTERISTICSOF THE TEST ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

examinee’s equated score, then, is the score on theJuly 1995 (or base) form for each section correspond-ing to the examinee’s score for each section on thecurrent form. The examinee’s converted, or reported,scores are obtained by applying the nonlinear conver-sion table originally obtained for each section on thebase form to the examinee’s equated section scores.

Adequacy of Time AllowedAlthough no single statistic has been widely acceptedas a measure of the adequacy of time allowed for aseparately timed section, two rules of thumb are usedat ETS: (1) 80 percent of the group ought to be ableto finish almost every question in each section, and(2) 75 percent of the questions in a section ought tobe completed by almost all of the group. The ListeningComprehension section of the TOEFL test is pacedby a recording; thus, every question is presented toevery examinee and the criteria for speededness donot apply.

For Sections 2 and 3 of the 13 forms administeredbetween July 1995 and June 1996, at least 94 percentof each group of examinees were able to complete allthe questions in each section, and the three-quarterpoint in the sections was reached by 99.1 to 100.0percent. Thus, one may reasonably conclude that,by these criteria, speed is not an important factor inTOEFL scores.

Reliabilities and the StandardError of MeasurementThe TOEFL test is an accurate and dependablemeasure of proficiency in English as a foreign lan-guage. However, no test score is entirely withoutmeasurement error. This does not mean that someonehas made a mistake in constructing or scoring thetest. It means only that examinees’ scores are notperfectly consistent, due to a number of factors. Theextent to which test scores are free from errors in themeasurement process is known as reliability. Reliabil-ity describes the tendency of individual examinees’scores to have the same relative positions in thegroup, no matter which form of the test the examin-ees take. Test reliability can be estimated by a varietyof different statistical procedures. The two mostcommonly used statistical indices are the reliabilitycoefficient and the standard error of measurement.



30

The standard error of measurement (SEM) is anestimate of the probable extent of the error inherentin a test score due to the imprecision of the measure-ment process. As an example, suppose that a numberof persons, all possessing the same degree of Englishlanguage proficiency, were to take the same TOEFLtest form. Despite their equal proficiency, thesepersons would not all get the same TOEFL score. Afew would get much higher scores than the rest, a fewmuch lower; however, most would obtain TOEFLscores that were close to the scores that representedtheir actual proficiency. The variation in scores couldbe attributable to differences in motivation, attentive-ness, the particular items on the TOEFL test, andother factors such as those mentioned above. Thestandard error of measurement is an index of howmuch the scores of examinees having the same actualproficiency can be expected to vary.

Interpretation of the standard error of measure-ment is based on concepts in statistical theory andis applied with the understanding that errors ofmeasurement can be expected to follow a particularsampling distribution. In the above example, the scorethat each of the persons with the same proficiencywould achieve on the test if there were no errors ofmeasurement is called the “true score.” The observedscores that these persons could be expected to actuallyreceive are assumed to be normally distributed aboutthis true score. That is, the true score is assumed to bethe expected value (i.e., the mean) of the distributionof observed scores. The standard deviation of thisdistribution is the standard error of measurement.

Note that the standard error of measurementdefined this way is actually the conditional standarderror of measurement (CSEM) given a particular truescore. That is, the standard deviation of the distribu-tion for the observed scores corresponding to aparticular true score is the CSEM given that truescore. Typically the CSEMs for particular true scorespeak in the middle of the score range and decrease asthe true scores increase. This is because for highertrue scores the corresponding observed scores have asmaller range of possible variation. As evidenced byTOEFL data from July 1995 to June 1996, for Section2 the CSEM for a scaled score of 45 is 3.16, muchbigger than 1.94, the CSEM for a scaled score of 60.

The term “reliability coefficient” is generic, reflect-ing the fact that a variety of coefficients exist becauseerrors in the measurement process can arise froma variety of sources. For example, sources of errorcan be found from variations in the sample of tasksrequired by the testing instrument, or in the way thatexaminees respond during the course of a single testadministration. Reliability coefficients that quantifythese sources are known as measures of internalconsistency, and they refer to the reliability of ameasurement instrument at a single point in time.It is also possible to obtain reliability coefficients thattake into account additional sources of error, such aschanges in the performance of examinees from dayto day and/or variations due to different test forms.Typically, these latter measures of reliability aredifficult to obtain because they require that a groupof examinees be retested with the same or anothertest form on another occasion.

In numerical value, reliability coefficients arealways between .00 and .99, and generally between.60 and .95. The closer the value of the reliabilitycoefficient to the upper limit, the greater the freedomof the test from error in measurement. Table 2 givesaverage internal consistency reliabilities of the scaledscores for each of the three multiple-choice sectionsand for the total test based on TOEFL test formsadministered between July 1995 and June 1996. For asomewhat different view of reliability that looks atlocal dependence in TOEFL reading comprehensionitems and some listening comprehension items, seeWainer and Lukhele (in press).

Table 2. Reliabilities and Standard Errorsof Measurement (SEM)*

Section Reliability SEM

1. Listening Comprehension .90 2.02. Structure and Written Expression .86 2.73. Reading Comprehension .89 2.4

Total Score .95 13.9

* The medians of forms administered between July 1995 and June 1996.Based only on examinees tested in the United States and Canada.



31

Once the CSEMs are defined and calculated, theSEM for a section scaled score can be computed as theweighted average of the CSEMs, with the weightsbased on the scaled score distribution.

When computing the CSEMs and then the SEM,because the true item and ability parameters areunknown, estimated item and ability parameters areused. The resulting CSEMs and SEM will likely differsomewhat from their actual true values (they are notnecessarily just underestimates of the true values).However, the effect of estimation error on the re-ported values of the CSEMs and SEM is likely to besmall for two reasons: (1) the effect of estimationerror of item and ability parameters on the CSEMs(and SEM) is through its effect on the item character-istic curves, and in general the item characteristiccurves are robust to modest changes in item andability parameters; and (2) the CSEMs (and the SEM)are related to the item characteristic curves, through asummation process, and in the summation process,each item contributes only a small amount to theCSEMs. Unless estimation error causes the itemcontributions to all be inaccurate in the same direc-tion (which is very unlikely), the effect will becanceled out through the summation process.

In most instances the SEM is treated as an averagevalue and applied to all scores in the same way. It canbe expressed in the same units as the reported score,which makes it quite useful in interpreting the scoresof individuals. Table 2 shows that the SEM forSection 1 is 2.0 points; for Section 2, 2.7 points; forSection 3, 2.4 points; and for the total score, 13.9points. There is, of course, no way of knowing justhow much a particular person’s actual proficiencymay have been under- or overestimated from a singleadministration. However, the SEM can be used toprovide score bands or confidence bands aroundobserved scores to arrive at estimates of true scoresof persons in a particular reference group.

Because the section and total score reliabilities(given in Table 2) are quite high for TOEFL, if theobserved scores of examinees are not extreme it isfairly likely that their true scores lie within one SEMof their observed scores. For example, from the data inTable 2, we can be fairly confident that for Section 1,the examinees’ true scores lie within 2 points of theirobserved scores. For the total score, it is fairly likely

that the examinees have true scores within 13.9points of their reported scores. Alternatively, supposea given examinee had a reported score of 50 onSection 3 of the test. We could then say that it is likelythis person’s true score was between 48 and 52. Moreprecise methods for calculating score bands aroundobserved scores to estimate true scores are available(see, for example, Harvill, 1991).

In comparing total scores for two examinees, thestandard errors of measurement need to be taken intoaccount. The standard error of the difference betweenTOEFL scores for two examinees is 2 (or 1.414)times the standard error of measurement presentedin Table 2 and takes into account the contribution oftwo error sources in the different scores. One shouldnot conclude that one score represents a significantlyhigher level of proficiency in English than anotherscore unless there is a difference of at least 39 pointsbetween them. In comparing section scores for twopersons, the difference should be at least 6 points forSection 1, at least 8 points for Section 2, and at least 7points for Section 3. (For additional information on thestandard errors of score differences, see Anastasi,1968, and Magnusson, 1967.)

Consideration of the standard error of measurementunderscores the fact that no test score is entirely with-out measurement error, and that cut-off scores shouldnot be used in a completely rigid fashion in evaluatingan applicant’s performance on the TOEFL test. Somejustification for this position follows.

TOEFL scores are used by many different under-graduate and graduate programs in conjunction withcandidates’ other profiles to make admissions deci-sions. Each program has its own requirement as tocandidates’ English proficiency levels. Some mayrequire higher spoken communication skills andothers may require higher writing skills, demandingdifferential consideration of the section scores. Attimes TOEFL scores are used to prescreen candidates,and factors such as applicant pool as well as projectedclassroom size come into play. All these circumstancesmake setting a universal cut-off score impossible aswell as unnecessary. However, many programs dohave their own cut-off scores set to reflect perhaps thebasic level of candidate English proficiency to survivetheir programs, as well as simply to prescreen and


32

reduce the prospective applicant pool. Keep in mind,however, that it is extremely difficult to defend anyparticular cut-off score. The process of setting cut-offscores has been identified by researches as an exampleof a judgment or decision-making task (JDM), andas Jaeger (1994) noted, “responses to JDM tasks,including standard-setting tasks (cut-off scores beingthe outcome) are…responses to problem statementsthat are replete with uncertainty and less thancomplete information.” Also as clearly articulatedby Brennan (1994), “standard setting is a difficultactivity, involving many a priori decisions and manyassumptions.”

Another problem with cut-off scores is that theyare often perceived as arbitrary. As noted by van deLinden (1994, page 100):

The feelings of arbitrariness…stem from thefact that although cut scores have an “all or none”character, their exact location can never bedefended sufficiently. Examinees with achievementjust below a cut score differ only slightly from thosewith achievements immediately above the score.However, the personal consequences of this smalldifference may be tremendous, and it should be nosurprise that these examinees can be seen as thevictims of arbitrariness in the standard-settingprocedure.

Still another problem with the setting of cut-offscores is that the particular method used to set thestandard will clearly affect the results, i.e., differentprocedures will provide different cut-off scores.Standards are constructed rather than discovered, andthere are no “true” standards. As Jaeger (1994)pointed out, “a right answer does not exist, exceptperhaps in the minds of those providing judgments.”All these factors support not using a cut-off score in acompletely rigid fashion in evaluating an applicant’sperformance on TOEFL. (For additional guidelinesfor using TOEFL test scores, see pages 25-28.)

Reliability of Gain ScoresSome users of the TOEFL test are interested in therelationship between TOEFL scores that are obtainedover time by the same examinees. For example, anEnglish language instructor may be interested in thegains in TOEFL scores obtained by students in anintensive English language program. Typically, theavailable data will consist of differences calculated bysubtracting TOEFL scores obtained at the completionof the program from those obtained at the beginningof the program. In interpreting these gain scores,we must inquire how reliable our estimates of thesedifferences are, taking into account the characteristicsof each of the two tests administered.

Unfortunately, it is a fact that the assessment ofthe difference between two test scores usually hassubstantially lower reliability than the reliabilitiesof the two tests taken separately. This is due to twofactors. First, the errors of measurement that occurin each of the tests are accumulated in the differencescore. Second, the common aspects of languageproficiency that are measured on the two occasionsare canceled out in the difference score. This latterfactor means that, other things being equal, thereliability of the difference scores decreases as thecorrelation between pretest and posttest increases.This is because more of what is common between thetwo tests is canceled out of the difference score, andmore of what is left over is made up of the accumu-lated errors of measurement in each of the two tests.As a numerical example, if the reliability of both thepretest and the posttest is about .90 and if the stan-dard deviations of the scores are assumed to be equal,the reliability of the gain scores decreases from .80 to.50 as the correlation between pretest and posttestincreases from .50 to .80. If the correlation betweenpretest and posttest is as high as the reliabilities of thetwo tests, the reliability of the gain scores is zero. Forfurther discussion on the limitations in interpretingdifference scores, see Linn and Slinde (1977), andThorndike and Hagan (1977, pages 98-100).


33

Table 3. Intercorrelations Among the Scores*

Section 1 2 3 Total

1. Listening Comprehension — .68 .69 .862. Structure and Written Expression .68 — .79 .923. Reading Comprehension .69 .79 — .92

Total Score .86 .92 .92 —

* The medians of correlation coefficients for forms administered betweenJuly 1995 and June 1996. Based only on examinees tested in theUnited States and Canada.

The attribution of gain scores in a local settingrequires caution, because gains may reflect increasedlanguage proficiency, a practice effect, and/or astatistical phenomenon called “regression toward themean” (which essentially means that, upon repeatedtesting, high scorers tend to score lower and lowscorers tend to score higher). Swinton (1983)analyzed data from a group of students at San Fran-cisco State University that indicated that TOEFLscore gains decrease as a function of proficiency levelat the time of initial testing. For this group, studentscores were obtained at the start of an intensiveEnglish language program and at its completion 13weeks later. Students whose initial scores were in the353-400 range showed an average gain of 61 points;students whose initial scores were in the 453-500range showed an average gain of 42 points.

As part of this study, an attempt was made toremove the effects of practice and regression towardthe mean by administering another form of theTOEFL test one week after the pretest. Initial scoresin the 353-400 range increased about 20 points onthe retest, and initial scores in the 453-500 rangeimproved about 17 points on the retest. The greaterpart of these gains can be attributed to practice andregression toward the mean, although a small partmay reflect the effect of one week of instruction.

Subtracting the retest gain (20 points) from theposttest gain (61 points), it was possible to determinethat, within this sample, students with initial scoresin the 353-400 range showed a real gain on theTOEFL test of 41 points during 13 weeks of instruc-tion. Similarly, students in the 453-500 initial scorerange showed a 25-point gain in real language profi-ciency after adjusting for the effects of practice andregression. Thus, the lower the initial score, thegreater will be the probable gain over a fixed periodof instruction. Other factors, such as the nature ofthe instructional program, will affect gain scores also.

The TOEFL program has published a manual(Swinton, 1983) that describes a methodology suitablefor conducting local studies of gain scores. University-affiliated and private English language institutes maywish to conduct gain score studies with their ownstudents to determine the amount of time that isordinarily required to improve from one score levelto another.

Intercorrelations Among ScoresThe three multiple-choice sections of the TOEFL testare designed to measure different skills within thegeneral domain of English proficiency. It is commonlyrecognized that these skills are interrelated; personswho are highly proficient in one area tend to beproficient in the other areas as well. If this relation-ship were perfect, there would be no need to reportscores for each section. The scores would representthe same information repeated several times, ratherthan different aspects of language proficiency.

Table 3 gives the correlation coefficients measuringthe extent of the relationships among the threesections and with the total test score. A correlationcoefficient of 1.0 would indicate a perfect relationshipbetween the two scores, and 0.0 would indicate a totallack of relationship. The table shows average correla-tions over the forms administered between July 1995and June 1996. Correlations between the sectionscores and the total score are spuriously high becausethe section scores are included in the total. Theobserved correlations, ranging from .68 to .79,indicate that there is a fairly strong relationshipamong the skills tested by the three multiple-choicesections of the test, but that the section scores providesome unique information.


34

ValidityIn addition to evidence of reliability, there should bean indication that a test is valid — that it actuallymeasures what it is intended to measure. For example,a test of basic mathematical skills that yielded veryconsistent scores would be considered reliable. Butif those scores showed little relationship to students’performance in basic mathematics courses, thevalidity of the test would be questionable. This wouldbe particularly true if the scores showed a strongerrelationship to the students’ performance in lessrelevant areas, such as language or social studies. Thequestion of validity of the TOEFL test relates to howwell it measures a person’s proficiency in English asa second or foreign language.

Establishing the validity of a test is admittedly oneof the most difficult tasks facing those who design thetest. For this reason, validity is usually confirmed byanalyzing the test from a number of perspectives.

Although researchers have stated definitions formany different types of validity, it is generally recog-nized that validity refers to the usefulness of infer-ences made from test scores (APA, 1985; Messick,1987). To support inferences, validation shouldinclude several types of evidence, e.g., content-related,criterion-related, and construct-related. The nature ofthe evidence should depend on the specific inferenceor use of the test.

To establish content-related evidence, one mustdemonstrate that the content exhibited and behaviorelicited on a test constitute an adequate sample of thecontent and behaviors of the subject or field tested.Criterion-related evidence of validity applies whenone wishes to draw a relationship between a scoreon the test under consideration and a score on someother variable, called a criterion. Construct-relatedvalidity evidence should support the integrity of theintended constructs or behavioral domains as mea-sured on the test. For a test that reports a total scoreand three section scores, such as TOEFL, researchshould provide evidence of the integrity of constructsand the validity of inferences associated with everyscore reported. Of the three kinds of validity evidence,content-related evidence is established by examiningthe content of the test, whereas criterion-related andconstruct-related evidence frequently involve judg-ments based on statistical relationships.

Content Validity

Content-related evidence for the TOEFL test is amajor concern of the TOEFL Committee of Examin-ers (see page 8), which has developed a comprehen-sive list of specifications for items appearing in thedifferent sections of the test. The specificationsidentify the aspects of English communication, ability,and proficiency that are to be tested and describeappropriate techniques for testing them. The specifi-cations are continually reviewed and revised asappropriate to ensure that the test reflects bothcurrent English usage and current theory as to thenature of second language proficiency.

A TOEFL research study by Duran, Canale,Penfield, Stansfield, and Liskin-Gasparro (1985)analyzed one form of the TOEFL test from severaldifferent frameworks related to contemporary ideasabout aspects of communicative competence. Theseframeworks take into account the grammatical,sociolinguistic, and discourse competencies requiredto answer TOEFL items correctly. Although thecompetencies and the degree to which the TOEFL testmeasures them vary considerably across sections, theresults indicate that successful performance on thetest requires a wide range of competencies.

Information regarding the perceptions of collegefaculty of the validity of the Listening Comprehensionsection is available in A Survey of Academic DemandsRelated to Listening Skills (Powers, 1985). Powersfound that the kinds of listening comprehensionquestions used in the TOEFL test were rated (byfaculty) as being among the most appropriate ofthose considered.

Bachman, Kunnan, Vanniarajan, and Lynch(1988) suggest that the reading passages in Section 3tend to be entirely academic in focus. This is consis-tent with the intended use of the test as a measure ofproficiency in English for academic purposes.

Although American cultural content is present inthe test, care has been taken to ensure that knowledgeof such content is not required to succeed in respond-ing to any of the items. Angoff (1989), in a studyusing one form of the TOEFL test with more than20,000 examinees tested abroad and more than 5,000examinees tested in the United States, establishedthat there was no detected cultural advantage forexaminees who had resided more than one year inthe United States.


35

In 1984, the TOEFL program held an invitationalconference to discuss the content validity of the test.The conference brought together some two dozenspecialists in the testing of English as a secondlanguage. The papers presented at the conferenceare available in Toward Communicative CompetenceTesting: Proceedings of the Second TOEFL InvitationalConference (Stansfield, 1986). These papers provideadditional information about the language tasks thatappear on the TOEFL test and are an importantreference for an understanding of the content validityof the test. Subsequent changes in the test, designedto make it more reflective of communicative compe-tence, are enumerated on pages 92 and 93 ofthe proceedings.

Criterion-Related Validity

Some of the earliest and most basic TOEFL researchattempted to match performance on the test withother indicators of English language proficiency,thus providing criterion-related evidence of TOEFL’svalidity. In some cases these indicators were teststhemselves.

A study conducted by Maxwell (1965) at theBerkeley campus of the University of California foundan .87 correlation between total scores on the TOEFLtest and the English proficiency test used for theplacement of foreign students at that campus. Thiscorrelation was based on a total sample of 238students (202 men and 36 women, 191 graduates and47 undergraduates) enrolled at the university duringthe fall of 1964. Upshur (1966) conducted a study todetermine the correlation between TOEFL and theMichigan Test of English Language Proficiency. Thiswas based on a total group of 100 students enrolled atSan Francisco State College (N = 50), IndianaUniversity (N = 38), and Park College (N = 12) andyielded a correlation of .89. Other studies comparingTOEFL and Michigan Test scores have been done byPack (1972) and Gershman (1977). In 1966 a studywas carried out at the American Language Institute(ALI) at Georgetown University comparing scores onTOEFL with scores on the ALI test developed atGeorgetown. The correlation of the two tests for 104students was .79.

In addition to comparing TOEFL with other tests,some of these studies included investigations of howperformance on TOEFL related to teacher ratings. Inthe ALI Georgetown study the correlation betweenTOEFL and these ratings for 115 students was .73.Four other institutions reported similar correlations.Table 4 gives the data from these studies. At eachof the institutions (designated by code letters in thetable) the students were ranked in four, five, or sixcategories based on their proficiency in English asdetermined by university tests or other judgmentsof their ability to pursue regular academic courses(American Language Institute, 1966).

In a study conducted on the five-section version ofthe test used prior to 1976, Pike (1979) investigatedthe relationship of the TOEFL test and its subsectionsto a number of alternate criterion measures, includingwriting samples, cloze tests, oral interviews, andsentence-combining exercises. In general, the resultsconfirmed a close relationship between the fivesections of the TOEFL test and the English skills theywere intended to measure. Among the most significantfindings of this study were the correlations betweenTOEFL subscores and two nonobjective measures: oralinterviews and writing samples (essays).

Table 4. Correlations of Total TOEFL Scoreswith University Ratings

Number CorrelationsUniversity of Students with Ratings

A 215 .78B 91 .87C 45 .76D 279 .79


36

Table 5 gives the correlation coefficients for thethree language groups participating in the study.Moreover, the figures are shown for both the totalinterview ratings and the grammar and vocabularysubscores; the essay ratings are listed according to twodifferent scoring schemes — one focusing on essaycontent and one on essay form. The strong correla-tions and common variances found in Pike’s studybetween some of the sections of the TOEFL test ledto the combining and revising of those sections toform the current three-part version of the test.

Further evidence for the criterion-related validityof the TOEFL, TSE, and TWE tests was provided byHenning and Cascallar (1992) in a study relatingperformance on these examinations to independentratings of oral and written communicative languageability over a variety of controlled academic commu-nicative functions.

Construct Validity

In early attempts to obtain construct-related evidenceof validity for the TOEFL test, two studies wereconducted comparing the performance of native andnonnative speakers of English on the test. Angoff andSharon (1970) found that the mean TOEFL scoresof native speakers in the United States were muchhigher than those of foreign students who had takenthe same test. Evidence that the test was quite easyfor the American students is found in the observa-tions that their mean scores were not only high buthomogeneously high relative to those of the foreignstudents; that their score distributions were highlynegatively skewed; and that a high proportion ofthem earned maximum or near-maximum scoreson the test.

A more detailed study of native speaker perfor-mance on the TOEFL test was conducted by Clark(1977). Once again, performance on the test as awhole proved similar to that of the native speakersincluded in the Angoff and Sharon study. The meanraw score for the native speakers, who took twodifferent forms of the TOEFL test, was 134 (out of150). This compared to mean scores of 88 and 89 forthe nonnative speakers who had originally taken thesame forms. However, additional analysis showedthat the native speakers did not perform equally wellon all three sections of the test.

Such information is useful for test developmentbecause it provides guidelines on which to baseevaluations of questions at the review stage. Theinformation from these comparisons of native andnonnative speakers of English also provides evidenceof the construct validity of the TOEFL test as ameasure of English language proficiency.

More recent evidence for the construct validityof the TOEFL test is available in a series of studiesinvestigating the factor structure and dimensionalityof the test (Boldt, 1988; Hale, Rock, and Jirele, 1989;Oltman, Stricker, and Barrows, 1988). Evidence forthe validity of constructs measured by current andprospective listening and vocabulary item types ispresented in Henning (1991a, 1991b). A number ofother construct validity studies are available in theTOEFL Research Report Series (see pages 43-45), themost recent of which bear on some construct validityevidence for the reading and listening portions ofTOEFL (Freedle and Kostin, 1993, 1996; Nissan,DeVincenzi, and Tang, 1996; and Schedl, Thomas,and Way, 1995).

Table 5. Correlations of TOEFL Subscoreswith Interview and Essay Ratings

Interview EssayN Gram. Vocab. Total Content Form

Peru 95 .84 .84 .84 .83 .91Listening Chile 143 .76 .75 .78 .76 .83Comprehension

Japan 192 .84 .83 .82 .59 .72

Peru 95 .86 .87 .87 .86 .92English Chile 143 .88 .87 .87 .88 .98Structure

Japan 192 .70 .69 .71 .55 .81

Peru 95 .82 .83 .82 .80 .84Vocabulary Chile 143 .77 .77 .75 .74 .83

Japan 192 .55 .62 .59 .45 .66

Peru 95 .88 .87 .87 .84 .85Reading

Chile 143 .74 .76 .75 .67 .82Comprehension Japan 192 .62 .62 .62 .61 .73

Peru 95 .86 .85 .86 .85 .93Writing

Chile 143 .79 .78 .75 .77 .88Ability Japan 192 .59 .62 .60 .64 .73


37

Other evidence of TOEFL’s validity is presentedin studies that have focused on the relationship of theTOEFL test to some widely used aptitude tests. Thefindings of these studies contribute to the construct-related validity evidence by showing the extent towhich the test has integrity as a measure of profi-ciency in English as a foreign language. One of thesestudies (Angelis, Swinton, and Cowell, 1979) com-pared the performance of nonnative speakers ofEnglish on the TOEFL test with their performanceon the verbal portions of the GRE Aptitude (nowGeneral) Test (graduate-level students) or both theSAT and the Test of Standard Written English(undergraduates). As indicated in Table 6, the GREverbal performance of the nonnative speakers wasmuch lower and less reliable than the performanceof the native speakers. Similar results were reportedfor undergraduates on the SAT verbal and the TSWE(Table 7).

Wilson (1982) conducted a similar study of allGRE, TOEFL, and GMAT examinees during a two-year period extending from 1977 to 1979. Theseresults, depicted in Table 8, combined with thoseobtained in the earlier study by Angelis, Swinton,and Cowell (1979), warrant an important conclusionfor admissions officers: verbal aptitude test scoresof nonnative examinees are significantly lower onaverage than the scores earned by native Englishspeakers. On the other hand, quantitative aptitudescores are not greatly affected by a lack of languageproficiency. Further, analyses of each study showthat only when TOEFL scores reach approximatelythe 625 level do verbal aptitude test scores of foreigncandidates reach the level normally obtained by nativeEnglish speakers.

Table 6. TOEFL/GRE Verbal Score Comparisons

Mean S.D. Rel. S.E.M.

TOEFL 523 69 .95 15(Nonnatives) (N = 186) GRE-V 274 67 .78 30Native Speakers (N = 1,495) GRE-V 514 128 .94 32

Table 7. TOEFL/SAT and TSWE Score Comparisons

Mean S.D Rel. S.E.M.

TOEFL 502 63 .94 16(Nonnatives) (N = 210) SAT-V 269 67 .77 33Native Speakers (N = 1,765) SAT-V 425 106 .91 32(Nonnatives) (N = 210) TSWE 28 8.8 .84 4Native Speakers (N = 1,765) TSWE 42.35 11.09 .89 3.7

Table 8. TOEFL, GRE, and GMATScore Comparisons, 1977-79

GRE Sample GMAT Sample

All Foreign ESL TOEFL All Foreign ESL TOEFL

N 831,650 2,442 2,442 563,849 3,918 3,918

Verbal Mean 479 345 NA 26 15.7 NASD 129 95 NA 9 7.7 NA

Quantitative Mean 518 606 NA 27 29 NASD 135 136 NA 8 9.2 NA

Analytical Mean 496 400 NA NA NA NASD 120 114 NA NA NA NA

Total Mean NA NA 552 462 389.8 541.8SD NA NA 61 105 97.5 71.7

To provide guidelines for those who may beevaluating applicants presenting scores from morethan one of the above tests, Angelis, Swinton, andCowell (1979) conducted special analyses. Resultsindicated that, for graduate-level applicants, 475on the TOEFL test is a critical decision point forinterpretations of GRE verbal scores. Applicantsabove that level tend to have GRE verbal scores that,although lower than scores for native speakers, fallwithin an interpretable range of verbal ability forstudents with homogeneous TOEFL scores. Thosebelow the 475 TOEFL level tend to have such lowGRE verbal scores that such interpretations cannoteasily be made. At the undergraduate level, 435 onTOEFL is a key decision point. SAT verbal scores forapplicants below that level are not likely to be infor-mative. Similarly, Powers (1980) found that a TOEFLscore of 450 is required before GMAT verbal scoresbegin to discriminate effectively among examinees.

These results suggest that, when TOEFL scoresenter the range normally considered for admissionsdecisions, it is also possible to draw valid inferencesfrom scores on aptitude tests.

As noted earlier, interpreting the relationshipbetween language proficiency and aptitude andachievement test scores in verbal areas can be com-plex. Few of even the most qualified internationalapplicants approach native proficiency in English.Thus, verbal aptitude scores of nonnative Englishspeakers are likely to be depressed somewhat evenwhen TOEFL test scores are high. Only whenTOEFL scores are at an average native speaker level(approximately 625 or above) does the distributionof scores on a verbal aptitude test become similar tothe distribution obtained by native English speakers.Cultural factors and cross-national differences ineducational programs may also affect performanceon tests of verbal ability.


38

As noted above, the TOEFL program has publishedthree research reports that can assist in evaluatingthe effect of language proficiency on an applicant’sperformance on specific standardized tests. ThePerformance of Nonnative Speakers of English onTOEFL and Verbal Aptitude Tests (Angelis, Swinton,and Cowell, 1979) gives comparative data about theperformance of a group of foreign students on theTOEFL test and either the GRE verbal or the SATverbal and the TSWE. The Relationship between Scoreson the Graduate Management Admission Test and theTest of English as a Foreign Language (Powers, 1980)compares performance on TOEFL and GMAT.Additional information and comparisons are availablein GMAT and GRE Aptitude Test Performance inRelation to Primary Language and Scores on TOEFL(Wilson, 1982).

TOEFL is currently a three-section test. Supportfor the three-section format is provided by the patternof correlations between each of the TOEFL sectionsand other tests (Angelis, Swinton, and Cowell, 1979).The GRE verbal score correlates highest with theReading Comprehension section of TOEFL (.623).The same section correlates highest (.681) with theSAT verbal score. This is to be expected since bothverbal aptitude tests rely heavily on reading andvocabulary. For the College Board’s TSWE, the highestcorrelation (.708) is with Section 2 of TOEFL, Struc-ture and Written Expression. Again, this is tobe expected because the TSWE uses knowledge ofgrammar and related linguistic elements as indicatorsof writing ability. In all three cases, the lowest correla-tions are those with TOEFL Section 1, ListeningComprehension. Because none of the other testsincludes items that attempt to measure ability tounderstand spoken English, this again is to be expected.

Table 9. Correlations Between GMATand TOEFL Scores*

TOEFL Scores

Structureand

Listening Written ReadingGMAT Score Comprehension Expression Comprehension Total

GMAT Verbal .58 .66 .69 .71GMAT Quantitative .29 .37 .39 .39GMAT Total .52 .61 .64 .66

*Based on 5,781 examinees with TOEFL and GMAT scores.

In another study cited earlier, comparing perfor-mance of nonnative speakers of English on TOEFLand the Graduate Management Admission Test,Powers (1980) reported the same pattern of correla-tions. As indicated in Table 9, the highest GMATverbal-TOEFL correlation is that for the Vocabularyand Reading Comprehension section. Correlations forSection 2 are slightly lower and those for Section 1(listening) are the lowest. The fact that the correla-tions for the quantitative section of the GMAT arethe lowest of all (ranging from .29 to .39) providessupport for the discriminating power of the TOEFLtest as a measure of verbal skills in contrast toquantitative skills.


39

OTHER TOEFL PROGRAMSAND SERVICES ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

TWE Test (Test of Written English)This 30-minute essay test provides the examineewith an opportunity to perform writing tasks similarto those required of students in North Americanuniversities. This includes the ability to generate andorganize ideas on paper, to support those ideas withexamples or evidence, and to use the conventions ofstandard written English.

The examinee is given one topic on which to write.As with other TOEFL test items, the TWE essayquestions are developed by specialists in Englishor ESL, and each essay question is field-tested andreviewed by a committee of composition specialists,the TWE Committee. A pretested topic will beapproved for use in the TWE test only if it elicits arange of responses at a variety of proficiency levels,does not appear to unfairly advantage or disadvantageany examinee or group of examinees, and does notrequire special subject matter knowledge. The essayquestions are also reviewed for racial and cultural biasand content appropriateness according to establishedETS sensitivity review procedures.

After a test administration, each TWE essay is readby two trained and qualified raters, who assign scoresbased on a six-point, criterion-referenced scoringguide. Neither reader knows the score assigned bythe other. In the case of a discrepancy of more thanone point, a third reader scores the essay.

The Test of Written English score is not incor-porated into the total TOEFL score. Instead, a separateTWE score is reported on the TOEFL score report.Score recipients receive a copy of the TWE ScoringGuide, which describes the proficiency levels associ-ated with the six holistic score points. Sample essaysat the six score levels are published in the TOEFL Testof Written English Guide.

TWE test results can assist institutions in evaluat-ing the academic writing proficiency of their ESL andEFL students and in placing these students in appro-priate writing courses.

TSE Test (Test of Spoken English)The Test of Spoken English was developed by ETSunder the direction of the TOEFL Policy Council andTSE Committee to provide a reliable measure ofproficiency in spoken English. Because the TSE test isa test of general oral language ability, it is appropriatefor examinees regardless of native language, type ofeducational training, or field of employment.

The TSE test has broad applicability in thatperformance on the test indicates how oral communi-cative language ability might affect the examinee’sability to communicate successfully in an academicor professional environment. TSE scores are used atmany North American institutions of higher educa-tion in the selection of international teaching assis-tants, sometimes called ITAs. The scores are also usedfor selection and certification purposes in the healthprofessions, such as medicine, nursing, pharmacy,and veterinary medicine.

The Test of Spoken English is administered 12times a year on the same dates as the TOEFL test.The test takes approximately 20 minutes and can beadministered to individuals with cassette tape record-ers or to a group using a language laboratory.

The TSE test requires examinees to demonstratetheir ability to communicate orally in English byresponding orally under timed conditions to a varietyof printed and aural stimuli that are designed to elicita variety of responses. All examinee responses arerecorded on tape.

The test consists of 12 items, each of whichrequires examinees to perform a particular speech act.Examples of these speech activities, also called“language functions,” include narrating, recommend-ing, persuading, and giving and supporting an opin-ion. The time allotted for each answer ranges from30 to 90 seconds and is written in parentheses aftereach question.

TSE answer tapes are rated by trained specialistsin the field of English or English as a second language.(The rating scale is different from the rating scaleused for the TSE test prior to July 1995.) Ratersassign a score level for the response to each item byusing a set of descriptors that describe performance atvarious levels of proficiency based on communicativelanguage features. Examinee scores are produced fromthe average of these two ratings and are reported ona score scale of 20 to 60. Official score reports are sentto institutions designated by the examinees.

The TSE rating scale and a sample score report areprinted in the TSE Manual for Score Users. A TSEsample response tape is also available to provide scoreusers with sample examinee responses at the levels ofcommunicative effectiveness represented by particu-lar TSE scores.


40

SPEAKT Kit (Speaking ProficiencyEnglish Assessment Kit)The Speaking Proficiency English Assessment Kit(SPEAK) was developed by the TOEFL program toprovide institutions with a valid and reliable instrumentfor assessing the spoken English of nonnative speakers.

SPEAK consists of several components including aRater Training Kit, test materials, and an ExamineePractice Set, each of which is purchased separately.The Rater Training Kit includes the materials neces-sary for training individuals to rate examinees’recorded responses to the test. The training materialsconsist of a Rater Training Guide, sample responsecassette, training cassettes, testing cassettes, practicerating sheet, and scoring key. Raters determinewhether they have mastered the necessary ratingskills by comparing the ratings they assign to therater-testing cassettes with the correct ratings pro-vided in the Guide.

SPEAK test results can be used to evaluate thespeaking proficiency of applicants for teaching assis-tantships who are not native speakers of English, tomeasure improvement in speaking proficiency over aperiod of time, or to identify teaching assistants andothers who may need additional instruction in English.

Two SPEAK test forms (Test Forms A and B) areavailable to purchasers of the SPEAK kit. The SPEAKtesting materials, which allow repeated test adminis-tration at any convenient location, consist of 30reusable examinee test books, a cassette tape foractual administration of the test, a scoring key, anda pad containing 100 rating sheets.

It is important to be aware that SPEAK is designedfor internal or local use only. SPEAK tests are avail-able for direct purchase by university-affiliatedlanguage institutes, institutional or agency testingoffices, and other organizations or offices servingeducational programs.

SLEPT Test (Secondary LevelEnglish Proficiency Test)The Secondary Level English Proficiency test isdesigned for students entering grades 7 through 12who are nonnative speakers of English. The test is ameasure of proficiency in two primary areas: under-standing spoken English and understanding written

English. The SLEP test is based on the assumptionthat language proficiency is a critical factor in deter-mining the degree to which students can benefit frominstruction. It is not an aptitude test or a measure ofacademic achievement; it is a measure of Englishlanguage ability. The results of the test can be veryhelpful in making placement decisions related toassignment to ESL classes, placement in a mainstreamEnglish-medium program, or exit from an ESLprogram. Because the SLEP scale is sensitive to smallgains in language skills, the test can be useful forprogram evaluation purposes.

There are three different forms of the SLEP test, alldeveloped to the same test specifications, equated, andnorm referenced. Each test form contains 150 mul-tiple-choice questions of eight different item types andis divided into two sections, Listening Comprehensionand Reading Comprehension. The questions in thefirst section of the test use taped samples of spokenEnglish to test listening comprehension and do notrely heavily on written materials. The questions inthe second section measure vocabulary, grammar,and overall reading comprehension and are based onwritten and visual materials. Answer sheets are easilyscored, and technical data for interpreting test resultsare provided in the SLEP Test Manual.

SLEP testing materials are available for directpurchase. The basic package of testing materials foreach form contains 20 SLEP test books, 100 two-plyanswer sheets, a copy of the SLEP Test Manual, anda cassette recording of the listening comprehensionquestions. Each item in the basic package may alsobe purchased separately.

Fee Voucher Service for TOEFLand TSE Score UsersThe TOEFL program offers a fee voucher service forthe convenience of organizations and agencies thatpay TOEFL and/or TSE test fees for some or all oftheir students or applicants. Each fee voucher cardshows the name and code number of the participatinginstitution and is valid only for the specified testingyear and the specific program (TOEFL or TSE)indicated thereon. To participate in the service,institutions must sponsor a minimum of 10 candi-dates per year.


41

Fee voucher cards are distributed by the participatinginstitution or agency directly to the applicants forwhom it will pay the test fees. The applicants, in turn,submit the completed cards in lieu of personal paymentwith their completed registration forms. Following eachTOEFL test administration, the sponsor receives thetest scores of each sponsored examinee who submitteda fee voucher card and an invoice for the number ofcards accepted and processed at ETS.

It is important that applicants register before theregistration closing date for the administration atwhich they wish to test. Test centers will not acceptfee voucher cards as admission documents.

TOEFL Fee Certificate ServiceThe TOEFL Fee Certificate Service allows familyor friends in the United States, Canada, and othercountries where US dollars are available to purchasecertificates from the TOEFL program office. A pur-chaser can then send the certificate to an individualliving in a country with currency exchange restric-tions to use as proof of payment for the test fee whenthe prospective test taker registers for a TOEFLadministration.

Although the fee certificates are especially usefulto individuals living in countries or areas in whichUS dollars are difficult or impossible to obtain, thecertificates will be accepted as a valid form of TOEFLregistration fee payment anywhere in the world(except Japan, Taiwan, and the People’s Republicof China) up to 14 months from the date of issue.

TOEFL MagneticScore-Reporting ServiceA magnetic score-reporting service for TOEFL officialscore-report recipients is available by subscription. Theservice provides TOEFL score reports twice a month toparticipating institutions and agencies for a nominalannual charge. Although individual paper score reportscontinue to be sent to institutions and agencies that aredesignated TOEFL score recipients, the scores can besent only to the central address or admissions officelisted in the TOEFL files. This service can be orderedfor specific offices or departments.

The score records are in single record format on9 track/1600/6250 bpi magnetic tapes, 31/2 inchfloppy disks, formatted for an IBM or IBM-compat-ible personal computer, or cartridges. Each tape ordisk is accompanied by a roster containingall examinee data included on the tape or disk.

The tapes or disks are prepared for each institutionor agency with only the score records of TOEFLexaminees who correctly marked the code numberof the institution or agency on their answer sheetswhen they took the test or who submitted a writtenrequest that their scores be reported to that institu-tion or agency. The magnetic score-reporting serviceprovides a convenient way to merge students’ TOEFLscore data with other student data.

Subscription to this service is for one year (Julyto June) and may begin at any time during the year.

Examinee Identification Servicefor TOEFL and TSE Score UsersThis service provides photo identification of examin-ees taking the TOEFL and TSE tests. The photo filerecord is collected by the test center supervisor fromeach examinee before he or she is admitted to thetesting room.

The official score reports routinely sent to institu-tions and other score recipients designated by thetest taker, and the examinee’s own copy of the scorereport, bear an electronically reproduced photo imageof the examinee and a copy of the test taker’s signa-ture. In a small number of cases, it may not bepossible to reproduce an examinee’s photo image onthe score report. Instead, the words “Photo AvailableUpon Request” will be printed on the reports. Copiesof photographs for these examinees may be obtainedby using the Examinee Identification Service.

If there is reason to suspect an inconsistencybetween a high test score and relatively weak Englishproficiency, an institution or agency that has receivedeither an official score report from ETS or anexaminee’s score record from an examinee mayrequest a copy of that examinee’s photo file recordup to 18 months following the test date shown onthe score report. The written request for examineeidentification must be accompanied by a photocopyof the examinee’s score record or official score report.


42

Support for ExternalResearch StudiesThe TOEFL program will make available certaintypes of test data or perform analyses of pertinentdata requested by external researchers for studiesrelated to assessing English language proficiency.The researchers must agree to (1) protect the confi-dentiality of the data, (2) assume responsibility forthe analyses and conclusions of the studies, and (3)reimburse the TOEFL program for the costs associ-ated with the compilation and formatting of the data.

TOEFL program funding of independent research,if requested and granted, is usually limited to provid-ing test materials and related services without chargeand/or the cost of the data access and data analysis.

Individuals interested in utilizing TOEFL test dataor materials for research studies should write to theTOEFL program office.

For more information about the programsand services described on pages 39-42, visitTOEFL OnLine at http://www.toefl.orgor write to:

TOEFL Program OfficeEducational Testing Service

P.O. Box 6155Princeton, NJ 08541-6155


43

RESEARCH PROGRAM

The purpose of the TOEFL research program is tofurther knowledge in the field of language assessmentand second language acquisition about issues related topsychometrics, language learning and pedagogy, and theproper use and interpretation of language assessmenttools. In light of these diverse goals, the TOEFL re-search agenda calls for continuing research in severalbroad areas of inquiry, including test validation, reliabil-ity, use, construction, and examinee performance.

The TOEFL Research Committee reviews andapproves all research projects and sets guidelinesfor the scope of the TOEFL research program.

TOEFL Research Report SeriesThe results of research studies conducted under thedirection of the TOEFL Research Committee areavailable to the public through the TOEFL ResearchReport and Technical Report Series. In addition tothose listed below, a number of new projects are inprogress or under consideration.

Research report titles available as of July 1997:

1. The Performance of Native Speakers of English on theTest of English as a Foreign Language. Clark.November 1977.

2. An Evaluation of Alternative Item Formats forTesting English as a Foreign Language. Pike. June1979.

3. The Performance of Nonnative Speakers of Englishon TOEFL and Verbal Aptitude Tests. Angelis,Swinton, and Cowell. October 1979.

4. An Exploration of Speaking Proficiency Measures inthe TOEFL Context. Clark and Swinton. October1979.

5. The Relationship between Scores on the GraduateManagement Admission Test and the Test of Englishas a Foreign Language. Powers. December 1980.

6. Factor Analysis of the Test of English as a ForeignLanguage for Several Language Groups. Powers andSwinton. December 1980.

7. The Test of Spoken English as a Measure of Commu-nicative Ability in English-Medium InstructionalSettings. Clark and Swinton. December 1980.

8. Effects of Item Disclosure on TOEFL Performance.Angelis, Hale, and Thibodeau. December 1980.

9. Item Performance Across Native Language Groupson the Test of English as a Foreign Language.Alderman and Holland. August 1981.

10. Language Proficiency as a Moderator Variable inTesting Academic Aptitude. Alderman. November1981.

11. A Comparative Analysis of TOEFL ExamineeCharacteristics, 1977-1979. Wilson. September1982.

12. GMAT and GRE Aptitude Test Performance inRelation to Primary Language and Scores onTOEFL. Wilson. October 1982.

13. The Test of Spoken English as a Measure ofCommunicative Proficiency in the HealthProfessions. Powers and Stansfield. January 1983.

14. A Manual for Assessing Language Growth inInstructional Settings. Swinton. February 1983.

15. A Survey of Academic Writing Tasks Required ofGraduate and Undergraduate Foreign Students.Bridgeman and Carlson. September 1983.

16. Summaries of Studies Involving the Test of Englishas a Foreign Language, 1963-1982. Hale, Stansfield,and Duran. February 1984.

17. TOEFL from a Communicative Viewpoint onLanguage Proficiency: A Working Paper. Duran,Canale, Penfield, Stansfield, and Liskin-Gasparro.February 1985.

18. A Preliminary Study of Raters for the Test of SpokenEnglish. Bejar. February 1985.

19. Relationship of Admission Test Scores to WritingPerformance of Native and Nonnative Speakersof English. Carlson, Bridgeman, Camp, andWaanders. August 1985.

20. A Survey of Academic Demands Related to ListeningSkills. Powers. December 1985.

21. Toward Communicative Competence Testing:Proceedings of the Second TOEFL InvitationalConference. Stansfield. May 1986.

22. Patterns of Test Taking and Score Change forExaminees Who Repeat the Test of English asa Foreign Language. Wilson. January 1987.

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○


44

23. Development of Cloze-Elide Tests of English asa Second Language. Manning. April 1987.

24. A Study of the Effects of Item Option Rearrangementon the Listening Comprehension Section of the Testof English as a Foreign Language. Golub-Smith.August 1987.

25. The Interaction of Student Major-Field Group andTest Content in TOEFL Reading Comprehension.Hale. January 1988.

26. Multiple-Choice Cloze Items and the Test of Englishas a Foreign Language. Hale, Stansfield, Rock,Hicks, Butler, and Oller. March 1988.

27. Native Language, English Proficiency, and theStructure of the Test of English as a ForeignLanguage. Oltman, Stricker, and Barrows.July 1988.

28. Latent Structure Analysis of the Test of Englishas a Foreign Language. Boldt. November 1988.

29. Context Bias in the Test of English as a ForeignLanguage. Angoff. January 1989.

30. Accounting for Random Responding at the End ofthe Test in Assessing Speededness on the Test ofEnglish as a Foreign Language. Secolsky.January 1989.

31. The TOEFL Computerized Placement Test:Adaptive Conventional Measurement. Hicks.January 1989.

32. Confirmatory Factor Analysis of the Test of Englishas a Foreign Language. Hale, Rock, and Jirele.December 1989.

33. A Study of the Effects of Variations of Short-termMemory Load, Reading Response Length, andProcessing Hierarchy on TOEFL Listening Compre-hension Item Performance. Henning. February 1991.

34. Note Taking and Listening Comprehension on theTest of English as a Foreign Language. Hale.February 1991.

35. A Study of the Effects of Contextualization andFamiliarization on Responses to TOEFL VocabularyTest Items. Henning. February 1991.

36. A Preliminary Study of the Nature of Communica-tive Competence. Henning and Cascallar.February 1992.

37. An Investigation of the Appropriateness of theTOEFL Test as a Matching Variable to EquateTWE Topics. DeMauro. May 1992.

38. Scalar Analysis of the Test of Written English.Henning. August 1992.

39. Effects of the Amount of Time Allowed on the Testof Written English. Hale. June 1992.

40. Reliability of the Test of Spoken English Revisited.Boldt. November 1992.

41. Distributions of ACTFL Ratings by TOEFL ScoreRanges. Boldt, Larsen-Freeman, Reed, andCourtney. November 1992.

42. Topic and Topic Type Comparability on the Testof Written English. Golub-Smith, Reese, andSteinhaus. March 1993.

43. Uses of the Secondary Level English Proficiency(SLEP) Test: A Survey of Current Practice. Wilson.March 1993.

44. The Prediction of TOEFL Reading ComprehensionItem Difficulty for Expository Prose Passages forThree Item Types: Main Idea, Inference, andSupporting Idea Items. Freedle and Kostin.May 1993.

45. Test-Retest Analyses of the Test of Englishas a Foreign Language. Henning. June 1993.

46. Multimethod Construct Validation of the Testof Spoken English. Boldt and Oltman.December 1993.

47. An Investigation of Proposed Revisions to Section 3of the TOEFL Test. Schedl, Thomas, and Way.March 1995.

48. Analysis of Proposed Revisions of the Test ofSpoken English. Henning, Schedl, and Suomi.March 1995.

49. A Study of Characteristics of the SPEAK Test.Sarwark, Smith, MacCallum, and Cascallar.March 1995.


45

50. A Comparison of the Performance of Graduateand Undergraduate School Applicants on theTest of Written English. Zwick and Thayer.May 1995.

51. An Analysis of Factors Affecting the Difficultyof Dialogue Items in TOEFL ListeningComprehension. Nissan, DeVincenzi, and Tang.February 1996.

52. Reader Calibration and Its Potential Role inEquating for the Test of Written English. Myford,Marr, and Linacre. May 1996.

53. An Analysis of the Dimensionality of TOEFLReading Comprehension Items. Schedl, Gordon,Carey, and Tang. March 1996.

54. A Study of Writing Tasks Assigned in AcademicDegree Programs. Hale, Taylor, Bridgeman, Carson,Kroll, and Kantor. June 1996.

55. Adjustment for Reader Rating Behavior in theTest of Written English. Longford. August 1996.

56. The Prediction of TOEFL Listening ComprehensionItem Difficulty for Minitalk Passages: Implicationsfor Construct Validity. Freedle and Kostin.August 1996.

57. Survey of Standards for Foreign StudentApplicants. Boldt and Courtney. August 1997.

58. Using Just Noticeable Differences to Interpret Testof Spoken English Scores. Stricker. August 1997.

TOEFL Technical Report SeriesThis series presents reports of a technical nature, suchas those related to issues of multidimensional scalingor item response theory. As of July 1997 there are 13reports in the series.

1 . Developing Homogeneous Scales byMultidimensional Scaling. Oltman and Stricker.February 1991.

2. An Investigation of the Use of Simplified IRT Modelsfor Scaling and Equating the TOEFL Test. Way andReese. February 1991.

3. Development of Procedures for ResolvingIrregularities in the Administration of the ListeningComprehension Section of the TOEFL Test. Way andMcKinley. February 1991.

4. Cross-Validation of the Proportional Item ResponseCurve Model. Boldt. April 1991.

5. The Feasibility of Modeling Secondary TOEFLAbility Dimensions Using Multidimensional IRTModels. McKinley and Way. February 1992.

6. An Exploratory Study of Characteristics Relatedto IRT Item Parameter Invariance with the Testof English as a Foreign Language. Way, Carey, andGolub-Smith. September 1992.

7. The Effect of Small Calibration Sample Sizes onTOEFL IRT-Based Equating. Tang, Way, andCarey. December 1993.

8. Simulated Equating Using Several Item ResponseCurves. Boldt. January 1994.

9. Investigation of IRT-Based Assembly of the TOEFLTest. Chyn, Tang, and Way. March 1995.

10. Estimating the Effects of Test Length and Test Timeon Parameter Estimation Using the HYBRIDModel. Yamamoto. March 1995.

11. Using a Neural Net to Predict Item Difficulty. Boldtand Freedle. December 1996.

12. How Reliable is the TOEFL test? Wainer andLukhele. August 1997.

13. Concurrent Calibration of Dichotomously andPolytomously Scored TOEFL Items Using IRTModels. Tang and Eignor. August 1997.


46

5. TOEFL 2000 — Writing: Composition,Community, and Assessment. Hamp-Lyons andKroll. March 1997.

6. A Review of Research into Needs in English forAcademic Purposes of Relevance to the NorthAmerican Higher Education Context. Waters.November 1996.

7. The Revised Test of Spoken English (TSE):Discourse Analysis of Native Speaker andNonnative Speaker Data. Lazaraton and Wagner.December 1996.

8. Testing Speaking Ability in Academic Contexts:Theoretical Considerations. Douglas. April 1997.

9. Theoretical Underpinnings of the Test of SpokenEnglish Revision Project. Douglas and Smith.May 1997.

10. Communicative Language Proficiency: Definitionand Implications for TOEFL 2000. Chapelle,Grabe, and Berns. May 1997.

TOEFL Monograph SeriesAs part of the foundation for the TOEFL 2000 project(see page 10), a number of papers were commissionedfrom experts within the fields of measurement andlanguage teaching and testing. Critical reviews andexpert opinions were invited to inform TOEFLprogram development efforts with respect to testconstruct, test user needs, and test delivery. Thesemonographs are also of general scholarly interest.Thus, the TOEFL program is pleased to make thesereports available to colleagues in the fields of languageteaching and testing and international studentadmissions in higher education.

1. A Review of the Academic Needs of Native English-Speaking College Students in the United States.Ginther and Grant. September 1996.

2. Polytomous Item Response Theory (IRT) Modelsand Their Applications in Large-Scale TestingPrograms: Review of Literature. Tang.September 1996.

3. A Review of Psychometric and ConsequentialIssues Related to Performance Assessment. Carey.September 1996.

4. Assessing Second Language Academic Readingfrom a Communicative Competence Perspective:Relevance for TOEFL 2000. Hudson.September 1996.

See TOEFL OnLine at http://www.toefl.orgfor new reports as they are published.


47

TOEFL Products andServices CatalogThe catalog provides summaries with photographsof the priced training and student study materialsdeveloped by the TOEFL program staff. There are alsobrief descriptions of the testing programs and relatedservices.

Bulletin of Information forTOEFL, TWE, and TSEThis publication is the primary source of informationfor individuals who wish to take the TOEFL, TWE,and TSE tests at Friday or Saturday testing programadministrations. The Bulletin tells examinees howto register, lists the test centers, provides a briefdescription of the tests, and explains score reportingand other procedures. It also contains the TOEFL,TWE, and TSE calendar, which includes the testdates, registration deadline dates, and mailing datesfor official score reports. In addition, there are practicequestions, detailed instructions for filling out theanswer sheet on the day of the test, an explanationof procedures to be followed at the test center, andinformation about interpreting scores on the tests.

Copies of the Bulletin are available at manycounseling or advising centers, United States embas-sies, and offices of the United States InformationService (USIS). In countries and regions whereregistration is handled by TOEFL representatives,the representatives distribute appropriate editionsof the Bulletin to examinees and local institutions.

Test Center Reference ListThe Test Center Reference List provides TOEFL, TWE,and TSE test dates, registration deadline dates, scorereport mailing dates, and test center locations for theFriday and Saturday testing programs. It also tells howto obtain the appropriate edition of the Bulletin. Thefree list is distributed at the beginning of each testingyear to institutions and organizations that useTOEFL, TWE, and TSE test scores.

PUBLICATIONS ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Test Forms Availableto TOEFL ExamineesAt some test center locations, examinees who actuallytake the test on dates announced in advance by theTOEFL office may obtain the test books used at theseadministrations free of charge. In addition, theseexaminees may order a list of the correct answers, acassette recording of Section 1 (Listening Comprehen-sion), and a copy of their answer sheet with the rawscores marked.

Information about when and how examinees mayavail themselves of this service is given in the appro-priate Bulletin editions for the areas where the serviceis available.

An order form with information about how toorder and pay for the materials is printed on theinside back covers of the test books for these testadministrations.

The availability of this material is subject tochange without notice.

Guidelines for TOEFL Institutional Validity StudiesThis publication provides institutions currentlyusing the TOEFL test with a set of general guidelinesto consider when planning local predictive validitystudies. It covers preliminary considerations, selectingcriteria, specifying subgroups, determining size ofgroup to be studied, selecting predictors, and determin-ing decision standards, and provides reference sources.

TOEFL Test and ScoreData SummaryThe performance of groups of examinees who tookthe TOEFL test during the most recently completedtesting year (June-July) is summarized here. Percen-tile ranks for section and total scaled scores are givenfor graduate and undergraduate students, as well asfor applicants applying for a professional license.Means and standard deviations are provided in tableformat for both males and females. Of particularinterest to many admissions administrators are thedata on section and mean scores for examineesclassified by native language and geographic regionand native country.


48

Institutional TestingProgram BrochureThe Institutional Testing Program (ITP) brochurecontains a description of the TOEFL and Pre-TOEFLtests offered under this program. The brochure alsoprovides sample test questions, details about ETSpolicy regarding testing, information about TOEFLand Pre-TOEFL score interpretation and the releaseof examinee score data, and an order form.

TOEFL Test of WrittenEnglish GuideThis publication provides a detailed description ofthe TWE test as well as the TWE scoring criteria andprocedures. It also provides guidelines for the inter-pretation and use of TWE scores, statistical datarelated to examinee performance on the test, andsample TWE items and essays.

TSE Score User’s ManualThe Manual details the development, use, and scoringof the Test of Spoken English and its off-the-shelfversion, SPEAK (Speaking Proficiency EnglishAssessment Kit). Guidelines for score use andinterpretation are also provided.

To obtain additional copies of the TOEFLTest and Score Manual or any of the freepublications described above, order on-line athttp://www.toefl.org or write to:

TOEFL Program OfficeEducational Testing Service

P.O. Box 6155Princeton, NJ 08541-6155

Secondary Level EnglishProficiency Test BrochureThis publication describes the SLEP test, a conve-nient, off-the-shelf testing program for nonnativeEnglish speaking students entering grades 7 through12. It includes sample test questions and orderinginformation.

The ResearcherThis publication contains brief descriptions of allthe studies done by ETS researchers specific to theTOEFL tests and testing programs. Published annu-ally, The Researcher is available to anyone interestedin ongoing research in such areas as language assess-ment, examinee performance, reliability, and testvalidation. (See pages 43-46 for a list of titles inthe series.)


49

TOEFL STUDY MATERIALSFOR THE PAPER-BASED TESTINGPROGRAM

The study materials described here are official publi-cations of the TOEFL program. They are produced bytest specialists at ETS to help individuals planning totake TOEFL understand the specific linguistic skillsthe test measures and become familiar with themultiple-choice formats used.

TOEFL Sample TestThis popular and very economical study product hasbeen expanded and completely updated. It containsinstructions for taking the TOEFL test and markingthe answers, one practice test, answer sheets for“gridding” the answers to the multiple-choicequestions, an answer key, recorded material for theListening Comprehension section of the test, andscoring information. It also contains practice exercisesfor the Test of Written English.

TOEFL Practice Tests, Volume 1TOEFL Practice Tests, Volume 2These products were created for those who want morethan one test form for practice. Volume 1 contains twotests; Volume 2 contains four. Each volume providesinstructions for taking the test, answer sheets, keys,recorded listening comprehension material with corre-sponding scripts, and scoring information. TOEFLPractice Tests provide hours of exercise material.

TOEFL Test Preparation Kit(new edition available Spring 1998)

The Test Preparation Kit is the most comprehensiveTOEFL study product produced by ETS test special-ists. This kit provides the user with extensive practicematerial in all three sections of the TOEFL test, aswell as the Test of Written English. The kit contains

n four audio cassettes with more than 230 minutesof recorded answer sheet instructions and listeningcomprehension material

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

n workbook with practice and review materials,answer sheets, lists of the correct answers, and aunit devoted to the TWE test

n sealed Test Exercise Book containing the TOEFLand TWE tests — just like the material distributedat the test center

The TOEFL Test Preparation Kit gives the studentan opportunity to hear and practice the kinds ofquestions that are contained in the paper-basedTOEFL test.

Econo Ten-packsEcono Ten-packs help to reduce costs for those work-ing in group settings, such as ESL study programs,language laboratories, and training classes andworkshops. Ten-packs are available for the TOEFLTest Preparation Kit and the TOEFL Practice Tests,Volume 2. Each pack contains 10 sets of printedmaterial from the corresponding study product.

Note: The instructor needs to purchase only oneproduct package containing the recorded materials.

Information about ordering TOEFL study materialscan be found in the TOEFL Products and ServicesCatalog (see page 47) or on our website at

http://www.toefl.org


50

REFERENCES

Alderman, D. L. TOEFL item performance acrossseven language groups (TOEFL Research Report 9).Princeton, NJ: Educational Testing Service, 1981.

American Association of Collegiate Registrars andAcademic Officers. AACRAO-AID participant selec-tions and placement study (Report to the Office ofInternational Training Agency for InternationalDevelopment, U.S. Department of State). Washington,DC: Author, 1971.

American Language Institute (Georgetown). Areport on the results of English testing during the 1966Pre-University Workshop at the American LanguageInstitute. Unpublished manuscript. GeorgetownUniversity, 1966.

American Psychological Association, AmericanEducational Research Association, and NationalCouncil on Measurement in Education. Standards foreducational and psychological testing. Washington, DC:American Psychological Association, 1985.

Anastasi, A. Psychological testing (3rd ed.). NewYork: Macmillan, 1968.

Angelis, P. J., Swinton, S. S., and Cowell, W. R. Theperformance of nonnative speakers of English on TOEFLand verbal aptitude tests (TOEFL Research Report 3).Princeton, NJ: Educational Testing Service, 1979.

Angoff, W. A. Context bias in the test of English as aforeign language (TOEFL Research Report 29).Princeton, NJ: Educational Testing Service, 1989.

Angoff, W. A., and Sharon, A. T. A comparison ofscores earned on the Test of English as a Foreign Languageby native American college students and foreign appli-cants to United States colleges (ETS Research BulletinNo. 70-8). Princeton, NJ: Educational Testing Service,1970.

Bachman, L. F., Kunnan, A., Vanniarajan, S., andLynch, B. Task and ability analysis as a basis forexamining content and construct comparability in twoEFL proficiency test batteries. Language Testing, 1988,5(2), 128-159.

Boldt, R. F. Latent structure analysis of the Test ofEnglish as a Foreign Language (TOEFL ResearchReport 28). Princeton, NJ: Educational TestingService, 1988.

Brennan, R. L. (1994) Standard Setting from thePerspective of Generalizability Theory. Proceedings ofthe Joint Conference on Standard Setting for Large-Scale Assessments. Washington, DC.

Chase, C. I., and Stallings, W. M. Tests of Englishlanguage as predictors of success for foreign students.Indiana Studies in Prediction No. 8. Monograph of theBureau of Educational Studies and Testing.Bloomington, IN: Bureau of Educational Studies andTesting, Indiana University, 1966.

Clark, J. L. D. The performance of native speakers ofEnglish on the Test of English as a Foreign Language(TOEFL Research Report 1). Princeton, NJ: Educa-tional Testing Service, 1977.

Cook, L. and Eignor, D. R.. IRT equating methods.Educational Measurement: Issues and Practice, 10(3),37-45. 1991.

Cowell, W. R. Item-response theory pre-equating inthe TOEFL testing program. In P. W. Holland and D.B. Rubin (Eds.), Test equating (pp. 149-161). NewYork: Academic Press, 1982.

Duran, R. P., Canale, M., Penfield, J. Stansfield, C.W., and Liskin-Gasparro, J. TOEFL from a communica-tive viewpoint on language proficiency: A working paper(TOEFL Research Report 17). Princeton, NJ: Educa-tional Testing Service, 1985.

Freedle, R., and Kostin, I. The prediction of TOEFLreading comprehension item difficulty for expositoryprose passages for three item types: Main idea, inference,and supporting idea items. (TOEFL Research Report44). Princeton, NJ: Educational Testing Service, 1993.

Freedle, R., and Kostin, I. The prediction of TOEFLlistening comprehension item difficulty for minitalkpassages: Implications for construct validity (TOEFLResearch Report 56). Princeton, NJ: EducationalTesting Service, 1996.

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○


51

Gershman, J. Testing English as a foreign language:Michigan/TOEFL study. Unpublished manuscript.Toronto Board of Education, 1977.

Hale, G. A., Rock, D. A., and Jirele, T. Confirmatoryfactor analysis of the Test of English as a Foreign Lan-guage (TOEFL Research Report 32). Princeton, NJ:Educational Testing Service, 1989.

Hambleton, R. K., and Swaminathan, H. Itemresponse theory: Principles and applications. Boston:Kluwer-Nijhoff, 1985.

Harvill, L. M., (1991). An NCME InstructionalModule on Standard Error of Measurement. Educa-tional Measurement: Issues and Practice, 10(2), 33-41.Washington, DC.

Heil, D. K., and Aleamoni, L. M. Assessment of theproficiency in the use and understanding of English byforeign students as measured by the Test of English as aForeign Language (Report No. RR-350). Urbana:University of Illinois. (ERIC Document ReproductionService No. ED 093 948), 1974.

Henning, G. A study of the effects of variations ofshort-term memory load, reading response length, andprocessing hierarchy on TOEFL listening comprehensionitem performance (TOEFL Research Report 33).Princeton, NJ: Educational Testing Service, 1991a.

Henning, G. A study of the effects of contextualizationand familiarization on responses to TOEFL vocabularytest items (TOEFL Research Report 35). Princeton, NJ:Educational Testing Service, 1991b.

Henning, G., and Cascallar, E. A preliminary studyof the nature of communicative competence (TOEFLResearch Report 36). Princeton, NJ: EducationalTesting Service, 1992.

Homburg, T. J. TOEFL and GPA: an analysis ofcorrelations. In R. Silverstein (Ed.), Proceedings of theThird International Conference on Frontiers in LanguageProficiency and Dominance Testing. Occasional Paperson Linguistics, No. 6. Carbondale: Southern IllinoisUniversity, 1979.

Hwang, K. Y., and Dizney, H. F. Predictive validityof the Test of English as a Foreign Language forChinese graduate students at an American university.Educational and Psychological Measurement, 1970, 30,475-477.

Jaeger, R. M. (1994). On the Cognitive Constructionof Standard-Setting Judgments: The Case of ConfiguralScoring. Proceedings of the Joint Conference onStandard Setting for Large-Scale Assessments.Washington, DC.

Linn, R., and Slinde, J. The determination of thesignificance of change between pre- and posttestingperiods. Review of Educational Research, 1977, 47(1),121-150.

Livingston, S. A., and Zieky, M. J. Passing scores: Amanual for setting standards of performance on educa-tional and occupational tests. Princeton, NJ: Educa-tional Testing Service, 1982.

Lord, F. M. Applications of item response theory topractical testing problems. Hillsdale, NJ: Erlbaum, 1980.

Lord, F. M., and Novick, M. R. Statistical theories ofmental test scores. Reading, MA: Addison Wesley, 1968.

Magnusson, D. Test theory. Boston: Addison-Wesley,1967.

Maxwell, A. A comparison of two English as a foreignlanguage tests. Unpublished manuscript. University ofCalifornia (Davis), 1965.

McKinley, R. L. An introduction to item responsetheory. Measurement and Evaluation in Counseling andDevelopment, 1989, 22(1), 37-57.

Messick, S. Validity (ETS Research Bulletin No. 87-40). Princeton, NJ: Educational Testing Service, 1987.Also appears in R. L. Linn (Ed.), Educational measure-ment (3rd ed.). New York: MacMillan, 1988.

Nissan, S., DeVincenzi, F., and Tang, K.L. Ananalysis of factors affecting the difficulty of dialog itemsin TOEFL listening comprehension (TOEFL ResearchReport 51). Princeton, NJ: Educational Testing Service,1996.


52

Stansfield, C. W., Ed. Toward communicative compe-tence testing: Proceedings of the Second TOEFL Invita-tional Conference (TOEFL Research Report 21).Princeton, NJ: Educational Testing Service, 1986.

Swineford, F. Test analysis: Test of English as aForeign Language (Statistical Report 71-112). Princeton,NJ: Educational Testing Service, 1971.

Swinton, S. S. A manual for assessing languagegrowth in instructional settings (TOEFL ResearchReport 14). Princeton, NJ: Educational Testing Service,1983.

Test of English as a Foreign Language: Interpretiveinformation. Princeton, NJ: College Entrance Examina-tion Board and Educational Testing Service, 1970.

Thorndike, R. I., and Hagen, E. P. Measurement andevaluation in education (4th ed.). New York: Wiley,1977.

Upshur, J. A. Comparison of performance on “Test ofEnglish as a Foreign Language” and “Michigan Test ofEnglish Language Proficiency.” Unpublished manu-script. University of Michigan, 1966.

Van der Linden, W. J. (1994). A Conceptual Analysisof Standard Setting in Large-Scale Assessments. Proceed-ings of the Joint Conference on Standard Setting forLarge-Scale Assessments. Washington, DC.

Wainer, H., and Lukhele, R. How reliable is theTOEFL test?. (TOEFL Technical Report 12). Princeton,NJ: Educational Testing Service, in press.

Wilson, K. M. GMAT and GRE Aptitude Testperformance in relation to primary language and scoreson TOEFL (TOEFL Research Report 12). Princeton,NJ: Educational Testing Service, 1982.

Odunze, O. J. Test of English as a Foreign Languageand first year GPA of Nigerian students (Doctoraldissertation, University of Missouri-Columbia, 1980).Dissertation Abstracts International, 42, 3419A-3420A.(University Microfilms No. 8202657), 1982.

Oltman, P. K., Stricker, L. J., and Barrows, T. Nativelanguage, English proficiency, and the structure of theTest of English as a Foreign Language (TOEFL ResearchReport 27). Princeton, NJ: Educational Testing Service,1988.

Pack, A.C. A comparison between TOEFL andMichigan Test scores and student success in (1)freshman English and (2) completing a college pro-gram. TESL Reporter, 1972,5, 1-7, 9.

Pike, L. An evaluation of alternative item formats fortesting English as a foreign language (TOEFL ResearchReport 2). Princeton, NJ: Educational Testing Service,1979.

Powers, D. E. The relationship between scores on theGraduate Management Admission Test and the Test ofEnglish as a Foreign Language (TOEFL ResearchReport 5). Princeton, NJ: Educational Testing Service,1980.

Powers, D. E. A survey of academic demands relatedto listening skills (TOEFL Research Report 20).Princeton, NJ: Educational Testing Service, 1985.

Schrader, W. B., and Pitcher, B. Interpreting perfor-mance of foreign law students on the Law School Admis-sion Test and the Test of English as a Foreign Language(Statistical Report 70-25). Princeton, NJ: EducationalTesting Service, 1970.

Schedl, M., Thomas, N., and Way, W. An investiga-tion of proposed revisions to the TOEFL test (TOEFLResearch Report 47). Princeton, NJ: EducationalTesting Service, 1995.

Sharon, A. T. Test of English as a Foreign Languageas a moderator of Graduate Record Examinations scoresin the prediction of foreign students’ grades in graduateschool (ETS Research Bulletin No. 71-50). Princeton,NJ: Educational Testing Service, 1971.


TOEFL ○ ○ ○ ○ ○ ○ ○ ○ ○ ○

Th i r t y - F o u r Yea r s o f Comm i tmen t t oH i gh

S t anda r d sand

Qua l i t y

Ser vi c eA

roundthe

World


54

ETS OFFICES SERVINGTOEFL CANDIDATES ANDSCORE USERS

MidwestEducational Testing ServiceSuite 300One Rotary Center1560 Sherman AvenueEvanston, IL 60201

Phone: 847-869-7700Fax: 847-492-5141TTY: 847-869-7738

SouthwestEducational Testing ServiceSuite 7002 Renaissance Square40 North Central AvenuePhoenix, AZ 85004

Phone: 602-252-5400Fax: 602-252-7499TTY: 602-252-0276

WestEducational Testing ServiceSuite 310Trans Pacific Centre1000 BroadwayOakland, CA 94607

Phone: 510-873-8000Fax: 510-873-8118TTY: 510-465-5571

Puerto RicoEducational Testing ServiceSuite 315American International

Plaza250 Munoz Rivera AvenueHato Rey, PR 00918

Phone: 787-753-6363Fax: 787-250-7426TTY: 787-758-4598

Main OfficeTOEFL/TSE ServicesEducational Testing ServiceP.O. Box 6151Princeton, NJ 08541-6151

Phone: 609-771-7100Fax: 609-771-7500TTY: 609-734-9362

Washington, DCEducational Testing ServiceSuite 6201776 Massachusetts

Avenue, NWWashington, DC 20036

Phone: 202-659-0616Fax: 202-659-8075TTY: 202-659-8067

SouthEducational Testing ServiceSuite 400Lakeside Centre1979 Lakeside ParkwayTucker, GA 30084

Phone: 770-934-0133Fax: 770-723-7436TTY: 770-934-2624

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○


55

TOEFL REPRESENTATIVES

IndonesiaInternational Education Foundation (IEF)Menara ImperiumLantai 28, Suite BJl. H.R. Rasuna Said, Kav. 1Kuningan, Jakarta Sealtan 12980Indonesia

JapanCouncil on International Educational Exchange (CIEE)TOEFL DivisionCosmos Aoyama B15-53-67 Jingumae, Shibuya-kuTokyo 150, Japan

JordanAMIDEASTP.O. Box 1249Amman, Jordan

KoreaKorean-American Educational

Commission (KAEC)K.P.O. Box 643Seoul 110-606, Korea

KuwaitAMIDEASTP.O. Box 44818Hawalli 32063, Kuwait

LebanonAMIDEASTP.O. Box 135-155Ras Beirut, Lebanon

orAMIDEASTP.O. Box 70-744Antelias, Beirut, Lebanon

Malaysia/SingaporeMACEETesting Services191 Jalan Tun Razak50400 Kuala Lumpur, Malaysia

MexicoInstitute of International EducationLondres 16, 2nd FloorApartado Postal 61-11506600 Mexico D.F., Mexico

MoroccoAMIDEAST25 bis, Patrice LumumbaApt. No. 8Rabat, Morocco

PakistanWorld Learning Inc.P.O. Box 13042Karachi 75350, Pakistan

People’s Republic of ChinaChina International Examinations

Coordination BureauNo. 167 Haidian RoadHaidian DistrictBeijing 100080People’s Republic of China

SyriaAMIDEASTP.O. Box 2313Damascus, Syria

TaiwanThe Language Training & Testing

CenterP.O. Box 23-41Taipei, Taiwan

Thailand, Cambodia, Laos, VietnamInstitute of International EducationG.P.O. Box 2050Bangkok 10501, Thailand

TunisiaAMIDEASTBP 351 Tunis-Belvedere 1002Tunis, Tunisia

United Arab EmiratesAMIDEASTc/o Higher Colleges of TechnologyP.O. Box 5464Abu Dhabi, UAE

All Other Countries and AreasTOEFL/TSE ServicesP.O. Box 6161Princeton, NJ 08541-6161USA

Algeria, Oman, Qatar, Saudi Arabia,Sudan, United Arab Emirates

AMIDEASTTesting Programs1730 M Street, NW, Suite 1100Washington, DC 20036-4505, USA

Australia, New Zealand, Papua NewGuinea, Solomon Islands, Vanuatu

Australian Council for Educational ResearchACER-ETS Administration OfficePrivate Bag 55Camberwell, Victoria 3124Australia

BahrainAMIDEASTP.O. Box 10410Manama, Bahrain

BrazilInstituto Brasil-Estados UnidosAv. Nossa Senhora de Copacabana690-6° Andar22050-000 Rio de Janeiro, RJBrasil

EgyptAMIDEAST6 Kamel El Shennawy StreetSecond Floor, Apartment 5Garden City, Cairo, Egypt

orAMIDEASTAmerican Cultural Center3 Pharaana StreetAzarita, AlexandriaEgypt

Europe, (East/West)CITO-TOEFLP.O. Box 12036801 BE ArnhemNetherlands

Hong KongHong Kong Examinations AuthoritySan Po Kong Sub-Office17 Tseuk Luk StreetSan Po KongKowloon, Hong Kong

India/BhutanInstitute of Psychological and

Educational MeasurementPost Box No. 19119/25-A Mahatma Gandhi MargAllahabad, U.P. 211 001, India

○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○ ○


Test of English as a Foreign LanguageP.O. Box 6155Princeton, NJ 08541-6155USA

To obtain more informationabout TOEFL products and services,use one of the following:

Phone: 609-771-7100E-mail: [email protected]: http://www.toefl.org

57516-03047 • DY87M60 • 678096 • Printed in U.S.A.


1997 edition - list-english.rulist-english.ru/pdf/tests/testandscoremanual.pdf · 1997 edition ®;...

Documents