ptea preliminary estimates of concordance 4aug10 v2
TRANSCRIPT
8/6/2019 PTEA Preliminary Estimates of Concordance 4Aug10 v2
http://slidepdf.com/reader/full/ptea-preliminary-estimates-of-concordance-4aug10-v2 1/7www.pearsonpte.com
Preliminary Estimates of Concordance
between Pearson Test of English Academicand Other Measures of English
Language Competencies
Introduction
This document outlines the research carried out to produce preliminary concordance between Pearson Test o English Academic(PTE Academic) and other measures o English language, including the Common European Framework (CEF or CEFR, Council o
Europe, 2001 ), TOEFL® iBT and IELTS™.
Tests comparisons using feld test data
PTE Academic has been eld tested using over 10,400 test takers. Field testing took place in 2007 and 2008. Test takers were
representative o the global population o students seeking admission to universities and other tertiary education institutions
where English is the language o instruction. Test takers were born in158 dierent countries and spoke 126 dierent languages.
During the eld tests several sets o secondary data were collected. Among these were ratings or all test takers on descriptive
scales published by the Council o Europe ( 2001 ). In addition, a number o test takers reported their scores on other tests o
English, including TOEIC, TOEFL PBT, TOEFL CBT, TOEFL iBT and IELTS.
A limited number o the sel-reported data were invalid as the reported scores were outside the possible score range or the
particular test. A small number o the test takers also submitted copies o their ocial score reports on the tests or which they ha
provided sel-reported data. Table1 ( p. 2) shows the ollowing or each test: the numbers o sel-reported data, how many o thes
were valid, the mean sel-reported scores, the number o ocial score reports sent in, the mean ocial scores and the correlation
with the PTE Academic eld test scores. All correlations are signicant at p<.011.
From Table 1 it can be concluded that the sel-reported scores are in general, quite accurate. Indeed, the correlation between the
sel-reported results and the ocial score reports was .82 or TOEFL iBT and .89 or IELTS. This nding is in agreement with earlier
research on sel-reported data. For example, Cassady ( 2001 ) ound students’ sel-reported Grade Point Average (GPA) scores to be
‘remarkably similar’ to ocial records. The data are also consistent. According to ETS ( 2005, p. 7 ) the score range 75–95 on TOEFL
iBT is comparable to the score range 213–240 on TOEFL CBT and to the score range 550–587 on TOEFL PBT. The mean sel-reportescores in Table 1 or these three tests are thereore comparable.
1 Signicant at p<.01 means there is less than 1% chance to observe this correlation i the measures are not related.
© Copyright 2010. All rights reserved; no part of this publication may be reproduced without the prior written permission of Edexcel Limited (a Pearson company).
July 2010 Version 2.
8/6/2019 PTEA Preliminary Estimates of Concordance 4Aug10 v2
http://slidepdf.com/reader/full/ptea-preliminary-estimates-of-concordance-4aug10-v2 2/7www.pearsonpte.com
Table 1: PTE Academic eld tests: test takers on other tests o English
Sel-reported data Ofcial score report
n Total
328
96
110
144
2436
n Valid
327
92
107
140
2432
Mean
831.5
572.3
240.5
92.9
6.49
Correlation
0.76
0.64
0.46
0.75
0.76
Test
TOEIC
TOEFL PBT
TOEFL CBT
TOEFL iBT
IELTS
n
no data
no data
no data
19
169
Mean
-
-
-
92.1
6.61
Correlation
-
-
-
0.95
0.73
In addition, according to ETS ( 2001, p. 3 ) a score range o 800–850 on TOEIC corresponds to a score range o 569–588 on TOEFL
PBT, which makes the sel-reported TOEIC mean score o the test takers on the PTE Academic eld test also all in line with data
published by ETS.Based on the data presented in Table 1, concordance between PTE Academic and other tests o English can be estimated,
taking into account a less than optimal eort o test takers during eld testing where test results have no direct relevance to the
test takers.
Relation to the Common European Framework
The preliminary relation o the PTE Academic score scale with the descriptive scale o the Common European Framework or
Languages (CEF) is based on both an item-centered and a test taker-centered method. For the item-centered method the
CEF level o all items was estimated by item writers, reviewed and, i necessary, adapted in the item-reviewing process. For
the test taker-centered method, three extended responses (one written and two spoken) per test taker were each rated by two
independent, trained raters. (On disagreement between the two independent raters, a third rating was gathered and the twoclosest ratings were retained). A dataset o over 26,000 ratings (by test takers sel-reporting, by items, by raters) on up to 100
dierent items was analyzed using the computer program FACETS (Linacre, 1988; 2005 ). Estimates o the lower boundaries o the
CEF levels, based on the item-centered method, correlated at.996 with those based on the test taker-centered method.
Validity check using BETA testing data
In addition to the initial eld testing o 10,400 students during 2007–08, a urther 364 test takers participated in the 2009 BETA
testing o PTE Academic. The concordance between the score scale o PTE Academic and the score scales o TOEFL iBT and IELTS,
each estimated rom the eld test data, were used as predictors o TOEFL iBT and IELTS scores o test takers participating in BETA
testing. Test takers provided sel-reported scores and a smaller, partially overlapping number o test takers sent in
copies o their ocial score reports.
8/6/2019 PTEA Preliminary Estimates of Concordance 4Aug10 v2
http://slidepdf.com/reader/full/ptea-preliminary-estimates-of-concordance-4aug10-v2 3/7www.pearsonpte.com
Table 2 shows the mean scores as sel-reported and rom the ocial score reports, the mean scores or the same test takers as
predicted rom their PTE Academic score and the correlations between the reported scores and the predictions rom PTE Academic
All correlations are signicant at p<.011.
It can be concluded that this preliminary concordance produces airly accurate and coherent predictions.
Table 2: PTE Academic BETA: test takers on other tests o English
Sel-reported data Ofcial score report
n
42
57
Mean
98.9
6.80
Predicted
97.3
6.75
Correlation
0.75
0.73
Test
TOEFL iBT
IELTS
n
13
15
Mean
92.2
6.60
Correlation
0.77
0.83
Predicted
98.2
6.51
Concordance o PTE Academic with other measures o English
Based on the research described, Pearson has produced preliminary concordance tables. The rst table (pp. 4–5 ) shows the
relationship between scores on TOEFL iBT and PTE Academic. The second table (pp. 6–7 ) shows the relationship between scores o
IELTS and PTE Academic. Both tables also show Pearson’s current best estimate o the relationship between these scores and the
CEF. Pearson will conduct more research in collaboration with independent scholars to increase the evidence or the validity and
accuracy o these concordance tables.
Caution while interpreting concordance tables
It must be noted that any individual level prediction o a score on a particular test based on the score observed on another test
will contain measurement error, caused by error in each o the tests in the comparison and in the estimate o the concordance.
Furthermore, tests in the comparison do not measure exactly the same construct.
For more inormation and to contact us, visit our website at www.pearsonpte.com
Reerences
Cassady, Jerrell C. ( 2001 ) Sel-Reported GPA and SAT Scores. ERIC Digest. ERIC Identier: ED458216.
Council o Europe ( 2001 ) Common European Framework o Reerence or Languages: Learning, Teaching, Assessment.
Cambridge: CUP.
ETS ( 2001 ) TOEFL Institutional Testing Program (ITP) and TOEIC Institutional Program (IP): Two On-Site Testing Tools rom ETS
at a Glance. Handout Berlin Conerence 2001. Princeton: Educational Testing Service.
ETS ( 2005 ) TOEFL ® Internet-based test: Score comparison tables. Princeton: Educational Testing Service.
Linacre, J.M ( 1988; 2005 ) A Computer Program or the Analysis o Multi-Faceted Data. Chicago, IL: Mesa Press.
8/6/2019 PTEA Preliminary Estimates of Concordance 4Aug10 v2
http://slidepdf.com/reader/full/ptea-preliminary-estimates-of-concordance-4aug10-v2 4/7www.pearsonpte.com
Preliminary estimates o concordance between PTE Academic, TOEFL iBT and the descriptive scaleo the Common European Framework o Reerence or Languages
TOEFL
iBTScore
120
119
118
117
115 – 116
114
113
112
110 – 111
109
107 – 108106105
103 – 104102
10199 – 100
98
9795 – 96
9493
91 – 929089
87 – 88
PTE
AcademicScore
90
89
88
87
86
85
84
83
82
81
80
79
78
77
76
7574
73
7271
70
6968
6766
65
6463
62
6160
59
Common
EuropeanFramework
C2
C1
B2
Level Descriptor
© Council o Europe
Can understand with ease virtuallyeverything heard or read. Cansummarize inormation romdierent spoken and written sources,reconstructing arguments andaccounts in a coherent presentation.Can express him/hersel spontaneously,very fuently and precisely,dierentiating ner shades o meaningeven in more complex situations.
Can understand a wide range o demanding, longer texts, and recognizeimplicit meaning. Can express him/hersel fuently and spontaneouslywithout much obvious searchingor expressions. Can use languagefexibly and eectively or social,academic and proessional purposes.Can produce clear, well-structured,detailed text on complex subjects,showing controlled use o organizational patterns, connectors
and cohesive devices.
Can understand the main ideas o complex text on both concrete andabstract topics, including technicaldiscussions in his/her eld o specialization. Can interact with adegree o fuency and spontaneity thatmakes regular interaction with nativespeakers quite possible withoutstrain or either party. Can produceclear, detailed text on a wide range o subjects and explain a viewpoint ona topical issue giving the advantages
and disadvantages o various options.
What does this mean or a score user?
C2 is a highly procient level and astudent at this level would be extremelycomortable engaging in academicactivities at all levels.
C1 is a level at which a studentcan comortably participate in allpostgraduate activities, includingteaching. It is not required or studentsentering university at undergraduatelevel. Most international students whoenter university at a B2 level wouldacquire a level close to, or at, C1 aterliving in the country or several yearsand actively participating in alllanguage activities encounteredat university.
B2 was designed as the level requiredto participate independently in higherlevel language interaction. It is typicallythe level required to be able to ollowacademic level instruction and toparticipate in academic education,including both coursework andstudent lie.
© The copyright o the descriptive scales reproduced in this document belongs to the Council o Europe.
8/6/2019 PTEA Preliminary Estimates of Concordance 4Aug10 v2
http://slidepdf.com/reader/full/ptea-preliminary-estimates-of-concordance-4aug10-v2 5/7www.pearsonpte.com
TOEFLiBT
Score
8685
83 – 84
82
81
79 – 80
78
76 – 77
74 – 75
72 – 73
70 – 71
67 – 69
65 – 66
63 – 64
60 – 62
57 – 59
54 – 56
52 – 53
48 – 51
45 – 47
40 – 44
PTE
AcademicScore
5857
56
55
54
53
52
51
50
49
48
47
46
45
44
43
42
41
40
39
38
35
34
3332
31
30
29
10
CommonEuropean
Framework
B1
A2
A1 orbelow
Level Descriptor© Council o Europe
Can understand the main points o clear standard input on amiliarmatters regularly encountered inwork, school, leisure, etc. Can dealwith most situations likely to arisewhilst in an area where the languageis spoken. Can produce simpleconnected text on topics, which areamiliar, or o personal interest. Candescribe experiences and events,dreams, hopes and ambitions andbriefy give reasons and explanationsor opinions and plans.
Can understand sentences andrequently used expressions relatedto areas o most immediate relevance(e.g. very basic personal and amilyinormation, shopping, local geography,employment). Can communicate insimple and routine tasks requiringa simple and direct exchange o
inormation on amiliar and routinematters. Can describe in simple termsaspects o his/her background,immediate environment and mattersin areas o immediate need.
Can understand and use amiliareveryday expressions and very basicphrases aimed at the satisaction o needs o a concrete type. Canintroduce him/hersel and othersand can ask and answer questionsabout personal details such as where
he/she lives, people he/she knowsand things he/she has. Can interactin a simple way provided the otherperson talks slowly and clearly and isprepared to help.
What does this mean or a score user?
B1 is insucient or ull academic levelparticipation in language activities. Astudent at this level could ‘get by’ ineveryday situations independently.To be successul in communication inuniversity settings, additional Englishlanguage courses are required.
A2 is an insucient level or academiclevel participation.
A1 is an insucient level or academiclevel participation.
N o
d a t a
a v a i l a
b l e
8/6/2019 PTEA Preliminary Estimates of Concordance 4Aug10 v2
http://slidepdf.com/reader/full/ptea-preliminary-estimates-of-concordance-4aug10-v2 6/7www.pearsonpte.com
IELTS
Score
9.0
8.5
8.0
7.5
7.0
6.5
PTE
AcademicScore
90
89
88
87
86
85
84
83
82
81
80
79
78
77
76
7574737271706968676665
646362616059
Common
EuropeanFramework
C2
C1
B2
Level Descriptor
© Council o Europe
Can understand with ease virtuallyeverything heard or read. Cansummarize inormation romdierent spoken and written sources,reconstructing arguments andaccounts in a coherent presentation.Can express him/hersel spontaneously,very fuently and precisely, dierentiatingner shades o meaning even in morecomplex situations.
Can understand a wide range o demanding, longer texts, andrecognize implicit meaning. Canexpress him/hersel fuently andspontaneously without much obvioussearching or expressions. Can uselanguage fexibly and eectively orsocial, academic and proessionalpurposes. Can produce clear,well-structured, detailed text on complexsubjects, showing controlled use o organizational patterns, connectorsand cohesive devices.
Can understand the main ideas o complex text on both concrete andabstract topics, including technicaldiscussions in his/her eld o specialization. Can interact with adegree o fuency and spontaneity thatmakes regular interaction with nativespeakers quite possible withoutstrain or either party. Can produceclear, detailed text on a wide range o subjects and explain a viewpoint on
a topical issue giving the advantagesand disadvantages o various options.
What does this mean or a score user?
C2 is a highly procient level and astudent at this level would be extremelycomortable engaging in academicactivities at all levels.
C1 is a level at which a studentcan comortably participate in allpostgraduate activities, includingteaching. It is not required or studentsentering university at undergraduatelevel. Most international students whoenter university at a B2 level wouldacquire a level close to, or at, C1 aterliving in the country or several yearsand actively participating in all languageactivities encountered at university.
B2 was designed as the level requiredto participate independently in higherlevel language interaction. It is typicallythe level required to be able to ollowacademic level instruction and toparticipate in academic education,including both coursework andstudent lie.
Preliminary estimates o concordance between PTE Academic, IELTS and the descriptive scale o theCommon European Framework o Reerence or Languages
8/6/2019 PTEA Preliminary Estimates of Concordance 4Aug10 v2
http://slidepdf.com/reader/full/ptea-preliminary-estimates-of-concordance-4aug10-v2 7/7www.pearsonpte.com
IELTSScore
6.5
6.0
5.5
5.0
4.5
PTE
AcademicScore
5857
56
55
54
53
52
51
50
49
48
47
4645
44
43
42
41
40
39
38
37
36
3534
33
32
31
30
29
10
CommonEuropean
Framework
B1
A2
A1 or
below
Level Descriptor© Council o Europe
Can understand the main points o clear standard input on amiliar mattersregularly encountered in work, school,leisure, etc. Can deal with mostsituations likely to arise whilst in anarea where the language is spoken.Can produce simple connected texton topics, which are amiliar, oro personal interest. Can describeexperiences and events, dreams,hopes and ambitions and briefy givereasons and explanations or opinionsand plans.
Can understand sentences andrequently used expressions relatedto areas o most immediate relevance(e.g. very basic personal and amilyinormation, shopping, local geography,employment). Can communicate insimple and routine tasks requiringa simple and direct exchange o
inormation on amiliar and routinematters. Can describe in simpleterms aspects o his/her background,immediate environment and mattersin areas o immediate need.
Can understand and use amiliareveryday expressions and very basicphrases aimed at the satisaction o needs o a concrete type. Can introducehim/hersel and others and can askand answer questions about personaldetails such as where he/she lives,people he/she knows and things he/she has. Can interact in a simple wayprovided the other person talks slowlyand clearly and is prepared to help.
What does this mean or a score user?
B1 is insucient or ull academic levelparticipation in language activities. Astudent at this level could ‘get by’ ineveryday situations independently.To be successul in communication inuniversity settings, additional Englishlanguage courses are required.
A2 is an insucient level or academiclevel participation
A1 is an insucient level or academiclevel participation.
N o
d a t a
a v a i l a
b l e