What kind of vocabulary is in course books and graded readers?

Rob WaringNotre Dame Seishin University

JALT Vocab SiG SymposiumJune 29, 2013

Steps

Decide a scale to use (ERF Scale)Make a base wordlist based on the scaleScan in the texts and remove proper nounsRun the analysis in AntWordCount the running words in each text at each of the wordlist levelsIdentify a typical average frequency profile (by baseword level) at each reading level for the GRs and course booksDecide the number of average texts to be ‘read’ (30)Decide how many times a word has to be met before it’s learnt (20)

#Titles Total running

words Average length Wordlist Level ERF1 (50) 64 4,138 65 Wordlist Level ERF2 (100) 39 9,343 240 Wordlist Level ERF3 (200) 68 42,846 630 Wordlist Level ERF4 (300) 57 82,962 1,455 Wordlist Level ERF5 (400) 59 189,697 3,215 Wordlist Level ERF6 (600) 106 223,740 2,111 Wordlist Level ERF7 (800) 121 403,641 3,336 Wordlist Level ERF8 (1000) 61 514,319 8,431 Wordlist Level ERF9 (1250) 51 645,368 12,654 Wordlist Level ERF10 (1500) 54 544,035 10,075 Wordlist Level ERF11 (1800) 65 1,113,465 17,130 Wordlist Level ERF12 (2100) 14 90,870 6,491 Wordlist Level ERF13 (2400) 33 596,979 18,090 Wordlist Level ERF14 (3000) 34 445,038 13,089 Wordlist Level ERF15 (3600) - Wordlist Level ERF16 (4500) 2 78,455 39,228

828 4,984,896

Graded Reader Corpus

Percentage of words at each ERF Reading level by Wordlist level

ERF1 ERF2 ERF3 ERF4 ERF5 ERF6 ERF7 ERF8 ERF9 ERF10 ERF11 ERF12 ERF13 ERF14 ERF15 ERF16ERF1 (50) 34.4% 37.6% 33.8% 30.8% 30.0% 27.6% 27.6% 27.4% 27.7% 27.3% 27.8% 26.7% 26.4% 26.3% 27.6%ERF2 (100) 8.2% 13.3% 12.1% 11.6% 11.2% 10.0% 9.3% 9.5% 9.8% 9.4% 9.2% 8.9% 9.0% 8.6% 8.9%ERF3 (200) 17.6% 13.1% 18.9% 20.3% 22.0% 21.8% 21.4% 21.2% 21.2% 20.9% 19.6% 19.5% 19.4% 18.8% 17.4%ERF4 (300) 5.3% 2.4% 4.7% 6.4% 6.9% 7.9% 7.8% 8.5% 8.2% 8.7% 8.3% 7.9% 8.2% 7.8% 6.8%ERF5 (400) 1.4% 1.7% 1.8% 1.8% 2.3% 2.5% 2.9% 3.2% 3.1% 3.1% 3.1% 3.0% 3.2% 3.1% 2.7%ERF6 (600) 1.9% 1.1% 2.1% 2.2% 2.2% 2.7% 3.0% 3.5% 3.4% 4.0% 3.8% 3.9% 4.0% 4.0% 3.4%ERF7 (800) 1.5% 0.7% 0.8% 0.7% 0.7% 1.1% 1.4% 1.6% 1.5% 1.9% 2.1% 2.1% 2.4% 2.7% 2.2%ERF8 (1000) 1.3% 0.5% 1.0% 0.9% 1.0% 1.5% 1.3% 1.6% 1.6% 1.9% 2.0% 2.5% 2.5% 2.9% 2.4%ERF9 (1250) 1.6% 0.5% 0.6% 0.4% 0.6% 0.7% 0.8% 0.7% 0.7% 0.8% 1.0% 1.1% 1.3% 1.4% 1.3%ERF10 (1500) 0.8% 0.5% 0.5% 0.5% 0.5% 0.6% 0.7% 0.8% 0.8% 0.8% 1.0% 1.3% 1.2% 1.5% 1.7%ERF11 (1800) 0.9% 0.3% 0.3% 0.6% 0.5% 0.7% 0.8% 0.7% 0.6% 0.7% 0.9% 0.8% 1.0% 1.1% 1.3%ERF12 (2100) 1.4% 0.2% 0.5% 0.5% 0.4% 0.7% 0.7% 0.5% 0.5% 0.5% 0.6% 0.8% 0.9% 1.0% 1.5%ERF13 (2400) 0.8% 0.6% 0.5% 0.2% 0.4% 0.4% 0.4% 0.3% 0.3% 0.3% 0.4% 0.2% 0.5% 0.6% 0.7%ERF14 (3000) 0.6% 0.1% 0.8% 0.7% 0.5% 0.8% 0.6% 0.5% 0.5% 0.4% 0.5% 0.3% 0.4% 0.6% 0.7%ERF15 (3600) 1.3% 0.2% 1.1% 0.9% 0.7% 1.1% 1.0% 0.9% 0.8% 0.8% 0.9% 0.9% 0.9% 1.3% 1.4%ERF16 (4500) 0.8% 0.5% 0.3% 0.2% 0.1% 0.3% 0.2% 0.2% 0.2% 0.2% 0.2% 0.2% 0.3% 0.3% 0.7%ERF17 (6000) 0.5% 0.4% 0.3% 0.2% 0.3% 0.4% 0.4% 0.3% 0.3% 0.4% 0.4% 0.3% 0.4% 0.4% 1.0%ERF18 (8000) 0.9% 0.2% 0.4% 0.2% 0.6% 0.9% 0.7% 0.3% 0.4% 0.3% 0.4% 0.3% 0.3% 0.4% 1.1%ERF19 (12000) 0.7% 0.6% 0.8% 0.3% 0.3% 0.4% 0.5% 0.3% 0.2% 0.3% 0.3% 0.2% 0.2% 0.2% 0.2%ERF20 (18000) 2.2% 1.1% 1.5% 1.4% 1.7% 1.5% 1.6% 1.5% 1.6% 1.6% 1.7% 1.5% 1.8% 1.9% 1.7%Out of level 0.2% 2.2% 0.2% 2.5% 0.9% 1.8% 1.9% 2.0% 1.0% 0.7% 0.5% 3.7% 0.8% 0.9% 0.2%Proper nouns 15.7% 22.3% 16.9% 16.5% 16.0% 14.7% 14.7% 14.3% 15.4% 14.7% 15.0% 14.0% 14.7% 13.6% 14.5%Not in lists 0.1% 0.0% 0.1% 0.1% 0.2% 0.2% 0.2% 0.2% 0.2% 0.2% 0.3% 0.2% 0.2% 0.4% 0.5%

Wordlist levelERF Reading level

% of families at each level which occur more than 20 times (minus proper nouns)

ERF1 ERF2 ERF3 ERF4 ERF5 ERF6 ERF7 ERF8 ERF9 ERF10 ERF11 ERF12 ERF13 ERF14 ERF15 ERF16ERF1 (50) 41% 48% 41% 37% 36% 32% 32% 32% 33% 32% 33% 31% 31% 30% 32%ERF2 (100) 10% 17% 15% 14% 13% 12% 11% 11% 12% 11% 11% 10% 11% 10% 10%ERF3 (200) 21% 17% 23% 24% 26% 26% 25% 25% 25% 24% 23% 23% 23% 22% 20%ERF4 (300) 6% 3% 6% 8% 8% 9% 9% 10% 10% 10% 10% 9% 10% 9% 8%ERF5 (400) 2% 2% 2% 2% 3% 3% 3% 4% 4% 4% 4% 3% 4% 4% 3%ERF6 (600) 2% 1% 3% 3% 3% 3% 4% 4% 4% 5% 5% 4% 5% 5% 4%ERF7 (800) 2% 1% 1% 1% 1% 1% 2% 2% 2% 2% 2% 2% 3% 3% 3%ERF8 (1000) 2% 1% 1% 1% 1% 2% 2% 2% 2% 2% 2% 3% 3% 3% 3%ERF9 (1250) 2% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 2% 2%ERF10 (1500) 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 2% 2%ERF11 (1800) 1% 0% 0% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 2%ERF12 (2100) 2% 0% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 2%ERF13 (2400) 1% 1% 1% 0% 0% 0% 0% 0% 0% 0% 0% 0% 1% 1% 1%ERF14 (3000) 1% 0% 1% 1% 1% 1% 1% 1% 1% 1% 1% 0% 1% 1% 1%ERF15 (3600) 2% 0% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 1% 2% 2%ERF16 (4500) 1% 1% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 0% 1%ERF17 (6000) 1% 0% 0% 0% 0% 1% 1% 0% 0% 0% 0% 0% 0% 0% 1%ERF18 (8000) 1% 0% 0% 0% 1% 1% 1% 0% 0% 0% 0% 0% 0% 0% 1%ERF19 (12000) 1% 1% 1% 0% 0% 0% 1% 0% 0% 0% 0% 0% 0% 0% 0%ERF20 (18000) 3% 1% 2% 2% 2% 2% 2% 2% 2% 2% 2% 2% 2% 2% 2%Out of level 0% 3% 0% 3% 1% 2% 2% 2% 1% 1% 1% 4% 1% 1% 0%Proper nouns 19% 29% 20% 20% 19% 17% 17% 17% 18% 17% 18% 16% 17% 16% 17%

Wordlist levelERF Reading level

Average book length without proper nouns

ERF1 ERF2 ERF3 ERF4 ERF5 ERF6 ERF7 ERF8 ERF9 ERF10 ERF11 ERF12 ERF13 ERF14 ERF15 ERF16

40.8% 65% 78% 83% 86% 85% 86% 89% 91% 92% 93% 91% 94% 93% 94%

% of each book in level

ERF1 ERF2 ERF3 ERF4 ERF5 ERF6 ERF7 ERF8 ERF9 ERF10 ERF11 ERF12 ERF13 ERF14 ERF15 ERF16

54 186 524 1,215 2,700 1,801 2,846 7,225 10,699 8,593 14,556 5,583 15,428 11,306 33,531

How many words do you meet if you read 30 books at each level?

ERF1 only

ERF1-2

ERF1-3

ERF1-4

ERF1-5

ERF1-6

ERF1-7

ERF1-8

ERF1-9

ERF1-10

ERF1-11

ERF1-12

ERF1-13

ERF1-14

ERF1-15

ERF1-16

Number of books read /

level 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30

Accumulated books read 60 90 120 150 180 210 240 270 300 330 360 390 420 450

Running words met

/level 65 240 630 1,455 3,215 2,111 3,336 8,431 12,654 10,075 17,130 6,491 18,090 13,089 39,228

Running words for 30 books / level 1,940

7,187

18,903 43,664 96,456 63,323

100,076 252,944

379,628 302,242 513,907 194,721 542,708 392,681 - 1,176,825

Accumulated having read

all 450 books 1,940

9,127

28,029 71,693

168,150 231,472

331,548 584,492

964,120 1,266,362 1,780,269

1,974,991 2,517,699

2,910,379 2,910,379 4,087,204

Accumulated coverage for 30 books per level to 95% coverage of the families at 20 meetings for each type at each level

ERF1 only ERF1-2 ERF1-3 ERF1-4 ERF1-5 ERF1-6 ERF1-7 ERF1-8 ERF1-9 ERF1-10 ERF1-11 ERF1-12 ERF1-13 ERF1-14 ERF1-15 ERF1-16

# books read 30 60 90 120 150 180 210 240 270 300 330 360 420 450 480

ERF1 (50) 16.7% 28.2% 35.5% 53.2% 71.5% 70.3% 70.1% 73.0% 73.3% 73.6% 77.8% 76.1% 78.2% 78.1% 76.7%ERF2 (100) 1.0% 14.3% 41.9% 60.0% 77.1% 81.9% 82.9% 87.6% 95.2% 96.2% 98.1% 99.0% 99.0% 99.0% 99.0%ERF3 (200) 0.8% 4.6% 23.2% 52.9% 76.8% 79.8% 83.7% 86.3% 89.0% 90.5% 92.8% 93.2% 95.1% 95.8% 96.2%ERF4 (300) 0.8% 2.3% 9.8% 34.6% 67.7% 77.4% 86.5% 89.5% 91.0% 91.7% 94.0% 94.0% 94.7% 95.5% 96.2%ERF5 (400) 0.0% 1.9% 3.7% 15.7% 49.1% 64.8% 74.1% 88.9% 91.7% 91.7% 92.6% 92.6% 95.4% 95.4% 96.3%ERF6 (600) 0.0% 0.6% 0.6% 9.4% 28.1% 38.1% 58.1% 79.4% 88.1% 91.3% 93.8% 94.4% 95.6% 96.3% 96.9%ERF7 (800) 0.0% 0.7% 2.6% 2.6% 7.9% 15.8% 26.3% 48.7% 59.2% 68.4% 82.2% 87.5% 94.7% 96.1% 97.4%ERF8 (1000) 0.0% 0.0% 0.9% 3.7% 11.6% 15.3% 25.5% 39.4% 50.9% 61.1% 78.7% 86.6% 91.7% 94.9% 97.2%ERF9 (1250) 0.0% 0.0% 0.5% 0.5% 6.3% 8.2% 14.4% 26.0% 39.4% 52.4% 63.0% 72.1% 80.3% 86.5% 89.9%ERF10 (1500) 0.0% 0.4% 0.8% 1.2% 5.8% 9.1% 12.8% 23.5% 36.2% 43.2% 58.0% 65.0% 73.3% 83.1% 89.7%ERF11 (1800) 0.0% 0.0% 0.0% 1.5% 3.3% 4.4% 9.5% 18.2% 32.1% 39.8% 54.4% 58.4% 70.4% 79.9% 88.3%ERF12 (2100) 0.0% 0.0% 0.3% 1.2% 3.4% 4.3% 7.4% 13.3% 20.4% 27.6% 40.6% 43.0% 53.6% 63.8% 76.8%ERF13 (2400) 0.0% 0.0% 0.4% 0.4% 2.7% 4.2% 6.2% 9.7% 16.2% 22.0% 30.1% 32.8% 43.2% 51.4% 68.7%ERF14 (3000) 0.0% 0.0% 0.3% 0.9% 2.3% 4.1% 5.2% 10.5% 15.2% 19.0% 25.9% 27.1% 32.4% 39.4% 54.2%ERF15 (3600) 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%ERF16 (4500) 0.0% 0.1% 0.5% 0.9% 1.6% 2.7% 3.2% 6.5% 9.4% 12.4% 18.5% 19.4% 25.4% 30.6% 45.4%

Accumulated reading amount

How many words are you likely to ‘know’ (20 meetings) after reading all that?

ERF1 only

ERF1-2

ERF1-3

ERF1-4

ERF1-5

ERF1-6

ERF1-7

ERF1-8

ERF1-9

ERF1-10

ERF1-11

ERF1-12

ERF1-13

ERF1-14

ERF1-15

ERF1-16

ERF1 (50) 8 14 18 27 36 35 35 36 37 37 39 38 39 39 0 38ERF2 (100) 1 14 42 60 77 82 83 88 95 96 98 99 99 99 0 99ERF3 (200) 1 5 23 53 77 80 84 86 89 90 93 93 95 96 0 96ERF4 (300) 1 2 10 35 68 77 86 89 91 92 94 94 95 95 0 96ERF5 (400) 0 2 4 16 49 65 74 89 92 92 93 93 95 95 0 96ERF6 (600) 0 1 1 19 56 76 116 159 176 183 188 189 191 193 0 194ERF7 (800) 0 1 5 5 16 32 53 97 118 137 164 175 189 192 0 195ERF8 (1000) 0 0 2 7 23 31 51 79 102 122 157 173 183 190 0 194ERF9 (1250) 0 0 1 1 16 20 36 65 99 131 157 180 201 216 0 225ERF10 (1500) 0 1 2 3 14 23 32 59 91 108 145 163 183 208 0 224ERF11 (1800) 0 0 0 4 10 13 28 55 96 119 163 175 211 240 0 265ERF12 (2100) 0 0 1 4 10 13 22 40 61 83 122 129 161 191 0 230ERF13 (2400) 0 0 1 1 8 13 19 29 49 66 90 98 130 154 0 206ERF14 (3000) 0 0 2 5 14 24 31 63 91 114 156 163 194 236 0 325ERF15 (3600) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0ERF16 (4500) 0 1 5 8 15 24 29 58 85 111 167 174 229 276 0 409

11 42 117 248 488 608 780 1092 1371 1580 1926 2036 2296 2520 0 2894

Summary

450 books = 2894 ‘known’ words (20 meetings)Many words at each level won’t be met enough times to ‘learn’ them even after having read 30 titles at each level

Course books

6 Japanese Junior High texts21 Japanese High school texts18 Korean Middle School texts15 Korean High School texts5 Mexican Middle and Senior High texts

How many words will a learner meet on average in these texts in a middle or high school?

Middle School High School TotalMexico (Sequences) 126,043 106,493 232,536

Korea (averaged) 23,483 37,950 61,433

Japan (averaged) 14,066 20,977 35,043

How many words are in each book by ERF level? Japanese Korea Mexico

JH SH Middle HS Middle HS ERF1 (50) 2,906 4,278 4,770 6,756 41,038 30,059 ERF2 (100) 1,012 1,638 1,757 2,369 10,755 8,911 ERF3 (200) 1,906 3,450 3,479 5,358 20,963 18,664 ERF4 (300) 616 1,339 1,233 2,056 7,550 7,472 ERF5 (400) 379 827 753 1,107 3,949 3,113 ERF6 (600) 407 932 878 1,312 4,059 3,787 ERF7 (800) 304 551 621 1,059 2,457 2,941 ERF8 (1000) 274 650 635 1,353 2,686 3,238 ERF9 (1250) 251 419 535 996 1,653 1,815 ERF10 (1500) 171 385 455 1,002 1,091 1,868 ERF11 (1800) 196 218 458 885 1,569 1,320 ERF12 (2100) 222 368 407 993 878 1,413 ERF13 (2400) 120 227 264 520 390 532 ERF14 (3000) 114 212 326 726 871 928 ERF15 (3600) 154 334 388 981 1,949 2,045 ERF16 (4500) 105 119 127 447 247 299 ERF17 (6000) 102 181 194 686 433 614 ERF18 (8000) 223 259 182 672 380 417 ERF19 (12000) 125 139 161 374 298 394 ERF20 (18000) 286 479 453 995 2,657 2,040

Out of level 247 206 266 404 686 830

Proper nouns 2,637 2,920 3,812 4,474 16,992 11,964

Not in lists 1,310 851 1,329 2,426 2,492 1,829

Total 14,066 20,977 23,483 37,950 126,043 106,493

words met vs number of words probably learnt (>20 meetings) in various course books

Japanese Korean Mexico#

meetings JH SH Both JH SH Both JH SH Both50+ 24 39 68 37 64 124 310 271 492

31-49 20 39 72 39 55 93 127 131 19120-30 23 57 75 53 71 172 138 139 19910-19 81 159 261 207 282 536 279 348 394

5-9 182 266 380 422 686 685 291 393 4091-4 792 713 792 802 1225 996 497 625 567

1122 1273 1648 1560 2383 2606 1642 1907 2252

50+ 2.1% 3.1% 4.1% 2.4% 2.7% 4.8% 18.9% 14.2% 21.8%31-49 1.8% 3.1% 4.4% 2.5% 2.3% 3.6% 7.7% 6.9% 8.5%20-30 2.0% 4.5% 4.6% 3.4% 3.0% 6.6% 8.4% 7.3% 8.8%10-19 7.2% 12.5% 15.8% 13.3% 11.8% 20.6% 17.0% 18.2% 17.5%

5-9 16.2% 20.9% 23.1% 27.1% 28.8% 26.3% 17.7% 20.6% 18.2%1-4 70.6% 56.0% 48.1% 51.4% 51.4% 38.2% 30.3% 32.8% 25.2%

100% 100% 100% 100% 100% 100% 100.0% 100.0% 100.0%

Course book plus a book a week = ?Japan Korea Mexico

# meetingsJH course book

Plus ERF1-3 (90 Books)

JH & SH course books Plus ERF 1-6 (180 books)

Middle course books Plus ERF1-3 (90 Books)

Middle & SH course

books plus ERF 1-6

(180 books)

Middle course books plus

ERF1-3 (90 Books)

Middle and SH course books plus

ERF 1-6 (180 books)

50+ 101 523 121 568 354 78031-49 63 182 90 229 165 25820-30 77 182 103 204 149 22810-19 162 300 288 467 291 411

5-9 267 372 368 593 323 4041-4 564 771 767 824 562 605

1234 2330 1737 2885 1844 2686

50+ 8.2% 22.4% 7.0% 19.7% 19.2% 29.0%31-49 5.1% 7.8% 5.2% 7.9% 8.9% 9.6%20-30 6.2% 7.8% 5.9% 7.1% 8.1% 8.5%10-19 13.1% 12.9% 16.6% 16.2% 15.8% 15.3%

5-9 21.6% 16.0% 21.2% 20.6% 17.5% 15.0%1-4 45.7% 33.1% 44.2% 28.6% 30.5% 22.5%

100.0% 100.0% 100.0% 100.0% 100.0% 100.0%

Number of words met

Japan Korea MexicoCourse books

onlyJH 14,066 23,483 126,043

JH & SH 35,043 61,433 232,536

Course books plus reading

JH 35,989 45,405 147,966JH & SH 219,242 245,632 416,735

Likely uptake (words met more than 20 times from reading 30 texts at each level)

Japan Korea MexicoCourse books

onlyJH 147 184 854

JH & SH 476 925 1,276

Course books plus reading

JH 403 602 959JH & SH 1,187 1,468 1,677

Summary

Course books only leads to low gainsmost words forgotten

Course books plus reading doubles vocabulary

BUT these data underestimate learning because the data• do not include partially known words (probably double that),

collocations, colligations, multi-word phrases etc.• are unfair to the Mexico group who were restricted to low

level reading (so we could compare)

It’s a work in progress ….

Some levels in my wordlist need redoing• level 3 has lots of past forms and irregular verbs -> bump in data• level 6, 8, 15 & 16 are short of familiesSome levels short of texts• level 12 and level 15

Next I’ll …• add higher level texts 17-20 when they become available• replicate Paul’s study on how many words you need to meet to learn X,000 words

with this corpus of SL texts• analyze which GR series best represents their stated levels• find out how many texts are needed before learners have covered say 05% of the

words at a set level• re-do the stats for 12, 30 meetings

Phew!

Yes Paul, I’ll publish it!