study of my word coach
TRANSCRIPT
-
8/13/2019 Study of My Word Coach
1/36
1
WORD COACH DOES IT COACH WORDS?
Tom Cobb
Universit du Qubec Montral
ABSTRACT
This study reports on the motivation, design, development, and testing of a
research informed video game for vocabulary learning,My Word Coach,played
on a Nintendo D or !ii and distributed commercially since late "##$% The
learning effects of the game &ere tested over four months in "##' &ith (# age)
appropriate *nglish as a second language +*- students in a Montreal school% .
battery of observational and empirical tests trac/ed e0perimental and uasi)
control groups2 le0ical development on the dimensions of meaning recognition,
free production, and speed of access% T&o months2 game use coincided &ith an
average 1#3)"#3 increase in recognition vocabulary si4e5 6#)(#3 reduction in
code)s&itching in oral productions5 and 1'3)783 increase in speed of le0ical
access% 9ncreased fluency appears to be the most interesting outcome% Questions
are raised about the importance of post)game follo& up, and suggestions made for
principled modification of the game2s learner model
INTRODUCTION & BACKGROUND
:ideo games occupy more and more of the time and attention of school age learners, &ith
an effect on learning that almost definitely e0ists but is presently un/no&n% .rguments in
principle for the learning po&er of such games are many +;ee, "##75
-
8/13/2019 Study of My Word Coach
2/36
"
learning effects, 9 had also been a linguistic consultant for its design and development,
and first presented the game to the public at the Getting the Word Outsymposium at the
.merican .ssociation of .pplied inguistics conference in "##$% The symposium &as a
collection of pro>ects &here some part of the voluminous vocabulary research of the past
"# years had been applied to a real)&orld problem and &ould potentially help large
numbers of real learners meet their vocabulary needs%
The vocabular !roblem
!hat are the vocabulary needs of real learners? 9f *nglish users are not reuired to read,
&rite or listen to lectures, these needs may not be all that great or difficult to meet%
@esearch has sho&n that the most freuent 1### &ord families of *nglish account for
almost A#3 of the individual &ords other than proper nouns in typical conversations
+.dolphs B chmitt, "##7-% These &ords can be learned fairly easily because of their
freuent recurrence in the environment% Cut if the learners have to read, &rite, or listen to
lectures, the picture changes dramatically% @esearchers have sho&n that the minimal
vocabulary needed to read a basic academic te0t in *nglish is probably about ',### &ord
families +Nation, "##8-% This figure presents little challenge to educated adult native
spea/ers of *nglish, &ho typically /no& roughly "#,### &ord families +;oulden, Nation
B @ead, 1AA#-% Cut for 1 children, there is a notable vocabulary divide along socio)
economic lines even before school begins +=hall B a/obs, "##7- &hich only &idens
over the school years% =hildren of immigrant families are particularly at ris/, particular
those &hose uic/ acuisition of conversational ability hides a vocabulary deficit that
only appears in high school &hen literacy s/ills become crucial and remediation is
difficult +@oessingh B *lgie, "##A-% aufer2s +"###- survey of seven vocabulary si4e
studies of *E learners on three continents found that many students attempting to study
academic sub>ects through *nglish medium &ere functioning &ith an average of only
",1## +D A$$- &ord families% 9t seems uncontroversial to conclude that vast numbers of
learners of *nglish both as 1 and " are several thousand &ords short of a functional
-
8/13/2019 Study of My Word Coach
3/36
7
le0icon in the tas/s they have set for themselves% There is clearly a place in this &orld for
a cheap, principled, and effective means of rapid vocabulary e0pansion%
A role "or #ech$olo%
The idea that a computer)based training system could be a good &ay to meet the
vocabulary instruction challenge has been around for some time% The large amount of
material to be covered, the li/elihood of strong individual differences in both goals and
learning rates, and the need for recycling and record)/eeping &ere identified early on as
reasons to loo/ to computeri4ed instruction +e%g%, .t/inson, 1A$"-% @ecent developments
add strength to the argumentF increased capacity allo&s for the provision of
concordances or glossaries as learning tools, increased processing speed allo&s for fine)
tuned control of procedural interactions, and the advent of net&or/s frees the learner
from particular times or places of learning% The still more recent development of game)
playing devices carries the potential benefits of a computational medium to a logical
conclusion%
-
8/13/2019 Study of My Word Coach
4/36
6
driven, resource)informed systems +=obb, "##A-% =urrent development &or/ involves
e0panding each of these approaches via game activities% Cased on
-
8/13/2019 Study of My Word Coach
5/36
(
met% The games focus on one or more of the follo&ing s/ill areas as mentioned above J
form, form)meaning connection, and le0ical access%
The user loads Word Coachinto the Nintendo console, enters his or her name +four can
play on one machine, plus guests, and many more &irelessly- and begins to play the one
game available at this point, &hich is effectively a placement test, a sample item from
&hich appears in the leftmost screen of Eigure 1% The testing starts by pitching &ord sets
from a medium freuency level +the 6### &ord level-5 if these &ords are not /no&n, the
level goes do&n until a 4one containing at least 7#3 un/no&n &ords is determined +or if
mainly /no&n, the level goes up-% 9n this &ay, players are challenged at their o&n level%
More games are introduced as play proceeds% There are t&o form)based games5 Missing
etter, &hich involves using the stylus to supply the letter missing from a &ord +e%g%,
new_paper, see Eig% ", left screen-, and Cloc/ etters, &hich involves clic/ing on falling
letters to form one of four given &ords as unused letters pile up to&ard an e0plosion +in a
version of Tetris, see Eig% ", middle screen-% Eour of the games involve connecting &ords
and simplified " glosses,in various combinations of both receptive and productive
tas/s% 9n plit Decision, the upper screen displays a definition &hile the lo&er screen
presents &ords that players toggle through until they identify the one that matches the
definition% 9n !ord huffle a &ord from the bottom screen is dragged and dropped on
one of four definitions in the top screen% 9n
-
8/13/2019 Study of My Word Coach
6/36
-
8/13/2019 Study of My Word Coach
7/36
$
Eigure "F !ord =oach2s form)based games and session feedbac/ +7 split screens-
Eigure 7F !ord =oach2s form and meaning games +7 split screens-
RE+IEW O, THE -ITERATURE
Eollo&ing on this introduction to the game itself, &e turn to a revie& of the relevant
research in three parts ) the research behind the game components, the research behind
the learning choices, and the research precedents for vocabulary and literacy training
games%
Reearch beh'$) #he %ame. com!o$e$#
-
8/13/2019 Study of My Word Coach
8/36
'
=ontent% nly relatively recently has it become possible to specify &hat the basic non)
specialist le0icon of *nglish consists in% This specification follo&s from three connected
research pro>ects over a 1#)year periodF the assembly of the 1## million &ord Critish
National =orpus +0ford =omputing ervices, "##(-5 its brea/do&n into a list organi4ed
by freuency and range +eech et al, "##1-5 and the pedagogical adaptation of the lists by
Nation, &hich involved recombining items from the &ritten and spo/en sections to Kmore
closely represented the order Oof learningP the &ords +Nation B Ceglar, "##$, p% 11- and
organi4ing them into 16 family)grouped thousand lists +Cauer B Nation, 1AA75 Nation,
"##8-% @andom items from the resulting lists are sho&n in Table 1 by &ay of illustration%
. final adaption of the lists for !ord =oach purposes &as a ran/ing of each list into
more and less difficult spellings according to number of syllables +si0 in
unreasonableness-, number of double consonants +t&o pairs in committed-, presence of
triple vo&els +agreeable-, and /no&n spelling conundrums +li/e admissible-, in order to
provide seuences according to difficulty for the form)oriented games +see
le0tutor%caHspellingHbncIconundrumsH-%
Table 1% !ords at five CN= freuency levels
Eirst 1### Third 1### Eifth 1### *ighth 1### Tenth 1###held
transport
point
lighten
degree
line
understand
high&ay
forty
sale
steam
adapt
stream
fiddle
urge
cheat
clip
trivial
polite
heal
diagnose
minimal
deer
gloomy
void
spine
captive
glossary
ra4or
&indscreen
garlic
bac/drop
mai4e
fret
draughtsman
bipolar
caption
tingle
moron
staunch
hairspray
beehive
vestry
into0icate
ban/note
deliverance
clang
fallible
temperance
disservice
.lso relatively recent is the publication of learner dictionaries &ith simplified definitions,
follo&ing the principle that as fe& &ords as possible in the definition should be less
common than the head&ord itself +as &as al&ays the case &ith the format Ka car is a
vehicle &hich-% The Cambridge Advanced Learners ictionary +"##(- &as used in
http://lextutor.ca/spelling/bnc_conundrums/http://lextutor.ca/spelling/bnc_conundrums/ -
8/13/2019 Study of My Word Coach
9/36
A
!ord =oach mainly o&ing to its definitional format and adeuate number of entries%
Many of the definitions &ere further simplified by research assistants to match the screen
si4e available and to be comprehensible to as many learners as possible%
-
8/13/2019 Study of My Word Coach
10/36
1#
The principles of the hand computer &ere programmed into !ord =oach e0actly as
presented in Mondria et al, such that all game &ords even if previously /no&n are
recycled at least five times, &hile un/no&n or difficult &ords are recycled possibly a
do4en or more times and are not dropped from the game +Ksucceeded, in the designers2
vernacular- until they have made it through five correct trials at increasing intervals%
;ames, learning, and motivation% The basic cognitive operations that promote initial
&ord retention and learning +moving &ords from &or/ing to long)term memory- are
fairly &ell /no&n% These are operations that reuire decisions about &ords, including
rehearsal, repetition, recall, retrieval, and use, in conditions of attention and arousal, as
discussed in Thornbury +"##", p% "6-% .ll of these operations are fairly straightfor&ard to
embody in a game format +apart from Kuse in the type of games &e are discussing-%
*ach of !ord =oach2s games gives learners practice in one or more of the basic
operations of &ord learning, from rehearsal of form, recall of form cued by meaning, to
recall of meaning cued by form, and all of them give learners motivation and opportunity
to increase their speed of le0ical access%
Reearch beh'$) #he lear$'$% cho'ce
!ord =oach is based on a classic paired associate learning model +&ords and glosses-,
&ith some e0tra attention paid to processing speed and forms of &ords% .s argued by
Nation +e%g%, "##1, pp% "A8)718-, paired)associate learning is +a- the learning model &e
/no& most about and +b- the best &ay to meet the goal of establishing a critical mass of
vocabulary as early as possible in language learning% The learning approach can also be
positioned on a number of schemes from the more recent research literature% 9n terms of
incidental or deliberate &ord learning +Nation, "##'-, the approach here is clearly
deliberate% 9n terms of conte0tual vs% definitional +=obb, 1AAA5
-
8/13/2019 Study of My Word Coach
11/36
11
forms +aufer, "##8-, the approach is clearly focus on forms%
-
8/13/2019 Study of My Word Coach
12/36
1"
le0icon- as &ell as being less amenable to e0plicit learning, as suggested by *llis +1AA6-
and Gulsti>n +"##"-% The games that give learners practice &ith forms alone are Missing
etter and Cloc/ !ords, and these are designed to &or/ in con>unction &ith form)
meaning games to establish the basis for high uality entries in the mental le0icon
+n, "##(-% 9t is often argued that
le0ical access speed li/e other implicit or procedural /no&ledge types can only be built
up over thousands of hours of e0posure to a language +e%g%, *llis, 1AA6-, but some recent
laboratory studies by nellings et al+"##"- and *lgort +"##$- suggests that le0ical access
may in fact be trainable using activities built &ith research soft&are ) and replicable in
principle on a game console such as My !ord =oach%
Third, on the matter of " definitions, a training study by *lgort +"##$- appears to sho&
that le0ical acuisition at all levels including the implicit level can be achieved through
e0plicit paired)associate training using " glosses% he gave advanced * learners
practice in acuiring 6' *nglish)li/e n, "##(-% *lgort e0plained these results in terms of iang2s +"##6-
-
8/13/2019 Study of My Word Coach
13/36
17
semantic transfer model of adult " vocabulary acuisition% iang2s model postulates that
ne& " &ords are almost inevitably associated &ith old 1 concepts for e0tensive
periods, or forever, even if the &ord is learned through conte0tual inference, unless steps
are ta/en to prevent the association from forming% This can apparently be achieved
through focus on an " definition through several rehearsals, provided the definition is
readable, &hich is &hy as mentioned above that the game definitions &ere doubly
simplified%
(rev'ou reearch o$ %ame "or vocabular a$) l'#erac
.s mentioned above, the claims for learning from video games are many and the
empirical studies fe&% There are some good reasons for the lac/ of studies% :ideo games
are played on special machines that, unli/e computers, are not supplied by schools, so
that getting an e0perimental group together is difficult% .lso, fe& such games attempt to
teach school sub>ects, so school authorities might be reluctant to donate their learners2
time for purposes they see as unclear%
*ven &hen literacy instruction is the presumed ob>ective +as can be inferred from the title
of ;ee2s, "##7, What video games have to teach us about learning and literacy-,
empirical evidence for learning from games is still thin% This is because literacy is often
construed in some special &ay in this literature ) critical literacy, ne& literacy, and game
playing as a literacy, in ;ee2s te0t ) &hich does not generate falsifiable propositions%
!hat is the test that could establish that reading and &riting should not be vie&ed only
as mental achievements going on inside peoples heads but also as social and cultural
practices &ith economic, historical, and political implications +p% '-? :ideo game
research, especially for games in the narrative, immersive, and Massive Multiplayer
nline modes, appears to be at the stage of frame&or/ building and uestion forming, or,
as the editors of a recent gaming issue of!ducational "echnology #esearch and
evelopment+pector B @oss, "##', p% (1#- put it, is Kin its infancy% ;ee2s student
tein/uehler +"##'- describes the current games research agenda in this &ayF K.s part of
-
8/13/2019 Study of My Word Coach
14/36
16
this success Oof ma/ing games and learning a serious topic on the intellectual and social
agendaP, &e have begun to move beyond mere plausibility arguments and are poised to
start as/ing more focused and empirically driven research uestions +p% "(1, emphasis
added-% .pparently &e must &ait a &hile for anything definite about learning from this
sort of gaming%
There is a legitimate uestion about K!hat is a game? The &ord &as !ittgenstein2s
+1A(7- classic e0ample of a concept &ith almost no common features across instances%
The games under discussion in the present study are clearly much simpler than the
multiplayer versions mentioned above% Go&ever, even simpler video games &ith clear
and even curricular ob>ectives have not generated strong research as yet% . study
referenced by
-
8/13/2019 Study of My Word Coach
15/36
1(
1% !hat is the e0tent of game use, and are there any game or learner characteristics
that predict game use?
"% Go& many &ords are /no&n receptively before and after game use?
7% Go& many learned &ords are used productively after game use?
6% 9s there a difference in speed of le0ical access before and after game use?
-ear$er a$) Se##'$%
The learners &ho participated in this study &ere t&o intact classes of "( ;rade 8
Erancophone * learners in a middle)class suburban school in Montreal% The ethnicity
of both groups &as roughly 7#3 Quebec Erancophone children and $#3 immigrant
children mainly from Erancophone countries% The medium of instruction at the school
&as Erench, e0cept for t&o hours per &ee/ of *, &here the teacher used *nglish as
much as possible follo&ing a communicative language teaching approach% The children
had various amounts of *nglish e0posure out of class, from e0tensive to none, and &idely
varying levels of *nglish proficiency% The teacher reported that the parents of these
children had complained that non)educational video games &ere eating into their
children2s home&or/ time and that they &ere more than &illing to try the educational
variety +A'3 of parents supported this e0periment-% The school supported the research,
&hich too/ place over a four month period in the spring of "##$% n receiving the game,
each group received (# minutes training and roughly eual encouragement during *
class to use the game%
The role o" #he %am'$% com!a$
Ubisoft 9nc% of Montreal provided 8# Nintendo D players andMy Word Coachgame
dis/s, &ithout &hich the study could not have ta/en place% Cy mutual agreement &ith
Ubisoft, no conditions &ere placed on the /inds of results e0pected or on ho& or &here
these &ould be publici4ed%
-
8/13/2019 Study of My Word Coach
16/36
18
Reearch De'%$
Cecause of the school2s reuirements that all children have a chance to use the game, and
that the research groups be intact classes, a pre)e0perimental, &ithin)sub>ects design &as
chosen for this study% ne group used the game for t&o months &hile the other served as
uasi)control, and then the roles &ere reversed% The same &ord /no&ledge tests &ere
administered to both groups at the beginning of the e0periment, at the changeover point,
and at the end of the four months% Eor learners &ho received the game first, the seuence
&as as follo&sF pre)test, t&o months &ith the game, post)test 1, t&o months of normal
classes &ithout the game, and post)test "% +The second post)test served as a measure of
delayed retention%- Eor learners &ho received the game second, the seuence &as pre)test
1, t&o months &ithout the game, pre)test ", t&o months &ith the game, and a post)test%
+The t&o pre)tests serves as a baseline for normal le0ical development against &hich
game inspired development could be compared%- The design is sho&n schematically in
Table "%
Table "F Diagram of research design
T1 T" T7
Eirst game
group
-
8/13/2019 Study of My Word Coach
17/36
1$
game period% .ll measures &ere compared at the three test points using basic .N:.s,
since the usual arrivals and absences of an intact setting created slightly uneual groups
and made a repeated measure impossible%
Reearch I$#rume$#
The research instruments &ere as follo&sF
1% The game itself provides detailed trac/ing of &hich &ords &ere played and ho&
often, as &ell as session si4e and freuency +Eig 1-%
"% The recognition /no&ledge measure is Nation and Ceglar2s +"##$- revision of the
classic :ocabulary evels Test +Nation, 1AA#-% The levels for this version of the
test are sampled from the first fourteen CN= freuency lists, as elaborated by
Nation, and thus correspond precisely to the content of the game% The test
measures only recognition /no&ledge, in that the test ta/er is not as/ed to
produce the &ord or its meaning but merely to match the underlined &ord in a
short non)defining conte0t to one of five short glosses +Eigure 6-% The test glosses
are not the same as the glosses encountered playing !ord =oach e0cept
coincidentally%
3. PERIOD: It was a difficult period.a. questionb. timec. thing to do
d. book
Eigure 6F evels Test +"##$- format
There are ten test uestions at each 1###)family level, such that the score
multiplied by 1## gives an estimate of the number of &ord families /no&n at that
level +eight out of 1# suggests '## families /no&n-% Nation and Ceglar +"##$-
discuss the test2s sampling and reliability% 9n this e0periment only the first 1#
levels of the test &ere administered, in vie& of the learners2 predicted level and
institutional time constraints% ince only one validated version of the test e0isted
-
8/13/2019 Study of My Word Coach
18/36
1'
at the time of the e0periment, the same test &as used &ith each learner three
times%
7% The production measure &as an oral telling of the "( line dra&ings of Mayer2s
+1A8$- &ordless storyA %oy, a og, and a &rogas told +untimed- to a research
assistant and recorded% The recordings &ere transcribed as te0t files for processing
by the CN= version of :ocabprofile +httpFHH&&&%le0tutor%caHvpHbncH-% This
program categori4es each &ord of an input te0t by 1### level, according to
Nation2s famili4ed lists as described above, resulting in a freuency profile of the
learner2s production +the percentage of &ord families, types, and to/ens at each
1### level-%
6% e0ical access speed &as measured &ith a simple instrument developed by
UN*= for literacy testing in developing countries% 9t is simply a list of 8#
&ords in order of decreasing freuency and increasing length +all &ithin the first
1### freuency 4one-% The learner is as/ed to read the list aloud to a research
assistant for one minute as uic/ly as possible% The assistant notes the last &ord
reached &ithin the time and stri/es out any mispronunciations that appear to
indicate unfamiliarity &ith the &ord, resulting in a tally of &ords read correctly in
one minute%
.ll four measures &ere administered in one hour in each of the three testing periods% 9n
addition, the teacher solicited &ritten comments using a uestionnaire format%
RESU-TS
Game ue
.ccording to Word Coach2s tally of K&ords succeeded +in the game2s vernacular- the
amount of game use in each of the t&o)month periods &as e0tensive and similar bet&een
the groups, although &ith considerable variance among individuals% ;roup 1 players
http://www.lextutor.ca/vp/bnc/http://www.lextutor.ca/vp/bnc/ -
8/13/2019 Study of My Word Coach
19/36
1A
succeeded an average of "'6A &ords +DW1'$A-, &hile ;roup " succeeded "(78 &ords
+D 1A(A-, &ith the difference bet&een means not statistically significant +pW%('-% The
standard deviations are high, sho&ing that some too/ to the game more than others +range
for &ords succeeded is 86## do&n to "$"-%
These numbers translate into days and hours as follo&sF .t the high end, there &ere si0
players in each group &ho succeeded at over 6### &ords in 8# days, &hich at an
estimated average of eight appearances of each &ord needed to succeed it, totals roughly
7",### &ords played, or >ust over (## &ords per day, or about "( game sets of "# &ords
apiece% .t five minutes per game this &ould amount to roughly t&o hours of play per
day, or a total of 1"# hours over t&o months% .t the lo& end, there &ere seven in each
group &ho succeeded at fe&er than 1### &ords +',### recurrences, 176 &ords per day,
si0 or seven game sets-% .t five minutes per game, this amounts to about 7# minutes per
day, or total of 7# hours over t&o months%
*ea$'$% reco%$'#'o$
Coth groups too/ the first ten 1### levels of Nation2s +"##$- CN=)based evels Test of
meaning recognition at all three testing points% 9t does not appear that the learners had
learned the test +as opposed to its content- to any degree, despite ta/ing the same 1##)
item test, since the T1)T" scores for ;roup " &hen they did not use the game are not
significantly different%
The pre)test results at T1 &ere similar bet&een groups +sho&n in Eigure ( and Table 7-,
&ith the appearance of a slight advantage for the first game group +not significant for any
single level belo& the fifth, or for the first five levels ta/en together-% ome&hat
surprising in the results is the roughly eual numbers of &ords /no&n across the second
through fifth levels5 it is more normal to see a decline as the &ords become less freuent%
-
8/13/2019 Study of My Word Coach
20/36
"#
Cecause the scores drop sharply after the fifth level, and because the first (### &ords are
a sufficient immediate goal for these learners, it &as decided to calculate learning gains
on the basis of only the first (### &ord families% The mean number of &ords /no&n from
the first (,### &as ",6"' +D $$1- for the first group, and ","78 +D '$7- for the second,
leaving a comfortable learning space of at least "(## &ords%
Eigure (F !ord families /no&n at Time 1
Table 7F Mean evels Test scores for first 1#,### &ord families at T1 +D-
S1 S" S7 S6 S( S8 S$ S' SA S1#
1stgame group 8%77
+1%(1-
(%#7
+"%7A-
6%17
+"%1$-
6%A#
+1%86-
6%'$
+1%A7-
"%$7
+"%#8-
"%A7
+1%($-
"%'#
+"%#$-
1%$#
+1%(7-
1%6$
+1%8$-
'ndgame
group
8%##
+"%#6-
6%78
+"%(7-
7%A8
+"%68-
6%66
+1%'8-
7%8#
+"%86-
"%##
+1%A6-
1%86
+1%6A-
1%(8
+1%$"-
#%'6
+1%#(-
#%(8
+#%A#-
Eollo&ing game play, learners in both groups acuired ne& &ords across the first five
levels of the test, more or less eually, as sho&n in Eigure 8 for the first group +the group
for &hich pre, post and delayed post scores are available in the present design-% Table 6
sho&s the number of &ords learned across five levels by both groups%
-
8/13/2019 Study of My Word Coach
21/36
"1
Eigure 8F Mean evels Test scores by level for ;roup 1 at three times
Table 6F !ord families from (### /no&n at three times Xgain, +D-
Time 1 Time " Time 7
Eirst game group "6"'+$$1-
"("$+$7A-XAA
"A7"YY
+$#"-X6#(
$econd game
group
""78+'$7-
"77"+$$A-X"A
"((8Y+'7'-X""6
YY pZ%##15 YpZ%#15 underlining indicates game periods
The results at T7 sho& significant and e0tensive gro&th of &ord /no&ledge but in a
pattern that &ould have been difficult to predict% Eor the first game group, the average
number of meanings recogni4ed had increased by only AA immediately after the game
+n%s%d-, but then at a delay of t&o months increased by a further 6#( &ords +EWA%1A,
pZ%##1-, for a total of (#6 &ords over the four)month period, or a gain of "#%$3 &ith
respect to the starting /no&ledge of "6"' &ords% Eor the second group, the average
number of meanings recogni4ed had increased by only "A &ords through t&o months2
normal classroom e0perience +n%s%d%-, but follo&ing the game had increased by ""6 &ords
+significant at T1)T7, EW(%11, pZ%#1-, a gain of 1#3 &ith respect to the starting
/no&ledge of ","78 average &ords%
-
8/13/2019 Study of My Word Coach
22/36
""
,ro% S#or'e 12or) '$ ue3
The story accounts &ere transcribed by research assistants, spell)chec/ed manually,
stripped of immediately repeated &ords and phrases, and run through :ocabprofile)CN=
both as corpora and individually% The uestion as/ed of the corpus &as &hether any of
the 7/, 6/, and (/ &ords that had been learned for meaning recognition &ould sho& up in
the tellings% 9f this &as not the case, it &ould not necessarily indicate that no learning had
ta/en place, since the transformation of receptive into productive /no&ledge is rarely
instantaneous +aufer, 1AA'-% The :
-
8/13/2019 Study of My Word Coach
23/36
"7
Eam Typ To/ Eam
Typ To/ Eam
Typ To/
Eirst game group 6(+""-
(1+"8
-
18$+'$
-
6'+"1-X7
(6+"8-
X7
"#$YY
+A#-
X6#
66+"1-)6
(#+"8-
)6
1$A+$(-
)1(econd game group 6#
+17-6(+16
-
1($+6"
-
7'+18-)"
61+1A-)6
1(A+("-X"
66+1$-X7
6'+"1-
X"
1$A+8#-
X11
YYpZ%#15 off)list items are not included
Table ( sho&s :ocabprofile2s family, type, and to/en counts for the t&o groups at three
times% There are an average three additional &ord families for each group follo&ing the
game periods +at T" for ;roup 1, T7 for ;roup "-, but these gains are statistically
significant only at the pZ%1# criterion in both cases% There &ere ho&ever large increases
in the number of total &ords used to tell the stories, as represented by the to/en counts, at
T" for ;roup 1 +6# mean increase from an original mean of 18$, about "(3, EW(%$',
pZ%#1- and T7 for ;roup " +"# mean increase from an original 1(A at T", about 1"3 but
not significant-% There are declines for all units in repeated re)tellings follo&ing non)
game periods, &hether preceding game use +;roup " at T"- or follo&ing +;roup 1 at T7-%
-e4'cal acce !ee)
Coth groups made small gains in le0ical access in the non)game periods +presumably a
practice effect- and strong gains follo&ing game periods% Erom a similar starting point of
(6%A average &ords read correctly in one minute for ;roup 1 and 87%8 &ords for ;roup "
+n%s%d%-, ;roup 1 increased the number by 1A%( &ords or 783 from T1)T" and another 8%A
by T7 +EW"#%'", pZ%###1-% ;roup " increased its &ords read by an average '%" at T1)T",
and by 1"%" more &ords or 1'3 increase follo&ing the game period T")T7 +EW6%"A,
pZ%#(-% This information is summari4ed in Table $%
Table $F Number of error)free &ords read in 1 minute at three times +D-+Xgain-
-
8/13/2019 Study of My Word Coach
24/36
"6
Time 1 Time " Time 7
Eirst game group (6%A+11%6-
$6%(YY+"$%A-X1A%(
'1%6Y+7#%6-X8%A
$econd gamegroup
87%8+"#%6-
$1%'+"(%"-X'%"
'6%#Y+"'%"-X1"%"
YY pZ%#1 YpZ%#(5 underlining indicates game period
S#u)e$# 5ue#'o$$a're
.fter each game period, the teacher too/ &ritten reports from the students on their
e0perience of the game% Most students e0pressed satisfaction &ith the game, seeming to
prefer the games &ith less to read +single &ords rather than definitions- and faster action
+li/e Cloc/ etters-% ne recurring comment stands out, that the players perceived the
systematic reappearance of &ords that &ere either ne& or had been involved in an error
as Kboring% o much for the careful recycling algorithm J and a reminder that there are
probably limits to the possibilities of edutainment%
(re)'c#or o" %ame ue
;iven the fact of very heavy and very light users, &ere there any obvious learner
characteristics that &ould predict game use? The first guess is that some learners might
not have enough *nglish to get started &ith the games% The participants &ere ran/
ordered by number of &ords succeeded and then divided into t&o groupings, those &ho
had succeeded in more than "### game &ords +86#7 do&n to "#'#-, and those &ho had
succeeded in fe&er than "### +1A8' do&n to 6(#-% The number of &ords succeeded &as
then compared to performance on the different levels of the evels Test at T1% The only
interesting relationship &as bet&een &ords succeeded and scores at the first 1### level%
-
8/13/2019 Study of My Word Coach
25/36
"(
f the heavy game users, $#3 had 1/ evels Test scores of 8#3 to $#3, suggesting they
probably /ne& about 8## or $## of the first thousand &ords of *nglish% f the light
users, $(3 had scores that &ere either belo& 6## &ords or else above '## &ords% 9n
other &ords, too much or too little /no&ledge of the first 1### &ords predicted light
game use5 a moderate /no&ledge predicted heavy game use%
;iven the crucial role of the first 1### &ords in *nglish at large +comprising at least $#3
of an average te0t and far more of an average conversation- or !ord =oach in particular
+it is the language of all the game2s instructions, feedbac/, definitions, and tutorial
messages-, a /no&ledge of only 7## or 6## &ords in this 4one &ould reasonably predict a
tough time ma/ing sense of the game% =onversely, a score of over '## may have given
learners the impression they &ere already good at *nglish +in the conte0t of their school
and neighbourhood- and had little to learn%
DISCUSSION
. pattern to notice across the measures is that &hile the results are in the same direction
for the t&o groups &ith respect to the game periods, they are larger and more consistently
significant for the first group that used the game despite their eual starting points and
amount of game use% 9t is probably safe to say there &as an e0citement to the first run that
had dissipated slightly &hen the game had become habitual% .nother pattern is that
fluency gains +reaction speed, access to *nglish &ords rather than code s&itches- &hile
strong appear to &ane as soon as the game period is over% ectives%
Wor) lear$e)
-
8/13/2019 Study of My Word Coach
26/36
"8
Cased on the performance of the t&o groups of learners, it appears that an average of
bet&een ""6 and (#6 of the &ords presumably met largely on !ord =oach &ere at least
remembered and understood, in that they could be matched to a definition on the evels
Test that &as not simply the one originally learned in the game% This &as an e0pansion of
vocabulary si4e of about 1#3 and "#3 respectively +giving some credibility to the
finding cited by ective of matching
game &ords to player levels%
The importance of these gains should be assessed against t&o bac/drops, +1- the mean
numbers of game &ords that had been through a full recycling +succeeded-, and +"- the
normal vocabulary gro&th that occurs &ith classroom learning% The average number of
&ords that had been succeeded &as roughly "(## and "'## &ords for the t&o groups,
such that (#6 &ords or >ust under "#3 of succeeded &ords &ere remembered by the first
group, and ""6 &ords or >ust under 1#3 &ere remembered for the second group%
The typical or habitual vocabulary gro&th for learners in Quebec schools has never been
measured, to my /no&ledge% The second group2s "A average &ords T1)T" is probably a
good start to&ard building a baseline% The average gro&th e0perienced in this period
appears plausible, in that "A &ords e0trapolates to "A# for a 1#)month school year and
about "7## for eight years of school *, and this tallies &ith studies &ith comparable
but older Quebec learners +Rahar et al, "##1-%
9t is interesting that the strongest /no&ledge gains +;roup 12s 6#( &ords T")T7- &ere
registered not right after the game period but at a t&o)month delay% Delayed appearance
of le0ical gro&th is not unprecedented +=obb, 1AAA- and may even be the norm
+unac/no&ledged o&ing to the general lac/ of delayed posttest measures in vocabulary
acuisition studies, as discussed in chmitt, in press-% 9t ma/es sense that &ords learned
inside one2s gro&th area are encountered again &ithin a short period and that further
consolidation and de)conte0tuali4ation ta/es place%
-
8/13/2019 Study of My Word Coach
27/36
"$
The freuency levels of the &ords learned deserves comment% @esults sho& the
participants had learned &ords across the first five freuency levels, but not to the $#3
criterion at any level +Eig% (-% ;iven the high coverage of the first and second thousand
&ords +Nation, "##1-, it &ould seem an important element in any designed approach to
vocabulary gro&th to fill these levels to at least '#3 /no&n items +achieving &hat
;rabe, "##A, p% 7"( calls a Kcritical mass of le0is- before moving the focus to less
freuent items% !ord =oach2s placement test may have failed to do this because it
assumed that learners entering the game had been learning &ords up to then roughly in
order of freuency% 9f this &as the case, then once a level had been identified &ith (#)
$#3 of &ords /no&n, this &ould be set as the &or/ing level% These learners, ho&ever,
had several such gro&th 4ones J their mean percentages &ere 6#)(#3 at T1 right across
the second through fifth levels +Eig% (-, ma/ing any of them plausible candidates for
learning 4one% 9t is li/ely the learning algorithms &ere confused by these uneven profiles,
and in their ongoing assessment /ept assigning learners to ne& levels, &here indeed the
learners /ept on pic/ing up ne& &ords%
Wor) '$ ue
ral production results sho& that &hile only a fe& ne& &ord families &ent into
immediate use in the Erog tories, the stories nonetheless changed on other units% tory
si4e gre& significantly follo&ing game use, but &here did these e0tra &ords come from ?
The presence of e0tra &ord to/ens indicates some additional elaboration of incidents in
the story +not >ust repetitions as these had been removed-% Cut it also indicates a strong
but unanticipated reduction in code s&itching, i%e% recourse to Erench &ords to flesh out
the accounts% The off)list component of the :ocabprofiles &as mainly 1 +Erench- &ords,
and this component dropped by ("3 for ;roup 1 +EW'%A$, pZ%#1 T1)T", pZ%#( T1)T7-
and 6#3 for ;roup " +pW%#8-% The non)game periods produced a pattern similar to the
one &e have seen before, for ;roup 1 some loss of gains at T7, and for ;roup " a T1)T"
baseline sho&ing little movement%
-
8/13/2019 Study of My Word Coach
28/36
"'
Table 8F n +"##(- found that ostensible bilinguals in Montreal often had
slo&er and more variant le0ical access times for common &ords than native spea/ers,
suggesting the use of attention)demanding rather than automati4ed le0ical processing in
reading% peed)up is not the same as automaticity, but it is presumably a step along the
&ay, and *lgort2s +"##$- strong finding for a similar training regime suggests that
gaming and le0ical access is a line &orth further development%
CONC-USION & ,UTURE
9s !ord =oach or something li/e it a solution to the problem &e started &ith, that many
learners are in serious need of a means of rapid vocabulary e0pansion? The results
presented above suggest it is a definite step along the &ay% 9n terms of ra& learning
-
8/13/2019 Study of My Word Coach
29/36
"A
po&er, an interesting game set attached to the currently available freuency lists seems a
clear alternative or supplement to the slo& accumulation of a le0icon through e0perience
and chance encounter% Cet&een 1#3 and "#3 of fully played &ords seem to have been
remembered and transferred, at least as far as being attached to a novel definition% 9n the
case of the heaviest ten players in the present study +over 8### &ords fully recycled-, this
could amount to as many as 1"## reasonably stable acuisitions in a couple of months
&ith corresponding gains in accessibility% uch an augmentation could ma/e a big
difference to an academic learner2s le0icon, say from 1'## to 7### /no&n &ord families
+or, from about $#3 to A#3 coverage in average academic te0ts- provided the &ords
&ere the right ones% !as that the case here?
These learners &ere pic/ing up &ords from across the first five 1###)levels, meaning that
they &ould largely not e0perience a coverage gain in &hat they &ere li/ely to be reading
in *nglish at school or else&here% The ma>ority of school te0ts or even home ne&spapers
they &ould be reading comprise mainly &ords from the first t&o 1### levels, and at these
levels their gains &ere minimal% !ith $#3 of first 1### items and 8#3 of second /no&n
after game use +Eig% (-, a simplified reader still presents roughly 1(3 un/no&n le0ical
items, as can be calculated at httpFHHle0tutor%caHclo4eHvpH%!hat this means is that despite
/no&ing "(## and more &ords across the first five levels, these learners could still find
their school materials heavy going% !hat it also means is that some of the gains at the
lo&er freuency levels &ould probably be lost &ithin a fe& months through lac/ of
further reinforcement and learning from the environment%
The /ey to better targeting of game content lies in the design of !ord =oach2s placement
and progress tests, or learner modeling% .t present, the game begins by testing a player2s
&ord /no&ledge at a medium level +fourth thousand &ords li/e rigged, sigh, inhale,and
poach- and &or/s do&n +third thousand, second thousand- until it finds a level &ith (#)
$#3 of &ords /no&n% The game assumes this to be the primary gro&th 4one, since the
level belo& that is presumably largely /no&n items% Cehind this choice, ho&ever, is the
further assumption that le0icons gro& more or less linearly according to freuency% This
appears to be the case in 1 le0icon building, as research by Ciemiller and lonim +"##1-
http://lextutor.ca/cloze/vp/http://lextutor.ca/cloze/vp/http://lextutor.ca/cloze/vp/ -
8/13/2019 Study of My Word Coach
30/36
7#
sho&s% 9t cannot be assumed in ", ho&ever, &here many factors +li/e accessibility of
lo&er freuency cognates and transfer of items from technical domains- could encourage
more top)heavy constructions% Eor many of the learners in this study, game play &ould
have begun at the third or even fourth 1###)level, &hen in fact there &ere further gro&th
4ones &ell belo& that, and more useful ones in terms of the greater coverage afforded by
higher freuency items and the greater gaps caused by not /no&ing them%
The alternative approach to learner modeling &ould be to start lo& and &or/ up, as doing
this &ould catch the most basic level of &ea/ness% 9t could, ho&ever, compromise the
allure of the game for 1 players to start &ith a test of basic &ords li/e meet, speed,
touch, andgrow)9t is a uestion of &hether a single game can serve both 1 and "
needs% 9t probably can, if the possibility of mi0ed profiles is incorporated into the learner
modeling process%
!or/ing at a less)than)ideal level does not, ho&ever, appear to affect implicit or
procedural level gains% Most of the &ords being played and learned &ere at the third
through fifth 1### levels, yet the le0ical access speed gains &ere achieved &ith first
1###)level &ords, and the e0tra &ord to/ens that replaced Erench &ords in the Erog
stories &ere also first)1### items% 9n other &ords, it appears that retrieving and rehearsing
medium freuency &ords under time pressure increases the access of high)freuency
items in the player2s true gro&th 4one, or 4one of productive ability% There appears to be
a generali4ed le0ical processing effect in the " that does not necessarily apply to the
e0act &ords played%
There is thus much more &or/ to do &ith this type of &ord game, and follo&ing the
resolution of the issues >ust mentioned still more testing is needed to determine &hether
learned game &ords are retained long term, &hether and &hen they go into active use,
and &hether fluency gains on the implicit level are permanent% The sense of previous
research &ould suggest that production and deeper comprehension depend on meeting
&ords not >ust matched to a gloss and recycled 1# times, although that is a reliable start,
but also in meaningful and varied conte0ts and situations% This of course could also
-
8/13/2019 Study of My Word Coach
31/36
71
happen in a game conte0t% The challenge follo&ing the successful revision and further
testing of !ord =oach is the integration of vocabulary design into a more immersive,
narrative game format%
RE,ERENCES
.dolphs, %, B chmitt, N% +"##7-% e0ical coverage of spo/en discourse%Applied
Linguistics, '*+6-, 6"(J67'%
.t/inson, @%=% +1A$"-% ptimi4ing the earning of a econd)anguage :ocabulary%
+ournal o( !perimental -sychology, ./+1-, 1"6)1"A%
Cauer, %, B Nation, 9%%
-
8/13/2019 Study of My Word Coach
32/36
7"
=obb, T% +1AA$-% Erom concord to le0iconF Development and test of a corpus)based
le0ical tutor% Unpublished
-
8/13/2019 Study of My Word Coach
33/36
77
;oulden, @%, Nation, n, %G% +"##"-% To&ards a unified account of the representation, processing and
acuisition of second language /no&ledge% $econd Language #esearch, l
-
8/13/2019 Study of My Word Coach
34/36
76
Lightspan #eport on Goodman !lementary $chool in Aldine, ">)+"##7-% @etrieved from
httpFHH&&&%plato%comHmediaH*valuation3"#tudiesH@H@alph3"#;%3"#;oodman
3"#*lementary%pdf
Mayer, M% +1A8$-%A boy, a dog, and a (rog)Ne& Lor/F
-
8/13/2019 Study of My Word Coach
35/36
7(
0ford University =omputing ervices +"##(-% "he %ritish =ational Corpus) 9nformation
at httpFHH&&&%natcorp%o0%ac%u/
-
8/13/2019 Study of My Word Coach
36/36
78
chmitt, N%, chmitt, D%, B =lapham, =% +"##1-% Developing and e0ploring the behaviour
of t&o ne& versions of the :ocabulary evels Test% Language "esting 3n, % +"##(-% .utomaticity in bilingualism and second)language
learning% 9n % Sroll B .% de ;root +*ds%-,:andboo9 o( %ilingualism5 -sycholinguistic
Approaches+pp% 7$1)7''-% 0ford, USF 0ford University