Ventral and dorsal streams for choosing word orderduring sentence productionMalathi Thothathiri1 and Michelle Rattinger

Department of Speech and Hearing Science, The George Washington University, Washington, DC 20052

Edited by Willem J. M. Levelt, Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands, and approved October 27, 2015 (received for review July25, 2015)

Proficient language use requires speakers to vary word order andchoose between different ways of expressing the samemeaning. Priorstatistical associations between individual verbs and different wordorders are known to influence speakers’ choices, but the underlyingneural mechanisms are unknown. Here we show that distinct neuralpathways are used for verbs with different statistical associations. Wemanipulated statistical experience by training participants in a lan-guage containing novel verbs and two alternative word orders(agent-before-patient, AP; patient-before-agent, PA). Some verbsappeared exclusively in AP, others exclusively in PA, and yet othersin both orders. Subsequently, we used sparse sampling neuroimagingto examine the neural substrates as participants generated new sen-tences in the scanner. Behaviorally, participants showed an overallpreference for AP order, but also increased PA order for verbs experi-enced in that order, reflecting statistical learning. Functional activationand connectivity analyses revealed distinct networks underlying theincreased PA production. Verbs experienced in both orders duringtraining preferentially recruited a ventral stream, indicating the useof conceptual processing for mapping meaning to word order. In con-trast, verbs experienced solely in PA order recruited dorsal pathways,indicating the use of selective attention and sensorimotor integrationfor choosing words in the right order. These results show that thebrain tracks the structural associations of individual verbs and thatthe same structural output may be achieved via ventral or dorsalstreams, depending on the type of regularities in the input.

language production | statistical learning | verb bias | sentence choice |fMRI

During communication, a speaker converts a thought or messageinto a sequence of words understood by a listener. Languages

contain different word orders that can indicate the same meaning.For example, having observed a dog chase a cat, an English-speakercould say either that “the dog chased the cat” or that “the cat waschased by the dog.” Similarly, “Bob gave John some money” and“Bob gave some money to John” are alternative descriptions of thesame event. These structural options, formally referred to as alter-nations, involve differential arrangement of the event participants,placing for example, the agent before the patient in active voicesentences, or the patient before the agent in passive voice sentences.Although languages permit different word orders, prior researchsuggests that some orders are both more prevalent across the world’slanguages and more preferred by speakers within a language. Thesepreferences could reflect the shaping of language by cognitive orcommunicative pressures (1, 2). One commonly noted preference isto mention the agent before the patient (3, 4). This leads to thequestion of how and when speakers choose alternate orders, forexample, in mentioning the patient before the agent when using thepassive voice. We investigated the influence of one proven factor:namely, differential language experience with different verbs.Verbs are central to sentence formation because they specify the

type of event being described, the number of participants in theevent, and the order in which different participants are mentionedin the sentence. Many studies have now demonstrated that thestatistical associations between verbs and structures in the languageinput (hereafter “verb bias”) influence subsequent sentence

comprehension and production (5–9). For example, both healthyadults and aphasic patients perform better in sentence in-terpretation when a verb is used in the structure with which it istypically associated than when it is used in a different structure(7). Similarly, people produce some verbs in some structuresmore than others, depending on past statistical tendencies (4, 8, 9).In this study, we investigated the neural instantiation of such verb-specific structural preferences.Psycholinguistic models have postulated two parallel routes for

learning to produce sentences (10). The “sequencing” route choosesthe next word in the sentence from the current word primarily usingpast knowledge of which word categories go together (10). The“meaning” route supports the same function via greater reliance onmeaning, including the roles that different participants play in anevent (e.g., agent) and how those roles map on to word order (10).Interestingly, these theoretically motivated learning routes mayalign with dual neural streams suggested by the neuropsychologicaland neuroimaging literature (10–13). The dorsal language “how”stream, which comprises suprasylvian projections between superiortemporal/inferior parietal and frontal regions, is thought to supportsensorimotor integration for sequencing (11–13). In contrast, theventral language “what” stream, which comprises infrasylvian con-nections within the temporal lobe and between anterior temporaland frontal regions, is thought to support the processing of meaning(12, 13). Although traditional models linked language production tothe dorsal pathway and language comprehension to the ventralpathway, recent models suggest that both streams can supportspeech (14). We explored the relative weighting of these streamsduring free sentence generation in a miniature artificial language.We used a miniature language paradigm to control the lan-

guage experience of each participant and investigate the effectsof this experience on subsequent language production. Such

Significance

Languages allow multiple word orders for expressing the samemeaning. Prior statistical experience with verbs in less-commonsentence structures is known to facilitate subsequent use of thosestructures. To investigate how this is implemented in the brain,we trained participants in an artificial language and examinedhow they used that language to describe novel scenarios. Dif-ferent verbs were associated with different word orders duringtraining. Functional activation and connectivity patterns revealedthat the choice of word order was supported by a ventral path-way for verbs that appeared in two competing orders, and dorsalpathways for verbs that appeared exclusively in the less-commonorder. Thus, the brain accomplishes the same language output viadifferent routes, depending on past experience.

Author contributions: M.T. designed research; M.T. and M.R. performed research; M.T.analyzed data; and M.T. and M.R. wrote the paper.

The authors declare no conflict of interest.

This article is a PNAS Direct Submission.1To whom correspondence should be addressed. Email: malathi@gwu.edu.

This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1514711112/-/DCSupplemental.

15456–15461 | PNAS | December 15, 2015 | vol. 112 | no. 50 www.pnas.org/cgi/doi/10.1073/pnas.1514711112

Dow

nloa

ded

by g

uest

on

July

24,

202

0

control is difficult to accomplish with the native language of theparticipants. The relevance of findings from miniature languageparadigms for psychological understanding of natural languageprocessing has been demonstrated previously (15). Furthermore,the processing of grammatical violations in an artificial languageshows similar brain signatures to those observed during natural lan-guage processing (16). In the present study, we manipulated partic-ipants’ exposure to novel verbs and sentence structures. The novelverbs (e.g., pelk) described transitive actions. The two structural al-ternatives began with the verb and placed either the agent or thepatient of the transitive action earlier in the sentence, akin to theactive versus passive voice distinction. Each participant completedthree sessions of training in the new language (Fig. 1). During thistraining phase, some verbs appeared exclusively in agent-before-patient order (hereafter AP-bias verbs), others appeared exclu-sively in patient-before-agent order (hereafter PA-bias verbs), andyet others appeared equally in the two orders (hereafter alternatingor Alt-bias verbs). After training, participants returned for a behav-ioral test in which they were asked to describe videos not seen before.Importantly, these videos could not be described using sentencesmemorized from training. They necessitated generating new sen-tences (Fig. 1A). Other than providing the verb to use and instructingthe participants to use the new language, the task was unconstrained.A subset of the participants returned for a scan test in which theyviewed and described new videos inside the scanner. We used asparse sampling design to measure neural activation associated withovert language production without motion-induced artifacts.Our analyses focused on two main questions: (i) Do statistical

verb biases induced during training influence subsequent sentenceproduction? Specifically, does prior exposure to some verbs in PAorder preferentially increase PA order production for those verbs?(ii) If so, how is this instantiated in the brain? Which regions showdifferential activity and connectivity for PA vs. AP order production,

and are the regions involved different for verbs with different sta-tistical associations? As a supplementary question, we hypothesizedthat producing the less-preferred PA order over the default APorder might require executive function. Accordingly, we gave eachparticipant well-established executive function tasks and examinedcorrelations between those measures and behavioral and neuralresponses during sentence production.

ResultsDo Statistical Verb Biases Influence Subsequent Production? In thebehavioral analyses, we sought to replicate previous evidence forstatistical learning of word order preferences (4–9). In the mainproduction task, participants saw novel videos and described them.We coded the responses as containing AP or PA order. Onlycomplete sentences containing the correct verb and the correcttwo nouns were included in the AP and PA order categories. Allother sentences, including those with incorrect verb or noun, ormore than one verb or two nouns, were classified as “Other.”Participants’ spoken responses showed an overall preference for APorder [AP order = 56.54%, PA order = 36.17%, Other = 7.29%. μ =50; t(35)= 2.06, P < 0.05]. Proportion of PA order produced differedaccording to two factors. There was an effect of the previous trialsuch that participants were less likely to use PA order if they hadproduced that order on the previous trial (PA order after previousAP order = 45.81%, PA order after previous PA order = 35.06%.β = −0.64, SE = 0.15, z = −4.38, P < 0.001). More importantly, therewas an effect of verb-bias (PA order used with Alt-bias verbs =42.47%; PA-bias verbs = 52.19%; AP-bias verbs = 29.88%). Plannedpairwise comparisons showed increased PA use for Alt-bias verbscompared with AP-bias verbs (β = 0.68, SE = 0.20, z = 3.35, P <0.001) and for PA-bias verbs compared with AP-bias verbs (β = 1.12,SE = 0.21, z = 5.46, P < 0.001). Alt-bias and PA-bias verbs did notdiffer from one another (z = 1.49, P > 0.1).Spoken responses during the scan test showed the same patterns,

including an overall preference for AP order [AP order = 55.62%,PA order = 40.48%, Other = 3.9%. μ = 50; t(14) = 3.45, P < 0.01]and lower PA use if PA order was used on the previous trial (PAorder after previous AP order = 52.99%, PA order after previousPA order = 28.6%. β = −1.13, SE = 0.11, z = −10.33, P < 0.001).As before, there was an effect of verb bias (PA order used withAlt-bias verbs = 44.29%; PA-bias verbs = 50.94%; AP-bias verbs =27.39%). Planned pairwise comparisons showed increased PA usefor Alt-bias verbs compared with AP-bias verbs (β = 0.86, SE = 0.18,z = 4.81, P < 0.001) and for PA-bias verbs compared with AP-biasverbs (β = 1.17, SE = 0.18, z = 6.32, P < 0.001). Alt-bias and PA-biasverbs did not differ from one another (z = 1.63, P > 0.1). Thus, thespoken responses of the participants who were scanned were rep-resentative of the larger set of participants in the behavioral test.These results show that speakers generally preferred the AP

order, consistent with an agent-first preference seen in many of theworld’s languages (3). In addition, overlying this overall AP pref-erence, there was an effect of verb bias on sentence production suchthat verbs experienced in the PA order (Alt-bias and PA-bias verbs)showed increased production of that order, consistent with thestatistical learning observed in natural and artificial language studies(4–9) (Fig. 1B). Proportion of PA order did not differ between Alt-bias and PA-bias verbs, despite the fact that the latter appeared inPA order 100% of the time during training. This might reflect aceiling effect because of the overarching preference for AP order.In addition to the production task, all participants in the be-

havioral test completed an acceptability judgment task, whichprovided a comprehension-based measure of learning. Participantssaw videos, heard accompanying sentences, and rated the accept-ability of those sentences on a scale of 1–5. For each participant, wecomputed a learning score, which was the difference between themean ratings for AP-bias and PA-bias verbs when they appearedin the statistically associated structure versus when they appearedin the unassociated structure. The average learning score across

A

B

Fig. 1. Training procedure, test procedure, and behavioral results. (A) Duringtraining, participants observed transitive events and heard and repeated thecorresponding sentences. During tests, participants observed and described newtransitive events that were different from training, thereby requiring thegeneration of new sentences. (B) Behavioral results from the scan test (n = 15).There was an overall preference for AP over PA order (Left). PA order was moreprevalent with Alt-bias and PA-bias than AP-bias verbs, reflecting statisticallearning (Right). These results mirror the patterns found during the behavioraltest (n = 36) (see text). Error bars denote SEs here and elsewhere.

Thothathiri and Rattinger PNAS | December 15, 2015 | vol. 112 | no. 50 | 15457

NEU

ROSC

IENCE

PSYC

HOLO

GICALAND

COGNITIVESC

IENCE

S

Dow

nloa

ded

by g

uest

on

July

24,

202

0

participants was significantly greater than zero [mean = 0.49, μ = 0,t(35) = 2.53, P < 0.05]. Taken together, the results above show thatparticipants encoded the statistical preferences of individual verbsand used those preferences during both language production andcomprehension, as language users are known to do in studies usingother paradigms (5–9).

How Is PA Order Facilitated for Verbs with Different StatisticalBiases? We first examined the regions involved in sentence pro-duction in general. Across all verbs and response types, sentenceproduction relative to baseline activated similar regions to thoseobserved for speech in natural languages, including the bilateralperisylvian cortices, bilateral motor cortices, and supplementarymotor cortex (17, 18) (Fig. S1). (Note: The baseline task involvedmanual rather than spoken responses. Therefore, this set of re-gions likely includes areas supporting language and speech aswell as those involved in the basic control of oral motor effec-tors.) In subsequent analyses, we specifically sought to determinethe neural substrates that facilitated PA order for Alt-bias andPA-bias verbs. Because PA and AP orders were more compa-rable after a previous AP structure, we analyzed that subset oftrials. We conducted three kinds of analyses, separately for Alt-bias and PA-bias verbs: (i) Whole brain PA > AP contrast toidentify regions showing increased activation for PA order pro-duction. (ii) Psychophysiological interaction (PPI) contrasts toidentify regions showing differential connectivity for PA vs. APorder. For these analyses, we chose two seed regions—left pre-central gyrus (left PG) and supplementary motor area (SMA)—that are consistently associated with speech planning and execution(18). We reasoned that brain areas that show differential connec-tivity with these seed regions were likely influencing participants’choice of word order during speech. And (iii), covariate analyses todetermine where functional connectivity correlates with individualdifferences in executive function.For Alt-bias verbs, whole-brain analysis showed PA > AP acti-

vation in bilateral superior temporal lobes, white matter tractsin the right hemisphere, and left precentral/superior frontal gyrus(Fig. 2A and Table S1). In the PPI analyses, we found increasedfunctional connectivity between the SMA and more superior por-tions of the right middle temporal gyrus, between the SMA and

right cerebellum, and between the left PG and more inferior por-tions of the right middle temporal gyrus (Fig. 2B and Table S2).Analysis of executive function measures revealed that PA orderproduction with Alt-bias verbs was correlated with performanceon the Stroop task (Fig. 2C) but not the number-letter task (rs < 0.4,Ps > 0.2). Those who were better at Stroop (i.e., in inhibitingthe default response), produced more PA order with these verbs(r = 0.61, corrected P < 0.05). To examine whether individual dif-ferences in inhibition correlated with differences in functionalconnectivity during sentence production, we included the Stroopmeasure as a covariate in the PPI analyses. This revealed a negativecorrelation between Stroop performance and connectivity betweenSMA and right hippocampus (Fig. 2D and Table S3).For PA-bias verbs, whole-brain analysis showed PA > AP acti-

vation in the right superior temporal lobe (Fig. 3A and Table S1).PPI analyses showed increased connectivity between the SMA andleft visual cortex and decreased connectivity between the left PGand right visual cortex (Fig. 3B and Table S2). Analysis of executivefunction measures revealed that PA order production for PA-biasverbs was correlated with mixing performance on the number-lettertask (Fig. 3C) but not other measures (rs < 0.03, Ps > 0.3). Thosewho were better able to maintain and execute multiple task rulesduring the number-letter task produced more PA order with PA-bias verbs (r = 0.74, corrected P < 0.05). When mixing performancewas entered as a covariate in the PPI analyses, we found a positivecorrelation with connectivity between the SMA and left opercularcortex and between the SMA and left visual cortex, and a negativecorrelation with connectivity between the left PG and right frontalpole (Fig. 3D and Table S3).The analyses thus far suggest different neural substrates for

PA order production with Alt-bias and PA-bias verbs. This mightbe especially surprising because all verbs were trained in anintermixed fashion and the proportion of PA order producedduring the test did not differ between the two conditions. Becausecluster thresholding could lead to quantitative differences ap-pearing as qualitative differences, we conducted confirmatory re-gion-of-interest (ROI) analyses within key regions identified above(Fig. 4). In the bilateral superior temporal clusters identified by thewhole-brain PA > AP analysis for Alt-bias verbs there was a sig-nificant effect of bias in both hemispheres [left: Huynh–Feldt F

A B

DC

Fig. 3. Neural substrates of PA order production with PA-bias verbs. (A) In-creased activation in right superior temporal lobe. Axial z = 2. (B) Increasedconnectivity between the SMA and left visual cortex, and decreased connec-tivity between the left PG and right visual cortex. (Upper) seed region; (Lower)functionally connected region; sagittal x = −20, 38. (C) Positive correlation be-tween percentage of PA produced and task mixing accuracy. (D) Task mixingaccuracy was positively correlated with connectivity between the SMA and leftopercular and visual cortices, and negatively correlated with connectivity be-tween the left PG and right frontal pole. Axial z = 4, 30, 30.

A C

B D

Fig. 2. Neural substrates of PA order production with Alt-bias verbs. (A) In-creased activation in bilateral superior temporal lobes, right white mattertracts, and left precentral/superior frontal gyrus. Axial z = 2, 58. (B) Increasedconnectivity between the SMA and right anterior/middle temporal lobe, SMAand right cerebellum, and left PG and right middle/inferior temporal lobe.(Upper) seed region; (Lower) functionally connected region; sagittal x = 42, 14,64. (C) Positive correlation between percentage of PA produced and Stroopaccuracy. (D) Stroop accuracy was negatively correlated with connectivity be-tween SMA and right hippocampus. Axial z = 0.

15458 | www.pnas.org/cgi/doi/10.1073/pnas.1514711112 Thothathiri and Rattinger

Dow

nloa

ded

by g

uest

on

July

24,

202

0

(2, 20)= 6.74, corrected P < 0.05; right: Huynh–Feldt F(2, 20)= 4.57,corrected P < 0.05]. (Note: Some subjects did not produce both APand PA order for some verb types in both runs, resulting in lowerdegrees of freedom for the repeated-measures analyses.) Bothclusters showed significant linear Alt-bias > PA-bias > AP-biastrends [left: F(1, 10) = 8.80, P < 0.05; right: F(1, 10) = 7.26, P <0.05] (Fig. 4 A and B). Similarly, in the two temporal lobeclusters identified by the PPI analyses for Alt-bias verbs, therewere significant bias effects [SMA-right temporal connectivity:Huynh–Feldt F(2, 20) = 17.35, corrected P < 0.05; left PG-righttemporal connectivity: Greenhouse–Geisser F(1.38, 13.75) = 10.06,corrected P < 0.05]. Both clusters showed significant linear Alt-bias> PA-bias >AP-bias trends [SMA-right temporal: F(1, 10) = 52.55,P < 0.05; left PG-right temporal: F(1, 10) = 19.24, P < 0.05] (Fig. 4C and D). Among the visual cortex clusters identified by the PPIanalyses for PA-bias verbs, there was a significant bias effect in leftPG–right visual connectivity [Huynh–Feldt F(1.84, 18.42) = 6.58,corrected P < 0.05] but not SMA–left visual connectivity (F < 2,P > 0.1). For the former, there was a significant quadratic Alt-bias >PA-bias < AP-bias trend [F(1, 10) = 15.49, P < 0.05] (Fig. 4E).Supplementary analyses confirmed that activation and connectivitydifferences between Alt-bias and PA-bias verbs remained aftertaking into account the frequency of different verbs during training(SI Materials and Methods and Fig. S2).These results show that neural activation and connectivity

patterns were different during PA order production with Alt-biasand PA-bias verbs. Specifically, greater activation within bilateraltemporal cortices and greater connectivity between speech-motor regions and the right temporal cortex suggests recruitment ofa ventral pathway for producing PA order with Alt-bias verbs. Incontrast, differential connectivity between speech-motor regionsand bilateral visual cortices and the left opercular cortex suggestsmodulation of dorsal pathways involved in visual attention andsensorimotor integration for producing PA order with PA-biasverbs (Fig. 5).

DiscussionWe found that prior experience with a less preferred word orderled to greater use of that order in a manner consistent withstatistical learning. There was increased PA production with Alt-bias and PA-bias verbs than AP-bias verbs. Proportion of PAorder produced with Alt-bias and PA-bias verbs did not differbut the underlying neural substrates were significantly different.Dual language streams have been proposed for extracting

meaning from sound during language comprehension (ventralstream) and for sequencing sounds during language production(dorsal stream) (11–13). Our finding of ventral stream involvementduring production with Alt-bias verbs challenges a strict divisionbetween the two modalities. It is consistent with growing evidencefor a unified system for understanding and speaking (14, 21). Thisunification may be particularly important during language acqui-sition, which is essentially learning to speak via listening to input(10, 12). Our results show that speakers used a comprehension-guided route to production, especially for Alt-bias verbs. Thisfinding suggests that encountering the same verb in multiplemeaning-to-order mappings may have triggered elaborate con-ceptual processing that in turn guided subsequent production. Incomparison with Alt-bias verbs, the results for PA-bias verbs showsignificantly weaker ventral stream involvement. Instead, PA pro-duction for these verbs showed modulation within the visual andopercular cortices. The opercular locus extended from the parietalto the frontal operculum. The parietal and frontal opercula arepart of a dorsal stream involved in sensorimotor integration andsequencing during articulation (12, 13). Such sensorimotor in-tegration may be particularly engaged when language production isnot highly practiced or automatic (e.g., nonnative speech) (22). Inthe present study, AP and PA order differed in the order of the twonouns, which were both familiar English words. Thus, our resultssuggest that lack of automaticity in producing novel word ordersmight recruit sensorimotor integration even if the words them-selves are well practiced. The visual cortex loci for PA-bias verbsspanned the hierarchy from early to midlevel regions, including theoccipital pole, cuneus, lingual gyrus, and lateral occipital cortex.The differential patterns obtained for the right versus left visualcortices likely reflect differential processing of stimuli in the leftversus right visual field. Although topographic organization isusually ascribed to early visual regions, recent results show that itmight extend to midlevel regions, such as the lateral occipital cortex

Fig. 5. Ventral and dorsal streams summary. The ventral language stream(solid lines) is involved in processing meaning (12–14, 19). Although traditionalmodels assumed left lateralization, recent models propose bilateral in-volvement (12, 19). Connectivity results for Alt-bias verbs (shown in red;reproduced from Fig. 2) suggest a ventral route to PA production for theseverbs. The dorsal language stream (dashed lines) is involved in sensorimotorintegration and phonological processing (11–14). It may be more left lateralizedthan the ventral stream (12). Connectivity results for PA-bias verbs (shown inblue; reproduced from Fig. 3) suggest that sensorimotor integration was usedfor these verbs. The dorsal frontoparietal attention network (dotted lines)supports top-down modulation and shifting of visual attention (20). Connec-tivity results for PA-bias verbs (shown in blue. Reproduced from Fig. 3) suggestthat the tuning of visual attention played a role in determining word order.(a)MTG, (anterior) middle temporal gyrus; fOP, frontal operculum; fORB/TRI,orbital/triangular inferior frontal gyrus; IPL/IPS, inferior parietal lobe/sulcus;sT/iP, superior temporal/inferior parietal cortex.

A B

DC E

Fig. 4. ROI Analyses. Each panel shows PA vs. AP order contrast estimates (n =11). Insets show parameter estimates for each order separately relative tobaseline. Text insets indicate significant polynomial trends (*P < .05). (A) Lineartrend in left superior temporal activation, with the highest activation for Alt-bias verbs. (B) Linear trend in right superior temporal activation, with thehighest activation for Alt-bias verbs. (C) Linear trend in the SMA-right anterior/middle temporal connectivity, with the highest connectivity for Alt-bias verbs.(D) Linear trend in the left PG–right middle/inferior temporal connectivity, withthe highest connectivity for Alt-bias verbs. (E) Quadratic trend in the left PG–right visual connectivity, with deactivation for PA-bias verbs.

Thothathiri and Rattinger PNAS | December 15, 2015 | vol. 112 | no. 50 | 15459

NEU

ROSC

IENCE

PSYC

HOLO

GICALAND

COGNITIVESC

IENCE

S

Dow

nloa

ded

by g

uest

on

July

24,

202

0

(23). In the present study, the videos depicted the agent on the leftand the patient on the right. Differential attention to the participanton the right over the one on the left could therefore facilitate PAorder, where the patient is mentioned first. Although such visuo-motor mappings are likely to be much less uniform in real-worldexperience, these results offer proof-of-principle that the languageproduction system uses regularities that are available in the input.A key finding of this study is that the relative involvement of the

ventral stream during sentence production varies depending on theverb-specific statistics experienced during learning. Experiencingcompetitive interactions between multiple meaning-to-order map-pings led to the recruitment of a meaning pathway during sub-sequent production. In contrast, the lack of such competitiveinteractions led to more direct integration between different sen-sory (visual and auditory) and motor representations. In accordancewith previous theoretical proposals, this suggests commonality be-tween the mechanisms used for language learning and languageproduction (10). The results are consistent with “division of labor”between semantic and nonsemantic processes, which has been ex-plored extensively within connectionist frameworks of single wordprocessing (24, 25). These models show that irregular words thatinvolve more opaque mapping between input and output—for ex-ample, between letter and sound during reading—come to relymore on a semantic route than regular words that can rely on amore direct nonsemantic route (24). Recent neurocomputationalimplementations link such division of labor to the differential re-cruitment of ventral and dorsal streams (14, 25). The present studyextends this principle to sentence level production by showinggreater ventral stream involvement for Alt-bias than PA-bias verbs.The results support the viewpoint that language use emerges in partvia flexible weighting of the interactions between primary neuralsystems involved in vision, semantics, phonology, and other mentalprocesses (25).PA order production with Alt-bias and PA-bias verbs corre-

lated with different executive function measures, namely Stroopand task-mixing performance, respectively. The Stroop task in-dexes inhibition of prepotent responses (26). Thus, positivecorrelation with Stroop accuracy suggests that producing the PAorder required inhibiting the prepotent AP order for Alt-biasverbs. Mixing accuracy in the number-letter task indexes partic-ipants’ ability to maintain and execute multiple task rules (27).Positive correlation with mixing accuracy, therefore, suggeststhat producing the PA order might have used a different taskrule rather than competitive interactions between the two wordorders for PA-bias verbs. This interpretation is supported by thefinding that participants who were poor at task mixing showedgreater left PG–right frontal pole connectivity during sentenceproduction. The right frontal pole is routinely implicated in taskswitching (28, 29).For Alt-bias verbs, and PA-bias verbs to a lesser extent, we

detected increased activation for PA production in the superior-most temporal lobes, including the auditory cortices, bilaterally.These regions are involved in auditory imagery and self-moni-toring during speech (17, 30, 31). Our results show that speakersparticularly monitored their utterances for verbs that hadappeared in both word orders, suggesting that competitive in-teractions play a role in triggering monitoring.The findings add to growing evidence for bilateral temporal lobe

involvement in language (12, 21). Right temporal lobe recruitmentfor Alt-bias verbs may be particularly related to the fact that theconceptual content of the task was significantly visual (19) or thefact that participants were learning a new language (32). Futurestudies are needed to extend the present findings to other languageproduction tasks and increased language experience.In conclusion, we have shown that different neural pathways

can give rise to the same structural output during languageproduction in a manner consistent with past learning experience.Support for a dual-stream model of language—analogous to the

one for vision—has grown in recent years. Our findings challengeaspects of the standard model and argue for a unified frameworkfor studying language acquisition, comprehension, and pro-duction. Statistical learning is a flexible and lifelong process thattakes advantage of regularities in the input. The neural sub-strates of language production are therefore unlikely to be in-dependent of the speaker’s learning history.

Materials and MethodsSubjects. Forty-two right-handed native English speakers (18–27 y; Mean age= 20.55 y; 29 female) completed all behavioral sessions (three training + onebehavioral test). Eighteen (18–27 y; Mean age = 21.06 y; 10 female) returnedfor a final scanning session. All participants gave consent under a protocolapproved by The George Washington University. Those who were scannedwere screened for MRI safety and provided additional MRI-specific consent.

Stimuli. The miniature language (4) contained novel verbs and sentencestructures. The 12 novel verbs (e.g., flern, stoom) described transitive actions(e.g., hit, hug). The verb forms corresponding to different meanings werecounterbalanced across five lists. Each participant was pseudorandomlyassigned to a list. During training, four verbs appeared only in AP order (AP-bias), four only in PA order (PA-bias), and two equally in the two orders(alternating or Alt-bias). Two other verbs were included to mimic synonymywithin language. These verbs were not tested during the scan test and arenot discussed further. Actions were depicted using videos involving puppets.Each video was roughly 4-s long and showed one puppet acting on another.For the training sentences, a female native English speaker recorded eachword at different sentential positions. These recordings were combined us-ing the SoX command line utility (sox.sourceforge.net).

Training. Stimuli were presented using E-prime. Each participant underwentthree identical training sessions, with no more than 3 d between sessions.Training involved watching videos where animal puppets acted on one an-other (e.g., zebra jumping on giraffe), listening to a prerecorded sentence(e.g., “pelk zebra giraffe”) and repeating it out loud. Each session contained48 trials with AP-bias, 48 trials with PA-bias, and 24 trials with Alt-bias verbs.Half of the AP-bias and PA-bias verbs appeared at low frequency (6 times persession) and half at high frequency (18 times per session). Alt-bias verbsappeared at an intermediate frequency (12 times per session).

Behavioral Test. The behavioral test was conducted within 3 d from the lasttraining session. Participants watched and described new videos containinganimal puppets not seen during training. They were familiarized with the labelsfor the newpuppets before testing. On each test trial, they saw the verb to use indescribing the upcoming video, watched the video clip (4,000 ms), and thenprovided a freely generated verbal response (maximum 5,000 ms). Each AP-bias,PA-bias, and Alt-bias verb appeared four times. Test trials were split across twoblocks. To simulate an interactive setting, participants were told that a computerprogram was trying to analyze and understand their utterances. Before eachtesting block, they played a picture-matching game with the computer that didnot involve content from the miniature language.

Subsequent to the production task, participants completed the accept-ability judgment task. Videos contained the same animal puppets as thoseseen during training. Each verb appeared in two critical trials, once in an APstructure and once in a PA structure.

Scan Test. The scan test was conducted within 3 d from the behavioral test.Outside the scanner, participants completed a short practicewith the baselinetask and a shortened training session for the miniature language. Inside thescanner, the test was comprised of two runs. Each run contained 70 criticaltrials, 15 baseline trials, and 5 null trials intermixed in an event-related design.Each trial lasted 9 s and was followed by 3 s of image acquisition. Duringcritical trials, participants saw the verb to use in describing the subsequentscene (500 ms), viewed a video clip (4,000 ms), and then provided a verbalresponse when a green circle appeared on the screen (maximum 4,500 ms).Participants were instructed to cease speaking when a red stop sign replacedthe green circle (3,000 ms). Each run contained 20 AP-bias trials, 30 PA-biastrials, and 20 Alt-bias trials. We included a greater number of PA-bias trialsanticipating an overall preference for the AP order. Each verb appearedbetween 5 and 10 times. The videos presented during the scan test weredifferent from those presented during the behavioral test and training.During baseline trials, participants viewed scrambled versions of the puppetvideos and pressed a button whenever a white plus sign appeared in arandom location on the screen (one to three times per trial). Foils consisted of

15460 | www.pnas.org/cgi/doi/10.1073/pnas.1514711112 Thothathiri and Rattinger

Dow

nloa

ded

by g

uest

on

July

24,

202

0

characters from an unfamiliar foreign script. During null trials, participantsviewed a white screen with a centered black fixation.

Executive Function Measures. Participants completed the Stroop (26) andnumber-letter (27) tasks at the end of different training sessions. In the Strooptask, they indicated the font color of words (blue, green or yellow) using key-board presses. Performance was calculated as the difference in accuracy be-tween incongruent-eligible and neutral trials as a percentage of accuracy onneutral trials (SI Materials and Methods). In the number-letter task, they atten-ded to the number or the letter portion of number-letter combinations (e.g., 9K)and responded using a number-based or letter-based rule respectively. Mixingperformance was calculated as the difference in accuracy between mixed taskand single task blocks as a percentage of accuracy on the single task blocks.Switching performance was calculated as the difference in accuracy betweenswitch and no-switch trials within the mixed block (SI Materials and Methods).

MRI Data Acquisition. Structural and functional images were acquired using a3T Siemens Trio scanner at the Center for Functional and Molecular Imagingat Georgetown University. Structural images were acquired using a sagittalT1-weighted MPRAGE sequence (TR = 1,900 ms, TE = 2.52 ms, flip angle = 9°,TI = 900 ms, slice thickness = 1 mm). Functional images were acquired usingan echoplanar imaging sequence (TR = 12,000 ms, TE = 30 ms, flip angle =90°, slice thickness = 3 mm) (see SI Materials and Methods and Fig. S3 fortemporal signal-to-noise ratio information). In the sparse sampling design,images were acquired only during the phase where subjects saw a red stopsign and were instructed to cease speaking. Overt responses were collectedusing a scanner-compatible microphone (FOMRI-III, Optoacoustics).

Behavioral Analyses. Audio recordings were transcribed and coded using Au-dacity. Six participants during thebehavioral test and three during the scan test didnot use PAorder evenonceper testingblock andwere excluded from the analyses.Because the dependent variable (structure produced) was binomial, we used logitmixed modeling. The models contained an intercept, two fixed factors (structureproduced on previous trial, verb-bias), and two random factors (subject, item).

Functional MRI Analyses. Images were processed and analyzed using FSL.Nonbrain voxels were removed using BET. Images were motion-correctedusingMCFLIRT, spatially smoothed using a Gaussian kernel (full-width at half-

maximum = 8 mm) and high-pass filtered (100 Hz). Statistical maps werenormalized to Montreal Neurological Institute (MNI)-152 space. Because weused a sparse sampling design, we did not apply any slice timing correction.For the whole-brain analyses, the general linear model contained 14 re-gressors. Twelve regressors modeled the spoken response phase of thecritical trials, one regressor modeled the video phase of the critical trials, andone regressor modeled the entirety of baseline trials. We were primarilyinterested in activity during the spoken-response phase. Behavioral resultsindicated that the structure produced on the current trial was influenced byboth the previous structure produced and the bias of the current verb.Therefore, the 12 regressors represented all 12 unique combinations of thesefactors. For example, AP-AP-AP represented trials where the previous struc-ture produced was AP, the current verb was AP-bias, and the current structureproduced was AP; PA-Alt-PA represented trials where the previous structureproduced was PA, the current verb was Alt-bias, and the current structure pro-duced was PA, and so on.Wemodeled the first 3 s of the spoken-response phasebecause behavioral measurements indicated that responses were usually com-pleted before that time (Mean duration = 2662.69 ms). First-level contrast mapsfor each subject and run were entered into fixed-effects second-level analysesfor each subject and then third level random-effects group analyses. Resultswere thresholded at Z > 2.58, cluster P < 0.05.

For the PPI analyses, we used the generalized PPI (gPPI) approach (SIMaterialsand Methods). Seed regions were extracted by identifying suprathresholdvoxels from the whole-brain contrast of all sentences versus baseline that laywithin the left PG and supplementary motor cortex (Harvard-Oxford structuralatlas, threshold > 30). Results were thresholded at Z > 2.3, cluster P < 0.05. Forthe ROI analyses, we adjusted P values using the Bonferroni correction (twicefor the whole-brain superior temporal ROIs, four times for the PPI ROIs). Weapplied the Greenhouse–Geisser or the Huynh–Feldt correction depending onwhether epsilon was less than or greater than 0.75.

Correlation Analyses.Wecomputed a total of six correlations, between PAorderpercentage for Alt-bias and PA-bias verbs and three executive functionmeasures(Stroop, number-letter mixing, and number-letter switching) (SI Materials andMethods). All P values were false-discovery rate-corrected (α = 0.05).

ACKNOWLEDGMENTS. We thank Lee Anne Steinberg, Fiacre Douglas, andother research assistants for preparing and running these experiments.

1. Goldin-Meadow S, So WC, Ozyürek A, Mylander C (2008) The natural order of events:How speakers of different languages represent events nonverbally. Proc Natl Acad SciUSA 105(27):9163–9168.

2. Gibson E, et al. (2013) A noisy-channel account of crosslinguistic word-order variation.Psychol Sci 24(7):1079–1088.

3. Jackendoff R (2002) Foundations of Language: Brain, Meaning, Grammar, Evolution(OUP, Oxford, UK).

4. Wonnacott E, Newport EL, Tanenhaus MK (2008) Acquiring and processing verb ar-gument structure: Distributional learning in a miniature language. Cognit Psychol56(3):165–209.

5. Trueswell JC, Tanenhaus MK, Kello C (1993) Verb-specific constraints in sentenceprocessing: Separating effects of lexical preference from garden-paths. J Exp PsycholLearn Mem Cogn 19(3):528–553.

6. Garnsey SM, Pearlmutter NJ, Myers E, Lotocky MA (1997) The contributions of verbbias and plausibility to the comprehension of temporarily ambiguous sentences.J Mem Lang 37(1):58–93.

7. Gahl S (2002) Lexical biases in aphasic sentence comprehension: An experimental andcorpus linguistic study. Aphasiology 16(12):1173–1198.

8. Stallings LM, MacDonald MC, O’Seaghdha PG (1998) Phrasal ordering constraints insentence production: Phrase length and verb disposition in heavy-NP shift. J MemLang 39(3):392–417.

9. Ferreira VS, Schotter ER (2013) Do verb bias effects on sentence production reflect sen-sitivity to comprehension or production factors? Q J Exp Psychol (Hove) 66(8):1548–1571.

10. Chang F, Dell GS, Bock K (2006) Becoming syntactic. Psychol Rev 113(2):234–272.11. Rauschecker JP (2011) An expanded role for the dorsal auditory pathway in sensori-

motor control and integration. Hear Res 271(1-2):16–25.12. Hickok G, Poeppel D (2007) The cortical organization of speech processing. Nat Rev

Neurosci 8(5):393–402.13. Saur D, et al. (2008) Ventral and dorsal pathways for language. Proc Natl Acad Sci USA

105(46):18035–18040.14. Ueno T, Saito S, Rogers TT, Lambon Ralph MA (2011) Lichtheim 2: Synthesizing

aphasia and the neural basis of language in a neurocomputational model of the dualdorsal-ventral language pathways. Neuron 72(2):385–396.

15. Romberg AR, Saffran JR (2010) Statistical learning and language acquisition. WileyInterdiscip Rev Cogn Sci 1(6):906–914.

16. Friederici AD, Steinhauer K, Pfeifer E (2002) Brain signatures of artificial languageprocessing: Evidence challenging the critical period hypothesis. Proc Natl Acad Sci USA99(1):529–534.

17. Indefrey P, Levelt WJ (2004) The spatial and temporal signatures of word productioncomponents. Cognition 92(1-2):101–144.

18. Guenther FH (2006) Cortical interactions underlying the production of speech sounds.J Commun Disord 39(5):350–365.

19. Rice GE, Lambon Ralph MA, Hoffman P (2015) The roles of left versus right anteriortemporal lobes in conceptual knowledge: An ALE meta-analysis of 97 functionalneuroimaging studies. Cereb Cortex 25(11):4374–4391.

20. Ptak R (2012) The frontoparietal attention network of the human brain: Action, sa-liency, and a priority map of the environment. Neuroscientist 18(5):502–515.

21. Silbert LJ, Honey CJ, Simony E, Poeppel D, Hasson U (2014) Coupled neural systemsunderlie the production and comprehension of naturalistic narrative speech. ProcNatl Acad Sci USA 111(43):E4687–E4696.

22. Simmonds AJ, Wise RJ, Dhanjal NS, Leech R (2011) A comparison of sensory-motoractivity during speech in first and second languages. J Neurophysiol 106(1):470–478.

23. Niemeier M, Goltz HC, Kuchinad A, Tweed DB, Vilis T (2005) A contralateral preference inthe lateral occipital area: Sensory and attentional mechanisms. Cereb Cortex 15(3):325–331.

24. Plaut DC, McClelland JL, Seidenberg MS, Patterson K (1996) Understanding normaland impaired word reading: Computational principles in quasi-regular domains.Psychol Rev 103(1):56–115.

25. Ueno T, et al. (2014) Not lost in translation: Generalization of the primary systemshypothesis to Japanese-specific language processes. J Cogn Neurosci 26(2):433–446.

26. Milham MP, et al. (2001) The relative involvement of anterior cingulate and pre-frontal cortex in attentional control depends on nature of conflict. Brain Res CognBrain Res 12(3):467–473.

27. Miyake A, et al. (2000) The unity and diversity of executive functions and their contributionsto complex “Frontal Lobe” tasks: A latent variable analysis. Cognit Psychol 41(1):49–100.

28. Sohn MH, Ursu S, Anderson JR, Stenger VA, Carter CS (2000) The role of prefrontalcortex and posterior parietal cortex in task switching. Proc Natl Acad Sci USA 97(24):13448–13453.

29. Rushworth MF, Hadland KA, Paus T, Sipila PK (2002) Role of the human medial frontalcortex in task switching: A combined fMRI and TMS study. J Neurophysiol 87(5):2577–2592.

30. Hashimoto Y, Sakai KL (2003) Brain activations during conscious self-monitoring ofspeech production with delayed auditory feedback: An fMRI study. Hum Brain Mapp20(1):22–28.

31. Fu CH, et al. (2006) An fMRI study of verbal self-monitoring: Neural correlates ofauditory verbal feedback. Cereb Cortex 16(7):969–977.

32. Dehaene S, et al. (1997) Anatomical variability in the cortical representation of firstand second language. Neuroreport 8(17):3809–3815.

Thothathiri and Rattinger PNAS | December 15, 2015 | vol. 112 | no. 50 | 15461

NEU

ROSC

IENCE

PSYC

HOLO

GICALAND

COGNITIVESC

IENCE

S

Dow

nloa

ded

by g

uest

on

July

24,

202

0