issues on language processing

7/31/2019 Issues on Language Processing

1/46

UNIVERSIT DEGLI STUDI DI CATANIA

SCUOLA SUPERIORE DI CATANIA

Daniele Virgillito

ISSUES ON LANGUAGE PROCESSING:

FROM THEORY TO BRAIN INVESTIGATIONS

DIPLOMA DI LICENZA

Relatore: Chiar.mo Prof. Marco Mazzone

ANNO ACCADEMICO 2006/2007


2/46

Daniele Virgillito 2

Contents

Abstract ......................................................................................................................3

Sommario ..................................................................................................................4

Acknowledgments...................................................................................................5

1. Linguistic theories and findings ...................................................................71.1 The layers of language...............................................................................7

1.2 Chomskys perspective .............................................................................9

1.3 Models of language processing..............................................................10

1.4 Time for meaning.....................................................................................12

1.5 Looking at the brain.................................................................................16

2. ERPs: what brainwaves can tell about language......................................18

2.1 Event-related Potentials ..........................................................................18

2.2 ERP correlates of semantics and syntax................................................19

2.3 More than meaning and structure.........................................................22

2.4 Semantics or syntax?................................................................................27

2.5 Chomskian account..................................................................................29

3. Mood and language........................................................................................32

3.1 Positive mood effects on language ........................................................32

Conclusions.............................................................................................................42

Bibliography ...........................................................................................................44


3/46

Issues on language processing: from theory to brain investigations 3

Abstract

The debate on language processing has seen its centre of gravity

progressively moving from the investigation about inner architecture

of language to the empirical observation of how language is

instantiated in the human brain.

The present work firstly intends to offer a review of the

proposed theoretical models defining general dynamics of meaning

and syntactic processing.It will present the main findings on this topic deriving from

studies on language carried out through the recording of event-related

potentials (ERPs), a research methodology based on the statistical

analysis of data coming from electroencephalogram (EEG) in

cooccurrence with linguistic stimuli.

These data seem to globally speak in favour of those

frameworks that, detaching themselves from the theoretical paradigm

imposed by Noam Chomsky in the last fifty years, conceive syntactic

component as interagent with other components.

It will then show the preliminary results of a study on the effect

of emotive states on language processing, carried out at ERP Lab of

NICI (Nijmegen Institute for Cognition and Information, Nijmegen,

Netherlands) in collaboration with Dorothee Chwilla and Constance

Vissers.

N400 ERP component (index of the cognitive effort of

semantic integration) in correspondance with unexpected linguistic

stimuli, appears to be mitigated, as predicted on the basis of

Federmeier et al. (2001), by positive mood.

This seems to encourage too the idea that language processing is

closely tied to other components of our cognitive system.


4/46


Sommario

Il dibattito sullelaborazione del linguaggio ha visto il suo

baricentro progressivamente spostarsi dallindagine sullarchitettura

interna dei sistemi linguistici allosservazione empirica di come il

linguaggio instanziato nel cervello umano.

Il presente lavoro intende offrire in primo luogo una rassegna

dei modelli teorici messi in campo per la definizione dei processi di

elaborazione del significato e della struttura sintattica.Presenter le principali conquiste dei relativi studi sul

linguaggio realizzati attraverso la rilevazione dei potenziali evento-

relati (ERPs), una metodologia di ricerca basata sullanalisi statistica

dei dati provenienti dallelettroencefalogramma (EEG) in cooccorrenza

con stimoli linguistici.

Questi dati sembrano favorire complessivamente quei modelli

che, prendendo le distanze dal modello teorico imposto da NoamChomsky negli ultimi cinquantanni, concepiscono la componente

sintattica del linguaggio come strettamente interagente con le altre

componenti.

Illustrer infine i risultati preliminari di uno studio sulleffetto

degli stati emotivi sullelaborazione del significato, condotto presso il

Laboratorio ERP del NICI (Nijmegen Institute for Cognition and

Information, Nijmegen, Paesi Bassi) in collaborazione con Dorothee

Chwilla e Constance Vissers.

La componente ERP N400 (indice dello sforzo cognitivo di

integrazione semantica) in corrispondenza ad elementi linguistici

inaspettati, appare essere mitigata, come previsto sulla base di

Federmeier et al. (2001), da uno stato emotivo positivo.

Anche questo sembra incoraggiare lidea che lelaborazione del

linguaggio sia strettamente legata ad altre componenti del nostro

sistema cognitivo.


5/46


Acknowledgments

This work is one of the fruits of a scientific path begun in 2004,

which led me from Sicily to Bologna to the Netherlands.

It would not have been possible without the continuous

guidance and support of its inspirator: Prof. Marco Mazzone.

I had the privilege of attending his course on Language Theory

(2005), fortunately still running at Scuola Superiore di Catania, which

broadened my horizons on the nature of language and cognition

through enlightening explanations and fruitful discussions on a vast

range of topics.

I hope that the SSC intellectual space devoted to neurosciences

of language which he created and promoted will be constantly

growing and attracting more students.

The experimental part of my thesis comes from five months

spent in Nijmegen, a Dutch small and ancient town very close to

Germany. I was a trainee at NICI (Nijmegen Institute for Cognition

and Information) under the supervision of Dr. Dorothee Chwilla,

director of the ERP Lab.

I want to thank her and Dr. Constance Vissers for offering me

the chance of getting hands-on experience on event-related potentials

and for their support during the project.

A dank u wel goes to all the people who helped me with and

during the experiments: Hubert, Gerardt, Nan, Sybrine, Dan and

many others; and to all the participants who lent me their heads and

three hours of their time.


6/46


Id like to thank all my mates at Scuola Superiore, for the

unforgettable moments spent with some of them, for letting me

improve my mimicking skills, and for their effort in keeping SSC alive.

A special thank goes to all the people who suffer(ed) the

distance from me: it is hard to go far from who you love in order to

follow your dreams.

In conclusion, Id like to give my endless and deepest thanks to

the one who gave me, alone, most of the things I have now, my

mother. Without her love, sacrifices and encouragement some of thethings I have done would have sounded simply impossible to me.

Thanks to all the people I love and who believe in me.


7/46


1. Linguistic theories and findings

1.1The layers of language

A large part of theories on language has depicted it as a rigidly

structured system, made up of different parts, combining but not

interacting each other. Each level of language structure would then be

separated from the others and ruled by its own inner principles.

Even in a simple sentence, like John is an American boy, we could

distinguish the different plans we are addressing. We can analyze

sounds, and the way they are structured in abstract entities (called

phonemes) that allow us to recognize words because of slight

articulatory differences (boy is not toy); we can observe the

morphological structure (from the ancient greek morph, shape), i.e. the

way words are built and modified by adding certain linguistic

elements (American is an adjective derived from the noun America plus

a suffix: [[America]N+an]A); it is also possible to look at the way words

are ordered and structured in a sentence, and how these rules differ or

converge in world languages (the phrase American boy lets us see how


8/46


English orders coupled adjectives and nouns, namely in a different

way than many other languages do).

In addition to this, we can understand the message the sentence

intends to convey because single words are either linked to non-

linguistic referents or help us to understand the existing relations

between them: this is called the semantic aspect of language.

As proposed by Evans1, one could imagine this theoretical

framework like a cake, composed of several different layers, each

corresponding to a plan of language (phonetic, morphologic, syntactic,

semantic). As we will explain soon, many linguistic theories insist in

addressing empirical questions taking just one of the layers into

account, never looking at a whole slice of the cake. This isautomatically translated in viewing linguistics as a collection of

separated sub-disciplines.

For the present purposes, we will attempt to look more in depth

at the relationship between two of the mentioned linguistic levels,

namely the syntactic and the semantic one, i.e. between the way words

are structured and their meaning. As we will see, this has represented

a huge battleground for modern studies on language.

1 See Evans et al. (2007).


9/46


We will now move to a brief sketch of some of the assumptions

underlying the approach outlined above.

1.2 Chomskys perspective

The separatist perspective is consistent with a particular view

of mind structure, the one interpreting it as a set of different modules,

each devoted to a specific cognitive function and dealing with a certain

type of information (see Fodor 1983).

Modules would be organized in a hierarchic way, leading from

the lowest levels to the higher ones, namely from the simplest specific

cognitive operations to the most complex and general ones. Modelsbased on such view of mental processes have for this reason been

labeled as bottom up, as opposed to top-down, i.e. based on high-

level influence over lower stages of elaboration.

In the realm of linguistics, one theoretical approach has tried to

offer a totally modular perspective on language, inspired by the work

of Noam Chomsky and his Generative Grammar paradigm. Chomsky

(1957) argues that a sentence like Colorless green ideas sleep furiously

shows the necessary features to be recognized as grammatical by a

native speaker of English. This happens independently from other

kinds of knowledge: in this sense it is particularly significant to point


10/46


out that, according to the chomskian view, the syntactic elaboration of

the sentence proceeds autonomously from the meaning of the

individual words. It represents an algorithmic and modular process,

separated at all from semantics.

This means that during sentence processing, a syntactic

structure would be built up first, according to syntactic information

alone (order of constituents, word inflections), and at a later stage

semantic and pragmatic information would be activated.

As we will now quickly see, this neat separation of syntax and

semantics has been questioned by other models of language

processing, drawing also on experimental evidence coming from

psycholinguistic research.

1.3 Models of language processing

Different approaches to how language is processed differ in the

stage at which syntactic structure is built and semantic information is

activated. The generative model, as outlined in 1.2, can be defined as

a syntax-first perspective on sentence perception.

The so-called constraint-based models (see for example

MacDonald, Pearlmutter & Seidenberg 1994) propose that meaning is

partially accessed during syntactic structure building, and can


11/46


somehow influence sentence structuring. Some semantic features, like

animacy, would tend to be active during the processing and so play an

early and important role in the final outcome of the analysis. In

addition to this, a series of factors such as expectancy and frequency of

use of a syntactic structure would influence the recognition of the

actual configuration of a sentence.

Trueswell et al. (1994) experimentally shows with an eye-

tracking study the effect of meaning over syntactic parsing. The test

reveals longer fixation-durations on disambiguating phrases like by the

lawyer in The defendant examined by the lawyer (see fig.1): one could

plausibly imagine, on a semantic basis, that examined is not a past

participle but the finite verb bound to the subject the defendant.

Figure 1


12/46


Bever, Sanz & Townsend (1998) assign a much more important

role to semantics. According to them, only semantic and pragmatic

information is used in a first stage to build an initial representation of

the sentence, that consequently constrains its syntactic analysis. This

and similar approaches have been labeled as semantics-first and

their model can be approximately summarized by the phrase

semantics proposes, syntax disposes. It is natural that these theories

are mainly based on non-modular accounts of mental processes, since

they outline a top-down model of language processing.

Psycholinguistic research has been trying to empirically test the

approaches sketched above, attempting to question theoretical models

by experimentally looking at the way human beings process language.This has led to a strong criticism towards the chomskian framework.

In the next paragraph, we will recall one of the first and best

known findings against Chomskys claims.

1.4Time for meaning

Language works as part of the human cognitive system and it

can then be subject to objective scientific measurement. Human

behaviour is linked to inner mental processes and variations in the


13/46


latter ones can lead to variation in behaviour. It is generally assumed

that statistically significant time differences between behavioural

responses point to differences in the corresponding mental processes.

One of Chomskys first claims was that sentence complexity

depended only on the numbers of syntactic transformations that the

so-called deep structure (specifying the logical role of each element) had

to undergo in order to get to the final outcome, the superficial structure

(i.e., the actual sentence including the final phrase order and

structure). This has been called the Derivational Theory of Complexity

(henceforth, DTC).

For instance, the deep structure

[[John]NP [[read]V [the book]NP ]VP ]S

can give rise to the following sentences, differing just in the

transformational rule(s) applied:

Transformational Rule Sentence

Active John read the book.

Passive The book was read by John.


14/46


Negative John didnt read the book.

Wh-Question What did John read?

Passive + Negative The book was not read by John.

Passive + Wh-Question What was read by John?

One of the first studies addressing the psychological reality of

the DTC hypothesis was the one carried out by Slobin (1966). His

participants had to listen to a sentence while looking at a picture,

depicting objects and human beings interacting in different ways. The

task assigned was judging as quickly as possible, by pressing a button,

whether the picture matched with the sentence.

The linguistic materials were organized according to two

independent variables, i.e. active/passive and reversible/irreversible,

mixed in four final conditions.

The latter variable (reversibility) consisted in the possibility of

inverting Agent and Theme in the same sentence with a plausible

result: The sentence The horse was kicked by the cow is reversible,

whereas The fence was kicked by the cow is not, since the Theme the fence

is inanimate and cannot replace, due to the verbs restrictions, the

Agent the cow.


15/46


Although the data obtained in the active condition was

compatible with the chomskian approach, the surprising finding was

that participants mean reaction times (RTs) in verifying passive

reversible sentences were higher than those shown for passive

irreversible ones (see fig.2 for an example).

Figure 2

This result was interpreted in the following way: since no

syntactic divergence can be found between these two kinds of

sentence, being reversibility a purely semantic feature, the difference

in the RTs represented a reliable index of semantic influence over

language processing.

Slobins is one of the first experimental studies addressing the

congruence of a theoretical linguistic model with the actual way

language is processed.


16/46


1.5Looking at the brain

The natural place for studying cognition in vivo is where mental

processes are born and spread in time: the human brain. Over the last

years many brain-imaging techniques have been developed in order to

find neural correlates of cognitive models. They attempt to find

reliable relationships between certain patterns of brain activation and

specific supposed cognitive (sets of) operations.

At the present state of affairs, these techniques mainly differ in

their capacity and goals: resonance-based methods, such as fMRI,

manage to obtain a very detailed spatial resolution of the studied

processes, but no accurate time resolution. That is to say that they

easily localize brain activity in specific areas, but cannot exactly

correlate it with external occurring factors since the latter are usually

presented very fast.

EEG (electroencephalogram)-based techniques, on the other

hand, dont succeed in obtaining an accurate space mapping of

cerebral activation, but find their strength in their extremely accurate

time resolution. They can then easily correlate external stimuli


17/46


presented at a very fast rate with variations in brain activity unfolding

in a limited time.

One of the best known EEG-based techniques, namely ERPs

(event-related potentials), will be the object of the next chapter. As we

will see, this experimental methodology has proven to be very useful

to investigate language: it provided evidence for unravelling the

complexity of linguistic processes and trying to link some of them to

more general cognitive domains.


18/46


2. ERPs: what brainwaves can tell about language

2.1 Event-related Potentials

Event-related Potentials (ERPs) represent one of the most

widespread experimental techniques in brain imaging studies about

language.2

They are based on Electro-Encephalogram (EEG), a recording of

the spontaneous electrical human brain activity and its variations over

time. It is generally measured via electrodes attached on the scalp,

mounted in an elastic cap.

ERPs are extracted from the raw EEG by time-locking it to a

particular sensorimotor or cognitive event, so that it is possible to

examine the brain response to a specific kind of stimulus. To properly

extract ERPs, it is necessary to average a large number of records time-

locked to the same stimulus type.

The outcome of the averaged signal is a series of negative and

positive voltage peaks, each one having a specific latency and

2 See Kutas & Van Petten (1988).


19/46


distribution across the scalp. These peaks are called ERP components:

they can be either exogenous or endogenous.

The first ones are the earliest components (around 100 ms after

stimulus onset) and their features mirror the physical properties of

stimuli. Cognitive studies (in particular linguistic ones) only address

the so-called endogenous components: these derive from higher levels

of information processing (their latencies are often over 300 ms) and

are mainly influenced by psychological factors.

Endogenous ERP components are usually labeled according to

two parameters: polarity and peak latency. P300, for instance is a

positive voltage fluctuation peaking around 300 milliseconds after the

stimulus onset. We will now review two of the best known language-related ERP components.

2.2 ERP correlates of semantics and syntax

The "separatist" approach sketched in chapter 1 has influenced

the early stage of ERP studies on language. Many scholars encouraged

the view that semantic and syntactic processing proceeded in

autonomous ways and were mirrored by different and independent

ERP components too (see fig. 5 for an example in the same sentential


20/46


context). This assumption was particularly clear at the beginning, but

changed somehow as new results appeared.

The N400 component (a negative potential peaking around 400

ms after word onset) has traditionally been labeled as an index of

semantic integration. Kutas and Hillyard (1980) discovered that all

open class words elicited this kind of negativity. In addition to this,

they found out that words that rendered sentences semantically

anomalous elicited a more negative component.

Figure 3

One of their most famous examples was He spread the warm bread

with socks: compared to the brainwave elicited by the final word butter

in the same sentence, a clearly larger negativity was observed. For this

reason, N400 has been linked to the ease with which a word is

semantically integrated in a sentential context (see figure 3).


21/46


The brain response to syntactic processing was quite different. A

later positivity (peaking at about 600 ms after word onset) was

observed in relation to verb agreement anomalies, or wrong

constituent order, e.g. The spoilt child throw the toys on the floor, or The

expensive very tulip... (Hagoort et al. 1993). A late positivity was also

elicited by garden-path sentences, i.e. initially syntactically ambiguous

sentences in which the first (more plausible) structural analysis turns

out to be the wrong one, like in The woman convinced her children are

noisy (see fig. 4 for an example), where the parser plausibly supposes

at the beginning that her is a possessive adjective while it is a object

pronoun, as clear from the overall structure of the sentence.

Figure 4

The so called P600 was then defined as the brain reaction to

grammaticality anomalies, and was thought to mirror a process of

syntactic reanalysis and reprocessing.


22/46


Figure 5

As we will now see, new experimental results coming from

more recent studies suggested partially different and broader

perspectives on the roles played by N400 and P600 components.

2.3More than meaning and structure

As outlined above, the N400 was thought to represent a

semantic processing component, modulated by meaning relations

existing in the mental lexicon. As shown by many studies addressing

the function of N400 in non-anomalous sentences, an N400 effect was

clearly associated with a lower cloze probability. That is to say that


23/46


this negativity was not only depending on semantic factors (although

some results could speak in favour of this conclusion: see fig. 6).

Figure 6

An N400 was clearly elicited by non-anomalous but unexpected

words, as the final word in He mailed the letter without a thought,

compared to He mailed the letter without a stamp (Kutas & Hillyard

1984). It was also observed in world knowledge violations, such as in

Dutch trains are white: Dutch speakers know, in fact, that they are

yellow (Hagoort et al. 2004). As a matter of fact, N400 can be defined

as a general index of word integration, sensitive to semantic,

pragmatic and discourse factors.


24/46


Several ERP studies showed a clear P600 effect as a response to

non syntactic anomalies. Vissers, Chwilla, & Kolk (2006) investigated

the brain response to misspellings at the word level (see fig.7)

Figure 7

Their paradigm involved the comparison between a high-cloze

condition (the critical, misspelled pseudohomophone word was the

most expected one) and a low-cloze one (the critical, misspelled

pseudohomophone word was not expected at all), e.g. In the library the

pupils borrow books/boeks vs. In the library the pupils borrow chairs/chayrs.


25/46


A clear late positivity (P600) was found in high-cloze sentences

containing misspellings. This encouraged the perspective according to

which P600 would reflect a more general process of monitoring in

language perception, aiming at the detection of a possible processing

error.

In the study cited above, this component appeared only in the

high-cloze pseudohomophone condition, namely the one in which it is

plausible to suppose that a strong conflict occurred between two

elements: the phonological representation of the word (perfectly

congruent with the sentential constraints, and highly expected) and its

orthographic realization, which triggered a monitoring response.

As summarized by Vissers et al. (2006),

The ERP data confirmed the present prediction in that only

pseudohomophones embedded in a high-cloze context gave rise to a

P600 effect. Because the words from which the pseudohomophones

were derived were highly expected, initially the pseudohomophones

were easily integrated into the higher order meaning representation

of the context. After all, the phonological representation of the

pseudohomophone is congruent with the sentential constraints. But

when the subject detected the misspelling, which signals a possible

processing error, a monitoring response was triggered.


26/46


Such perspective on P600s functional role has been confirmed

by further studies addressing a different kind of mismatch at the

sentence level. Vissers, Chwilla, Van de Meerendonk & Kolk (2008),

for instance, presented participants several pictures of spatial arrays

followed by a sentence giving a correct or incorrect description of the

picture.

For example:

Picture Sentence

The triangle stands above the square.

The triangle stands below the circle.

As predicted, the mismatches created a conflict between the

conceptual representation on the basis of the picture and the actual

sentence, and therefore led to a P600 effect.


27/46


2.4Semantics or syntax?

According to the experimental results illustrated above, no clear

definition of the functional significance of language-related

components can be established. Until a few years ago, however, N400

and P600 were thought to belong to separated, albeit broad, areas of

sentence processing.

Further language ERP studies have shown quite controversial

late positivities in unusual situations (see Kolk & Chwilla 2007). The

most surprising finding was what we could define a semantic P600.

As outlined in 2.2, late positivities traditionally occurred in

sentences showing syntactic anomalies or ambiguities. In particular,

many experiments addressed a morphosyntactic P600, namely the one

elicited by anomalies in subject-verb agreement.

The unexpected result came from semantically anomalous

sentences like The cat that from the mice fledsing (literal translation

from Dutch: De kat die voor de muizen vluchtte, see Kolk, Chwilla, Van

Herten & Oor 2003). Meaning anomalies were supposed to elicit an

N400 effect and no positivity, but the opposite happened (see fig. 8).


28/46


Figure 8

It was however possible to reconcile this surprising finding with

a syntactic view of P600. The sentence type reported above did not

involve any kind of syntactic anomaly or ambiguity. Although it is a

highly implausible sentence (mice are supposed to flee from cats), its

structure doesnt violate any grammatical rule.

As a matter of fact, readers processing language dont always

apply a rigid parse ruled by grammar. The processing of sentences is

often guided by heuristics-based principles. Heuristic is represented in

this case by a set of strategies based on plausibility, intended to

process sentence meanings by overcoming a rule-based analysis of

sentence structure. In everyday language it is in fact often possible to

derive a correct structural representation of a sentence simply drawing


29/46


on the semantic features of the lexical items, and combining them in a

plausible way.

Processing the sentence The cat that from the mice fledsing. could

have led to assume that the plausible subject would be the mice, and

then trigger a P600 due to the morphosyntactic anomaly: no agreement

between the mice (plural) and the verbal form (singular). No

correspondence between the predicted and the observed inflection.

This interpretation was questioned by Van Herten, Kolk &

Chwilla (2005). The same kind of sentences was presented to

participants, with one difference: the two noun phrases never differed

in grammatical number, avoiding then agreement errors between the

plausible subject and its related verb form.Notwithstanding this last modification, a late positivity occurred

at the verb level. This challenged the mentioned syntactic account of

P600 and can thus be explained only in terms of monitoring during

language perception (see 2.3).

2.5 Chomskian account

Electrophysiology of language has proven to show a very

heterogeneous panorama of brain responses to linguistic processes,


30/46


leading to difficulties in deciphering the functional origins and range

limits of each ERP component.

In spite of the limits shown by current accounts of language-

related brain activation patterns, a remainder of the chomskian

approach is still present in the debate.

In particular, language-specificity of ERP components, in spite

of the counterevidence presented above, has been defended by

Friederici and other scholars trying to outline a brain activation model

consistent with the separatist thesis, i.e. the one supporting the

autonomy of syntactic processes. We will now move to the analysis of

part of the chomskian approach to neurolinguistic findings (see

Friederici et al. 2006).The so-called ELAN (early left anterior negativity) was the

brainwave found in correlation with phrase structure violations. Due

to its very short latency (between 100 and 250 ms from the stimulus

onset) and its independence from attentional factors, ELAN was

defined as an index of early syntactic processing, involving the

computation of local phrase structures according to word category

information. The definition of a low-level phrase would be influenced

by word-level features. A P600 would then be correlated with a final

integration process leading to the structuring of the overall sentence.


31/46


Although these results point to a syntactic-only brain mapping

of linguistic processes, they seem as pointed out by Mazzone (in press)

somehow to contradict the theoretical assumptions of the chomskian

Minimalist Program (Chomsky 1995). According to the latter, the same

operation of Merge (the making up of a new syntactic unit from two

syntactic objects) should be working both at the local phrase level (e.g.

the phrase The old man) and at the highest phrase level, i.e. the sentence

(e.g. The old man bought a small cake).

The model outlined above seems to correlate Merge at different

levels with qualitative differences in the cerebral activation timecourse

and mapping. fMRI studies, in fact, showed the predominance of two

distinct brain areas: the frontal operculum and Brocas area, and ERPrecording highlighted two different components too.

These and many other findings discourage the view that

linguistic processes proceeds through fixed serial stages and are

centered on syntax. On the one hand, sub-components of language, i.e.

the layers (see 3.3) in which language is normally dissected, seem

to continuously interact during related cognitive dynamics; on the

other hand, it emerges that non-linguistic factors can affect linguistic

processes, for example their timecourse, and the effort required.

It is the case of the role of emotive states in semantic processing:

we will talk about this in the following chapter.


32/46


3. Mood and language

3.1 Positive mood effects on language

Event-related potentials have proven to represent a very

profitable experimental paradigm for assessing not only the basic

nature of cognitive processing, but also how external psychological

factors can globally affect the dynamic of specific processes.

One of the factors involved in these assessments has been

emotive state. As clearly stated in Federmeier et al. (2001),

Empirical research concurs with the everyday intuition that

our moods influence our thinking, judgment, and perceptions.

Transient, mild positive mood has been shown to increase

integrative decision-making and subjective risk assessment and

to facilitate flexibility of thinking and problem solving.

We will now review two of the most acknowledged findings on

the effect of positive mood over the language domain.


33/46


Kirson (1990) used the sentence verification task (asking subjects

to state if a sentence is true or false). The experiment provided

participants with sentences describing category exemplar relationships

(e.g. A robin/coin is a bird). True sentences contained both exemplars

that were a near associate of the category word and those with a more

distant association (e.g. A robin/parrot is a bird).

After inducing either a neutral or a positive mood by showing

subjects respectively either a boring or an amusing (as rated by a

group of students) videotape, reaction times were recorded during

sentence verification. As expected, sentences depicting a close

relationship were verified in less time than those describing a distant

relationship. This difference was smaller, however, with positivemood induction, as it specifically shortened participants time to verify

distant sentences (see fig. 9).

These findings, replicated in a lexical decision task, supported

the notion that the semantic distance between distant exemplars and

their categories was functionally reduced due to the positive mood

condition.


34/46


Figure 9

A second study carried out by Federmeier, Kirson, Moreno &

Kutas (2001) investigated positive mood effect on language processing

through ERP recording.

Participants were presented with sentence pair contexts (e.g.

They wanted to make the hotel look more like a tropical resort. So, along the

driveway they planted rows of) ended with (1) the most expected

ending, as determined by cloze probability ratings (expected

exemplars, e.g. palms), (2) an unexpected item from the expected

semantic category (within category violations, e.g. pines), or (3) an

equally unexpected item from a different, though related, semantic

category (between category violations, e.g. tulips).

Mood was manipulated exposing participants to different kind

of photos, intended to engender particular emotive responses. The

effectiveness of the induction was positively tested after the

experiment through a questionnaire.


35/46


As summarized by Federmeier et al. (2001),

Under neutral mood, N400 amplitudes were smallest for

expected items and smaller for unexpected items when these

came from the expected category. In contrast, under positive

mood, N400 amplitudes to the two types of unexpected items

did not differ (see fig.10). Positive mood seemed to specifically

facilitate the processing of distantly-related, unexpected items.

Figure 10

In both studies, positive mood contributes to the process of

semantic retrieval and integration. It facilitates in both cases the access


36/46


to distant concepts and, according to the N400 effect found, makes

their processing easier.

We will now present the preliminary results of our study on the

effect of emotive state on language processing.

Our study replicated Vissers, Chwilla, & Kolk (2006)

investigating brain response to misspellings at the word level, with the

addition of a mood manipulation.

We first constructed 127 simple declarative sentence fragments

and used these in a cloze test with 25 subjects to obtain highly

expected (high-cloze) critical words. Of these 127 sentences, 116

sentences were completed with the same word by 91% of the

participants. These were used as the high-cloze context sentencefragments in this study.

We then created 116 low-cloze context sentences by exchanging

the critical word from a high-cloze context fragment with the critical

word from another high-cloze context fragment. For example, we

exchanged the critical word from In that library the pupils borrow books

to take home with the critical word from The pillows are stuffed with

feathers which makes them feel soft resulting in the following low-cloze

fragment The pillows are stuffed with books which makes them feel hard. The

critical word was always in mid-sentence position.


37/46


A further experimental manipulation was Lexicality (correct

word vs. pseudohomophone derived from the correct word).

Every critical word occurred in a correct version and a

pseudohomophonic version. The pseudohomophone was created by

changing the vowel of the second syllable, keeping phonology the

same.

The two experimental manipulations Context and Lexicality

were crossed. As a result, there were four conditions and thus, four

experimental sentence types: high-cloze correct word sentences, high-

cloze pseudohomophone sentences, low-cloze correct word sentences,

and low-cloze pseudohomophone sentences; yielding a total set of 464

sentences. The four versions of each sentence were counterbalancedacross lists. Each list contained each sentence context (in a high-cloze

or a low-cloze version) and each critical word (in a correct word or a

pseudohomophone version) only once.

So, each list contained 29 high-cloze correct word sentences, 29

high-cloze pseudohomophone sentences, 29 low-cloze correct word

sentences, and 29 low-cloze pseudohomophone sentences. To each list,

60 filler sentences were added: 30 correct sentences, 10 sentences with

a pseudohomophone at the beginning of the sentence, 10 sentences

with a pseudohomophone in the middle of the sentence, and 10

sentences with a pseudohomophone at the end of the sentence.


38/46


With the purpose of comparing the monitoring effect supposed

to be elicited by high-cloze pseudohomophones with the classical

syntactic P600, we added to the material a set of sentences

containing morphosyntactic violations. The related ERP results will

not be included in the present work.

For the EEG study, participants were seated in an experimental

room. Sentences were presented in serial visual presentation mode at

the center of a PC monitor. Word duration was 345 ms and the

stimulus-onset asynchrony (SOA) was 645 ms. Sentence final words

were followed by a full stop. The intertrial interval was 2 s.

Words were presented in black capitals on a white background

in a 9 cm by 2 cm window at a viewing distance of approximately 1m. Each sentence was preceded by a fixation cross (duration 510 ms)

followed by a 500 ms blank screen. The experimental list was split up

into six blocks; there was a brief pause between blocks and each block

was preceded by two filler items.

The mood manipulation was carried out by means of two sets of

video fragments, each shown before and between each experimental

block to two groups of participants. They were intended to engender

positive and negative mood conditions (henceforth, PMC and NMC).


39/46


The clips were taken from Happy Feet (G. Miller, Warner

Bros. 2006) and Sophies Choice (A.J. Pakula, Universal Pictures

1982) for the positive and negative manipulation respectively.

The effectiveness of the mood manipulation was assessed

through an online self-rating of participants emotive state.

Participants were instructed to attentively read the sentences.

Because eye movements distort the EEG recording, participants were

trained to make eye movements, e.g., blinks, only in the period

between the end of the last sentence and the beginning of the next one.

The electroencephalogram (EEG) was recorded with 27 tin

electrodes mounted in an elastic electrode cap (Electrocap

International; see fig.11 for the montage).

Figure 11


40/46


The electrode positions included standard International 1020

system locations over the left and right hemispheres at the frontal (F3,

F4, F7, and F8), midline (Fz, Cz, Pz, and Oz), parietal (P3 and P4), and

temporal (T5 and T6) sites. Eight extra electrodes were placed at the

frontal (F3A, F4A, F7A, and F8A), midline (Fza and Oz), and parietal

(P3P and P4P) sites. In addition, eight electrodes were placed at

nonstandard electrode positions previously found to be sensitive to

language manipulations (e.g., Holcomb & Neville, 1990): left and right

anteriortemporal sites (LAT and RAT: 50% of the distance between

T3/4 and F7/8), left and right temporal sites (LT and RT: 33% of the

interaural distance lateral to Cz), left and right temporoparietal (LTP

and RTP: Wernicke's area and its right hemisphere homologue: 30% ofthe interaural distance lateral to a point 13% of the nasioninion

distance posterior to Cz), and left and right occipital sites (OL and OR:

50% of the distance between T5/6 and O1/2). The left mastoid served

as reference. Electrode impedance was less than 3 k. The electro-

oculogram (EOG) was recorded bipolarly; vertical EOG was recorded

by placing an electrode above and below the right eye and the

horizontal EOG was recorded via a right to left canthal montage. The

signals were amplified (time constant = 8 s, bandpass = 0.0230 Hz),

and digitized online at 200 Hz.


41/46


Presentation of stimuli and recording of performance data were

accomplished by a Macintosh computer.

For N400 (300 to 500 ms epoch), an interaction of cloze

probability and mood was found for the midline and the lateral sites.

For the midline sites an N400 was present for the positive mood

condition but absent for the sad condition. For the lateral sites the

interaction indicated that the N400 in the PMC was more broadly

distributed across the scalp than in the NMC.

Further analyses are still under way on the pseudohomophone

and syntactic condition.


42/46


43/46


governed by a set of segregated modules, independent from other

cognitive factors and must be separately studied in all its different

compartments.

This paradigm could be extended to other kinds of cognitive

integration tasks, such as the visual one.

Moreover, future ERP works could also address different facets

of semantic integration (e.g. at the verb level), or be integrated with

computational tools in order to compare and refine ontologies with

live data on how the brain seems to deal with language.


44/46


Bibliography

Chomsky, N., 1957. Syntactic Structures. Mouton & Co., The Hague/Paris.

Chomsky, N. (1995). The Minimalist Program. Cambridge, Mass: MIT Press.

Chwilla, D. J., Brown, C. M., & Hagoort, P. (1995). The N400 as a function of

the level of processing. Psychophysiology, 32(3), 274285.

Bever, T.G., Sanz, M., Townsend, D.J., 1998. The emperors psycholinguistics.

Journal of Psycholinguistic Research 27 (2), 261284.

Evans, V., Benjamin Bergen & Joerg Zinken, 2007. The Cognitive Linguistics

Reader. London: Equinox.

Federmeier, K. D., Kirson, D. A., Moreno, E. M., and Kutas, M. (2001). Effects

of transient, mild mood states on semantic memory organization and use: An

event-related potential investigation in humans. Neuroscience Letters,

305, 149-152.

Fodor, J., 2003. The modularity of mind. An essay on faculty psychology,

Cambridge, Mass: MIT Press.

Friederici, A., Bahlmann, J., Heim, S., Schubotz, R.I., Anwander, A., 2006. The

brain differentiates human and non-human grammars: Functional

localization and structural connectivity, in PNAS, 103 (7), 2458-2463.

Hagoort, P., Brown, C., Groothusen, J., 1993. The syntactic positive shift (SPS)

as an ERP measure of syntactic processing. In: Garnsey, S.M. (Ed.),


45/46


Language and Cognitive Processes. Special Issue: Event-Related Brain

Potentials in the Study of Language, vol. 8(4). Lawrence Erlbaum

Associates, Hove, pp. 439483Hagoort, P., Hald, L., Bastiaansen, M., Petersson, K.M., 2004. Integration of

word meaning and world knowledge in language comprehension. Science

304, 438441.

Kirson, D.A., 1990. The Effect of Mood on Access to Remote Category Associates ,

Department of Psychology, University of Denver.Kolk, H.H., Chwilla, D.J., van Herten, M., Oor, P.J., 2003. Structure and limited

capacity in verbal working memory: a study with event-related potentials .

Brain Lang. 85, 136.

Kolk, H.H., Chwilla, D.J., 2007. Late Positivities in unusual situations.

Brain Lang. 100, 257262.

Kutas, M., Hillyard, S.A., 1980. Reading senseless sentences: brain potentials

reflect semantic incongruity. Science 207, 203205.

Kutas, M., Hillyard, S.A., 1984. Brain potentials during reading reflect word

expectancy and semantic association. Nature 307, 161163.

Kutas, M., Van Petten, C.K.F., 1994. Psycholinguistics electrified. Event-related

brain potential investigations. In: Gernsbacher, M.A. (Ed.), Handbook of

Psycholinguistics. Academic Press, San Diego, CA, pp. 83143.

MacDonald, Maryellen C., Pearlmutter, Neal, and Seidenberg, Mark S., 1994.

Lexical Nature of Syntactic Ambiguity Resolution. Psychological Review

101:676-703.


46/46


Mazzone, M., in press. Autonomia del linguistico? Alcune tendenze delle

neuroscienze attuali, in Atti del II Convegno CODISCO

(Coordinamento Dottorati Italiani Scienze Cognitive).

Osterhout, L., Holcomb, P.J., 1992. Event-related potentials elicited by syntactic

anomaly. J. Mem. Lang. 31, 785806.

Slobin, D. I., 1966. Grammatical transformations and sentence comprehension in

childhood and adulthood. Journal of Verbal Learning and Verbal

Behavior, 5, 219-227.

Trueswell, J. C. & Tanenhaus, M. K., 1994. Toward a lexicalist framework for

constraint-based syntactic ambiguity resolution. In Clifton, C., Frazier, L.

and Rayner, K. (Eds.) Perspectives in Sentence Processing. Lawrence

Erlbaum Assoc.: Hillsdale, NJ, pp. 155-179.

Van Herten, M., Kolk, H.H.J., Chwilla, D.J., 2005. An ERP study of P600 effects

elicited by semantic anomalies. Cogn. Brain Res. 22, 241255.

Van Herten, M., Kolk, H. H. J., & Chwilla, D. (2005). An ERP study of

P600 effects elicited by semantic anomalies. Cognitive Brain Research, 22,

241255.

Vissers, C.T., Chwilla, D. J., & Kolk, H. H. J., 2006. Monitoring in Language

Perception: the effect of misspellings of words in highly constrained

sentences. Brain Research.

Vissers C.T., Kolk H.H., van de Meerendonk N., Chwilla D.J., 2008.

Monitoring in language perception: evidence from ERPs in a picture-

issues on language processing

Documents