PDP Models of Morphology

Psych 419/719

April 3, 2001

What is Morphology?

• Derives from MORPH, meaning “to change”

• Defines the rules of a language governing how words can be changed into new words.

Different Kinds of Morphology

• Inflectional– Plural (DOG->DOGS)– Past tense (BUG->BUGGED)– 3rd person singular (LIKE->LIKES)

• Derivational:– GOVERN->GOVERNMENT– POMPOUS->POMPOSITY

• Compound Words:– Houseboat, boathouse

• Infixing

The General Character...

Sound

Meaning

Cat

Meows, purrs

s+

plural+

Exceptions are Legion..

• Plural: MICE, MEN, GEESE, KNIVES• Past tense: RAN, LIT• Derivational: there’s GOVERN in

GOVERNMENT, but no DEPART in DEPARTMENT

• Compounds: A bookcase is a container for books, a suitcase is a container for suits, but a staircase isn’t really a container for stairs, a pocketbook isn’t a book, a bookworm isn’t a worm, etc.

Despite This, We Can Generalize

• WUGS are either more than one WUG, or the act of the verb WUG (e.g., “Bob wugs his car”)

• GLORPED is doing GLORP in the past

• ESTRANGEMENTALITY is the processes of ESTRANGEMENT

The Standard Account• Part of your knowledge of your language is

knowing the rules that govern such transformations

• Rules operate over standard linguistic units – Stems, affixes, suffixes, etc.

• Knowledge of rules is independent of knowledge of mapping from sound to meaning

• The exceptions must be memorized by a separate system

What PDP Networks Are Like

• They can memorize exceptions, but attend to statistical regularities as well.

• They’re good at such tasks, but not as good at partitioning training into qualitatively different categories, like rules and exceptions

The PDP Story

• You learn to associate word forms to their meanings

• Regularities are easy to learn– So you pick up on them,– And languages evolve to use them.

• “Morphology” is simply attending to these regularities

Non-Morphological Regularities• Onomatopoeia - where the sound of a word has

something to do with its meaning– whistle, whir, whip, whiz– boom, bang, clang

• Words with similar sounds mean similar things– glitter glisten, gleam– sparkle, sputter, sprite

• Common Latin root– include, exclude, preclude

.. Leads to Different Interpretations

• On the standard account, these kinds of regularities are not “morphological” and as such are not handled by the morphological system

• On the PDP account, they’re just another regularity for the sound to meaning system to learn. Less predictable, but still there.

Other Effects in Morphology

• Claim: for some items, you regularize the inflection if the meaning is far from the original meaning– The batter flied out to left field– Maple Leafs vs. Timber Wolves

• Sometimes the reverse:– The suspect has been held up in the apartment

for 6 hours (c.f. HOLED/HOLD)

U-Shaped Learning

• Initially, children seem to learn word forms whether they are regular or exception

• At some point, performance on exceptions drops.. Children regularize them (saying “eated” for “ate”)

• Ultimately, performance recovers

The Standard Account of U-Shaped learning

• Initially children are memorizing word forms.

• Then, they infer the rule for morphology

• This results in interference between memorized forms and rule-generated forms– Competition between two systems

• Eventually, rule is learned, exceptions are put in “exception box”

The PDP Account of U-Shaped Learning

• A single homogenous system is learning the task

• Interference between regular and irregular forms is not competition between two separate, atomic systems

• … but rather, results from the normal dynamics of learning in a single system

The Facts of U-Shaped Learning

• When you look closely at children’s performance, there isn’t a global “switch” from good to poor performance on all exceptions– As might be predicted by standard account

• Rather, there are micro-U trends by item over development

• And by the way, it doesn’t happen that often...

Micro-U Shaped Learning

• This falls out naturally from the PDP account.

• Not quite as straightforward for the standard account.

Review: Rumelhart & McClelland’s Past Tense Model

• Created a two layer network to map uninflected forms to inflections.– Initially, introduced high frequency (mostly

exception) words in training.– Then, switched to whole training set

• One system learned exceptions and regulars

• Demonstrated U-Shaped learning at a global scale

Criticisms of R&M Past Tense Model

• The frequency shift was bogus

• The representation was poor

• The errors it made were implausible

• Didn’t account for semantic effects

• The U-Shaped learning it modeled isn’t what actually happens

• Wrong theory: people aren’t just inflecting base forms when making the past tense in normal language

Another Try:Plunkett & Marchman ‘91

• Used more reasonable phonological representation

• Did not introduce explicit frequency shift

• Found that parameters for type/token frequency to induce best match to people was that of actual English

• … But, had very few items (500), and did not master vocabulary early in training

One More Time:Plunkett and Marchman ‘93

• Increased training set size gradually, one item at a time.

• Two conditions:– Add new item when existing ones mastered.

Result: Got Stuck– Add new item when certain amount of time

passed. Result: Much better

Stem To Inflected Word:Summary

• Models were able to reproduce U-Shaped learning with a high degree of fidelity to what children do

• Provided an account of effect of vocabulary size on interference and generalization that is absent in standard account

A New Attack:Marslen-Wilson and Tyler

• Looked at priming effects in lexical decision– You get a prime such as BAKE, then have to

make lexical decision on BAKER

• Crossed semantic overlap with phonological overlap

Marslen-Wilson’s Manipulation• Semantic, Phonological, Morphological related

– bake / baker

• Phonological but not semantic– corn / corner

• Semantic but not phonological– cook / baker

• Result: only morphological condition primed reliably

• Conclusion: Morphology is special

The Counter to Marslen-Wilson:Gonnerman ‘98

• Marslen-Wilson observed weak (ns) priming in other conditions

• Maybe just phonological or semantic isn’t enough; need both

• M-W’s semantic primes weren’t very closely related

• Should get graded priming if items sufficiently related, and enough subjects

Gonnerman’s Results

• If semantic relatedness is high, get priming even with no morphological relationship– Jubilee - jubilant, fork - spoon

• Priming effects are in fact graded: semantic overlap gives some priming, semantic and phonological gives more

• No effect of morphology independent of semantic and phonological overlap!

The Gonnerman & Devlin Model

• Maps forms onto meaning• Manipulated phonological

and semantic overlap• Results broadly replicated

that of empirical study

Word Form

Meaning

Brain Damage Can YieldDouble Dissociations

• Some patients exhibit an impairment in generating irregular past tenses – Say runned instead of ran

• While others have an impairment in generating novel past tenses– Can’t say glorped for the past tense of glorp

• Taken as evidence (Ullman and colleagues) for two systems in morphology

The Patients

• Those impaired on rules:– Patients with Parkinson’s disease, or left

inferior cortex (including Broca’s area)

• Those impaired on exceptions:– Patients with damage to Wernike’s area, either

from Alzheimer’s disease or lesion

The Two Accounts

• The “rules” are in the left inferior frontal area; Broca’s or the basal ganglia

• The “exceptions” are part of declarative memory, near Wernike’s area

• Phonological knowledge is needed more for rule-like items

• Semantic knowledge is needed more for exception processing

Standard Account PDP Account

The Joanisse & Seidenberg Model

• Four tasks:– Speaking

– Hearing

– Repeating

– Past tense formation

Semantics Speech in

Speech Out

Modeled rule impairmentwith phonological damage

And exception impairmentwith semantic damage

Results of Simulation

• Phonological damage impaired rule performance more than exceptions

• Semantic damage impaired exceptions more than rule performance

Another Angle: Why Do WeGeneralize At All?

• In English, the regulars are by far more common than exceptions. Easy to decide it’s the default.

• But: in German, there are several forms of the past tense. The “default” is low in frequency– But: Other forms are phonologically conditioned!

The Case of the Rats Eaters

• People can form compound nouns, by glueing two words together (rat eater, pig farmer, etc).

• It has been observed that people don’t like to use plurals as the head of a compound– rat eater (ok)– mice eater (ok)– rats eater (bad)

The Standard Account: The Level-Ordering Hypothesis

• Rules are applied at different levels of representation

• Each level doesn’t have access to input to previous levels– Irregular inflections stored in lexicon– Compounding applies to items in lexicon– Regular inflection happens after compounding

Problems With This Account

• People use regular compounds in plurals all the time– parks department, weapons inspector, pilots

union, communications industry, compounds research

• If compounds are generated by rule, acceptability shouldn’t be frequency sensitive– But it is

The PDP-Inspired Alternative

• There are cues to what is acceptable• Account hinges on what makes a good

modifier– Children learn that modifiers generally aren’t

semantically or phonologically plural– Consider adjectives: red balloons, not reds

balloons– Irregulars are semantically but not phonologically

plural, so not great, but better than pure plurals

Next Time: Reading and Dyslexia

• Optional reading on the class web page

• No class April 10