What Is the “Context” forContextual Vocabulary Acquisition?

William J. RapaportDepartment of Computer Science & Engineering

Department of Philosophy

Center for Cognitive Science

NSF ROLE Grant REC-0106338

Outline

• People can figure out a meaning for a word “from context”

• What does “context” mean in this context?

Definition of “CVA”

“Contextual Vocabulary Acquisition” =def

• the acquisition of word meanings from text– “incidental”

– “deliberate”

• by reasoning about– contextual cues

– background knowledge• Including prior word-meaning hypotheses, language knowledge…

• without external sources of help– no dictionaries

– no people

CVA: From Algorithm to Curriculum

1. Computational theory of CVA– Based on:

• algorithms developed by Karen Ehrlich (1995)• verbal protocols (case studies)

– Implemented in a semantic-network-basedknowledge-representation & reasoning system• SNePS (Stuart C. Shapiro & colleagues)

2. Educational curriculum to teach CVA– Based on our algorithms & protocols– To improve vocabulary & reading comprehension– Joint work with Michael Kibby

• Center for Literacy & Reading Instruction

People Do “Incidental” CVA

• We know more words than explicitly taught– Average high-school grad knows ~45K words learned ~2.5K words/year (over 18 yrs.)– But only taught ~400/school-year

• ~ 4800 in 12 years of school (~ 10% of total)

Most word meanings learned from context– “incidentally” (unconsciously)

• How?

People Also Do “Deliberate” CVA

• You’re reading;• You understand everything you read, until…• You come across a new word• Not in dictionary• No one to ask• So, you try to “figure out” its meaning from “context”• How?

– guess? derive? infer? deduce? educe? construct? predict? …

– our answer: Compute it! Via inferential search of “context”/KB

• But what KB?

CVA as Cognitive Science

• Studied in:– AI / computational linguistics– Psychology– Child-language development (L1 acquisition)– L2 acquisition (e.g., ESL)– Reading education (vocabulary development)

• Thus far: “multi-”disciplinary• Not yet: “inter-”disciplinary!

What does ‘brachet’ mean?

(From Malory’s 15th century Morte d’Arthur [page # in brackets])

1. There came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them. [66]

• People: brachet = animal? inanimate object? don’t know.

• Computer: brachet = physical object• (because only physical objects have color)

2. As the hart went by the sideboard, the white brachet bit him. [66]

• People: brachet = animal

• Computer: brachet = animal• (because only animals bite)

Malory, continued

3. The knight arose, took up the brachet and rode away with the brachet. [66]

• People: brachet = animal / small animal

• Computer: brachet = small animal• (because: picked up and carried)

4. A lady came in and cried aloud to King Arthur, “Sire, the brachet is mine”. [66]

• People: brachet = pet / small, valuable animal

• Computer: brachet = small, valuable animal• (because: what’s wanted is valuable)

Malory, continued

10. There was the white brachet which bayed at him fast. [72]

• People: brachet = dog

• Computer: brachet = hound (i.e., dog that hunts)• (because only hounds, which are hunting dogs, bay)

18. The hart lay dead; a brachet was biting on his throat, and other hounds came behind. [86]

• People: brachet = hound

• Computer: brachet = hound (i.e., dog that hunts)• (because “x and other y” x is a y)

How (Not) to Teach CVA:Vague Strategies

• Clarke & Nation 1980: a “strategy” (algorithm)1.Look at word & context; determine POS

2.Look at grammatical context• E.g., “who does what to whom”?

3.Look at wider context• [E.g., for clues re: causal, temporal, class-membership, etc.]

4.Guess the word; check your guess

Vague strategies:

• “guess the word”

=

“then a miracle occurs”

• Surely,

we computer scientists

can “be more explicit”!

A More Precise, Teachable Algorithm

• Treat “guess” as a procedure call– Fill in the details with our algorithm

• Convert the algorithm into a curriculum– To enhance students’ abilities to use deliberate

CVA strategies

Figure out meaning of word from what?

• context (i.e., the text)?– Werner & Kaplan 52, McKeown 85, Schatz & Baldwin 86

• context and reader’s background knowledge?– Granger 77, Sternberg 83, Hastings 94

• context including background knowledge?– Nation & Coady 88, Graesser & Bower 90

• Note:– “context” = text context is external to reader’s mind

• Could also be spoken/visual/situative (still external)

– “background knowledge”: internal to reader’s mind

• What is (or should be) the “context” for CVA?

Some Proposed Preliminary Definitions(to extract order out of confusion)

• Unknown word for a reader =def

– Word or phrase that reader has never seen before– Or only has vague idea of its meaning

• Different levels of knowing meaning of word

– Notation: “X”

Proposed preliminary definitions

• Text =def

– (written) passage – containing X– single phrase or sentence … several paragraphs

Proposed preliminary definitions

• Co-text of X in some text =def

– The entire text “minus” X; i.e., entire text surrounding X

– E.g., if X = ‘brachet’, and text =• “There came a white hart running into the hall with a white

brachet next to him, and thirty couples of black hounds came running after them.”

Then X’s co-text in this text =• “There came a white hart running into the hall with a white

______ next to him, and thirty couples of black hounds came running after them.”

– Cf. “cloze” tests in psychology• But, in CVA, reader seeks meaning or definition

– NOT a missing word or synonym: There’s no “correct” answer!

– “Co-text” is what many mean by “context”• BUT: they shouldn’t!

Proposed preliminary definitions

• The reader’s prior knowledge =def – the knowledge that the reader has when s/he

begins to read the text– and is able to recall as needed while reading

• “knight picks up & carries brachet” ? small

• Warnings:– “knowledge” truth

• so, “prior beliefs” is better

– “prior” vs. “background” vs. “world”, etc.• See next slide!

Proposed preliminary definitions

• Possible synonyms for “prior knowledge”, each with different connotation:

– Background knowledge:• Can use for information that author assumes reader to have

– World knowledge:• General factual knowledge about things other than the text’s topic

– Domain knowledge:• Specialized, subject-specific knowledge about the text’s topic

– Commonsense knowledge:• Knowledge “everyone” has

– E.g., CYC, “cultural literacy” (Hirsch)

• These overlap:– PK should include some CSK, might include some DK– BK might include much DK

Steps towards a

Proper Definition of “Context”• Step 1:

– The context of X for a reader =def

1. The co-text of X

2. “+” the reader’s prior knowledge

• Both are needed!– After reading:

• “the white brachet bit the hart in the buttock”

most subjects infer that brachets are (probably) animals, from:

• That text, plus:

• Available PK premise: “If x bites y, then x is (probably) an animal.

– Inference is not an enthymeme! (because …)

Proper definition of “context”:

• But (inference not an enthymeme because):– When you read, you “internalize” the text

• You “bring it into” your mind

• Gärdenfors 1997, 1999; Jackendoff 2002

– This “internalized text” is more important than the actual words on paper:• Text: “I’m going to put the cat out”

• Misread as: “I’m going to put the car out” – leads to different understanding of “the text”

– What matters is what the reader thinks the text is,• Not what the text actually is

• Therefore …

Proper definition of “context”:

• Step 2:

– The context of X for a reader =def

• A single KB, consisting of:

1. The reader’s internalized co-text of X

2. “+” the reader’s prior knowledge

Proper definition of “context”:

• But: What is “+”?– Not: mere conjunction or union!

– Active readers make inferences while reading.• From text = “a white brachet”

& prior commonsense knowledge = “only physical objects have color”,

reader might infer that brachets are physical objects

• From “The knight took up the brachet and rode away with the brachet.”

& prior commonsense knowledge about size,

reader might infer that brachet is small enough to be carried

– Whole > Σ parts:• inference from [internalized text + PK] new info not in text or in PK

• I.e., you can learn from reading!

Proper definition of “context”:

• But: Whole < Σ parts!– Reader can learn that some prior beliefs were mistaken

• Or: reader can decide that text is mistaken (less likely)

• Reading & CVA need belief revision!

• operation “+”: – input: PK & internalized co-text– output: “belief-revised integration” of input, via:

• Expansion:– addition of new beliefs from ICT into PK, plus new inferences

• Revision:– retraction of inconsistent prior beliefs together with inferences from them

• Consolidation:– eliminate further inconsistencies

Prior Knowledge Text

PK1

PK2

PK3

PK4

Prior Knowledge Text

PK1

PK2

PK3

PK4

T1

Integrated KB Text

PK1

PK2

PK3

PK4

T1

I(T1)

internalization

B-R Integrated KB Text

PK1

PK2

PK3

PK4

T1

I(T1)

internalization

P5

inference

B-R Integrated KB Text

PK1

PK2

PK3

PK4

T1

I(T1)

internalization

P5

inference

T2

I(T2)

P6

B-R Integrated KB Text

PK1

PK2

PK3

PK4

T1

I(T1)

internalization

P5

inference

T2

I(T2)

P6

T3

I(T3)

B-R Integrated KB Text

PK1

PK2

PK3

PK4

T1

I(T1)

internalization

P5

inference

T2

I(T2)

P6

T3

I(T3)

B-R Integrated KB Text

PK1

PK2

PK3

PK4

T1

I(T1)

internalization

P5

inference

T2

I(T2)

P6

T3

I(T3)

P7

Note: All “contextual” reasoning is done in this “context”:

Proper definition of “context”:

• One more detail: X needs to be internalized• Context is a 3-place relation among:

– Reader, word, and text

• Final(?) def.:

– Let T be a text

– Let R be a reader of T

– Let X be a word in T (that is unknown to R)

– Let T-X be X’s co-text in T.

– Then:• The context that R should use to hypothesize a meaning

for R’s internalization of X as it occurs in T =def

– The belief-revised integration of R’s prior knowledge with R’s internalization of T-X.

This definition agrees with…

• Cognitive-science & reading-theoretic views of text understanding– Schank 1982, Rumelhart 1985, etc.

• & KRR techniques for text understanding:– Reader’s mind modeled by KB of prior knowledge

• Expressed in KR language (for us: SNePS)

– Computational cognitive agent reads the text,• “integrating” text info into its KB, and• making inferences & performing belief revision along the way

– When asked to define a word,• Agent deductively searches this single, integrated KB for

information to fill slots of a definition frame

– Agent’s “context” for CVA = this single, integrated KB

Distinguishing Prior Knowledge from Integrated Co-Text

• So KB can be “disentangled” as needed for belief revision or to control inference:

• Each proposition in the single, integrated KB is marked with its “source”:– Originally from PK– Originally from text– Inferred

• Sources of premises

Some Open Questions

• Roles of spoken/visual/situative contexts• Relation of CVA “context” to formal theories of

context (e.g., McCarthy, Guha…)

• Relation of I(T) to prior-KB; e.g.:– Is I(Ti) true in prior-KB?

• It is “accepted pro tem”.

– Is I(T) a “subcontext” of pKB or B-R KB?

• How to “activate” relevant prior knowledge.• Etc.

Summary

• People can figure out a meaning for a word “from context”, where…

• “Context” = belief-revised integration of:– reader’s prior knowledge, with – internalized information from the text

• This clearer concept of relevant notion of “context” will help us:– evaluate other research– develop our curriculum