XformationsXformations

Statistical Natural Language Processing Statistical Natural Language Processing Presented by : Charu JainPresented by : Charu Jain

MotivationMotivation

Language Learning, a Generalization Language Learning, a Generalization problemproblem

Bayes’ theorem : P(Bayes’ theorem : P(ΘΘ).P(D/ ).P(D/ ΘΘ))Size of the LexiconSize of the LexiconExtract higher level generalizations Extract higher level generalizations

from D,from D, hypothesize for unseen hypothesize for unseen lexical entries.lexical entries.

Generalizations are Transformational.Generalizations are Transformational.

ParsingParsing

Probabilistic parsing requires a lexicon that Probabilistic parsing requires a lexicon that specifies each word's syntactic preferences specifies each word's syntactic preferences in terms of probabilities. To estimate these in terms of probabilities. To estimate these probabilities for words that were poorly probabilities for words that were poorly observed during training, we may assume observed during training, we may assume the existence of arbitrarily powerful the existence of arbitrarily powerful transformations (also known to linguists as transformations (also known to linguists as lexical redundancy rules or metarules) that lexical redundancy rules or metarules) that can add, delete, retype or reorder the can add, delete, retype or reorder the argument and adjunct positions specified by argument and adjunct positions specified by a lexical entry.a lexical entry.

Transformation GraphTransformation Graph

PCFGs, Independence AssumptionPCFGs, Independence AssumptionVertices – Lexical EntriesVertices – Lexical EntriesArcs – Linguistically possible Arcs – Linguistically possible

transformations, single edits and transformations, single edits and generation of lexical entries (if from generation of lexical entries (if from Start).Start).

WeightsWeights

WeightsWeights

Exception WeightException WeightGeneralization WeightGeneralization WeightNarrower generalization WeightNarrower generalization WeightTransformationally related rules have Transformationally related rules have

related probabilities that rise and fall related probabilities that rise and fall toghether.toghether.