lecture1_fsnlp

8/13/2019 Lecture1_FSNLP

1/49

Sangeeta

Foundations of Statistical Natural Language Processing

Introduction To Course


2/49

FSNLP - Introduction

Course Book

Foundations of Statistical Natural LanguageProcessing: Christopher D. Manning and

Hinrich Schtze


3/49


Computational Linguistics

The Study of computer systems for

understanding and generating naturallanguages

How sentences are generated

How people communicate to eachother


4/49


Why To Study NLP

Written Aids (Spelling Checker, Grammar Checker)

Speech Recognition

OCR and OLCR

Intelligent Information Retrieval

We will study some of the applications


5/49


Computational Linguistics

Rules

To distinguish well formed and Ill formed

utterancesAll Grammar Leak: people bend grammar

rules to meet their communication needs

Rationalist Approach

Common Patterns

Statistical NLP

Known as counting things

Empiricist approach


6/49


Rationalist Approach

1960-1985

Noam Chromsky

Chromskyan linguistics

A significant part of theknowledge inthe human mind is not derived by thesenses but is fixed in advance,presumably by genetic inheritance.

Key parts are hardwired in the brain


7/49


Rationalist Approach cont

Children run complex task such as natural

language with limited input

Hence rules are hardwired in brain during birth


8/49


Empiricist Approach

1920-1960, 198X-

Some cognitive abilities at birth

Some initial structure to prefer certain ways of

organizing and generalizing

not tabula rasa

General operations upon senses Patter

Recognition, Association and Generalization

Language structure can be understood by

general language model and statistical

processing on large amount of language use


9/49


Difference between both the approaches

The difference is not absolute

Differ in level of initial knowledge brain have


10/49


Rationalist approach: linguistic competence Knowledge of language structure in the mind of native-

speaker

Empiricist Approach: Linguistic performance Delivery of language by speaker

Affected by many factors, distracting/noise in the

environment, memory limitations etc.


11/49


Rationalist Approach: Categorical Either a sentence will follow rule or not

Empiricist Approach: Non-categorical Commonly occurring patterns

Finding probabilities whether a sentence is usual or

not


12/49


Rule based approach

Categorical view of language

Measures linguistic competences : Language

structure in mindof speaker

Either a sentence is correct/in-correct (using

rules)

Sometime its difficult for average humans being

Any answers why?


13/49


Questions that linguistic should

answer

What kind of things do people say?

What do these things say/sak/request aboutworld?


14/49


say?

Traditionally people used to describe competentgrammar

On the basis of competent grammar

grammatically correct and wrong sentences are

identified

Checks only syntaxof the sentence


15/49


say?

See only syntax (Rule based approach)

Colorless green ideas sleep

Valid as per syntax

No one uses such sentence


16/49


17/49


#Exercise

Identify which sentences are grammaticallycorrect

1. John I believe Sally said Bill believed Sue saw.

2. What did Sally whisper that she had

secretly read?

3. John wants very much for himself to win.

4. (Those are) the books you should read

before it becomes difficult to talk about.5. (Those are) the books you should read

before talking about becomes difficult.

6. Who did Jo think said John saw him?


18/49


#Exercise

Identify which sentences are grammaticallycorrect

1. John I believe Sally said Bill believed Sue saw.

2. What did Sally whisper that she had

secretly read?

3. John wants very much for himself to win.

4. (Those are) the books you should read

before it becomes difficult to talk about.5. (Those are) the books you should read

before talking about becomes difficult.

6. Who did Jo think said John saw him?


19/49


What kind of people say?

Changes in the language pattern

Words can change their meaning and part of

speech

Example;

While: Time

Take a whi le

While : Complementizer

While you were ou t

Although valid today, but was invalid before


20/49

FSNLP I t d ti


21/49




Blending of part of speech

Example: Near

Can be used as adjective or preposition

(simultaneously)

FSNLP I t d ti


22/49




Language change

Example: kind of / sort of

Kind and sort were basically noun

But over the period of time their meaning is

changed and ofis attached to them We are kind of hungry

We can not attach o f to any o ther noun

FSNLP I t d ti


23/49


say?

Example:

In add it ion to th is, she ins isted that

women were regarded as a dif ferent

existence from men unfair ly

This sentence in grammatically correct

This sentence can be expressed in better form i.e

convention

FSNLP Introduction


24/49


convention (How frequently people express theidea)

Convection changes gradually and can be

identified by measuring frequencies of the pattern

Empiricist approach

FSNLP Introduction


25/49


say?

Empiricist approach find common pattern

Simple sentences are clearly acceptable or

unacceptable

FSNLP Introduction


26/49


Non-Categorical

Meaning of words change gradually kind of / sort of

Does not behave as normal Noun +

Proposition pair

Example:

He is kind of hungry

He sort of understood whats going wrong

FSNLP Introduction


27/49


Probabilistic

The argument for a probabilistic approach tocognition is that we live in a world filled with

uncertainty and incomplete information.

Unseen events

Ambiguity

FSNLP Introduction


28/49


Disadvantages of Rule Based

Approach

Hand coding rule is time consuming

Performs poorly on natural occurring text, Not

scalable

Example:

Verb: swallow

Rule: Animate being as subject and a physical

object

I swallowed his story

The supernova swallowed the planet



29/49


#Exercise

Dis-advantages of Statistical Approach

Preparing database is a time consuming

Generalization is poor for small-size database



30/49

FSNLP Introduction

Rule-based VS Corpus-based:

Advantages

Rule-based

No need to prepare database

Reasoning processes are explainable andtraceable



31/49

FSNLP Introduction

Rule-based VS Corpus-based:

Advantages

Corpus-based Knowledge acquisition can beautomatically

achieved by the computer

Offers a good solution to ambiguity problems, by

identifying words that form group together commonly



32/49

FSNLP Introduction

Why NLP is difficult??



33/49

FSNLP Introduction

Why NLP is difficult

Ambiguities

Example: Our Company is t ra in ing wo rkers

What is the meaning of this sentence?


34/49



35/49

How many possible parses following

sentence will have?

List the sales of the produ cts produced in

1973 w ith the produc ts p roduced in 1972.

Any guess?

455 possible parses



36/49

Lexical resources (corpora)

Canadian Hansards

Bilingual corpus, parallel texts

WordNet

Electronic dictionary

Synset

Relations between words Meronymy (part-whole relations)


37/49



38/49

Zipfs laws

Principle of Least Effort : People try to minimizetheir work

In a conversation speaker tries to use mostgeneral words, and listener tries to hear mostrare words



39/49

Zipfs laws

Principle of Least Effort

Zipfs law (Language):

or1

krfr

f



40/49

Any Problem with the equation?


41/49



42/49

Mandelbrot: Approximation


43/49


44/49



45/49

The significance of power laws

Zipfs law also stands for randomly-generated text?



46/49

Applications of NLP

Machine Translation

Meaning in English

( /

six writings



47/49

Applications of NLP cont

Information Extraction E-mail:

Make automated calendar entry:

Hi, we have exam on 5thJan,

2014 at 12.00 PM.

Event : Eaxm

Date: 5-1-2014

Time: 12:PM



48/49

Applications of NLP contd

Unbeatable package of image quality

You really need to spend some quality time going

through all the settings before using the OlympusOM-D E-M1.



49/49

Applications of NLP contd

lecture1_fsnlp

Documents