NLP in the Real WorldStudent Open HouseSeptember 30, 2015 David MurgatroydVP, Engineering

@dmurga

1

Raise Your Hand If You...

➔Know what NLP stands for?➔Speak a language other than English?➔Have ever written a computer program?➔...that tries to process language?

2

A Little More About You...

➔Think of this room as a map of the world➔The TV is Iceland, the table is Antarctica➔Stand where you were born.➔Take a minute and ask the folks standing

near you where they were born.

3

A Little More About You...

➔Now stand where you most want to travel.➔Take a minute...

4

A Little More About You...

➔Now stand at the travel destination that’s a the top of your bucket list.

➔Take a minute…➔How many of you have used Airbnb?

5

A Little More About You...

➔Now stand at the travel destination that’s a the top of your bucket list.

➔Take a minute…➔How many of you have used Airbnb?➔How do you think NLP might help Airbnb?

6

How does NLP Help Airbnb?

Booking a place requires Sharing

Sharing requires Trust

Trust requires Verification

Verification requires NLP

Airbnb Added Government ID

Gov. ID

Airbnb: Scan a Government ID

First Name: J A N E

Last Name: C I Y I Z E N

Scanned Name

Match Scanned Name Against Profile

C I T I Z E N

First Name:

Last Name:

J A N I E

C I Y I Z E N

J A N E

Scanned Passport Name Profile Name

85%

Match Against Local Property Record

Scanned Passport Name

C I T I Z E N

Profile Name

First Name:

Last Name:

J A N I E

C I Y I Z E N

J A N E

시 티 즌

자 넷

Property Record

85% 90%

Lesson...

No one has a problem that involves just language.

So start with the user and work your way back.

12

7 Steps to Real-world NLP Happiness

1. Whose life are you trying to improve?

13

About us...

14

7 Steps to Real-world NLP Happiness

1. Whose life are you trying to improve?2. What precise problem do you want to solve?

15

Demo...

16

7 Steps to Real-world NLP Happiness

1. Whose life are you trying to improve?2. What precise problem do you want to solve?3. Do you have any data?

17

Match Against Local Property Record

Scanned Passport Name

C I T I Z E N

Profile Name

First Name:

Last Name:

J A N I E

C I Y I Z E N

J A N E

시 티 즌

자 넷

Property Record

Match Against Local Property Record

Scanned Passport Name

C I T I Z E N

Profile Name

First Name:

Last Name:

J A N I E

C I Y I Z E N

J A N E

시 티 즌

자 넷

Property Record

7 Steps to Real-world NLP Happiness

1. Whose life are you trying to improve?2. What precise problem do you want to solve?3. Do you have any data?

◆input data◆‘true’ output data

20

7 Steps to Real-world NLP Happiness

1. Whose life are you trying to improve?2. What precise problem do you want to solve?3. Do you have any data?4. How will you measure success?

21

How to measure success?

22

➔At the level of the user?◆Better reviews?◆More bookings per host/user?

How to measure success?

➔At the level of the user?◆Better reviews?◆More bookings per host/user?

➔At the level of the problem?◆Define accuracy?

23

TruthSystem

How to measure success?

➔At the level of the user?◆Better reviews?◆More bookings per host/user?

➔At the level of the problem?◆Define accuracy?

24

TruthSystem

True Positives

How to measure success?

➔At the level of the user?◆Better reviews?◆More bookings per host/user?

➔At the level of the problem?◆Define accuracy?

25

TruthSystem

FalsePositives

FalseNegatives

True Positives

How to measure success?

➔At the level of the user?◆Better reviews?◆More bookings per host/user?

➔At the level of the problem?◆Define accuracy?

26

TruthSystem

FalsePositives

FalseNegatives

True Positives

Metrics:

Precision = TPs / System

Recall = TPs / Truth

7 Steps to Real-world NLP Happiness

1. Whose life are you trying to improve?2. What precise problem do you want to solve?3. Do you have any data?4. How will you measure success?5. How fast does it need to be?

27

How fast?

28

➔User interface waiting on it?➔Some automated decision waiting on it?➔Can I get it done on the hardware I have?➔Metrics:

◆Latency: how long to process one thing.◆Throughput: how many things finish on

average in a period.

7 Steps to Real-world NLP Happiness

1. Whose life are you trying to improve?2. What precise problem do you want to solve?3. Do you have any data?4. How will you measure success?5. How fast does it need to be?6. How should it be integrated?

29

How should it be integrated?

30

➔Bring the computation to the data (Cluster)➔Bring the data to the computation (Cloud

APIs)➔Other complexities:

◆Containers (e.g., Docker)◆Programming interface (e.g., Java,

RESTful)◆Where to persist the data?◆Combine with existing systems?

7 Steps to Real-world NLP Happiness

1. Whose life are you trying to improve?2. What precise problem do you want to solve?3. Do you have any data?4. How will you measure success?5. How fast does it need to be?6. How should it be integrated?7. How do I make it better?

31

How to make it better?

32

➔Error analysis➔Adaptation

◆Overrides◆Configuration◆Field training◆Field paving◆Trading off speed for accuracy

➔Upgrades: Consistency v. Correctness

Questions

33

➔Doesn’t Deep Learning solve everything?➔What should I do in college to be well-

prepared for this real world?➔What distinguishes Basis from other places I

might think about for an internship?