OutlineNaive BayesLaplace smoothingEvent models

comments on applied ML

kernel Methods

Recap

XEaby

X I word appears on email

Generative Model

PCxly p yp Hy ITp xilyParameters P y D yP X L I y 0 Oily 0P X II y L lo ly L

Max likelihood estimates

y I I y 13T

lo'sly o Eh I xj L y o

I gil 03l l

s ly I

At prediction time

peg Ha pcxty D pcy.snopCx1yD.PCy D PCX1y o3 PCy o 0

Near IPSj 5500

P 45500 1 9 1g0 05500151

P X5500 11 y 0 g0 05500 ly O

paly D IIEpcxjly.IM 9Tooly1PCx1yo jFI pcxs1y o o55ooly L

Laplace smoothing2009

412 WakeforestWoon

1010 OSU O10117 Arizona O11 21 Caltech O12 31 Oklahoma 0

PEX L IIst o's 2

L I0 4 60 Ig

Laplace Smoothing has IO's c I

More generally XEEL K

Estimate 2 xin j I

PCx j I

Qjly o IE IE L g 03 I P Xg L g o

I y 0 t 2 pasty o

Xi C I Kthx 019 4 1

SIZE 4010feet 4002800

80031200

71200X 4

PCxly ftp.pcx.lyMultinomial Cvs bernoulli

Is P X 1 P D L

PCXg4y o tPCXg oly o I

Ig1 bank

beneficiary

XiE 0,1

account bank account800 1600 800

E IRD di lengthof email i

Xj C I 10,0003

So far Multivariate Bernoulli event modelNew Multinomial event model

Generative model

PCx y paly peg

paly II P x Lg

Parameters

y My L

4k1g o P X k ly o

Chanceofwordj bang k f g 0

Assume that this does not depend on j10Kly I P X k ly L

MLE01k1g so 2 I yakoy 7 I x Ky L

E I y o de 10,000

Laplacesmoothing

Xj of jeh worden emailptosetcon in the dictionary

P

map fare wordstoUNLa

mortgage mafrtgageIUNK

0k1g I P yahword k ly L

independent of j

PCxly II p x lysq

PCH1197 PCH 01g Pas Ily

PCHg III palslyk 4 sum

linear f tquadratic fcubicfh of data

ho X Oo 10 X t 02 4 03 3

01cmng 4 IR IN

holm Coo O 0297beg

tote

sea y X yI

Cx y Colin y

Recall linear regression0 0

opoi o d.IECy otx x

t tRd scalar IRd

New dataset0 0Loop

oi q x.E.cya q EY.MEIRP IRPscalar

4 Rd IRP d L p 4 in example

Terminologyfeaturemap

0cal Chew featurere attributes

kernel methods

d L X Xi XD

Glx ft

OT a can represent anyxD Jd degree3 poly in Xi Xd

1es

lid3XdXdXd

time per iteration ocnp pndImprove this to 0cm per iteration

Even storing 0 takes time Ocp

p n