svm questions - seas.upenn.educis520/lectures/svm_questions.pdf · see if the svm is separable and...
TRANSCRIPT
![Page 1: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/1.jpg)
Lyle Ungar, University of Pennsylvania
SVM Questions Lyle Ungar
![Page 2: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/2.jpg)
Lyle H Ungar, University of Pennsylvania 2
Hyperplanes u Given the hyperplane defined by the line
l y = x1 - 2x2 l y = (1,-2)T x = wT x
u Is this point correctly predicted? 1) y = 1, x = (1,0) ? 2) y = 1, x = (1,1) ?
A. Yes B. No
![Page 3: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/3.jpg)
Lyle H Ungar, University of Pennsylvania 3
Hyperplanes u Given the hyperplane defined by the line
l y = x1 - 2x2 l y = (1,-2)T x
x1
x2
(1,-2) y > 0
y < 0 (1,0)
(1,1)
So (1,0) having y=1 is correct and (1,1) having y=1 is not correct
![Page 4: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/4.jpg)
Lyle H Ungar, University of Pennsylvania 4
Projections u The projection of a point x onto a line w is xT w/|w|2
u The distance of a point x to a hyperplane defined by the line w is
xT w/|w|2
Why?
![Page 5: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/5.jpg)
Lyle H Ungar, University of Pennsylvania 5
Hyperplanes u The projection of a point x on a plane defined by a
line w is xT w/|w|2
u The distance of x from the hyperplane defined by (1, -2) is what l for x = (-1,2) l for x = (1,0) l for x = (1,1)
x1
x2
(1,-2)
(1,0)
(1,1)
A) 1/√5 B) - 1/√5 C) 1/5 D) √5 E) None of the above
![Page 6: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/6.jpg)
Lyle H Ungar, University of Pennsylvania 6
Hyperplanes u The projection of a point x on a plane defined by a
line w is xT w/|w|2
u The distance of x from the hyperplane defined by (1, -2) is what l for x = (-1,2) √5 l for x = (1,0) 1/√5 l for x = (1,1) 1/√5
x1
x2
(1,-2)
(1,0)
(1,1)
project onto the line xT (1 -2)/√(12 + (-2) 2) = xT (1 -2)/√5
![Page 7: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/7.jpg)
Lyle H Ungar, University of Pennsylvania 7
Hyperplanes u True or False? When solving for a hyperplane
specified by , 0 one can always set the margin to 1;
![Page 8: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/8.jpg)
Lyle H Ungar, University of Pennsylvania 8
Hyperplanes u True or False? When solving for a hyperplane
specified by , 0 one can always set the margin to 1;
True (as long as it is separable)
![Page 9: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/9.jpg)
Lyle H Ungar, University of Pennsylvania 9
Hyperplanes u True or False? When solving for a hyperplane
specified by , 0 one can always set the margin to 1;
u True or False? This then implies that the margin, the distance of the support vectors from the separating hyperplane, is
T
![Page 10: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/10.jpg)
Lyle H Ungar, University of Pennsylvania 10
Hyperplanes u True or False? When solving for a hyperplane
specified by , 0 one can always set the margin to 1;
True u True or False? This then implies that the
margin, the distance of the support vectors from the separating hyperplane, is
True
T
![Page 11: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/11.jpg)
Lyle H Ungar, University of Pennsylvania 11
(non)separable SVMs u True or False? In a real problem, you should check to
see if the SVM is separable and then include slack variables if it is not separable.
u True or False? Linear SVMs have no hyperparameters that need to be set by cross-validation
![Page 12: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/12.jpg)
Lyle H Ungar, University of Pennsylvania 12
(non)separable SVMs u True or False? In a real problem, you should check to
see if the SVM is separable and then include slack variables if it is not separable.
False: you can just run the slack variable problem in either case (but you need to pick C) u True or False? Linear SVMs have no hyperparameters
that need to be set by cross-validation False: you need to pick C u True or False? Adding slack variables is equivalent to
requiring that all of the αi are less than a constant in the dual
![Page 13: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/13.jpg)
Lyle H Ungar, University of Pennsylvania 13
Non-separable SVMs u True or False? A support vector could be inside
the margin.
x3
w
![Page 14: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/14.jpg)
Lyle H Ungar, University of Pennsylvania 14
Non-separable SVMs u x2, x3 have y=1, while x4 has y = -1 u All are co-linear At what location does x3 become a support vector? True/False? It just needs to be closer to x4, than x2 is
w
x2
x4
x3
1
1
![Page 15: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/15.jpg)
Lyle H Ungar, University of Pennsylvania 15
Non-separable SVMs - dual
w
Consider the following cases 1) a point x1 is on the correct side of the margin 2) a point x2 is on the margin 3) a point x3 is on the wrong side of the margin 4) a point x4 is on the wrong side of the decision hyperplane A) αi = 0 B) αi < C C) αi = C
x2 x1
x3
x4
![Page 16: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/16.jpg)
Lyle H Ungar, University of Pennsylvania 16
Non-separable SVMs - dual
Consider the following cases 1) a point x1 is on the correct side of the margin αi = 0 nonbinding 2) a point x2 is on the margin αi < C 3) a point x3 is on the wrong side of the margin αi = C 4) a point x4 is on the wrong side of the decision hyperplane αi = C If a point is on the wrong side of the margin (cases 3 and 4), ξi > 0 and hence λi = 0 (last term of the first equation below; λi is the Lagrange multiplier for the ith slack variable ξi ), and hence αi = C If a point is on the margin (case 2), then ξi = 0, and λi can be greater than zero so in general αi < C.
![Page 17: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/17.jpg)
Lyle H Ungar, University of Pennsylvania
Why do SVMs work well?
u Why are SVMs fast?
u Why are SVMs often more accurate than logistic regression?
17
![Page 18: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/18.jpg)
Lyle H Ungar, University of Pennsylvania
Why do SVMs work well?
u Why are SVMs fast? l Quadratic optimization (convex!) l They work in the dual, with relatively few points l The kernel trick
u Why are SVMs often more accurate than logistic regression? l SVMs use kernels –but regression can, too l SVMs assume less about the model form
n Logistic regression uses all the data points, assuming a probabilistic model, while SVMs ignore the points that are clearly correct, and give less weight to ones that are wrong
18
![Page 19: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/19.jpg)
Lyle H Ungar, University of Pennsylvania 19
Hyperplanes u Given the hyperplane defined by the line
l y = x1 - 2x2 l y = (1,-2)T x = wT x
u What is the minimal adjustment to w to make a new point y = 1, x = (1,1) be correctly classified?
![Page 20: SVM questions - seas.upenn.educis520/lectures/SVM_questions.pdf · see if the SVM is separable and then include slack variables if it is not separable. False: you can just run the](https://reader034.vdocuments.site/reader034/viewer/2022042021/5e78b257172e6f49cd592c12/html5/thumbnails/20.jpg)
Lyle H Ungar, University of Pennsylvania 20
Hyperplanes u Given the hyperplane defined by the line
l y = (1,-2)T x = wT x u What is the minimal adjustment to w to make a
new point y = 1, x = (1,1) be correctly classified?
x1
x2
(1,-2) y > 0
y < 0
(1,1)
Make w=(1,-1) (1,-1)