exam preparation and hw7 - learning & adaptive systems group · introduction to machine...

Exam Preparation and HW7Introduction to Machine Learning 2020

Julian Mäder

Schedule

- Exam 2019, Question 1- Exam 2019, Question 2- HW7 Questions 13, 14 and 15

Exam 2019, Question 1

Exam 2019, Question 1.1

Exam 2019, Question 1.1

Exam 2019, Question 1.1

Exam 2019, Question 1.1

Exam 2019, Question 1.2

Exam 2019, Question 1.2

Exam 2019, Question 1.2

First we reformulate the maximum likelihood estimate:

Exam 2019, Question 1.2

Next we look at our assumptions about the data:

Exam 2019, Question 1.2

Exam 2019, Question 1.2So let’s compare the maximum likelihood estimate to the weighted empirical risk:

Exam 2019, Question 1.3

Exam 2019, Question 1.3

...because is not differentiable!

Exam 2019, Question 1.4

Exam 2019, Question 1.5

Recap Kernels

Perceptron:

Recap Kernels

Exam 2019, Question 2.1

⟶ See Kernel Nearest-Neighbor Algorithm, Yu et al. 2002⟶ Lecture Slides: Dimensionality Reduction ��, slides 6 - 13⟶ Lecture Slides: Kernels ��, slides 34 - 37

⟶ Lecture Slides: Dimensionality Reduction ��, slides 6 - 12

Exam 2019, Question 2.2

Exam 2019, Question 2.2

Exam 2019, Question 2.2

Kernel Definition

Kernel Definition

Kernel Rules

Exam 2019, Question 2.3

Exam 2019, Question 2.3

Because c needs to be bigger than Zero!

Exam 2019, Question 2.3

Exam 2019, Question 2.4

Exam 2019, Question 2.4

Exam 2019, Question 2.4

Exam 2019, Question 2.4

Exam 2019, Question 2.5

Exam 2019, Question 2.5

Exam 2019, Question 2.6

Expectation of a (discrete) Random Variable

HW7 Question 13-15: Important Tipps

HW7 Question 13-15: Important Tipps

HW7 Question 13

HW7 Question 13

* Expectation ** Jensen’s Inequality

HW7 Question 14

HW7 Question 14

* Expectation ** Jensen’s Inequality

HW7 Question 14

HW7 Question 15

There is a more detailed explanation in the CS229 lecture notes (Part IX, The EM Algorithm) by Andrew Ng:(https://course.ccs.neu.edu/cs6220f16/sec3/assets/pdf/cs229-notes8.pdf)