exam preparation and hw7 - learning & adaptive systems group · introduction to machine...
TRANSCRIPT
![Page 1: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/1.jpg)
Exam Preparation and HW7Introduction to Machine Learning 2020
Julian Mäder
![Page 2: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/2.jpg)
Schedule
- Exam 2019, Question 1- Exam 2019, Question 2- HW7 Questions 13, 14 and 15
![Page 3: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/3.jpg)
Exam 2019, Question 1
![Page 4: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/4.jpg)
Exam 2019, Question 1.1
![Page 5: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/5.jpg)
Exam 2019, Question 1.1
![Page 6: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/6.jpg)
Exam 2019, Question 1.1
![Page 7: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/7.jpg)
Exam 2019, Question 1.1
![Page 8: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/8.jpg)
Exam 2019, Question 1.2
![Page 9: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/9.jpg)
Exam 2019, Question 1.2
![Page 10: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/10.jpg)
Exam 2019, Question 1.2
First we reformulate the maximum likelihood estimate:
![Page 11: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/11.jpg)
Exam 2019, Question 1.2
Next we look at our assumptions about the data:
![Page 12: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/12.jpg)
Exam 2019, Question 1.2
![Page 13: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/13.jpg)
Exam 2019, Question 1.2So let’s compare the maximum likelihood estimate to the weighted empirical risk:
![Page 14: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/14.jpg)
Exam 2019, Question 1.3
![Page 15: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/15.jpg)
Exam 2019, Question 1.3
...because is not differentiable!
![Page 16: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/16.jpg)
Exam 2019, Question 1.4
![Page 17: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/17.jpg)
Exam 2019, Question 1.5
![Page 18: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/18.jpg)
Recap Kernels
Perceptron:
![Page 19: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/19.jpg)
![Page 20: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/20.jpg)
Recap Kernels
![Page 21: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/21.jpg)
![Page 22: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/22.jpg)
Exam 2019, Question 2.1
⟶ See Kernel Nearest-Neighbor Algorithm, Yu et al. 2002⟶ Lecture Slides: Dimensionality Reduction ��, slides 6 - 13⟶ Lecture Slides: Kernels ��, slides 34 - 37
⟶ Lecture Slides: Dimensionality Reduction ��, slides 6 - 12
![Page 23: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/23.jpg)
Exam 2019, Question 2.2
![Page 24: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/24.jpg)
Exam 2019, Question 2.2
![Page 25: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/25.jpg)
Exam 2019, Question 2.2
![Page 26: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/26.jpg)
Kernel Definition
![Page 27: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/27.jpg)
Kernel Definition
![Page 28: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/28.jpg)
Kernel Rules
![Page 29: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/29.jpg)
Exam 2019, Question 2.3
![Page 30: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/30.jpg)
Exam 2019, Question 2.3
Because c needs to be bigger than Zero!
![Page 31: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/31.jpg)
Exam 2019, Question 2.3
![Page 32: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/32.jpg)
Exam 2019, Question 2.4
![Page 33: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/33.jpg)
Exam 2019, Question 2.4
![Page 34: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/34.jpg)
Exam 2019, Question 2.4
![Page 35: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/35.jpg)
Exam 2019, Question 2.4
![Page 36: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/36.jpg)
Exam 2019, Question 2.5
![Page 37: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/37.jpg)
Exam 2019, Question 2.5
![Page 38: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/38.jpg)
Exam 2019, Question 2.6
![Page 39: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/39.jpg)
![Page 40: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/40.jpg)
Expectation of a (discrete) Random Variable
HW7 Question 13-15: Important Tipps
![Page 41: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/41.jpg)
HW7 Question 13-15: Important Tipps
![Page 42: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/42.jpg)
HW7 Question 13
![Page 43: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/43.jpg)
HW7 Question 13
* Expectation ** Jensen’s Inequality
![Page 44: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/44.jpg)
HW7 Question 14
![Page 45: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/45.jpg)
HW7 Question 14
* Expectation ** Jensen’s Inequality
![Page 46: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/46.jpg)
HW7 Question 14
![Page 47: Exam Preparation and HW7 - Learning & Adaptive Systems Group · Introduction to Machine Learning 2020 Julian Mäder. Schedule - Exam 2019, Question 1 - Exam 2019, Question 2 - HW7](https://reader033.vdocuments.site/reader033/viewer/2022060606/605c77c539fba31a8638401f/html5/thumbnails/47.jpg)
HW7 Question 15
There is a more detailed explanation in the CS229 lecture notes (Part IX, The EM Algorithm) by Andrew Ng:(https://course.ccs.neu.edu/cs6220f16/sec3/assets/pdf/cs229-notes8.pdf)