machine learning (cse 446): introductionmachine learning (cse 446): introduction sham m kakade c...
TRANSCRIPT
![Page 1: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/1.jpg)
Machine Learning (CSE 446):Introduction
Sham M Kakadec© 2018
University of [email protected]
Jan 3, 2018
1 / 18
![Page 2: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/2.jpg)
Learning and Machine Learning?
I Broadly, what is “learning”?Wikipedia, “ Learning is the process of acquiring new or modifying existingknowledge, behaviors, skils, values, or preferences. Evidences that learning hasoccurred may be seen in changes in behavior from simle to complex.”
I What is “machine learning”?An AI centric viewpoint: ML is about getting computers to do the types of thingspeople are good at.
I How is it...I different from statistics?I different from AI?
(When people say “AI” they almost always mean “ML.”)
2 / 18
![Page 3: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/3.jpg)
What is ML about?
I Easy for a computer: (42384 ∗ 3421.82)1/3
I Easy for a child:I speech recognitionI object recognitionI question/answering (“what color is the sky?”)
I Computers are designed to execute mathematically precise computationalprimitives (and they have become much faster!).
I This class: The algorithmic and statistical thinking (and techniques) for how wetrain computers to get better at these more ’easy-for-human’ tasks.
3 / 18
![Page 4: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/4.jpg)
ML is starting to work....
I No longer just an academic pursuit...
I Almost “overnight” impacts to society:(threshold) improvements in performance translate into societal impact
4 / 18
![Page 5: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/5.jpg)
Today, ML is begin used for:
I Video and image processing
I Speech and language processing
I Search engines
I Robot control
I Medical and health analysis
I not just “AI-ish” problems:sensor networks, traffic navigation, medical imaging, computational biology,finance
5 / 18
![Page 6: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/6.jpg)
Is it Magic?
I “sort of, yes”: why is the future (and never-before-seen instances) predictablefrom the past?“inductive bias” is critical for learning.
I “in practice, no”: we will examine the algorithmic tools and statistical methodsappropriately.
I “responsibly, NO”: there are consequences and limitations.
6 / 18
![Page 7: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/7.jpg)
Course logistics
6 / 18
![Page 8: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/8.jpg)
Your Instructors
I Sham Kakade (instructor)Research interests:
I theory: rigorous algorithmic and statistical analysis of these methodsI practice: understanding how to advance the state of the art (robotics, music +comp.
vision, NLP)
I TAs:Kousuke Ariga, Benjamin Evans, Xingfan Huang, Sean Jaffe, Vardhman Mehta,Patrick Spieker, Jeannette Yu, Kaiyu Zheng.
7 / 18
![Page 9: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/9.jpg)
Info
Course website: https://courses.cs.washington.edu/courses/cse446/18wi/
Contact: [email protected]
Please only use this email for course related questions (unless privacy is needed).
Canvas: https://canvas.uw.edu/courses/1124156/discussion_topics
Office hours: TBA.
8 / 18
![Page 10: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/10.jpg)
Textbooks
I “A Course in Machine Learning”, Hal Daume.
I “Machine Learning: A Probabilistic Perspective”, Kevin Murphy.
9 / 18
![Page 11: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/11.jpg)
Outline of CSE 446
I Problem formulations: classification, regression
I Techniques: decision trees, nearest neighbors, perceptron, linear models,probabilistic models, neural networks, kernel methods, clustering
I “Meta-techniques”: ensembles, expectation-maximization
I Understanding ML: limits of learning, practical issues, bias & fairness
I Recurring themes: (stochastic) gradient descent, the “scope” of ML, overfitting
10 / 18
![Page 12: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/12.jpg)
Grading
I Assignments (40%)I 5 in totalI both mathematics pencil and paper, mostly programmingI Graded based on attempt and correctnessI Late policy: 33% off for (up to) one day late; 66% off for (up to) two days late; ...
I Midterm (20%)
I Final exam (40%)
I Caveat: Your grade may go up or down in extreme cases.(down) Failure to hand in all the HW, (up) very strong exam scores
I You MUST make the exam dates (unless you have an exception based on UWpolicies). Do not enroll in the course otherwise.
11 / 18
![Page 13: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/13.jpg)
“Can I Take The Class?”
I Short answer: if you are qualified and can register, yes
I Math prerequisites: probability, statistics, algorithms, and linear algebra background.I Programming prereqs: strong programmer (e.g. comfortable in python)
I We will move fast; lectures will focus on concepts and mathematics
I work hard, do the readings, etc...
12 / 18
![Page 14: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/14.jpg)
To-Do List
I Quiz section meetings start tomorrow. Bring your laptop! Python review
I Readings (do them, before the class)
I Academic integrity statement: on the course web page. ultimately, it is up to youto carry yourself with integrity.
I Gender and diversity statement (an acknowledgement): please try to actappropriately, knowing that.
13 / 18
![Page 15: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/15.jpg)
Integrity
I Academic integrity policy: on the course web page. ultimately, it is up to you tocarry yourself with integrity.
I Gender and diversity statement: (an acknowledgement) the current state is notbalanced in any reasonable way; please try to act appropriately. people cansurprise you...
14 / 18
![Page 16: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/16.jpg)
The Standard Learning Framework
14 / 18
![Page 17: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/17.jpg)
“Inductive” Supervised Machine Learning
I Training: a learning algorithm takes aset of example input-output pairs,{(x1, y1), . . . (xN , yN )}, and returns afunction f (the ’hypothesis’); the goalis for f(x) to recover the true label y,for each example, and on futureexamples
I Testing: we check how well f predictson a set of test examples,{(x′1, y′1), . . . (x′M , y′M )}, by measuringhow well f(x′) matches y.
(x, y)(x, y)(x, y)(xi, yi)learning
algorithm f
x
f(x)
training data
y
15 / 18
![Page 18: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/18.jpg)
Inputs and Output
I x can be pretty much anything we can representI To start, we’ll think of x as a vector (really, a “tuple”) of features, where each
feature φ(x) maps the instance into some set. Sometimes Φ(x) denotes the tuple(the “vector” of all the features).
I y can beI a real value (regression)I a label (classification)I an ordering (ranking)I a vector (multivariate regression)I a sequence/tree/graph (structured prediction)I . . .
16 / 18
![Page 19: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/19.jpg)
“Classification” Examples
I Predict an object in image:
I (structured prediction) Predict words from an audio signal:
I (structured prediction) predict a sentence from a sentence:
17 / 18
![Page 20: Machine Learning (CSE 446): IntroductionMachine Learning (CSE 446): Introduction Sham M Kakade c 2018 University of Washington cse446-staff@cs.washington.edu Jan 3, 2018 1/18](https://reader030.vdocuments.site/reader030/viewer/2022041021/5ed23704c75eb6475719e6eb/html5/thumbnails/20.jpg)
More Examples:
I Regression:Predict the depth of an object (e.g. a pedestrian) in an image.
I Ranking:What order of ads should be displayed?
18 / 18