cs 445/545 machine learning winter, 2014
DESCRIPTION
CS 445/545 Machine Learning Winter, 2014. See syllabus at http://web.cecs.pdx.edu/~mm/ MachineLearningWinter2014 / Lecture slides will be posted on this website before each class. . Use of laptops, phones, etc. in class: - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/1.jpg)
CS 445/545 Machine Learning
Winter, 2014
See syllabus at http://web.cecs.pdx.edu/~mm/MachineLearningWinter2014/
Lecture slides will be posted on this website before each class.
![Page 2: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/2.jpg)
• Use of laptops, phones, etc. in class:
– Please don’t, unless you are using it to take notes or view class slides.
![Page 3: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/3.jpg)
What is machine learning?
• Textbook definitions of “machine learning”:
– Detecting patterns and regularities with a good and generalizable approximation (“model” or “hypothesis”)
– Execution of a computer program to optimize the parameters of the model using training data or past experience.
![Page 4: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/4.jpg)
Training Examples: Class 1 Training Examples: Class 2
Test example: Class = ?
![Page 5: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/5.jpg)
Training Examples: Class 1 Training Examples: Class 2
Test example: Class = ?
![Page 6: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/6.jpg)
Training Examples: Class 1 Training Examples: Class 2
Test example: Class = ?
![Page 7: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/7.jpg)
Training Examples: Class 1 Training Examples: Class 2
Test example: Class = ?
![Page 8: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/8.jpg)
Training Examples: Class 1 Training Examples: Class 2
Test example: Class = ?
![Page 9: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/9.jpg)
Training Examples: Class 1 Training Examples: Class 2
Test example: Class = ?
![Page 10: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/10.jpg)
Types of machine learning tasks
• Classification– Output is one of a number of classes (e.g., ‘A’)
• Regression– Output is a real value (e.g., ‘$35/share”)
![Page 11: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/11.jpg)
Types of Machine Learning Methods
• Supervised
– provide explicit training examples with correct answers• e.g. neural networks with back-propagation
• Unsupervised
– no feedback information is provided• e.g., unsupervised clustering based on similarity
![Page 12: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/12.jpg)
• “Semi-supervised”
– some feedback information is provided but it is not detailed
• e.g., only a fraction of examples are labeled
• e.g., reinforcement learning: reinforcement single is single-valued assessment of current state
![Page 13: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/13.jpg)
Relation between “artificial intelligence” and “machine learning”?
![Page 14: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/14.jpg)
Key Ingredients for Any Machine Learning Method
• Features (or “attributes”)
![Page 15: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/15.jpg)
![Page 16: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/16.jpg)
From: "TopShop" <[email protected]> To: <[email protected]> Subject: Dear mm get 77% offList-Unsubscribe: <mailto:[email protected]>MIME-Version: 1.0Content-Type: text/plain; charset=ISO-8859-1; format=flowedContent-Transfer-Encoding: 7bit
Wassup mm, how are you?!Hurry up, last day huge discount...
Our shop have more than 3085 pisl online.Buy more, pay less!
Today TopSellers:
** Proepcia - 0.12$** Levitra - 1.64$** Ciails - 1.78$** Vaigra - 0.79$
Purchase here http://TvBqeUP.ngadoctor.ru/
![Page 17: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/17.jpg)
![Page 18: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/18.jpg)
![Page 19: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/19.jpg)
http://www.kaggle.com/c/dogs-vs-cats
![Page 20: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/20.jpg)
Data: Instances and Features
Instance: Individual example of a particular category or class
56 instances of the class “A” 6 instances of the class “B”
![Page 21: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/21.jpg)
Data: Instances and Features
Features: Collection of attributes of a single instance
Feature Vector: N-dimensional vector describing a single instance
0 0 0 0 0 0 0 00 1 1 1 1 1 0 00 0 0 0 0 1 0 00 0 0 0 1 0 0 00 0 0 1 0 0 0 00 0 1 1 0 0 0 00 0 1 0 0 0 0 00 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 1 1 1 ...
![Page 22: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/22.jpg)
Another example of computing features32 x 32
![Page 23: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/23.jpg)
Another example of computing featuresf1 f2 f3 f4 f5 f6 f7 f8
f9
x = (f1, f2, ..., f64)
= (0, 2, 13, 16, 16, 16, 2, 0, 0, ...) Etc. ..
![Page 24: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/24.jpg)
Notation for Instances and FeaturesInstance: x (boldface vector)
Set of M instances: {x1, x2, ... , xM}
Instance as feature vector, with N features: x = (x1, x2, ..., xN)
Instance as a point in an N-dimensional space:
x
x1
x2
x3
![Page 25: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/25.jpg)
Key Ingredients for Any Machine Learning Method
• Features (or “attributes”)
• Underlying Representation for “hypothesis”, “model”, or “target function”:
![Page 26: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/26.jpg)
Key Ingredients for Any Machine Learning Method
• Features (or “attributes”)
• Underlying Representation for “hypothesis”, “model”, or “target function”:
• Hypothesis space
![Page 27: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/27.jpg)
• Learning method
![Page 28: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/28.jpg)
• Learning method
• Data:
![Page 29: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/29.jpg)
• Learning method
• Data: – Training data
• Used to train the model
![Page 30: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/30.jpg)
• Learning method
• Data:– Training data
• Used to train the model
– Validation data • Used to select model complexity, to determine when to stop
training, or to alter training method
![Page 31: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/31.jpg)
• Learning method
• Data: – Training data
• Used to train the model
– Validation data • Used to select model complexity, to determine when to stop
training, or to alter training method
– Test data• Used to evaluate trained model
![Page 32: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/32.jpg)
• Learning method
• Data: – Training data
• Used to train the model
– Validation data • Used to select model complexity, to determine when to stop
training, or to alter training method
– Test data• Used to evaluate trained model
• Evaluation method
![Page 33: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/33.jpg)
Assumption of all ML methods:
Inductive learning hypothesis:
Any hypothesis that approximates target concept well over sufficiently large set of training examples will also approximate the concept well over other examples outside of the training set.
Difference between “induction” and “deduction”?
![Page 34: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/34.jpg)
This class
• Perceptrons• Evaluating Classifiers• Neural Networks• Support Vector Machines• Ensemble Learning• Bayesian Learning• Hidden Markov Models• Unsupervised Learning• Hierarchical Networks and “Deep Learning”• Feature Selection• Evolutionary Learning
![Page 35: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/35.jpg)
Class Logistics
• My office hours– Tu, Th. 2-3 or by appt.
• Mailing list– [email protected]
• Readings / Slides– On-line
• Homework– Four assignments– Late HW policy
• Weekly Quizzes (closed book, etc.)– 30 minutes, on Wednesdays
• Final Exam (open notes, open everything)
• Grading: – Homework 50%– In-Class Quizzes 30%– Final exam 20%
• Website:
http://www.cs.pdx.edu/~mm/MachineLearningWinter2014
![Page 36: CS 445/545 Machine Learning Winter, 2014](https://reader035.vdocuments.site/reader035/viewer/2022081520/5681622e550346895dd25bb2/html5/thumbnails/36.jpg)
Don’t forget to ask questions!