cpsc 540 machine learning
DESCRIPTION
CPSC 540 Machine Learning. Nando de Freitas. http://www.cs.ubc.ca/~nando/540-2007. Acknowledgement. Many thanks to the following people for making available some of the slides, figures and videos used in these slides: Kevin Murphy (UBC) Kevin Leyton-Brown (UBC) Tom Griffiths (Berkeley) - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/1.jpg)
CPSC 540 Machine Learning
Nando de Freitas
http://www.cs.ubc.ca/~nando/540-2007
![Page 2: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/2.jpg)
Acknowledgement
Many thanks to the following people for making available some of the slides, figures and videos used in these slides:
• Kevin Murphy (UBC)
• Kevin Leyton-Brown (UBC)
• Tom Griffiths (Berkeley)
• Josh Tenenbaum (MIT)
• Kobus Barnard (Arizona)
• All my awesome students at UBC
![Page 3: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/3.jpg)
Introduction to machine learning• What is machine learning?• How is machine learning related to other fields?• Machine learning applications• Types of learning
– Supervised learning • regression • classification
– Unsupervised learning • clustering• data association• abnormality detection• dimensionality reduction• structure learning
– Semi-supervised learning– Active learning – Reinforcement learning and control of partially observed Markov decision
processes.
![Page 4: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/4.jpg)
What is machine learning?``Learning denotes changes in the system that are adaptive in the sense
that they enable the system to do the task or tasks drawn from the same population more efficiently and more effectively the next time.'' -- Herbert Simon
WORLD
ActionPercept
AGENT
![Page 5: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/5.jpg)
Learning concepts and words“tufa”
“tufa”
“tufa”
Can you pick out the tufas? Source: Josh Tenenbaum
![Page 6: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/6.jpg)
Why “Learn” ?Learning is used when:
– Human expertise is absent (navigating on Mars)
– Humans are unable to explain their expertise (speech recognition, vision, language)
– Solution changes in time (routing on a computer network)
– Solution needs to be adapted to particular cases (user biometrics)
– The problem size is to vast for our limited reasoning capabilities (calculating webpage ranks)
![Page 7: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/7.jpg)
Introduction to machine learning• What is machine learning?• How is machine learning related to other fields?• Machine learning applications• Types of learning
– Supervised learning • regression • classification
– Unsupervised learning • clustering• data association• abnormality detection• dimensionality reduction• structure learning
– Semi-supervised learning– Active learning – Reinforcement learning and control of partially observed Markov decision
processes.
![Page 8: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/8.jpg)
How is machine learning related to other fields?
ML
CS Statistics
Electricalengineering
Neuroscience
Psychology Philosophy
![Page 9: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/9.jpg)
Introduction to machine learning• What is machine learning?• How is machine learning related to other fields?• Machine learning applications• Types of learning
– Supervised learning • regression • classification
– Unsupervised learning • clustering• data association• abnormality detection• dimensionality reduction• structure learning
– Semi-supervised learning– Active learning – Reinforcement learning and control of partially observed Markov decision
processes.
![Page 10: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/10.jpg)
Chess• In 1996 and 1997, Gary Kasparov, the world chess
grandmaster played two tournaments against Deep Blue, a program written by researchers at IBM
Source: IBM Research
![Page 11: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/11.jpg)
Deep Blue’s Results in the first tournament:
won 1 game, lost 3 and tied 1• first time a reigning world champion lost to a computer
• although Kasparov didn’t see it that way…
Source: CNN
![Page 12: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/12.jpg)
Deep Blue’s Results in the second tournament: – second tournament: won 3 games, lost 2, tied 1
![Page 13: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/13.jpg)
Learning is essential to building autonomous robots
Source: RoboCup web site
![Page 14: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/14.jpg)
Autonomous robots and self-diagnosis
Y1 Y3
X1 X2 X3
Y2
Unknown continuous signals
Sensor readings
M1 M2 M3
Unknown internal discrete state
![Page 15: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/15.jpg)
Simultaneous localization and map learning
![Page 16: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/16.jpg)
Tracking and activity recognitionTracking and activity recognition
![Page 17: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/17.jpg)
Animation and control Animation and control
Source: Aaron Hertzmann
![Page 18: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/18.jpg)
Learning agents that play poker
• In full 10-player games Poki is better than a typical low-limit casino player and wins consistently; however, not as good as most experts
• New programs being developed for the 2-player game are quite a bit better, and we believe they will very soon surpass all human players
Source: The University of Alberta GAMES Group
![Page 19: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/19.jpg)
Speech recognition
P(words | sound) P(sound | words) P(words)Final beliefs Likelihood of data Language model
“Recognize speech” “Wreck a nice beach”
eg mixture of Gaussians eg Markov model
Hidden Markov Model (HMM)
![Page 20: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/20.jpg)
Natural language understanding • P(meaning | words) P(words | meaning) P(meaning)
• We do not yet know good ways to represent "meaning"
(knowledge representation problem)
• Most current approaches involve "shallow parsing", where the meaning of a sentence can be represented by fields in a database, eg
– "Microsoft acquired AOL for $1M yesterday"
– "Yahoo failed to avoid a hostile takeover from Google"
Buyer Buyee When Price
MS AOL Yesterday $1M
Google Yahoo ? ?
![Page 21: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/21.jpg)
Example: Handwritten digit recognition for postal codes
![Page 22: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/22.jpg)
Example: Face Recognition
Training examples of a person
Test images
AT&T Laboratories, Cambridge UKhttp://www.uk.research.att.com/facedatabase.html
![Page 23: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/23.jpg)
Interactive robots
![Page 24: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/24.jpg)
Introduction to machine learning• What is machine learning?• How is machine learning related to other fields?• Machine learning applications• Types of learning
– Supervised learning • classification • regression
– Unsupervised learning • clustering• data association• abnormality detection• dimensionality reduction• structure learning
– Semi-supervised learning– Active learning – Reinforcement learning and control of partially observed Markov decision
processes.
![Page 25: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/25.jpg)
Classification• Example: Credit scoring• Differentiating between
low-risk and high-risk customers from their income and savings
Discriminant: IF income > θ1 AND savings > θ2 THEN low-risk ELSE high-risk
Input data is two dimensional, output is binary {0,1}
![Page 26: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/26.jpg)
Classification of DNA arrays
Y1
X1
YN
XN
Y*
X*
Cancer/not cancer
DNA array
Not cancer
Cancer
![Page 27: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/27.jpg)
Classification
Color Shape Size
Blue Square Small
Red Ellipse Small
Red Ellipse Large
Label
Yes
Yes
NoBlue Crescent Small
Yellow Ring Small
?
?
Training set:X: n by py: n by 1
Test set
n cases
p features (attributes)
![Page 28: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/28.jpg)
Hypothesis (decision tree)
blue?
big?
oval?
no
no
yes
yes
yes
no
![Page 29: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/29.jpg)
Decision Tree
blue?
big?
oval?
no
no
yes
yes
?
![Page 30: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/30.jpg)
Decision Tree
blue?
big?
oval?
no
no
yes
yes
?
![Page 31: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/31.jpg)
What's the right hypothesis?
![Page 32: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/32.jpg)
What's the right hypothesis?
![Page 33: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/33.jpg)
How about now?
![Page 34: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/34.jpg)
How about now?
![Page 35: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/35.jpg)
Noisy/ mislabeled data
![Page 36: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/36.jpg)
Overfitting• Memorizes irrelevant details of training set
![Page 37: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/37.jpg)
Underfitting• Ignores essential details of training set
![Page 38: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/38.jpg)
Now we’re given a larger data set
![Page 39: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/39.jpg)
Now more complex hypothesis is ok
![Page 40: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/40.jpg)
Linear regression
• Example: Price of a used car• x : car attributes
y : price
y = g (x,θ)
g ( ) model,
= (w,w0) parameters (slope and intercept)
y = wx+w0
Regression is like classification except the output is a real-valued scalar
![Page 41: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/41.jpg)
Nonlinear regression
Useful for:• Prediction• Control• Compression• Outlier detection• Knowledge extraction
![Page 42: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/42.jpg)
Introduction to machine learning• What is machine learning?• How is machine learning related to other fields?• Machine learning applications• Types of learning
– Supervised learning • classification • regression
– Unsupervised learning • clustering• data association• abnormality detection• dimensionality reduction• structure learning
– Semi-supervised learning– Active learning – Reinforcement learning and control of partially observed Markov decision
processes.
![Page 43: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/43.jpg)
Clustering
Input
Desired output
Soft labelingHard labeling
K=3 is the number of clusters, here chosen by hand
![Page 44: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/44.jpg)
’s
Y1 Y2 YN
group
Clustering
“yellow fish”“angel fish” “kitty”
![Page 45: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/45.jpg)
Introduction to machine learning• What is machine learning?• How is machine learning related to other fields?• Machine learning applications• Types of learning
– Supervised learning • classification • regression
– Unsupervised learning • clustering• data association• abnormality detection• dimensionality reduction• structure learning
– Semi-supervised learning– Active learning – Reinforcement learning and control of partially observed Markov decision
processes.
![Page 46: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/46.jpg)
Semi-supervised learning
![Page 47: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/47.jpg)
Introduction to machine learning• What is machine learning?• How is machine learning related to other fields?• Machine learning applications• Types of learning
– Supervised learning • classification • regression
– Unsupervised learning • clustering• data association• abnormality detection• dimensionality reduction• structure learning
– Semi-supervised learning– Active learning – Reinforcement learning and control of partially observed Markov decision
processes.
![Page 48: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/48.jpg)
Active learning
• In active learning, the machine can query the environment. That is, it can ask questions.
Does reading this improve your knowledge of Gaussians?
• Decision theory leads to optimal strategies for choosing when and what questions to ask in order to gather the best possible data. Good data is often better than a lot of data.
• Active learning is a principled way of integrating decision theory with traditional statistical methods for learning models from data.
![Page 49: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/49.jpg)
Nonlinear regressionUseful for predicting:• House prices• Drug dosages• Chemical processes• Spatial variables• Output of control action
![Page 50: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/50.jpg)
Active learning example
![Page 51: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/51.jpg)
Introduction to machine learning• What is machine learning?• How is machine learning related to other fields?• Machine learning applications• Types of learning
– Supervised learning • classification • regression
– Unsupervised learning • clustering• data association• abnormality detection• dimensionality reduction• structure learning
– Semi-supervised learning– Active learning – Reinforcement learning and control of partially observed Markov decision
processes.
![Page 52: CPSC 540 Machine Learning](https://reader035.vdocuments.site/reader035/viewer/2022082208/568144ca550346895db192f6/html5/thumbnails/52.jpg)
Partially Observed Markov Decision Processes (POMDPs)
x1
a1
r1
x2
a2
r2
x3
a3
r3
x4
a4
r4
During learning: we can estimate the transition p(x_t|x_t-1,a_t-1) and reward r(a,x) models by, say observing a human expert.
During planning: we learn the best sequence of actions (policy) so as to maximize the discounted sum of expected rewards.