1

Machine Learning CMPUT 466/551

Nilanjan Ray

Department of Computing Science

University of Alberta

2

Introduction

• What is machine learning (ML)?

• Taxonomy in ML

• Applications/Examples

• Related disciplines

• References

• Resources

3

What is machine learning (ML)?

• Definition of “learning” from Merriam-Webster: “To gain knowledge or understanding of or skill in by study, instruction, or experience”

• ML = Learning in machines (computers)• ML techniques are algorithms that enable the

machines to improve its performance at some task through experience

• Tasks: recognition, diagnosis, prediction, planning, data mining, robot control, and so on.

4

Why Machine Learning?

• Some tasks can be specified only by training data/examples

• Human expertise may be scarce and/or very costly

• Amount of knowledge might be too large for explicit encoding by humans

• Modeling/Hidden parameter estimation: Often only data from measurements are available

• Computational power is ever increasing• Growing data pool and storage capacity

5

A Brief History about ML

• 1950’s: – Samuel’s checker program– Rosenblatt’s perceptron

• 1960’s: – Neural network– Pattern recognition

• 1970’s:– Winston’s ARCH– Buchanan and Mitchell’s Meta-Dendral: mass spectrometry

prediction rules– Quinlan’s ID3: Chess end-game rules– Michalski’s AQ11: Soybean disease diagnosis rules– MACROPS: macro operators in block world planning

6

A Brief History about ML…

• 1980’s: Gains momentum– Learning theory– Symbolic learning algorithms– Connectionist learning algorithm– Clustering– Explanation-based learning– Knowledge guided inductive learning– Genetic algorithm

• 1990’s: Maturity– Data mining– Ensemble learning: bagging, boosting etc.– Kernel methods– Reinforcement learning– Theoretical analysis

7

Typical Taxonomy for ML

• Supervised learning

• Unsupervised learning

• Semi-supervised learning

• Reinforcement learning

8

Supervised Learning

• Training data available in the form of (input, output) pairs

• When output is continuous valued the problem is called regression; if the output is qualitative or categorical the problem is called classification

• The goal here is to estimate the output for a novel (never-seen-before) input, after learning the training data

9

A Supervised Learning Example(From [DHS] book)

• “Sorting incoming Fish on a conveyor according to species using optical sensing”

Sea bass

Species

Salmon

10

• Set up a camera and take some sample images to extract features– Length– Lightness– Width– Number and shape of fins– Position of the mouth, etc…

Problem Analysis

11

• Use a segmentation operation to isolate fishes from one another and from the background

• Information from a single fish is sent to a feature extractor whose purpose is to reduce the data by measuring certain features

• The features are passed to a classifier

Preprocessing

12

13

Feature for Classification

Can we select the length of the fish as a possible feature for discrimination?

14

Let’s Try Another Feature

Lightness (Intensity of image pixels)

15

• Adopt the lightness and add the width of the fish

Fish xT = [x1, x2]

Lightness Width

Yet Another Feature: Width

16

A Classifier

So, how does the “Width” feature help?

17

Another Classifier

18

Unsupervised Learning

• No training data in the form of (input, output) pair is available

• Applications:– Dimensionality reduction– Data compression– Outlier detection– Classification– Segmentation/clustering– Probability density estimation– …

19

Example: Unsupervised Learning

DNA microarray data (taken from [HTF])

20

• Applications on DNA microarray– Clustering: Group genes or samples into similar

expression profiles– Bi-clustering: Subset of genes exhibiting similar

expression pattern along a subset of samples– Dimension reduction– …

Example: Unsupervised Learning (contd..)

21

Semi-supervised Learning

• Uses both labeled data (in the form (input, output) pairs) and unlabelled data for learning

• When labeling of data is a costly affair semi-supervised techniques could be very useful

• Examples: Generative models, self-training, co-training

22

Example: Semi-supervised Learning

Source: Semi-supervised literature survey by X. Zhu, Technical Report

23

Reinforcement Learning

• Reinforcement learning is the problem faced by an agent that must learn behavior through trial-and-error interactions with a dynamic environment.

• There is no teacher telling the agent wrong or right• There is critic that gives a reward / penalty for the

agent’s action• Applications:

– Robotics– Combinatorial search problems, such as games– Industrial manufacturing– Many others!

24

Example: Reinforcement Learning

  Tic Tac Toe TD-Gammon

Goal Learn to play optimal game

Learn to play game at master level

States All possible board states - 9

All possible board states - 1020

Action A new X in an empty field

21 dice combinations & avg. 20 legal moves

Reinforcement Signal

+10 winning -1 for every move

that did not win

‘1 ‘ for a reward ‘0‘ for a penalty

25

Related Disciplines

• Statistics

• Artificial Intelligence

• Psychology

• Vision and Neuroscience

• Control Theory

• Signal and Image Processing

• …..

26

References and Journals• Text: The Elements of Statistical Learning by Hastie, Tibshirani, and

Friedman (book website: http://www-stat.stanford.edu/~tibs/ElemStatLearn/) • Reference books:

– Pattern Classification by Duda, Hart and Stork– Pattern Recognition and Machine Learning by C.M. Bishop– Machine Learning by T. Mitchell– Introduction to Machine Learning by E. Alpaydin

• Some related journals / associations:– Machine Learning (Kluwer). – Journal of Machine Learning Research. – Journal of AI Research (JAIR). – Data Mining and Knowledge Discovery - An International Journal. – Journal of Experimental and Theoretical Artificial Intelligence (JETAI). – Evolutionary Computation. – Artificial Life. – Fuzzy Sets and Systems – IEEE Intelligent Systems (Formerly IEEE Expert) – IEEE Transactions on Knowledge and Data Engineering – IEEE Transactions on Pattern Analysis and Machine Intelligence – IEEE Transactions on Systems, Man and Cybernetics– Journal of AI Research – Journal of Intelligent Information Systems – Journal of the American Statistical Association – Journal of the Royal Statistical Society

27

References and Journals…– Pattern Recognition – Pattern Recognition Letters – Pattern Analysis and Applications. – Computational Intelligence . – Journal of Intelligent Systems . – Annals of Mathematics and Artificial Intelligence. – IDEAL, the online scientific journal library by Academic Press. – ECCAI (European Coordinating Committee on Artificial Intelligence). – AAAI (American Association for Artificial Intelligence). – IJCAI (International Joint Conferences on Artificial Intelligence, Inc.). – ACM (Association for Computing Machinery). – Association for Uncertainty in Artificial Intelligence. – ACM SIGAR– ACM SIGMOD– American Statistical Association. – Artificial Intelligence – Artificial Intelligence in Engineering – Artificial Intelligence in Medicine – Artificial Intelligence Review – Bioinformatics – Data and Knowledge Engineering – Evolutionary Computation

28

Some Conferences & Workshops• Congress on Evolutionary Computation

• European Conference on Machine Learning and Principles and Practice of Knowledge Discovery

• The ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

• National Conference on Artificial Intelligence

• Genetic and Evolutionary Computation Conference

• International Conference on Machine Learning

• Conference on Autonomous Agents and Multiagent Systems

• European Symposium on Artificial Neural Networks Advances in Computational Intelligence and Learning

• Artificial and Ambient Intelligence

• Computational Intelligene in Biomedical Engineering

• IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning

• International Joint Conference on Artificial Intelligence