shirish shevade - visvesvaraya technological...

Machine Learning

Shirish Shevade

Department of Computer Science and AutomationIndian Institute of ScienceBangalore 560 012, India.

[email protected]

July 02, 2010

Shirish Shevade (IISc) Machine Learning July 02, 2010 1 / 18

Machine Learning

Humans have developed sophisticated skills for recognizing patterns

Spam RecognitionReading handwritingUnderstanding spoken wordsFace recognitionWeather Prediction

Machine Learning

Can we write computer programs which learn these skills from pastexperience?

Machine Learning

Can we write computer programs which learn these skills from pastexperience?

Machine Learning: Study of theory, algorithms and implementationsthat enable computers to learn from experience

Applications of Machine Learning

Data Mining

Speech Recognition

Bioinformatics [Baldi and Brunak, 1998]

Statistical Debugging [Zheng et al, ICML, 2006]

Compiler Design [Stephenson et al, PLDI, 2003; Joseph et al, TheCompiler Design Handbook, 2008]

Computer vision

Stock market index prediction

Data Mining

Speech Recognition

Computer vision

Data Mining

Speech Recognition

Computer vision

Data Mining

Speech Recognition

Computer vision

Data Mining

Speech Recognition

Computer vision

Data Mining

Speech Recognition

Computer vision

Data Mining

Speech Recognition

Computer vision

Different Learning Techniques

Supervised

Unsupervised

Semi-supervised

Reinforcement

Types of Learning Problems

Classification

Classification- Binary, multi-class,

multi-label etc.

Classify an image as

star or galaxy

Regression

Regression- Real valued output

Predict tomorrow’s

rainfall

Clustering

- Find similar data items

Application: MarketResearch

Ranking

- Order examples by preference

- Application: Ordering of web search results

Classification

Regression

Clustering

Ranking

Algorithms

Linear models for classification and Regression

Naive Bayes Classifiers

Decision Trees

Perceptron

Support Vector Machines (SVM)

Gaussian Processes

Clustering algorithms (k-means, hierarchical)

Research Challenges

Relational Learning

- Finding events, relationships in the data- Use of these relationships to achieve better classification accuracy- Application - Web page classification

Information Extraction

- Extraction of structure from unstructured, heterogeneous sources- Applications: Tracking News, Ad placement on webpages

Cross-language information retrieval

Finance

- Credit card fraud detection- Detection of stock market manipulation

Research Challenges

Relational Learning

Finance

Research Challenges

Relational Learning

Finance

Research Challenges

Relational Learning

Finance

Research Challenges

Structured Prediction

Research Challenges

Structured Prediction

- Output is not a scalar -sequence, tree, graphetc

- Use interdependence inthe outputs

- Applications:Computational Biology,Natural LanguageAnalysis

Applications in Machine Learning

Linear Algebra

Non-negative matrixfactorization

Singular valuedecomposition

Optimization

Duality ideas

Efficient solutions ofoptimization problems

Probability

Bayesian networks

Markov networks

Graph Theory

Graph cut algorithms

Max flow algorithms

Game Theory

Feature selection

Adversarial learning

Linear Algebra

Optimization

Duality ideas

Probability

Bayesian networks

Markov networks

Graph Theory

Max flow algorithms

Game Theory

Feature selection

Linear Algebra

Optimization

Duality ideas

Probability

Bayesian networks

Markov networks

Graph Theory

Max flow algorithms

Game Theory

Feature selection

Linear Algebra

Optimization

Duality ideas

Probability

Bayesian networks

Markov networks

Graph Theory

Max flow algorithms

Game Theory

Feature selection

Linear Algebra

Optimization

Duality ideas

Probability

Bayesian networks

Markov networks

Graph Theory

Max flow algorithms

Game Theory

Feature selection

Some Reputed Journals and Conferences

Journals

Machine Learning

Jl. of Machine LearningResearch

Neural Computation

Neural Networks

IEEE PAMI

IEEE NN

Pattern Recognition

Conferences

Intl Conf on MachineLearning (ICML)

Neural Inf. ProcessingSystems (NIPS)

Intl Joint Conf on AI(IJCAI)

IEEE Intl Conf on DataMining (IEEE ICDM)

SIGKDD

Some Reputed Journals and Conferences

Journals

Machine Learning

Jl. of Machine LearningResearch

Neural Computation

Neural Networks

IEEE PAMI

IEEE NN

Pattern Recognition

Conferences

Intl Conf on MachineLearning (ICML)

Neural Inf. ProcessingSystems (NIPS)

Intl Joint Conf on AI(IJCAI)

IEEE Intl Conf on DataMining (IEEE ICDM)

SIGKDD

Readings

Pattern Recognition and Machine Learning

- C. M. Bishop

The Elements of Statistical Learning

- Hastie, Tibshirani and Friedman

Pattern Classification

- Duda, Hart and Stork

Machine Learning

- Mitchell

Learning with Kernels

- Scholkopf and Smola

Data Mining: Practical Machine Learning Tools and Techniques,

- Witten and Frank

Probabilistic Graphical Models: Principles and Techniques

- Koller and Friedman

. . . Some Resources

Tutorials

The Discipline of Machine Learning

- Mitchell

A Tutorial on Support Vector Machines for Pattern Recognition

- C.J.C. Burges

- Sunita Sarawagi

A Tutorial on Spectral Clustering

- Ulrike von Luxburg

. . . Some Resources

Popular Sites

David Aha’s page

- http://home.earthlink.net/~dwaha/research/tutorials.html

Andrew Moore’s page

- http://www.cs.cmu.edu/~awm/tutorials

Tom Dietterich’s page

- http://web.engr.oregonstate.edu/~tgd/projects/tutorials.html

Kernel methods page

- http://www.kernel-machines.org

LIBSVM - Kernel Methods Software

- http://www.csie.ntu.edu.tw/~cjlin/libsvm/

Weka - Machine Learning Software

- http://sourceforge.net/projects/weka/

Machine Learning Database Repository

- http://mlearn.ics.uci.edu/MLRepository.html

Thank You.

shirish shevade - visvesvaraya technological...

Documents