8. machine learning - darmstadt university of applied …€¦ · · 2016-05-308. machine...

Prof. Dr. Bernhard Humm, Darmstadt University of Applied Sciences. www.fbi.h-da.de/~b.humm. 30.05.20161

8. Machine LearningApplied Artificial Intelligence

Prof. Dr. Bernhard HummFaculty of Computer ScienceHochschule Darmstadt – University of Applied Sciences


RetrospectiveNatural Language Processing

• Name and explain different areas of NLP

• What are the “7 levels of language understanding“?

• What is tokenizing, sentence splitting, POS tagging, and parsing?

• What do language resources offer to NLP? Give examples

• What do NLP frameworks offer? Give examples

• What do NLP web services offer? Give examples

2

Agenda

• Overview

• ML Applications

• ML Tasks

• ML Approaches

• Methodology

• ML Tools

• Services / Product Map



What is Machine Learning (ML)?

4

Generating a model based on inputs and using it for making decisions or predictions

( rather than programming instructions explicitly )

Agenda

• Overview

• ML Applications

• ML Tasks

• ML Approaches

• Methodology

• ML Tools




Applications of ML:Spam filtering

• Task: classify new e-mails as spam or not spam

6

Spam filter

New e-mails

Automaticallyclassified

Manuallyclassified

Corrections

ML input


Stock market analysis

• Task: make recommendations on buying and selling stocks

7

Prediction

Current stock values

History ofstock values

ML input

Recommendation

Decision

Image source: Wikimedia


Detecting credit card fraud

• Task: Detect fraud in credit card payments

8

Fraud detection

CC payments

Automaticallyclassified

Manuallyclassified

Corrections

ML input


Recommender systems

• Task: Recommending customers suitable products

9

Recommender system

Order

Recommendationof related products

ML input

Purchasing behaviourof other customersor customer groups

Agenda

• Overview

• ML Applications

• ML Tasks

• ML Approaches

• Methodology

• ML Tools




Categories of ML tasks

• P.S. Other categorizations / groupings are possible

11

Machine Learning Task

SupervisedLearning

UnsupervisedLearning

ReinforcementLearning

Classifi-cation

Regression ClusteringFeature

selection / extraction

Topic modeling


Categories of ML tasks

• Given: Example inputs and desired outputs

• Goal: Learn a general rule that maps inputs to outputs

Supervisedlearning

• Given: Data inputs (e.g., documents)

• Goal: Find structure in the inputs

Unsupervisedlearning

• Setting: An agent interacts with a dynamic environment in which it must perform a goal

• Goal: Improving the agent‘s behaviour

Reinforcement learning

12


Supervised learning subcategories

• Given: Training inputs (records) which aredivided into two or more classes

• Goal: Produce model to classify new inputs

• Examples: spam filter, fraud detection, …

Classification

• Given: Training data (records) withcontinuous (not discrete) output values

• Goal: Produce model to predict outputvalues for new inputs

• Example: stock value prediction

Regression

13


Unsupervised learning subcategories

•Given: Set of input records

•Goal: Identifying clusters (groups of similar records)

•Example: Customer groupingClustering

•Given: Set of input records with attributes („features“)

•Goal: Find a subset of the original attributes that areequally well suited for classification / clustering tasks

Feature selection / extraction

•Given: Set of text documents

•Goal: Find abstract topics that occur in severaldocuments and classify documents accordingly

Topic modeling

14

Agenda

• Overview

• ML Applications

• ML Tasks

• ML Approaches

• Methodology

• ML Tools




Decision Tree Learning

• Used for supervised learning

(classification, regression)

• Training input: Training data

(records) with output values

(discrete or continuous)

• Learning result: decision tree that

allows classifying / predicting output

values of new data records

• Example (figure): Decision tree for

classfying passengers on the Titanic

in survived / died

16 Image source: Wikipedia


Artificial Neural Networks (ANN)

• Inspired by brain / nervous system:

- Neurons connected via dentrites

- Reduce resistance if fired repeatedly

• Artificial Neuron:

- Weighted inputs

- Function, e.g., weighted sum

- Filter, e.g, threshold output

• Artificial Neural Network (ANN):

- Input layer, output layer, and possibly

intermediate layers of neurons

- Training phase: weights are adjusted via

known cases

- Regognition phase: output is produced for

new cases

17 Source: Ivan Galkin, U. MASS Lowell ( http://ulcar.uml.edu/~iag/CS/Intro-to-ANN.html )

http://ulcar.uml.edu/~iag/CS/Intro-to-ANN.html


Deep learning

18

• Cascade of many layers for feature extraction and transformation

• Levels form a hierarchy of concepts.

• Each successive layer uses the output from the previous layer as

input

• Applications include feature selection / extraction (unsupervised)

and classification (supervised).

• ANNs are often used, but

other approaches are possible,

too


BayesianNetworks

• Directed acyclic graph (DAG) with:

- Nodes: random variables

+ probability function

- Edges: conditional

dependencies

• Example:

- Causes, diseases, symptoms

• Bayes Network inference allows answering questions like:

- What is the probability of a lung disease in case of a cough?

19

Source: Goodman & Tenenbaum https://probmods.org/

https://probmods.org/


Inductive Logic Programming

• Given:

- Set of logic facts (background knowledge), e.g.

male(Tom), female(Eve), parent (Tom, Eve)

- Positive and / or negative examples, e.g.,

daughter (Eve, Tom)

• Learning goal:

- General rules that are consistent with the examples and the

background knowledge, e.g.,

parent(p1, p2) and female(p2) daughter(p2, p1)

20

George

TomMary

Helen

Nancy

Eve

parent

male female

Agenda

• Overview

• ML Applications

• ML Tasks

• ML Approaches

• Methodology

• ML Tools



Evaluation,Planning exams


Training / test data setsin supervised machine learning

1. Obtain a set of data with input and output values,

e.g., manually classifying paintings as portraits, still lifes,

landscapes, etc.

2. Separate the data set into two

disjoint subsets:

a. Training set

b. Test set

3. Apply machine learning, e.g., train an Artificial Neural Network

(ANN) with the training set

4. Feed the ANN with the test set (omitting the ouput values) and

collect the output values computed by the ANN

5. Compare the computed output values with the expected ones and

compute measures like precision, recall, and F-Measure22

Source: Wikipedia


The problem of overfitting

• It is easy to optimize precision, recall, and F-measure for the

training set

• A model that simply memorizes all data points will leads to an

F-Measure of 100% on the

training set

• However, the F-Measure

for the test set will be worse

• Memorizing is an example of

overfitting:

the model has too many

parameters (is too complex)

relative to the number of training data points

• Find the appropriate level of abstraction for the problem domain

23

Image Source: Wikipedia


Process Overview

24

Agenda

• Overview

• ML Applications

• ML Tasks

• ML Approaches

• Methodology

• ML Tools




RapidMiner example:Customer segregation

26

https://rapidminer.com/

https://rapidminer.com/


Analyzing the data

27


Process for generating a decision tree

28


Decisiontree

29


Using the decision tree forcustomer segregation

30


Validation

31


Precision, recall, accuracy

32

Agenda

• Overview

• ML Applications

• ML Tasks

• ML Approaches

• Methodology

• ML Tools




ML Services Map

38

Machine learning libraries

Machine learning web services

Machine learning development environments / frameworks

IDEs and frameworksfor experimenting with

different ML approaches and

configuring solutions

Web services for forexperimenting with

different ML approaches and

configuring solutions

Algorithms for classification, regression, clustering, feature selection / extraction, topic modeling, etc. using different approaches, e.g., decisiontree learning, Artificial Neural Networks, Bayes networks, inductive logic

programming, Support Vector machines, Hidden Markov Chains, etc.


ML Product Map

39

Machine learning libraries

Machine learning web services

Machine learning development environments / frameworks

Google Prediction API, Microsoft Azure Machine Learning, bigml,

wise.io, procog, ersatz, …

TensorFlow, DL4J, Torch, Caffeee, Theano, Eblearn, OpenNN, aisolver,

CURRENNT, …

SPSS Modeler, RapidMiner, WEKA, Orange, Shogun, scikt-learn, …


ML product table

40

Product Library IDE / Framework Web service

aisolver *

bigml *

Caffee *

CURRENNT *

DL4J *

eblearn *

Encog *

ersatz *

Fast Artificial Neural Network Library *

Google Prediction API *

Jaden * *

Java Neural Network Framework Neuroph * *

Joone * *

Microsoft Azure Machine Learning *

OpenNN - Open Neural Networks Library *


ML product table (cont‘d)

41

Product Library IDE / Framework Web service

Orange * *

procog *

RapidMiner * *

scikit-learn * *

Shogun * *

SPSS Modeler * *

TensorFlow *

Theano *

Torch *

WEKA * *

wise.io *

8. machine learning - darmstadt university of applied …€¦ · · 2016-05-308. machine...

Documents