Engineering

Intelligent System

with Machine

Learning Saurabh Kaushik

Agenda

Why ML is significant?

What is ML Technology?

How to Engineering an Intelligent System?

What is Next in ML Technology?

Use Cases & Demo

1

2

3

4

5

Machine Learning vs Traditional Learning

Machine Learning "A computer program is said to learn from experience E with respect to some

class of tasks T and performance measure P, if its performance at tasks in T,

as measured by P, improves with experience E“ – T. Michell (1997)

Example: A program for soccer tactics

• Task : Win the game

• Performance : Goals

• Experience : (x) Players’ movements (y) Evaluation

Why ML is Significant?

Why do Automate?

A few thousand years ago:

Manual Plowing

Today: Automated Plowing

Path of Machine Evolution…

Automation Evolution

System that Do

• Replicate repetitive human actions

System that Think

• Cognitive capabilities handle judgment-oriented tasks

System that Learn/Adapt

• Learn to understand context and adapt to users and systems

Robotic Automation

Cognitive Automation

Intelligent Automation

Natural Language

Processing

Big Data Analytics

Artificial Intelligence

Machine Learning

Large Scale Processing

Adaptive Alteration

Rule Engine

Screen Scraping

Workflow

Unstructured Data

Processing (Extraction)

Knowledge Modelling

(Ontologies)

Implementation: • Macro-based applets

• Screen Scraping data collection

• Workflow Implementation

• Process Mapping

• Business Process Management

Implementation: • Built-in Knowledge repository

• Learning capabilities

• Ability to work with unstructured data

• Pattern recognition

• Reading source data manuals

Implementation: • Artificial Intelligence Systems

• Natural Language Understanding and Generation

• Self Optimizing / Self Learning

• Predictive Analytics / hypothesis generation

• Evidence based learning

Cap

ab

ilit

ies

Cap

ab

ilit

ies

Cap

ab

ilit

ies

Evolution of Machine Intelligence

• Raw computing power can automate complex tasks! Great Algorithms

+ Fast Computers

• Automating automobiles into autonomous automata! More Data + Real-

Time Processing

• Automating question answering and information retrieval! Big Data + In-

Memory Clusters

• Deep Learning + Smart Algorithms = Master Gamer

Deep Learning

• New algorithm learns handwriting of unseen symbols from very few training examples (unlike typical Deep Learning)

Improve Training Efficiency

IBM Deep

Blue

Google Self

Driven Cars

Watson

Jeopardy

Deepmind

Atari Game

One Shot

Learning

Why Machine Learning?

Human Behavior & their Life are not logical like Code, not linear like a Formulas and not consistent like Rules, so it is hard for Machines

to understand & respond to humans, that is the challenge for todays Digital world. Unless, Machine starts to Learn this ever changing

human behavior, it can neither understand effectively nor respond intelligently & personally with its human counterpart. Machine

Learning algorithms offers a mechanism to understand this non-linear, non-consistent and intuitive behavior.

Formula

Behavior

Actual

Behavior

Machine Learning to help machine Learn

about Human World.

Where can we Apply?

What is ML Technology?

What is Machine Learning Process ?

Types of Tasks for ML

Decide between two classes

Group data points tightly

Fit the target values

Cla

ssif

icati

on

Regre

ssio

n

Clu

steri

ng

An

om

aly

Dete

cti

on

Find something out of place

Calls to Customer Care

Delta Change in Calls

Duration

Grouping by distance from

tower

Call drops due to technical

issues

How to build Model?

Task : Prove Hypothesis

Experience : Nature of Training Data

Goal : Minimize Loss Function

Loss Function = | Predicted Value – Actual Value |

How to evaluate Model Performance?

Cross Validation

Major Reasons:

• Less relevant Feature

• Smaller Training Data Set

• Higher Polynomials

• High/Low Learning Rate

• High/Low Regularization Value “Underfitting”

What are Key Data Learning Algorithms?

Reinforcement

Learning

Learning from Data Paradigm

• Learning by fully

labelled Data

• Used For: Prediction,

Classification (discrete

labels), Regression (real

values)

• Learning by Data

interrelationship

• Used for: Clustering,

Probability distribution

estimation, Finding association

(in features)

• Learning by Feedback

Loop

• Used for: Decision making

(robot, chess machine)

• Learning by partially

labelled and Data

interrelationships

• Used For: Prediction,

Classification (discrete

labels), Regression (real

values)

What are Key Problem Solving Algorithms? Problem Type Paradigm

What is probable

effect of it?

How can we generalize

given model?

Is this A or B? Is

this A or B or C?

What is its decision

flow/reasoning?

Can we draw straight

rules from it?

How is it Organized?

Can combining models gives

better output?

Classification

Algorithms

How much/How

many it is?

Can we get higher

abstraction from it?

What is common in

it?

What is the similarity

in it? Can it draw finer

feature from it?

Is it weird? What should I do

Next?

Anomaly Detection Reinforcement

Learning

How to choose amongst algorithms?

How to Engineer an Intelligent

System?

Engineering Intelligent System

Architecture B

uild

Ph

ase

Op

era

tio

n

Ph

ase

What is difference between Software vs Intelligent System Engineering?

Deployment

Monitoring Support

Testing

Regression/ Integration System Testing

NFR / Performance Testing

Implementation

Code Implementation Unit Testing

Designing

HLD - Architecture Level LLD – Class and method level

System Analysis

Requirement Gathering Technical Specification of

Requirements

Model Deployment

Monitoring Evaluating Managing

Model Evaluation

Error Analysis Tuning Model

Model Training

Model Selection Model Training

Feature Engineering

Feature Extraction / Processing Feature Ranking / Selection /

Reduction

Data Preparation

Data Acquisition Data Preprocessing

Software System Engineering Process Intelligent System Engineering Process

WHAT IS NEXT IN ML TECHNOLOGY?

What is NEXT in ML?

What is DL?

• “Deep Learning is a set of algorithms in Machine Learning that Attempts to model high level abstractions in data by using architecture composed of multiple non-linear transformations.”

• Deep Learning don’t need to provide explicit Feature Engineering. It learns based on algorithm’s non learner transformation logics.

What is current landscape?

Use case & Demos

Demo – Predicting Consumer Churn Scenario:

• Company has been managing CRM Process for a large US based Telecom giant.

• Lately, Client has been showing concerns about Customer churn due to various reasons.

• Company wants to help its client by developing an Intelligent System to predict/detect customers which are likely to abandon their subscription.

Problem Analysis

Data Acquisition

Feature Engineering

Model Training

Model Evaluation

State Account

Length Area Code Phone Int'l Plan VMail Plan

Night

Charge Intl Mins Intl Calls

Intl

Charge

CustServ

Calls

Subscribed

(Churn)

True/False

Predicted Column

Hypothesis:

• Customer Churning can be predicted by their Usage of Calls as well as Frequency of Customer Care calls.

Objective of Demo:

• To evaluate and select best performing ML Model for predicting Customer Churn. (Build Phase)

Customer Data:

Irrelevant Columns Binary Value Columns (Yes/No)

Binary Classification Reading CSV File into

Data Frame

Removing irrelevant

columns and modifying

data value

Train models with

three best with Cross

Validation Technique

Using Confusion

Matrix – Find best

most suitable Algo

Confusion Matrix

Actual Value

Predicted Value

Correct Value

Incorrect Value

Demo - Evaluating Models

• Precision - When a classifier predicts an

individual will churn, how often does that

individual actually churn? (Accuracy)

Precision = 235 / 269

Recall = 235 / 483

Precision = 330 / 256

Recall = 330 / 483

Precision = 167 / 211

Recall = 167/ 483

• Recall - When an individual churns, how often

does my classifier predict that correctly?

(Coverage)

Thank You

Saurabh Kaushik