just add imagination
TRANSCRIPT
© 2016 KNIME.com AG. All Rights Reserved.
The Evolution of Data Analytics
3
Classic Analytics of CRM Data
More Data: Customer Intelligence
Change Objective: Restocking Strategy in Retail
No Historical Examples: Anomaly Detection in IoTPredictive Maintenance
Change Perspective: User Experience
© 2016 KNIME.com AG. All Rights Reserved.
Classic CRM Analytics: Churn Prediction
5
CRM SystemData about your customer• Demographics• Behavior• Revenues
Model
• Churn Prediction• Upselling Likelihood• Product Propensity /NBO• Customer Segmentation• Campaign Management• …
© 2016 KNIME.com AG. All Rights Reserved.
Churn Score for each Customer
7
YouTube: “Building a basic Model for Churn Prediction with KNIME” https://www.youtube.com/watch?v=RHsO10q7e2Y
© 2016 KNIME.com AG. All Rights Reserved.
Classic CRM Analytics: CRM Data, Train a Model, Use the Model
11
Daily CallersNon-Daily Callers
No Voice Messages
© 2016 KNIME.com AG. All Rights Reserved.
Include other Data Sources: Social Media
12
Why only CRM Data?Web Pages, Google APIs, Sensors, Social Media, …
© 2016 KNIME.com AG. All Rights Reserved.
The Problem
A major European Telco
14
• Can you tell us what people say about our new product?
• Can you tell us who is supporting the product and who trashing it?
• Of those, can you tell us who is an influencer?
Its Forum Site
© 2016 KNIME.com AG. All Rights Reserved.
The Data
16
• The Data Set unfortunately cannot be shared
• Slashdot Forum Data are!
• Slashdot was a public forum built in 1997 and hosting a number of discussions: from software to philosophy, from science fiction to politics.
• Politics was the biggest discussion group
• So, politics is what we analyzed to find out:– What users were thinking about a political issue
– Who was pro and who was con
– Who was an influencer
© 2016 KNIME.com AG. All Rights Reserved.
Text Analytics and Network Mining: Workflow
Read Data
Network Mining
Sentiment Analysis
Joiner
Scatter Plots
© 2016 KNIME.com AG. All Rights Reserved.
Text Analytics: Results
Most negative user pNutz
Most positive and most
talkative user dada21
© 2016 KNIME.com AG. All Rights Reserved.
Network Mining: Results
Dada21
Carl Bialik from the WSJ
Doc Ruby
© 2016 KNIME.com AG. All Rights Reserved.
Text Analytics and Network Mining: Results
pNutz
Carl Bialik
dada21
Doc Ruby
99BottlesOfBeerInMyF
WebHosting Guy
Tube Steak
Catbeller
from the WSJ
© 2016 KNIME.com AG. All Rights Reserved.
New Analysis Domains: Text, Networks, Images, …
21
Why only Prediction? Text Analysis, Sentiment,
Network Graphs, …
© 2016 KNIME.com AG. All Rights Reserved. 22
Social Media for User Experience:Twitter & Online Dating
© 2016 KNIME.com AG. All Rights Reserved.
New Prediction Objectives: Prices, Numbers, Failures, Restocking, …
26
Why only Churn? We could predict Stock Prices, Iot Mechanical
Failures, Traffic Numbers, Bike Restocking Needs, …
© 2016 KNIME.com AG. All Rights Reserved.
The Challenges
• Handling very large amounts of data created over time
• Forcing sensor-equipped objects (house or city) to learn, and therefore to become smarter
© 2016 KNIME.com AG. All Rights Reserved.
Even MORE of a Business Challenge
• Any Station without bikes for 1 hour:
$XXXX Per Violation
• Any Station with no free slots for 1 hour:
$XXXX Per Violation
© 2016 KNIME.com AG. All Rights Reserved.http://bikeportland.org/2013/03/10/behind-the-scenes-of-capital-bikeshare-84006
Capital Bikeshare Response
Over 3 years
307 Stations2963 Bikes19.4% Casual Bikers5.9m Bike Moves
© 2016 KNIME.com AG. All Rights Reserved.
The Goal
• Restocking alert signal
• 1 hour warning! Lag(flag-1)
© 2016 KNIME.com AG. All Rights Reserved.
Lean Restocking Alert System
The input feature subset with the smallest error (81% accuracy):
• Hour of the day
• Working day (Y/N)
• Current Bike Ratio
• Terminal (station code)
Past of bike ratio and weather infos do not seem to be relevant!
Is this because most bikers are registered members?
© 2016 KNIME.com AG. All Rights Reserved.
The hardest Case: No Class Examples
38
Can I still learn to predict if I have no Historical Examples for that
class?
© 2016 KNIME.com AG. All Rights Reserved. 39
The ultimate Challenge: No ExamplesAnomaly Detection in Predictive Maintenance
© 2016 KNIME.com AG. All Rights Reserved.
3. Dynamic Unsupervised Anomaly Detection.
Here some measures change over time till their values are not normal anymore. For example, while a motor is slowly deteriorating, one of the measurements might change till it gets out of control and the motor breaks. We want to stop the motor before it completely breaks producing even more damages.
This problem is similar to number 2 but slightly more challenging because it is not pattern based and it changes slowly over time.
Anomaly Detection: The Approach
© 2016 KNIME.com AG. All Rights Reserved.
Until Now! A Motor and its Sensors
28 time series from 28 sensors on 8 different parts of a mechanical engine.
A1 (input shaft vertical)
A2 (second shaft horizontal upper bearing)
A3 (third shaft horizontal lower bearing)
A4 (internal gear 275 degrees)
A5 (internal gear 190,5 degree)
A6 (input shaft bearing 150)
A7 (input shaft bearing 151)
M1 (torque KnM)
© 2016 KNIME.com AG. All Rights Reserved.
The Data
• Time Series are FFT-derived Spectral Amplitudes
• There is only one motor breakdown episodes on July 21, 2008
• The breakdown is visible only from some sensors and only in some frequency bands
• The engine was substituted with a new one after the breakdown
© 2016 KNIME.com AG. All Rights Reserved.
Data Visualization: Time Plots by Frequency Bands
A1-SV3 [500, 600] Hz
New motor piece
Old motor piece
Breaking pointJuly 21, 2008
Only some Spectral Time Series shows the break down
A1-SV3 [0, 100] Hz
© 2016 KNIME.com AG. All Rights Reserved.
Learn “normal”: Training Set
A1-SV3 [0, 100] Hz
A1-SV3 [500, 600] HzBreaking pointJuly 21, 2008
Only some Spectral Time Series shows the break down
31 August 2007
Training Set
Predictive Maintenance
© 2016 KNIME.com AG. All Rights Reserved.
Time Series Production: Second Level Alarms R Stacked Plot
Sep 2007
Mar 06 2008
May 05 2008
MA
(alm
(t))
Jul 2008
A7, SA1 [200,300]Hz
© 2016 KNIME.com AG. All Rights Reserved. 46
Time Series Production: Use KNIME to Send Alarms !!!
Using RESTful Services:
© 2016 KNIME.com AG. All Rights Reserved.
Any New Ideas?
• Find Analytics Areas for your Business: Customers, IoT, User Experience, …
• Adjust the use case to your Data
• Invent your Analytics Goals!
• Learn about the new techniques
• Spread the Knowledge through Communication
48
© 2016 KNIME.com AG. All Rights Reserved.
Thank You
Free copy of the e-book “KNIME Beginner’s Luck”From KNIME Press https://www.knime.org/knimepress
Promotional Code:
KNIMEBayArea_Meetup2017