deep learning for anomaly detection in manufacturing · • western electric or nelson rules •...
TRANSCRIPT
![Page 1: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/1.jpg)
David Katz, Data Scientist, TIBCO Software
Mike Alperin, Manufacturing Industry Consultant, TIBCO Software
July, 2018
Deep Learning for Anomaly Detection in Manufacturing
© Copyright 2000-2018 TIBCO Software Inc.
![Page 2: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/2.jpg)
2
•Anomaly Detection in Manufacturing•Deep Learning Autoencoder• Neural Networks• Autoencoders• Software Tools
•Virtual Demo
Agenda
© Copyright 2000-2018 TIBCO Software Inc.
![Page 3: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/3.jpg)
3
Anomaly Detection in Manufacturing
© Copyright 2000-2018 TIBCO Software Inc.
![Page 4: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/4.jpg)
4
• Detecting new problems• Supervised vs. Unsupervised Learning
• Some Types of Anomalies• Facilities Equipment - sensor & environmental data • Process Equipment – sensor FDC data• Process Results – Process history and measurements• Physical Defects – defect images & characteristics• Device and Product – PCM and Sort data
• A General Method1. Detect anomalies2. Cluster them3. Classify with fingerprints or signatures 4. Determine causes of anomaly classes 5. Develop Action Plans to address causes 6. Predict cluster for new material and intervene to mitigate potential problems
Anomaly Detection in Manufacturing
© Copyright 2000-2018 TIBCO Software Inc.
![Page 5: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/5.jpg)
5
Univariate Statistical Process Control
© Copyright 2000-2018 TIBCO Software Inc.
Detect changes from baseline –one variable at a time
Shewhart Process Control Charts• Statistically derived Control Limits• Western Electric or Nelson rules
• Automated Alerting
Individual – Moving Range Control Charts
![Page 6: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/6.jpg)
• Suppose we measured 2 parameters y1 and y2 (e.g., person’s height & 1/weight)• Univariate charts would not detect some obvious outliers• This happens in many real applications
The Power of Multivariate Control Charts
Bad Tester
© Copyright 2000-2018 TIBCO Software Inc.
![Page 7: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/7.jpg)
7
Univariate & Multivariate Methods
© Copyright 2000-2018 TIBCO Software Inc.
![Page 8: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/8.jpg)
8
Real-time equipment anomaly prediction & clustering
© Copyright 2000-2018 TIBCO Software Inc.
High Tech Manufacturing Accelerator
https://community.tibco.com/modules/high-tech-manufacturing-accelerator
M
6
![Page 9: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/9.jpg)
9
Deep Learning Autoencoder
© Copyright 2000-2018 TIBCO Software Inc.
![Page 10: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/10.jpg)
10
• Frank Rosenblatt, Cornell, inventor of the Perceptron.• Brain mechanisms and models.
• Why the explosion?• New algorithms and techniques
• Convolutional NN, Recursive NN, Generative Adversarial NNs
• New Hardware capabilities• GPU
• Multicore
• Clusters
• More Data
• New Tools from the Open Source world.
From Neural Networks to Deep Learning
© Copyright 2000-2018 TIBCO Software Inc.
cs231n.github.io/neural-networks-1
![Page 11: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/11.jpg)
11
• Create an identity transformation with constraints• Analogy to Principal Components – but much more
flexible/accurate.• Anomalies – the output is the reconstructed input, but it
does not fully match the original input => Reconstruction Error• Reconstruction Error:
• By component
• By sample.
Autoencoders
© Copyright 2000-2018 TIBCO Software Inc.
![Page 12: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/12.jpg)
12
• Fraud
• Credit• Natural Language
• Speech
• Video
• Manufacturing
© Copyright 2000-2018 TIBCO Software Inc.
AbstractSmart manufacturing refers to using advanced data analytics to complement physical science for improving system performance and decision making. With the widespread deployment of sensors and Internet of Things, there is an …
Autoencoder Applications
![Page 13: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/13.jpg)
13
•Sparse Autoencoders•Denoising Autoencoders•Generative Adversarial Networks•Variational Autoencoders
Autoencoders – Types and Variants
© Copyright 2000-2018 TIBCO Software Inc.
![Page 14: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/14.jpg)
14
• H2O DeepLearning• Simple Structure of networks – just specify number of fully-
connected layers (and optionally dropout)• Settings for Sparse data can outperform GPU.• H2O Deep Water Project –
• uses GPU but no longer being developed;• H2O recommends Keras for new projects.
• Keras • Front end for Tensorflow, CNTK, Theano, MXNet• Specify complex network topologies• Use different types of layers – CNN, RNN,…• Can leverage GPU
Deep Learning Software
© Copyright 2000-2018 TIBCO Software Inc.
![Page 15: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/15.jpg)
15
Virtual Demo
© Copyright 2000-2018 TIBCO Software Inc.
![Page 16: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/16.jpg)
Industrial Plant: Raw Time series Data
© Copyright 2000-2018 TIBCO Software Inc.
![Page 17: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/17.jpg)
Industrial Plant: Raw Time series Data
© Copyright 2000-2018 TIBCO Software Inc.
![Page 18: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/18.jpg)
Industrial Plant: Raw Time series Data
© Copyright 2000-2018 TIBCO Software Inc.
![Page 19: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/19.jpg)
19
Tag Training, Validation & Test Data Sets
© Copyright 2000-2018 TIBCO Software Inc.
![Page 20: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/20.jpg)
Variable Selection
© Copyright 2000-2018 TIBCO Software Inc.
![Page 21: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/21.jpg)
Model Configuration & Evaluation
Validation Error has clear minimumNote Distribution of Reconstruction Error
© Copyright 2000-2018 TIBCO Software Inc.
![Page 22: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/22.jpg)
Problems converging
Model Configuration & Evaluation
Problems Converging
© Copyright 2000-2018 TIBCO Software Inc.
![Page 23: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/23.jpg)
Model Configuration & Evaluation
© Copyright 2000-2018 TIBCO Software Inc.
![Page 24: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/24.jpg)
Anomalies & Component Signatures
© Copyright 2000-2018 TIBCO Software Inc.
![Page 25: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/25.jpg)
Anomalies & Component Signatures
© Copyright 2000-2018 TIBCO Software Inc.
![Page 26: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/26.jpg)
Anomalies & Component Signatures
© Copyright 2000-2018 TIBCO Software Inc.
![Page 27: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/27.jpg)
Anomalies & Component Signatures
Incident not detected on Univariate Chart
© Copyright 2000-2018 TIBCO Software Inc.
![Page 28: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/28.jpg)
Reconstruction Error
Identify Incidents Programmatically
© Copyright 2000-2018 TIBCO Software Inc.
![Page 29: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/29.jpg)
Elbow Plot of Cluster Config
© Copyright 2000-2018 TIBCO Software Inc.
![Page 30: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/30.jpg)
Cluster Similar Incidents, View Signatures
© Copyright 2000-2018 TIBCO Software Inc.
![Page 31: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/31.jpg)
31
Visit the TIBCO Industry 4.0page
To Learn & Do More
© Copyright 2000-2018 TIBCO Software Inc.
Visit the TIBCO Community Manufacturing Solutions page
Download AI & Machine Learning Manufacturing Solutions from the
TIBCO Exchange
![Page 32: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/32.jpg)
32
• Thanks to Dr. Thomas Hill, Venkata Jagannath, Glenn Hoskins and Nico Rode for their contributions to this work
• Thanks to Michael O’Connell, Steven Hillion and HeleenSnelting for their support and encouragement.
Acknowledgments
© Copyright 2000-2018 TIBCO Software Inc.
![Page 33: Deep Learning for Anomaly Detection in Manufacturing · • Western Electric or Nelson rules • Automated Alerting Individual – Moving Range Control Charts • Suppose we measured](https://reader033.vdocuments.site/reader033/viewer/2022060803/6086e6bce5dc123491117de7/html5/thumbnails/33.jpg)
Contacts:Mike Alperin
Manufacturing Industry [email protected]
David KatzData Scientist
Thank you!
Visit us in Booth 1021See this and other Manufacturing Demos
Learn more about the Technology
© Copyright 2000-2018 TIBCO Software Inc.