machine learning for manufacturing and materials
TRANSCRIPT
![Page 1: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/1.jpg)
Machine Learning for Manufacturing and Materials
Prof . Randy Paf fenro thA s s o c i a t e P r o f e s s o r o f
M a t h e m a t i c a l S c i e n c e s , C o m p u t e r S c i e n c e a n d D a t a S c i e n c e ,
W o r c e s t e r P o l y t e c h n i c I n s t i t u t e
Predictive Maintenance November 23, 2020
![Page 2: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/2.jpg)
Students and collaborators!
Chong ZhouWenjing LiNitish Bahadur Kelum Gajamannage Rasika Karkare Matt Weiss
Louis Scharf
Anura Jayasumana
Les ServiPartha Pal
BBN/Raytheon
Josh UzarskiYingnan Liu
Patricia Medina
Robert Casoni
Lane Harrison Alex Wyglinski
![Page 3: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/3.jpg)
http://www.azquotes.com/quote/850928
We are a machine learning research
group that focuses on problems in the
physical sciences
![Page 4: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/4.jpg)
A selection of current applications
Chemical Sensors
Supported by The U.S. Army CCDC-SC
Supported by Nanocomp
Technologies
Nano-materialsCyber Warfare
Supported by BBN/Raytheon
and MITRE Corp
Manufacturing
Supported by The Advanced Casting
Research Center
![Page 5: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/5.jpg)
A selection of current applications
Chemical Sensors
Supported by The U.S. Army CCDC-SC
Supported by Nanocomp
Technologies
Nano-materialsCyber Warfare
Supported by BBN/Raytheon
and MITRE Corp
Manufacturing
Supported by The Advanced Casting
Research Center
![Page 6: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/6.jpg)
Students and collaborators!
Chong ZhouWenjing LiNitish Bahadur Kelum Gajamannage Matt Weiss
Louis Scharf
Anura Jayasumana
Les ServiPartha Pal
BBN/Raytheon
Josh UzarskiYingnan Liu
Patricia Medina
Robert Casoni
Lane Harrison Alex Wyglinski
Rasika Karkare
![Page 7: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/7.jpg)
Root cause analysis of foundry defect formations drives appropriate corrective action for overall product quality enhancement
Porosity35 %
Other defects32 %
Supplier quality22 %
Tool costs/life11 %
Types of DefectsDepiction of Severe Internal Porosity
7
Source: NADCA & Ultraseal International.
![Page 8: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/8.jpg)
There are many terms flying around these days.
https://sastat.org.za/sasa2017/big-data-dictionary
![Page 9: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/9.jpg)
Data vs. ML approaches quadrant
9
Good Physical Model
Good Data
Bad Data
• Small size
• Unbalanced
• Biased
• Missing
• Irrelevant
Features
• Anomalous
Bad Physical Model
Source: Aref et.al., Clinical applications of machine learning in cardiovascular disease and its relevance to cardiac imaging
Source: Barata, Using 3D visualizations to tune hyperparameters in ML models
• Balanced data
• Large size
• Unbiased
• Complete
• Noise-free
• Relevant Features
Full Physics PDE model
![Page 10: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/10.jpg)
Data vs. ML approaches quadrant
10
Good Data
Bad Data
• Small size
• Unbalanced
• Biased
• Missing
• Irrelevant
Features
• Anomalous
Bad Physical Model
Source: Aref et.al., Clinical applications of machine learning in cardiovascular disease and its relevance to cardiac imaging
Source: Barata, Using 3D visualizations to tune hyperparameters in ML models
• Balanced data
• Large size
• Unbiased
• Complete
• Noise-free
• Relevant Features
Full Physics PDE model
Good Physical Model
![Page 11: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/11.jpg)
Deep learning vs Machine Learning4
![Page 12: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/12.jpg)
Results – Comparison with RF and XGB12
![Page 13: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/13.jpg)
Criteria for good DL approaches
Source: Jason Brownlee, How touse Learning Curves to Diagnose Machine Learning Model Performance
13
Bias-Variance Tradeoff
Model choice based on the size of the dataset
Underfit Robust Overfit
![Page 14: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/14.jpg)
Challenges in data collection
14
Semi-Supervised Unbalanced and small-size Heterogeneous
Siloed Multi-modal data
![Page 15: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/15.jpg)
Key idea: Need to work together!
15
A s s o c i a t e P r o f e s s o r o f M a t h e m a t i c a l S c i e n c e s , C o m p u t e r S c i e n c e a n d D a t a S c i e n c e
Professor Diran ApelianMetal Processing InstituteDirector, Advanced Casting Research Center (ACRC)
![Page 16: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/16.jpg)
Going “into the weeds”…
16
![Page 17: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/17.jpg)
Challenges in data collection
17
Semi-Supervised Unbalanced and small-size Heterogeneous
Siloed Multi-modal data
![Page 18: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/18.jpg)
Dealing with missing and noisy data in manufacturing processes
18
f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 f11 f12 f13
r1
r3
r3
r4
r5
r6
r7
r8
r9
Noise as an item
Noise as a feature
Noise as a record
Such algorithms exist!
![Page 19: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/19.jpg)
19
Input X
N * m
Hidden
LayerN * k
Reconstruction
N * m
Cost1
Outlier Filter SN * m
Cost2
Wm * k
Wk * m
T
There is hope!Robust Hadamard Autoencoders
Karkare et.al, Blind Image Denoising and inpainting using Robust Hadamard Autoencoders, in progress
![Page 20: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/20.jpg)
Standard Autoencoder(sae)-tsne – Fully Observed Data Projection
20
![Page 21: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/21.jpg)
Hadamard Autoencoder(ha)-tsne20% Missing Data
21
![Page 22: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/22.jpg)
Ha-tsne40% Missing Data
22
![Page 23: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/23.jpg)
Ha-tsne60% Missing Data
23
![Page 24: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/24.jpg)
Conclusions
• Machine learning is a powerful tool for predictive analytics• But, like any tool, it must be used properly
• Manufacturing data is different than the types of data that machine learning is used on• Semi-supervised
• Unbalanced
• Heterogenous
• However, when the correct algorithms are selected, machine learning can be used to solve difficult problems.
24
![Page 25: Machine Learning for Manufacturing and Materials](https://reader031.vdocuments.site/reader031/viewer/2022012211/61df10a1f7010223531cce87/html5/thumbnails/25.jpg)
Acknowledgements
25