![Page 1: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/1.jpg)
1© 2014 The MathWorks, Inc.
Building a Big Engineering Data Analytics System
using MATLAB
Arvind Hosagrahara
Principal Technical Consultant
Tel: 310-819-3960
![Page 2: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/2.jpg)
2
Trend: Data Economy
“Information is the oil of the 21st
century, and analytics is the combustion engine”
Peter Sondergaard, Gartner Research
Develop
Predictive ModelsAccess and
Explore DataPreprocess Data
Integrate Analytics
with Systems
![Page 3: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/3.jpg)
3
Problem Statement
Can MATLAB® scale up and meet increasingly demanding fleet test data
analytics requirements?
Is it possible to build data analytics algorithms in a flexible, scalable manner
and yet satisfy production requirements?
![Page 5: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/5.jpg)
5
A quick recap of our connected cars
OBD2 Bluetooth 4G LTE
HTTP
Hadoop Ecosystem
MATLAB
Desktop
Apache, the Apache feather logo, Hadoop are registered trademarks or trademarks of the Apache Software Foundation
AmazonTM EC2TM
Deployed
MATLABLAMR Stack
![Page 6: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/6.jpg)
6
Fleet Data Analysis System (Test Bed Summary)
8-25 Mb per day
Non-telemetric data files
Up to 25 operators
660Mb of data and counting
![Page 7: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/7.jpg)
7
Insights (Engine Fuel Consumption and Efficiency)
![Page 8: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/8.jpg)
8
Insights (8 mile traffic)
Traffic Patterns (the case for roundabouts)
0.0351 Gal/car at the intersection
12 cars a minute on the average
A saving of 121.3 gallons of
gasoline per day if the traffic lights
were replaced with a round-about.
A rough saving of 4.5 million
pounds of CO2 per year.
![Page 9: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/9.jpg)
9
What has changed?
What has improved?
What is new?
![Page 10: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/10.jpg)
10
Infrastructure Changes
Enrichment of incoming streams Non-telemetric data
Drag/drop addition and
management of data
MATLAB with visualization tools
Direct Connectivity with MATLAB
Caching of data for performance
Upsized compute resources
Load balancers
Simulation sources
– MATLAB
– Simulink
Web
App
![Page 11: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/11.jpg)
11
Location and Methodology changes
Web enabled dashboards
Privacy and Security
Enabling relational query
Reference architectures offer
solutions with an emphasis on
selecting the right tool for the task
resulting in more efficient
workflows
![Page 12: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/12.jpg)
12
Insights from the field
![Page 13: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/13.jpg)
13
Study of Driving Patterns
“By the way, this is pretty neat data to look
at in MATLAB. Even without the GPS, I can
pinpoint (1) the moment I nearly clobbered a
deer this morning and (2) the merging onto
and off of route 9…”
![Page 14: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/14.jpg)
14
Performance
Bonneville Salt Flats
Waterford Hills
% Fetch the data
dataFile = urlwrite('http://ec2-54-187-132-60.us-west-
2.compute.amazonaws.com/upload.csv','trackdata.csv');
% Read the data into MATLAB
trackData = xlsread(dataFile);
% Isolate area of interest
idx = timeVec>1403714100426 & timeVec<1403717061373;
% Gather statistics
topSpeed = max(trackData(idx,7)); % in kmph.
% topSpeed =
% 167.4000
% Compute fuel usage
ccMin = trackData(:,17);
fuelUsedInCC = trapz(ccMin(idx))/(60);
fuelUsedInUSGal = 0.000264172052*fuelUsedInCC;
fuelCostInUSD = fuelUsedInUSGal*3.91;
% fuelCostInUSD =
6.1048
![Page 15: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/15.jpg)
15
Discovering problems in veracity of collected data
Software
modifications to give
higher peak ft-lbs of
torque
Increased boost
pressure and
optimized ignition
timing
Aftermarket software
was conditioning the
data to ensure
proper operation
Stock
Stage 1
![Page 16: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/16.jpg)
16
Operationalization of Analytics
![Page 17: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/17.jpg)
17
Product Changes (1) - Big Data Capabilities in MATLAB
Memory and Data Access
64-bit processors
Memory Mapped Variables
Disk Variables
Databases
Datastores
Platforms
Desktop (Multicore, GPU)
Clusters
Cloud Computing (MDCS on EC2)
Hadoop
Programming Constructs
Streaming
Block Processing
Parallel-for loops
GPU Arrays
SPMD and Distributed Arrays
MapReduce
![Page 18: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/18.jpg)
18
Product Changes (2) – Datastores and MapReduce
Datastore
Incremental data processing
feature for reading collections of
data containing tabulartext or
keyvalue pairs.
Enables access to relational
databases using Database
Toolbox™
MATLAB® MapReduce
Allows analysis of out-of-memory
data.
Deployment of mapreduce
algorithms to:
– Serial Mapreduce using local workers
– Parallel Computing Toolbox™
– MATLAB® Distributed Computing
Server™
– Hadoop® using the MATLAB
Compiler
![Page 19: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/19.jpg)
19
Out of memory processing using Datastores
![Page 20: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/20.jpg)
20
Studying ride quality using MapReduce
![Page 21: Building a Big Engineering Data Analytics System …...Building a Big Engineering Data Analytics System using MATLAB Arvind Hosagrahara Principal Technical Consultant Arvind.Hosagrahara@mathworks.com](https://reader033.vdocuments.site/reader033/viewer/2022042310/5ed73142c30795314c175ea6/html5/thumbnails/21.jpg)
21
Conclusion
Single stack, open and extensible analytics toolset that plays well with other
technologies
New features support clean workflows for use in production systems and
with Big Data
A capacity for complexity