solving industrial data integration with machine intelligence
TRANSCRIPT
1 Bit Stew Systems Inc. Confidential and Proprietary
Solving Industrial Data Integration with Machine Intelligence
By: Mike Varney
Executive Director, Product Management & Strategic Initiatives at Bit Stew
2 Bit Stew Systems Inc. Confidential and Proprietary
The Problem with Big Data
With 50 billion devices connected to the Internet by 2020 rapid data proliferation will continue to choke the progress of data analytics projects.
Analytics is only the tip of the iceberg. Data integration, data quality, and challenges in extracting value lurk beneath the surface and are amplified by the growth of Industrial data from IIoT.
Analytics are the tip of the iceberg
Data cleansing and quality challenges lurk beneath the surface
3 Bit Stew Systems Inc. Confidential and Proprietary
Two Different Worlds, One Common Problem
Operational Technology (OT)
Information Technology (IT)
Industrial IoT
• Asset Performance • Real-Time Status Monitoring• Diagnostics & Service• Predictive Maintenance• Software Updates
• Data Integration• Data Cleansing • Systems Management• Analytics• Value-Added Services
0101010010101
0101
4 Bit Stew Systems Inc. Confidential and Proprietary
Hidden Cost of Integration
• Industrial data is not easy• Complex data, variable size, frequency, and format • Disparate sources • No view of data relationships across the enterprise • Often rely on traditional ETL or BI tools• Large integration teams costing thousands of dollars
5 Bit Stew Systems Inc. Confidential and Proprietary
Complex Environments Lead to Complex Data
Why is Industrial Data so complicated?
Variety of source types Messy Data Complex Data Relations
Different Sizes Massive Volume Varied Frequency
6 Bit Stew Systems Inc. Confidential and Proprietary
Traditional ETL Compounds the Problem
This model does not work for large volumes of complex OT and IT data
Extracts source data without contextual understanding
Then a team of integrators, architects, and data scientists apply transformations
After which, finally it can be provided to business and operational teams
7 Bit Stew Systems Inc. Confidential and Proprietary
Why MIx Core?
Small teams can solve BIG industrial problems with MIx Core™
• No longer need large and costly integration teams
• IoT endpoints are being added at an accelerated pace that is unmanageable by human computing
• Machine Intelligence solves the compounding data problem in Industrial environments
8 Bit Stew Systems Inc. Confidential and Proprietary
Using Machine Intelligence to Automate Data Integration
• Comprehensive Semantic Model
• Automatic sensing of data and mapping to Semantic Model
• Dynamic and adaptive to source data
• Examine, transform and clean data
MachineIntelligence
Create Semantic ModelSiloed Data Sources
Intelligent Integration to Target Model
• Integrate with APIs, services, files, streams and network traffic
• Embedded guaranteed messaging
• Interactive UI for viewing and editing
9 Bit Stew Systems Inc. Confidential and Proprietary
Step 1: Create Feature Vector
– Source data is read and sent to the Hierarchical Feature Extractor
– The Hierarchical Feature Extractor produces Feature Vectors
– The Hierarchical Feature Extractor can be customized for domain specific feature extraction and machine learning.
– Feature vectors are used to fingerprint and classify the input data.
Step 2: Store denormalized data into the Data Index
– The Feature Vectors are stored into the Data Index
– Data is denormalized and fed into the Relationship Association
– A Relationship Matrix is created and stored into the Data Index
How it Works?
10 Bit Stew Systems Inc. Confidential and Proprietary
How it Works?
Step 3: Create Semantic Model
– The Feature Vectors are sent to the Field Classifier
– The Field Classifier uses supervised and unsupervised machine learning algorithms to relate and map data
– Each algorithm gets a vote to determine the best model
– The related and mapped data is sent to the Modeler to create a Semantic Model
Step 4: Publish the Semantic Model to Target Systems
– Model is published and available for use in the platform
– The semantic model may be exported to 3rd party systems for further analysis
– Changes to published model are tracked over time for the system to learn how relationships in the data change over time
11 Bit Stew Systems Inc. Confidential and Proprietary
Learn more about Bit StewVisit www.info.bitstew.com
Follow us on
Download the whitepaper: http://info.bitstew.com/whitepaper-mike-machine-intelligence
12 Bit Stew Systems Inc. Confidential and Proprietary
Mike Varney spent over 20 years in the US Navy, where his experience included commanding the most advanced nuclear-powered submarines in complex operations around the globe, leading a special operations team in reconstruction efforts in Afghanistan, and directing a Naval Operations Centre. He has also served as a Strategic Advisor for the US Department of Defense, a Senior Evaluation Officer at nuclear power plants, and an advisor to companies providing smart grid technologies to the energy industry.
Mike holds Bachelor of Science degrees in Nuclear and Marine Engineering as well as Master of Science degrees in Engineering Management and National Security Strategy. Today, Mike is the Executive Director, Product Management & Strategic Initiatives, where he leads the strategy for Bit Stew Systems MIx Core platform, MIx Developer Network and Bit Stew University.
About the Author