AI + ENGINEERINGFrom Physical to Virtual Sensors using DEEP LEARNING

#aiwtb

We developed a novel approach by adapting Deep Learning algorithms to the classical Engineering applications like signal filtering, sensor fusion and control systems.

This gave us a competitive advantage over the existing solutions.

AI + ENGINEERING

In Addfor, we developed a novel approach by adapting the state of the art Artificial Intelligence algorithms to the classical Engineering applications like signal filtering, sensor fusion and control systems.This gave us a competitive advantage over the existing solutions.

PRESENTER NOTES

add-for.com

This is made possible by what we call

COMBINATORIAL INNOVATION

We Re-Engineer the Deep Learning Algorithms developed for the weband adapt it to work on generic sensor data.

Many big companies like Google, Facebook and Baidu are Open-Sourcing their most powerful algorithms that are build mainly for Natural Language and Image understanding.This is an enormous opportunity for small and medium high-tech companies to develop Smart Vertical Applications: we call this process COMBINATORIAL INNOVATION.Since all the Heavy Work of creating the fundamental algorithms is already done, developing powerful industrial application is just a matter of Combining the existing code in a proper way.

PRESENTER NOTES

add-for.com

HUGE demand for Deep Learning Vertical Applicationsin Industrial Environments

Examples - some of our projects:

We’ve seen there is a huge demand in industry for this kind of applications: everyone is collecting large amounts of data but very few companies have the technical knowledge to extract the information that the customer need.This is just a brief list of the projects we have developed for our customers:

PRESENTER NOTES

add-for.com

TIRE DOT CODE RECOGNITION

DOT XB 4J R523 4213

This is a quite standard application. Basically is OCR with some special twists since the tire can be shiny or dull, scratched or dirty.The Deep Convolutional Neural Network are great here !

PRESENTER NOTES

add-for.com

Target Detected Class: Missile Launcher Type: MAZ-543 Uragan

MILITARY TARGETS DETECTION AND RECOGNITION

The Detection and Recognition of military target is a more interesting task.Here the problem was the lack of training data.We managed to put tighter a solution using Unreal Engine to automatically generate the training dataset and the ground truth. The final algorithm was deployed on an NVIDIA TX1 embedded platform.

PRESENTER NOTES add-for.com

ADVANCED TRACTION CONTROLSVEHICLE DYNAMICS

DRIVING STYLE DETECTIONMost of our applications are developed for the Automotive Industry: our Deep Learning algorithms are embedded in the can Electronic Control Units.

PRESENTER NOTES

add-for.com

VIRTUAL THERMOMETERSVIRTUAL FLOW-METERSEMISSION CONTROLSTURBO SPEED

The same happens for the powertrain: here we develop both Real-Time algorithms for the ECUs and software for the analysis of the data sent by the vehicles to a centralized data center.

PRESENTER NOTES

add-for.com

ADVANCED NONLINEAR CONTROLSWASHING MACHINES

The Home Appliances market is another huge nice for deep learning: from washing machines to dryers.

PRESENTER NOTES

add-for.com

ADAPTIVE LIGHTING CONTROLSEARLY FAULT DETECTION

Prognostics and Early Fault Detection is one of the major request we have from our customers.

PRESENTER NOTESadd-for.com

PREDICTIVE MAINTENANCEPRODUCTION FORECASTA similar application is the production forecast and the predictive maintenance of large Photovoltaic Plants. Here one common request is the optimization of the cleaning operations.

PRESENTER NOTES

add-for.com

TEMPERATURE FORECASTINGHVAC OPTIMIZATIONThe last application is the one on which our coding example is based: Temperature Forecasting in Buildings. This is used to feed the predictive control logic of the HVAC systems.

PRESENTER NOTES

add-for.com

AI (RNN) ENGINEERING+

u y+

σ

φ π + X π φ

σ π

σ

Input Node

Input Gate

Output Gate

Forgetting Gate

Output Node

DATA SCIENTIST vs ENGINEER ? You Need Both

Now, who do you trust to develop such Vertical Applications: the Engineer or the Data Scientist?The answer is that you need both: the common engineer don’t know the recent development in Data Science and hasn’t been trained in that field.On the other side, the Data Scientist struggles when it comes to deal with real sensor data and machinery.

PRESENTER NOTES

add-for.com

u y+ φ π + X π φ

σ π

σ

Input Node

Input Gate

Output Gate

Forgetting Gate

Output Node

σ

WHAT ENGINEERSKNOW VERY WELL

WHAT ENGINEERSKNOW VERY WELL

WHAT ENGINEERSDON’T KNOW

The Engineer is great in doing Data cleaning, filtering and transformation to make easier the task for the Data Scientist.Our recipe is to inject in the project as much Engineering Knowledge as we can, as far as we are absolutely sure about our theories and formulas.Then, when the going gets tough, the Data Scientist gets going…

PRESENTER NOTES

Lets focus on FORECASTING

add-for.com

t

y

t0

future

Typical approach with FFNNThe typical Feed Forward Neural Network

approach will be to feed the net with the sensor readings and train it to predict the future value.

PRESENTER NOTES

add-for.com

t

y

t0

Dimensionality problem The problem is that in the real world the systems are complex and your hypothetical network will need so many

time samples to figure out an accurate prediction.

PRESENTER NOTES

add-for.com

Recurrent Neural Networks RNN

u(t) X y(t)

Mind This LOOP !!

The solution are the Recurrent Neural Networks: here you have internal states that keep memory of the incoming data.

PRESENTER NOTES

add-for.com

Xu(t0) y(t0)

Xu(t1) y(t1)

Xu(t2) y(t2)

Xu(tn) y(tn)

Xu(t2) y(t2)

Something Happens at a certain moment in time

The INTERNAL STATE keeps the event in memory

But the MEMORY of the event is soon forgotten

TheVANISHING GRADIENT PROBLEM

TIME FLOW

But now you face a second problem: the Vanishing Gradient.Basically, if something happens at a certain time, the INTERNAL STATE keeps memory of that event just for few time steps and soon forget it.This is a problem when the system should remember important events that happened far in the past

PRESENTER NOTES

Input Node

Input Gate

Output Gate

Forgetting Gate

Output Node

u

y

X

σ

φ

Input

Output

State

Sigmoid [0 ÷ 1]

Tanh [-1 ÷ +1]

⊕+

π

Sum

Concatenation

Product

σ

σ

σ

φ φu yX⊕ +

π

ππ

Long-Short Term Memory LSTM

The good news is that there is a class of network topologies that manage the Vanishing Gradient Problem very well: the Long-Short Term Memory also known as LSTM.Here there are two fundamental elements:The Hyperbolic Transformation layers (displayed in yellow) are magnifying glasses (or deforming mirrors) that stretch or squish the incoming vector. Mathematically they are called HOMEOMORPHISMS.The Sigmoid Layers instead are gates that allows just selected components of the vector to pass through.

PRESENTER NOTES

add-for.com

Input Node

Input Gate

u

y

X

σ

φ

Input

Output

State

Sigmoid [0 ÷ 1]

Tanh [-1 ÷ +1]

⊕+

π

Sum

Concatenation

Productσ

φu X⊕ +π

Long-Short Term Memory LSTM

The Internal State sees the input through the deformation induced by the Input Node: the most important part of the input vector are enhanced by the Hyperbolic Tangent, and the least important are reduced.Moreover the view is filtered by the Input Gate: this way, the useless signals are filtered out.

PRESENTER NOTES

add-for.com

Forgetting Gate

u

y

X

σ

φ

Input

Output

State

Sigmoid [0 ÷ 1]

Tanh [-1 ÷ +1]

⊕+

π

Sum

Concatenation

Product

σ

u X⊕ +

π

Long-Short Term Memory LSTM

This is the forgetting section: the forgetting gate defines which component of the vector that is memorized by the internal state should be forgotten and how fast.

PRESENTER NOTES

add-for.com

Output Gate

Output Node

u

y

X

σ

φ

Input

Output

State

Sigmoid [0 ÷ 1]

Tanh [-1 ÷ +1]

⊕+

π

Sum

Concatenation

Product

σ

φu yX⊕ π

Long-Short Term Memory LSTM

And finally the output section: the output is a gated nonlinear transformation of what is stored inside the Internal State.

PRESENTER NOTES

add-for.com

How to build the Input Vector for the LSTM

Of course there are many things to consider while working with real-world signals and we don’t have enough time to go thru every detail but we can focus on one of the most important things: how to build the input vector.

PRESENTER NOTES

add-for.com

t

y PR

ESEN

T FUTUREPAST

This is a CONTROLLED VARIABLE: you know PAST and FUTURE

IF NEEDED here you can use a causal digital filter

The Target Signal should only be filtered with a NON-CAUSAL digital filter to avoid LAG

Say we want to forecast the internal temperature of a big building. You can imagine the temperature plotted over time by the BLUE LINE.Usually we can measure the at the present moment the quantity we want to forecast.Then there are other variables, say for example that the PURPLE line represents the external temperature and the GREEN line represent the number of people inside the building.The ORANGE line represent a CONTROLLED VARIABLE, for example it can be the SET TEMPERATURE for the HVAC. The important concept that you should take in mind is that you ALWAYS KNOW the future behaviour of the controlled variables.So, after the training, when you will make a forecast you will use the model to make a prediction under some assumption like: WHAT WILL BE THE TEMPERATURE if I WILL SET THE HVAC to 25% of its power for the next two hours?

PRESENTER NOTES

add-for.com

This is the INPUT VECTOR for the LSTM u

This is the TARGET VALUE for the LSTM (training) and PREDICTION (inference)

y

This is the TARGET VALUE for the LSTM (training) and PREDICTION (inference)

Future Values of the CONTROLLED VARIABLES can be shifted back to present

This is the (vanishing) PAST: you do not have to manage it: the internal states of the LSTM will manage (forget) it for you.

PR

ESEN

T FUTUREPASTThis is how the input vector should be build:The green, purple and blue values are the sensors readings at t0 (the time in which you make the prediction).The ORANGE value instead is the FUTURE value, because, since it is a controlled variable you know which value it will have in the future.

PRESENTER NOTES

add-for.com

Remember to Subscribe to theMACHINE LEARNING ITALY Meetup

To be Updated on Examples - Code - Benchmarks

meetup.com/it-IT/Machine-Learning-Italy

Now I pass the mike to Matteo that will show you how to implement the LSTM in Tensorflow and Neon.Just remember to subscribe our meetup if you want to receive updates when we publish example code and benchmarks.

PRESENTER NOTES

add-for.com

Visitadd-for.com/training-material

To download the code and the sample data

Contact Meit.linkedin.com/in/ebusto

Or visit our website to download the code we will show you today and be directed to our GitHub repository.If you have any additional question we can answer you just after the presentation or you can contact me directly on LinkedIn or by mail.

PRESENTER NOTES

add-for.com

Time for CODING

add-for.com