Neural Networks on STM32Artificial Intelligence Solutions

The Key Steps Behind Neural Networks 3

2

5

Process & analyze new data using trained NN

1

Capture data

4

Convert NN intooptimized code for MCU

3

Train NN Model

Neural Network (NN) Model Creation Operating Mode

Clean, label dataBuild NN topology

ST Toolbox for Neural Networks 4

Process & analyze new data using trained NNCapture data

Convert NN intooptimized code for MCU

Clean, label dataBuild NN topology

STM32CubeMX ExtensionAI Conversion Tool

5

Input your framework-dependent, pre-trained Neural Network into the STM32Cube.AI conversion tool

Automatic and fast generation of an STM32-optimized library

STM32Cube.AI offers interoperability with state-of-the-art Deep Learning design frameworks

Convert NN intooptimized code for MCU

Train NN Model

*

Process & analyze new data using trained NN

*

Collecting Data& Architecting a NN Topology

6

Selected partners Neural Networks engineering services support.Data scientists and Neural network architects.

ST BLE Sensor mobile phone applicationCollect and label data from the SensorTile.

Clean, label DataBuild NN topology

Services provided by Partners ST tools to support

ST BLE Sensor

Capture data

ST Toolbox for Neural NetworksMore Than Just a Conversion Tool

7

• Function packs for quick prototyping• Audio, Motion and Vision examples

• STM32 Community with dedicated Neural Networks topic

• For support and idea exchange

Convert NN intooptimized code for MCU

Process & analyze new data using trained NN

STM32 Solutions for AIMore Than Just the STM32Cube.AI

8

An extensive toolbox to support easy creation of your AI application

AI extension for STM32CubeMXto map pre-trained Neural Networks

Software examples for Quick prototypingAudio, Motion and Vision Function packs On ST development Hardware

STM32 Community with dedicated Neural Networks topic

STM32 AI Partner Program with dedicated Partners providing Machine or Deep Learning engineering services

Trainings, hands on, MOOCs andpartners videos

Example Form Factor Hardwareto Capture and Process Data

9

Motion MEMSMotion MEMS

www.st.com/SensorTilewww.st.com/SensorTile-edu

SensorTile

Process & analyze new data using trained NNCapture data

Fast Go to Market Moduleto Capture Data with More Accuracy

10

www.st.com/SensorTileBox

More advanced, high accuracy and low power sensors• First Inertial module with Machine Learning capabilities.• Motion (accelerometer and gyroscope, magnetometer) and slow motion (inclinometer)• Altitude (pressure), environment (pressure, temperature, humidity, compass) and sound

(sound and ultrasound analog microphone)• Microsoft IoT services ready to make available on a web dashboard the result of the

embedded processing

SensorTile.Box

Microsoft IoTServices ready

Process & analyze new data using trained NNCapture data

Distributed AI: Sensor + STM32 Optimize Performance and Power Consumption

11

FSMup to 16

MLCup to 8 Raw Data

Event Decision

FSM and MLCRe-configuration

Inertial SensorNew LSM6DSOX

Deep LearningNeural NetworksMachine Learning

• More advanced and complex NNs• Decisions on multiple sensors• NN input can be sensor data and/or

sensor Machine Learning decisions• Multiple Neural Networks support• Actuation & communication

Smart Sensorwith Machine Learning Core

Smart STM32second level of AI processing

• Best ultra-low-power sensing at high performance:• 550µA (gyroscope and accelerometer) 200µA less than closest competitor

• 20~40µA (Accelerometer only for HAR)• Efficient Finite State Machines: 2µA• Configurable Machine Learning Core: 4~8µA

Form Factor HardwareAI IoT Node for More Connectivity

12

More debug capabilities• Integrated ST-Link/V2.1• PMOD extension connector• Arduino Uno extension connectors

Sub-1GHz

Wi-Fi

Dynamic NFC Tag+

https://www.st.com/IoTnode

IoTNode

Process & analyze new data using trained NNCapture data

STM32Cube.AITool Roadmap

STM32Cube.AI Roadmap 14

2019 Jul 2020

+

IoTNode

NN Training

ToolsSupported

softwareExamples

SensorTile.Box STM32H747 Discovery Kit

+

• Floating Point Support

• 8-bit Quantization• TensorFlow for MCU• Command line interface• UI Improvements• Additional layers

Oct

• Integer Quantization• External memory support

(weights & activations)• Lambda layers support

(Alpha customers only)

• support introduction• Additional layers• Debug improvements

OpenMV H7 CAM

FP-AI-VISION1support

NN libraries generated by STM32Cube.AI

compatible OpenMV MicroPython ecosystem

FP-AI-SENSING1support

FP-AI-SENSING1support

FP-AI-SENSING1support

SensorTile

+

Function Packs(Software examples)

Audio Scene Classification (ASC) Audio Example in FP-AI-SENSING1 Package

16

Audio Data capture Data stored on the device SD card for future learning

Inference result displayed on mobile app

Inferences running on the microcontroller

Embedded audiopre-processing

NN & example dataset provided

Indoor, Outdoor, In vehiclelabelling

3 classesLabelling controlled

by smartphone application

Human Activity Recognition (HAR)Motion Example in FP-AI-SENSING1 Package

17

Motion Data Capture Stationary, walking, running, biking, driving labelling

Inference result displayed on mobile app

Inferences running on the microcontroller

Embedded motionpre-processing

5 classes example

NN & example dataset provided

Data stored on the device SD card for future learning

Labelling controlled by smartphone application

Image ClassificationVision Example in FP-AI-VISION1 Package

18

Inferences running on the microcontroller

Embedded imagepre-processing (SW) on

the STM32H747

NN & example dataset provided

Pizza99%

150ms

Enjoy the food classification demo• Default demo based on 18 classes

(224x224 RGB pictures)• Several camera image output size possible

Inference result displayed on STM32H747

Discovery board LCD display

Full end-to-end optimized software example• from camera acquisition to image pre-processing before feeding

the NN• Multiple memory mapping possibilities to optimise and test

impact on performances• Retrain this NN with your own dataset• Quantize your trained network to optimized inference time and

memory usage

Making AI Accessible Now 19

World 1st

Cortex-M MCU

World 1st

Cortex-M Ultra-low-power

1st High Perf.120 MHz, 90nm

1st High Perf.Cortex-M4 168 MHz

Entry CostCortex-M0

1st Mixed SignalDSP + Analog

Cortex-M4

Entry CostUltra-low-power

World 1st

Cortex-M7

LeadershipUltra-low-power

Cortex-M4

447 ULPBench™#1 ULP

#1 Performance

2400 CoreMark

Ultra-low-powerExcellence

Dual-core, multi-protocol

and open radio

Introduction of Cortex-M33Excellence in ULPwith more security

Mainstream Cortex-M0+ MCUs

Efficiency at its best!

20182007 2009 2010 2011 2012 2013 2014 2015 2016 20192017

First STM32 MPUDual Cortex-A7 + Cortex-M4

STM32 meets Linux

Compatible with Deep LearningSTM32Cube.AI ecosystem

Compatible with Machine Learning Partner ecosystems

More than 40,000 customers Over 4 Billion STM32 shipped since 2007

Leader in Arm® Cortex®-M 32-bit General Purpose MCU

For More Information 20

www.st.com/STM32CubeAI