bluecoin: the robotic ear - home - stmicroelectronics · 2021. 7. 19. · •bluetooth 4.1,...
TRANSCRIPT
BlueCoin: The Robotic Ear
Edoardo GallizioMicrosystems Product Marketing Manager
STMicroelectronics
BlueCoin: The Robotic Ear
SoundLocalization
Embedded Processing
Orientation, 3D TrackingAbsolute Altitude
Acoustic Beamforming & Source Localization
Bluetooth Low Energy Connectivity
BlueCoin: The Robotic EarAugmented hearing, ToF and motion sensing• Features
– Advanced audio processing• Sound Source Localization• Beamforming
– Wide band audio over BLE (BlueVoice)– 10 DOF Sensor fusion + Contextual Awareness
• Inertial, environmental, MEMS Microphones
– Complete Reference Design• SW Compatible with STM32 ODE• CoinStation + small Cradle
25mm
Sense
Connect
BlueCoin: The Robotic Ear• Main components
– STM32F446• ARM Cortex-M4F@180MHz - 128KB RAM
– u4 Microphone Array (4x MP34DT04)– Bluetooth-Low-Energy radio (BlueNRG-MS)
• Bluetooth 4.1, multiple role simultaneously
– 6+3 axis inertial module (LSM6DSM+LSM303AGR)– Absolute pressure sensor (LPS22HB)– 2x VL53L0X Time-of-Flight (ToF) ranging sensor
BlueCoin + battery
BlueCoin: Open Software Support
4 x Microphones
Accelerometer + Gyroscope
BLE
STM32F4 ARM Cortex-M4
HW-SNS-ALLMEMS1: STM32 ODE function pack for IoT node with BLE connectivity, digital microphone, environmental and motion sensors
Sense
E-compass
Pressure SensorConnect
Processing
BlueCoin: STEVAL-BCNKT01V1
Sensors
Ultra Low PowerConnectivity
Low-Power MCU
4 x MP34DT04
LPS22HB
LSM6DSMLSM303AGR
STM32F446
BlueNRG-MS
Motion MEMS
Environmental sensors
MEMS microphone array
High-Performance brain
Sensor fusion
Bluetooth Smart
Power/Battery Management Li-ion Battery Charger STBC03J
BlueCoin: The Robotic Ear
4 x MP34DT04-C1
BOTTOM VIEW TOP VIEW
LSM6DSM/L
ExpansionConnectors
2 x Buttons
STBC03J
SMD Antenna
BLUENRG-MS
STM32F446
LPS22HB
LSM303AGR 8 x LEDS
Ø 2
5 m
m
Expansions:I2S, I2C, USB, TIMx,
CAN, UART, SWD, SDIO
BALF-NRG-01D3
Enabling Multiple SW Libraries• Example project in source code built on STM32Cube software technology• Includes acoustic Beam Forming, Echo Cancellation, and Source Localization. • Immediate test and performance evaluation
Beamforming
Acoustic Echo Cancellation
reference audio
Source Localization4-MEMSmicrophones array
Audio SW IP and Eco-system
AudioAnalytics
StatisticalDereverberation
Auto Gain Control
Trigger ASR
Noise Reduction
reference
- Voice Activity Detection - Statistical moments- Noise estimation
3rd party ASR
cloudembedded
MEMS microphone array
osxAcousticSL
osxAcousticBF osxAcousticEC
Each osxAcoustic library may be easily replaced by 3rd party SW IP
All are released under free evaluation and production licensing
Sound Source Localization LibraryTwo algorithms implemented
• XCORR• GCC-PHAT
A simple Voice Activity Detector is included, based on energy threshold.
– Avoids false recognitions in case of low signal energy
Range 2 microphones cover a range of 180°
4 microphones cover a range of 360°
Acoustic Echo CancellationRemoves echo of playback audio in speech capture application
AEC(estimates
room reverberation)
Reverberant Room
Known Audio Sourcee. g. music / voice
Single Microphone application
STM32 is connected to both the microphone and the loudspeaker
The Open.AUDIO AEC library is an optimized STM32 port based on the Open Source project Speex: http://www.speex.org/
BlueCoin – Audio Augmented for IoT
BlueCoin Development
Platform
Reference Software and Applications
Cloud-based Conversational
Interface
Advanced Audio Capture frontend
