![Page 1: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/1.jpg)
1
Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks
Gonzalo Mateos, Ioannis Schizas and Georgios B. GiannakisECE Department, University of Minnesota
Acknowledgment: ARL/CTA grant no. DAAD19-01-2-0011
![Page 2: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/2.jpg)
2
Motivation Estimation using ad hoc WSNs raises exciting challenges
Communication constraints Limited power budget Lack of hierarchy / decentralized processing Consensus
Unique features Environment is constantly changing (e.g., WSN topology) Lack of statistical information at sensor-level
Bottom line: algorithms are required to be Resource efficient Simple and flexible Adaptive and robust to changes
Single-hop communications
![Page 3: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/3.jpg)
3
Prior Works Single-shot distributed estimation algorithms
Consensus averaging [Xiao-Boyd ’05, Tsitsiklis-Bertsekas ’86, ’97] Incremental strategies [Rabbat-Nowak etal ’05] Deterministic and random parameter estimation [Schizas etal ’06]
Consensus-based Kalman tracking using ad hoc WSNs MSE optimal filtering and smoothing [Schizas etal ’07] Suboptimal approaches [Olfati-Saber ’05], [Spanos etal ’05]
Distributed adaptive estimation and filtering LMS and RLS learning rules [Lopes-Sayed ’06 ’07]
![Page 4: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/4.jpg)
4
Problem Statement Ad hoc WSN with sensors
Single-hop communications only. Sensor ‘s neighborhood Connectivity information captured in Zero-mean additive (e.g., Rx, quantization) noise
Each sensor , at time instant Acquires a regressor and scalar observation Both zero-mean w.l.o.g and spatially uncorrelated
Least-mean squares (LMS) estimation problem of interest
![Page 5: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/5.jpg)
5
Centralized Approaches If , jointly stationary Wiener solution
If global (cross-) covariance matrices , available Steepest-descent converges avoiding matrix inversion
If (cross-) covariance info. not available or time-varying Low complexity suggests (C-) LMS adaptation
Goal: develop a distributed (D-) LMS algorithm for ad hoc WSNs
![Page 6: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/6.jpg)
6
A Useful Reformulation
Introduce the bridge sensor subset1) For all sensors , such that2) For , there must such that
Consider the convex, constrained optimization
Proposition [Schizas etal’06]: For satisfying 1)-2) and the WSN is connected, then
![Page 7: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/7.jpg)
7
Algorithm Construction Problem of interest
Two key steps in deriving D-LMS1) Resort to the alternating-direction method of multipliers
Gain desired degree of parallelization
2) Apply stochastic approximation ideasCope with unavailability of statistical
information
![Page 8: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/8.jpg)
8
Derivation of Recursions Associated augmented Lagrangian
Alternating-direction method of Lagrange multipliersThree-step iterative update process
Multipliers Dual iteration Local estimates Minimize w.r.t. Bridge variables Minimize w.r.t.
Step 1:Step 2:Step 3:
![Page 9: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/9.jpg)
9
Multiplier Updates Recall the constraints
Use standard method of multipliers type of update
Requires from the bridge neighborhood
![Page 10: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/10.jpg)
10
Local Estimate Updates Given by the local optimization
First order optimality condition
Proposed recursion inspired by Robbins-Monro algorithm
1) is the local prior error2) is a constant step-size
Requires Already acquired bridge variables Updated local multipliers
![Page 11: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/11.jpg)
11
Bridge Variable Updates
Similarly,
Requires from the neighborhood from the neighborhood in a startup phase
![Page 12: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/12.jpg)
12
D-LMS Recap and Operation In the presence of communication noise, for
Simple, fully distributed, only single-hop exchanges needed
Step 1:Step 2:
Step 3:
Sensor
Rxfrom
Tx
toBridge sensor
Txto
Rx
from
Steps 1,2:
Step 3:
![Page 13: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/13.jpg)
13
Further Insights Manipulating the recursions for and yields
Introduce the instantaneous consensus error at sensor
The update of becomes
Superposition of two learning mechanisms Purely local LMS-type of adaptation PI consesus loop tracks the consensus set-point
![Page 14: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/14.jpg)
14
Network-wide information enters through the set-point Expect increased performance with Flexibility
D-LMS ProcessorLocal LMS Algorithm
Sensor j
PI RegulatorTo
Consensus Loop
![Page 15: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/15.jpg)
15
Mean Analysis Independence setting signal assumptions for
(As1) is a zero-mean white random vector , with spectral radius
(As2) Observations obey a linear model where is a zero-mean white noise
(As3) and are statistically independent
Define and
Goal: derive sufficient conditions under which
![Page 16: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/16.jpg)
16
Dynamics of the MeanLemma: Under (As1)-(As3), consider the D-LMS algorithm initialized with .Then for , is given by the second-order recursion with and , where
Equivalent first-order system by state concatenation
![Page 17: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/17.jpg)
17
First-Order Stability ResultProposition: Under (As1)-(As3), the D-LMS algorithm whose positive step-sizes and relevant parameters are chosen such that , achieves consensus in the mean sense i.e.,
Step-size selection based on local information only Local regressor statistics Bridge neighborhood size
![Page 18: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/18.jpg)
18
Simulations node WSN, Regressors: i.i.d.Observations:
D-LMS: ,
True time-varying weight:
![Page 19: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/19.jpg)
19
Loop Tuning Adequately selecting actually does make a difference
Compared figures of merit: MSE (Learning curve): MSD (Normalized estimation error):
![Page 20: 1 Consensus-Based Distributed Least-Mean Square Algorithm Using Wireless Ad Hoc Networks Gonzalo Mateos, Ioannis Schizas and Georgios B. Giannakis ECE](https://reader036.vdocuments.site/reader036/viewer/2022062311/5a4d1b287f8b9ab0599983b8/html5/thumbnails/20.jpg)
20
Concluding Summary Developed a distributed LMS algorithm for general ad hoc WSNs Intuitive sensor-level processing
Local LMS adaptation Tunable PI loop driving local estimate to consensus
Mean analysis under independence assumptions step-size selection rules based on local information
Simulations validate mss convergence and tracking capabilities
Ongoing research Stability and performance analysis under general settings Optimality: selection of bridge sensors, D-RLS. Estimation/Learning performance Vs complexity tradeoff