Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.

Course EE210B Spring Quarter 2015

1

Professor Ali H. Sayed UCLA Electrical Engineering

INFERENCE OVER NETWORKS

LECTURE #8: Useful Matrix Results

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 2

References

Appendix F (Useful Matrix and Convergence Results, pp. 761-776): A. H. Sayed, ``Adaptation, learning, and optimization over networks,'' Foundations and Trends in Machine Learning, vol. 7, issue 4-5, pp. 311-801, NOW Publishers, 2014.

Appendices C (Stochastic Matrices) and D (Block Maximum Norm): A. H. Sayed, ``Diffusion adaptation over networks,'' in Academic Press Library in Signal Processing, vol. 3, R. Chellapa and S. Theodoridis, editors, pp. 323-454, Academic Press, Elsevier, 2014.

Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.

Course EE210B Spring Quarter 2015

Kronecker Products

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 4

Kronecker Products

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 5

Kronecker Products

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 6

Kronecker Products

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 7

Block Kronecker Products

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 8

Block Kronecker Products

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 9

Block Kronecker Products

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 10

Block Kronecker Products

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 11

Block Kronecker Products

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 12

Block Kronecker Products

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 13

Block Kronecker Products

Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.

Course EE210B Spring Quarter 2015

Vector and Matrix Norms

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 15

Vector Norms

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 16

Matrix Norms

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 17

Matrix Norms

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 18

Equivalent Matrix Norms

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 20

One Useful Matrix Norm

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 21

One Useful Matrix Norm

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 22

One Useful Matrix Norm

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 23

One Useful Matrix Norm

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 24

One Useful Matrix Norm

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 25

One Useful Matrix Norm

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 26

Statement

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 27

Proof

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 28

Proof

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 29

Proof

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 30

Block Maximum Norm

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 31

Block Maximum Norm

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 32

Properties

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 33

Properties

Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.

Course EE210B Spring Quarter 2015

Jensen’s Inequality

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 35

Jensen’s Inequality

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 36

Jensen’s Inequality

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 37

Jensen’s Inequality

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 38

Jensen’s Inequality

Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.

Course EE210B Spring Quarter 2015

Perturbation Bounds On Eigenvalues

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 40

Weyl’s Theorem

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 41

Weyl’s Theorem

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 42

Weyl’s Theorem

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 43

Gershgorin’s Theorem

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 44

Gershgorin’s Theorem

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 45

Gershgorin’s Theorem

Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.

Course EE210B Spring Quarter 2015

Lyapunov Equations

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 47

Setting

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 48

Discrete-Time Lyapunov Eqs

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 49

Statement

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 50

Proof

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 51

Proof

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 52

Proof

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 53

Continuous-Time Lyapunov Eqs

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 54

Statement

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 55

Proof

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 56

Proof

Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.

Course EE210B Spring Quarter 2015

Stochastic Matrices

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 58

Stochastic Matrices

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 59

Properties

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 60

Properties

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed)

Proof 61

Strong primitiveness ensures a unique eigenvalue at one by the Perron Frobenius Theorem.

Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.

Course EE210B Spring Quarter 2015

Inequality Recursions

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 66

Deterministic Recursion

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 67

Deterministic Recursion

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 68

Big and Little-O Notation

Lecture #8: Useful Matrix Results EE210B: Inference over Networks (A. H. Sayed) 69

Stochastic Recursion

Proc. IEEE, vol. 102, no. 4, pp. 460-497, April 2014. Foundations and Trends in Machine Learning, vol. 7, no. 4-5, pp. 311-801, July 2014.

Course EE210B Spring Quarter 2015

End of Lecture