Numerical linear algebra

Purdue UniversityCS 51500

Fall 2019Fall 2020

David Gleich

David F. Gleich

Call me …“Prof Gleich”“Dr. Gleich”

Please not“Hey matrix guy!”

Numerical linear algebra

Or

Matrix computations

Purpose

Matrix computations underlie much (most?) of applied computations.

It’s the language of computational algorithms.

PageRank (from the paper)

BFGS78 C. G. BROYDEN

3. The Generalized MethodIn this method (Broyden, 1967) the vector p, is given by

P, = -H,f(> (3.1)where H, is positive definite. Hi is chosen to be an arbitrary positive definite matrix(often the unit matrix) and H,+i is given by

H / + 1 = Ht-Hgrf+Wfif, i = 1,2,..., (3.2a)where

y; = f1+,-f(, (3.2b)q[ = a i P i r - ^ r H ( > (3.2c)

(3.2d)(3.2e)(3.20

The parameter /?, is arbitrary and setting it equal to zero gives the DFP method(Fletcher & Powell, 1963). It was shown by Broyden (1967) that the matrices H,constructed in this way are always positive definite if /?, > 0.

In order to analyse the convergence of this algorithm we define three more quantities.Let B be the positive definite matrix that satisfies the equation

B2 = A, (3.3)and define z,, K, by

i, = Be,, (3.4)and

K, = BH,B. (3.5)These definitions coupled with equations (2.12) and (3.2) then yield, after some

tedious and uninstructive algebra involving relationships established in Section 2(above),

|ZJ+I = Z|-K,z,f,, (3.6)

K,+, = X]-K?z,tj

Circular antennae design

612 EZEE TRANSACTIONS ON ANTENNAS AND PROPAGA~ON; SEPTEMBER 1973

with

where

2a N

a=---

I 1, N even 2, N odd. E N = and [ N / P ] is t.he largest integer not exceeding N I P , (N,P integers). All submatrices appearing in equat,ions (6) and (7) a.re symmet,ric, as is Y , so that Yi = (i = 1,2, - .,X - 1 ) ‘and only [ N / 2 ] + 1 of the sub- matrices Y i need be computed. The closely related case of a circular array having an additional dipole at its center may be considered by simila.r methods to those used in the preceding [7, sect. 3.51.

B. Port Description of Array Since relatively few of the elements of V are nonzero

some reduction in (3) is possible. Only those columns of Y which correspond to indices of triangles centered at. the dipole mid-point,s need be retained in ( 3 ) ; denote as YR the rectangular matzix obtained by delet.ing all columns of Y not having such a column index. Then,

I = YRVT (8 1 where VT is the N vector formed by deleting all ident.ically zero elements of V . Y R is denoted t,he “reduced admittance matrix.”

Furthermore, if only the feed-point. currents a.re of interest, a similar reduction may be performed on rows of YR and I to yield,

IT = YTVT (9 1 where IT,YT,VT are, respectively, the port (terminal) cur- rent vector, admittance matrix, and voltage vector which describe the array as an N-port network for feed con- siderations.

111. PATTERN SYNTHESIS It has been shorn [ 9 ] that for an array of parallel

x-directed filamentary currents the far-field radiation pat- tern on a surface of constant 0 may be considered as arising from a pla.nm array of isotropic uncoupled point sources whose excitations are given by a linear transformat,ion of the filamentary current, excitations. An array of thin

parallel wires may be considered as an army of iilamentmy currents for far-field purposes if the wire diameters are small compared to a wavelength at the frequency of ex- citation. In the following, the “surface of constant fP is taken as the principal H plane (0 = ~ / 2 ) without serious loss of generality.

The result of [9] as it applies to the principal H-plane field for the circular array of Fig. 1 may be stated,

hr ee = Wn exp [ jkR COS (4 - na)] (10)

where e8 is the normalized 0 component of the far electric field, W , is the transformed excitat-ion to be discussed, and 12 = with X the wavelength at the exciting fre- quency. This expression is of the same form as for the far field in t.he zy plane of an array of uncoupled isotxopic point sources of strength W , at the points (p,+,z) = (R,ncu,O) (n = 1,2, . - , N ) . It should be noted that this reduction to a point,-source problem depends for its suc- cess on the a.zimut.ha1 omnidirectionality of the radiating sources (hence the restriction t.0 parallel wires) and is not simply a rest,at.ement of t,he principle of patttern mult.ipli- cation; the present t,reatment. takes full account of all mut.ua1 coupling effects.

The vector W = [ l V ~ , W ~ , - - . , W N } is generated by a transformat.ion of the terminal excitation [9]

n=l

IF‘ = AVT = AYTIT (11 1 where A is a circulant matrix having elements Aij = ali-jl, such that, in terms of the elements yij of the reduced admittance matrix YR,

bf ‘-1 an-l 2 yin + yMiln, n = 1,2,.--,X. (12)

i=l

Thus the circuit description of the array serves as the connecting link betweea t,he physical voltage or current excitat,ions of the array and the point.-source model. In t.he following, attention is confined to principal H-plane pattern synthesis for circular axrays of point-sources; (1 1 ) and (12) can then be used t.o rea.lize t.he synthesized pattern with physical dipole elements.

In order to achieve some depth of treatment of t.he more common pathern synthesis requirements, t,he following restrictions are imposed.

1) Patterns are symmet.rica1 about. 4 = 0. 2) Patterns are purely real (or constant phase). 3 ) The number of dipoles is even.

It can be shown [7, sect.. 5.61 that, subject to these rest,rictions, Wn = v’hr-n = Tvhr/2-n*, so that. there are only N/2 + 1 independent real va.riab1es.I (Real and imaginary parts of WO,WI, - - , W [ N / ~ ] , nit,h w [ N / 4 ] purely real if [ N / 4 ] = X/4.)

The pattern synthesis problem mag be considered as a problem of approximation. Let, F ( 4 ) be a pattern which

1 Complex conjugation is denoted by the asterisk.

Authorized licensed use limited to: The University of British Columbia Library. Downloaded on November 13, 2009 at 12:27 from IEEE Xplore. Restrictions apply.

Dynamic mode decomposition

Electrical circuits

“A matrix version of Kirchhoff’s circuit law is the basis of most circuit simulation software”-- Wikipedia

Other applications

BiologyPDEs/Mechanical Engineering/AeroAstroMachine learningStatisticsGraphics

Purpose

The purpose this class is to teach you how to “speak matrix computations like a native” so that you can understand, implement, interpret, and extend work that uses them.

Examples

Why should we avoid the “normal equations”?

Why do I get strange looks if I talk about the SVD of a symmetric positive definite matrix?

Why not write things element-wise?

Please pay attention for a second, this next bit is important!

The new class scheduleBasic Problems• Least Squares, Linear Systems,

Singular Values, Eigenvalues, Sparse Matrices,

Simple Algorithms• Gradient descent, power

method • Convergence analysisFinite Termination• Coordinate fixing -> Cholesky• LU with pivoting• QR factorization

Conditioning & Stability (after midterm)• How to choose algorithms? Advanced Problems• Sequences of linear systems• Generalized eigenvalue

problemsKrylov Methods• Arnoldi, LanczosEigenvalue algorithms• All eigenvalues• Some eigenvaluesGetting high performance, randomized?

Why did I change this?

• One weakness of a classic presentation is that it discourages interplay between pre/post midterm.

• The new presentation makes the class more exciting and highlights the interplay between materials.

• One downside, it doesn’t really follow an existing book.

Textbooks

No best reference.

Golub and van Loan – “The Bible” – but sometimes a bit terse

Trefethen & Bau, Numerical Linear AlgebraDemmel, Applied Numerical Linear AlgebraSaad, Iterative Methods for Sparse Linear Systems

Background books

Strang, Linear Algebra and its ApplicationsMeyer, Matrix Analysis

Why I like Julia & Matlab

Julia Designed as a technical computing languageMatlab it’s a modeling language for matrix methods!

The power method described in Wikipedia

while 1a = b; b = A*b;b = b/norm(b);if test_converge(a,b); break; end

end

Matlab & Julia code

x = b # make a reference to Ay = zeros(length(b)) # allocatewhile 1

A_mul_B!(y,A,x) # y = Axscale!(y,1/norm(x)) # scaleif test_converge(x,y); break; endx,y == y,x # swap pointers

end

Super efficient Julia code

SoftwareYou will have to write matrix programs in class.

Julia & Atom my recommendation (what I use!)Julia & Jupyter notebook my 2nd recommendationJulia & Text Editor (your call!)

Matlab what I used to useSciPy, NumPy okay (look at spyder/pythonxy)

R not recommended, best to avoidScilab you’re on your ownC/C++ with LAPACK okay, but ill-advisedFortran (same!)

THE SYLLABUS

Cut to website!www.cs.purdue.edu/homes/dgleich/cs515-2020

Quiz

• Write down any questions, concerns, issues, etc. you think you have after hearing about the class logistics.