Linear Algebra and Applications: Numerical Linear Algebra Algebra and Applications: Numerical Linear Algebra ... Linear Algebra, SIAM, 1997. James W. Demmel, ... Templates for the Solution of Linear
Post on 23-May-2018
226 views
Embed Size (px)
TRANSCRIPT
Linear Algebra and Applications:Numerical Linear Algebra
David S. Watkinswatkins@math.wsu.edu
Department of Mathematics
Washington State University
IMA Summer Program, 2008 p. 1
My Pledge to You
IMA Summer Program, 2008 p. 2
My Pledge to YouI promise not to cover as much materialas I previously claimed I would.
IMA Summer Program, 2008 p. 2
Resources
IMA Summer Program, 2008 p. 3
Resources (a biased list)
IMA Summer Program, 2008 p. 3
Resources (a biased list)David S. Watkins, Fundamentals of MatrixComputations, Second Edition, John Wiley andSons, 2002. (FMC)
IMA Summer Program, 2008 p. 3
Resources (a biased list)David S. Watkins, Fundamentals of MatrixComputations, Second Edition, John Wiley andSons, 2002. (FMC)
David S. Watkins, The QR algorithm revisited,SIAM Review, 50 (2008), pp. 133145.
IMA Summer Program, 2008 p. 3
Resources (a biased list)David S. Watkins, Fundamentals of MatrixComputations, Second Edition, John Wiley andSons, 2002. (FMC)
David S. Watkins, The QR algorithm revisited,SIAM Review, 50 (2008), pp. 133145.
David S. Watkins, The Matrix Eigenvalue Problem,GR and Krylov Subspace Methods, SIAM, 2007.
IMA Summer Program, 2008 p. 3
Leslie Hogben, Handbook of Linear Algebra,Chapman and Hall/CRC, 2007.
IMA Summer Program, 2008 p. 4
Leslie Hogben, Handbook of Linear Algebra,Chapman and Hall/CRC, 2007.
Lloyd N. Trefethen and David Bau, III, NumericalLinear Algebra, SIAM, 1997.
IMA Summer Program, 2008 p. 4
Leslie Hogben, Handbook of Linear Algebra,Chapman and Hall/CRC, 2007.
Lloyd N. Trefethen and David Bau, III, NumericalLinear Algebra, SIAM, 1997.
James W. Demmel, Applied Numerical LinearAlgebra, SIAM, 1997.
IMA Summer Program, 2008 p. 4
Leslie Hogben, Handbook of Linear Algebra,Chapman and Hall/CRC, 2007.
Lloyd N. Trefethen and David Bau, III, NumericalLinear Algebra, SIAM, 1997.
James W. Demmel, Applied Numerical LinearAlgebra, SIAM, 1997.
G. H. Golub and C. F. Van Loan, MatrixComputations, Third Edition, Johns HopkinsUniversity Press, 1996.
IMA Summer Program, 2008 p. 4
Common Linear AlgebraComputations
IMA Summer Program, 2008 p. 5
Common Linear AlgebraComputations
linear system Ax = b
IMA Summer Program, 2008 p. 5
Common Linear AlgebraComputations
linear system Ax = b
overdetermined linear system Ax = b
IMA Summer Program, 2008 p. 5
Common Linear AlgebraComputations
linear system Ax = b
overdetermined linear system Ax = b
eigenvalue problem Av = v
IMA Summer Program, 2008 p. 5
Common Linear AlgebraComputations
linear system Ax = b
overdetermined linear system Ax = b
eigenvalue problem Av = v
various generalized eigenvalue problems,e.g. Av = Bv
IMA Summer Program, 2008 p. 5
Linear Systems
IMA Summer Program, 2008 p. 6
Linear SystemsAx = b, n n, nonsingular, real or complex
IMA Summer Program, 2008 p. 6
Linear SystemsAx = b, n n, nonsingular, real or complex
Examples: FMC 1.2, 7.1; any linear algebra text
IMA Summer Program, 2008 p. 6
Linear SystemsAx = b, n n, nonsingular, real or complex
Examples: FMC 1.2, 7.1; any linear algebra text
Major tools:Gaussian elimination (LU Decomp.)various iterative methods
IMA Summer Program, 2008 p. 6
Overdetermined Linear Systems
IMA Summer Program, 2008 p. 7
Overdetermined Linear SystemsAx = b, n m, n > m
IMA Summer Program, 2008 p. 7
Overdetermined Linear SystemsAx = b, n m, n > m
often n m
IMA Summer Program, 2008 p. 7
Overdetermined Linear SystemsAx = b, n m, n > m
often n m
Example: fitting data by a straight line
IMA Summer Program, 2008 p. 7
Overdetermined Linear SystemsAx = b, n m, n > m
often n m
Example: fitting data by a straight line
minimize b Ax2
(least squares)
IMA Summer Program, 2008 p. 7
Overdetermined Linear SystemsAx = b, n m, n > m
often n m
Example: fitting data by a straight line
minimize b Ax2
(least squares)
Major tools:QR decompositionsingular value decomposition
IMA Summer Program, 2008 p. 7
Eigenvalue Problems
IMA Summer Program, 2008 p. 8
Eigenvalue Problemsstandard: Av = v, n n, real or complex
IMA Summer Program, 2008 p. 8
Eigenvalue Problemsstandard: Av = v, n n, real or complex
Examples: FMC 5.1
IMA Summer Program, 2008 p. 8
Eigenvalue Problemsstandard: Av = v, n n, real or complex
Examples: FMC 5.1
generalized: Av = Bv
IMA Summer Program, 2008 p. 8
Eigenvalue Problemsstandard: Av = v, n n, real or complex
Examples: FMC 5.1
generalized: Av = Bv
Examples: FMC 6.7
IMA Summer Program, 2008 p. 8
Eigenvalue Problemsstandard: Av = v, n n, real or complex
Examples: FMC 5.1
generalized: Av = Bv
Examples: FMC 6.7
product: A1A2
IMA Summer Program, 2008 p. 8
Eigenvalue Problemsstandard: Av = v, n n, real or complex
Examples: FMC 5.1
generalized: Av = Bv
Examples: FMC 6.7
product: A1A2
Examples: generalized (AB1), SVD (AA)
IMA Summer Program, 2008 p. 8
Eigenvalue Problemsstandard: Av = v, n n, real or complex
Examples: FMC 5.1
generalized: Av = Bv
Examples: FMC 6.7
product: A1A2
Examples: generalized (AB1), SVD (AA)
quadratic: (2K + G + M)v = 0
IMA Summer Program, 2008 p. 8
Sizes of Linear Algebra Problems
IMA Summer Program, 2008 p. 9
Sizes of Linear Algebra Problemssmall
IMA Summer Program, 2008 p. 9
Sizes of Linear Algebra Problemssmall
medium
IMA Summer Program, 2008 p. 9
Sizes of Linear Algebra Problemssmall
medium
large
IMA Summer Program, 2008 p. 9
Solving Linear Systems:
IMA Summer Program, 2008 p. 10
Solving Linear Systems:small problems
IMA Summer Program, 2008 p. 10
Solving Linear Systems:small problems
Ax = b, n n, n small
IMA Summer Program, 2008 p. 10
Solving Linear Systems:small problems
Ax = b, n n, n small
store A conventionally
IMA Summer Program, 2008 p. 10
Solving Linear Systems:small problems
Ax = b, n n, n small
store A conventionally
solve using Gaussian elimination
IMA Summer Program, 2008 p. 10
Solving Linear Systems:small problems
Ax = b, n n, n small
store A conventionally
solve using Gaussian elimination
A = LU
IMA Summer Program, 2008 p. 10
Solving Linear Systems:small problems
Ax = b, n n, n small
store A conventionally
solve using Gaussian elimination
A = LU
PA = LU (partial pivoting)
IMA Summer Program, 2008 p. 10
Solving Linear Systems:small problems
Ax = b, n n, n small
store A conventionally
solve using Gaussian elimination
A = LU
PA = LU (partial pivoting)
forward and back substitution
IMA Summer Program, 2008 p. 10
Solving Linear Systems:small problems
Ax = b, n n, n small
store A conventionally
solve using Gaussian elimination
A = LU
PA = LU (partial pivoting)
forward and back substitution
Questions: cost?,
IMA Summer Program, 2008 p. 10
Solving Linear Systems:small problems
Ax = b, n n, n small
store A conventionally
solve using Gaussian elimination
A = LU
PA = LU (partial pivoting)
forward and back substitution
Questions: cost?, accuracy? (FMC Ch. 2)
IMA Summer Program, 2008 p. 10
Positive Definite Case
IMA Summer Program, 2008 p. 11
Positive Definite CaseA = A, xAx > 0 for all x 6= 0
IMA Summer Program, 2008 p. 11
Positive Definite CaseA = A, xAx > 0 for all x 6= 0
A = RR Cholesky decomposition
IMA Summer Program, 2008 p. 11
Positive Definite CaseA = A, xAx > 0 for all x 6= 0
A = RR Cholesky decomposition
symmetric variant of Gaussian elimination
IMA Summer Program, 2008 p. 11
Positive Definite CaseA = A, xAx > 0 for all x 6= 0
A = RR Cholesky decomposition
symmetric variant of Gaussian elimination
flop count is halved
IMA Summer Program, 2008 p. 11
Solving Linear Systems:
IMA Summer Program, 2008 p. 12
Solving Linear Systems:medium problems
IMA Summer Program, 2008 p. 12
Solving Linear Systems:medium problems
Larger problems are usually sparser.
IMA Summer Program, 2008 p. 12
Solving Linear Systems:medium problems
Larger problems are usually sparser.
Use sparse data structure.
IMA Summer Program, 2008 p. 12
Solving Linear Systems:medium problems
Larger problems are usually sparser.
Use sparse data structure.
sparse Gaussian elimination
IMA Summer Program, 2008 p. 12
Solving Linear Systems:medium problems
Larger problems are usually sparser.
Use sparse data structure.
sparse Gaussian elimination
