linear algebra and applications: numerical linear algebra algebra and applications: numerical linear...

Download Linear Algebra and Applications: Numerical Linear Algebra Algebra and Applications: Numerical Linear Algebra ... Linear Algebra, SIAM, 1997. James W. Demmel, ... Templates for the

Post on 23-May-2018

248 views

Category:

Documents

4 download

Embed Size (px)

TRANSCRIPT

  • Linear Algebra and Applications:Numerical Linear Algebra

    David S. Watkinswatkins@math.wsu.edu

    Department of Mathematics

    Washington State University

    IMA Summer Program, 2008 p. 1

  • My Pledge to You

    IMA Summer Program, 2008 p. 2

  • My Pledge to YouI promise not to cover as much materialas I previously claimed I would.

    IMA Summer Program, 2008 p. 2

  • Resources

    IMA Summer Program, 2008 p. 3

  • Resources (a biased list)

    IMA Summer Program, 2008 p. 3

  • Resources (a biased list)David S. Watkins, Fundamentals of MatrixComputations, Second Edition, John Wiley andSons, 2002. (FMC)

    IMA Summer Program, 2008 p. 3

  • Resources (a biased list)David S. Watkins, Fundamentals of MatrixComputations, Second Edition, John Wiley andSons, 2002. (FMC)

    David S. Watkins, The QR algorithm revisited,SIAM Review, 50 (2008), pp. 133145.

    IMA Summer Program, 2008 p. 3

  • Resources (a biased list)David S. Watkins, Fundamentals of MatrixComputations, Second Edition, John Wiley andSons, 2002. (FMC)

    David S. Watkins, The QR algorithm revisited,SIAM Review, 50 (2008), pp. 133145.

    David S. Watkins, The Matrix Eigenvalue Problem,GR and Krylov Subspace Methods, SIAM, 2007.

    IMA Summer Program, 2008 p. 3

  • Leslie Hogben, Handbook of Linear Algebra,Chapman and Hall/CRC, 2007.

    IMA Summer Program, 2008 p. 4

  • Leslie Hogben, Handbook of Linear Algebra,Chapman and Hall/CRC, 2007.

    Lloyd N. Trefethen and David Bau, III, NumericalLinear Algebra, SIAM, 1997.

    IMA Summer Program, 2008 p. 4

  • Leslie Hogben, Handbook of Linear Algebra,Chapman and Hall/CRC, 2007.

    Lloyd N. Trefethen and David Bau, III, NumericalLinear Algebra, SIAM, 1997.

    James W. Demmel, Applied Numerical LinearAlgebra, SIAM, 1997.

    IMA Summer Program, 2008 p. 4

  • Leslie Hogben, Handbook of Linear Algebra,Chapman and Hall/CRC, 2007.

    Lloyd N. Trefethen and David Bau, III, NumericalLinear Algebra, SIAM, 1997.

    James W. Demmel, Applied Numerical LinearAlgebra, SIAM, 1997.

    G. H. Golub and C. F. Van Loan, MatrixComputations, Third Edition, Johns HopkinsUniversity Press, 1996.

    IMA Summer Program, 2008 p. 4

  • Common Linear AlgebraComputations

    IMA Summer Program, 2008 p. 5

  • Common Linear AlgebraComputations

    linear system Ax = b

    IMA Summer Program, 2008 p. 5

  • Common Linear AlgebraComputations

    linear system Ax = b

    overdetermined linear system Ax = b

    IMA Summer Program, 2008 p. 5

  • Common Linear AlgebraComputations

    linear system Ax = b

    overdetermined linear system Ax = b

    eigenvalue problem Av = v

    IMA Summer Program, 2008 p. 5

  • Common Linear AlgebraComputations

    linear system Ax = b

    overdetermined linear system Ax = b

    eigenvalue problem Av = v

    various generalized eigenvalue problems,e.g. Av = Bv

    IMA Summer Program, 2008 p. 5

  • Linear Systems

    IMA Summer Program, 2008 p. 6

  • Linear SystemsAx = b, n n, nonsingular, real or complex

    IMA Summer Program, 2008 p. 6

  • Linear SystemsAx = b, n n, nonsingular, real or complex

    Examples: FMC 1.2, 7.1; any linear algebra text

    IMA Summer Program, 2008 p. 6

  • Linear SystemsAx = b, n n, nonsingular, real or complex

    Examples: FMC 1.2, 7.1; any linear algebra text

    Major tools:Gaussian elimination (LU Decomp.)various iterative methods

    IMA Summer Program, 2008 p. 6

  • Overdetermined Linear Systems

    IMA Summer Program, 2008 p. 7

  • Overdetermined Linear SystemsAx = b, n m, n > m

    IMA Summer Program, 2008 p. 7

  • Overdetermined Linear SystemsAx = b, n m, n > m

    often n m

    IMA Summer Program, 2008 p. 7

  • Overdetermined Linear SystemsAx = b, n m, n > m

    often n m

    Example: fitting data by a straight line

    IMA Summer Program, 2008 p. 7

  • Overdetermined Linear SystemsAx = b, n m, n > m

    often n m

    Example: fitting data by a straight line

    minimize b Ax2

    (least squares)

    IMA Summer Program, 2008 p. 7

  • Overdetermined Linear SystemsAx = b, n m, n > m

    often n m

    Example: fitting data by a straight line

    minimize b Ax2

    (least squares)

    Major tools:QR decompositionsingular value decomposition

    IMA Summer Program, 2008 p. 7

  • Eigenvalue Problems

    IMA Summer Program, 2008 p. 8

  • Eigenvalue Problemsstandard: Av = v, n n, real or complex

    IMA Summer Program, 2008 p. 8

  • Eigenvalue Problemsstandard: Av = v, n n, real or complex

    Examples: FMC 5.1

    IMA Summer Program, 2008 p. 8

  • Eigenvalue Problemsstandard: Av = v, n n, real or complex

    Examples: FMC 5.1

    generalized: Av = Bv

    IMA Summer Program, 2008 p. 8

  • Eigenvalue Problemsstandard: Av = v, n n, real or complex

    Examples: FMC 5.1

    generalized: Av = Bv

    Examples: FMC 6.7

    IMA Summer Program, 2008 p. 8

  • Eigenvalue Problemsstandard: Av = v, n n, real or complex

    Examples: FMC 5.1

    generalized: Av = Bv

    Examples: FMC 6.7

    product: A1A2

    IMA Summer Program, 2008 p. 8

  • Eigenvalue Problemsstandard: Av = v, n n, real or complex

    Examples: FMC 5.1

    generalized: Av = Bv

    Examples: FMC 6.7

    product: A1A2

    Examples: generalized (AB1), SVD (AA)

    IMA Summer Program, 2008 p. 8

  • Eigenvalue Problemsstandard: Av = v, n n, real or complex

    Examples: FMC 5.1

    generalized: Av = Bv

    Examples: FMC 6.7

    product: A1A2

    Examples: generalized (AB1), SVD (AA)

    quadratic: (2K + G + M)v = 0

    IMA Summer Program, 2008 p. 8

  • Sizes of Linear Algebra Problems

    IMA Summer Program, 2008 p. 9

  • Sizes of Linear Algebra Problemssmall

    IMA Summer Program, 2008 p. 9

  • Sizes of Linear Algebra Problemssmall

    medium

    IMA Summer Program, 2008 p. 9

  • Sizes of Linear Algebra Problemssmall

    medium

    large

    IMA Summer Program, 2008 p. 9

  • Solving Linear Systems:

    IMA Summer Program, 2008 p. 10

  • Solving Linear Systems:small problems

    IMA Summer Program, 2008 p. 10

  • Solving Linear Systems:small problems

    Ax = b, n n, n small

    IMA Summer Program, 2008 p. 10

  • Solving Linear Systems:small problems

    Ax = b, n n, n small

    store A conventionally

    IMA Summer Program, 2008 p. 10

  • Solving Linear Systems:small problems

    Ax = b, n n, n small

    store A conventionally

    solve using Gaussian elimination

    IMA Summer Program, 2008 p. 10

  • Solving Linear Systems:small problems

    Ax = b, n n, n small

    store A conventionally

    solve using Gaussian elimination

    A = LU

    IMA Summer Program, 2008 p. 10

  • Solving Linear Systems:small problems

    Ax = b, n n, n small

    store A conventionally

    solve using Gaussian elimination

    A = LU

    PA = LU (partial pivoting)

    IMA Summer Program, 2008 p. 10

  • Solving Linear Systems:small problems

    Ax = b, n n, n small

    store A conventionally

    solve using Gaussian elimination

    A = LU

    PA = LU (partial pivoting)

    forward and back substitution

    IMA Summer Program, 2008 p. 10

  • Solving Linear Systems:small problems

    Ax = b, n n, n small

    store A conventionally

    solve using Gaussian elimination

    A = LU

    PA = LU (partial pivoting)

    forward and back substitution

    Questions: cost?,

    IMA Summer Program, 2008 p. 10

  • Solving Linear Systems:small problems

    Ax = b, n n, n small

    store A conventionally

    solve using Gaussian elimination

    A = LU

    PA = LU (partial pivoting)

    forward and back substitution

    Questions: cost?, accuracy? (FMC Ch. 2)

    IMA Summer Program, 2008 p. 10

  • Positive Definite Case

    IMA Summer Program, 2008 p. 11

  • Positive Definite CaseA = A, xAx > 0 for all x 6= 0

    IMA Summer Program, 2008 p. 11

  • Positive Definite CaseA = A, xAx > 0 for all x 6= 0

    A = RR Cholesky decomposition

    IMA Summer Program, 2008 p. 11

  • Positive Definite CaseA = A, xAx > 0 for all x 6= 0

    A = RR Cholesky decomposition

    symmetric variant of Gaussian elimination

    IMA Summer Program, 2008 p. 11

  • Positive Definite CaseA = A, xAx > 0 for all x 6= 0

    A = RR Cholesky decomposition

    symmetric variant of Gaussian elimination

    flop count is halved

    IMA Summer Program, 2008 p. 11

  • Solving Linear Systems:

    IMA Summer Program, 2008 p. 12

  • Solving Linear Systems:medium problems

    IMA Summer Program, 2008 p. 12

  • Solving Linear Systems:medium problems

    Larger problems are usually sparser.

    IMA Summer Program, 2008 p. 12

  • Solving Linear Systems:medium problems

    Larger problems are usually sparser.

    Use sparse data structure.

    IMA Summer Program, 2008 p. 12

  • Solving Linear Systems:medium problems

    Larger problems are usually sparser.

    Use sparse data structure.

    sparse Gaussian elimination

    IMA Summer Program, 2008 p. 12

  • Solving Linear Systems:medium problems

    Larger problems are usually sparser.

    Use sparse data structure.

    sparse Gaussian elimination

Recommended

View more >