monte carlo for linear operator equations fall 2012
DESCRIPTION
Monte Carlo for Linear Operator Equations Fall 2012. By Hao Ji. Review. Last Class Quasi-Monte Carlo This Class Monte Carlo Linear Solver v on Neumann and Ulam method Randomize Stationary iterative methods Variations of Monte Carlo solver - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/1.jpg)
Monte Carlo for Linear Operator Equations
Fall 2012
By Hao Ji
![Page 2: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/2.jpg)
Review• Last Class– Quasi-Monte Carlo
• This Class– Monte Carlo Linear Solver
• von Neumann and Ulam method• Randomize Stationary iterative methods• Variations of Monte Carlo solver
– Fredholm integral equations of the second kind– The Dirichlet Problem– Eigenvalue Problems
• Next Class– Monte Carlo method for Partial Differential Equations
![Page 3: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/3.jpg)
Solving Linear System
• The simultaneous equations,
where is a matrix , is a given vector and is the unknown solution vector.
• Define the norm of matrix to be
![Page 4: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/4.jpg)
Solving Linear System
• Direct methods– Gaussian elimination– LU decomposition– …
• Iterative methods– Stationary iterative methods (Jacobi method, Gauss Seidel method,
…)– Krylov subspace methods(CG, Bicg, GMRES,…)– …
• Stochastic linear solvers– Monte Carlo methods– …
![Page 5: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/5.jpg)
Monte Carlo Linear Solver
• The Monte Carlo method proposed by von Neumann and Ulam:1. Define the transition probabilities and the terminating
probabilities.2. Build an unbiased estimator of the solution.3. Produce Random Walks and calculate the average value.
![Page 6: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/6.jpg)
Monte Carlo Linear Solver
• Let be a matrix based on the matrix , such that
and • A special case:
![Page 7: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/7.jpg)
Monte Carlo Linear Solver
• A terminating random walk stopping after k steps is
which passes through the sequence of integers (the row indices)
• The successive integers (states) are determined by the transition probabilities
and the termination probabilities
![Page 8: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/8.jpg)
Monte Carlo Linear Solver
• Define
whereThen,
is an unbiased estimator of in the solution if the Neumann series converges.
![Page 9: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/9.jpg)
Monte Carlo Linear Solver
• Proof:The expectation of is
(Since )
If the Neumann Series converges,
then .
![Page 10: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/10.jpg)
Monte Carlo Linear Solver
• Produce random walks starting from ,
can evaluate only one component of the solution.
• The transition matrix is critical for the convergence of the Monte Carlo Linear Solver.
In the special case: – Monte Carlo breaks down– Monte Carlo is less efficient than a conventional method
( 1% accuracy n<=554, 10% accuracy n<=84)– (1% accuracy n<=151, 10% accuracy n<=20)
![Page 11: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/11.jpg)
Monte Carlo Linear Solver
• To approximate the sum based on sampling, define a random variable with possible values , and the probabilities
Since
we can use random samples of to estimate the sum .
• The essence of Monte Carlo method in solving linear system is to sample the underlying Neumann series
![Page 12: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/12.jpg)
Randomize Stationary iterative methods
• Consider – Jacobi method: decompose A into a diagonal component and the
reminder .
where and
– Gauss Seidel method: decomposed A into a lower triangular component , and a strictly upper triangular component
where and
• Stationary iterative methods can easily be randomized by using Monte Carlo to statistically sample the underlying Neumann Series.
![Page 13: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/13.jpg)
Variations of Monte Carlo Linear Solver
• Wasow uses another estimator
in some situations to obtain smaller variance than .
• Adjoint Method
to find the solution instead of only.
![Page 14: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/14.jpg)
Variations of Monte Carlo Linear Solver
• Sequential Monte Carlo method To accelerate Monte Carlo method of simultaneous equations, Halton uses a rough estimate for to transform the original linear system.
Let and , then
Since the elements of are much smaller than , the transformed linear system could be much faster to get solution than solving the original one.
![Page 15: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/15.jpg)
Variations of Monte Carlo Linear Solver
• Dimov uses a different transtion matrix
Since the terminating probabilities not exist anymore, the random walk terminates when is small enough, where
![Page 16: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/16.jpg)
Fredholm integral equations of the second kind
• The integral equation
may be solved by Monte Carlo methods.
Since the integral can be approximated by a quadrature formula:
![Page 17: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/17.jpg)
Fredholm integral equations of the second kind
• The integral equation can be transformed to be
evaluate it at the quadrature points:
Let be the vector , be the vector and be the matrix , the integral equation becomes
where is the unknown vector.
![Page 18: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/18.jpg)
The Dirichlet Problem
• Dirichlet’s problem is to find a function , which is continuous and differentiable over a closed domain with boundary , satisfying
where is a prescribed function, and is the Laplacian operator.
Replacing by its finite-difference approximation,
![Page 19: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/19.jpg)
The Dirichlet Problem
• Suppose the boundary lies on the mesh, the previous equations can be transformed into
– The order of is equal to the number of mesh points in .– has four elements equal to in each row corresponding to an interior
point of , all other elements being zero.– has boundary values corresponding to an boundary point of , all other
interior elements being zero.– The random walk starting from an interior point , terminates when it
hits a boundary point . The is an unbiased estimator of .
![Page 20: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/20.jpg)
Eigenvalue Problems
• For a given symmetric matrix
assume that >, so that is the dominant eigenvalue and is the corresponding eigenvector.
For any nonzero vector , according to the power method,
We can obtain a good approximation of the dominant eigenvector of from the above.
![Page 21: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/21.jpg)
Eigenvalue Problems
Similar to the idea behinds Monte Carlo solver that we can do sampling on to estimate its value, and then evaluate the dominant eigenvector by a proper scaling. From the Rayleigh quotient,the dominant eigenvalue be approximated based on the estimated vector of .
![Page 22: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/22.jpg)
Summary
• This Class– Monte Carlo Linear Solver• von Neumann and Ulam method• Randomize Stationary iterative methods• Variations of Monte Carlo solver
– Fredholm integral equations of the second kind– The Dirichlet Problem– Eigenvalue Problems
![Page 23: Monte Carlo for Linear Operator Equations Fall 2012](https://reader035.vdocuments.site/reader035/viewer/2022070423/568166e6550346895ddb245b/html5/thumbnails/23.jpg)
What I want you to do?
• Review Slides• Work on Assignment 4