numerical algorithms
DESCRIPTION
Numerical Algorithms. .Matrix Multiplication .Gaussian Elimination .Jacobi Iteration .Gauss-Seidel Relaxation. Numerical Algorithms. Matrix addition. Numerical Algorithms. Matrix Multiplication. Numerical Algorithms. Matrix-Vector Multiplication. Implementing Matrix Multiplication. - PowerPoint PPT PresentationTRANSCRIPT
Numerical Algorithms
.Matrix Multiplication
.Gaussian Elimination
.Jacobi Iteration
.Gauss-Seidel Relaxation
Numerical Algorithms
Matrix addition
Numerical Algorithms
Matrix Multiplication
1
0
l
kijijij bac
Numerical Algorithms
Matrix-Vector Multiplication
Implementing Matrix Multiplication
for(i=0 ; i<n ; i++) for(j=0 ; j<n ; j++){ c[i][j] = 0; for(k=0 ; k<n ; k++) c[i][j] = c[i][j] + a[i][k] * b[k][j];}
Sequential Code O(n3)
Implementing Matrix Multiplication
Partitioning into Submatrices
for(p=0 ; p<s ; p++) for(q=0 ; q<s ; q++){ Cp,q = 0; for(r=0 ; r<m ; r++) Cp,q = Cp,q + Ap,r * Br,q; }
Implementing Matrix Multiplication
Implementing Matrix Multiplication
Implementing Matrix Multiplication
Implementing Matrix Multiplication
Analysis
communication
)()(
)()2(
22
22
datastartupdatastartupcomm
datastartupdatastartupcomm
ttntntt
broadcast
ttnnttnt
computation
nt
additionsnandtionsmultiplican
comp 2
Implementing Matrix Multiplication
O(n2) with n2 processorsO(log n) with n3 processors
Implementing Matrix Multiplication
submatricess=n/m
communication
)}()(2{ 22datastartupdatastartupcomm tmtmttst
computation
)()()2( 2323 nmOsmOmmstcomp
Recursive Implementation
mat_mult(App, Bpp, s) { if( s==1) C=A*B; else{ s = s/2; P0 = mat_mult(App, Bpp, s); P1 = mat_mult(Apq, Bqp, s); P2 = mat_mult(App, Bpq, s); P3 = mat_mult(Apq, Bqq, s); P4 = mat_mult(Aqp, Bpp, s); P5 = mat_mult(Aqq, Bqp, s); P6 = mat_mult(Aqp, Bpq, s); P7 = mat_mult(Aqq, Bqq, s); Cpp = P0 + P1; Cpq = P2 + P3; Cqp = P4 + P5; Cqq = P6 + P7; } return(C);}
Mesh Implementation
Connon's Algorithm
1. initially processor Pij has element Aij and Bij2. Elements are moved from their initial position to an "aligned" position. The complete ith row of A is shifted i places left and the complete jth column of B is shifted j places downward. this has the effect of placing the elements aij+1 and the element bi+jj in processor Pij, as illusrated in figure 10.10. These elements are pair of those required in the accumulation of cij3. Each processor, P1j, multiplies its elements.4. The ith row of A is shifted one place right, and the jth column of B is shifted one place downward. this has the effect of bringing together the adjacent elements of A and B, which will also be required in the accumulation, as illustrated in Figure 10.11.5. Each processor, Pij, multiplies the elements brought to it and adds the result to the accumulation sum.6. Step 4 and 5 are repeated until the final result is obtained
Mesh Implementation
Mesh Implementation
Analysis
communication
))(1(2 2datastartupcomm tmtst
computation
O(sm2)
nmsmmstcomp233 22)2)(1(
Two dimensional pipeline--- Systolic array
recv(&a, Pi,j-1);recv(&b, Pi-1,j);c=c+a*b;send(&a, Pi,j+1);send(&b, Pi+1,j);
Two dimensional pipeline--- Systolic array
Solving a System of Linear Equations
11,11,022,011,000,0
11,11,122,111,100,1
11,11,122,111,100,1
...............
................
....
....
.....
nnnn
nnnn
nnnnnnnn
bxaxaxaxa
bxaxaxaxa
bxaxaxaxa
Ax=bDense matrixSparse matrix
Solving a System of Linear Equations
Gaussian Elimination
0)(
ii
jiiijiji a
aaaa
Solving a System of Linear Equations
for(i=0 ; i<n-1 ; i++) for(j=i+1 ; j<n ; j++){ m = a[j][i]/a[i][i]; for(k=i ; k<n ; k++) a[j][k] = a[j][k] - a[i][k] * m; b[j] = b[j] - b[i] * m;
O(n3)
Solving a System of Linear Equations
communication
2
0
))1((n
idatastartupcomm tintt O(n2)
Solving a System of Linear Equations
computation
)(32
)2)(3()2(2 2
1
1
nOnn
jntn
jcomp
Solving a System of Linear Equations
Pipeline configuration
Solving a System of Linear Equations
Iterative Methods
Jacobi Iteration
ij
kjiji
ii
ki xab
ax ][
1 1
Iterative Methods
022
2
22
2
y
f
x
f
)],(),(2),([1
)],(),(2),([1
222
2
222
2
yxfyxfyxfy
f
yxfyxfyxfx
f
Iterative Methods
0)],(4),(),(),(),([1
2
yxfyxfyxfyxfyxf
4
)],(),(),(),([),(
yxfyxfyxfyxfyxf
4
)],(),(),(),([),(
1111
yxfyxfyxfyxfyxf
kkkkk
Relationship with a General System of Linear Equations
411 niiini
i
xxxxx
Iterative Methods
Iterative Methods
Iterative Methods
Iterative Methods
4
)],(),(),(),([),(
11
yxfyxfyxfyxfyxf
kkkkk
Gauss-Seidel Relaxation
Iterative Methods
Iterative Methods
Red-Black Ordering
Iterative Methods
forall(i=0 ; i<n ; i++) forall(j=1 ; j<n ; j++) if((i+j)%2 == 0) f[i][j] = 0.25*(f[i-1][j]+f[i][j-1]+f[i+1][j]+f[i][j+1]);
forall(i=1 ; i<n ; i++) forall(j=1 ; j<n ; j++) if((i+j)%2 !=0 ) f[i][j] = 0.25*(f[i-1][j]+f[i][j-1]+f[i+1][j]+f[i][j+1]);
Iterative Methods
High-Order Difference Methods
)]2,(),2()2,(),2(
),(16),(16),(16),(16[60
1
1111
1111
yxfyxfyxfyxf
yxfyxfyxfyxf
kkkk
kkkk
Iterative Methods
Multigrid Method
10,)1(][ 1
1
1
ki
j
kiiji
ii
ki xxab
ax