intro to scientific libraries

Intro to Intro to Scientific LibrariesScientific Libraries

Blue Waters Undergraduate Petascale Education Program

May 29 – June 10 2011

Scientific LibrariesBWUPEP2011, UIUC, May 29 - June 10 2011 2

Matrix Multiplication How would/did you program it? How about parallelizing it?

-Hint: N2 operations that look like “ab+cd” that all can be done independently

What is the most efficient way?

Answers to the previous questions How would/did you program it? How about parallelizing it?

Supercomputing in Plain English: Apps & Par Types BWUPEP2011, UIUC, May 29 - June 10

2011 3

Lazy answer #1: You don’t need to do either!

What is the most efficient way?

Lazy answer #2: It depends.

Explanation of Lazy Answer #1 Solution Quality:

Don’t reinvent the wheel


2011 4

The solution of a large number

of very smart people over many years

Your solution

Scientific Libraries Loose Definition – A collection of mathematical optimized

for various data types, computer architectures, numeric types, and scientific fields of study.

Common example:

Lapack- linear algebra library


2011 5

How to identify your matrix General Matrix – Nothing too special going on with it Banded Matrix – A lines of data going diagonally down the

matrix.

Tri-diagonal matrix- A 3-width

banded matrix

Symmetric matrix-


2011 6

The Sparse Matrix A larger matrix that is mostly empty. A good rule of thumb

is approximately 90% - 95% empty. The opposite of a sparse matrix is a dense matrix.


2011 7

BLAS Basic Linear Algebra System Fundamental level of linear algebra libraries Various highly optimized implementations by vendors Many other libraries built on top of BLAS Create your own optimized implementation using ATLAS Three levels:

Level 1- Vector-vector operations -

Level 2- Matrix-vector operations -

Level 3- Matrix-matrix operations –


2011 8

ATLAS Automatically Tuned Linear Algebra Software Performs a series of timed tests upon installation. These

tests are used to tune the libraries for the individual system. Substantially faster on many systems.


2011 9

Atlas Performance


2011 10

Lapack Written on top of Basic Linear Algebra Subprograms

(BLAS) Incorporates/retools EISPACK (eigenvalues) and LINPACK

(least squares) Optimized for most modern shared memory architectures


2011 11

Eigenvalue

=

Eigenvector

Why LAPACK? Widely available. Lots of documentation and examples. Easy to use. Mostly preinstalled and tuned on most

computers. Many versions are tuned for shared memory architectures. Since it is built on top of your BLAS installation, it’ll

perform at least as well as BLAS.


2011 12

Beginner’s Guide to LAPACK

First step: Install LAPACK

-May involve installing BLAS first (Optional activity!)


2011 13

#1 RULE – Documentation is your friend!!

Beginner’s Guide to LAPACK Second step- Look up what you want to do.

-Handy naming scheme:


2011 14

GE General Matrix

DI Diagonal Matrix

GB General Band Matrix

GT Tridiagonal Matrix

HB Hermitian Band matrix

SY Symmetric matrix

TRF Factorize

TRS Solve

TRI Invert

RFS Error bounds

S Real

D Double

C Complex

Z Complex Double

Data type + Matrix Type + Problem

Not all subroutines have handy naming scheme. USE DOCUMENTATION

EXAMPLES Simple LAPACK call- DGEMM A bunch of calls repeated 1000 times Bonus demo-Difficult part of

libraries!


2011 15

Leading dimensions?


2011 16

You have this:

You want to perform an operation on this:

LDA = M

This occurs the most

LDA = 3M=2N=2

General “Don’t”s Don’t invert a matrix if it can be avoided. It is

computationally very expensive. Figure out a way around it. Don’t write your own solvers. If it is a common

mathematical operation, someone else most likely spent a lot of time and effort solving for the most efficient method.

Don’t attempt to use a library without consulting the documentation and attempting a few example problems.

Don’t attempt to program a solution (especially with libraries) if you don’t understand the problem’s mathematics/science!


2011 17

Preview: Next Week DOE ACTs collection:

2nd lazy answer- How do we decide what to use to optimize efficiency?


2011 18

ScaLAPACK

intro to scientific libraries

Documents