development tools for hpc applications
DESCRIPTION
Development Tools For HPC Applications. Deniz Savas, Michael Griffiths Corporate Information and Computing Services The University of Sheffield Email [email protected], [email protected]. Outline. Building Applications Gnu compilers, g++, g++, g77 - PowerPoint PPT PresentationTRANSCRIPT
Development ToolsFor HPC Applications
Deniz Savas, Michael GriffithsCorporate Information and Computing ServicesThe University of SheffieldEmail [email protected], [email protected]
• Building Applications– Gnu compilers, g++, g++, g77– Portland compilers, pg++, pgf77
• Calling fortran from C and C from fortran
• Using, Building and installing libraries• Using the make utility• The Eclipse Development Environment• Links
Outline
Compilers
Language GNU Portland
C gcc pgcc
C++ g++ pgCC
Fortran 77 g77 pgf77
Fortran90/95 pgf90
Invoking the Compiler
• Compiling FORTRAN Programs– pgf77 –o mycode [options] mycode.f
• Compiling c/c++ Programs– pgcc –o mycode [options] mycode.c
Options Used with Both gnu and Portland Compilers
Option Action-c Compile, do not link.
-o exefile Specifies a name for the resulting executable.
-g Produce debugging information (no optimization).
-Ilibrary_name (lower case L)
Link the given library into the program.
e.g. include math library by using option -lm
-ldirectory-name (upper case I)
Add directory to search path for include files
-O N Set optimisation level to N
-Dmacro[=defn] Define a macro
Options Used with Portland Compilers
Option Action-tp k8-64 Specify target processor type to be opteron processor
running 64 bit system (Portland only).
-Mvect=sse2 Vectorizer option, turn on streaming SIMD extensions (SSE) and SSE2 instructions. SSE2 instructions operate on 64bit floating point data
-Mvect=prefetch Generate prefetch instructions
-Mconcur Auto parallelization option
-fastsse Optimal set of options for processors supporting SSE/SSE2, Streaming SIMD Extensions
-fast Full optimisation with function unrolling and code reordering (Portland only).
-g77 libs Link time option allowing object files generated by g77 to be linked into programs (may cause problems with parallel libraries)
-Mbounds Check arrays for out of bounds access
Options Used with gnu CompilersOption Action TO CHECK--version Return version information
-mtune=cpu-type
-march=cpu-type
I386-I64 processor specification options.
-mmmx -msse
-msse2
Options to enable sse/sse2 or mmx(windows)
-Wall Show all warnings
-Wno-deprecated Switch off warnings about use of deprecated function (c++ only)
Linking a FORTRAN application with NAG libraries• NAG best and most comprehensive library of
numerical computing routines available• Mark20 on iceberg use –lnag and –lacml with the
pgf77 or pgf90 compiler• Example
– pgf90 myprogf90 –lnag –lacml
• See comprehensive documentation at– https://iceberg.shef.ac.uk/docs/nag/index.html
The AMD core math libraries (acml)
• ACML consists of the following main components:– A full implementation of Level 1, 2 and 3 Basic Linear
Algebra Subroutines (BLAS), with key routines optimized for high performance on AMD Opteron™ processors.
– A full suite of Linear Algebra (LAPACK) routines. As well as taking advantage of the highly-tuned BLAS kernels, a key set of LAPACK routines has been further optimized to achieve considerably higher performance than standard LAPACK implementations.
– A comprehensive suite of Fast Fourier Transforms (FFTs) in both single-, double-, single-complex and double-complex data types.
Using the acml libraries
• Building an application using the portland compilers– pgcc myapp.c -I/opt/acml-pg2.6.0/pgi64/include -L/opt/acml-
pg2.6.0/pgi64/lib -lm –lacml
• Building an application using the gnu compilers– gcc myapp.c -I/opt/acml-gnu2.6.0/gnu64/include
-L/opt/acml-gnu2.6.0/gnu64/lib -lm –lacml
• Examples– Documentation at
https://iceberg.shef.ac.uk/docs/acmldoc/html/index.html– See
http://www.shef.ac.uk/wrgrid/documents/hpc/numlibs.html#acmlexamples
Building Large Applications
• Typically compile program using– g++ –o myprog myprog.c –lm –g
• Large programs– Modularized– Combine into a single executable
• Building large applications is a multi step process– Compile each source file– Link resulting objects into an executable
Example Multi Source Program:1
• To build the Monte-carlo model, mc, we do the following.– g++ –c –g mc.cpp– g++ –c –g mc_system.cpp– g++ –c –g mc_particle.cpp– g++ –c -g mc_statistics.cpp– g++ –o mc mc.o mc_system.o mc_particle.o mc_statistics.o
–lm• Note: only one of the sources has a main function
Example Multi Source Program:2
• If mc_system.cpp is edited we don’t need to recompile– mc_statistics, mc_particle or mc
• Rebuild the application as follows– g++ –c –g mc_system.cpp
– g++ –o mc mc.o mc_system.o mc_particle.o mc_statistics.o –lm
• Automate these steps using make
Libraries
• Libraries are packaged collections of object files– Standard library contains printf… etc..– Maths library contains sin, cos etc..
• Specify additional libraries with –l<name>– Only standard library is provided automatically
• To compile a program with a maths library– g++ –c myprog myprog.c -lm
Building your own library
• Benefits of building libraries– Share standardised functions with community
– Separate functionality from detailed code
– Good way of packing up your most useful routines and reusing them
• How to build– Build libraries using
– Named as lib<name>.a or lib<name>.so
– http://www-cs.canisius.edu/PL_TUTORIALS/C/C-UNIX/libraries
Example
• Example my util library– g++ -c vec.cc
• Generates vec.o– g++ -c mat.cc
• Generates mat.o• Add object files to library
– ar r myutillib.a vec.o– ar r mylibutil.a mat.o
• Don’t use –l for your own libraries link as follows– g++ myprog.cc mylib.a –o myprog
Installing a Library
• General steps– Download and uncompress source– Read documentation and build e.g. using configure– make and make install to build and install– Update your environment
• Set LD_LIBRARY_PATH
• Compile with -lMyNewLib
Using the Make Utility
• Used to compile and link programs• Makefile tells make how to perform link and
compilation• Consists of rules with the following shape
target …… : dependencies ……
command
……………
make
• target name of file generated by a program• dependency used as input to create target• Target files are created whenever a dependency has changed• Commands can include
– cc, CC, g++, f77, f95, mpf77
• make• make clean
make target
• Perform actions to obtain a target from a set of dependecies
• Make checks when dependencies last updated
target : dependencies
rule
Simple Makefile ….. almost trivial!game : game.o
gcc -o game game.o game.o : game.c
gcc -c game.c
clean :rm game game.o
Simple Makefile
• Generates executable called game from a single source file called game.c
• Has a sequence of rules– game
• Rule for building target executable file
– game.o• Rule for building object files
– clean• Rule for cleaning executable and object files
Make multiple source file projectproject : main.o data.o io.o
CC -o project main.o data.o io.o main.o : main.c io.h data.h
CC -c main.c
data.o : data.c io.h data.hCC -c data.c
io.o : io.c io.hCC -c io.c
clean :rm project main.o data.o io.o
Hints for Building Makefiles
• Use # at the start of a line for comments• Use \ at the end of a line for line continuation• The line defining the rule that follows the definition of
target and dependencies should normally be indented using a tab character and NOT whitespace characters
Makefile with implict rules for compiling a static library
objects = vec.o vecpair.o mat.o
flags = -fast -tp k8-64 libmyutil.a : $(objects)
ar -r -o myutil.a $(objects) $(flags)
vec.o : vec.cpgCC -c vec.c $(flags)
vecpair.o : vecpair.cpgCC -c vecpair.c $(flags)
mat.o : mat.cpgCC -c mat.c $(flags)
clean :rm myutil.a $(objects)
Macros Used with Makefiles
$@ Full name of the current target . $< The source file of the current (single) dependency . $* The part of a filename which matched a suffix rule. $? The names of all the dependencies newer than the target separated by spaces. $^ The names of all the dependencies separated by spaces, but with duplicate names removed.
Suffixes
• Make uses a special target, named .SUFFIXES to allow you to define your own suffixes.
• For example, the dependency line:.SUFFIXES: .foo .bar– tells make that you will be using these special suffixes to
make your own rules.
Custom Suffix Rule
• Similar to how make already knows how to make a .o file from a .c file, you can define rules in the following manner:
.foo.bar: tr '[A-Z][a-z]' '[N-Z][A-M][n-z][a-m]' < $< > $@ .c.o: $(CC) $(CFLAGS) -c $<
• The first rule allows you to create a .bar file from a .foo file. (Don't worry about what it does, it basically scrambles the file.)
• The second rule is the default rule used by make to create a .o file from a .c file.
Makefile with suffix rule
objects = blastest.o
flags = -fast -tp k8-64
mk4 : $(objects)pgCC -o mk4 $(objects) $(flags)
.c.o:pgCC -c $(flags) $<
clean :rm mk4 $(objects)
Using Eclipse
• Advantages• Starting
Using Eclipse: Advantages
• Open source• Available for many platforms
– Windows requires cygwin and gnu development tools g++, g77, gdb, gmake, stl etc..
• Use to develop wide variety of applications in a single development environment– e.g. c, c++, f77, f90, java
Eclipse features
• Perspectives for java, C/C++, debug and soon fortran development
• Multiple projects• Browsers
– Help, members, types, namespaces
• Interactive debugging• Build projects using make or ant
Starting eclipse
• Type “eclipse”• Requests directory for workspace
– N.B. Sometimes necessary to start eclipse using– eclipse -vm $JAVA_HOME/jre/bin/java
Eclipse C/C++ Perspective Layout
Creating a new project
• Managed make project– Helloworld
• Standard make project– Hellotest– Provide a make file
Creating a standard make project
Editing Project Settings
Finishing Steps
Debug WindowSwitch
between perspecti
ves
Debug window
s
Debug steppingcontrols
Running the Debugger
Setting and Modifying Breakpoint Properties
Right click here to toggle breakpoi
nt
Right click here
to edit breakpoin
t propertie
s
Links
• http://www.eclipse.org/• http://www-106.ibm.com/developerworks/library/os-e
cc/– A useful tutorial on the eclipse cdt
• http://www.cplusplus.com– Very useful reference section