Introduction to Python

for Scientific Computinghttp://tinyurl.com/cq-intro-python-20160217

By: Bart Oldeman, Calcul Quebec – McGill HPC

Bart.Oldeman@calculquebec.ca, Bart.Oldeman@mcgill.ca

1

Partners and Sponsors

2

Outline of the workshop

• What is Python? Why is it useful?

• How do we run Python codes, including on the Guillimin cluster?

• How to program with Python?

• How to use scientific packages in Python (NumPy, SciPy,Matplotlib)?

• Practical exercises on Guillimin

• Note: These slides are based on the lectures notes here:http://scipy-lectures.github.io/

3

What is Python?

• Python is a high-level, general-purpose programming language.

• Readable code.

• (Usually) Interpreted, not compiled.

• Dynamically typed.

• Original author and “Benevolent Dictator For Life (BFDL)”: Guidovan Rossum (originally at CWI, Amsterdam, now at Dropbox).

• Named after Monty Python, Van Rossum’s favourite TV show.

• History: Started 1989, Python 1.0 (1994), Python 2.0 (2000),Python 3.0 (2008). Using Python 3.5.0 in this workshop.

4

Why is it useful?

• Provides a general-purpose, open-source, high-level glue betweencomputations.

• Alternative to programs such as MATLAB, R, Julia, IDL, etc., formany numerical computations. Python is less specialized, which haspros:

• easier to interface with non-scientific code• can do almost everything

and cons• slightly more cumbersome syntax for arrays

• Used in many disciplines, e.g. bioinformatics, mathematics,statistics, biology, economics, physics, electrical engineering,geosciences, signal processing.

• Groups on Guillimin use it, for instance, to study• Gene flow in animal populations• Climate change• Searching for pulsars• Computational Fluid Dynamics

5

Scientific Python stack

Python The base language. It has “batteries included” but no fastarray type by default.

Numpy Provides efficient powerful numerical array type withassociated routines. Almost all scientific software written inPython uses Numpy.

Scipy Higher level than Numpy, provides many data processingroutines, for instance optimization, regression,interpolation.

Matplotlib Most popular 2D (and basic 3D) plotting library.

IPython Advance Python shell that integrates well with scientificPython packages.

Mayavi 3D visualization.

Mpi4py Use MPI from Python.

Cython C-extensions for Python.

Sympy Symbolic mathematics.

6

Exercise 1:Log in to Guillimin, setting up the environmentWindows users: please use http://mobaxterm.mobatek.net/.

1) Log in to Guillimin:ssh -X class##@guillimin.hpc.mcgill.ca

2) Check for loaded software modules:guillimin> module list

3) See all available modules:guillimin> module av

4) Load necessary modules: (needs ~/.lmod legacy until March 15)guillimin> module load iomkl/2015b Python/3.5.0

5) Check loaded modules again6) Verify that you have access to the correct Python package:

guillimin> which python3

output: /software/CentOS-6/eb/software/Toolchain/iomkl/2015b/Python/3.5.0/bin/python3

7

Interactive job submission

1) Copy all workshop files to your home directory:

guillimin> cp -a /software/workshop/python/* ./

2) Inspect the file “interactive.pbs”:

guillimin> cat interactive.pbs

#!/bin/bash

#PBS -l nodes=1:ppn=1,walltime=06:00:00

#PBS -N python-interactive

#PBS -V

#PBS -X

#PBS -I

8

Interactive job submission

3) guillimin> qsub interactive.pbs

qsub: waiting for job 28812458.gm-1r16-n04.guillimin.clumeq.ca to start

qsub: job 28812458.gm-1r16-n04.guillimin.clumeq.ca ready

----------------------------------------

Begin PBS Prologue Wed Mar 11 12:51:09 EDT 2015 1426092669

Job ID: 28812458.gm-1r16-n04.guillimin.clumeq.ca

Username: class01

Group: class

Nodes: sw-2r08-n24

End PBS Prologue Wed Mar 11 12:51:09 EDT 2015 1426092669

----------------------------------------

[classxx@sw-2r08-n24 ~]$

9

Run “Hello World”

4) [classxx@sw-2r08-n24 ~]$ cat hello.py

print("Hello, world!")

[classxx@sw-2r08-n24 ~]$ python3 hello.py

Hello, world!

[classxx@sw-2r08-n24 ~]$ python3Python 3.5.0 (default, Sep 30 2015, 10:10:22)

[GCC Intel(R) C++ gcc 4.9 mode] on linux

Type "help", "copyright", "credits" or "license" for more information.

>>> print("Hello, world!")

Hello, world!

>>> quit()

Note: for Python 2 (unless you use Python 2.6+ with from future

import print function) use print "Hello, world!" instead.

10

Run “Hello World” (IPython)

5) [classxx@sw... ~]$ module load IPython/3.2.3-Python-3.5.0

[classxx@sw... ~]$ ipythonPython 3.5.0 (default, Sep 30 2015, 10:10:22)

Type "copyright", "credits" or "license" for more information.

IPython 3.2.3 -- An enhanced Interactive Python.

? -> Introduction and overview of IPython’s features.

%quickref -> Quick reference.

help -> Python’s own help system.

object? -> Details about ’object’, use ’object??’ for extra details.

In [1]: print("Hello, world!")

Hello, world!

In [2]: %run hello.py

Hello, world!

In [3]: quit

[classxx@sw... ~]$ module load PyQt/4.11.4-Python-3.5.0

[classxx@sw... ~]$ ipython qtconsole

# Brings up a GUI

11

Python script for I/O: examples/io.py

import math

import os

infile = "numbers"

outfile = "f_numbers"

f = open(infile , ’r’)

g = open(outfile , ’w’)

def func(y):

if y >= 0.0:

return y**5.0* math.exp(-y)

else:

return 0.0

for line in f:

line = line.split()

x, y = float(line [0]), float(line [1])

g.write("%g %12.5e\n" % (x,func(y)))

f.close()

g.close()

Run as cd data; python3 ../examples/io.py12

Setting up Jupyter

Jupyter (formally known as IPython Notebook) is a web-based interactivecomputational environment where execution, text, mathematics, plotsand rich media combine into a single document. We use it for theremainder of the workshop. Setting up Jupyter:

1. Open https://guillimin2.hpc.mcgill.ca:8443 in a webbrowser.

2. In the web page, trust this web site and login using your classaccount. Please “Continue” any unresponsive scripts.

Alternatively, you could install a scientific Python distribution such asContinuum’s Anaconda or Enthought’s Canopy and use Jupyter locally.In that case you can obtain the workshop files as follows:

yourlaptop> git clone -b mcgill \https://github.com/calculquebec/cq-formation-intro-python.git

yourlaptop> cd cq-formation-intro-python

13

Further reading:

• Python, NumPy, SciPy, Matplotlib and IPython can all be found atthe .org website, for instance:http://www.python.org

• Official non-scientific Python tutorial:http://docs.python.org/3/tutorial

• Two more scientific tutorials:http://github.com/jrjohansson/scientific-python-lectures

http://python4astronomers.github.io/

• Software Carpentry:http://swcarpentry.github.io/python-novice-inflammation/

• Followup questions? Contact guillimin@calculquebec.ca.

14