mpi4py.pdf
TRANSCRIPT
![Page 1: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/1.jpg)
MPI for Pythonhttp://mpi4py.scipy.org
Lisandro [email protected]
Centro Internacional de Metodos Computacionales en IngenierıaConsejo Nacional de Investigaciones Cientıficas y Tecnicas
Santa Fe, Argentina
January, 2011Python for parallel scientific computing
PASI, Valparaıso, Chile
![Page 2: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/2.jpg)
Outline
Overview
Communicators
Point to Point Communication
Collective Operations
Compute Pi
Mandelbrot Set
Dynamic Process Management
![Page 3: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/3.jpg)
Overview
Communicators
Point to Point Communication
Collective Operations
Compute Pi
Mandelbrot Set
Dynamic Process Management
![Page 4: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/4.jpg)
What is mpi4py?
I Full-featured Python bindings for MPI.
I API based on the standard MPI-2 C++ bindings.I Almost all MPI calls are supported.
I targeted to MPI-2 implementations.I also works with MPI-1 implementations.
![Page 5: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/5.jpg)
Implementation
Implemented with Cython
I Code base far easier to write, maintain, and extend.
I Faster than other solutions (mixed Python and C codes).
I A pythonic API that runs at C speed !
![Page 6: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/6.jpg)
Features – MPI-1
I Process groups and communication domains.I intracommunicatorsI intercommunicators
I Point to point communication.I blocking (send/recv)I nonblocking (isend/irecv + test/wait)
I Collective operations.I Synchronization (barrier)I Communication (broadcast, scatter/gather)I Global reductions (reduce, scan)
![Page 7: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/7.jpg)
Features – MPI-2
I Dynamic process management (spawn, connect/accept).
I Parallel I/O (read/write).
I One sided operations, a.k.a. RMA (put/get/accumulate).
I Extended collective operations.
![Page 8: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/8.jpg)
Features – Python
I Communication of Python objects.I high level and very convenient, based in pickle serializationI can be slow for large data (CPU and memory consuming)
<object> −→ pickle.dump() −→ send()↓
<object> ←− pickle.load() ←− recv()
I Communication of array data (e.g. NumPy arrays).I lower level, slightly more verboseI very fast, almost C speed (for messages above 5-10 KB)
message = [<object>, (count, displ), datatype]
![Page 9: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/9.jpg)
Point to Point Throughput – Gigabit Ethernet
100 101 102 103 104 105 106 107
Array Size [Bytes]
0
20
40
60
80
100
120
Thro
ughp
ut [M
iB/s
]
PingPong
PickleBufferC
![Page 10: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/10.jpg)
Point to Point Throughput – Shared Memory
100 101 102 103 104 105 106 107
Array Size [Bytes]
0
500
1000
1500
2000
2500
3000
3500
4000
4500
Thro
ughp
ut [M
iB/s
]
PingPong
PickleBufferC
![Page 11: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/11.jpg)
Features – IPython
Integration with IPython enables MPI to be used interactively.
I Start engines with MPI enabled
$ ipcluster mpiexec -n 16 --mpi=mpi4py
I Connect to the engines
$ ipythonIn [1]: from IPython.kernel import clientIn [2]: mec = client.MultiEngineClient()In [3]: mec.activate()
I Execute commands using %px
In [4]: %px from mpi4py import MPIIn [5]: %px print(MPI.Get_processor_name())
![Page 12: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/12.jpg)
Features – Interoperability
Good support for wrapping other MPI-based codes.
I You can use Cython (cimport statement).
I You can use SWIG (typemaps provided).
I You can use F2Py (py2f()/f2py() methods).
I You can use Boost::Python or hand-written C extensions.
mpi4py will allow you to use virtually any MPI basedC/C++/Fortran code from Python.
![Page 13: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/13.jpg)
Hello World!
1 from mpi4py import MPI2
3 rank = MPI.COMM_WORLD.Get_rank()4 size = MPI.COMM_WORLD.Get_size()5 name = MPI.Get_processor_name()6
7 print ("Hello, World! "8 "I am process %d of %d on %s" %9 (rank, size, name))
![Page 14: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/14.jpg)
Hello World! – Wrapping with SWIG
C source
1 /* file: helloworld.c */
2 void sayhello(MPI_Comm comm)
3 {
4 int size, rank;
5 MPI_Comm_size(comm, &size);
6 MPI_Comm_rank(comm, &rank);
7 printf("Hello, World! "
8 "I am process "
9 "%d of %d.\n",
10 rank, size);
11 }
SWIG interface file
1 // file: helloworld.i
2 %module helloworld
3 %{
4 # include <mpi.h>
5 # include "helloworld.c"
6 }%
7
8 %include mpi4py/mpi4py.i
9 %mpi4py_typemap(Comm, MPI_Comm);
10
11 void sayhello(MPI_Comm comm);
At the Python prompt . . .
>>> from mpi4py import MPI
>>> import helloworld
>>> helloworld.sayhello(MPI.COMM_WORLD)
Hello, World! I am process 0 of 1.
![Page 15: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/15.jpg)
Hello World! – Wrapping with Boost.Python
1 // file: helloworld.cxx
2 # include <boost / python.hpp>
3 # include <mpi4py / mpi4py.h>
4 using namespace boost::python;
5
6 # include "helloworld.c"
7 static void wrap_sayhello(object py_comm) {
8 PyObject* py_obj = py_comm.ptr();
9 MPI_Comm *comm_p = PyMPIComm_Get(py_obj);
10 if (comm_p == NULL) throw_error_already_set();
11 sayhello(*comm_p);
12 }
13
14 BOOST_PYTHON_MODULE(helloworld) {
15 if (import_mpi4py() < 0) return;
16 def("sayhello", wrap_sayhello);
17 }
![Page 16: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/16.jpg)
Hello World! – Wrapping with F2Py
Fortran 90 source
1 ! file: helloworld.f90
2 subroutine sayhello(comm)
3 use mpi
4 implicit none
5 integer :: comm, rank, size, ierr
6 call MPI_Comm_size(comm, size, ierr)
7 call MPI_Comm_rank(comm, rank, ierr)
8 print *, ’Hello, World! I am process ’,rank,’ of ’,size,’.’
9 end subroutine sayhello
At the Python prompt . . .
>>> from mpi4py import MPI
>>> import helloworld
>>> fcomm = MPI.COMM_WORLD.py2f()
>>> helloworld.sayhello(fcomm)
Hello, World! I am process 0 of 1.
![Page 17: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/17.jpg)
Overview
Communicators
Point to Point Communication
Collective Operations
Compute Pi
Mandelbrot Set
Dynamic Process Management
![Page 18: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/18.jpg)
Communicators
communicator = process group + communication context
I Predefined instancesI COMM WORLDI COMM SELFI COMM NULL
I AccessorsI rank = comm.Get rank() # or comm.rankI size = comm.Get size() # or comm.sizeI group = comm.Get group()
I Constructors
I newcomm = comm.Dup()I newcomm = comm.Create(group)I newcomm = comm.Split(color, key)
![Page 19: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/19.jpg)
Communicators – Create()
1 from mpi4py import MPI2
3 comm = MPI.COMM_WORLD4 group = comm.Get_group()5
6 newgroup = group.Excl([0])7 newcomm = comm.Create(newgroup)8
9 if comm.rank == 0:10 assert newcomm == MPI.COMM_NULL11 else:12 assert newcomm.size == comm.size - 113 assert newcomm.rank == comm.rank - 114
15 group.Free(); newgroup.Free()16 if newcomm: newcomm.Free()
![Page 20: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/20.jpg)
Communicators – Split()
1 from mpi4py import MPI2
3 world_rank = MPI.COMM_WORLD.Get_rank()4 world_size = MPI.COMM_WORLD.Get_size()5
6 if world_rank < world_size//2:7 color = 558 key = -world_rank9 else:
10 color = 7711 key = +world_rank12
13 newcomm = MPI.COMM_WORLD.Split(color, key)14 # ...
15 newcomm.Free()
![Page 21: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/21.jpg)
Exercise #1
a) Create a new process group containing the processes in thegroup of COMM WORLD with even rank. Use the new group tocreate a new communicator.Tip: use Group.Incl() or Group.Range incl()
b) Use Comm.Split() to split COMM WORLD in two halves.I The first half contains the processes with even rank in
COMM WORLD. The process rank ordering in the newcommunication is ascending.
I The second half contains the processes with odd rank inCOMM WORLD. The process rank ordering in the newcommunication is descending.
![Page 22: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/22.jpg)
Overview
Communicators
Point to Point Communication
Collective Operations
Compute Pi
Mandelbrot Set
Dynamic Process Management
![Page 23: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/23.jpg)
I Blocking communicationI Python objects
comm.send(obj, dest=0, tag=0)
obj = comm.recv(None, src=0, tag=0)
I Array data
comm.Send([array, count, datatype], dest=0, tag=0)
comm.Recv([array, count, datatype], src=0, tag=0)
I Nonblocking communicationI Python objects
request = comm.isend(object, dest=0, tag=0)}
request.Wait()
I Array data
req1 = comm.Isend([array, count, datatype], dest=0, tag=0)
req2 = comm.Irecv([array, count, datatype], src=0, tag=0)
MPI.Request.Waitall([req1, req2])}
![Page 24: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/24.jpg)
PingPong
1 from mpi4py import MPI2 comm = MPI.COMM_WORLD3 assert comm.size == 24
5 if comm.rank == 0:6 sendmsg = 7777 comm.send(sendmsg, dest=1, tag=55)8 recvmsg = comm.recv(source=1, tag=77)9 else:
10 recvmsg = comm.recv(source=0, tag=55)11 sendmsg = "abc"12 comm.send(sendmsg, dest=0, tag=77)
![Page 25: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/25.jpg)
PingPing
1 from mpi4py import MPI2 comm = MPI.COMM_WORLD3 assert comm.size == 24
5 if comm.rank == 0:6 sendmsg = 7777 target = 18 else:9 sendmsg = "abc"
10 target = 011
12 request = comm.isend(sendmsg, dest=target, tag=77)13 recvmsg = comm.recv(source=target, tag=77)14 request.Wait()
![Page 26: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/26.jpg)
Exchange
1 from mpi4py import MPI2 comm = MPI.COMM_WORLD3
4 sendmsg = [comm.rank]*35 right = (comm.rank + 1) % comm.size6 left = (comm.rank - 1) % comm.size7
8 req1 = comm.isend(sendmsg, dest=right)9 req2 = comm.isend(sendmsg, dest=left)
10 lmsg = comm.recv(source=left)11 rmsg = comm.recv(source=right)12
13 MPI.Request.Waitall([req1, req2])14 assert lmsg == [left] * 315 assert rmsg == [right] * 3
![Page 27: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/27.jpg)
PingPing with NumPy arrays
1 from mpi4py import MPI2 import numpy3 comm = MPI.COMM_WORLD4 assert comm.size == 25
6 if comm.rank == 0:7 array1 = numpy.arange(10000, dtype=’f’)8 array2 = numpy.empty(10000, dtype=’f’)9 target = 1
10 else:11 array1 = numpy.ones(10000, dtype=’f’)12 array2 = numpy.empty(10000, dtype=’f’)13 target = 014
15 request = comm.Isend([array1, MPI.FLOAT], dest=target)16 comm.Recv([array2, MPI.FLOAT], source=target)17 request.Wait()
![Page 28: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/28.jpg)
Exercise #2
a) Modify PingPong example to communicate NumPy arrays.Tip: use Comm.Send() and Comm.Recv()
b) Modify Exchange example to communicate NumPy arrays.Use nonblocking communication for both sending and receiving.Tip: use Comm.Isend() and Comm.Irecv()
![Page 29: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/29.jpg)
Overview
Communicators
Point to Point Communication
Collective Operations
Compute Pi
Mandelbrot Set
Dynamic Process Management
![Page 30: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/30.jpg)
9 2 C H A P T E R 4 . C O L L E C T I V E C O M M U N I C A T I O N
F i g u r e 4 . 1 : C o l l e c t i v e m o v e f u n c t i o n s i l l u s t r a t e d f o r a g r o u p o f s i x p r o c e s s e s . I n e a c h c a s e ,
e a c h r o w o f b o x e s r e p r e s e n t s d a t a l o c a t i o n s i n o n e p r o c e s s . T h u s , i n t h e b r o a d c a s t , i n i t i a l l y
j u s t t h e r s t p r o c e s s c o n t a i n s t h e d a t a A
0
, b u t a f t e r t h e b r o a d c a s t a l l p r o c e s s e s c o n t a i n i t .
1
2
3
4
5
6
7
8
9
1 0
1 1
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6
2 7
2 8
2 9
3 0
3 1
3 2
3 3
3 4
3 5
3 6
3 7
3 8
3 9
4 0
4 1
4 2
4 3
4 4
4 5
4 6
4 7
4 8
A0
A0
A0
A0
A0
A0
A0
A1
A2
A3
A4
A5 scatter
gather
A0
A1
A2
A3
A4
A5
A0
broadcast
data
pro
cess
es
data
pro
cess
es
![Page 31: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/31.jpg)
9 2 C H A P T E R 4 . C O L L E C T I V E C O M M U N I C A T I O N
F i g u r e 4 . 1 : C o l l e c t i v e m o v e f u n c t i o n s i l l u s t r a t e d f o r a g r o u p o f s i x p r o c e s s e s . I n e a c h c a s e ,
e a c h r o w o f b o x e s r e p r e s e n t s d a t a l o c a t i o n s i n o n e p r o c e s s . T h u s , i n t h e b r o a d c a s t , i n i t i a l l y
j u s t t h e r s t p r o c e s s c o n t a i n s t h e d a t a A
0
, b u t a f t e r t h e b r o a d c a s t a l l p r o c e s s e s c o n t a i n i t .
1
2
3
4
5
6
7
8
9
1 0
1 1
1 2
1 3
1 4
1 5
1 6
1 7
1 8
1 9
2 0
2 1
2 2
2 3
2 4
2 5
2 6
2 7
2 8
2 9
3 0
3 1
3 2
3 3
3 4
3 5
3 6
3 7
3 8
3 9
4 0
4 1
4 2
4 3
4 4
4 5
4 6
4 7
4 8
A0
A1
A2
A3
A4
A5
B0
B1
B2
B3
B4
B5
C0
C1
C2
C3
C4
C5
D0
D1
D2
D3
D4
D5
E0
E1
E2
E3
E4
E5
F0
F1
F2
F3
F4
F5
A0
B0
C0
D0
E0
F0
A1
B1
C1
D1
E1
F1
A2
B2
C2
D2
E2
F2
A3
B3
C3
D3
E3
F3
A4
B4
C4
D4
E4
F4
A5
B5
C5
D5
E5
F5
alltoall
A0
B0
C0
D0
E0
F0
allgather
A0
B0
C0
D0
E0
F0
A0
B0
C0
D0
E0
F0
A0
B0
C0
D0
E0
F0
A0
B0
C0
D0
E0
F0
A0
B0
C0
D0
E0
F0
A0
B0
C0
D0
E0
F0
data
pro
cess
es
data
pro
cess
es
![Page 32: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/32.jpg)
Broadcast
1 from mpi4py import MPI2 comm = MPI.COMM_WORLD3
4 if comm.rank == 0:5 sendmsg = (7, "abc", [1.0,2+3j], {3:4})6 else:7 sendmsg = None8
9 recvmsg = comm.bcast(sendmsg, root=0)
![Page 33: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/33.jpg)
Scatter
1 from mpi4py import MPI2 comm = MPI.COMM_WORLD3
4 if comm.rank == 0:5 sendmsg = [i**2 for i in range(comm.size)]6 else:7 sendmsg = None8
9 recvmsg = comm.scatter(sendmsg, root=0)
![Page 34: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/34.jpg)
Gather & Gather to All
1 from mpi4py import MPI2 comm = MPI.COMM_WORLD3
4 sendmsg = comm.rank**25
6 recvmsg1 = comm.gather(sendmsg, root=0)7
8 recvmsg2 = comm.allgather(sendmsg)
![Page 35: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/35.jpg)
Reduce & Reduce to All
1 from mpi4py import MPI2 comm = MPI.COMM_WORLD3
4 sendmsg = comm.rank5
6 recvmsg1 = comm.reduce(sendmsg, op=MPI.SUM, root=0)7
8 recvmsg2 = comm.allreduce(sendmsg)
![Page 36: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/36.jpg)
Exercise #3
a) Modify Broadcast, Scatter, and Gather example tocommunicate NumPy arrays.
b) Write a routine implementing parallel matrix–vector producty = matvec(comm,A,x).
I the global matrix is dense and square.I matrix rows and vector entries have matching block distribution.I all processes own the same number of matrix rows.
Tip: use Comm.Allgather() and numpy.dot()
![Page 37: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/37.jpg)
Overview
Communicators
Point to Point Communication
Collective Operations
Compute Pi
Mandelbrot Set
Dynamic Process Management
![Page 38: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/38.jpg)
Compute Pi
π =
∫ 1
0
4
1 + x2dx ≈ 1
n
n−1∑i=0
4
1 + ( i+0.5n )2
![Page 39: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/39.jpg)
Compute Pi – sequential
1 import math2
3 def compute_pi(n):4 h = 1.0 / n5 s = 0.06 for i in range(n):7 x = h * (i + 0.5)8 s += 4.0 / (1.0 + x**2)9 return s * h
10
11 n = 1012 pi = compute_pi(n)13 error = abs(pi - math.pi)14
15 print ("pi is approximately %.16f, "16 "error is %.16f" % (pi, error))
![Page 40: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/40.jpg)
Compute Pi – parallel [1]
1 from mpi4py import MPI2 import math3
4 def compute_pi(n, start=0, step=1):5 h = 1.0 / n6 s = 0.07 for i in range(start, n, step):8 x = h * (i + 0.5)9 s += 4.0 / (1.0 + x**2)
10 return s * h11
12 comm = MPI.COMM_WORLD13 nprocs = comm.Get_size()14 myrank = comm.Get_rank()
![Page 41: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/41.jpg)
Compute Pi – parallel [2]
1 if myrank == 0:2 n = 103 else:4 n = None5
6 n = comm.bcast(n, root=0)7
8 mypi = compute_pi(n, myrank, nprocs)9
10 pi = comm.reduce(mypi, op=MPI.SUM, root=0)11
12 if myrank == 0:13 error = abs(pi - math.pi)14 print ("pi is approximately %.16f, "15 "error is %.16f" % (pi, error))
![Page 42: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/42.jpg)
Exercise #4
Modify Compute Pi example to employ NumPy.
Tip: you can convert a Python int/float object to a NumPyscalar with x = numpy.array(x).
![Page 43: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/43.jpg)
Overview
Communicators
Point to Point Communication
Collective Operations
Compute Pi
Mandelbrot Set
Dynamic Process Management
![Page 44: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/44.jpg)
Mandelbrot Set
![Page 45: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/45.jpg)
Mandelbrot Set – sequential [1]
1 def mandelbrot(x, y, maxit):2 c = x + y*1j3 z = 0 + 0j4 it = 05 while abs(z) < 2 and it < maxit:6 z = z**2 + c7 it += 18 return it9
10 x1, x2 = -2.0, 1.011 y1, y2 = -1.0, 1.012 w, h = 150, 10013 maxit = 127
![Page 46: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/46.jpg)
Mandelbrot Set – sequential [2]
1 import numpy2 C = numpy.zeros([h, w], dtype=’i’)3 dx = (x2 - x1) / w4 dy = (y2 - y1) / h5 for i in range(h):6 y = y1 + i * dy7 for j in range(w):8 x = x1 + j * dx9 C[i, j] = mandelbrot(x, y, maxit)
10
11 from matplotlib import pyplot12 pyplot.imshow(C, aspect=’equal’)13 pyplot.spectral()14 pyplot.show()
![Page 47: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/47.jpg)
Mandelbrot Set – partitioning
0
1
2
Block distribution
012
012
012
Cyclic distribution
![Page 48: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/48.jpg)
Mandelbrot Set – parallel, block [1]
1 def mandelbrot(x, y, maxit):2 c = x + y*1j3 z = 0 + 0j4 it = 05 while abs(z) < 2 and it < maxit:6 z = z**2 + c7 it += 18 return it9
10 x1, x2 = -2.0, 1.011 y1, y2 = -1.0, 1.012 w, h = 150, 10013 maxit = 127
![Page 49: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/49.jpg)
Mandelbrot Set – parallel, block [2]
1 from mpi4py import MPI2 import numpy3
4 comm = MPI.COMM_WORLD5 size = comm.Get_size()6 rank = comm.Get_rank()7
8 # number of rows to compute here
9 N = h // size + (h % size > rank)10
11 # first row to compute here
12 start = comm.scan(N)-N13
14 # array to store local result
15 Cl = numpy.zeros([N, w], dtype=’i’)
![Page 50: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/50.jpg)
Mandelbrot Set – parallel, block [3]
1 # compute owned rows
2
3 dx = (x2 - x1) / w4 dy = (y2 - y1) / h5 for i in range(N):6 y = y1 + (i + start) * dy7 for j in range(w):8 x = x1 + j * dx9 Cl[i, j] = mandelbrot(x, y, maxit)
![Page 51: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/51.jpg)
Mandelbrot Set – parallel, block [4]
1 # gather results at root (process 0)
2
3 counts = comm.gather(N, root=0)4 C = None5 if rank == 0:6 C = numpy.zeros([h, w], dtype=’i’)7
8 rowtype = MPI.INT.Create_contiguous(w)9 rowtype.Commit()
10
11 comm.Gatherv(sendbuf=[Cl, MPI.INT],12 recvbuf=[C, (counts, None), rowtype],13 root=0)14
15 rowtype.Free()
![Page 52: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/52.jpg)
Mandelbrot Set – parallel, block [5]
1 if comm.rank == 0:2 from matplotlib import pyplot3 pyplot.imshow(C, aspect=’equal’)4 pyplot.spectral()5 pyplot.show()
![Page 53: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/53.jpg)
Exercise #5
Measure the wall clock time Ti of local computations at eachprocess for the Mandelbrot Set example with block and cyclic rowdistributions.What row distribution is better regarding load balancing?
Tip: use Wtime() to measure wall time, compute the ratioTmax/Tmin to compare load balancing.
![Page 54: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/54.jpg)
Overview
Communicators
Point to Point Communication
Collective Operations
Compute Pi
Mandelbrot Set
Dynamic Process Management
![Page 55: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/55.jpg)
Dynamic Process Management
I Useful in assembling complex distributed applications. Cancouple independent parallel codes written in differentlanguages.
I Create new processes from a running program.– Comm.Spawn() and Comm.Get parent()
I Connect two running applications together.– Comm.Connect() and Comm.Accept()
![Page 56: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/56.jpg)
Dynamic Process Management – Spawning
Spawning new processes is a collective operation that creates anintercommunicator.
I Local group is group of spawning processes (parent).
I Remote group is group of new processes (child).
I Comm.Spawn() lets parent processes spawn the childprocesses. It returns a new intercommunicator.
I Comm.Get parent() lets child processes findintercommunicator to the parent group. Child processes haveown COMM WORLD.
I Comm.Disconnect() ends the parent–child connection. Afterthat, both groups can continue running.
![Page 57: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/57.jpg)
Dynamic Process Management – Compute Pi (parent)
1 from mpi4py import MPI2 import sys, numpy3
4 comm = MPI.COMM_SELF.Spawn(sys.executable,5 args=[’compute_pi-child.py’],6 maxprocs=5)7
8 N = numpy.array(10, ’i’)9 comm.Bcast([N, MPI.INT], root=MPI.ROOT)
10 PI = numpy.array(0.0, ’d’)11 comm.Reduce(None, [PI, MPI.DOUBLE],12 op=MPI.SUM, root=MPI.ROOT)13 comm.Disconnect()14
15 error = abs(PI - numpy.pi)16 print ("pi is approximately %.16f, "17 "error is %.16f" % (PI, error))
![Page 58: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/58.jpg)
Dynamic Process Management – Compute Pi (child)
1 from mpi4py import MPI2 import numpy3
4 comm = MPI.Comm.Get_parent()5 size = comm.Get_size()6 rank = comm.Get_rank()7
8 N = numpy.array(0, dtype=’i’)9 comm.Bcast([N, MPI.INT], root=0)
10 h = 1.0 / N; s = 0.011 for i in range(rank, N, size):12 x = h * (i + 0.5)13 s += 4.0 / (1.0 + x**2)14 PI = numpy.array(s * h, dtype=’d’)15 comm.Reduce([PI, MPI.DOUBLE], None,16 op=MPI.SUM, root=0)17
18 comm.Disconnect()
![Page 59: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/59.jpg)
Exercise #5
a) Implement the Compute Pi child code in C or C++ . Adjustthe parent code in Python to spawn the new implementation.
b) Compute and plot the Mandelbrot Set using spawning withparent/child codes implemented in Python.Tip: Reuse the provided parent code in Python and translatethe child code in Fortran 90 to Python.
![Page 60: mpi4py.pdf](https://reader035.vdocuments.site/reader035/viewer/2022062319/55504411b4c9058f768b4c4e/html5/thumbnails/60.jpg)
Do not hesitate to ask for help . . .
I Mailing List: [email protected]
I Mail&Chat: [email protected]
Thanks!