University of Minnesota

Comments on Co-Array Fortran

Robert W. NumrichMinnesota Supercomputing Institute

University of Minnesota, Minneapolis

2

Philosophy Behind the CAF Model• A minimum number of new features that look and feel like

Fortran.• The rules for co-dimensions are the same as those for

normal dimensions with a few exceptions.• The CAF model is purely local.• The compiler performs normal optimization between

synchronization points.• Compiler is not required to perform, is not expected to

perform, is deliberately prevented from performing, global optimization.

• Programmer is responsible for explicit data distribution, explicit communication, and explicit synchronization.

• Programmer is responsible for memory consistency.

3

The Essential Features of CAF

Co-array syntax• Co-dimensions (How images are related to each other)• Data communication (How data moves between images)

Synchronization• Full barrier• Pair-wise handshake

Dynamic Memory management• Allocatable co-arrays• Allocatable/pointer components of co-array derived

types

4

Recommendations

Delete all of Section 8.5• Substitute SYNC[([myPal])]

Delete everything on I/O except • Only image 1 reads stdin• Records from different images not mixed to stdout

Delete all the stuff on collectives• Substitute the functions glbSum(x), glbMin(x), glbMax(x)• Argument is not necessarily a co-array.

5

Ragged Arrays

1. The most important, the most powerful, feature of the CAF model.

• Allows each image to have data structures with different sizes and different locations on each image.

• No other model can handle this.• Fortran with CAF extensions handles it in a very natural way.

2. Allocatable/pointer components of co-array derived typestype x real,allocatable,target :: z(:) real,pointer :: ptr(:) type(x) :: y[]end type x allocate(z(someRule(this_image))) y%ptr => z sync y[p]%ptr = 25.0

• Most difficult feature to implement on crummy hardware and/or crummy operating systems.

• Good systems should be rewarded for getting this right.• We should not allow bad systems to drag everything down to

the lowest common denominator.

6

Memory Consistency

• The programmer is responsible for maintaining memory consistency, not the compiler.

• The rules must be very simple and very clear.• With just SYNC, it is very simple• The addition of NOTIFY/QUERY makes it not so simple• Segment boundary statements

- Compiler may optimize between segment boundaries but not across segment boundaries

- Processor must make co-arrays “visible” to all images across segment boundaries

- The programmer must make sure that one and only one image defines a co-array variable “at the same time”

- The programmer must make sure that no image tries to reference a co-array variable “at the same time” as another image is trying to define it.

7

Input/Output

• What’s the minimum needed?– stdin/stdout are special cases

– Always connected to all images– Only image 1 can read stdin– System must not mix records to stdout from different

images

– Shared files– Allow each image to open to same unit– Direct access only

– Allowing sequential access requires changes to backspace, rewind, etc.

– open(unit=u,access=,…)– open(unit=u,access=connectList,…)

– Do we need teams?– Do we need to sync?

8

Collectives• Avoid language bloat

- Not everything in the MPI Library needs to be reproduced in CAF as intrinsic procedures

- What do we really need and want?- What should the interface be?

• CAF is intended as a low-level language- Collectives are easy to write in CAF for any specific

procedure- Throwing a long list of new intrinsic procedures over the

wall may discourage vendors from adopting CAF

• If anything, supply intrinsic functions: glbSum(x), glbMin(x), glbMax(x) argument x need not be a co-array

• Propose a supporting library for CAF.

9

VOLATILE Co-arrays?

• What do we want the VOLATILE attribute to mean for co-arrays?

- Inhibit optimization? - Always read from memory?- Always flush to memory?

• Can VOLATILE make spin-loops work without the need for an artificial sync-memory() from the programmer?

- Statements with co-arrays are segment boundary statements

10

SYNC/BARRIER

• One new statement SYNC- Implies full BARRIER for all images - No arguments

• No SYNC_IMAGES- Synchronization between subsets of images

can be done with NOTIFY/QUERY

11

NOTIFY/QUERY

• Why do we want NOTIFY/QUERY?- Split-phase sync - Subset sync- Master-slave work distribution- Should they match in pairs?- Should we expose which notify matches which query?

• Maybe we really want EVENTS?- EVENT_POST(tag)- EVENT_WAIT(tag)- EVENT_CLEAR(tag)