a constraint-based approach to automatic data …...2019/09/05 · a constraint-based approach to...

A Constraint-Based Approach to Automatic Data Partitioning

Wonchan Lee Computer Science Department

Stanford University

Advanced Modeling & Simulation (AMS) Seminar Series NASA Ames Research Center, September 5th, 2019

Auto-Parallelizers for Distributed Systems

• Goal: automatically generate distributed memory code from sequential programs

• Focus on data parallel programs

• Numerous efforts in the past several decades

• High Performance Fortran and its predecessors (Fortran D, Vienna Fortran)

• Polyhedral compilers for distributed memory machines

2Why are they not being used more widely?

Issue 1: Configurability

3

for i in A: A[i] = f(i)

for i in B: B[i] = g(A[i+1])

Sequential Code

parallel for p = 0, P: int S = N/P // N = |A| int lA[S+1], lB[S]

for i = 0, S: lA[i] = f(p*S+i)

if p > 1: send(p-1, lA[0]) if p < P-1: recv(p+1, &lA[S])

for i = 0, S: lB[i] = g(lA[i+1])

Distributed Code

Partitioning of A and B

Decomposition of loop iterations

Inter-node communication

{<latexit sha1_base64="+oiEDX8T5j+i1nAJVMd2QImdMsA=">AAAEHHicdZNNb9MwGMe9ZcAIbxscuUREk7pLFZdqHbcJLhwHotukJJps12mtOk5kP2GronwDrnDk03BDXJH4NrhpqmxdZsnKX8//97zYkWkuhYEg+Le17ew8ePho97H75Omz5y/29l+emazQjI9ZJjN9QYnhUig+BgGSX+Sak5RKfk7nH5b++VeujcjUF1jkPE7JVIlEMAI29DkqL/f8oP/u+GgwPPKCfhCM8AAvxWA0fDv0sI0sl4+adXq57+xEk4wVKVfAJDEmxEEOcUk0CCZ55UaF4TlhczLloZWKpNzEZT1q5R3YyMRLMm23Aq+O3swoSWrMIqWWTAnMzKa3DHZ5YQHJcVwKlRfAFVs1SgrpQeYtz+1NhOYM5MIKwrSws3psRjRhYG/HdSPFr1iWpkRNyiiltApxbIVtQ2np46raIFgLMCK7CNESAroA0wIm6QRuNDGsiwBoCehqAmsC+DXcQ9BbRP2lq3kq1z3wcqLtn55xI8xmZm012Ta1V/m4FocdTexNzzm0cLiG426Yt2hUrtmountJ66J24uiOS9RUNpVkretSupGbdHueno8PN11Kde2FPo4r176b9ePw7hdngz4O+vjT0D9537ygXfQavUE9hNEInaCP6BSNEUMJ+oa+ox/OT+eX89v5s0K3t5qcV+jWcv7+B5ioZ7o=</latexit> <latexit sha1_base64="+oiEDX8T5j+i1nAJVMd2QImdMsA=">AAAEHHicdZNNb9MwGMe9ZcAIbxscuUREk7pLFZdqHbcJLhwHotukJJps12mtOk5kP2GronwDrnDk03BDXJH4NrhpqmxdZsnKX8//97zYkWkuhYEg+Le17ew8ePho97H75Omz5y/29l+emazQjI9ZJjN9QYnhUig+BgGSX+Sak5RKfk7nH5b++VeujcjUF1jkPE7JVIlEMAI29DkqL/f8oP/u+GgwPPKCfhCM8AAvxWA0fDv0sI0sl4+adXq57+xEk4wVKVfAJDEmxEEOcUk0CCZ55UaF4TlhczLloZWKpNzEZT1q5R3YyMRLMm23Aq+O3swoSWrMIqWWTAnMzKa3DHZ5YQHJcVwKlRfAFVs1SgrpQeYtz+1NhOYM5MIKwrSws3psRjRhYG/HdSPFr1iWpkRNyiiltApxbIVtQ2np46raIFgLMCK7CNESAroA0wIm6QRuNDGsiwBoCehqAmsC+DXcQ9BbRP2lq3kq1z3wcqLtn55xI8xmZm012Ta1V/m4FocdTexNzzm0cLiG426Yt2hUrtmountJ66J24uiOS9RUNpVkretSupGbdHueno8PN11Kde2FPo4r176b9ePw7hdngz4O+vjT0D9537ygXfQavUE9hNEInaCP6BSNEUMJ+oa+ox/OT+eX89v5s0K3t5qcV+jWcv7+B5ioZ7o=</latexit> <latexit sha1_base64="+oiEDX8T5j+i1nAJVMd2QImdMsA=">AAAEHHicdZNNb9MwGMe9ZcAIbxscuUREk7pLFZdqHbcJLhwHotukJJps12mtOk5kP2GronwDrnDk03BDXJH4NrhpqmxdZsnKX8//97zYkWkuhYEg+Le17ew8ePho97H75Omz5y/29l+emazQjI9ZJjN9QYnhUig+BgGSX+Sak5RKfk7nH5b++VeujcjUF1jkPE7JVIlEMAI29DkqL/f8oP/u+GgwPPKCfhCM8AAvxWA0fDv0sI0sl4+adXq57+xEk4wVKVfAJDEmxEEOcUk0CCZ55UaF4TlhczLloZWKpNzEZT1q5R3YyMRLMm23Aq+O3swoSWrMIqWWTAnMzKa3DHZ5YQHJcVwKlRfAFVs1SgrpQeYtz+1NhOYM5MIKwrSws3psRjRhYG/HdSPFr1iWpkRNyiiltApxbIVtQ2np46raIFgLMCK7CNESAroA0wIm6QRuNDGsiwBoCehqAmsC+DXcQ9BbRP2lq3kq1z3wcqLtn55xI8xmZm012Ta1V/m4FocdTexNzzm0cLiG426Yt2hUrtmountJ66J24uiOS9RUNpVkretSupGbdHueno8PN11Kde2FPo4r176b9ePw7hdngz4O+vjT0D9537ygXfQavUE9hNEInaCP6BSNEUMJ+oa+ox/OT+eX89v5s0K3t5qcV+jWcv7+B5ioZ7o=</latexit> <latexit sha1_base64="+oiEDX8T5j+i1nAJVMd2QImdMsA=">AAAEHHicdZNNb9MwGMe9ZcAIbxscuUREk7pLFZdqHbcJLhwHotukJJps12mtOk5kP2GronwDrnDk03BDXJH4NrhpqmxdZsnKX8//97zYkWkuhYEg+Le17ew8ePho97H75Omz5y/29l+emazQjI9ZJjN9QYnhUig+BgGSX+Sak5RKfk7nH5b++VeujcjUF1jkPE7JVIlEMAI29DkqL/f8oP/u+GgwPPKCfhCM8AAvxWA0fDv0sI0sl4+adXq57+xEk4wVKVfAJDEmxEEOcUk0CCZ55UaF4TlhczLloZWKpNzEZT1q5R3YyMRLMm23Aq+O3swoSWrMIqWWTAnMzKa3DHZ5YQHJcVwKlRfAFVs1SgrpQeYtz+1NhOYM5MIKwrSws3psRjRhYG/HdSPFr1iWpkRNyiiltApxbIVtQ2np46raIFgLMCK7CNESAroA0wIm6QRuNDGsiwBoCehqAmsC+DXcQ9BbRP2lq3kq1z3wcqLtn55xI8xmZm012Ta1V/m4FocdTexNzzm0cLiG426Yt2hUrtmountJ66J24uiOS9RUNpVkretSupGbdHueno8PN11Kde2FPo4r176b9ePw7hdngz4O+vjT0D9537ygXfQavUE9hNEInaCP6BSNEUMJ+oa+ox/OT+eX89v5s0K3t5qcV+jWcv7+B5ioZ7o=</latexit>

• Decomposition of program and data must be determined at compile time, often by hard-coded heuristics in the compiler

• Indirect accesses make the output program even more obscure (inspect/executor)

• Compilers have to make decisions without the information only available at runtime

Issue 2: Composability

4

Auto-parallelized

Auto-parallelized

Hand-parallelized

Hand-parallelized

In practice, programs look like this:

Interface

Interface

Interface

We need interface for seamless integration, but auto-parallelized parts are opaque to the rest of the program

Data Distributions in HPF

• Annotation language to describe the primary partition of data

• E.g., tiling on the first dimension of A:

• Can serve as an interface for both configuration and composition

• Support for sharing data partitions is key to configurability and composability

• Limited because “data distributions were not themselves data objects”

5

REAL A(1000,10000) !HPF$ DISTRIBUTE A(BLOCK,*)

Ken Kennedy, Charles Koelbel, and Hans Zima. 2007. The Rise and Fall of High Performance Fortran: An Historical Object Lesson. In Proceedings of the Third ACM SIGPLAN Conference on History of Programming Languages. ACM, 7–1.

†<latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit>


Programming Models with First-Class Partitions

6

Use data partitions as programmable objects


Examples: Legion, StarPU, PaRSEC

// pA1 is a partition of A parallel for x in pA1: A = pA1[x]

A[0] A[1] A[2]

pA1[0] pA1[1]

pA1 is an abstraction over partitions of A constructed at runtime:

A[0] A[1] A[2]

pA1[0] pA2[2]pA1[1]

· · ·<latexit sha1_base64="Pr3PvsqCbrDf/uWV7h5idwi1BMA=">AAAIzXicnZVLj9MwEMfDc0t4Lhy5WFQrLagqDUKCy0rL4wBIiOWxC1JTIcdxu6aOHewJEEK4cuUK34KPw7dhkiZkm81KhUhNJ56f/56xk5kglsLCaPT72PETJ0+dXuudcc+eO3/h4qX1y3tWJ4bxXaalNm8CarkUiu+CAMnfxIbTKJD8dTB/UPhff+DGCq1eQRrzSURnSkwFo4BDez4LNdi3l/qj4ai8yGHDq4y+U107b9fXfvmhZknEFTBJrR17oxgmGTUgmOS56yeWx5TN6YyP0VQ04naSleHmZANHQjLVBn8KSDl6cEZGI2vTKEAyorBv275isMs3TmB6d5IJFSfAFVssNE0kAU2K3EkoDGcgUzQoMwJjJWyfGsoAd2hplXg2jSXuzFImGYj558VIYUkRGGpSTNroj3ZAE9AYEh3EFFBP4UjpGEYc6PKOKG0iKnk0yRK8567r+op/ZDqKqAozPwqCfOxN0MAMgyDre3neIlgDMCq7CNEQAroA2wB22gkcWMSyLgKgIaBrEagJ4J/gCCJYIsr/YBEPbswGianBl2yfW2HbM0tXNRunbuZ9rzSudyyChzzn0MDjGp50w7xB/axm/fzwJtWiGLF/yEvVTFZKsrRLKVOZbbrJZ7PvXW97g8CUvnHfm7RDro+q2Lz6qFy/fokxxswlhKX4aZKipmwVj4RYLUU4IIbjjWORSElEzfzmkMYxVyGxkEq+lVXUVEi5hWg+KKmtG76/cVAlkAlfVaZgax37PsG0/6qFXGExwy+U2v0irgA/IbWybgHXwhoEFp0jhYFTuapuwdayWDoircJGt9JjKV05zoLNB51yUuulMFcTnBmaVoJHJvzPSu2TQR1sFvBfWu3DoBalVlSrwwFqunKrVP45pOWdz928VYXjopkVTTKvCvE027n34lX7q0Ujlhx4Qz149nSnTYXCvtNCQUM9fPzySd5eU2B3LpSwBmULu6WDvf0g8/exhXE8OlkxCzt3scl77ZZ+2Ni7NfRGQ+/57f72/ard95yrzjVn0/GcO86288jZcXYd5rxzvjs/nJ+9Z72k96X3dYEeP1bNueIsXb1vfwDdTh5a</latexit><latexit sha1_base64="Pr3PvsqCbrDf/uWV7h5idwi1BMA=">AAAIzXicnZVLj9MwEMfDc0t4Lhy5WFQrLagqDUKCy0rL4wBIiOWxC1JTIcdxu6aOHewJEEK4cuUK34KPw7dhkiZkm81KhUhNJ56f/56xk5kglsLCaPT72PETJ0+dXuudcc+eO3/h4qX1y3tWJ4bxXaalNm8CarkUiu+CAMnfxIbTKJD8dTB/UPhff+DGCq1eQRrzSURnSkwFo4BDez4LNdi3l/qj4ai8yGHDq4y+U107b9fXfvmhZknEFTBJrR17oxgmGTUgmOS56yeWx5TN6YyP0VQ04naSleHmZANHQjLVBn8KSDl6cEZGI2vTKEAyorBv275isMs3TmB6d5IJFSfAFVssNE0kAU2K3EkoDGcgUzQoMwJjJWyfGsoAd2hplXg2jSXuzFImGYj558VIYUkRGGpSTNroj3ZAE9AYEh3EFFBP4UjpGEYc6PKOKG0iKnk0yRK8567r+op/ZDqKqAozPwqCfOxN0MAMgyDre3neIlgDMCq7CNEQAroA2wB22gkcWMSyLgKgIaBrEagJ4J/gCCJYIsr/YBEPbswGianBl2yfW2HbM0tXNRunbuZ9rzSudyyChzzn0MDjGp50w7xB/axm/fzwJtWiGLF/yEvVTFZKsrRLKVOZbbrJZ7PvXW97g8CUvnHfm7RDro+q2Lz6qFy/fokxxswlhKX4aZKipmwVj4RYLUU4IIbjjWORSElEzfzmkMYxVyGxkEq+lVXUVEi5hWg+KKmtG76/cVAlkAlfVaZgax37PsG0/6qFXGExwy+U2v0irgA/IbWybgHXwhoEFp0jhYFTuapuwdayWDoircJGt9JjKV05zoLNB51yUuulMFcTnBmaVoJHJvzPSu2TQR1sFvBfWu3DoBalVlSrwwFqunKrVP45pOWdz928VYXjopkVTTKvCvE027n34lX7q0Ujlhx4Qz149nSnTYXCvtNCQUM9fPzySd5eU2B3LpSwBmULu6WDvf0g8/exhXE8OlkxCzt3scl77ZZ+2Ni7NfRGQ+/57f72/ard95yrzjVn0/GcO86288jZcXYd5rxzvjs/nJ+9Z72k96X3dYEeP1bNueIsXb1vfwDdTh5a</latexit><latexit sha1_base64="Pr3PvsqCbrDf/uWV7h5idwi1BMA=">AAAIzXicnZVLj9MwEMfDc0t4Lhy5WFQrLagqDUKCy0rL4wBIiOWxC1JTIcdxu6aOHewJEEK4cuUK34KPw7dhkiZkm81KhUhNJ56f/56xk5kglsLCaPT72PETJ0+dXuudcc+eO3/h4qX1y3tWJ4bxXaalNm8CarkUiu+CAMnfxIbTKJD8dTB/UPhff+DGCq1eQRrzSURnSkwFo4BDez4LNdi3l/qj4ai8yGHDq4y+U107b9fXfvmhZknEFTBJrR17oxgmGTUgmOS56yeWx5TN6YyP0VQ04naSleHmZANHQjLVBn8KSDl6cEZGI2vTKEAyorBv275isMs3TmB6d5IJFSfAFVssNE0kAU2K3EkoDGcgUzQoMwJjJWyfGsoAd2hplXg2jSXuzFImGYj558VIYUkRGGpSTNroj3ZAE9AYEh3EFFBP4UjpGEYc6PKOKG0iKnk0yRK8567r+op/ZDqKqAozPwqCfOxN0MAMgyDre3neIlgDMCq7CNEQAroA2wB22gkcWMSyLgKgIaBrEagJ4J/gCCJYIsr/YBEPbswGianBl2yfW2HbM0tXNRunbuZ9rzSudyyChzzn0MDjGp50w7xB/axm/fzwJtWiGLF/yEvVTFZKsrRLKVOZbbrJZ7PvXW97g8CUvnHfm7RDro+q2Lz6qFy/fokxxswlhKX4aZKipmwVj4RYLUU4IIbjjWORSElEzfzmkMYxVyGxkEq+lVXUVEi5hWg+KKmtG76/cVAlkAlfVaZgax37PsG0/6qFXGExwy+U2v0irgA/IbWybgHXwhoEFp0jhYFTuapuwdayWDoircJGt9JjKV05zoLNB51yUuulMFcTnBmaVoJHJvzPSu2TQR1sFvBfWu3DoBalVlSrwwFqunKrVP45pOWdz928VYXjopkVTTKvCvE027n34lX7q0Ujlhx4Qz149nSnTYXCvtNCQUM9fPzySd5eU2B3LpSwBmULu6WDvf0g8/exhXE8OlkxCzt3scl77ZZ+2Ni7NfRGQ+/57f72/ard95yrzjVn0/GcO86288jZcXYd5rxzvjs/nJ+9Z72k96X3dYEeP1bNueIsXb1vfwDdTh5a</latexit><latexit sha1_base64="Pr3PvsqCbrDf/uWV7h5idwi1BMA=">AAAIzXicnZVLj9MwEMfDc0t4Lhy5WFQrLagqDUKCy0rL4wBIiOWxC1JTIcdxu6aOHewJEEK4cuUK34KPw7dhkiZkm81KhUhNJ56f/56xk5kglsLCaPT72PETJ0+dXuudcc+eO3/h4qX1y3tWJ4bxXaalNm8CarkUiu+CAMnfxIbTKJD8dTB/UPhff+DGCq1eQRrzSURnSkwFo4BDez4LNdi3l/qj4ai8yGHDq4y+U107b9fXfvmhZknEFTBJrR17oxgmGTUgmOS56yeWx5TN6YyP0VQ04naSleHmZANHQjLVBn8KSDl6cEZGI2vTKEAyorBv275isMs3TmB6d5IJFSfAFVssNE0kAU2K3EkoDGcgUzQoMwJjJWyfGsoAd2hplXg2jSXuzFImGYj558VIYUkRGGpSTNroj3ZAE9AYEh3EFFBP4UjpGEYc6PKOKG0iKnk0yRK8567r+op/ZDqKqAozPwqCfOxN0MAMgyDre3neIlgDMCq7CNEQAroA2wB22gkcWMSyLgKgIaBrEagJ4J/gCCJYIsr/YBEPbswGianBl2yfW2HbM0tXNRunbuZ9rzSudyyChzzn0MDjGp50w7xB/axm/fzwJtWiGLF/yEvVTFZKsrRLKVOZbbrJZ7PvXW97g8CUvnHfm7RDro+q2Lz6qFy/fokxxswlhKX4aZKipmwVj4RYLUU4IIbjjWORSElEzfzmkMYxVyGxkEq+lVXUVEi5hWg+KKmtG76/cVAlkAlfVaZgax37PsG0/6qFXGExwy+U2v0irgA/IbWybgHXwhoEFp0jhYFTuapuwdayWDoircJGt9JjKV05zoLNB51yUuulMFcTnBmaVoJHJvzPSu2TQR1sFvBfWu3DoBalVlSrwwFqunKrVP45pOWdz928VYXjopkVTTKvCvE027n34lX7q0Ujlhx4Qz149nSnTYXCvtNCQUM9fPzySd5eU2B3LpSwBmULu6WDvf0g8/exhXE8OlkxCzt3scl77ZZ+2Ni7NfRGQ+/57f72/ard95yrzjVn0/GcO86288jZcXYd5rxzvjs/nJ+9Z72k96X3dYEeP1bNueIsXb1vfwDdTh5a</latexit>

A[0] A[2] A[1]

pA1[0] pA1[1]

Data partitions can naturally serve as an interface between different parts

Programming Models with First-Class Partitions

7

Can provide synchronization and communication from multiple data partitions



// pA1 and pA2 are partitions of A parallel for x in pA1: A = pA1[x]

parallel for x in pB: A = pA2[x] B = pB[x]

Implicit communication and synchronization for every i and j such that pA1[i] ∩ pA2[j] ≠ ∅


‡<latexit sha1_base64="SF40vHoiirzA7OGN6QO0+4ISLr8=">AAAIz3icnZVRj9w0EMdzhdIlbWlLH3mxWJ10RavtBiHBy0mlxwNFQr3SXnvSZlVNHO+eWccO9oQjREG89pVX+BJ8nH4bJtmkuc3lpC2WNjvx/Pz3jB2Po1RJh7PZ271rH3x4/aMbo4/9m7duf3Ln7r1PXzqTWS5OuFHGnkbghJJanKBEJU5TKyCJlHgVrY8q/6tfhXXS6BeYp2KRwErLpeSA1HUaxjGsVsK+vjueTWd1Y5eNoDHGXtOOX9+78W8YG54lQiNX4Nw8mKW4KMCi5EqUfpg5kQJfw0rMydSQCLco6oBLtk89MVsaSz+NrO69OKKAxLk8iYhMAM9c31d1DvnmGS6/WRRSpxkKzTcTLTPF0LAqexZLKziqnAzgVlKsjJ+BBY60RluzpKtlqgy6rUwKlOvfNz2VpWRkweaUtDXnbgIZGgoJJikg6WnqqR3TRCBsr4g2NgElkkWR0bP0fT/U4pybJAEdF2ESReU8WJBBGUZRMQ7KskfwDuCghgjZERKHANcBbjkIXJjE8SECsSNwaBJsCRS/4RVEtEXU/9EmHlqYfZaCpY/sTDjp+iNrVzOahh6U46A2HgxMQpu8FtjB8xZeDMOiQ8OiZcPy8iK1ohRxeMkLeqUaJVXbtZRtzD7d5XMwDh70vVFka998HCz6IbdbVS1eu1V+2H7EFGPhM8ZzOpqsqiqH1StjzigZT5gV9BBUJnKWgF0/nEKaCh0zh7kSh0VDLaVSh4SWk5o6/CIM9y+qRCoTu8pUbKvjfsko7XdqsdBUzuiEgjur4oroCOmddSu4FTYoqehcKYwC1K66FdvKUulIjI473UaP57BznBVbTgbllDFbYe4muLKQN4JXJvzeSv2dIZ3YUGn7P1r9zQBHUjuqteEg2KHcGpX3Dml75Uu/7FXhtLrMqmuybArxsjj+9qcX/VNLRqoEio46evrjcZ+KpfvZSI0d9d2T5z+U/Tkl3c+VEtWgYmP3dOh2v8i8e+1hgrZONczGLn265IP+lX7ZePnlNJhNg2dfjR89bq77kfeZ97l34AXe194j73vv2DvxuKe8v7y/vX9Gz0bnoz9Gf27Qa3vNmPveVhu9+Q8HXx8f</latexit><latexit sha1_base64="SF40vHoiirzA7OGN6QO0+4ISLr8=">AAAIz3icnZVRj9w0EMdzhdIlbWlLH3mxWJ10RavtBiHBy0mlxwNFQr3SXnvSZlVNHO+eWccO9oQjREG89pVX+BJ8nH4bJtmkuc3lpC2WNjvx/Pz3jB2Po1RJh7PZ271rH3x4/aMbo4/9m7duf3Ln7r1PXzqTWS5OuFHGnkbghJJanKBEJU5TKyCJlHgVrY8q/6tfhXXS6BeYp2KRwErLpeSA1HUaxjGsVsK+vjueTWd1Y5eNoDHGXtOOX9+78W8YG54lQiNX4Nw8mKW4KMCi5EqUfpg5kQJfw0rMydSQCLco6oBLtk89MVsaSz+NrO69OKKAxLk8iYhMAM9c31d1DvnmGS6/WRRSpxkKzTcTLTPF0LAqexZLKziqnAzgVlKsjJ+BBY60RluzpKtlqgy6rUwKlOvfNz2VpWRkweaUtDXnbgIZGgoJJikg6WnqqR3TRCBsr4g2NgElkkWR0bP0fT/U4pybJAEdF2ESReU8WJBBGUZRMQ7KskfwDuCghgjZERKHANcBbjkIXJjE8SECsSNwaBJsCRS/4RVEtEXU/9EmHlqYfZaCpY/sTDjp+iNrVzOahh6U46A2HgxMQpu8FtjB8xZeDMOiQ8OiZcPy8iK1ohRxeMkLeqUaJVXbtZRtzD7d5XMwDh70vVFka998HCz6IbdbVS1eu1V+2H7EFGPhM8ZzOpqsqiqH1StjzigZT5gV9BBUJnKWgF0/nEKaCh0zh7kSh0VDLaVSh4SWk5o6/CIM9y+qRCoTu8pUbKvjfsko7XdqsdBUzuiEgjur4oroCOmddSu4FTYoqehcKYwC1K66FdvKUulIjI473UaP57BznBVbTgbllDFbYe4muLKQN4JXJvzeSv2dIZ3YUGn7P1r9zQBHUjuqteEg2KHcGpX3Dml75Uu/7FXhtLrMqmuybArxsjj+9qcX/VNLRqoEio46evrjcZ+KpfvZSI0d9d2T5z+U/Tkl3c+VEtWgYmP3dOh2v8i8e+1hgrZONczGLn265IP+lX7ZePnlNJhNg2dfjR89bq77kfeZ97l34AXe194j73vv2DvxuKe8v7y/vX9Gz0bnoz9Gf27Qa3vNmPveVhu9+Q8HXx8f</latexit><latexit sha1_base64="SF40vHoiirzA7OGN6QO0+4ISLr8=">AAAIz3icnZVRj9w0EMdzhdIlbWlLH3mxWJ10RavtBiHBy0mlxwNFQr3SXnvSZlVNHO+eWccO9oQjREG89pVX+BJ8nH4bJtmkuc3lpC2WNjvx/Pz3jB2Po1RJh7PZ271rH3x4/aMbo4/9m7duf3Ln7r1PXzqTWS5OuFHGnkbghJJanKBEJU5TKyCJlHgVrY8q/6tfhXXS6BeYp2KRwErLpeSA1HUaxjGsVsK+vjueTWd1Y5eNoDHGXtOOX9+78W8YG54lQiNX4Nw8mKW4KMCi5EqUfpg5kQJfw0rMydSQCLco6oBLtk89MVsaSz+NrO69OKKAxLk8iYhMAM9c31d1DvnmGS6/WRRSpxkKzTcTLTPF0LAqexZLKziqnAzgVlKsjJ+BBY60RluzpKtlqgy6rUwKlOvfNz2VpWRkweaUtDXnbgIZGgoJJikg6WnqqR3TRCBsr4g2NgElkkWR0bP0fT/U4pybJAEdF2ESReU8WJBBGUZRMQ7KskfwDuCghgjZERKHANcBbjkIXJjE8SECsSNwaBJsCRS/4RVEtEXU/9EmHlqYfZaCpY/sTDjp+iNrVzOahh6U46A2HgxMQpu8FtjB8xZeDMOiQ8OiZcPy8iK1ohRxeMkLeqUaJVXbtZRtzD7d5XMwDh70vVFka998HCz6IbdbVS1eu1V+2H7EFGPhM8ZzOpqsqiqH1StjzigZT5gV9BBUJnKWgF0/nEKaCh0zh7kSh0VDLaVSh4SWk5o6/CIM9y+qRCoTu8pUbKvjfsko7XdqsdBUzuiEgjur4oroCOmddSu4FTYoqehcKYwC1K66FdvKUulIjI473UaP57BznBVbTgbllDFbYe4muLKQN4JXJvzeSv2dIZ3YUGn7P1r9zQBHUjuqteEg2KHcGpX3Dml75Uu/7FXhtLrMqmuybArxsjj+9qcX/VNLRqoEio46evrjcZ+KpfvZSI0d9d2T5z+U/Tkl3c+VEtWgYmP3dOh2v8i8e+1hgrZONczGLn265IP+lX7ZePnlNJhNg2dfjR89bq77kfeZ97l34AXe194j73vv2DvxuKe8v7y/vX9Gz0bnoz9Gf27Qa3vNmPveVhu9+Q8HXx8f</latexit><latexit sha1_base64="SF40vHoiirzA7OGN6QO0+4ISLr8=">AAAIz3icnZVRj9w0EMdzhdIlbWlLH3mxWJ10RavtBiHBy0mlxwNFQr3SXnvSZlVNHO+eWccO9oQjREG89pVX+BJ8nH4bJtmkuc3lpC2WNjvx/Pz3jB2Po1RJh7PZ271rH3x4/aMbo4/9m7duf3Ln7r1PXzqTWS5OuFHGnkbghJJanKBEJU5TKyCJlHgVrY8q/6tfhXXS6BeYp2KRwErLpeSA1HUaxjGsVsK+vjueTWd1Y5eNoDHGXtOOX9+78W8YG54lQiNX4Nw8mKW4KMCi5EqUfpg5kQJfw0rMydSQCLco6oBLtk89MVsaSz+NrO69OKKAxLk8iYhMAM9c31d1DvnmGS6/WRRSpxkKzTcTLTPF0LAqexZLKziqnAzgVlKsjJ+BBY60RluzpKtlqgy6rUwKlOvfNz2VpWRkweaUtDXnbgIZGgoJJikg6WnqqR3TRCBsr4g2NgElkkWR0bP0fT/U4pybJAEdF2ESReU8WJBBGUZRMQ7KskfwDuCghgjZERKHANcBbjkIXJjE8SECsSNwaBJsCRS/4RVEtEXU/9EmHlqYfZaCpY/sTDjp+iNrVzOahh6U46A2HgxMQpu8FtjB8xZeDMOiQ8OiZcPy8iK1ohRxeMkLeqUaJVXbtZRtzD7d5XMwDh70vVFka998HCz6IbdbVS1eu1V+2H7EFGPhM8ZzOpqsqiqH1StjzigZT5gV9BBUJnKWgF0/nEKaCh0zh7kSh0VDLaVSh4SWk5o6/CIM9y+qRCoTu8pUbKvjfsko7XdqsdBUzuiEgjur4oroCOmddSu4FTYoqehcKYwC1K66FdvKUulIjI473UaP57BznBVbTgbllDFbYe4muLKQN4JXJvzeSv2dIZ3YUGn7P1r9zQBHUjuqteEg2KHcGpX3Dml75Uu/7FXhtLrMqmuybArxsjj+9qcX/VNLRqoEio46evrjcZ+KpfvZSI0d9d2T5z+U/Tkl3c+VEtWgYmP3dOh2v8i8e+1hgrZONczGLn265IP+lX7ZePnlNJhNg2dfjR89bq77kfeZ97l34AXe194j73vv2DvxuKe8v7y/vX9Gz0bnoz9Gf27Qa3vNmPveVhu9+Q8HXx8f</latexit>

→ Can be handled automatically by compiler or runtime system†

<latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit>

‡<latexit sha1_base64="SF40vHoiirzA7OGN6QO0+4ISLr8=">AAAIz3icnZVRj9w0EMdzhdIlbWlLH3mxWJ10RavtBiHBy0mlxwNFQr3SXnvSZlVNHO+eWccO9oQjREG89pVX+BJ8nH4bJtmkuc3lpC2WNjvx/Pz3jB2Po1RJh7PZ271rH3x4/aMbo4/9m7duf3Ln7r1PXzqTWS5OuFHGnkbghJJanKBEJU5TKyCJlHgVrY8q/6tfhXXS6BeYp2KRwErLpeSA1HUaxjGsVsK+vjueTWd1Y5eNoDHGXtOOX9+78W8YG54lQiNX4Nw8mKW4KMCi5EqUfpg5kQJfw0rMydSQCLco6oBLtk89MVsaSz+NrO69OKKAxLk8iYhMAM9c31d1DvnmGS6/WRRSpxkKzTcTLTPF0LAqexZLKziqnAzgVlKsjJ+BBY60RluzpKtlqgy6rUwKlOvfNz2VpWRkweaUtDXnbgIZGgoJJikg6WnqqR3TRCBsr4g2NgElkkWR0bP0fT/U4pybJAEdF2ESReU8WJBBGUZRMQ7KskfwDuCghgjZERKHANcBbjkIXJjE8SECsSNwaBJsCRS/4RVEtEXU/9EmHlqYfZaCpY/sTDjp+iNrVzOahh6U46A2HgxMQpu8FtjB8xZeDMOiQ8OiZcPy8iK1ohRxeMkLeqUaJVXbtZRtzD7d5XMwDh70vVFka998HCz6IbdbVS1eu1V+2H7EFGPhM8ZzOpqsqiqH1StjzigZT5gV9BBUJnKWgF0/nEKaCh0zh7kSh0VDLaVSh4SWk5o6/CIM9y+qRCoTu8pUbKvjfsko7XdqsdBUzuiEgjur4oroCOmddSu4FTYoqehcKYwC1K66FdvKUulIjI473UaP57BznBVbTgbllDFbYe4muLKQN4JXJvzeSv2dIZ3YUGn7P1r9zQBHUjuqteEg2KHcGpX3Dml75Uu/7FXhtLrMqmuybArxsjj+9qcX/VNLRqoEio46evrjcZ+KpfvZSI0d9d2T5z+U/Tkl3c+VEtWgYmP3dOh2v8i8e+1hgrZONczGLn265IP+lX7ZePnlNJhNg2dfjR89bq77kfeZ97l34AXe194j73vv2DvxuKe8v7y/vX9Gz0bnoz9Gf27Qa3vNmPveVhu9+Q8HXx8f</latexit><latexit sha1_base64="SF40vHoiirzA7OGN6QO0+4ISLr8=">AAAIz3icnZVRj9w0EMdzhdIlbWlLH3mxWJ10RavtBiHBy0mlxwNFQr3SXnvSZlVNHO+eWccO9oQjREG89pVX+BJ8nH4bJtmkuc3lpC2WNjvx/Pz3jB2Po1RJh7PZ271rH3x4/aMbo4/9m7duf3Ln7r1PXzqTWS5OuFHGnkbghJJanKBEJU5TKyCJlHgVrY8q/6tfhXXS6BeYp2KRwErLpeSA1HUaxjGsVsK+vjueTWd1Y5eNoDHGXtOOX9+78W8YG54lQiNX4Nw8mKW4KMCi5EqUfpg5kQJfw0rMydSQCLco6oBLtk89MVsaSz+NrO69OKKAxLk8iYhMAM9c31d1DvnmGS6/WRRSpxkKzTcTLTPF0LAqexZLKziqnAzgVlKsjJ+BBY60RluzpKtlqgy6rUwKlOvfNz2VpWRkweaUtDXnbgIZGgoJJikg6WnqqR3TRCBsr4g2NgElkkWR0bP0fT/U4pybJAEdF2ESReU8WJBBGUZRMQ7KskfwDuCghgjZERKHANcBbjkIXJjE8SECsSNwaBJsCRS/4RVEtEXU/9EmHlqYfZaCpY/sTDjp+iNrVzOahh6U46A2HgxMQpu8FtjB8xZeDMOiQ8OiZcPy8iK1ohRxeMkLeqUaJVXbtZRtzD7d5XMwDh70vVFka998HCz6IbdbVS1eu1V+2H7EFGPhM8ZzOpqsqiqH1StjzigZT5gV9BBUJnKWgF0/nEKaCh0zh7kSh0VDLaVSh4SWk5o6/CIM9y+qRCoTu8pUbKvjfsko7XdqsdBUzuiEgjur4oroCOmddSu4FTYoqehcKYwC1K66FdvKUulIjI473UaP57BznBVbTgbllDFbYe4muLKQN4JXJvzeSv2dIZ3YUGn7P1r9zQBHUjuqteEg2KHcGpX3Dml75Uu/7FXhtLrMqmuybArxsjj+9qcX/VNLRqoEio46evrjcZ+KpfvZSI0d9d2T5z+U/Tkl3c+VEtWgYmP3dOh2v8i8e+1hgrZONczGLn265IP+lX7ZePnlNJhNg2dfjR89bq77kfeZ97l34AXe194j73vv2DvxuKe8v7y/vX9Gz0bnoz9Gf27Qa3vNmPveVhu9+Q8HXx8f</latexit><latexit sha1_base64="SF40vHoiirzA7OGN6QO0+4ISLr8=">AAAIz3icnZVRj9w0EMdzhdIlbWlLH3mxWJ10RavtBiHBy0mlxwNFQr3SXnvSZlVNHO+eWccO9oQjREG89pVX+BJ8nH4bJtmkuc3lpC2WNjvx/Pz3jB2Po1RJh7PZ271rH3x4/aMbo4/9m7duf3Ln7r1PXzqTWS5OuFHGnkbghJJanKBEJU5TKyCJlHgVrY8q/6tfhXXS6BeYp2KRwErLpeSA1HUaxjGsVsK+vjueTWd1Y5eNoDHGXtOOX9+78W8YG54lQiNX4Nw8mKW4KMCi5EqUfpg5kQJfw0rMydSQCLco6oBLtk89MVsaSz+NrO69OKKAxLk8iYhMAM9c31d1DvnmGS6/WRRSpxkKzTcTLTPF0LAqexZLKziqnAzgVlKsjJ+BBY60RluzpKtlqgy6rUwKlOvfNz2VpWRkweaUtDXnbgIZGgoJJikg6WnqqR3TRCBsr4g2NgElkkWR0bP0fT/U4pybJAEdF2ESReU8WJBBGUZRMQ7KskfwDuCghgjZERKHANcBbjkIXJjE8SECsSNwaBJsCRS/4RVEtEXU/9EmHlqYfZaCpY/sTDjp+iNrVzOahh6U46A2HgxMQpu8FtjB8xZeDMOiQ8OiZcPy8iK1ohRxeMkLeqUaJVXbtZRtzD7d5XMwDh70vVFka998HCz6IbdbVS1eu1V+2H7EFGPhM8ZzOpqsqiqH1StjzigZT5gV9BBUJnKWgF0/nEKaCh0zh7kSh0VDLaVSh4SWk5o6/CIM9y+qRCoTu8pUbKvjfsko7XdqsdBUzuiEgjur4oroCOmddSu4FTYoqehcKYwC1K66FdvKUulIjI473UaP57BznBVbTgbllDFbYe4muLKQN4JXJvzeSv2dIZ3YUGn7P1r9zQBHUjuqteEg2KHcGpX3Dml75Uu/7FXhtLrMqmuybArxsjj+9qcX/VNLRqoEio46evrjcZ+KpfvZSI0d9d2T5z+U/Tkl3c+VEtWgYmP3dOh2v8i8e+1hgrZONczGLn265IP+lX7ZePnlNJhNg2dfjR89bq77kfeZ97l34AXe194j73vv2DvxuKe8v7y/vX9Gz0bnoz9Gf27Qa3vNmPveVhu9+Q8HXx8f</latexit><latexit sha1_base64="SF40vHoiirzA7OGN6QO0+4ISLr8=">AAAIz3icnZVRj9w0EMdzhdIlbWlLH3mxWJ10RavtBiHBy0mlxwNFQr3SXnvSZlVNHO+eWccO9oQjREG89pVX+BJ8nH4bJtmkuc3lpC2WNjvx/Pz3jB2Po1RJh7PZ271rH3x4/aMbo4/9m7duf3Ln7r1PXzqTWS5OuFHGnkbghJJanKBEJU5TKyCJlHgVrY8q/6tfhXXS6BeYp2KRwErLpeSA1HUaxjGsVsK+vjueTWd1Y5eNoDHGXtOOX9+78W8YG54lQiNX4Nw8mKW4KMCi5EqUfpg5kQJfw0rMydSQCLco6oBLtk89MVsaSz+NrO69OKKAxLk8iYhMAM9c31d1DvnmGS6/WRRSpxkKzTcTLTPF0LAqexZLKziqnAzgVlKsjJ+BBY60RluzpKtlqgy6rUwKlOvfNz2VpWRkweaUtDXnbgIZGgoJJikg6WnqqR3TRCBsr4g2NgElkkWR0bP0fT/U4pybJAEdF2ESReU8WJBBGUZRMQ7KskfwDuCghgjZERKHANcBbjkIXJjE8SECsSNwaBJsCRS/4RVEtEXU/9EmHlqYfZaCpY/sTDjp+iNrVzOahh6U46A2HgxMQpu8FtjB8xZeDMOiQ8OiZcPy8iK1ohRxeMkLeqUaJVXbtZRtzD7d5XMwDh70vVFka998HCz6IbdbVS1eu1V+2H7EFGPhM8ZzOpqsqiqH1StjzigZT5gV9BBUJnKWgF0/nEKaCh0zh7kSh0VDLaVSh4SWk5o6/CIM9y+qRCoTu8pUbKvjfsko7XdqsdBUzuiEgjur4oroCOmddSu4FTYoqehcKYwC1K66FdvKUulIjI473UaP57BznBVbTgbllDFbYe4muLKQN4JXJvzeSv2dIZ3YUGn7P1r9zQBHUjuqteEg2KHcGpX3Dml75Uu/7FXhtLrMqmuybArxsjj+9qcX/VNLRqoEio46evrjcZ+KpfvZSI0d9d2T5z+U/Tkl3c+VEtWgYmP3dOh2v8i8e+1hgrZONczGLn265IP+lX7ZePnlNJhNg2dfjR89bq77kfeZ97l34AXe194j73vv2DvxuKe8v7y/vX9Gz0bnoz9Gf27Qa3vNmPveVhu9+Q8HXx8f</latexit>

E. Slaughter, W. Lee, S. Treichler, W. Zhang, M. Bauer, G. Shipman, P. McCormick, and A. Aiken, Control replication: Compiling implicit parallelism to efficient SPMD with logical regions, SC17.Michael Bauer, Sean Treichler, Elliott Slaughter, Alex Aiken, Legion: expressing locality and independence with logical regions. SC12.

Auto-Parallelization as Constraint Solving

8

Auto-parallelization amounts to finding legal partitions by solving partitioning constraints



parallel for x in pA1: A = pA1[x]

parallel for x in pB: A = pA2[x] B = pB[x]

Find partitions pA1, pA2, and pB that satisfy these constraints:

1

1 pA1 covers A

2

2 pB covers B

3

3 For any index i in pB[x], pA2[x] includes i+1

Constraint-Based Automatic Data Partitioning

9

pA = some_pA parallel for x in pA: ...

Discharges constraints with interface constraints

... // Hand-parallelized code ... assert(π(some_pA))

Parallelizes sequential program using data partitions for i in A:

A[i] = f(i)

parallel for x in pA: A = pA[x] for i in A: A[i] = f(i)

Infers partitioning constraints

require(π(pA)) Or, synthesizes partitioning code using constraint solver

pA = partition(A,...) parallel for x in pA: ...

DPL as Constraint Language

• DPL(Dependent Partitioning Language): domain specific language for data partitioning

• DPL programs construct data partitions using high-level operators

• DPL operators have well-defined semantics and scalable implementation

• DPL can be used to describe both partitioning constraints and their solutions

10Sean Treichler, Michael Bauer, Rahul Sharma, Elliott Slaughter, and Alex Aiken, Dependent Partitioning, OOPSLA16.†<latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit>


Constructing Partitions with DPL

11

parallel for x in pB: A = pA2[x] B = pB[x] for i in B: B[i] = g(A[i+1])

pBB[0] B[1] B[2]

pB[0] pB[1]

pA2

A function that maps i to i+1

A[1] A[2]

pA2[0]

A[3]

pA2[1]

Example:

8j, 8i 2 pB[j], i+1 2 pA2[j]<latexit sha1_base64="aCYWQvFK/vU26VccTh0Saj0dQa0=">AAAJl3icnZZtb9xEEMd9LdDe8ZTCK8SbFadIKVjHOaQPvIhoUwQFCZGWpq16tqL1eu+yufWu2R1TjOUPxRfhPd+Gsc+uY8eRrljKZW73N/+Z3fHOXphIYWE+/3d07fo777534+Z48v4HH3708c6tT55bnRrGT5iW2rwMqeVSKH4CAiR/mRhO41DyF+H6UTn/4g9urNDqGWQJD2K6UmIpGAUcOt35x4/4En0rpTzOtKFqxYv86Y9HRb5/547r3bvreveLSY/DGDKUaUMeIOR943rfekguUURK4gNAfl64pDMgCuILhTb/E/pBKyA5WpwHReH6Nf+VN+DRht/4PNyvnCanO9P5bF495LLh1cbUqZ/j01s3/vYjzdKYK2CSWrvw5gkEOTUgmOS4mtTyhLI1XfEFmorG3AZ5lUdBdnEkIrg8/FNAqtGLHjmNrc3iEMmYwpntz5WDQ3OLFJb3g1yoJAWu2CbQMpUENCmrSCJhOAOZoUGZEZgrYWfUUAZY606UZLVMpAbbWUkOYv3XZqS0pAgNNRku2ujX1qUpaEyJugkF1FM4Uk3MYg60uyNKm5hKHgd5ip/FZDLxFX/NdBxTFeV+HIbFwgvQwBWGYT71sEZdgrUAo3KIEC0hYAiwLWCXg8CFIJYNEQAtAUNBoCHKt/AKIuwQ1f9wkw9uzC5JqMGX7IxbYfue1VTtja57xdSrjNsDQbDIaw4tvGjgYBjmLernDesXlzepEcWM/UuzeEBlrSQru5Iytdmn2/XsTb3b/dkwNNXcYuoF/ZSbUpWb15Rq4jcvMeaYTwhhGR5NUnbHw/IrIVZLEbnEcPzg2O4yElOz/npGk4SriFjIJD/Ma2oppDxEFDtTSR1+6fu7F1XKvrKtTNWDah37e4rLfqMWcYVtGU8otWdlXiEeIbW1bgk3whoENp0rhQF74ba6JdvIYuuItYpa3VqPZXTrPEu2cAflpNadNLcTXBma1YJXLvitlfqVQZ1IY2v7P1r9YlCLUluqNekANUNrq1XeOqXuzheToteFk/IyK6/7om7Ey/z44dNn/VOLRiI58JZ69Osvx30qEvZcCwUt9f1Pv/1c9GMK/J2xuZ3DfGP3dPBXykXmzdcexrF0smY2dnXJe/0r/bLxfH/mzWfek4Ppg6P6ur/pfO584ew5nnPPeeA8do6dE4eNDkavRmwUjT8bfzf+Yfx4g14b1T6fOp1n/OQ/yologw==</latexit><latexit sha1_base64="aCYWQvFK/vU26VccTh0Saj0dQa0=">AAAJl3icnZZtb9xEEMd9LdDe8ZTCK8SbFadIKVjHOaQPvIhoUwQFCZGWpq16tqL1eu+yufWu2R1TjOUPxRfhPd+Gsc+uY8eRrljKZW73N/+Z3fHOXphIYWE+/3d07fo777534+Z48v4HH3708c6tT55bnRrGT5iW2rwMqeVSKH4CAiR/mRhO41DyF+H6UTn/4g9urNDqGWQJD2K6UmIpGAUcOt35x4/4En0rpTzOtKFqxYv86Y9HRb5/547r3bvreveLSY/DGDKUaUMeIOR943rfekguUURK4gNAfl64pDMgCuILhTb/E/pBKyA5WpwHReH6Nf+VN+DRht/4PNyvnCanO9P5bF495LLh1cbUqZ/j01s3/vYjzdKYK2CSWrvw5gkEOTUgmOS4mtTyhLI1XfEFmorG3AZ5lUdBdnEkIrg8/FNAqtGLHjmNrc3iEMmYwpntz5WDQ3OLFJb3g1yoJAWu2CbQMpUENCmrSCJhOAOZoUGZEZgrYWfUUAZY606UZLVMpAbbWUkOYv3XZqS0pAgNNRku2ujX1qUpaEyJugkF1FM4Uk3MYg60uyNKm5hKHgd5ip/FZDLxFX/NdBxTFeV+HIbFwgvQwBWGYT71sEZdgrUAo3KIEC0hYAiwLWCXg8CFIJYNEQAtAUNBoCHKt/AKIuwQ1f9wkw9uzC5JqMGX7IxbYfue1VTtja57xdSrjNsDQbDIaw4tvGjgYBjmLernDesXlzepEcWM/UuzeEBlrSQru5Iytdmn2/XsTb3b/dkwNNXcYuoF/ZSbUpWb15Rq4jcvMeaYTwhhGR5NUnbHw/IrIVZLEbnEcPzg2O4yElOz/npGk4SriFjIJD/Ma2oppDxEFDtTSR1+6fu7F1XKvrKtTNWDah37e4rLfqMWcYVtGU8otWdlXiEeIbW1bgk3whoENp0rhQF74ba6JdvIYuuItYpa3VqPZXTrPEu2cAflpNadNLcTXBma1YJXLvitlfqVQZ1IY2v7P1r9YlCLUluqNekANUNrq1XeOqXuzheToteFk/IyK6/7om7Ey/z44dNn/VOLRiI58JZ69Osvx30qEvZcCwUt9f1Pv/1c9GMK/J2xuZ3DfGP3dPBXykXmzdcexrF0smY2dnXJe/0r/bLxfH/mzWfek4Ppg6P6ur/pfO584ew5nnPPeeA8do6dE4eNDkavRmwUjT8bfzf+Yfx4g14b1T6fOp1n/OQ/yologw==</latexit><latexit sha1_base64="aCYWQvFK/vU26VccTh0Saj0dQa0=">AAAJl3icnZZtb9xEEMd9LdDe8ZTCK8SbFadIKVjHOaQPvIhoUwQFCZGWpq16tqL1eu+yufWu2R1TjOUPxRfhPd+Gsc+uY8eRrljKZW73N/+Z3fHOXphIYWE+/3d07fo777534+Z48v4HH3708c6tT55bnRrGT5iW2rwMqeVSKH4CAiR/mRhO41DyF+H6UTn/4g9urNDqGWQJD2K6UmIpGAUcOt35x4/4En0rpTzOtKFqxYv86Y9HRb5/547r3bvreveLSY/DGDKUaUMeIOR943rfekguUURK4gNAfl64pDMgCuILhTb/E/pBKyA5WpwHReH6Nf+VN+DRht/4PNyvnCanO9P5bF495LLh1cbUqZ/j01s3/vYjzdKYK2CSWrvw5gkEOTUgmOS4mtTyhLI1XfEFmorG3AZ5lUdBdnEkIrg8/FNAqtGLHjmNrc3iEMmYwpntz5WDQ3OLFJb3g1yoJAWu2CbQMpUENCmrSCJhOAOZoUGZEZgrYWfUUAZY606UZLVMpAbbWUkOYv3XZqS0pAgNNRku2ujX1qUpaEyJugkF1FM4Uk3MYg60uyNKm5hKHgd5ip/FZDLxFX/NdBxTFeV+HIbFwgvQwBWGYT71sEZdgrUAo3KIEC0hYAiwLWCXg8CFIJYNEQAtAUNBoCHKt/AKIuwQ1f9wkw9uzC5JqMGX7IxbYfue1VTtja57xdSrjNsDQbDIaw4tvGjgYBjmLernDesXlzepEcWM/UuzeEBlrSQru5Iytdmn2/XsTb3b/dkwNNXcYuoF/ZSbUpWb15Rq4jcvMeaYTwhhGR5NUnbHw/IrIVZLEbnEcPzg2O4yElOz/npGk4SriFjIJD/Ma2oppDxEFDtTSR1+6fu7F1XKvrKtTNWDah37e4rLfqMWcYVtGU8otWdlXiEeIbW1bgk3whoENp0rhQF74ba6JdvIYuuItYpa3VqPZXTrPEu2cAflpNadNLcTXBma1YJXLvitlfqVQZ1IY2v7P1r9YlCLUluqNekANUNrq1XeOqXuzheToteFk/IyK6/7om7Ey/z44dNn/VOLRiI58JZ69Osvx30qEvZcCwUt9f1Pv/1c9GMK/J2xuZ3DfGP3dPBXykXmzdcexrF0smY2dnXJe/0r/bLxfH/mzWfek4Ppg6P6ur/pfO584ew5nnPPeeA8do6dE4eNDkavRmwUjT8bfzf+Yfx4g14b1T6fOp1n/OQ/yologw==</latexit><latexit sha1_base64="aCYWQvFK/vU26VccTh0Saj0dQa0=">AAAJl3icnZZtb9xEEMd9LdDe8ZTCK8SbFadIKVjHOaQPvIhoUwQFCZGWpq16tqL1eu+yufWu2R1TjOUPxRfhPd+Gsc+uY8eRrljKZW73N/+Z3fHOXphIYWE+/3d07fo777534+Z48v4HH3708c6tT55bnRrGT5iW2rwMqeVSKH4CAiR/mRhO41DyF+H6UTn/4g9urNDqGWQJD2K6UmIpGAUcOt35x4/4En0rpTzOtKFqxYv86Y9HRb5/547r3bvreveLSY/DGDKUaUMeIOR943rfekguUURK4gNAfl64pDMgCuILhTb/E/pBKyA5WpwHReH6Nf+VN+DRht/4PNyvnCanO9P5bF495LLh1cbUqZ/j01s3/vYjzdKYK2CSWrvw5gkEOTUgmOS4mtTyhLI1XfEFmorG3AZ5lUdBdnEkIrg8/FNAqtGLHjmNrc3iEMmYwpntz5WDQ3OLFJb3g1yoJAWu2CbQMpUENCmrSCJhOAOZoUGZEZgrYWfUUAZY606UZLVMpAbbWUkOYv3XZqS0pAgNNRku2ujX1qUpaEyJugkF1FM4Uk3MYg60uyNKm5hKHgd5ip/FZDLxFX/NdBxTFeV+HIbFwgvQwBWGYT71sEZdgrUAo3KIEC0hYAiwLWCXg8CFIJYNEQAtAUNBoCHKt/AKIuwQ1f9wkw9uzC5JqMGX7IxbYfue1VTtja57xdSrjNsDQbDIaw4tvGjgYBjmLernDesXlzepEcWM/UuzeEBlrSQru5Iytdmn2/XsTb3b/dkwNNXcYuoF/ZSbUpWb15Rq4jcvMeaYTwhhGR5NUnbHw/IrIVZLEbnEcPzg2O4yElOz/npGk4SriFjIJD/Ma2oppDxEFDtTSR1+6fu7F1XKvrKtTNWDah37e4rLfqMWcYVtGU8otWdlXiEeIbW1bgk3whoENp0rhQF74ba6JdvIYuuItYpa3VqPZXTrPEu2cAflpNadNLcTXBma1YJXLvitlfqVQZ1IY2v7P1r9YlCLUluqNekANUNrq1XeOqXuzheToteFk/IyK6/7om7Ey/z44dNn/VOLRiI58JZ69Osvx30qEvZcCwUt9f1Pv/1c9GMK/J2xuZ3DfGP3dPBXykXmzdcexrF0smY2dnXJe/0r/bLxfH/mzWfek4Ppg6P6ur/pfO584ew5nnPPeeA8do6dE4eNDkavRmwUjT8bfzf+Yfx4g14b1T6fOp1n/OQ/yologw==</latexit>

3

3

Partition of the range of λi.i+1

pA2 = image(pB, λi.i+1)DPL program:

Partition of the domain of λi.i+1

Collecting image of λi.i+1

Characterizing Legal Partitions with DPL

12

parallel for x in pB: A = pA2[x] B = pB[x] for i in B: B[i] = g(A[i+1])

pBB[0] B[1] B[2]

pB[0] pB[1]

pA2A[1] A[2]

pA2[0]

A[3]

pA2[1]

8j, 8i 2 pB[j], i+1 2 pA2[j]<latexit sha1_base64="aCYWQvFK/vU26VccTh0Saj0dQa0=">AAAJl3icnZZtb9xEEMd9LdDe8ZTCK8SbFadIKVjHOaQPvIhoUwQFCZGWpq16tqL1eu+yufWu2R1TjOUPxRfhPd+Gsc+uY8eRrljKZW73N/+Z3fHOXphIYWE+/3d07fo777534+Z48v4HH3708c6tT55bnRrGT5iW2rwMqeVSKH4CAiR/mRhO41DyF+H6UTn/4g9urNDqGWQJD2K6UmIpGAUcOt35x4/4En0rpTzOtKFqxYv86Y9HRb5/547r3bvreveLSY/DGDKUaUMeIOR943rfekguUURK4gNAfl64pDMgCuILhTb/E/pBKyA5WpwHReH6Nf+VN+DRht/4PNyvnCanO9P5bF495LLh1cbUqZ/j01s3/vYjzdKYK2CSWrvw5gkEOTUgmOS4mtTyhLI1XfEFmorG3AZ5lUdBdnEkIrg8/FNAqtGLHjmNrc3iEMmYwpntz5WDQ3OLFJb3g1yoJAWu2CbQMpUENCmrSCJhOAOZoUGZEZgrYWfUUAZY606UZLVMpAbbWUkOYv3XZqS0pAgNNRku2ujX1qUpaEyJugkF1FM4Uk3MYg60uyNKm5hKHgd5ip/FZDLxFX/NdBxTFeV+HIbFwgvQwBWGYT71sEZdgrUAo3KIEC0hYAiwLWCXg8CFIJYNEQAtAUNBoCHKt/AKIuwQ1f9wkw9uzC5JqMGX7IxbYfue1VTtja57xdSrjNsDQbDIaw4tvGjgYBjmLernDesXlzepEcWM/UuzeEBlrSQru5Iytdmn2/XsTb3b/dkwNNXcYuoF/ZSbUpWb15Rq4jcvMeaYTwhhGR5NUnbHw/IrIVZLEbnEcPzg2O4yElOz/npGk4SriFjIJD/Ma2oppDxEFDtTSR1+6fu7F1XKvrKtTNWDah37e4rLfqMWcYVtGU8otWdlXiEeIbW1bgk3whoENp0rhQF74ba6JdvIYuuItYpa3VqPZXTrPEu2cAflpNadNLcTXBma1YJXLvitlfqVQZ1IY2v7P1r9YlCLUluqNekANUNrq1XeOqXuzheToteFk/IyK6/7om7Ey/z44dNn/VOLRiI58JZ69Osvx30qEvZcCwUt9f1Pv/1c9GMK/J2xuZ3DfGP3dPBXykXmzdcexrF0smY2dnXJe/0r/bLxfH/mzWfek4Ppg6P6ur/pfO584ew5nnPPeeA8do6dE4eNDkavRmwUjT8bfzf+Yfx4g14b1T6fOp1n/OQ/yologw==</latexit><latexit sha1_base64="aCYWQvFK/vU26VccTh0Saj0dQa0=">AAAJl3icnZZtb9xEEMd9LdDe8ZTCK8SbFadIKVjHOaQPvIhoUwQFCZGWpq16tqL1eu+yufWu2R1TjOUPxRfhPd+Gsc+uY8eRrljKZW73N/+Z3fHOXphIYWE+/3d07fo777534+Z48v4HH3708c6tT55bnRrGT5iW2rwMqeVSKH4CAiR/mRhO41DyF+H6UTn/4g9urNDqGWQJD2K6UmIpGAUcOt35x4/4En0rpTzOtKFqxYv86Y9HRb5/547r3bvreveLSY/DGDKUaUMeIOR943rfekguUURK4gNAfl64pDMgCuILhTb/E/pBKyA5WpwHReH6Nf+VN+DRht/4PNyvnCanO9P5bF495LLh1cbUqZ/j01s3/vYjzdKYK2CSWrvw5gkEOTUgmOS4mtTyhLI1XfEFmorG3AZ5lUdBdnEkIrg8/FNAqtGLHjmNrc3iEMmYwpntz5WDQ3OLFJb3g1yoJAWu2CbQMpUENCmrSCJhOAOZoUGZEZgrYWfUUAZY606UZLVMpAbbWUkOYv3XZqS0pAgNNRku2ujX1qUpaEyJugkF1FM4Uk3MYg60uyNKm5hKHgd5ip/FZDLxFX/NdBxTFeV+HIbFwgvQwBWGYT71sEZdgrUAo3KIEC0hYAiwLWCXg8CFIJYNEQAtAUNBoCHKt/AKIuwQ1f9wkw9uzC5JqMGX7IxbYfue1VTtja57xdSrjNsDQbDIaw4tvGjgYBjmLernDesXlzepEcWM/UuzeEBlrSQru5Iytdmn2/XsTb3b/dkwNNXcYuoF/ZSbUpWb15Rq4jcvMeaYTwhhGR5NUnbHw/IrIVZLEbnEcPzg2O4yElOz/npGk4SriFjIJD/Ma2oppDxEFDtTSR1+6fu7F1XKvrKtTNWDah37e4rLfqMWcYVtGU8otWdlXiEeIbW1bgk3whoENp0rhQF74ba6JdvIYuuItYpa3VqPZXTrPEu2cAflpNadNLcTXBma1YJXLvitlfqVQZ1IY2v7P1r9YlCLUluqNekANUNrq1XeOqXuzheToteFk/IyK6/7om7Ey/z44dNn/VOLRiI58JZ69Osvx30qEvZcCwUt9f1Pv/1c9GMK/J2xuZ3DfGP3dPBXykXmzdcexrF0smY2dnXJe/0r/bLxfH/mzWfek4Ppg6P6ur/pfO584ew5nnPPeeA8do6dE4eNDkavRmwUjT8bfzf+Yfx4g14b1T6fOp1n/OQ/yologw==</latexit><latexit sha1_base64="aCYWQvFK/vU26VccTh0Saj0dQa0=">AAAJl3icnZZtb9xEEMd9LdDe8ZTCK8SbFadIKVjHOaQPvIhoUwQFCZGWpq16tqL1eu+yufWu2R1TjOUPxRfhPd+Gsc+uY8eRrljKZW73N/+Z3fHOXphIYWE+/3d07fo777534+Z48v4HH3708c6tT55bnRrGT5iW2rwMqeVSKH4CAiR/mRhO41DyF+H6UTn/4g9urNDqGWQJD2K6UmIpGAUcOt35x4/4En0rpTzOtKFqxYv86Y9HRb5/547r3bvreveLSY/DGDKUaUMeIOR943rfekguUURK4gNAfl64pDMgCuILhTb/E/pBKyA5WpwHReH6Nf+VN+DRht/4PNyvnCanO9P5bF495LLh1cbUqZ/j01s3/vYjzdKYK2CSWrvw5gkEOTUgmOS4mtTyhLI1XfEFmorG3AZ5lUdBdnEkIrg8/FNAqtGLHjmNrc3iEMmYwpntz5WDQ3OLFJb3g1yoJAWu2CbQMpUENCmrSCJhOAOZoUGZEZgrYWfUUAZY606UZLVMpAbbWUkOYv3XZqS0pAgNNRku2ujX1qUpaEyJugkF1FM4Uk3MYg60uyNKm5hKHgd5ip/FZDLxFX/NdBxTFeV+HIbFwgvQwBWGYT71sEZdgrUAo3KIEC0hYAiwLWCXg8CFIJYNEQAtAUNBoCHKt/AKIuwQ1f9wkw9uzC5JqMGX7IxbYfue1VTtja57xdSrjNsDQbDIaw4tvGjgYBjmLernDesXlzepEcWM/UuzeEBlrSQru5Iytdmn2/XsTb3b/dkwNNXcYuoF/ZSbUpWb15Rq4jcvMeaYTwhhGR5NUnbHw/IrIVZLEbnEcPzg2O4yElOz/npGk4SriFjIJD/Ma2oppDxEFDtTSR1+6fu7F1XKvrKtTNWDah37e4rLfqMWcYVtGU8otWdlXiEeIbW1bgk3whoENp0rhQF74ba6JdvIYuuItYpa3VqPZXTrPEu2cAflpNadNLcTXBma1YJXLvitlfqVQZ1IY2v7P1r9YlCLUluqNekANUNrq1XeOqXuzheToteFk/IyK6/7om7Ey/z44dNn/VOLRiI58JZ69Osvx30qEvZcCwUt9f1Pv/1c9GMK/J2xuZ3DfGP3dPBXykXmzdcexrF0smY2dnXJe/0r/bLxfH/mzWfek4Ppg6P6ur/pfO584ew5nnPPeeA8do6dE4eNDkavRmwUjT8bfzf+Yfx4g14b1T6fOp1n/OQ/yologw==</latexit><latexit sha1_base64="aCYWQvFK/vU26VccTh0Saj0dQa0=">AAAJl3icnZZtb9xEEMd9LdDe8ZTCK8SbFadIKVjHOaQPvIhoUwQFCZGWpq16tqL1eu+yufWu2R1TjOUPxRfhPd+Gsc+uY8eRrljKZW73N/+Z3fHOXphIYWE+/3d07fo777534+Z48v4HH3708c6tT55bnRrGT5iW2rwMqeVSKH4CAiR/mRhO41DyF+H6UTn/4g9urNDqGWQJD2K6UmIpGAUcOt35x4/4En0rpTzOtKFqxYv86Y9HRb5/547r3bvreveLSY/DGDKUaUMeIOR943rfekguUURK4gNAfl64pDMgCuILhTb/E/pBKyA5WpwHReH6Nf+VN+DRht/4PNyvnCanO9P5bF495LLh1cbUqZ/j01s3/vYjzdKYK2CSWrvw5gkEOTUgmOS4mtTyhLI1XfEFmorG3AZ5lUdBdnEkIrg8/FNAqtGLHjmNrc3iEMmYwpntz5WDQ3OLFJb3g1yoJAWu2CbQMpUENCmrSCJhOAOZoUGZEZgrYWfUUAZY606UZLVMpAbbWUkOYv3XZqS0pAgNNRku2ujX1qUpaEyJugkF1FM4Uk3MYg60uyNKm5hKHgd5ip/FZDLxFX/NdBxTFeV+HIbFwgvQwBWGYT71sEZdgrUAo3KIEC0hYAiwLWCXg8CFIJYNEQAtAUNBoCHKt/AKIuwQ1f9wkw9uzC5JqMGX7IxbYfue1VTtja57xdSrjNsDQbDIaw4tvGjgYBjmLernDesXlzepEcWM/UuzeEBlrSQru5Iytdmn2/XsTb3b/dkwNNXcYuoF/ZSbUpWb15Rq4jcvMeaYTwhhGR5NUnbHw/IrIVZLEbnEcPzg2O4yElOz/npGk4SriFjIJD/Ma2oppDxEFDtTSR1+6fu7F1XKvrKtTNWDah37e4rLfqMWcYVtGU8otWdlXiEeIbW1bgk3whoENp0rhQF74ba6JdvIYuuItYpa3VqPZXTrPEu2cAflpNadNLcTXBma1YJXLvitlfqVQZ1IY2v7P1r9YlCLUluqNekANUNrq1XeOqXuzheToteFk/IyK6/7om7Ey/z44dNn/VOLRiI58JZ69Osvx30qEvZcCwUt9f1Pv/1c9GMK/J2xuZ3DfGP3dPBXykXmzdcexrF0smY2dnXJe/0r/bLxfH/mzWfek4Ppg6P6ur/pfO584ew5nnPPeeA8do6dE4eNDkavRmwUjT8bfzf+Yfx4g14b1T6fOp1n/OQ/yologw==</latexit>

A[1] A[2] A[3]

pA2[0] pA2[1]

A[3] A[4]

B[0] B[1] B[2]

pB[0] pB[1]

· · ·<latexit sha1_base64="Pr3PvsqCbrDf/uWV7h5idwi1BMA=">AAAIzXicnZVLj9MwEMfDc0t4Lhy5WFQrLagqDUKCy0rL4wBIiOWxC1JTIcdxu6aOHewJEEK4cuUK34KPw7dhkiZkm81KhUhNJ56f/56xk5kglsLCaPT72PETJ0+dXuudcc+eO3/h4qX1y3tWJ4bxXaalNm8CarkUiu+CAMnfxIbTKJD8dTB/UPhff+DGCq1eQRrzSURnSkwFo4BDez4LNdi3l/qj4ai8yGHDq4y+U107b9fXfvmhZknEFTBJrR17oxgmGTUgmOS56yeWx5TN6YyP0VQ04naSleHmZANHQjLVBn8KSDl6cEZGI2vTKEAyorBv275isMs3TmB6d5IJFSfAFVssNE0kAU2K3EkoDGcgUzQoMwJjJWyfGsoAd2hplXg2jSXuzFImGYj558VIYUkRGGpSTNroj3ZAE9AYEh3EFFBP4UjpGEYc6PKOKG0iKnk0yRK8567r+op/ZDqKqAozPwqCfOxN0MAMgyDre3neIlgDMCq7CNEQAroA2wB22gkcWMSyLgKgIaBrEagJ4J/gCCJYIsr/YBEPbswGianBl2yfW2HbM0tXNRunbuZ9rzSudyyChzzn0MDjGp50w7xB/axm/fzwJtWiGLF/yEvVTFZKsrRLKVOZbbrJZ7PvXW97g8CUvnHfm7RDro+q2Lz6qFy/fokxxswlhKX4aZKipmwVj4RYLUU4IIbjjWORSElEzfzmkMYxVyGxkEq+lVXUVEi5hWg+KKmtG76/cVAlkAlfVaZgax37PsG0/6qFXGExwy+U2v0irgA/IbWybgHXwhoEFp0jhYFTuapuwdayWDoircJGt9JjKV05zoLNB51yUuulMFcTnBmaVoJHJvzPSu2TQR1sFvBfWu3DoBalVlSrwwFqunKrVP45pOWdz928VYXjopkVTTKvCvE027n34lX7q0Ujlhx4Qz149nSnTYXCvtNCQUM9fPzySd5eU2B3LpSwBmULu6WDvf0g8/exhXE8OlkxCzt3scl77ZZ+2Ni7NfRGQ+/57f72/ard95yrzjVn0/GcO86288jZcXYd5rxzvjs/nJ+9Z72k96X3dYEeP1bNueIsXb1vfwDdTh5a</latexit><latexit sha1_base64="Pr3PvsqCbrDf/uWV7h5idwi1BMA=">AAAIzXicnZVLj9MwEMfDc0t4Lhy5WFQrLagqDUKCy0rL4wBIiOWxC1JTIcdxu6aOHewJEEK4cuUK34KPw7dhkiZkm81KhUhNJ56f/56xk5kglsLCaPT72PETJ0+dXuudcc+eO3/h4qX1y3tWJ4bxXaalNm8CarkUiu+CAMnfxIbTKJD8dTB/UPhff+DGCq1eQRrzSURnSkwFo4BDez4LNdi3l/qj4ai8yGHDq4y+U107b9fXfvmhZknEFTBJrR17oxgmGTUgmOS56yeWx5TN6YyP0VQ04naSleHmZANHQjLVBn8KSDl6cEZGI2vTKEAyorBv275isMs3TmB6d5IJFSfAFVssNE0kAU2K3EkoDGcgUzQoMwJjJWyfGsoAd2hplXg2jSXuzFImGYj558VIYUkRGGpSTNroj3ZAE9AYEh3EFFBP4UjpGEYc6PKOKG0iKnk0yRK8567r+op/ZDqKqAozPwqCfOxN0MAMgyDre3neIlgDMCq7CNEQAroA2wB22gkcWMSyLgKgIaBrEagJ4J/gCCJYIsr/YBEPbswGianBl2yfW2HbM0tXNRunbuZ9rzSudyyChzzn0MDjGp50w7xB/axm/fzwJtWiGLF/yEvVTFZKsrRLKVOZbbrJZ7PvXW97g8CUvnHfm7RDro+q2Lz6qFy/fokxxswlhKX4aZKipmwVj4RYLUU4IIbjjWORSElEzfzmkMYxVyGxkEq+lVXUVEi5hWg+KKmtG76/cVAlkAlfVaZgax37PsG0/6qFXGExwy+U2v0irgA/IbWybgHXwhoEFp0jhYFTuapuwdayWDoircJGt9JjKV05zoLNB51yUuulMFcTnBmaVoJHJvzPSu2TQR1sFvBfWu3DoBalVlSrwwFqunKrVP45pOWdz928VYXjopkVTTKvCvE027n34lX7q0Ujlhx4Qz149nSnTYXCvtNCQUM9fPzySd5eU2B3LpSwBmULu6WDvf0g8/exhXE8OlkxCzt3scl77ZZ+2Ni7NfRGQ+/57f72/ard95yrzjVn0/GcO86288jZcXYd5rxzvjs/nJ+9Z72k96X3dYEeP1bNueIsXb1vfwDdTh5a</latexit><latexit sha1_base64="Pr3PvsqCbrDf/uWV7h5idwi1BMA=">AAAIzXicnZVLj9MwEMfDc0t4Lhy5WFQrLagqDUKCy0rL4wBIiOWxC1JTIcdxu6aOHewJEEK4cuUK34KPw7dhkiZkm81KhUhNJ56f/56xk5kglsLCaPT72PETJ0+dXuudcc+eO3/h4qX1y3tWJ4bxXaalNm8CarkUiu+CAMnfxIbTKJD8dTB/UPhff+DGCq1eQRrzSURnSkwFo4BDez4LNdi3l/qj4ai8yGHDq4y+U107b9fXfvmhZknEFTBJrR17oxgmGTUgmOS56yeWx5TN6YyP0VQ04naSleHmZANHQjLVBn8KSDl6cEZGI2vTKEAyorBv275isMs3TmB6d5IJFSfAFVssNE0kAU2K3EkoDGcgUzQoMwJjJWyfGsoAd2hplXg2jSXuzFImGYj558VIYUkRGGpSTNroj3ZAE9AYEh3EFFBP4UjpGEYc6PKOKG0iKnk0yRK8567r+op/ZDqKqAozPwqCfOxN0MAMgyDre3neIlgDMCq7CNEQAroA2wB22gkcWMSyLgKgIaBrEagJ4J/gCCJYIsr/YBEPbswGianBl2yfW2HbM0tXNRunbuZ9rzSudyyChzzn0MDjGp50w7xB/axm/fzwJtWiGLF/yEvVTFZKsrRLKVOZbbrJZ7PvXW97g8CUvnHfm7RDro+q2Lz6qFy/fokxxswlhKX4aZKipmwVj4RYLUU4IIbjjWORSElEzfzmkMYxVyGxkEq+lVXUVEi5hWg+KKmtG76/cVAlkAlfVaZgax37PsG0/6qFXGExwy+U2v0irgA/IbWybgHXwhoEFp0jhYFTuapuwdayWDoircJGt9JjKV05zoLNB51yUuulMFcTnBmaVoJHJvzPSu2TQR1sFvBfWu3DoBalVlSrwwFqunKrVP45pOWdz928VYXjopkVTTKvCvE027n34lX7q0Ujlhx4Qz149nSnTYXCvtNCQUM9fPzySd5eU2B3LpSwBmULu6WDvf0g8/exhXE8OlkxCzt3scl77ZZ+2Ni7NfRGQ+/57f72/ard95yrzjVn0/GcO86288jZcXYd5rxzvjs/nJ+9Z72k96X3dYEeP1bNueIsXb1vfwDdTh5a</latexit><latexit sha1_base64="Pr3PvsqCbrDf/uWV7h5idwi1BMA=">AAAIzXicnZVLj9MwEMfDc0t4Lhy5WFQrLagqDUKCy0rL4wBIiOWxC1JTIcdxu6aOHewJEEK4cuUK34KPw7dhkiZkm81KhUhNJ56f/56xk5kglsLCaPT72PETJ0+dXuudcc+eO3/h4qX1y3tWJ4bxXaalNm8CarkUiu+CAMnfxIbTKJD8dTB/UPhff+DGCq1eQRrzSURnSkwFo4BDez4LNdi3l/qj4ai8yGHDq4y+U107b9fXfvmhZknEFTBJrR17oxgmGTUgmOS56yeWx5TN6YyP0VQ04naSleHmZANHQjLVBn8KSDl6cEZGI2vTKEAyorBv275isMs3TmB6d5IJFSfAFVssNE0kAU2K3EkoDGcgUzQoMwJjJWyfGsoAd2hplXg2jSXuzFImGYj558VIYUkRGGpSTNroj3ZAE9AYEh3EFFBP4UjpGEYc6PKOKG0iKnk0yRK8567r+op/ZDqKqAozPwqCfOxN0MAMgyDre3neIlgDMCq7CNEQAroA2wB22gkcWMSyLgKgIaBrEagJ4J/gCCJYIsr/YBEPbswGianBl2yfW2HbM0tXNRunbuZ9rzSudyyChzzn0MDjGp50w7xB/axm/fzwJtWiGLF/yEvVTFZKsrRLKVOZbbrJZ7PvXW97g8CUvnHfm7RDro+q2Lz6qFy/fokxxswlhKX4aZKipmwVj4RYLUU4IIbjjWORSElEzfzmkMYxVyGxkEq+lVXUVEi5hWg+KKmtG76/cVAlkAlfVaZgax37PsG0/6qFXGExwy+U2v0irgA/IbWybgHXwhoEFp0jhYFTuapuwdayWDoircJGt9JjKV05zoLNB51yUuulMFcTnBmaVoJHJvzPSu2TQR1sFvBfWu3DoBalVlSrwwFqunKrVP45pOWdz928VYXjopkVTTKvCvE027n34lX7q0Ujlhx4Qz149nSnTYXCvtNCQUM9fPzySd5eU2B3LpSwBmULu6WDvf0g8/exhXE8OlkxCzt3scl77ZZ+2Ni7NfRGQ+/57f72/ard95yrzjVn0/GcO86288jZcXYd5rxzvjs/nJ+9Z72k96X3dYEeP1bNueIsXb1vfwDdTh5a</latexit>

Many partitions can satisfy the constraint

Constraint that characterize all legal partitions for pA2:

image(pB, λi.i+1) ⊆ pA2

The program pA2 = image(pB, λi.i+1) is one solution of this constraint

Example:

Example: Particle Simulation

13C (Cells)PT (ParTicles)

for i in PT: c = PT[i].cell PT[i].pos = f(C[c].vel,C[n(c)].vel)

Updates the position of every particle using the velocity of cells

PT[∙].cell

n

for i in PT: c = PT[i].cell PT[i].pos = f(C[c].vel,C[n(c)].vel)

Identifies necessary data partitions

Constraint Inference

14

Needs two partitions pC1 and pC2 of C for different access patterns

Needs a partition pPT of PT

parallel for x in pPT: PT = pPT[x]; C1 = pC1[x]; C2 = pC2[x] for i in PT: c = PT[i].cell PT[i].pos = f(C1[c].vel,C2[n(c)].vel)

Infers partitioning constraints

require(complete(pPT,PT))require(image(pPT,PT[∙].cell) ⊆ pC1)require(image(pC1,n) ⊆ pC2)

pPT[j]

pC2[j]

pC1[j]

8j, 8i 2 pPT[j], PT[i].cell 2 pC1[j]<latexit sha1_base64="+haAItm5oWkHnhKwSPxGVjXayV0=">AAAJvXicnZZbb9s2FMflrltr7Zauj3shZgRIB8Gz0qSXh2BtM2AXYJjXJW0xSwgoinYYU6RGUks1QUBft2+zj7PvsQ+wo1sVyQrgTkDiI/J3/jyHl0MFMWfazGb/jG68d/P9D27dHtsffvTxJ5/u3PnshZaJIvSUSC7VqwBrypmgp4YZTl/FiuIo4PRlsD4u+l/+TpVmUpyYNKZ+hFeCLRnBBprOdv71QroE31Ipi1KpsFjRPHv+7bM82z9wHffQdQ4Pc7vHBTxpKPf+oePef+C4D/c3MEXDhno0cw4eOw8KZgmjcI48Y0x2kTuo08By5DEBNn1t+lGVQDw/WVz4ee54qPKAd+ZPCeV8wLUKtHI8dktH+2xnMpvOygdtGm5tTKz6mZ/dufW3F0qSRFQYwrHWC3cWGz/DyjDCKaSUaBpjssYrugBT4IhqPytjyNEutIQIcoQ/YVDZetUjw5HWaRQAGWFzrvt9ReNQ3yIxy0d+xkScGCpINdAy4chIVKw1CpmixPAUDEwUg1gROccKEwM7ojNKvFrGXBrdySQzbP1H1VJYnAUKqxSSVvJSOzgxEkLCTowN6AloKTumETW4OyNCqghzGvlZAv9z27Y9QS+JjCIswsyLgiBfuD4YkGEQZBMX1qhLkBYgmA8RrCWYGQJ0C+jlIHBlEE2GCGNawgwNYhqi2IHXEEGHKH+DKh6YmF0UYwWb7JxqpvueZVftDa57+cQtjXsDg8Air6lp4UUD+8MwbVEva1gv35ykRhQi9jZ64ZTyWomXdimlarNPt/nsTdx7/d4gUGXfYuL6/ZCbpSomr1kq22s2McSY2QiRFI4mKmroUfGKkJachQ6CmuQgCkUxRRFW66+mOI6pCJE2KadHWU0tGedHRflySuroS8/bvapS1JVtZcoaVOvo3xJI+61aSAUUbzihWJ8XcQVwhMTWugXcCEvDoOhcK2wo5tvqFmwjC6UjkiJsdWs9kuKt4yzY3BmU41J2wtxOcKVwWgtem/A7K/VXBnRCCaXt/2j1FwNrkNpSrQnHYDWUW63yziF1Zz63814VjovLrPgoyOtCvMzmT5+f9E8tGDGnhrbU8U8/zvtUyPSFZMK01Dff//JD3h+TwddIdTsHWWX3dOBb5irz9rWHUVg6XjOVXV7ybv9K3zRe7E/d2dT9+WDy5Fl93d+2Pre+sPYs13poPbG+s+bWqUVGv47ejP4c/TX+ekzHfCwq9Mao9rlrdZ7x5X91WHbK</latexit><latexit sha1_base64="+haAItm5oWkHnhKwSPxGVjXayV0=">AAAJvXicnZZbb9s2FMflrltr7Zauj3shZgRIB8Gz0qSXh2BtM2AXYJjXJW0xSwgoinYYU6RGUks1QUBft2+zj7PvsQ+wo1sVyQrgTkDiI/J3/jyHl0MFMWfazGb/jG68d/P9D27dHtsffvTxJ5/u3PnshZaJIvSUSC7VqwBrypmgp4YZTl/FiuIo4PRlsD4u+l/+TpVmUpyYNKZ+hFeCLRnBBprOdv71QroE31Ipi1KpsFjRPHv+7bM82z9wHffQdQ4Pc7vHBTxpKPf+oePef+C4D/c3MEXDhno0cw4eOw8KZgmjcI48Y0x2kTuo08By5DEBNn1t+lGVQDw/WVz4ee54qPKAd+ZPCeV8wLUKtHI8dktH+2xnMpvOygdtGm5tTKz6mZ/dufW3F0qSRFQYwrHWC3cWGz/DyjDCKaSUaBpjssYrugBT4IhqPytjyNEutIQIcoQ/YVDZetUjw5HWaRQAGWFzrvt9ReNQ3yIxy0d+xkScGCpINdAy4chIVKw1CpmixPAUDEwUg1gROccKEwM7ojNKvFrGXBrdySQzbP1H1VJYnAUKqxSSVvJSOzgxEkLCTowN6AloKTumETW4OyNCqghzGvlZAv9z27Y9QS+JjCIswsyLgiBfuD4YkGEQZBMX1qhLkBYgmA8RrCWYGQJ0C+jlIHBlEE2GCGNawgwNYhqi2IHXEEGHKH+DKh6YmF0UYwWb7JxqpvueZVftDa57+cQtjXsDg8Air6lp4UUD+8MwbVEva1gv35ykRhQi9jZ64ZTyWomXdimlarNPt/nsTdx7/d4gUGXfYuL6/ZCbpSomr1kq22s2McSY2QiRFI4mKmroUfGKkJachQ6CmuQgCkUxRRFW66+mOI6pCJE2KadHWU0tGedHRflySuroS8/bvapS1JVtZcoaVOvo3xJI+61aSAUUbzihWJ8XcQVwhMTWugXcCEvDoOhcK2wo5tvqFmwjC6UjkiJsdWs9kuKt4yzY3BmU41J2wtxOcKVwWgtem/A7K/VXBnRCCaXt/2j1FwNrkNpSrQnHYDWUW63yziF1Zz63814VjovLrPgoyOtCvMzmT5+f9E8tGDGnhrbU8U8/zvtUyPSFZMK01Dff//JD3h+TwddIdTsHWWX3dOBb5irz9rWHUVg6XjOVXV7ybv9K3zRe7E/d2dT9+WDy5Fl93d+2Pre+sPYs13poPbG+s+bWqUVGv47ejP4c/TX+ekzHfCwq9Mao9rlrdZ7x5X91WHbK</latexit><latexit sha1_base64="+haAItm5oWkHnhKwSPxGVjXayV0=">AAAJvXicnZZbb9s2FMflrltr7Zauj3shZgRIB8Gz0qSXh2BtM2AXYJjXJW0xSwgoinYYU6RGUks1QUBft2+zj7PvsQ+wo1sVyQrgTkDiI/J3/jyHl0MFMWfazGb/jG68d/P9D27dHtsffvTxJ5/u3PnshZaJIvSUSC7VqwBrypmgp4YZTl/FiuIo4PRlsD4u+l/+TpVmUpyYNKZ+hFeCLRnBBprOdv71QroE31Ipi1KpsFjRPHv+7bM82z9wHffQdQ4Pc7vHBTxpKPf+oePef+C4D/c3MEXDhno0cw4eOw8KZgmjcI48Y0x2kTuo08By5DEBNn1t+lGVQDw/WVz4ee54qPKAd+ZPCeV8wLUKtHI8dktH+2xnMpvOygdtGm5tTKz6mZ/dufW3F0qSRFQYwrHWC3cWGz/DyjDCKaSUaBpjssYrugBT4IhqPytjyNEutIQIcoQ/YVDZetUjw5HWaRQAGWFzrvt9ReNQ3yIxy0d+xkScGCpINdAy4chIVKw1CpmixPAUDEwUg1gROccKEwM7ojNKvFrGXBrdySQzbP1H1VJYnAUKqxSSVvJSOzgxEkLCTowN6AloKTumETW4OyNCqghzGvlZAv9z27Y9QS+JjCIswsyLgiBfuD4YkGEQZBMX1qhLkBYgmA8RrCWYGQJ0C+jlIHBlEE2GCGNawgwNYhqi2IHXEEGHKH+DKh6YmF0UYwWb7JxqpvueZVftDa57+cQtjXsDg8Air6lp4UUD+8MwbVEva1gv35ykRhQi9jZ64ZTyWomXdimlarNPt/nsTdx7/d4gUGXfYuL6/ZCbpSomr1kq22s2McSY2QiRFI4mKmroUfGKkJachQ6CmuQgCkUxRRFW66+mOI6pCJE2KadHWU0tGedHRflySuroS8/bvapS1JVtZcoaVOvo3xJI+61aSAUUbzihWJ8XcQVwhMTWugXcCEvDoOhcK2wo5tvqFmwjC6UjkiJsdWs9kuKt4yzY3BmU41J2wtxOcKVwWgtem/A7K/VXBnRCCaXt/2j1FwNrkNpSrQnHYDWUW63yziF1Zz63814VjovLrPgoyOtCvMzmT5+f9E8tGDGnhrbU8U8/zvtUyPSFZMK01Dff//JD3h+TwddIdTsHWWX3dOBb5irz9rWHUVg6XjOVXV7ybv9K3zRe7E/d2dT9+WDy5Fl93d+2Pre+sPYs13poPbG+s+bWqUVGv47ejP4c/TX+ekzHfCwq9Mao9rlrdZ7x5X91WHbK</latexit><latexit sha1_base64="+haAItm5oWkHnhKwSPxGVjXayV0=">AAAJvXicnZZbb9s2FMflrltr7Zauj3shZgRIB8Gz0qSXh2BtM2AXYJjXJW0xSwgoinYYU6RGUks1QUBft2+zj7PvsQ+wo1sVyQrgTkDiI/J3/jyHl0MFMWfazGb/jG68d/P9D27dHtsffvTxJ5/u3PnshZaJIvSUSC7VqwBrypmgp4YZTl/FiuIo4PRlsD4u+l/+TpVmUpyYNKZ+hFeCLRnBBprOdv71QroE31Ipi1KpsFjRPHv+7bM82z9wHffQdQ4Pc7vHBTxpKPf+oePef+C4D/c3MEXDhno0cw4eOw8KZgmjcI48Y0x2kTuo08By5DEBNn1t+lGVQDw/WVz4ee54qPKAd+ZPCeV8wLUKtHI8dktH+2xnMpvOygdtGm5tTKz6mZ/dufW3F0qSRFQYwrHWC3cWGz/DyjDCKaSUaBpjssYrugBT4IhqPytjyNEutIQIcoQ/YVDZetUjw5HWaRQAGWFzrvt9ReNQ3yIxy0d+xkScGCpINdAy4chIVKw1CpmixPAUDEwUg1gROccKEwM7ojNKvFrGXBrdySQzbP1H1VJYnAUKqxSSVvJSOzgxEkLCTowN6AloKTumETW4OyNCqghzGvlZAv9z27Y9QS+JjCIswsyLgiBfuD4YkGEQZBMX1qhLkBYgmA8RrCWYGQJ0C+jlIHBlEE2GCGNawgwNYhqi2IHXEEGHKH+DKh6YmF0UYwWb7JxqpvueZVftDa57+cQtjXsDg8Air6lp4UUD+8MwbVEva1gv35ykRhQi9jZ64ZTyWomXdimlarNPt/nsTdx7/d4gUGXfYuL6/ZCbpSomr1kq22s2McSY2QiRFI4mKmroUfGKkJachQ6CmuQgCkUxRRFW66+mOI6pCJE2KadHWU0tGedHRflySuroS8/bvapS1JVtZcoaVOvo3xJI+61aSAUUbzihWJ8XcQVwhMTWugXcCEvDoOhcK2wo5tvqFmwjC6UjkiJsdWs9kuKt4yzY3BmU41J2wtxOcKVwWgtem/A7K/VXBnRCCaXt/2j1FwNrkNpSrQnHYDWUW63yziF1Zz63814VjovLrPgoyOtCvMzmT5+f9E8tGDGnhrbU8U8/zvtUyPSFZMK01Dff//JD3h+TwddIdTsHWWX3dOBb5irz9rWHUVg6XjOVXV7ybv9K3zRe7E/d2dT9+WDy5Fl93d+2Pre+sPYs13poPbG+s+bWqUVGv47ejP4c/TX+ekzHfCwq9Mao9rlrdZ7x5X91WHbK</latexit>

8j, 8i 2 pC1[j], n(i) 2 pC2[j]<latexit sha1_base64="w10KbspzNr+xCirENdPPS++1m4o=">AAAJtHicnZbrbts2FMfl7tJau6Xrx30hZgRIBsGz0iRtPwTomgG7AMPSrmkLWEJAUZTDmCI1klqmCQK2R9rj7Bn2Eju6VZGsAO4E2D7m+Z3/OSSlQwUJZ9osFv9M7rz3/gcf3r03tT/6+JNPP9u5//krLVNF6DmRXKo3AdaUM0HPDTOcvkkUxXHA6etgfVr6X/9GlWZSvDRZQv0YrwSLGMEGhi52/vVCGkFspZTHmVRYrGiRv/juWZEfHLqOe+Q6R0eFPeACnraU+/DIcR8eO+6jgw1M0bClHi+cwyfOcclEkIVz5Blj8qvCQb0BViCPCbDp76afrXInp+7yyi8Kx6txscf26wi7F1JlbiIOqgj7Yme2mC+qC20abmPMrOY6u7h/928vlCSNqTCEY62X7iIxfo6VYYRTmEmqaYLJGq/oEkyBY6r9vCqhQLswEiKYGnyEQdXozYgcx1pncQBkjM2lHvrKwTHfMjXRYz9nIkkNFaROFKUcGYnKLUYhU5QYnoGBiWJQKyKXWGFi4EboZUlWUcKl0b2Z5Iat/6hHSouzQGGVwaSVvNYOTo2EkrCTYAN6AkYqxzymBvdXREgVY05jP0/hu7Bt2xP0msg4xiLMvTgIiqXrgwEzDIJ85sIe9QnSAQTzMYJ1BDNjgO4AHY0CN5JoMkYY0xFmLIlpifIGvIUIekT1G9T1wMLsogQruMkuqWZ6GFm5mmgI3StmbmXsjySBTV5T08HLFvbHYdqhXt6yXrG5SK0oVOxteKFl8EaJV3YlpRpzSHfz2Zu5+0NvEKjKt5y5/rDkdqvKxWu3yvbamxhqzG2ESAaPJipb50n5FyEtOQsdBA3BQRR6YYZirNZfz3GSUBEibTJOT/KGihjnJ2XvcCrq5CvP272pUnaibWWqrtXo6F9TmPZbtZAK6NnwhGJ9WdYVwCMkttYt4VZYGgZN51ZhQzHfVrdkW1loHbEUYafb6JEMb11nyRbOqByXslfmdoIrhbNG8NYJv7PScGdAJ5TQ2v6P1nAzsAapLdXacgxWY3NrVN65pP7KF3Yx6MJJeZiV7wJF04ij/OybFy+HTy0YCaeGdtTpzz+dDamQ6SvJhOmob3/45cdimJPBS0h9ngd5bQ904BXmJvP27wCjsHW8YWq7OuTd4ZG+abw6mLuLufv8cPb0WXPc37O+sL609izXemQ9tb63zqxzi0yeT64nf07+mh5PvSmZ0hq9M2liHli9ayr+A5nDcn8=</latexit><latexit sha1_base64="w10KbspzNr+xCirENdPPS++1m4o=">AAAJtHicnZbrbts2FMfl7tJau6Xrx30hZgRIBsGz0iRtPwTomgG7AMPSrmkLWEJAUZTDmCI1klqmCQK2R9rj7Bn2Eju6VZGsAO4E2D7m+Z3/OSSlQwUJZ9osFv9M7rz3/gcf3r03tT/6+JNPP9u5//krLVNF6DmRXKo3AdaUM0HPDTOcvkkUxXHA6etgfVr6X/9GlWZSvDRZQv0YrwSLGMEGhi52/vVCGkFspZTHmVRYrGiRv/juWZEfHLqOe+Q6R0eFPeACnraU+/DIcR8eO+6jgw1M0bClHi+cwyfOcclEkIVz5Blj8qvCQb0BViCPCbDp76afrXInp+7yyi8Kx6txscf26wi7F1JlbiIOqgj7Yme2mC+qC20abmPMrOY6u7h/928vlCSNqTCEY62X7iIxfo6VYYRTmEmqaYLJGq/oEkyBY6r9vCqhQLswEiKYGnyEQdXozYgcx1pncQBkjM2lHvrKwTHfMjXRYz9nIkkNFaROFKUcGYnKLUYhU5QYnoGBiWJQKyKXWGFi4EboZUlWUcKl0b2Z5Iat/6hHSouzQGGVwaSVvNYOTo2EkrCTYAN6AkYqxzymBvdXREgVY05jP0/hu7Bt2xP0msg4xiLMvTgIiqXrgwEzDIJ85sIe9QnSAQTzMYJ1BDNjgO4AHY0CN5JoMkYY0xFmLIlpifIGvIUIekT1G9T1wMLsogQruMkuqWZ6GFm5mmgI3StmbmXsjySBTV5T08HLFvbHYdqhXt6yXrG5SK0oVOxteKFl8EaJV3YlpRpzSHfz2Zu5+0NvEKjKt5y5/rDkdqvKxWu3yvbamxhqzG2ESAaPJipb50n5FyEtOQsdBA3BQRR6YYZirNZfz3GSUBEibTJOT/KGihjnJ2XvcCrq5CvP272pUnaibWWqrtXo6F9TmPZbtZAK6NnwhGJ9WdYVwCMkttYt4VZYGgZN51ZhQzHfVrdkW1loHbEUYafb6JEMb11nyRbOqByXslfmdoIrhbNG8NYJv7PScGdAJ5TQ2v6P1nAzsAapLdXacgxWY3NrVN65pP7KF3Yx6MJJeZiV7wJF04ij/OybFy+HTy0YCaeGdtTpzz+dDamQ6SvJhOmob3/45cdimJPBS0h9ngd5bQ904BXmJvP27wCjsHW8YWq7OuTd4ZG+abw6mLuLufv8cPb0WXPc37O+sL609izXemQ9tb63zqxzi0yeT64nf07+mh5PvSmZ0hq9M2liHli9ayr+A5nDcn8=</latexit><latexit sha1_base64="w10KbspzNr+xCirENdPPS++1m4o=">AAAJtHicnZbrbts2FMfl7tJau6Xrx30hZgRIBsGz0iRtPwTomgG7AMPSrmkLWEJAUZTDmCI1klqmCQK2R9rj7Bn2Eju6VZGsAO4E2D7m+Z3/OSSlQwUJZ9osFv9M7rz3/gcf3r03tT/6+JNPP9u5//krLVNF6DmRXKo3AdaUM0HPDTOcvkkUxXHA6etgfVr6X/9GlWZSvDRZQv0YrwSLGMEGhi52/vVCGkFspZTHmVRYrGiRv/juWZEfHLqOe+Q6R0eFPeACnraU+/DIcR8eO+6jgw1M0bClHi+cwyfOcclEkIVz5Blj8qvCQb0BViCPCbDp76afrXInp+7yyi8Kx6txscf26wi7F1JlbiIOqgj7Yme2mC+qC20abmPMrOY6u7h/928vlCSNqTCEY62X7iIxfo6VYYRTmEmqaYLJGq/oEkyBY6r9vCqhQLswEiKYGnyEQdXozYgcx1pncQBkjM2lHvrKwTHfMjXRYz9nIkkNFaROFKUcGYnKLUYhU5QYnoGBiWJQKyKXWGFi4EboZUlWUcKl0b2Z5Iat/6hHSouzQGGVwaSVvNYOTo2EkrCTYAN6AkYqxzymBvdXREgVY05jP0/hu7Bt2xP0msg4xiLMvTgIiqXrgwEzDIJ85sIe9QnSAQTzMYJ1BDNjgO4AHY0CN5JoMkYY0xFmLIlpifIGvIUIekT1G9T1wMLsogQruMkuqWZ6GFm5mmgI3StmbmXsjySBTV5T08HLFvbHYdqhXt6yXrG5SK0oVOxteKFl8EaJV3YlpRpzSHfz2Zu5+0NvEKjKt5y5/rDkdqvKxWu3yvbamxhqzG2ESAaPJipb50n5FyEtOQsdBA3BQRR6YYZirNZfz3GSUBEibTJOT/KGihjnJ2XvcCrq5CvP272pUnaibWWqrtXo6F9TmPZbtZAK6NnwhGJ9WdYVwCMkttYt4VZYGgZN51ZhQzHfVrdkW1loHbEUYafb6JEMb11nyRbOqByXslfmdoIrhbNG8NYJv7PScGdAJ5TQ2v6P1nAzsAapLdXacgxWY3NrVN65pP7KF3Yx6MJJeZiV7wJF04ij/OybFy+HTy0YCaeGdtTpzz+dDamQ6SvJhOmob3/45cdimJPBS0h9ngd5bQ904BXmJvP27wCjsHW8YWq7OuTd4ZG+abw6mLuLufv8cPb0WXPc37O+sL609izXemQ9tb63zqxzi0yeT64nf07+mh5PvSmZ0hq9M2liHli9ayr+A5nDcn8=</latexit><latexit sha1_base64="w10KbspzNr+xCirENdPPS++1m4o=">AAAJtHicnZbrbts2FMfl7tJau6Xrx30hZgRIBsGz0iRtPwTomgG7AMPSrmkLWEJAUZTDmCI1klqmCQK2R9rj7Bn2Eju6VZGsAO4E2D7m+Z3/OSSlQwUJZ9osFv9M7rz3/gcf3r03tT/6+JNPP9u5//krLVNF6DmRXKo3AdaUM0HPDTOcvkkUxXHA6etgfVr6X/9GlWZSvDRZQv0YrwSLGMEGhi52/vVCGkFspZTHmVRYrGiRv/juWZEfHLqOe+Q6R0eFPeACnraU+/DIcR8eO+6jgw1M0bClHi+cwyfOcclEkIVz5Blj8qvCQb0BViCPCbDp76afrXInp+7yyi8Kx6txscf26wi7F1JlbiIOqgj7Yme2mC+qC20abmPMrOY6u7h/928vlCSNqTCEY62X7iIxfo6VYYRTmEmqaYLJGq/oEkyBY6r9vCqhQLswEiKYGnyEQdXozYgcx1pncQBkjM2lHvrKwTHfMjXRYz9nIkkNFaROFKUcGYnKLUYhU5QYnoGBiWJQKyKXWGFi4EboZUlWUcKl0b2Z5Iat/6hHSouzQGGVwaSVvNYOTo2EkrCTYAN6AkYqxzymBvdXREgVY05jP0/hu7Bt2xP0msg4xiLMvTgIiqXrgwEzDIJ85sIe9QnSAQTzMYJ1BDNjgO4AHY0CN5JoMkYY0xFmLIlpifIGvIUIekT1G9T1wMLsogQruMkuqWZ6GFm5mmgI3StmbmXsjySBTV5T08HLFvbHYdqhXt6yXrG5SK0oVOxteKFl8EaJV3YlpRpzSHfz2Zu5+0NvEKjKt5y5/rDkdqvKxWu3yvbamxhqzG2ESAaPJipb50n5FyEtOQsdBA3BQRR6YYZirNZfz3GSUBEibTJOT/KGihjnJ2XvcCrq5CvP272pUnaibWWqrtXo6F9TmPZbtZAK6NnwhGJ9WdYVwCMkttYt4VZYGgZN51ZhQzHfVrdkW1loHbEUYafb6JEMb11nyRbOqByXslfmdoIrhbNG8NYJv7PScGdAJ5TQ2v6P1nAzsAapLdXacgxWY3NrVN65pP7KF3Yx6MJJeZiV7wJF04ij/OybFy+HTy0YCaeGdtTpzz+dDamQ6SvJhOmob3/45cdimJPBS0h9ngd5bQ904BXmJvP27wCjsHW8YWq7OuTd4ZG+abw6mLuLufv8cPb0WXPc37O+sL609izXemQ9tb63zqxzi0yeT64nf07+mh5PvSmZ0hq9M2liHli9ayr+A5nDcn8=</latexit>

A function that maps particles to cells

Must run all iterations at least once

Two solutions for :

Solving Constraints

15

pC1 = equal(C,N) pPT = preimage(PT[∙].cell,pC1)

✔

require(image(pPT,PT[∙].cell) ⊆ pC1)

pPT = equal(PT,N) pC1 = image(pPT,PT[∙].cell) pC2 = image(pC1,n)

require(complete(pPT,PT)) require(image(pPT,PT[∙].cell) ⊆ pC1) require(image(pC1,n) ⊆ pC2)

Solve constraints

pPT[0]

pPT[1]

pPT[0]

pPT[1]

pC1[0]

pC1[1] pC1[0] pC1[1]

pPT = equal(PT,N) pC1 = image(pPT,PT[∙].cell)

Handling Multiple Loops

16

for i in PT: c = PT[i].cell PT[i].pos = g(C[c].vel,C[n(c)].vel)

for i in C: C[i].vel = h(C[i].acc,C[n(i)].acc)

require(image(pC1,n) ⊆ pC2)require(image(pPT,PT[∙].cell) ⊆ pC1)require(complete(pPT,PT))

require(image(pC3,n) ⊆ pC4)require(complete(pC3,C))

One loop = One set of partitioning constraints

✔ Capture all possible partitioning strategies

✘ Can lead to excessive communication if solved naïvely



Handling Multiple Loops

17

for i in PT: c = PT[i].cell PT[i].pos = g(C[c].vel,C[n(c)].vel)

for i in C: C[i].vel = h(C[i].acc,C[n(i)].acc)

Constraint solver unifies partitions to maximize partition reuse

Similar access patterns Isomorphic constraints

pC3 = pC1 = equal(PT,N) pC4 = pC2 = image(pC1,n)

pPT = preimage(PT[∙].cell,pC1)Solution:

Unified!

External Constraints

18

while t < T: for i in PT: c = PT[i].cell PT[i].pos = g(C[c].vel, C[n(c)].vel)

for i in C: C[i].vel = g(C[i].acc, C[n(i)].acc)

for i in PT: PT[i].cell = h(PT[i].pos)

In the real simulation code, particles might move to different cells,

pC1 = equal(C,N) pC2 = image(pC1,n)

while t < T: pPT = preimage(PT[∙].cell,pC1)

parallel for x in pPT: ...

parallel for x in pC1: ...

parallel for x in pPT: ...

requiring pPT to be repartitioned every time step ☹

Solve constraints

External Constraints

19

while t < T: ... // Manual particle transfer code using // pParticle and pCell ... assert( image(pParticle,PT[∙].cell) ⊆ pCell)) ...

pC3 = pC1 = pCell pC4 = pC2 = image(pCell,n)

while t < T: ... // Manual particle transfer code ... pPT = pParticle ...

External constraints provide a precise control over the automated data partitioning process

User can manually parallelize particle transfer code and provide external constraints as an interface:

Solve constraints with external constraints

No more repartitioning 🙂



Partitioning constraints: Unifiable constraints

Evaluation

• Implemented the constraint inference and solver algorithms in Regent

• Regent is a high-level programming language with first-class data partitions and DPL

• Weak scaling performance of four benchmark programs

• Stencil: 9-point stencil in 2D grid

• MiniAero: explicit Navier-stokes solver on hexahedral 3D mesh

• Circuit: circuit simulator on unstructured circuit graphs

• PENNANT: Lagrangian hydrodynamics on unstructured 2D mesh

• Machine: Piz Daint (12-core Xeon E5-2690, NVIDIA P100, and 64 GB memory per node)

• All benchmark programs ran on GPUs

20Elliott Slaughter, Wonchan Lee, Sean Treichler, Michael Bauer, Alex Aiken, Regent: a high-productivity programming language for HPC with logical regions. SC15.†<latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit>


Weak Scaling: Stencil and MiniAero

21

Auto-parallelized programs match hand-parallelized programs within 3%

Auto-Parallelized Compute Tasks

Parallel Circuit Generator

Auto-Parallelized Circuit

Weak Scaling: Circuit

22

Add interface constraints

Interface constraints: complete(pNodes_private ⋃ pNodes_shared,Nodes)

Generator uses two node partitions:pNodes_private and pNodes_shared

piece 1

piece 3

piece 2

Weak Scaling: PENNANT

23

Add interface constraints

Auto-Parallelized Compute Tasks

Parallel Mesh Generator

Auto-Parallelized PENNANT

Interface constraints: complete(pPoints_private ⋃ pPoints_shared,Points)image(pSides,Sides[∙].prev_side) ⊆ pSidesimage(pSides,Sides[∙].next_side) ⊆ pSides

Each side of a polygon colocates with its previous and next sides

Two points partitions:pPoints_private and pPoints_shared

Case Study: Soleil-X

• Developed for the PSAAP II program at Stanford

• Eulerian Fluid + Lagrangian Particles + Radiation (DOM/Algebraic)

• DOM is manually parallelized

• Fluid and particles are auto-parallelized except for particle transfers

!24

Heated section of concentrated solar energy receiver

Soleil-X Performance

25

92X speedup at 512 for fluid 65X speedup at 256 with particles

17X speedup at 128 for fluid 15X speedup at 128 with particles

77% parallel efficiency (9,216 CPUs)

79% parallel efficiency (30,720 CPUs)

Soleil-X Performance

26

82% parallel efficiency (1,024 GPUs)

Case Study: HTR Solver

• Solves multi-component Navier-Stokes equations in compressible formulation

• Accounts for complex chemistry and multicomponent transport

• Heavy flux tasks are auto-parallelized

27Courtesy of Dr. Mario Di Renzo at Stanford ([email protected])†<latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit><latexit sha1_base64="a9UgLG//bCM4Nz+9ss2vOB7Gfko=">AAAB7nicbVBNS8NAEJ34WetX1aOXxSJ4KokIeix68VjBfkAbymYzSZduNmF3I5TQH+HFgyJe/T3e/Ddu2xy09cHA470ZZuYFmeDauO63s7a+sbm1Xdmp7u7tHxzWjo47Os0VwzZLRap6AdUouMS24UZgL1NIk0BgNxjfzfzuEyrNU/loJhn6CY0ljzijxkrdQUjjGNWwVncb7hxklXglqUOJ1rD2NQhTlicoDRNU677nZsYvqDKcCZxWB7nGjLIxjbFvqaQJar+Ynzsl51YJSZQqW9KQufp7oqCJ1pMksJ0JNSO97M3E/7x+bqIbv+Ayyw1KtlgU5YKYlMx+JyFXyIyYWEKZ4vZWwkZUUWZsQlUbgrf88irpXDY8t+E9XNWbt2UcFTiFM7gAD66hCffQgjYwGMMzvMKbkzkvzrvzsWhdc8qZE/gD5/MHSjqPhg==</latexit>




HTR Performance

28

81% efficiency 18,432 CPUs 400M points 1s/timestep

68% efficiency 1,024 GPUs 4.8B points

0.7s/timestep

Conclusion

• First-class data partitions enable composable and configurable auto-parallelization

• A constraint-based data partitioning brings scalability of manual parallelization to auto-parallelized programs

29

Questions?

https://legion.stanford.edu

a constraint-based approach to automatic data …...2019/09/05 · a constraint-based approach to...

Documents