enabling thread level speculation via a transactional memory system
DESCRIPTION
Helper Transactions. Enabling Thread Level Speculation via A Transactional Memory System. Richard M. YooGeorgia Tech Hsien-Hsin S ean LeeGeorgia Tech. In Workshop on Parallel Execution of Sequential Programs on Multi-core (PESPMA-08). Where are ILP Techniques ?. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/1.jpg)
Enabling Thread Level Speculationvia A Transactional Memory System
Richard M. Yoo Georgia TechHsien-Hsin Sean Lee Georgia Tech
Helper Transactions
In Workshop on Parallel Execution of Sequential Programs on Multi-core (PESPMA-08)
![Page 2: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/2.jpg)
2Helper Transactions: Yoo & Lee
Exploiting Multi-Core Performance • Thread Level Speculation (TLS)
– Extract new threads from single-threaded applications
• Transactional Memory (TM)– Help the existing threads perform better
Where are ILPTechniques ?
![Page 3: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/3.jpg)
3Helper Transactions: Yoo & Lee
TLS versus TM
Contention Management
Sequential Ordering
Context Passing
Task Spawning
Checkpointing
Dependency Violation Detection
Result Buffering
Replay
TransactionScheduling
TLS and TM share multiple hardware components
TLS TM
![Page 4: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/4.jpg)
4Helper Transactions: Yoo & Lee
Helper Transactions
Goal: Enable TLS with a TM-ready system
• Support “out-of-order procedure fall-thru speculation” on TM
• Amortize TLS implementation cost on a TM-ready system
![Page 5: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/5.jpg)
5Helper Transactions: Yoo & Lee
Agenda
• Thread-Level Speculation (TLS)
• Mapping TLS onto A Transactional Memory System
• Extending TM System
![Page 6: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/6.jpg)
6Helper Transactions: Yoo & Lee
Spawning Points of Thread Level Speculation
Non-speculative(loop iteration 0)
More-speculative(loop iteration 2)
Speculative(loop iteration 1)
Loop Speculation
Speculative(code after else)
Non-speculative(if-then-else)
If-then-else Speculation
Speculative(fall-through code)
Non-speculative(function body)
Procedure Fall-Thru Speculation
![Page 7: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/7.jpg)
7Helper Transactions: Yoo & Lee
Out-of-order Spawn
• The spawn order of tasks (0-1-2-3) disagrees with the sequential order (0-3-1-2)
• Complicate sequential ordering maintenance
task 0task 1task 2 task 3
Sequential order
function main() { … foo(); … hoo(); ... }
function foo() { ... goo(); ... }
function hoo() { ... }
foo()
goo()
hoo()Out-of-Order
Procedure Fall-Thru Speculation
Helper Transactions focus on out-of-order procedure fall-through speculation
Spawning
![Page 8: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/8.jpg)
8Helper Transactions: Yoo & Lee
Agenda
• Thread-Level Speculation (TLS)
• Mapping TLS onto A Transactional Memory System
• Extending TM System
![Page 9: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/9.jpg)
9Helper Transactions: Yoo & Lee
TLS||TM Basics main(){
foo()
foo() code
foo2()
depth=1
depth=1
foo2() code
depth=2
depth=2Green light to commit
buffer
Green light to commit
depth=1depth=0
Exec
utio
n tim
elin
e
fallthru code
fallthru code
• Differ from conventional TM– transactions execute different code – Sequential order among transactions
![Page 10: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/10.jpg)
10Helper Transactions: Yoo & Lee
Procedure Fall-Thru Speculation on TM• Each task in TLS = a transaction
– Function body is guarded with begin_transaction and commit_transaction
– Spawned thread starts a transaction itself upon start – TM detects memory dependency violation
begin_transaction
Function body
commit_transaction
End thread
begin_transaction
commit_transaction(implicit)
Fall-through code
Trigger commit
function main() { … foo(); begin_transaction; … … ...}
function foo() { begin_transaction; … commit_transaction;}
foo()
Trigger commit
![Page 11: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/11.jpg)
11Helper Transactions: Yoo & Lee
Alternative Approach to Out-of-Order Spawn
begin_transactionlevel = 1
begin_transactionlevel = 2
commit_transactionlevel = 2
begin_transaction level = 2
End thread
commit_transaction level = 2 (implicit)
commit_transaction, level = 1
End thread
begin_transaction, level = 1
commit_transactionlevel = 1 (implicit)
Trigger commit
Trigger commit
• Spawn a new thread with function body– Reduces traffic used to
convert register dependencies into memory dependencies
– Simplifies compiler implementation
• Requires partial abort support from the TM system
Out-of-Order Procedure Fall-Through Speculation on TM (Revised)
![Page 12: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/12.jpg)
12Helper Transactions: Yoo & Lee
Agenda
• Thread-Level Speculation (TLS)
• Mapping TLS onto A Transactional Memory (TM) System
• Extending TM System
![Page 13: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/13.jpg)
13Helper Transactions: Yoo & Lee
Required Support• Compared to TLS, TM lacks
1. Thread spawning mechanism2. Context passing mechanism3. Sequential ordering mechanism
• Compiler support– Thread spawning mechanism– Context passing mechanism
• Hardware extension– Sequential ordering mechanism
![Page 14: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/14.jpg)
14Helper Transactions: Yoo & Lee
Compiler Support
int main ( int argc, char* argv[]){ int a, b, c; … foo( a, b, c); …}
int foo ( int arg0, int arg1, int arg2){ … // function foo body}
Volatile int in_memory_a, in_memory_b, in_memory_c;
int main ( int argc, char* argv[]){ int a, b, c; … in_memory_a = a; in_memory_b = b; in_memory_c = c;
create_thread( _tls_foo);
begin_transaction; …}
void* _tls_foo ( void* arg){ int arg0, arg1, arg2;
arg0 = in_memory_a; arg1 = in_memory_b; arg2 = in_memory_c;
begin_transaction; foo( arg0, arg1, arg2); commit_transaction;}
int foo ( int arg0, int arg1, int arg2){ … // function foo body}
Sample Code Before TLS Transformation
Sample Code After TLS Transformation
Encountering a function
call
Guard the function body
with transaction
Function call is replaced with a thread spawn to
the clone function
Fall-through thread is also
guarded with a transaction
Create a clone function whose
body is the function call
Register dependencies
should be changed into memory dependencies
Store the function call arguments in
memory…
…and retrieve them back
![Page 15: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/15.jpg)
15Helper Transactions: Yoo & Lee
Hardware Extension 1: Binary Tree to Encode the Sequential Ordering• Sequential order determines
1. Which transaction to abort upon conflict2. Which transaction should stall on commit
• Use binary tree to represent sequential ordering– Child_X executing “function body” appends 1 to its parent’s
encoding– Child_X executing “fall-thru code” appends 0 root
00 1001 11
0 1
foo()main() FT
goo()foo() FThoo()main() FTgoo()
foo()
X3X1
hoo()
X0X2
main()
X3X1X2X0
Sequential ordering
![Page 16: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/16.jpg)
16Helper Transactions: Yoo & Lee
Hardware Extension 1: Binary Tree to Encode the Sequential Ordering• Sequential order determines
1. Which transaction to abort upon conflict2. Which transaction should stall on commit
• Use binary tree to represent sequential ordering– Child_X executing “function body” appends 1 to its parent’s
encoding– Child_X executing “fall-thru code” appends 0 root
00 1001 11
0 1
foo()main() FT
goo()foo() FThoo()main() FTgoo()
foo()
X3X1
hoo()
X0X2
main()
X3X1X2X0
![Page 17: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/17.jpg)
17Helper Transactions: Yoo & Lee
Hardware Extension 2: Aborting a Subtree of Transactions• Upon a transaction abort
– More speculative transactions are all aborted– Conservatively abort a subtree of transactions
More Speculative,Abort the entire
subtree of transactions
root
00 01
0 1
conflict
![Page 18: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/18.jpg)
18Helper Transactions: Yoo & Lee
Hardware Extension 3: Ordering the Commits• A central module to serialize the commits
– Similar to ROB– Transaction consults it to determine commit or stall
0 1
00 01
root
Sequential order
0
0 1
1
Can I commit?
00
Encoding value
1
01
>
>>>>>>>
01
Query value
1
1
1
Module Generating Stall Signal
![Page 19: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/19.jpg)
19Helper Transactions: Yoo & Lee
Summary• TM can be extended to support out-of-order
TLS
• Two-fold approach– Compiler support for thread spawning and context
passing– Hardware support for sequential ordering
• Extend the usage scope of a TM system
• Amortize TLS implementation cost onto a TM-ready system
![Page 20: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/20.jpg)
Thank You!
Georgia Tech ECEMARS Labshttp://arch.ece.gatech.edu
![Page 21: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/21.jpg)
21Helper Transactions: Yoo & Lee
TLS versus TM• Thread-Level Speculation
– Divide a program into possibly non-conflicting tasks– Hardware speculate tasks to execute in parallel– Inter-task dependency maintained by detecting,
squashing and rolling back conflicting tasks• Transactional Memory
– Transaction• A sequence of instructions that executes in atomic fashion• These instructions either commit or abort as a single large
operation– Speculatively execute transactions within a critical
section– Underlying TM system detects and aborts
transactions that violate memory dependency
![Page 22: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/22.jpg)
BACKUP FOILS
![Page 23: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/23.jpg)
23Helper Transactions: Yoo & Lee
Helper Transactions• Goal: Enable TLS with a TM-ready system
– Support “out-of-order procedure fall-through speculation” on TM
– Amortize TLS implementation cost on a TM-ready system
Implementation Category Architecture Operand Passing
Network
Out-of-order Spawn
Lock Parallelization
Multiscalar TLS Dedicated Y
SVC TLS SMP Y
Hydra TLS CMP Y
PolyFlow TLS SMT Y
TLS4OutOrder TLS CMP Y
Voltron TLS CMP Y Y ?
Helper Transactions TM + TLS CMP Y Y
TCC TM CMP Y
UTM TM CMP Y
LogTM TM CMP YComparison of Various Parallelization Techniques
![Page 24: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/24.jpg)
24Helper Transactions: Yoo & Lee
The Basics (Cont’d)• Differ from conventional TM
– transactions execute different code – Sequential order among
transactions
• In this example, function_X sequentially precedes fallthru_X
1. When conflict, TM should abort fall-thru_X in favor of function_X
2. Upon commit_transaction, fallthru_X should be stalled until function_X commits.
OR
Commit of function_X implicitly triggers the commit of the fallthru_X (implicit commit)
begin_transaction
Function body
commit_transaction
End thread
begin_transaction
commit_transaction(implicit)
Fall-through code
Trigger commit
foo()
Procedure Fall-Thru Speculation on TM
Aborted transaction may improveperformance due to cache warm-up
function_Xfallthru_X
![Page 25: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/25.jpg)
25Helper Transactions: Yoo & Lee
Spawning Points of Thread Level Speculation• Task boundaries• Determined by high level programming language
– E.g., Loops, if-then-else statements, procedure fall-throughs, etc.
Non-speculative(loop iteration 0)
More-speculative(loop iteration 2)
Speculative(loop iteration 1)
Loop Speculation
Speculative(code after else)
Non-speculative(if-then-else)
If-then-else Speculation
Speculative(fall-through code)
Non-speculative(function body)
Procedure Fall-Thru Speculation
![Page 26: Enabling Thread Level Speculation via A Transactional Memory System](https://reader035.vdocuments.site/reader035/viewer/2022062520/56815c1c550346895dc9f18f/html5/thumbnails/26.jpg)
26Helper Transactions: Yoo & Lee
Supporting Out-of-Order Spawn
begin_transactionlevel = 1
begin_transactionlevel = 2
commit_transactionlevel = 2
begin_transaction level = 2
End thread
commit_transaction level = 2 (implicit)
commit_transaction, level = 1
End thread
begin_transaction, level = 1
commit_transactionlevel = 1 (implicit)
Trigger commit
Trigger commit
• Map out-of-order spawning to nested transactions– A transaction may have multiple
concurrent transactions– At spawn point, the spawning
thread increments its nesting level
– The spawned thread starts a transaction at the same level
• Maintain sequential order by1. Upon conflict, abort more
speculative transaction2. Stall the explicit commit of the
more speculative transaction until the less speculative transaction commitsOut-of-Order Procedure Fall-Thru
Speculation on TM