pipelining concepts, datapath and hazards · 2014-07-16 · pipelining hazards a hazard is a...
TRANSCRIPT
![Page 1: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/1.jpg)
Pipelining concepts, datapath
and hazards
Lecture 17
CDA 3103
07-16-2014
![Page 2: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/2.jpg)
Boolean Exprs for Controller
rtype = ~op5 ~op4 ~op3 ~op2 ~op1 ~op0,
ori = ~op5 ~op4 op3 op2 ~op1 op0lw = op5 ~op4 ~op3 ~op2 op1 op0sw = op5 ~op4 op3 ~op2 op1 op0beq = ~op5 ~op4 ~op3 op2 ~op1 ~op0jump = ~op5 ~op4 ~op3 ~op2 op1 ~op0
Instruction<31:0>
<21:2
5>
<16
:20
>
<11
:15
>
<0:1
5>
Imm16RdRsRt
Adr
InstMemory
Op
<0:5
>
Fun
<26
:31
>
Op 0-5 are really Instruction bits 26-31
Func 0-5 are really Instruction bits 0-5
How do we implement this in gates?
add = rtype func5 ~func4 ~func3 ~func2 ~func1 ~func0
sub = rtype func5 ~func4 ~func3 ~func2 func1 ~func0
Dr Dan Garcia
![Page 3: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/3.jpg)
Boolean Exprs for ControllerRegDst = add + sub
ALUSrc = ori + lw + sw
MemtoReg = lw
RegWrite = add + sub + ori + lw
MemWrite = sw
nPCsel = beq
Jump = jump
ExtOp = lw + sw
ALUctr[0] = sub + beq
ALUctr[1] = ori
(assume ALUctr is 00 ADD, 01 SUB, 10 OR)
How do we implement this in gates?
Dr Dan Garcia
![Page 4: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/4.jpg)
Controller Implementation
add
sub
ori
lw
sw
beq
jump
RegDst
ALUSrc
MemtoReg
RegWrite
MemWrite
nPCsel
Jump
ExtOp
ALUctr[0]
ALUctr[1]
“AND” logic “OR” logic
opcode func
Dr Dan Garcia
![Page 5: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/5.jpg)
lw $t0, 0($2)lw $t1, 4($2)sw $t1, 0($2)sw $t0, 4($2)
High Level Language Program (e.g., C)
Assembly Language Program (e.g.,MIPS)
Machine Language Program (MIPS)
Hardware Architecture Description (e.g., block diagrams)
Compiler
Assembler
Machine Interpretation
temp = v[k];
v[k] = v[k+1];
v[k+1] = temp;
0000 1001 1100 0110 1010 1111 0101 1000
1010 1111 0101 1000 0000 1001 1100 0110
1100 0110 1010 1111 0101 1000 0000 1001
0101 1000 0000 1001 1100 0110 1010 1111
Logic Circuit Description(Circuit Schematic Diagrams)
Architecture Implementation
Call home, we’ve made HW/SW contact!
Dr Dan Garcia
![Page 6: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/6.jpg)
Review: Single-cycle Processor
• Five steps to design a processor:1. Analyze instruction set
datapath requirements
2. Select set of datapathcomponents & establish clock methodology
3. Assemble datapath meeting the requirements
4. Analyze implementation of each instruction to determine setting of control points that effects the register transfer.
5. Assemble the control logic• Formulate Logic Equations
• Design Circuits
Control
Datapath
Memory
Processor
Input
Output
Dr Dan Garcia
![Page 7: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/7.jpg)
Agenda
• Pipelining Performance
• Structural Hazards
• Data Hazards
– Forwarding
– Load Delay Slot
• Control Hazards
Dr Dan Garcia
![Page 8: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/8.jpg)
Performance Issues
Longest delay determines clock period
Critical path: load instruction
Instruction memory register file ALU
data memory register file
Not feasible to vary period for different
instructions
Violates design principle
Making the common case fast
We will improve performance by pipelining
![Page 9: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/9.jpg)
Single Cycle Performance• Assume time for actions are
– 100ps for register read or write; 200ps for other events
• Clock rate is?
Instr Instr fetch Register
read
ALU op Memory
access
Register
write
Total time
lw 200ps 100 ps 200ps 200ps 100 ps 800ps
sw 200ps 100 ps 200ps 200ps 700ps
R-format 200ps 100 ps 200ps 100 ps 600ps
beq 200ps 100 ps 200ps 500ps
• What can we do to improve clock rate?
• Will this improve performance as well?Want increased clock rate to mean faster programs
Dr Dan Garcia
![Page 10: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/10.jpg)
Single Cycle Performance• Assume time for actions are
– 100ps for register read or write; 200ps for other events
• Clock rate is?
Instr Instr fetch Register
read
ALU op Memory
access
Register
write
Total time
lw 200ps 100 ps 200ps 200ps 100 ps 800ps
sw 200ps 100 ps 200ps 200ps 700ps
R-format 200ps 100 ps 200ps 100 ps 600ps
beq 200ps 100 ps 200ps 500ps
• What can we do to improve clock rate?
• Will this improve performance as well?Want increased clock rate to mean faster programs
Dr Dan Garcia
![Page 11: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/11.jpg)
Gotta Do Laundry• Ann, Brian, Cathy, Dave
each have one load of clothes to wash, dry, fold, and put away– Washer takes 30 minutes
– Dryer takes 30 minutes
– “Folder” takes 30 minutes
– “Stasher” takes 30 minutes to put clothes into drawers
A B C D
Dr Dan Garcia
![Page 12: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/12.jpg)
Sequential Laundry
• Sequential laundry takes 8 hours for 4 loads
T
a
s
k
O
r
d
e
r
B
C
D
A
30Time
3030 3030 30 3030 3030 3030 3030 3030
6 PM 7 8 9 10 11 12 1 2 AM
Dr Dan Garcia
![Page 13: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/13.jpg)
Pipelined Laundry
• Pipelined laundry takes 3.5 hours for 4 loads!
T
a
s
k
O
r
d
e
r
B
C
D
A
12 2 AM6 PM 7 8 9 10 11 1
Time303030 3030 30 30
Dr Dan Garcia
![Page 14: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/14.jpg)
• Pipelining doesn’t help latencyof single task, it helps throughput of entire workload
• Multiple tasks operating simultaneously using different resources
• Potential speedup = Number pipe stages
• Time to “fill” pipeline and time to “drain” it reduces speedup:2.3X v. 4X in this example
6 PM 7 8 9
Time
B
C
D
A
3030 30 3030 30 30
T
a
s
k
O
r
d
e
r
Pipelining Lessons (1/2)
Dr Dan Garcia
![Page 15: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/15.jpg)
• Suppose new Washer takes 20 minutes, new Stasher takes 20 minutes. How much faster is pipeline?
• Pipeline rate limited by slowest pipeline stage
• Unbalanced lengths of pipe stages reduces speedup
6 PM 7 8 9
Time
B
C
D
A
3030 30 3030 30 30
T
a
s
k
O
r
d
e
r
Pipelining Lessons (2/2)
Dr Dan Garcia
![Page 16: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/16.jpg)
1) IFtch: Instruction Fetch, Increment PC
2) Dcd: Instruction Decode, Read Registers
3) Exec:Mem-ref: Calculate AddressArith-log: Perform Operation
4) Mem: Load: Read Data from MemoryStore: Write Data to Memory
5) WB: Write Data Back to Register
Steps in Executing MIPS
Dr Dan Garcia
![Page 17: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/17.jpg)
PC
instr
uction
me
mo
ry
+4
rt
rs
rd
reg
iste
rs
ALU
Da
ta
me
mo
ry
imm
1. Instruction
Fetch2. Decode/
Register Read3. Execute 4. Memory
5. Write
Back
Single Cycle Datapath
Dr Dan Garcia
![Page 18: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/18.jpg)
PC
instr
uction
me
mo
ry
+4
rt
rs
rd
reg
iste
rs
ALU
Da
ta
me
mo
ry
imm
1. Instruction
Fetch2. Decode/
Register Read3. Execute 4. Memory
5. Write
Back
Pipeline registers
• Need registers between stages
– To hold information produced in previous cycle
Dr Dan Garcia
![Page 19: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/19.jpg)
More Detailed Pipeline
Dr Dan Garcia
![Page 20: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/20.jpg)
IF for Load, Store, …
Dr Dan Garcia
![Page 21: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/21.jpg)
ID for Load, Store, …
Dr Dan Garcia
![Page 22: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/22.jpg)
EX for Load
Dr Dan Garcia
![Page 23: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/23.jpg)
MEM for Load
Dr Dan Garcia
![Page 24: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/24.jpg)
WB for Load – Oops!
Wrong
register
number
![Page 25: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/25.jpg)
Corrected Datapath for Load
Dr Dan Garcia
![Page 26: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/26.jpg)
Pipelined Execution Representation
• Every instruction must take same number of steps, so some stages will idle– e.g. MEM stage for any arithmetic instruction
IF ID EX MEM WB
IF ID EX MEM WB
IF ID EX MEM WB
IF ID EX MEM WB
IF ID EX MEM WB
IF ID EX MEM WB
Time
Dr Dan Garcia
![Page 27: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/27.jpg)
Graphical Pipeline Diagrams
• Use datapath figure below to represent pipeline:IF ID EX Mem WB
ALUI$ Reg D$ Reg
1. InstructionFetch
2. Decode/Register Read
3. Execute 4. Memory 5. WriteBack
PC
inst
ruct
ion
mem
ory
+4
RegisterFilert
rsrd
ALU
Dat
am
emo
ry
imm
MU
X
Dr Dan Garcia
![Page 28: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/28.jpg)
Instr
Order
Load
Add
Store
Sub
Or
I$
Time (clock cycles)
I$
ALU
Reg
Reg
I$
D$
ALU
ALU
Reg
D$
Reg
I$
D$
Reg
ALU
Reg Reg
Reg
D$
Reg
D$
ALU
• RegFile: left half is write, right half is read
Reg
I$
Graphical Pipeline Representation
Dr Dan Garcia
![Page 29: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/29.jpg)
Pipelining Performance (1/3)
• Use Tc (“time between completion of instructions”) to measure speedup
–
– Equality only achieved if stages are balanced(i.e. take the same amount of time)
• If not balanced, speedup is reduced
• Speedup due to increased throughput
– Latency for each instruction does not decrease
Dr Dan Garcia
![Page 30: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/30.jpg)
Pipelining Performance (2/3)
• Assume time for stages is
– 100ps for register read or write
– 200ps for other stages
• What is pipelined clock rate?
– Compare pipelined datapath with single-cycle datapath
Instr Instr
fetch
Register
read
ALU op Memory
access
Register
write
Total
time
lw 200ps 100 ps 200ps 200ps 100 ps 800ps
sw 200ps 100 ps 200ps 200ps 700ps
R-format 200ps 100 ps 200ps 100 ps 600ps
beq 200ps 100 ps 200ps 500ps
Dr Dan Garcia
![Page 31: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/31.jpg)
Pipelining Performance (3/3)
Single-cycleTc = 800 ps
PipelinedTc = 200 ps
Dr Dan Garcia
![Page 32: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/32.jpg)
Pipelining Hazards
A hazard is a situation that prevents starting the next instruction in the next clock cycle
1) Structural hazard– A required resource is busy
(e.g. needed in multiple stages)
2) Data hazard– Data dependency between instructions– Need to wait for previous instruction to complete
its data read/write
3) Control hazard– Flow of execution depends on previous instruction
Dr Dan Garcia
![Page 33: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/33.jpg)
Agenda
• Pipelining Performance
• Structural Hazards
• Data Hazards
– Forwarding
– Load Delay Slot
• Control Hazards
Dr Dan Garcia
![Page 34: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/34.jpg)
1. Structural Hazards
• Conflict for use of a resource
• MIPS pipeline with a single memory?
– Load/Store requires memory access for data
– Instruction fetch would have to stall for that cycle
• Causes a pipeline “bubble”
• Hence, pipelined datapaths require separate instruction/data memories
– Separate L1 I$ and L1 D$ take care of this
Dr Dan Garcia
![Page 35: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/35.jpg)
I$
Load
Instr 1
Instr 2
Instr 3
Instr 4
ALUI$ Reg D$ Reg
ALUI$ Reg D$ Reg
ALUI$ Reg D$ Reg
ALUReg D$ Reg
ALUI$ Reg D$ Reg
Instr
Order
Time (clock cycles)
Structural Hazard #1: Single Memory
Trying to read same memory twice in same clock cycle
Dr Dan Garcia
![Page 36: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/36.jpg)
Structural Hazard #2: Registers (1/2)
I$
Load
Instr 1
Instr 2
Instr 3
Instr 4
ALUI$ Reg D$ Reg
ALUI$ Reg D$ Reg
ALUI$ Reg D$ Reg
ALUReg D$ Reg
ALUI$ Reg D$ Reg
Instr
Order
Time (clock cycles)
Can we read and write to registers simultaneously?
Dr Dan Garcia
![Page 37: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/37.jpg)
Structural Hazard #2: Registers (2/2)
• Two different solutions have been used:
1) Split RegFile access in two: Write during 1st half and Read during 2nd half of each clock cycle
• Possible because RegFile access is VERY fast (takes less than half the time of ALU stage)
2) Build RegFile with independent read and write ports
• Conclusion: Read and Write to registers during same clock cycle is okay
Dr Dan Garcia
![Page 38: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/38.jpg)
Agenda
• Pipelining Performance
• Structural Hazards
• Data Hazards
– Forwarding
– Load Delay Slot
• Control Hazards
Dr Dan Garcia
![Page 39: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/39.jpg)
2. Data Hazards (1/2)
• Consider the following sequence of instructions:
add $t0, $t1, $t2
sub $t4, $t0, $t3
and $t5, $t0, $t6
or $t7, $t0, $t8
xor $t9, $t0, $t10
Dr Dan Garcia
![Page 40: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/40.jpg)
2. Data Hazards (2/2)
• Data-flow backwards in time are hazards
sub $t4,$t0,$t3ALUI$ Reg D$ Reg
and $t5,$t0,$t6
ALUI$ Reg D$ Reg
or $t7,$t0,$t8 I$
ALUReg D$ Reg
xor $t9,$t0,$t10
ALUI$ Reg D$ Reg
add $t0,$t1,$t2
IF ID/RF EX MEM WBALUI$ Reg D$ Reg
Instr
Order
Time (clock cycles)
Dr Dan Garcia
![Page 41: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/41.jpg)
Data Hazard Solution: Forwarding
• Forward result as soon as it is available– OK that it’s not stored in RegFile yet
sub $t4,$t0,$t3ALUI$ Reg D$ Reg
and $t5,$t0,$t6
ALUI$ Reg D$ Reg
or $t7,$t0,$t8 I$
ALUReg D$ Reg
xor $t9,$t0,$t10
ALUI$ Reg D$ Reg
add $t0,$t1,$t2
IF ID/RF EX MEM WBALUI$ Reg D$ Reg
Dr Dan Garcia
![Page 42: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/42.jpg)
Datapath for Forwarding (1/2)
• What changes need to be made here?
Dr Dan Garcia
![Page 43: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/43.jpg)
Datapath for Forwarding (2/2)
• Handled by forwarding unit
Dr Dan Garcia
![Page 44: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/44.jpg)
Data Hazard: Loads (1/4)
• Recall: Dataflow backwards in time are hazards
• Can’t solve all cases with forwarding
– Must stall instruction dependent on load, then forward (more hardware)
sub $t3,$t0,$t2
ALUI$ Reg D$ Reg
lw $t0,0($t1)
IF ID/RF EX MEM WBALUI$ Reg D$ Reg
Dr Dan Garcia
![Page 45: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/45.jpg)
Data Hazard: Loads (2/4)
• Hardware stalls pipeline
– Called “hardware interlock”
sub $t3,$t0,$t2
ALUI$ Reg D$ Reg
bub
ble
and $t5,$t0,$t4
ALUI$ Reg D$ Regbub
ble
or $t7,$t0,$t6 I$
ALUReg D$
bub
ble
lw $t0, 0($t1)IF ID/RF EX MEM WBA
LUI$ Reg D$ Reg
Schematically, this is what we want, but in reality stalls done “horizontally”
How to stall just part of pipeline?
Dr Dan Garcia
![Page 46: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/46.jpg)
Data Hazard: Loads (3/4)
• Stall is equivalent to nop
sub $t3,$t0,$t2
and $t5,$t0,$t4
or $t7,$t0,$t6 I$
ALUReg D$
lw $t0, 0($t1)
ALUI$ Reg D$ Reg
bub
ble
bub
ble
bub
ble
bub
ble
bub
ble
ALUI$ Reg D$ Reg
ALUI$ Reg D$ Reg
nop
Dr Dan Garcia
![Page 47: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/47.jpg)
Data Hazard: Loads (4/4)
• Slot after a load is called a load delay slot
– If that instruction uses the result of the load, then the hardware interlock will stall it for one cycle
– Letting the hardware stall the instruction in the delay slot is equivalent to putting a nop in the slot (except the latter uses more code space)
• Idea: Let the compiler put an unrelated instruction in that slot no stall!
Dr Dan Garcia
![Page 48: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/48.jpg)
Code Scheduling to Avoid Stalls
• Reorder code to avoid use of load result in the next instruction!
• MIPS code for D=A+B; E=A+C;# Method 1:
lw $t1, 0($t0)
lw $t2, 4($t0)
add $t3, $t1, $t2
sw $t3, 12($t0)
lw $t4, 8($t0)
add $t5, $t1, $t4
sw $t5, 16($t0)
# Method 2:
lw $t1, 0($t0)
lw $t2, 4($t0)
lw $t4, 8($t0)
add $t3, $t1, $t2
sw $t3, 12($t0)
add $t5, $t1, $t4
sw $t5, 16($t0)
Stall!
Stall!
13 cycles 11 cyclesDr Dan Garcia
![Page 49: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/49.jpg)
Agenda
• More Pipelining
• Structural Hazards
• Data Hazards
– Forwarding
– Load Delay Slot
• Control Hazards
Dr Dan Garcia
![Page 50: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/50.jpg)
3. Control Hazards
• Branch (beq, bne) determines flow of control– Fetching next instruction depends on branch
outcome
– Pipeline can’t always fetch correct instruction• Still working on ID stage of branch
• Simple Solution: Stall on every branch until we have the new PC value– How long must we stall?
Dr Dan Garcia
![Page 51: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/51.jpg)
Branch Stall
• When is comparison result available?
I$
beq
Instr 1
Instr 2
Instr 3
Instr 4
ALUI$ Reg D$ Reg
ALUI$ Reg D$ Reg
ALUI$ Reg D$ Reg
ALUReg D$ Reg
ALUI$ Reg D$ Reg
Instr
Order
Time (clock cycles)
TWO bubbles required per branch!
Dr Dan Garcia
![Page 52: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/52.jpg)
Summary
• Hazards reduce effectiveness of pipelining
– Cause stalls/bubbles
• Structural Hazards
– Conflict in use of datapath component
• Data Hazards
– Need to wait for result of a previous instruction
• Control Hazards
– Address of next instruction uncertain/unknown
Dr Dan Garcia
![Page 53: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/53.jpg)
Question: For each code sequences below, choose one of the statements below:
No stalls as isA)
No stalls with forwardingB)
Must stallC)
1:lw $t0,0($t0)
add $t1,$t0,$t0
2:add $t1,$t0,$t0
addi $t2,$t0,5
addi $t4,$t1,5
3: addi $t1,$t0,1
addi $t2,$t0,2
addi $t3,$t0,2
addi $t3,$t0,4
addi $t5,$t1,5
Dr Dan Garcia
![Page 54: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/54.jpg)
Code Sequence 1
I$
lw
add
instr
instr
instrALUI$ Reg D$ Reg
ALUI$ Reg D$ Reg
ALUI$ Reg D$ Reg
ALUReg D$ Reg
ALUI$ Reg D$ Reg
Instr
Order
Time (clock cycles)
Must stall
Dr Dan Garcia
![Page 55: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/55.jpg)
Question: For each code sequences below, choose one of the statements below:
No stalls as isA)
No stalls with forwardingB)
Must stallC)
1:lw $t0,0($t0)
add $t1,$t0,$t0
2:add $t1,$t0,$t0
addi $t2,$t0,5
addi $t4,$t1,5
3: addi $t1,$t0,1
addi $t2,$t0,2
addi $t3,$t0,2
addi $t3,$t0,4
addi $t5,$t1,5
Dr Dan Garcia
![Page 56: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/56.jpg)
Code Sequence 2
I$
add
addi
addi
instr
instrALUI$ Reg D$ Reg
ALUI$ Reg D$ Reg
ALUI$ Reg D$ Reg
ALUReg D$ Reg
ALUI$ Reg D$ Reg
Instr
Order
Time (clock cycles)
forwarding
no forwarding
No stalls with forwarding
Dr Dan Garcia
![Page 57: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/57.jpg)
Question: For each code sequences below, choose one of the statements below:
No stalls as isA)
No stalls with forwardingB)
Must stallC)
1:lw $t0,0($t0)
add $t1,$t0,$t0
2:add $t1,$t0,$t0
addi $t2,$t0,5
addi $t4,$t1,5
3: addi $t1,$t0,1
addi $t2,$t0,2
addi $t3,$t0,2
addi $t3,$t0,4
addi $t5,$t1,5
Dr Dan Garcia
![Page 58: Pipelining concepts, datapath and hazards · 2014-07-16 · Pipelining Hazards A hazard is a situation that prevents starting the next instruction in the next clock cycle 1) Structural](https://reader033.vdocuments.site/reader033/viewer/2022060418/5f15d56caa46db2fbd5328d8/html5/thumbnails/58.jpg)
Code Sequence 3
I$
addi
addi
addi
addi
addiALUI$ Reg D$ Reg
ALUI$ Reg D$ Reg
ALUI$ Reg D$ Reg
ALUReg D$ Reg
ALUI$ Reg D$ Reg
Instr
Order
Time (clock cycles)
No stalls as is
Dr Dan Garcia