computer performance and cost
DESCRIPTION
Computer Performance and Cost. Pradondet Nilagupta Fall 2005 (original notes from Randy Katz, & Prof. Jan M. Rabaey , UC Berkeley Prof. Shaaban, RIT). IR. Regs. Review: What is Computer Architecture?. Interfaces. API. ISA. Link. I/O Chan. Technology. Machine Organization. - PowerPoint PPT PresentationTRANSCRIPT
April 22, 2023204521 Digital System Architecture
Computer Performance and Cost
Pradondet NilaguptaFall 2005
(original notes from Randy Katz, & Prof. Jan M. Rabaey , UC Berkeley
Prof. Shaaban, RIT)
April 22, 2023204521 Digital System Architecture 2
Review: What is Computer Architecture?
Technology
Applications Computer
Architect
Interfaces
Machine Organization
Measurement &Evaluation
ISA
AP
I
Link
I/O C
han
Regs
IR
April 22, 2023204521 Digital System Architecture 3
Review: Computer System ComponentsReview: Computer System Components
SDRAMPC100/PC133100-133MHZ64-128 bits wide2-way inteleaved~ 900 MBYTES/SEC )64bit)
Double DateRate (DDR) SDRAMPC3200200 MHZ DDR64-128 bits wide4-way interleaved~3.2 GBYTES/SEC (one 64bit channel)~6.4 GBYTES/SEC(two 64bit channels)
RAMbus DRAM (RDRAM)400MHZ DDR16 bits wide (32 banks)~ 1.6 GBYTES/SEC
CPU
CachesFront Side Bus (FSB)
I/O Devices:
MemoryControllers
adapters
DisksDisplaysKeyboards
Networks
NICs
I/O BusesMemoryController Example: PCI, 33-66MHz
32-64 bits wide 133-528 MBYTES/SEC PCI-X 133MHz 64 bit 1024 MBYTES/SEC
CPU Core1 GHz - 3.8 GHz4-way SuperscalerRISC or RISC-core (x86): Deep Instruction Pipelines Dynamic scheduling Multiple FP, integer FUs Dynamic branch prediction Hardware speculation
L1
L2 L3
Memory Bus
All Non-blocking cachesL1 16-128K 1-2 way set associative (on chip), separate or unifiedL2 256K- 2M 4-32 way set associative (on chip) unifiedL3 2-16M 8-32 way set associative (off or on chip) unified
Examples: Alpha, AMD K7: EV6, 200-400 MHz Intel PII, PIII: GTL+ 133 MHz Intel P4 800 MHz
NorthBridge
SouthBridge
Chipset
Off or On-chip
Current Standard
I/O Subsystem
April 22, 2023204521 Digital System Architecture 4
The Architecture Process
New concepts created
EstimateCost &
PerformanceSort
Good ideasMediocre
ideasBad ideas
April 22, 2023204521 Digital System Architecture 5
Performance Measurement and Evaluation
Many dimensions to computer performance
– CPU execution time• by instruction or sequence
– floating point– integer– branch performance
– Cache bandwidth– Main memory bandwidth– I/O performance
• bandwidth• seeks• pixels or polygons per
second
Relative importance depends on applications
P
C
M
April 22, 2023204521 Digital System Architecture 6
Evaluation Tools
Benchmarks, traces, & mixes– macrobenchmarks & suites
• application execution time– microbenchmarks
• measure one aspect of performance
– traces• replay recorded accesses
– cache, branch, registerSimulation at many levels
– ISA, cycle accurate, RTL, gate, circuit
• trade fidelity for simulation rateArea and delay estimationAnalysis
– e.g., queuing theory
MOVE 39%BR 20%LOAD 20%STORE 10%ALU 11%
LD 5EA3ST 31FF….LD 1EA2….
April 22, 2023204521 Digital System Architecture 7
BenchmarksMicrobenchmarks– measure one
performance dimension• cache bandwidth• main memory bandwidth• procedure call overhead• FP performance
– weighted combination of microbenchmark performance is a good predictor of application performance
– gives insight into the cause of performance bottlenecks
Macrobenchmarks– application execution
time• measures overall
performance, but on just one application
Perf. Dimensions
App
licat
ions
Mic
ro
Macro
April 22, 2023204521 Digital System Architecture 8
Some Warnings about Benchmarks
Benchmarks measure the whole system– application– compiler– operating system– architecture– implementation
Popular benchmarks typically reflect yesterday’s programs– computers need to be
designed for tomorrow’s programs
Benchmark timings often very sensitive to– alignment in cache– location of data on disk– values of data
Benchmarks can lead to inbreeding or positive feedback– if you make an operation
fast (slow) it will be used more (less) often
• so you make it faster (slower)
– and it gets used even more (less)
» and so on…
April 22, 2023204521 Digital System Architecture 9
Choosing Programs To Evaluate Choosing Programs To Evaluate PerformancePerformance
Levels of programs or benchmarks that could be used to evaluate performance:
– Actual Target Workload: Full applications that run on the target machine.
– Real Full Program-based Benchmarks: • Select a specific mix or suite of programs that are typical
of targeted applications or workload (e.g SPEC95, SPEC CPU2000).
– Small “Kernel” Benchmarks: • Key computationally-intensive pieces extracted from real
programs.– Examples: Matrix factorization, FFT, tree search, etc.
• Best used to test specific aspects of the machine.– Microbenchmarks:
• Small, specially written programs to isolate a specific aspect of performance characteristics: Processing: integer, floating point, local memory, input/output, etc.
April 22, 2023204521 Digital System Architecture 10
Types of BenchmarksTypes of Benchmarks
Actual Target Workload
Full Application Benchmarks
Small “Kernel” Benchmarks
Microbenchmarks
Pros Cons
• Representative• Very specific.• Non-portable.• Complex: Difficult to run, or measure.
• Portable.• Widely used.• Measurements useful in reality.
• Easy to run, early in the design cycle.
• Identify peak performance and potential bottlenecks.
• Less representative than actual workload.
• Easy to “fool” by designing hardware to run them well.
• Peak performance results may be a long way from real application performance
April 22, 2023204521 Digital System Architecture 11
SPEC: System Performance SPEC: System Performance Evaluation CooperativeEvaluation Cooperative
The most popular and industry-standard set of CPU benchmarks.
SPECmarks, 1989:– 10 programs yielding a single number (“SPECmarks”).
SPEC92, 1992:– SPECInt92 (6 integer programs) and SPECfp92 (14 floating point programs).
SPEC95, 1995:– SPECint95 (8 integer programs):
• go, m88ksim, gcc, compress, li, ijpeg, perl, vortex– SPECfp95 (10 floating-point intensive programs):
• tomcatv, swim, su2cor, hydro2d, mgrid, applu, turb3d, apsi, fppp, wave5
– Performance relative to a Sun SuperSpark I (50 MHz) which is given a score of SPECint95 = SPECfp95 = 1
SPEC CPU2000, 1999:– CINT2000 (11 integer programs). CFP2000 (14 floating-point intensive
programs)– Performance relative to a Sun Ultra5_10 (300 MHz) which is given a score of
SPECint2000 = SPECfp2000 = 100
April 22, 2023204521 Digital System Architecture 12
SPEC First RoundOne program: 99% of time in single line of codeNew front-end compiler could improve dramatically
Benchmark
SPEC
Per
f
0
100
200
300
400
500
600
700
800
gcc
epre
sso
spice
dodu
c
nasa
7 li
eqnt
ott
mat
rix30
0
fppp
p
tom
catv
April 22, 2023204521 Digital System Architecture 13
Impact of Means on SPECmark89 for IBM 550(without and with special compiler option)
Ratio to VAX: Time: Weighted Time:
Program Before After Before After Before Aftergcc 30 29 49 51 8.91 9.22espresso 35 34 65 67 7.64 7.86spice 47 47 510 510 5.69 5.69doduc 46 49 41 38 5.81 5.45nasa7 78 144 258 140 3.43 1.86li 34 34 183 183 7.86 7.86eqntott 40 40 28 28 6.68 6.68matrix300 78 730 58 6 3.43 0.37fpppp 90 87 34 35 2.97 3.07tomcatv 33 138 20 19 2.01 1.94Mean 54 72 124 108 54.42 49.99
Geometric Arithmetic Weighted Arith.
Ratio 1.33 Ratio 1.16 Ratio 1.09
April 22, 2023204521 Digital System Architecture 14
SPEC CPU2000 ProgramsSPEC CPU2000 ProgramsBenchmark Language Descriptions 164.gzip C Compression 175.vpr C FPGA Circuit Placement and Routing 176.gcc C C Programming Language Compiler 181.mcf C Combinatorial Optimization 186.crafty C Game Playing: Chess 197.parser C Word Processing 252.eon C++ Computer Visualization 253.perlbmk C PERL Programming Language 254.gap C Group Theory, Interpreter 255.vortex C Object-oriented Database 256.bzip2 C Compression 300.twolf C Place and Route Simulator
168.wupwise Fortran 77 Physics / Quantum Chromodynamics171.swim Fortran 77 Shallow Water Modeling 172.mgrid Fortran 77 Multi-grid Solver: 3D Potential Field 173.applu Fortran 77 Parabolic / Elliptic Partial Differential Equations177.mesa C 3-D Graphics Library 178.galgel Fortran 90 Computational Fluid Dynamics 179.art C Image Recognition / Neural Networks 183.equake C Seismic Wave Propagation Simulation 187.facerec Fortran 90 Image Processing: Face Recognition 188.ammp C Computational Chemistry 189.lucas Fortran 90 Number Theory / Primality Testing191.fma3d Fortran 90 Finite-element Crash Simulation 200.sixtrack Fortran 77 High Energy Nuclear Physics Accelerator Design301.apsi Fortran 77 Meteorology: Pollutant Distribution
CINT2000(Integer)
CFP2000(Floating Point)
Source: http://www.spec.org/osg/cpu2000/Programs application domain: Engineering and scientific computation
April 22, 2023204521 Digital System Architecture 15
Top 20 SPEC CPU2000 Results (As of March 2002)
# MHz Processor int peak int base MHz Processor fp peak fp base
1 1300 POWER4 814 790 1300 POWER4 1169 1098 2 2200 Pentium 4 811 790 1000 Alpha 21264C 960 776 3 2200 Pentium 4 Xeon 810 788 1050 UltraSPARC-III Cu 827 7014 1667 Athlon XP 724 697 2200 Pentium 4 Xeon 802 7795 1000 Alpha 21264C 679 621 2200 Pentium 4 801 7796 1400 Pentium III 664 648 833 Alpha 21264B 784 6437 1050 UltraSPARC-III Cu 610 537 800 Itanium 701 7018 1533 Athlon MP 609 587 833 Alpha 21264A 644 5719 750 PA-RISC 8700 604 568 1667 Athlon XP 642 59610 833 Alpha 21264B 571 497 750 PA-RISC 8700 581 52611 1400 Athlon 554 495 1533 Athlon MP 547 50412 833 Alpha 21264A 533 511 600 MIPS R14000 529 49913 600 MIPS R14000 500 483 675 SPARC64 GP 509 37114 675 SPARC64 GP 478 449 900 UltraSPARC-III 482 42715 900 UltraSPARC-III 467 438 1400 Athlon 458 42616 552 PA-RISC 8600 441 417 1400 Pentium III 456 43717 750 POWER RS64-IV 439 409 500 PA-RISC 8600 440 39718 700 Pentium III Xeon 438 431 450 POWER3-II 433 42619 800 Itanium 365 358 500 Alpha 21264 422 38320 400 MIPS R12000 353 328 400 MIPS R12000 407 382
Source: http://www.aceshardware.com/SPECmine/top.jsp
Top 20 SPECfp2000Top 20 SPECint2000
April 22, 2023204521 Digital System Architecture 16
Common Benchmarking Mistakes (1/2)
Only average behavior represented in test workloadSkewness of device demands ignoredLoading level controlled inappropriatelyCaching effects ignoredBuffer sizes not appropriateInaccuracies due to sampling ignored
April 22, 2023204521 Digital System Architecture 17
Common Benchmarking Mistakes (2/2)
Ignoring monitoring overheadNot validating measurementsNot ensuring same initial conditionsNot measuring transient (cold start) performanceUsing device utilizations for performance comparisonsCollecting too much data but doing too little analysis
April 22, 2023204521 Digital System Architecture
Architectural Performance Laws and Rules of Thumb
April 22, 2023204521 Digital System Architecture 19
Measurement and Evaluation
Architecture is an iterative process:– Searching the space of possible
designs– Make selections– Evaluate the selections made
Good measurement tools are required to accurately evaluate the selection.
April 22, 2023204521 Digital System Architecture 20
Measurement Tools
Benchmarks, Traces, MixesCost, delay, area, power estimationSimulation (many levels)– ISA, RTL, Gate, Circuit
Queuing TheoryRules of ThumbFundamental Laws
April 22, 2023204521 Digital System Architecture 21
Measuring and Reporting Performance
What do we mean by one Computer is faster than another?– program runs less time
Response time or execution time– time that users see the output
Elapsed time– A latency to complete a task including disk accesses,
memory accesses, I/O activities, operating system overhead
Throughput – total amount of work done in a given time
April 22, 2023204521 Digital System Architecture 22
Performance
“Increasing and decreasing” ?????
We use the term “improve performance” or “ improve execution time” When we mean increase performance and decrease execution time .
improve performance = increase performance improve execution time = decrease execution time
April 22, 2023204521 Digital System Architecture
Metrics of Performance
Compiler
Programming Language
Application
DatapathControl
Transistors Wires Pins
ISA
Function Units
(millions) of Instructions per second: MIPS(millions) of (FP) operations per second: MFLOP/s
Cycles per second (clock rate)
Megabytes per second
Answers per monthOperations per second
April 22, 2023204521 Digital System Architecture
Does Anybody Really Know What Time it is?
User CPU Time (Time spent in program)System CPU Time (Time spent in OS)Elapsed Time (Response Time = 159 Sec.)(90.7+12.9)/159 * 100 = 65%, % of lapsed time that is CPU time. 45% of the time spent in I/O or running other programs
UNIX Time Command: 90.7u 12.9s 2:39 65%
April 22, 2023204521 Digital System Architecture 25
ExampleUNIX time command90.7u 12.95 2:39 65%
user CPU time is 90.7 secsystem CPU time is 12.9 secelapsed time is 2 min 39 sec. (159 sec)% of elapsed time that is CPU time is 90.7 + 12.9 = 65% 159
April 22, 2023204521 Digital System Architecture 26
TimeCPU time
– time the CPU is computing– not including the time waiting for I/O or running
other programUser CPU time
– CPU time spent in the programSystem CPU time
– CPU time spent in the operating system performing task requested by the program decrease execution time
CPU time = User CPU time + System CPU time
April 22, 2023204521 Digital System Architecture 27
Performance
System Performance– elapsed time on unloaded system
CPU performance– user CPU time on an unloaded system
April 22, 2023204521 Digital System Architecture 28
Two notions of “performance”
° Time to do the task (Execution Time)– execution time, response time, latency
° Tasks per day, hour, week, sec, ns. .. – throughput, bandwidth
Response time and throughput often are in opposition
DC to Paris
6.5 hours
3 hours
Plane
Boeing 747
BAD/Sud Concorde
Speed
610 mph
1350 mph
Passengers
470
132
Throughput
286,700
178,200
Which has higher performance?
April 22, 2023204521 Digital System Architecture 29
Example
• Time of Concorde vs. Boeing 747?• Concord is 1350 mph / 610 mph = 2.2 times faster = 6.5 hours / 3 hours
• Throughput of Concorde vs. Boeing 747 ?• Concord is 178,200 pmph / 286,700 pmph = 0.62 “times faster”• Boeing is 286,700 pmph / 178,200 pmph = 1.6 “times faster”
• Boeing is 1.6 times (“60%”)faster in terms of throughput• Concord is 2.2 times (“120%”) faster in terms of flying time
We will focus primarily on execution time for a single job
April 22, 2023204521 Digital System Architecture 30
Computer Performance Measures: Computer Performance Measures: Program Execution Time Program Execution Time (1/2)(1/2)
For a specific program compiled to run on a specific machine (CPU) “A”, the following parameters are provided: – The total instruction count of the
program.– The average number of cycles per
instruction (average CPI).– Clock cycle of machine “A”
April 22, 2023204521 Digital System Architecture 31
Computer Performance Measures: Computer Performance Measures: Program Execution Time Program Execution Time (2/2)(2/2)
How can one measure the performance of this machine running this program?– Intuitively the machine is said to be faster or has better
performance running this program if the total execution time is shorter.
– Thus the inverse of the total measured program execution time is a possible performance measure or metric:
PerformanceA = 1 / Execution TimeA
How to compare performance of different machines?What factors affect performance? How to improve
performance?!!!!
April 22, 2023204521 Digital System Architecture 32
Comparing Computer Performance Using Execution Time
To compare the performance of two machines (or CPUs) “A”, “B” running a given specific program:
PerformanceA = 1 / Execution TimeA
PerformanceB = 1 / Execution TimeB
Machine A is n times faster than machine B means (or slower? if n < 1) :
Speedup = n = =PerformanceA
PerformanceB
Execution TimeB
Execution TimeA
April 22, 2023204521 Digital System Architecture 33
ExampleFor a given program:
Execution time on machine A: ExecutionA = 1 second
Execution time on machine B: ExecutionB = 10 seconds
The performance of machine A is 10 times the performance of machine B when running this program, or: Machine A is said to
be 10 times faster than machine B when running this program.
The two CPUs may target different ISAs providedthe program is written in a high level language (HLL)
101
10
ATimeExecution BTimeExecution
BePerformancAePerformanc
Speedup
April 22, 2023204521 Digital System Architecture 34
CPU Execution Time: The CPU Equation
A program is comprised of a number of instructions executed , I– Measured in: instructions/program
The average instruction executed takes a number of cycles per instruction (CPI) to be completed. – Measured in: cycles/instruction, CPI
CPU has a fixed clock cycle time C = 1/clock rate – Measured in: seconds/cycle
CPU execution time is the product of the above three parameters as follows:CPU time = Seconds = Instructions x Cycles x Seconds
Program Program Instruction Cycle
T = I x CPI x C execution Timeper program in seconds
Number of instructions executed
Average CPI for program CPU Clock Cycle
April 22, 2023204521 Digital System Architecture 35
CPU Execution Time: ExampleA Program is running on a specific machine with the following parameters:– Total executed instruction count: 10,000,000 instructions Average
CPI for the program: 2.5 cycles/instruction.– CPU clock rate: 200 MHz. (clock cycle = 5x10-9 seconds)
What is the execution time for this program:
CPU time = Instruction count x CPI x Clock cycle = 10,000,000 x 2.5 x 1 / clock
rate = 10,000,000 x 2.5 x 5x10-9
= .125 seconds
CPU time = Seconds = Instructions x Cycles x Seconds
Program Program Instruction Cycle
April 22, 2023204521 Digital System Architecture 36
Aspects of CPU Execution TimeCPU Time = Instruction count x CPI x Clock cycle
Instruction Count Instruction Count II
ClockClockCycleCycle CC
CPICPIDepends on:CPU OrganizationTechnology (VLSI)
Depends on:Program UsedCompilerISACPU Organization
Depends on:
Program UsedCompilerISA
(executed)
(AverageCPI)
T = I x CPI x C
April 22, 2023204521 Digital System Architecture 37
Factors Affecting CPU PerformanceFactors Affecting CPU PerformanceCPU time = Seconds = Instructions x Cycles x
Seconds Program Program Instruction Cycle
CPI Clock Cycle CInstruction Count I
Program
Compiler
Organization (CPU Design)
Technology (VLSI)
Instruction SetArchitecture (ISA)
X
X
X
X
X
X
X X
X
April 22, 2023204521 Digital System Architecture 38
Performance Comparison: Example
From the previous example: A Program is running on a specific machine with the following parameters:– Total executed instruction count, I: 10,000,000 instructions– Average CPI for the program: 2.5 cycles/instruction.– CPU clock rate: 200 MHz.
Using the same program with these changes: – A new compiler used: New instruction count 9,500,000 New CPI: 3.0– Faster CPU implementation: New clock rate = 300 MHZ
What is the speedup with the changes?
Speedup = (10,000,000 x 2.5 x 5x10-9) / (9,500,000 x 3 x 3.33x10-9 )
= .125 / .095 = 1.32 or 32 % faster after changes.
Speedup = Old Execution Time = Iold x CPIold x Clock cycleold
New Execution Time Inew x CPInew x Clock Cyclenew
April 22, 2023204521 Digital System Architecture 39
Instruction Types & CPIGiven a program with n types or classes of instructions executed on a given CPU with the following characteristics:
Ci = Count of instructions of typei
CPIi = Cycles per instruction for typei
Then:Then:
CPI = CPU Clock Cycles / Instruction Count I Where:
Instruction Count I = Ci
CPU clock cycles i ii
n
CPI C
1
i = 1, 2, …. n
April 22, 2023204521 Digital System Architecture 40
Instruction Types & CPI: An Example
An instruction set has three instruction classes:
Two code sequences have the following instruction counts:
CPU cycles for sequence 1 = 2 x 1 + 1 x 2 + 2 x 3 = 10 cycles CPI for sequence 1 = clock cycles / instruction count = 10 /5 = 2
CPU cycles for sequence 2 = 4 x 1 + 1 x 2 + 1 x 3 = 9 cycles CPI for sequence 2 = 9 / 6 = 1.5
Instruction class CPI A 1 B 2 C 3
Instruction counts for instruction classCode Sequence A B C 1 2 1 2 2 4 1 1
For a specific CPU design
April 22, 2023204521 Digital System Architecture 41
Instruction Frequency & CPIGiven a program with n types or classes of instructions with the following characteristics:Ci = Count of instructions of typei
CPIi = Average cycles per instruction of typei
Fi = Frequency or fraction of instruction typei executed
= Ci/ total executed instruction count = Ci/ I
Then:
n
iii FCPICPI
1
Fraction of total execution time for instructions of type i = CPIi x Fi
CPI
April 22, 2023204521 Digital System Architecture 42
Instruction Type Frequency & CPI: Instruction Type Frequency & CPI:
A RISC ExampleA RISC Example
Typical Mix
Base Machine (Reg / Reg)Op Freq, Fi CPIi CPIi x Fi % TimeALU 50% 1 .5 23% = .5/2.2Load 20% 5 1.0 45% = 1/2.2Store 10% 3 .3 14% = .3/2.2Branch 20% 2 .4 18% = .4/2.2
n
iii FCPICPI
1
CPIi x Fi
CPI
Sum = 2.2
Program Profile or Executed Instructions Mix
April 22, 2023204521 Digital System Architecture 43
Performance Terminology“X is n% faster than Y” means:ExTime(Y) Performance(X) n--------- = -------------- = 1 + -----ExTime(X) Performance(Y) 100
n = 100(Performance(X) - Performance(Y)) Performance(Y)
n = 100(ExTime(Y) - ExTime(X)) ExTime(X)
April 22, 2023204521 Digital System Architecture
Example
Example: Y takes 15 seconds to complete a task, X takes 10 seconds. What % faster is X?
n = 100(ExTime(Y) - ExTime(X)) ExTime(X)
n = 100(15 - 10) 10
n = 50%
April 22, 2023204521 Digital System Architecture 45
SpeedupSpeedup due to enhancement E: ExTime w/o E Performance w/ ESpeedup(E) = ------------- = ------------------- ExTime w/ E Performance w/o E
Suppose that enhancement E accelerates a fractionenhanced of
the task by a factor Speedupenhanced , and the remainder ofthe task is unaffected, then what is ExTime(E) = ?Speedup(E) = ?
April 22, 2023204521 Digital System Architecture 46
Amdahl’s Law
States that the performance improvement to be gained from using some faster mode of execution is limited by the fraction of the time faster mode can be used
Speedup = Performance for entire task using the enhancement Performance for the entire task without using the enhancement
or Speedup = Execution time without the enhancement Execution time for entire task using the enhancement
April 22, 2023204521 Digital System Architecture
Amdahl’s Law
ExTimenew = ExTimeold x (1 - Fractionenhanced) + Fractionenhanced
Speedupoverall =ExTimeold
ExTimenew
Speedupenhanced
=1
(1 - Fractionenhanced) + Fractionenhanced
Speedupenhanced
April 22, 2023204521 Digital System Architecture
Example of Amdahl’s Law
Floating point instructions improved to run 2X; but only 10% of actual instructions are FP
Speedupoverall =
ExTimenew =
April 22, 2023204521 Digital System Architecture
Example of Amdahl’s Law
Floating point instructions improved to run 2X; but only 10% of actual instructions are FP
Speedupoverall = 10.95
= 1.053
ExTimenew = ExTimeold x (0.9 + .1/2) = 0.95 x ExTimeold
April 22, 2023204521 Digital System Architecture 50
Performance Enhancement Performance Enhancement Calculations: Amdahl's LawCalculations: Amdahl's Law
The performance enhancement possible due to a given design improvement is limited by the amount that the improved feature is used Amdahl’s Law:
Performance improvement or speedup due to enhancement E:
Execution Time without E Performance with E Speedup(E) = -------------------------------------- = --------------------------------- Execution Time with E Performance without E
– Suppose that enhancement E accelerates a fraction F of the execution time by a factor S and the remainder of the time is unaffected then:
Execution Time with E = ((1-F) + F/S) X Execution Time without E Hence speedup is given by:
Execution Time without E 1Speedup(E) = --------------------------------------------------------- = -------------------- ((1 - F) + F/S) X Execution Time without E (1 - F) + F/S
April 22, 2023204521 Digital System Architecture 51
Pictorial Depiction of Amdahl’s LawPictorial Depiction of Amdahl’s Law
Before: Execution Time without enhancement E: (Before enhancement is applied)
After: Execution Time with enhancement E:
Enhancement E accelerates fraction F of original execution time by a factor of S
Unaffected fraction: (1- F) Affected fraction: F
Unaffected fraction: (1- F) F/S
Unchanged
Execution Time without enhancement E 1Speedup(E) = ------------------------------------------------------ = ------------------ Execution Time with enhancement E (1 - F) + F/S
• shown normalized to 1 = (1-F) + F =1
April 22, 2023204521 Digital System Architecture 52
Performance Enhancement ExamplePerformance Enhancement Example
For the RISC machine with the following instruction mix given earlier:Op Freq Cycles CPI(i) % TimeALU 50% 1 .5 23%Load 20% 5 1.0 45%Store 10% 3 .3 14%
Branch 20% 2 .4 18%If a CPU design enhancement improves the CPI of load instructions from 5 to 2, what is the resulting performance improvement from this enhancement:
Fraction enhanced = F = 45% or .45Unaffected fraction = 100% - 45% = 55% or .55Factor of enhancement = 5/2 = 2.5Using Amdahl’s Law: 1 1Speedup(E) = ------------------ = --------------------- = 1.37 (1 - F) + F/S .55 + .45/2.5
April 22, 2023204521 Digital System Architecture 53
An Alternative Solution Using CPU Equation
Op Freq Cycles CPI(i) % TimeALU 50% 1 .5 23%Load 20% 5 1.0 45%Store 10% 3 .3 14%
Branch 20% 2 .4 18%If a CPU design enhancement improves the CPI of load instructions from 5 to 2, what is the resulting performance improvement from this enhancement:
Old CPI = 2.2New CPI = .5 x 1 + .2 x 2 + .1 x 3 + .2 x 2 = 1.6
Original Execution Time Instruction count x old CPI x clock cycleSpeedup(E) = ----------------------------------- = ---------------------------------------------------------------- New Execution Time Instruction count x new CPI x clock cycle
old CPI 2.2= ------------ = --------- = 1.37
new CPI 1.6
Which is the same speedup obtained from Amdahl’s Law in the first solution.
April 22, 2023204521 Digital System Architecture 54
Performance Enhancement Example Performance Enhancement Example (1/2)(1/2)
A program runs in 100 seconds on a machine with multiply operations responsible for 80 seconds of this time. By how much must the speed of multiplication be improved to make the program four times faster?
100 Desired speedup = 4 = ----------------------------------------------------- Execution Time with enhancement
Execution time with enhancement = 25 seconds 25 seconds = (100 - 80 seconds) + 80 seconds / n 25 seconds = 20 seconds + 80 seconds / n 5 = 80 seconds / n n = 80/5 = 16
Hence multiplication should be 16 times faster to get a speedup of 4.
April 22, 2023204521 Digital System Architecture 55
Performance Enhancement Example (2/2)
For the previous example with a program running in 100 seconds on a machine with multiply operations responsible for 80 seconds of this time. By how much must the speed of multiplication be improved to make the program five times faster?
100Desired speedup = 5 = ----------------------------------------------------- Execution Time with enhancement
Execution time with enhancement = 20 seconds 20 seconds = (100 - 80 seconds) + 80 seconds / n 20 seconds = 20 seconds + 80 seconds / n 0 = 80 seconds / n
No amount of multiplication speed improvement can achieve this.
April 22, 2023204521 Digital System Architecture 56
Extending Amdahl's Law To Multiple Enhancements
Suppose that enhancement Ei accelerates a fraction Fi of the execution time by a factor Si and the remainder of the time is unaffected then:
i ii
ii X
SFF
SpeedupTime Execution Original)1
Time Execution Original
)((
i ii
ii S
FFSpeedup
)( )1
1
(
Note: All fractions Fi refer to original execution time before the enhancements are applied..
April 22, 2023204521 Digital System Architecture 57
Amdahl's Law With Multiple Enhancements: Example
Three CPU performance enhancements are proposed with the following speedups and percentage of the code execution time affected:
Speedup1 = S1 = 10 Percentage1 = F1 = 20% Speedup2 = S2 = 15 Percentage1 = F2 = 15% Speedup3 = S3 = 30 Percentage1 = F3 = 10%
While all three enhancements are in place in the new design, each enhancement affects a different portion of the code and only one enhancement can be used at a time.What is the resulting overall speedup?
Speedup = 1 / [(1 - .2 - .15 - .1) + .2/10 + .15/15 + .1/30)] = 1 / [ .55 + .0333 ] = 1 / .5833 = 1.71
i ii
ii S
FFSpeedup
)( )1
1
(
April 22, 2023204521 Digital System Architecture 58
Pictorial Depiction of ExampleBefore: Execution Time with no enhancements: 1
After: Execution Time with enhancements: .55 + .02 + .01 + .00333 = .5833
Speedup = 1 / .5833 = 1.71
Note: All fractions (Fi , i = 1, 2, 3) refer to original execution time.
Unaffected, fraction: .55
Unchanged
Unaffected, fraction: .55 F1 = .2 F2 = .15 F3 = .1 S1 = 10 S2 = 15 S3 = 30
/ 10 / 30/ 15
April 22, 2023204521 Digital System Architecture 61
Means
n
iiTn 1
1Arithmetic
mean
n
i iR
n
1
1
Geometric mean
n
i ri
inn
i ri
i
TT
nTT
1
1
1
log1exp
Harmonic mean
Consistent independent of reference
Can be weighted. Represents total execution time
April 22, 2023204521 Digital System Architecture 62
How to Summarize Performance (1/2)Arithmetic mean (weighted arithmetic mean) tracks execution time:
Harmonic mean (weighted harmonic mean) of rates (e.g., MFLOPS) tracks execution time:
ii
i TWornT
*
i
ii R
Wnor
R
n1
April 22, 2023204521 Digital System Architecture 63
How to Summarize Performance (2/2)Normalized execution time is handy for scaling performance (e.g., X times faster than SPARCstation 10)– Arithmetic mean impacted by choice of
reference machineUse the geometric mean for comparison:
– Independent of chosen machine– but not good metric for total execution time
niT
April 22, 2023204521 Digital System Architecture 64
Summarizing Performance
Arithmetic mean– Average execution time– Gives more weight to longer-running programs
Weighted arithmetic mean– More important programs can be emphasized– But what do we use as weights?– Different weight will make different machines look better
• (see Figure 1.16 in your text book – Hennessey/Patterson 3rd Edition)
• (see board for extension of previous example)
April 22, 2023204521 Digital System Architecture 65
Normalizing & the Geometric Mean
Normalized execution time– Take a reference machine R– Time to run A on X / Time to run A on R
Geometric mean
Neat property of the geometric mean:Consistent whatever the reference machineDo not use the arithmetic mean for normalized execution times
nn
i
i1
on timeexecution Normalized
April 22, 2023204521 Digital System Architecture
Which Machine is “Better”?
Computer A Computer B Computer CProgram P1(sec) 1 10 20Program P2 1000 100 20Total Time 1001 110 40
April 22, 2023204521 Digital System Architecture
Weighted Arithmetic Mean
Assume three weighting schemes
P1/P2 Comp A Comp B Comp C.5/.5 500.5 55.0 20.909/.091 91.82 18.8 20.999/.001 2 10.09 20
April 22, 2023204521 Digital System Architecture
Performance Evaluation
Given sales is a function of performance relative to the competition, big investment in improving product as reported by performance summaryGood products created then have:– Good benchmarks– Good ways to summarize performance
If benchmarks/summary inadequate, then choose between improving product for real programs vs. improving product to get more sales;Sales almost always wins!Execution time is the measure of performance
April 22, 2023204521 Digital System Architecture 69
Computer Performance Measures : MIPS Rating (1/3)For a specific program running on a specific CPU the MIPS rating is a measure of how many millions of instructions are executed per second:
MIPS Rating = Instruction count / (Execution Time x 106) = Instruction count / (CPU clocks x Cycle time x 106) = (Instruction count x Clock rate) / (Instruction
count x CPI x 106) = Clock rate / (CPI x 106)
Major problem with MIPS rating: As shown above the MIPS rating does not account for the count of instructions executed (I). – A higher MIPS rating in many cases may not mean higher
performance or better execution time. i.e. due to compiler design variations.
April 22, 2023204521 Digital System Architecture 70
In addition the MIPS rating:– Does not account for the instruction set architecture
(ISA) used.• Thus it cannot be used to compare computers/CPUs with
different instruction sets.
– Easy to abuse: Program used to get the MIPS rating is often omitted.
• Often the Peak MIPS rating is provided for a given CPU which is obtained using a program comprised entirely of instructions with the lowest CPI for the given CPU design which does not represent real programs.
Computer Performance Measures : MIPS Rating (2/3)
April 22, 2023204521 Digital System Architecture 71
Computer Performance Measures : MIPS Rating (3/3)
Under what conditions can the MIPS rating be used to compare performance of different CPUs?
The MIPS rating is only valid to compare the performance of different CPUs provided that the following conditions are satisfied:
1 The same program is used (actually this applies to all performance metrics)
2 The same ISA is used
3 The same compiler is used
(Thus the resulting programs used to run on the CPUs and obtain the MIPS rating are identical at the machine code level including the same instruction count)
April 22, 2023204521 Digital System Architecture 72
Compiler Variations, MIPS, Performance: An Example (1/2)
For the machine with instruction classes:
For a given program two compilers produced the following instruction counts:
The machine is assumed to run at a clock rate of 100 MHz
Instruction class CPI A 1 B 2 C 3
Instruction counts (in millions) for each instruction class Code from: A B C Compiler 1 5 1 1 Compiler 2 10 1 1
April 22, 2023204521 Digital System Architecture 73
Compiler Variations, MIPS, Performance: An Example (2/2)
MIPS = Clock rate / (CPI x 106) = 100 MHz / (CPI x 106) CPI = CPU execution cycles / Instructions count
CPU time = Instruction count x CPI / Clock rate
For compiler 1:– CPI1 = (5 x 1 + 1 x 2 + 1 x 3) / (5 + 1 + 1) = 10 / 7 = 1.43– MIP Rating1 = 100 / (1.428 x 106) = 70.0– CPU time1 = ((5 + 1 + 1) x 106 x 1.43) / (100 x 106) = 0.10 seconds
For compiler 2:– CPI2 = (10 x 1 + 1 x 2 + 1 x 3) / (10 + 1 + 1) = 15 / 12 = 1.25– MIPS Rating2 = 100 / (1.25 x 106) = 80.0– CPU time2 = ((10 + 1 + 1) x 106 x 1.25) / (100 x 106) = 0.15 seconds
CPU clock cycles i ii
n
CPI C
1
April 22, 2023204521 Digital System Architecture 74
Computer Performance Measures :MFLOPS (1/2)
A floating-point operation is an addition, subtraction, multiplication, or division operation applied to numbers represented by a single or a double precision floating-point representation.MFLOPS, for a specific program running on a specific computer, is a measure of millions of floating point-operation (megaflops) per second:
MFLOPS = Number of floating-point operations / (Execution time x 106 )MFLOPS rating is a better comparison measure between different machines (applies even if ISAs are different) than the MIPS rating.– Applicable even if ISAs are different
April 22, 2023204521 Digital System Architecture 75
Computer Performance Measures :MFLOPS (2/2)
Program-dependent: Different programs have different percentages of floating-point operations present. i.e compilers have no floating- point operations and yield a MFLOPS rating of zero.Dependent on the type of floating-point operations present in the program.– Peak MFLOPS rating for a CPU: Obtained using a program
comprised entirely of the simplest floating point instructions (with the lowest CPI) for the given CPU design which does not represent real floating point programs.
April 22, 2023204521 Digital System Architecture 76
Other ways to measure performance
(1) Use MIPS (millions of instructions/second)
MIPS is a rate of operations/unit time.
Performance can be specified as the inverse of execution time so faster machines have a higher MIPS rating
So, bigger MIPS = faster machine. Right?
MIPS = Exec. Time x 106
Instruction CountCPI x 106
Clock Rate=
April 22, 2023204521 Digital System Architecture 77
Wrong!!!3 significant problems with using MIPS:– Problem 1:
• MIPS is instruction set dependent.• (And different computer brands usually have different
instruction sets)
– Problem 2:• MIPS varies between programs on the same computer
– Problem 3:• MIPS can vary inversely to performance!
Let’s look at an example of why MIPS doesn’t work…
April 22, 2023204521 Digital System Architecture 78
A MIPS Example (1)Consider the following computer:
Code from: A B C
Compiler 1 5 1 1
Compiler 2 10 1 1
Instruction counts (in millions) for each instruction class
The machine runs at 100MHz.Instruction A requires 1 clock cycle, Instruction B requires 2 clock cycles, Instruction C requires 3 clock cycles.
CPIi x Ci
i =1
n
CPI = Instruction Count
CPU Clock Cycles
Instruction Count=!
Noteimportantformula!
April 22, 2023204521 Digital System Architecture 79
A MIPS Example (2)
CPI1 = (5 + 1 + 1) x 106
[(5x1) + (1x2) + (1x3)] x 10610/7 = 1.43=
MIPS1 = 1.43
100 MHz69.9=
CPI2 = (10 + 1 + 1) x 106
[(10x1) + (1x2) + (1x3)] x 10615/12 = 1.25=
MIPS2 = 1.25
100 MHz80.0=
So, compiler 2 has a higherMIPS rating and should befaster?
count cycles
cycles
April 22, 2023204521 Digital System Architecture 80
A MIPS Example (3)
Now let’s compare CPU time:
CPU Time = Clock Rate
Instruction Count x CPI
= 0.10 secondsCPU Time1 = 100 x 106
7 x 106 x 1.43
= 0.15 secondsCPU Time2 = 100 x 106
12 x 106 x 1.25
Therefore program 1 is faster despite a lower MIPS!
!Note
importantformula!