performance and power benchmarks

Zhang Yu

A few years ago (like centuries in electronic world) frequency was considered the key element of a cpu`s performance, because of Intel`s dominance in the market place and their continuous bragging about frequency.

In November 2000 Intel's heavily advertised advances in clock speed reached an extreme with the release of the Pentium 4 which sacrificed per-cycle performance and used a deep instruction pipeline to gain higher clock speeds, ignoring problems that this introduced of heat production and power consumption.

Intel: “I am the fastest!” AMD, Apple etc. : “No you are not!”

They decided to settle this…. Let`s race!

Bunny is fast indeed but unfortunately prone to make mistakes….

Deep instruction pipeline (e.g. if a prediction is wrong

performance penalty will be greater because of deeper pipeline.)

higher clock speeds introduced heat production and power consumption

As computer architecture advanced, it became more difficult to compare the performance of various computer systems simply by looking at their specifications. Therefore, tests were developed that allowed comparison of different architectures

Benchmarks are particularly important in CPU design, giving processor architects the ability to measure and make tradeoffs in microarchitectural decisions

Benchmarks are designed to mimic a particular type of workload on a component or system

Industry Standard (audited and verifiable) Business Applications Performance

Corporation (BAPCo) Embedded Microprocessor Benchmark

Consortium (EEMBC) Standard Performance Evaluation

Corporation (SPEC) Transaction Processing Performance

Council (TPC)

BAPCo: MobileMark, SYSmark, WebMark Futuremark:3DMark, PCMark Whetstone PiFast Super PI WinSAT, exclusively for Windows Vista,

providing an index for consumers to rate their systems easily

Benchmarking is not easy and often involves several iterative rounds in order to arrive at predictable, useful conclusions.

Vendors tend to tune their products specifically for industry-standard benchmarks

Benchmarks do not address questions which you did not ask… only see what you measure

Specific application benchmarks will not tell you

about the performance of other applications without proper analysis => extrapolation is not straightforward

You must understand the benchmark itself to understand what it tells you

General benchmarks will not tell you about the details of your specific application

When more power is used, a portable system will have a shorter battery life and require recharging more often. This is often the antithesis of performance as most semiconductors require more power to switch faster

nVidia`s Geforce graphic card: The cooling fan on this card is so

powerful that it can be used to clean the door way.

In 2004 problems of overheating led Intel to abandon further development of its Netburst microarchitecture

The newest version of SPEC (SPECpower_ssj2008) counts power consumption as one of the key factor in performance.

overall ssj_ops/Watt Nowadays we often use Gflops/Watt as

an indicator of performance

http://en.wikipedia.org/wiki/Computer_benchmark

http://www.spec.org/power_ssj2008/ http://en.wikipedia.org/wiki/

Megahertz_myth http://en.wikipedia.org/wiki/

Performance_per_watt