Energy-Conscious HW ISW -Partitioning of Embedded Systems:A Case Study on an MPEG-2 Encoder

Jorg Henkel

C&C Research LaboratoriesNEC USA, Princeton, NJ 08540

henkel @ccr1.nj.nec.com

AbstractEnergy dissipation is a hot topic in the design of-especiallymobile - embedded systems. This is because applicationslike digital video cameras, cellular phones etc. draw theircurrentfrom batteries that spend a limited amount of energyonly. In this paper we show that energy-conscious HW/SW-partitioning can lead to drastic reductions of energy dissi-pation of a whole embedded system. Subject of investigationis an MPEG-2 encoder. Therefore, we introduce ourframe-work for estimating and optimizing system energy as wellas all conducted design steps. The obtained results showenergy savings up 59% while the performance remains ap-proximately the same or becomes even slightly higher. Asa main result, energy-conscious HW/SW-partitioning is apromising method to be deployed in addition to classicalenergy and/or power reduction methods.

1 IntroductionReducing energy dissipation is a prominent tasks during thedesign phase of embedded systems. One of the manifoldreasons is that the market share of one class - the mobileembedded systems - is steadily increasing. A characteris-tic peculiarity of such a system is that it draws the currentfrom batteries that spend a limited amount of energy only.Hence, the design goal is to minimize the energy dissipa-tion in order to enable a longer operation period. Practicalexamples of mobile embedded systems are consumer prod-ucts like digital video cameras, cellular phones etc. Reduc-tion of energy dissipation increases their "mobility". Thissmall embedded systems come often as a systems-on-a-chipwhere components like processors, caches, application spe-cific hardware and even the main memory [1] reside on thesame piece of silicon. The according design process is oftenreferred to as core-based system design. In this context,a core can be an already optimized hardware (a hard coreor firm core) like a processor core or a more abstract de-scription (e.g. algorithm in C or VHDL) that still offers thepossibility for parameterizing, optimizing and even givesthe leeway to choose the form of implementation (e.g. soft-wareor hardware implementation). The latter is called a softcore. As a result, the designer has powerful design aids totailor an embedded system that exactly matches the requiredconstraints. Energy-conscious system design requires a so-phisticated adaption and tailoring of all system resources toprevent energy dissipations caused by under-utilized systemparts. As one major step, tailoring includes the selection of

1092-6100/98 $10.00 @ 1998 IEEE

[- --

Yanbing Li

Dept. of EE, Princeton UniversityPrinceton, NJ 08544

yanbing@ee.princeton.edu

an appropriate HW/SW-partitioning as well as fixing of pa-rameters like cache sizes, cache associativities, cache linesizes, main memory size etc. Core-based system designmakes these design goals possible.In this case study we investigate the power dissipation ofan MPEG-2 encoder for different HW/SW-partitions anddifferent system configurations like data cache size, instruc-tion cache size, cache line size cache associativity and mainmemory size. Therefore, we deploy our research frameworkin order to estimate and optimize energy dissipation for mostof the system parts. The remaining system parts are treatedby in-house production tools.This paper is structured as follows: the next section summa-rizes the research activities for estimating and minimizingenergy and/or power dissipation of various system parts andalso recent approaches in co-synthesis for low energy/power.In 3 the estimation models of our framework are presented.The MPEG-2 application as well as the performed designsteps and the partitioning considerations are outlined in 4whereas the obtained results are presented and discussed in5. Finally, section 6 gives a conclusion.

2 Related ResearchEstimating and optimizing energy/power dissipation hasbeen addressed from a software point of view! as well asfrom a hardware point of view (application specific hard-ware, memory hierarchy).Tiwari and Malik [2] investigated the energy that is dissi-pated during the execution of programs within a processor.They found out that the energy dissipation is sensitive tothe executed instruction. From these investigations theyderived specific optimization techniques to minimize thesoftware energy. They did not take into consideration theenergy dissipation of the memory hierarchy and its impacton the whole system's energy dissipation. Ong and Ynn[3] have shown that the energy dissipation may drasticallyvary depending on the algorithms running on a processor..Apower and performance simulation tool for a RISC designhas been developed by Sato et al. [4]. Their tool has beenused by designers to conduct architectural-level optimiza-tions.

1This means the power that is dissipated during the execution of a soft-

ware program on a programmable processor. Since the power dissipationvaries according to the executed instruction, the term software energy isjustified. We will use this term in the following.

23

.

"

Further work deals with energy/power dissipation from ahardwarepoint of view. Investigations of the major sourcesof power dissipation within a processor have been under-taken by Burd and Peters [5]. Gonzales and Horowitz [6]exploredthe power dissipation depending on the processorarchitecture(pipelined, un-pipelined, super-scalar).Other systemparts like caches have been analyzed by Kam-ble and Ghose [7]. A model for main memory power dissi-pation froma behavioral level point of view has been derivedby Itoh et al. [20]. A method for estimating power dissipa-tion of a synthesized hardware (ASIC) from a behavioral-level of abstraction has been proposed by Raghunathan andDey [8]. Optimizing energy dissipation by means of high-level transformations has been addressed by Potkonjak etal. [9], for example.As a conclusion, none of the described approaches offers acomprehensive solution i.e. an approach that takes into con-sideration the interdependency of all relevant system partsin order estimate/minimize energy/power dissipation of thecomplete system. This lack has been recognized by recentresearch activities in low-power co-synthesis [10, 11] evenif these approaches do not reflect all system resources (nocaches). Co-synthesis driven by other constraints (perfor-mance and in parts hardware effort) has been explored by[12, 13, 14, 15] among others.The case study presented in this paper quantifies the in-terdependency of various system parts in terms of energydissipation. Our framework estimates and optimizes sys-tem energy dissipation and is prominent for the conductedcase study.

II

3 Estimation ModelsA brief summary of the energy estimation models withinour framework is presented in this section. More details canbe retrieved from [16].3.1 Data and Instruction CacheThe models for data and instruction caches are generic:cache size, associativity and line size are the parameters.Modeled are the switching capacitances of the major sourcesfor energy dissipation (due to [7]): word-line capacitances,bit-line capacitances, decoder capacitances and capaci-tances of the output driver. The topology of the tag arrayand the decoder depends on the cache size and caches pa-rameters whereas the topology of the data array can directlybe derived from the cache size. Data and tag array are com-posed of 6-transistor SRAM cells. Our model captures thedynamic energy dissipation i.e. the energy dissipated dur-ing accesses to the caches. This is the final formula in itssimplified version:

Ecache1 22" . VDD . (Nacc . Cbit,r + Nacc . Cword

+kl . Cbit,w + k2 . Cdec + k3 . Cod) (1)

Chit r, Cword, Chit w, Cdec and Cod are the effective capac-itances for a bit-line read access, a word-line access, a bit-line write access, decoder and output driver, respectively.2Nacc gives the number of caches accesses and is obtainedby a cache simulator (see section3.3). Note, that the capac-itances are itself complex expressions composed of basic

2k1, k2 and kJ are expressions that contain in parts statistical accessnumbers.

- -- -

capacitances3. The basic capacitances have been obtainedthrough the tool Cacti [17], an analytical suite of models fora 0.8Jlm CMOS technology.

3.2 Software EnergyThe model is based on the techniques described in [2] butenhanced by the impact the number of cache miss reads(Nmiss r), cache miss writes (Nmiss w) and instruction fetchmisses (Nmiss f) have on the energy dissipation of the pro-cessor. It is modeled as follows:

Eprg

N-I

VDD . (TIOO%hit. L (Iinstr,i . Ncyc,i) +i=O

Tcyc . (Nmiss,r . Ncyc,r_penalty . Iinstr,nop +, v '

data read miss penalty

Nmiss,w . Ncyc,w...penalty . Iinstr,nop +, v '

data write miss penalty

Nmiss,j . Ncyc,j_penalty . Iinstr,nop)), v '

instruction fetch miss penalty

(2)

There it is, N the number of instructions of the program,Ncyc,i the number of cycles instruction i resides in theprocessor pipeline and Iinstr,i the current that is drawnduring the execution of i. Furthermore, Tcyc is the cy-cle time, NCJ/c,r_penalt!j'Ncyc,w...penalty, and Ncyc,j _penaltyare the number of cycles that are imposed through a cacheread/write/fetc» penalty, respectively. The drawn currentduring penalty cycles is given by I;nstr,nop. TIOO% hit isthe execution time of the program with an assumption of-a 100% cache hit rate.4 In the current state this model isimplemented for a 33MHz SPARCLite processor. The totalprogram execution time is modeled by:

Tprg TIOO%hit + Nmiss,r . Ncyc,r_penalty +

Nmiss,w . Ncyc,w_penalty +Nmiss,j . Ncyc,j _penalty (3)

3.3 Design Flow of our Framework

The input is a C program (Fig. 1) of the applications. Theupper branch comprises a behavior simulator [23] that isattached to our software energy and performance models(Eq. 2 and 3). The branch below contains the trace toolQPT and the Dinero III [19]cache simulator who feedssoftware, cache and main memory energy models with thenumbers of total cache accesses, data cache hits, data cachemisses and instruction fetch misses. Output are the numbersfor software energy dissipation and performance numbers.The feedback loop contains an optimization algorithm6 thatadapts cache sizes, associativities and cache line sizes inorder to reduce the total energy dissipation.

3Gate and drain capacitances, line capacitances etc.

4This is because the deployed behavior simulator primarily does notreflect the impact of caches. It assumes a 100% cache hit rate. So, theequation reflects the corrected model.

5i.e. only those parts that are implemented as software.6The implementation of the optim.loop is described in [16].

24

,

nedfor

butIdstchro- ia.

~

t

2)

Figure 1: Design flow of our framework for estimating andoptimizing system energy dissipation.

4 Approach and Design MethodologySubject of investigation for this case study is an MPEG-2encoder which is intended to run on thearchitecture shown inFig. 2. It features a CPU, instruction and data cache, a mainmemory and an application specifichardware (ASIC). In thefollowing sub-section the approach for minimizing/reducingsystem energy dissipation is described.

n,Ie'0r-

4.1 The ApproachIt should be noted that decreasing power dissipation can notbe achieved by reducing the performance since an MPEG-2 application is a real-time application that has to meetperformance constraints. Rather than that, the principalidea to reduce energy dissipation comprises the followingtwo steps:A. Energy-conscious HW/SW-partitioning

A general purpose processor? is not designed to handle effi-cientlyspecialapplications like an MPEG-2 encoder. As oneexample, the number of available FUs (Functional Units) islimited. As a consequence, the processor executes slowercompared to a hardware with multiple FUs. During thistime, other FUs might be un-utilized (e.g. multiplier in aapplication that features no multiplications) but still con-tribute to the total energy dissipation without increasing theperformance.8 Ontheother side, a hardware can be designedin such a way that the utilization of each FU is much higheras opposed to the processor i.e. "wasting of energy" can beminimized through removement of low or non-utilized FUs.Even if thepower dissipation of the hardware is higher, thereis stilI a hope that the dissipated energy is smaller since theinstruction-level parallelism leads to much shorter execu-tion times and hence, to a smaller energy dissipation. Thiseffect of energy saving is the larger the smaller the selectedapplication is because a smaller part can be better adaptedi.e. high utilization of each FU is much more likely.For our case study that means, we have to detect smallsystemparts that can efficiently (high utilization) be synthe-sized in hardware. All other system parts are still carried

yeItsf

7In the following we will use the terms "processor" when we mean"general purpose processor" and "hardware" when we mean an "applicationspecific hardware" i.e. an "ASIC".

8This is correct in cases where no gated clocks are used.

1--

CHIP

Figure 2: The target architecture.out by the processor.B. Optimizing the remaining system parametersBesides the processor and the hardware, the memory hier-archy is an additional major source for energy dissipation.In case of dis-configuration it can even dominate. Due tothe chosen size of data cache, instruction cache and theirparameters (associativity, line size etc.) the contribution ofeach system part to the total energy dissipation can varydrastically and hence, it is hard to predict which parametersand sizes lead to a minimum energy dissipation of the wholesystem.

"MPEG"

Figure 2a: Energy dissipation (left) and execution time(right) of the MPEG-2 Encoder for various data and in-struction cache sizes.As an illustration, Fig. 2a shows the energy dissipationin Joule (left) and execution time in cycles (right) of theMPEG-2 encoder for different data and instruction cachesizes9. This is part of the results of a pre-investigation(i.e. no hardware considered). Surprisingly, neither thelargest nor the smallest caches sizes lead to a minimumenergy dissipation. Additionally, not even the largestcachesizes lead to the highest performance (i.e. smallest numberof cycles). What actually happens is, demonstrates Eq. I:large caches sizes increase the switching capacitances, suchthat each cache access dissipates more energy. In such acase the caches dominate the total energy dissipation. Smallcache sizes lead to more data and instruction caches misses.Due to Eq. 2, the software energy increases. Apparently,this is a complex optimization problem. It is addressed byour framework.

4.2 The MPEG-2 Encoder

Figure 3 shows a part of the data-flow within the MPEG-2encoder. Compression for both, spatial and temporal (mo-tion) reduction of redundancy is based on a DCT (discretecosine transformation). Quantization (Q) is the process thatdetermines what information can be discarded withouta sig-nificant loss. Following the flow, Huffman coding is used

9Given in the figures is the value of the exponent of the cache sizes:a value of 12 for example means a cache size of 212 = 2048 bytes. The

results have been obtained by our framework.

25

to code the entropylO. Dequantization (Q-l)11 and inverseDCT (IDCT) are deployed as a part of generating the futureframe. For more details, see [21], for example. The basisfor our case study is a complete MPEG-2 software encoder(~ 200KB of C source code). The code has been exploredby techniques of profiling, behavior simulation, schedulingfor different resource constraints and energy investigationon the software implementation (see last section). Finallythree functions have been selected for a hardware imple-mentation: (dctO as a part of the DCT, idctcolO as a partof the the IDCT as well as distO as part of the quantizationprocess (Q).

Huffman Coder

Figure 3: The interesting parts of the MPEG-2 encoder.

4.3 DesignMethodologyFor each of the selected co-design configurations, the fol-lowing steps have been performed:

I. The candidate hardware part is synthesized and an-alyzed. This incorporates the following sub-steps(see also Fig. 4):a) The respective functionality is encoded in BDL

and synthesized by means of the high-level syn-thesis tool Cyber12.

b) The VHDL-RTL code output of Cyber is simu-lated with VSIM [22]. It delivers the number ofclock cycles for execution and allows to evaluatethe RTL code.

c) The RTL and Logic synthesis tool Varchsyn13isfed with the VHDL code. Technology mapping isdone by means ofNEC's CMOS6library. Outputof Varchsynis a netlist in PWC format as well asinformation about the total number of cells andthe maximum possible clock frequency.

d) Gate-level simulation is performed by CSIMI4.One operation mode ofCSIM deliver~ the energydissipation based on the switching activities. Forthat step, stimuli vectors have to be provided.

II. Our framework optimizes the data cache size, theinstruction cache size and the according associativ-ities and line sizes. It provides furthermore detailedinformation about the energy dissipation of the soft-ware, the main memory and the caches. Also, theperformance of the software part is provided.

III. Results of the previous steps are energy and perfor-mance data of the whole HW/SW-system as wellas system configuration parameters that ensure lowenergy dissipation.

:J<

lOEntropy is a measure for randomness and disorder.11The inverse of quantification.12Cyber is NEe's in-house high-level synthesis tool. BDL is the HDL

used in Cyber.13Yarchsyn is NEC's in-house RTL and Logic synthesis tool14NEC's in-house gate-level simulator.

-

r----

NEC'sCMOS6

Lib.

Switching Energy

Figure 4: Hardware synthesis steps

Table 1: Contribution of all system resources to total energydissipation.

S Results

¥Ie distinguish five different cases in the following discus-sion:

case 0: the all-software solution

case 1: distO is synthesized in hardware, rest is softwarecase 2: fdctO in hardWhr"e,rest is softwarecase 3: dctcolO in hardware, rest is softwarecase 4: distO and fdctO in hardware, rest is software

Our primary goal is to reduce the total energy dissipationrather than the power dissipation (see section 1). In practicethat means, even an (ASIC-) hardware featuring an aver-age power dissipation close to, for example, the processorspower dissipation, might be an appropriate candidate forreducing system energy in case the throughput is high dueto high utilization of internal resources (FUs). Furthermore,we assume that the clock of the whole processor and hard-ware is gated. So, whenever the processor is performingand the hardware is idle (they perform mutual exclusive),no energy will be dissipated by the hardware. In the com-plementary case, the processor behaves accordingly. Forpractical reasons (keeping simulation times of VHDL sim-ulation and behavioral simulation reasonable), all obtainednumbers refer to a sequence of six frames running throughthe MPEG-2 encoder. Each frame has a size of 64 x 64pixels. This configuration is appropriate to ensure that allparts of the encoder work almost like in the steady-state.The current energy model of our framework is valid for aSPARCLite processor running at 33MHz. The hardware isworking at 15MHz(delivered by logic synthesis through useof NEC's CMOS6 library) The memory parameters havebeen optimized for low energy dissipation by use of ourframework: instruction cache size and data cache size are2KB each, the associativity is 2 and the line size is 4 for bothcaches. The main memory size is 128KB. The main resultsare illustrated in Tab. 1: Each of the selected co-designsof cases 1 to 4 yields a energy reduction (up to 59%, seealso Fig. 5) compared to the reference case O. The average

26

~

~

Ths

~1t

---,

case Energy [mJ]i-cache d-cache mem sw hw total

0 44.79 17.98 2.305 74.32 nfa 140.92I 40.54 14.84 2.464 49.33 14.09 121.262 35.36 13.14 2.944 27.16 0.709 79.313 39.43 15.93 2.520 49.51 0.156 107.554 14.59 4.127 1.372 16.99 22.27 59.35

III

Table2: Execution time of SW and HW. Last two columns:hardwareeffort and average power dissipation of synthe-sizedhardware.

powerdissipation is shown in Tab. 2, last column. Accord-ing to Tab. 1, a larger reduction of the energy dissipationthrough an even larger hardware is hardly possible sincea great amount of energy is dissipated within the memoryhierarchy(e.g. 47.7 % in case 1).Of course, there is a simple way to reduce energy dissi-pation by simply reducing the performance. But our aimwasto keep the performance approximately the same whilereducing the energy. These results are shown in Tab. 2,columns2 and 4 (time) and columns 3 and 5 (cycles). Theresultsin terms of energy reduction and performance change(comparedto case 0) are shown in Fig. 5. Only in case 1a performancedecrease has occured. In all other cases, theperformanceis even moderately higher.Thedecrease in energy dissipation is due to the high oper-ator parallelism within the hardware as well as to the highdegree of utilization of each FU. To achieve these results,the systemparts to be implemented in hardware, have to beselectedthoroughly. In case, the presupposition describedin 4.1 are not fulfilled, a hardware will even increase theenergydissipation.Other restrictions are the hardware costs of an ASIC hard-ware. In ourcase the largest ASIC comprised a number 51kcells,the smallest less than 9k cells (Tab.2). Furthermore, itshouldbe noted that thepartitioning was done from a design-ers point of view - the functional level. Investigation of theschedule revealed that in parts of the synthesized functionsthe utilization rate of HW resources was much smaller thanthe average utilization within the same function. Appar-ently, an even better reduction of energy dissipation wouldhave been achieved through a finer-grain partitioning.

6 Conclusion

We have presented a case study for energy-consciousHW/SW-partitioning. Based on the idea that an applica-tion specific hardware can be much more energy efficientdue to higher resource utilization, energy savings up to 59%have been achieved comp;u-edto a pure software solution.Furthermore, it has been shown that estimating energy dis-sipation of a whole system is not a trivial task since thesources of energy dissipation are manifold and depend oneach other. This requires tools that can capture these inter-dependencies. Therfore, we presented our framework forestimatingand optimizing system energy dissipation.

7 AcknowledgmentWe would like to thank Bo Tao from Princeton Universitywho provided us with his version of the MPEG-2 encodersource code.

75

Case 4

Case 250

~ Case 3

-30

Figure 5: Energy reduction and performance change in %.

References[I] Y. Nunomura, T. Shimizu, O. Tomisawa, M32/D-lntegrating DRAM and Mi-

croprocessor. IEEE Micro Magazine, VoU7. No.6, pp.40-48, ]997.[2] V. Tiwari, S. Malik, A. Wolfe, Instruction Level Power Analysis and Opti.

mization of Software, Kluwer Academic Publishers, Journal of VLSI Signa]Processing,pp. ]-18, ]996.

[3] P.-W.Ong, R-H. Ynn, Power-Conscious Software Design - a framework formodeling software on hardware, IEEE Proc. of Symposium on Low PowerE]ectronics, pp. 36-37, ]994.

[4] T. Sato, M. Nagamatsu, H. Tago, Power and PeiformanceSimulator: ESP andits Applicationfor /00 MIPS/w Class RISC Design, IEEE Proc. of Symposiumon Low Power E]ectronics, pp. 46-47, ]994.

[5] T. Burd, B. Peters, A Power Analysis of a Microproce.<sor: A Study of anImplementation of the MIPS R3000 Architecture, Technica] Report, Universityof California at Berkeley, May 1994.

[6] R Gonzales, M. Horowitz, Energy Dissipation in General Purpose Processors,IEEE Proc. of Symposium on Low Power E]ectronics, pp. 12-13, ]995.

[7] M.B. Kamble, K. Ghose,Analytical Energy Dissipation Models For Low PowerCaches, IEEE Proc. of Symposium on Low Power Electronics and Design,pp. 143-]48, ]997.

[8] A. Raghunathan, S. Dey, N. Jha, estimationRegister-Transfer Level EstimationTechniquesfor Switching Activity and Power Consumption, IEEE Proc. of Int.Conf. on CAD (ICCAD96), pp.]58-]65, ]996.

[9] I. Hong, D. Kirovski, M. Potkonjak, Potential-Driven Statistical Ordering ofTransformations, IEEE Proc. of 34th. Design Automation Conference (DAC97),pp.347-352, ]997.

[10] D. Kirovski, M. potkonjak, System-Level Synthesis of Low-Power Hard Real-TIme Systems, IEEE Proc. of 34th. Design Automation Conference (DAC97),pp.697-702, ]997.

[] I] B.P. Dave, G. Lakshminarayana, N.K. Jha, COSYN: Hardware-Software Co-Synthesis of Embedded Systems' IEEE Proc. of 34th. Design Automation Con-ference (DAC97), pp.703- 708,1997.

[12] RK. Gupta, G.D. Micheli, System-level Synthesis using Re-programmableComponents, IEEE/ACM Proc. ofEDAC'92, pp. 2-7,1992.

[13] F.Vahid,D.D. Gajski, J. Gong, ABinary-Constraint SearchAlgorithmfor Min-imizing Hardware during Hardware/Software Partitioning, IEEE/ACM Proc.ofEuroDAC'94,pp. 214-2]9, ]994.

[]4] P.v. Knudsen, J. Madsen, PACE:A Dynamic Programming Algorithmfor Hard-ware/Software Partitioning, IEEE Proc. ofCodes/CASHE'96, pp.85-n, ]996.

[]5] J. Henkel, R Ernst, A Hardware/Software Partitioner using a dynamicallydetermined Granularity, IEEE Proc. of 34th. Design Automation Conference(DAC97), pp.69]-696, 1997.

[16] J. Henkel, Y. Li, An Approachfor Estimating and Minimizing Energy Dis-sipation of Embedded HW/SW Systems, NEC USA Inc., Technical Report #97-C07I-4-5] 1O-2,Oct.l997.

[17] SJ.E Wilton, N.P. Jouppi, An Enhanced Access and Cycle TIme Model forOn-Chip Caches, DEC, WRL Research Report 93/5, Ju]y ]994.

[]8] D.L. Weaver, T. Germond, The SPARC Architecture Manual, Version9, PTRPrenticeHall,1994.

[] 9] M. D. Hill, J. R Lauros, A. R Lebeck et a!., WARTS: WisconsinArchitecturalResearch Tool Set, Computer Science Department University of Wiscocnsin.

[20] K. Itoh, K. Sasaki and Y. Nakagome, Trends in Low-Power RAMCircuit Tech-nologies, Proceedings of the IEEE, VOL. 83, No.4, April 1995.

[2]] P.K. Andleigh, K. Thakrar,Multimedia Systems Design, Prentice Hall, ]996.[22] V.SystemIWorkstation Users Manual, VHDL Simulation for Workstations,

Model Techno]ogy, ] 995.[23] R Ernst, W. Ye, Embedded program timing analysis based on path clustering

and architectural classification, IEEE Proc. of Int. Cont. on CAD (ICCAD97),pp.598-604, 1997.

27

Software Hardware (ASIC)cs time # eyes time # eyes # cells

av.Pw.

[s] (33MHz) [ms] (15MHz) [mW]0 0.155 5,167,958 n/a nla nla n/a1 0.110 3,682,978 87.27 1,309,068 43,364 161.482 0.051 1,697,951 23.82 357,408 8,600 29.793 0.111 3,693,138 3.31 49,632 18,412 47.074 0.033 1,093,911 110.09 nla 51,298 202.28