© 2009 ibm corporation active memory sharing overview carol hernandez, power firmware architect
TRANSCRIPT
© 2009 IBM Corporation
Active Memory Sharing Overview
Carol Hernandez, Power Firmware Architect
© 2009 IBM Corporation
IBM Power Systems
2
Outline Technology Overview
– What is Active Memory Sharing
– Value Proposition
– Requirements
– Configuration
– Major Sub-Systems
Deployment Considerations
– Usage and Cost Savings
– Performance
– Methodology for Deployment
Memory Utilization Improvement Use Cases
– Time Zone Variant Workloads
– High Availability Scenario
– Physical Over-commitment
© 2009 IBM Corporation
IBM Power Systems
3
What is Active Memory SharingActive Memory Sharing intelligently flows memory from one partition to
another for increased utilization and flexibility of memory usage
Memory virtualization enhancement for Power Systems– Latest innovation in PowerVM virtualization: extends resource optimization to include memory– Can improve overall memory utilization similar to the way micro-partitioning improves CPU
utilization– A pool of physical memory is dynamically allocated amongst logical partitions as needed to
optimize overall memory usage in the pool
Blends Power Systems hardware, firmware and software enhancements to optimize resources
– Supports over-commitment of logical memory with overflow going to VIOS managed paging devices
– Two paging VIOS partitions can be used for redundancy– Compatible with Live Partition Mobility
More efficient utilization of memory through collaboration with Operating System
– Enables fine-grained sharing of physical memory and automated expansion and contraction of a partition’s physical memory footprint
– Supports OS collaborative memory management to reduce hypervisor paging
© 2009 IBM Corporation
IBM Power Systems
4
Active Memory Sharing Value Proposition
Dynamically adjusts memory available on a physical system for multiple virtual images based on their workload activity levels:
– Different workload peaks due to time zones
– Mixed workloads with different time of day peaks (e.g. CRM by day, batch at night)
– Ideal for highly-consolidated workloads with low or sporadic memory requirements
Increases memory utilization in an autonomic manner– Memory is automatically re-allocated between
participating partitions
•No user intervention required after set-up
– Save minutes - hours compared to manual DLPAR memory between partitions
0
5
10
15
Night
Day
Mem
ory
Usa
ge
(GB
)
Time
0
5
10
15
Asia
Americas
Europe
Mem
ory
Usa
ge
(GB
)
Time
Dynamically optimize memory across virtual images to improve memory utilization
0
5
10
15#10
#9
#8
#7
#6
#5
#4
#3
#2
#1 Time
Mem
ory
Usa
ge
(GB
)
Workloads
© 2009 IBM Corporation
IBM Power Systems
5
Active Memory Sharing Requirements
Available with PowerVM Enterprise Edition– No additional cost
System requirements:– IBM Power Systems server or blade with POWER6 processors– Virtual I/O Server (VIOS) 2.1.1– Firmware level: eFW 3.4.2– HMC v7.342
Operating systems supported:– AIX 6.1 TL3– IBM i 6.1 plus PTFs– SUSE Linux Enterprise Server 11
Partition Configuration Requirements– Must use shared processors only - dedicated processor is not supported– All I/O must be virtualized through VIOS – dedicated I/O, including HEA and HCA, is
not supported– 4K pages only – 64K or larger pages are not supported*
* Linux kernel emulates 64K pages
© 2009 IBM Corporation
IBM Power Systems
6
Active Memory Sharing Configuration
Dedicated Processor
LPARFinance
70 GB
LPAR#3
LPAR#1
LPARVIOS
LPAR#2
Dedicated Processor
LPARPlanning
105 GB 105 GB105 GB 5 GB60 GB
Power Hypervisor
Micro-Partition Processor Pool
AIXIBM iLinuxAIXAIX
MMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMM
MMMMMMMMMMMMMMMMMMMMM
Memory Pool: 210 GB
M
MMMMMMMMMMMMMM
MMMMMMMMM
MMM
Total Defined Memory450 GB
Physical Memory350 GB
Shared Memory Pool210 GB
350 GB Memory
M
AIX
Shared Memory PoolPaging Devices
Disk4
Disk3
Disk2
Disk1
Shared Memory Pool– Specify desired and maximum pool size– Assign paging devices and paging VIOS
• Single or Redundant Paging VIOSes
Shared Memory Partition– Partition Attributes
• Min, Max, Assigned Memory refer to logical memory.
• I/O Entitled Memory: maximum amount of physical memory available for I/O mapping.
• Memory Weight: partition’s priority to get physical pages
• Paging VIOSes: single or redundant; primary and secondary paging VIOS (optional)
– DLPAR memory operations change logical memory
– Partition Mobility support: among AMS capable systems
© 2009 IBM Corporation
IBM Power Systems
7
Active Memory Sharing Major Sub-Systems
SM Partition 1
AIX
Paging VIOS(1 GB)
Page In / Out
VASI
Virtualization Control Point (VCP)Shared Memory Pool:16 GBPaging VIOSPaging Devices
SM P1: Desired Mem= 12GBEntitled Mem: 4 GBSM P2: Desired Mem= 8GBEntitled Mem: 3 GBSM P3: Desired Mem= 4GBEntitled Mem: 1 GB
Shared Memory Pool
(16 GB)
Free Memory (5.5 GB)
Hypervisor Memory (1.5 GB)
Dedicated Memory (9 GB)
Physical Memory (32 GB)
FC
SM Partition 2
Linux
SM Partition 3
IBM i
Dedicated Memory
Partition 4 (8 GB)
AIX
Paging Devices
vSCSI Server CMMCMMCMM
Shared Memory Manager (SMM)
HypervisorHypervisor
Page Loaning
Virtualization Control Point User Interface– Create Shared Memory Pool
– Create Shared Memory Partitions
– Change Shared Memory Pool Configuration and partition attributes.
– Switch between dedicated and shared memory partitions.
– Profile I/O Entitled Memory Usage
Firmware and OS Interfaces– Paging VIOS Interface to manage paging devices and
allocate them to Shared Memory partitions.
– Hypervisor interface to create and manage Shared Memory Partitions.
– Client interface for DLPAR operations and dynamic partition attributes changes.
© 2009 IBM Corporation
IBM Power Systems
8
Active Memory Sharing Major Sub-Systems (cont.)
SM Partition 1
AIX
Paging VIOS(1 GB)
Page In / Out
VASI
Virtualization Control Point (VCP)Shared Memory Pool:16 GBPaging VIOSPaging Devices
SM P1: Desired Mem= 12GBEntitled Mem: 4 GBSM P2: Desired Mem= 8GBEntitled Mem: 3 GBSM P3: Desired Mem= 4GBEntitled Mem: 1 GB
Shared Memory Pool
(16 GB)
Free Memory (5.5 GB)
Hypervisor Memory (1.5 GB)
Dedicated Memory (9 GB)
Physical Memory (32 GB)
FC
SM Partition 2
Linux
SM Partition 3
IBM i
Dedicated Memory
Partition 4 (8 GB)
AIX
Paging Devices
vSCSI Server CMMCMMCMM
Shared Memory Manager (SMM)
HypervisorHypervisor
Page Loaning
Shared Memory Manager (SMM)– Guarantee physical memory is available for I/O ops.
– Manage and allocate physical memory in the pool among shared memory partitions, using:
• Page stealing based on OS page usage hints, memory weight, page usage statistics.
• Page loaning mechanism• Hypervisor Paging (when all else fails).
Paging VIOS Partition– Help SMM move partition page frames in and out of the
Shared Memory Pool to a paging device
• Page In/Out requests received through VASI stream
Operating System– Manage partition’s I/O entitlement across device drivers
and provides page usage hints to hypervisor.
– Dynamically change partition’s memory footprint in response to hypervisor page loaning requests (CMM: Collaborative Memory Manager).
© 2009 IBM Corporation
IBM Power Systems
9
Outline Technology Overview
– What is Active Memory Sharing
– Active Memory Sharing Value Proposition
– Active Memory Sharing Requirements
– Active Memory Sharing Major Sub-Systems
– Active Memory Sharing Configuration
Deployment Considerations
– Usage and Cost Savings
– Performance
– OS and VIOS
– Methodology for Deployment
Memory Utilization Improvement Use Cases
– Time Zone Variant Workloads
– High Availability Scenario
– Physical Over-commitment
© 2009 IBM Corporation
IBM Power Systems
10
Deployment Considerations: Usage and Cost Savings Usage
– AMS provides the most benefit when the aggregate memory working sets for all partitions running concurrently can be backed by the physical memory in the pool.
• Variable workloads that peak at different times across the partitions • Workloads with low average memory residency requirements• Active/Inactive Partition Scenarios
– AMS provides limited benefit and is not recommended for the following types of applications:
• Workloads with high, sustained memory residency requirements• Response time and performance sensitive workloads• Workloads with high degree of load variation
– To understand the benefits of AMS, customers should run test trials on the new AMS functions prior to deploying in a production environment
• White paper and LBS are available to assist customer with their set-up / optimization
Cost Savings– Reduction in real memory requirements may reduce cost of system configuration
depending on specific workloads and performance requirements • AMS allows creation of more partitions than would be otherwise possible • Only actively referenced memory needs to stay resident in a workload memory footprint
– AMS can save time and money of system admin who otherwise would be manually reallocating memory
© 2009 IBM Corporation
IBM Power Systems
11
Deployment Considerations: Performance
--
---
-
--
-
--
-
-4.0
2.0
0.04.0
3.0
2.04.0
2.0
0.03.5
3.0
2.0
1.51.8
3.4
3.1 3.2
3.7
2.1
3.4 3.3
00:00 00:30 01:00 01:30 02:00 02:30 03:00 03:30 04:00 04:30 05:00 05:30 06:00
Mem
ory
in (M
B)
Partition 1 (becoming active)
Partition 3 (inactive / app idle)
(-1.6GB)
(+1.6 GB)
Application running full speed.
Partition 4 (active) application running full speed
Partition 2 (active) application running full speed
Elapsed time (Minutes)
Partition 2 and 4 workload performance protected
Partition 3 workload idle but, memory not released by application
Partition 1 started, memory removed from partition 3 until performance full speed
– Performance depends on characteristics and usage model of the workloads that share the memory pool, memory configuration, and over subscription levels
– Switching latency may vary depending on utilization across the shared memory partitions, configured memory, and paging devices
• When a large amount of memory is moved, there will be a ramp-up latency at the destination partition
– When memory demand increases, the shared memory pool can be increased dynamically to avoid paging and improve performance
• Latency has to be monitored to initiate DLPAR memory add to the shared pool
– High performance paging devices are required to minimize performance impact
• Solid State Devices and FASTt are recommended
Example: Memory Bandwidth Workload
© 2009 IBM Corporation
IBM Power Systems
12
Baseline – Dedicated Memory Partition– Determine the memory capacity needed for the workloads per partition
Base Overhead - AMS with the same physical memory as dedicated memory scenario
– Shared Memory Pool will have physical memory to cover the memory capacity determined in baseline measurements
Logical Overcommit – Workloads peak at different times– Shared Memory Pool will have enough physical memory to cover the peaks at
different time periods– Frequent change of loads might impact latency, additional memory may have to be
added to the Shared Memory Pool to meet response time criteria
Physical Overcommit – Workloads peak concurrently – Shared Memory Pool cannot back up all the memory in use at a time– If performance is not within acceptable level, go back to Logical Overcommit
Methodology For Deployment
© 2009 IBM Corporation
IBM Power Systems
13
Outline Technology Overview
– What is Active Memory Sharing
– Active Memory Sharing Value Proposition
– Active Memory Sharing Requirements
– Active Memory Sharing Major Sub-Systems
– Active Memory Sharing Configuration
Deployment Considerations
– Usage and Cost Savings
– Performance
– OS and VIOS
– Methodology for Deployment
Memory Utilization Improvement Use Cases
– Time Zone Variant Workloads
– High Availability Scenario
– Physical Over-commitment
© 2009 IBM Corporation
IBM Power Systems
14
Active Memory Sharing: Time Zone Variant Workloads (No DLPAR)
0
8
16
24
32
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39
Time
Mem
ory U
sage
(GB)
Time Zone Variant Workloads Logical Overcommit (2.4x) Required Physical Memory
Dedicated Memory 3x16GB = 48 GB
Shared Memory 1x16GB + 2x2GB = 20 GB
System Memory (Dedicated Memory mode) 12 x 4GB DIMMs = 48 GB
System Memory (Shared Memory mode) 5 x 4GB DIMMs = 20 GB
Memory Utilization Improvement (48-20) GB / 48 GB = 58.3%
0
8
16
24
32
40
48
1 4 7 10 13 16 19 22 25 28 31 34 37 40
Time
Mem
ory
Usa
ge (G
B)
Asia
Europe
Americas
© 2009 IBM Corporation
IBM Power Systems
15
Active Memory Sharing: High Availability Scenario
HA - Logical Overcommit (2.5x) Required Physical Memory
Dedicated Memory 3x100GB = 300 GB
Shared Memory 10x10GB+20x1GB = 120 GB
System Memory (Dedicated Memory Mode) 10 x 32GB DIMMs = 320 GB
System Memory(Shared Memory Mode) 4 x 32GB DIMMs = 128 GB
Memory Utilization Improvement (320-128) GB / 320 GB = 60%
100 GB10 LPARs
120 GB 30 LPARs
100 GB10 LPARs
100 GB10 LPARs
300 GB30 LPARs
P1
P3
P2
P1
BU1
BU1’
© 2009 IBM Corporation
IBM Power Systems
16
0
5
10
15
20
25
30
35
40
1 3 5 7 9 11 13 15 17 19 21 23
Time
Mem
ory
Usa
ge (G
B)
Workload3
Workload2
Workload1
Active Memory Sharing: Physical Overcommitment
Physical Overcommit(1.25x) Required Physical Memory
Dedicated Memory 16GB+12GB+10GB = 40 GB
Shared Memory 40GB/1.25 = 32 GB
System Memory (Dedicated Memory Mode) 10 x 4GB DIMMs = 40 GB
System Memory (Shared Memory Mode) 8 x 4GB DIMMs = 32 GB
Memory Utilization Improvement (40-32) GB / 40 GB = 20%
024
68
1012
141618
1 3 5 7 9 11 13 15 17 19 21 23
Time
Mem
ory
Usag
e (G
B)
© 2009 IBM Corporation
IBM Power Systems
17
Questions?
Thank you Questions?
– Carol Hernandez– [email protected]