Download - Power5 Vio Server
2
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
This document was developed for IBM offerings in the United States as of the date of publication. IBM may not make these offerings available in other countries, and the information is subject to change without notice. Consult your local IBM business contact for information on the IBM offerings available in your area.Information in this document concerning non-IBM products was obtained from the suppliers of these products or other public sources. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive, Armonk, NY 10504-1785 USA. All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. The information contained in this document has not been submitted to any formal IBM test and is provided "AS IS" with no warranties or guarantees either expressed or implied.All examples cited or described in this document are presented as illustrations of the manner in which some IBM products can be used and the results that may be achieved. Actual environmental costs and performance characteristics will vary depending on individual client configurations and conditions.IBM Global Financing offerings are provided through IBM Credit Corporation in the United States and other IBM subsidiaries and divisions worldwide to qualified commercial and government clients. Rates are based on a client's credit rating, financing terms, offering type, equipment type and options, and may vary by country. Other restrictions may apply. Rates and offerings are subject to change, extension or withdrawal without notice.IBM is not responsible for printing errors in this document that result in pricing or information inaccuracies.All prices shown are IBM's United States suggested list prices and are subject to change without notice; reseller prices may vary.IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.Many of the pSeries features described in this document are operating system dependent and may not be available on Linux. For more information, please check: http://www.ibm.com/servers/eserver/pseries/linux/whitepapers/linux_pseries.htmlAny performance data contained in this document was determined in a controlled environment. Actual results may vary significantly and are dependent on many factors including system hardware configuration and software design and configuration. Some measurements quoted in this document may have been made on development-level systems. There is no guarantee these measurements will be the same on generally-available systems. Some measurements quoted in this document may have been estimated through extrapolation. Users of this document should verify the applicable data for their specific environment.
Special Notices
Revised February 6, 2004
3
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Special Notices (Cont.)The following terms are registered trademarks of International Business Machines Corporation in the United States and/or other countries: AIX, AIX/L, AIX/L(logo), alphaWorks, AS/400, Blue Gene, Blue Lightning, C Set++, CICS, CICS/6000, CT/2, DataHub, DataJoiner, DB2, DEEP BLUE, developerWorks, DFDSM, DirectTalk, DYNIX, DYNIX/ptx, e business(logo), e(logo)business, e(logo)server, Enterprise Storage Server, ESCON, FlashCopy, GDDM, IBM, IBM(logo), ibm.com, IBM TotalStorage Proven, IntelliStation, IQ-Link, LANStreamer, LoadLeveler, Lotus, Lotus Notes, Lotusphere, Magstar, MediaStreamer, Micro Channel, MQSeries, Net.Data, Netfinity, NetView, Network Station, Notes, NUMA-Q, Operating System/2, Operating System/400, OS/2, OS/390, OS/400, Parallel Sysplex, PartnerLink, PartnerWorld, POWERparallel, PowerPC, PowerPC(logo), Predictive Failure Analysis, pSeries, PTX, ptx/ADMIN, RISC System/6000, RS/6000, S/390, Scalable POWERparallel Systems, SecureWay, Sequent, ServerProven, SP1, SP2, SpaceBall, System/390, The Engines of e-business, THINK, ThinkPad, Tivoli, Tivoli(logo), Tivoli Management Environment, Tivoli Ready(logo), TME, TotalStorage, TURBOWAYS, VisualAge, WebSphere, xSeries, z/OS, zSeries.
The following terms are trademarks of International Business Machines Corporation in the United States and/or other countries: Advanced Micro-Partitioning, AIX/L(logo), AIX 5L, AIX PVMe, AS/400e, BladeCenter, Chipkill, Cloudscape, DB2 OLAP Server, DB2 Universal Database, DFDSM, DFSORT, Domino, e-business(logo), e-business on demand, eServer, GigaProcessor, HACMP, HACMP/6000, Hypervisor, i5/OS, IBMLink, IBM Virtualization Engine, IMS, Intelligent Miner, Micro-Partitioning, iSeries, NUMACenter, OpenPower, POWER, Power Architecture, Power Everywhere, PowerPC Architecture, PowerPC 603, PowerPC 603e, PowerPC 604, PowerPC 750, POWER2, POWER2 Architecture, POWER3, POWER4, POWER4+, POWER5, POWER5+, POWER6, Redbooks, Sequent (logo), SequentLINK, Server Advantage, ServeRAID, Service Director, SmoothStart, SP, S/390 Parallel Enterprise Server, ThinkVision, Tivoli Enterprise, TME 10, TotalStorage Proven, Ultramedia, VideoCharger, Visualization Data Explorer, X-Architecture, z/Architecture.
A full list of U.S. trademarks owned by IBM may be found at: http://www.ibm.com/legal/copytrade.shtml.
UNIX is a registered trademark in the United States, other countries or both.
Linux is a registered trademark of Linus Torvalds in the United States, other countries or both.
Microsoft, Windows, Windows NT and the Windows logo are registered trademarks of Microsoft Corporation in the United States and/or other countries.
Intel, Itanium and Pentium are registered trademarks and Intel Xeon and MMX are trademarks of Intel Corporation in the United States and/or other countries
AMD Opteron is a trademark of Advanced Micro Devices, Inc.
Java and all Java-based trademarks and logos are trademarks of Sun Microsystems, Inc. in the United States and/or other countries.
TPC-C and TPC-H are trademarks of the Transaction Performance Processing Council (TPPC).
SPECint, SPECfp, SPECjbb, SPECweb, SPECjAppServer, SPEC OMP, SPECviewperf, SPECapc, SPEChpc, SPECjvm, SPECmail, SPECimap and SPECsfs are trademarks of the Standard Performance Evaluation Corp (SPEC).
NetBench is a registered trademark of Ziff Davis Media in the United States, other countries or both.
Other company, product and service names may be trademarks or service marks of others. Revised August 23, 2004
4
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
The IBM benchmarks results shown herein were derived using particular, well configured, development-level and generally-available computer systems. Buyers should consult other sources of information to evaluate the performance of systems they are considering buying and should consider conducting application oriented testing. For additional information about the benchmarks, values and systems tested, contact your local IBM office or IBM authorized reseller or access the website of the benchmark consortium or benchmark vendor.
IBM benchmark results can be found in the IBM eServer p5, pSeries and IBM RS/6000 Performance Report at http://www-1.ibm.com/servers/eserver/pseries/hardware/system_perf.html
Unless otherwise indicated for a system, the performance benchmarks were conducted using AIX V4.3 or AIX 5L. IBM C Set++ for AIX and IBM XL FORTRAN for AIX with optimization were the compilers used in the benchmark tests. The preprocessors used in some benchmark tests include KAP 3.2 for FORTRAN and KAP/C 1.4.2 from Kuck & Associates and VAST-2 v4.01X8 from Pacific-Sierra Research. The preprocessors were purchased separately from these vendors. Other software packages like IBM ESSL for AIX and MASS for AIX were also used in some benchmarks.
For a definition and explanation of each benchmark and the full list of detailed results, visit the web site of the benchmark consortium or benchmark vendor.
TPC http://www.tpc.orgSPEC http://www.spec.orgLinpack http://www.netlib.org/benchmark/performance.pdfPro/E http://www.proe.comGPC http://www.spec.org/gpcNotesBench http://www.notesbench.orgVolanoMark http://www.volano.comSTREAM http://www.cs.virginia.edu/stream/SAP http://www.sap.com/benchmark/Oracle Applications http://www.oracle.com/apps_benchmark/PeopleSoft - To get information on PeopleSoft benchmarks, contact PeopleSoft directly Siebel http://www.siebel.com/crm/performance_benchmark/index.shtmBaan http://www.ssaglobal.comMicrosoft Exchange http://www.microsoft.com/exchange/evaluation/performance/default.aspVeritest http://www.veritest.com/clients/reportsFluent http://www.fluent.com/software/fluent/fl5bench/fullres.htmTOP500 Supercomputers http://www.top500.org/Ideas International http://www.microsoft.com/exchange/evaluation/performance/default.aspStorage Performance Council http://www.storageperformance.org/results
Notes on Benchmarks and Values
Revised August 26, 2003
5
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
rPerf–rPerf (Relative Performance) is an estimate of commercial processing performance relative to other pSeries systems. It is derived from an IBM analytical model which uses characteristics from IBM internal workloads, TPC and SPEC benchmarks. The rPerf model is not intended to represent any specific public benchmark results and should not be reasonably used in that way. The model simulates some of the system operations such as CPU, cache and memory. However, the model does not simulate disk or network I/O operations.
–rPerf estimates are calculated based on systems with the latest levels of AIX 5L and other pertinent software at the time of system announcement. Actual performance will vary based on application and configuration specifics. The IBM ~ pSeries 640 is the baseline reference system and has a value of 1.0. Although rPerf may be used to approximate relative IBM UNIX commercial processing performance, actual system performance may vary and is dependent upon many factors including system hardware configuration and software design and configuration.
–All performance estimates are provided "AS IS" and no warranties or guarantees are expressed or implied by IBM. Buyers should consult other sources of information, including system benchmarks, and application sizing guides to evaluate the performance of a system they are considering buying. For additional information about rPerf, contact your local IBM office or IBM authorized reseller.
Notes on Performance Estimates
Revised June 28, 2004
6
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
What is Advanced POWER Virtualization - APV
Hardware Feature code for Power5 server, which enables:
Micro-partitioning – use of processor shared pool, or fractional cpu
Partition Load Manager (PLM)
VIO ServerPhysical disk can be shared as virtual disks to client partitionsShared Ethernet Adapter (SEA) – Physical adapter or EtherChannel in VIO server can be shared by client partitions. Clients have virtual Ethernet Adapters
Virtual Ethernet – LPAR to LPAR within a Power5 Server, does not require APV feature code
7
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Why Virtual I/O Server
Driven by a large number of partitions being configurable in a micr-partitioning environment
POWER5 systems will support more partitions than i/o slots available Allow partitions to be created without physical slot restrictionsNo more the need to have the typical slot requirement of 1 NIC and 1 storage adapter
There are exceptional servers – 90% active all the time. However,Broad statistical average utilization ? 10-15% in a large UNIX server farmCan some of this hardware resource be shared? Be virtualized?
Optimized utilization of resourcesPartitions can be created without adding any additional hardware resourcesEfficient utilization of physical resources through sharing on the serverFacilitates server consolidation
8
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Virtual I/O Server characteristicsVirtual I/O server configured as a VIO Server lpar
AIX 5.3 requiredSupported only on Power5 Server
VIO server lpar is a single function appliance lparHost based – LVMStorage based – ESS, EMC etc
Based on a client/server modelVIO Server LPAR has physical diskClient LPAR sees standard scsi disks, accesses lun’s via virtual scsi adapter
Inter-partition communication provided via PHYP
Virtual disk in client LPAR may be disk or logical volume in VIO server
A physical disk in VIO server can provide virtual disks to several client LPARs
10
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Virtual I/O Server installation
Packaged and shipped as AIX mksysb image – on VIO CD
VIO Server installation methodsCD installHMC install - Open rshterm and type “installios”, follow promptsNIM Install now supportedhttp://publib.boulder.ibm.com/infocenter/eserver/v1r2s/en_US/index.htm?info/iphb1/iphb1_vios_configuring_installnim.htm
VIO Server can support multiple clientsAIX 5.3SUSE Linux Enterprise Server 9 for POWERRed Hat Enterprise Linux AS for POWER Version 3
11
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Virtual I/O Server Administration
Closed box only supporting encapsulated commands – padmin userpadmin id runs a restricted shell onlypadmin initial login, prompted for password changepadmin user runs “license –accept” commandCommand Line Interface for administrative functions
no smitty with padmin userdevice management, lan, install, security, users, maintenance
padmin can obtain AIX root access by oem_setup_env command
13
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
VIO Server Configuration with MPIO
MPIO in client LPAR automatically configures
Client sees one hdisk –with two MPIO pathslspath –l hdisk0
Paths are fail_over only. Noload balancing in client MPIO
hdisk1 in each VIO serverattached to vscsi serveradapter. Did not bring hdisk1into volume group in VIO
Set reserve_policy attributeon hdisk1 to no_reserve ineach VIO server
LUN appears in each VIOserver as hdisk1
Single RAID5 LUN carvedin ESS, made visible to onefibre channel adapter inboth VIO servers
14
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
VIO Server Configuration with LVM Mirroring
15
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Internals AIX
Virtual I/O Server AIX client
LVM
OEMmultipathing
disk DDHBA DD
VSCSI client
disk DD
LVM
MPIOPCM
MPIOPCM
VSCSIserverinternal or
external storage
logical volumes
OEM device paths
physical volumes
16
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Supported configurations today
PHYP
LVM
PHYSICALADAPTERDRIVER
AIXClient LPAR
AIX LVM
VSCSIclient
DISKDRIVER
Virtual I/O Server
ESS SAN STORAGE
PHYSICALADAPTERDRIVER
SDD-PCM for ESS
VSCSIclient
MPIO
LVM
PHYSICALADAPTERDRIVER
Virtual I/O Server
PHYSICALADAPTERDRIVER
SDD-PCM for ESS
ESS SAN STORAGEPHYP
LVM
PHYSICALADAPTERDRIVER
AIXClient LPAR
AIX LVM
VSCSIclient
DISKDRIVER
Virtual I/O Server
ESS SAN STORAGE
PHYSICALADAPTERDRIVER
SDD or SDD-PCM for ESS
ESS SAN Storage ConfigurationAIX Client uses MPIO to protect against Virtual I/O Server failures Virtual I/O Server uses SDD-PCM to protect against adapter failures
ESS SAN Storage ConfigurationVirtual I/O Server uses SDD, or SDD-PCM to protect against adapter failures
PHYP
LVM
PHYSICALADAPTERDRIVER
AIXClient LPAR
AIX LVM
VSCSIclient
DISKDRIVER
Virtual I/O Server
FASTT SAN STORAGE
PHYSICALADAPTERDRIVER
FASTT RDAC Driver
ESS SAN Storage ConfigurationVirtual I/O Server uses FASTT RDAC Driver to protect against adapter failures
Other configurations with EMC and HDS are being tested currentlyOther configurations with EMC and HDS are being tested currently
17
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Virtual IO Server resource configuration
Add Physical devices to VIO Server
Create a volume group on one or more disks with mkvgmkvg [-f] [-vg VolumeGroup] PhysicalVolume
mkvg –f –vg rootvg_clients hdisk2rootvg_clients
Create logical volumes on the volume group
mklv [-mirror] [-lv NewLogicalVolume | -prefix Prefix ] VolumeGroup Size [PhysicalVolume … ]mklv –lv aix_sq07 rootvg_clients 7G hdisk2
18
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Create Virtual SCSI adapter - server
Same panel whether you are creating VIO server for first time, or DLPAR addingvirtual scsi server adapter to running VIO server later.
19
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
VIO Server virtual adapters configuration
DLPAR added virtual adapter doesn’t show up in VIO server until cfgdev
lsdev –virtualname status description
ent2 Available Virtual I/O Ethernet Adapter (l-lan)vhost0 Available Virtual SCSI Server Adaptervhost1 Available Virtual SCSI Server Adaptervsa0 Available LPAR Virtual Serial Adapter
cfgdev –dev vio0
lsdev –virtualname status description
ent2 Available Virtual I/O Ethernet Adapter (l-lan)vhost0 Available Virtual SCSI Server Adaptervhost1 Available Virtual SCSI Server Adaptervhost2 Available Virtual SCSI Server Adaptervsa0 Available LPAR Virtual Serial Adapter
20
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Virtual IO server resource mapping
Configuring virtual target devicemkvdev –vdev aix_sq07 –vadapter vhost0 –dev vt_aix_sq07mkvdev –vdev hdisk7 –vadapter vhost1 –dev vt_hdisk7
$ lsdev -virtualname status description
ent2 Available Virtual I/O Ethernet Adapter (l-lan)vhost0 Available Virtual SCSI Server Adaptervhost1 Available Virtual SCSI Server Adaptervsa0 Available LPAR Virtual Serial Adaptervt_aix_sq07 Available Virtual Target Device - Logical Volumevt_hdisk7 Available Virtual Target Device - Disk
21
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
VIO resources map$ lsmap -allSVSA Physloc Client Partition ID--------------- -------------------------------------------- ------------------vhost0 U9111.520.10C1C1C-V3-C2 0x00000001
VTD vtscsi0LUN 0x8100000000000000Backing device aix_sq07Physloc
SVSA Physloc Client Partition ID--------------- -------------------------------------------- ------------------vhost1 U9111.520.10C1C1C-V3-C4 0x00000001
VTD vtscsi1LUN 0x8100000000000000Backing device hdisk7Physloc U787A.001.DNZ00ZE-P1-C1-T1-L4-L0
$
22
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
More use of lsmap$for v in `ioscli lsdev -virtual | grep vhost | awk '{print $1}'`doioscli lsmap -vadapter $v -fmt : | awk -F: '{ print $1" "$2" "$4"
"$6}'donevhost0 U9117.570.10C0EDC-V16-C7 virt_aix_sq07 aix_sq07vhost1 U9117.570.10C0EDC-V16-C8 virt_suse_sq07 suse_sq07vhost2 U9117.570.10C0EDC-V16-C9 virt_aix_sq08 aix_sq08vhost3 U9117.570.10C0EDC-V16-C10 virt_suse_sq08 suse_sq08vhost4 U9117.570.10C0EDC-V16-C11 virt_aix_sq09 aix_sq09vhost5 U9117.570.10C0EDC-V16-C12 virt_suse_sq09 suse_sq09vhost6 U9117.570.10C0EDC-V16-C13 virt_aix_sq10 aix_sq10vhost7 U9117.570.10C0EDC-V16-C14 virt_suse_sq10 suse_sq10vhost8 U9117.570.10C0EDC-V16-C15 virt_rhel_sq07 rhel_sq07vhost9 U9117.570.10C0EDC-V16-C16 virt_rhel_sq08 rhel_sq08vhost10 U9117.570.10C0EDC-V16-C17 virt_rhel_sq09 rhel_sq09vhost11 U9117.570.10C0EDC-V16-C18 virt_rhel_sq10 rhel_sq10
23
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Create Virtual SCSI adapter - client
Similar panel to create client virtual scsi adapter as the panel for server virtualscsi adapter. “This slot connects to which slot in which remote LPAR?” thinkplanning, think spreadsheet
24
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Client virtual disk attributes
[email protected] / # lsdev -Cc diskhdisk0 Available Virtual SCSI Disk Drive
[email protected] / # lscfg -vl hdisk0hdisk0 U9117.570.10C0EDC-V7-C5-T1-L810000000000 Virtual SCSI Disk Drive
[email protected] / # lsattr -El hdisk0PCM PCM/friend/vscsi Path Control Module Falsealgorithm fail_over Algorithm Falsemax_transfer 0x20000 Maximum TRANSFER Size Truepvid 00cc0edc916c5bd80000000000000000 Physical volume identifier Falsequeue_depth 3 Queue DEPTH Falsereserve_policy no_reserve Reserve Policy False
[email protected] / # lscfg -vl vscsi1vscsi1 U9117.570.10C0EDC-V7-C6-T1 Virtual SCSI Client Adapter
Device Specific.(YL)........U9117.570.10C0EDC-V7-C6-T1
25
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Virtual Ethernet
Virtual EthernetEnable inter-lpar communications without a physical adapterIEEE-compliant ethernet programming modelImplemented through inter-partition in-memory communication
VLAN splits up groups of network users on a physical network onto segments of logical networks
Virtual switch provides support for multiple (up to 4K) VLANsEach partition can connect to multiple networks, through one or more adaptersVIO server can add VLAN ID tag to the ethernet frame as appropriate. Ethernet switch restricts frames to ports that are authorized to receive frames with specific VLAN ID
Virtual network can connect to physical network through "routing" partitionsipforwarding, routing at IP layer – this is not Virtual Ethernet switch
27
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Configure shared ethernet adapter - SEA
28
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
SEA configuration$ lsdev | grep ent[0-9]ent0 Available Virtual I/O Ethernet Adapter (l-lan)ent1 Available 10/100 Mbps Ethernet PCI Adapter II (1410ff01)
$ mkvdev –sea ent1 –vadapter ent0 –default ent0 –defaultid 1ent2 Available
$ lsdev | grep ent[0-9]ent0 Available Virtual I/O Ethernet Adapter (l-lan)ent1 Available 10/100 Mbps Ethernet PCI Adapter II (1410ff01)ent2 Available Shared Ethernet Adapter
$ lsattr -El ent2pvid 1 PVID to use for the SEA device Truepvid_adapter ent0 Default virtual adapter to use for non-VLAN-tagged packets Truereal_adapter ent1 Physical adapter associated with the SEA Truethread 0 Thread mode enabled (1) or disabled (0) Truevirt_adapters ent0 List of virtual adapters associated with the SEA (comma separated) True
29
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Shared Ethernet Adapter setup
VIO Server after mksysb InstallPhysicalEthernet
VirtualEthernet
VIO Server after sea config
mkvdev –sea ent1 –vadapter ent0 –default ent0 –defaultid 1ent2 Available
If VIO server requires a localIP address on this adapter con-figuration, the address is placedon shared adapter interfaceen2 (smitty chinet as root). It isnot configured on physical, noron virtual adapter.
ent1 ent0
9.19.126.98Shared Ethernet ent2
ent0ent1
30
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Supported configurations today
Virtual I/O Server
AIX Client LPAR
PHYP
Shared Ethernet Adapter
Virtual Ethernet
VirtualEthernet
Interface Failover
Virtual I/O Server
Shared Ethernet Adapter
Client LPAR: Interface Failover protects against Virtual I/O Server failures Restriction: VLAN Tags cannot be used
31
IBM Systems Group
POWER5 VIO Server © 2004 IBM Corporation
Reference
InfoCenterhttp://publib.boulder.ibm.com/infocenter/eserver/v1r2s/en_US/index.htm
Virtualizing your compute environmenthttp://publib.boulder.ibm.com/infocenter/eserver/v1r2s/en_US/info/iphb1/iphb2.pdf
VIO Server and PLM command line referencehttp://publib.boulder.ibm.com/infocenter/eserver/v1r2s/en_US/info/iphb1/commands/commands.pdf
Redbook Introduction to Advanced POWER Virtualization on IBM p5 Servershttp://www.redbooks.ibm.com search on SG24-7940http://www.redbooks.ibm.com/redpieces/pdfs/sg247940.pdf
IBM eServer P5 Architecture and Performance Considerationshttp://publib-b.boulder.ibm.com/redpieces/abstracts/sg245768.html