ibm power vc integration with openstack for sap hana

© Copyright IBM Corporation 2016. Technical University/Symposia materials may not be reproduced in whole or in part without the prior written permission of IBM.

IBM Power VC Integration with Openstack for SAP HANA Deployments /

Live SAP HANA Workload Movement Using LPM Including Storage

Hemantha GunasingheIT SpecialistIBM Australia – STG LAB Services

1IBM Systems Technical Events | ibm.com/training/events

© Copyright IBM Corporation 2016. Technical University/Symposia materials

may not be reproduced in whole or in part without the prior written permission of

IBM.

Session objectives(1/2)

IBM Power VC Integration with Openstack for SAP HANA VM Deployments

• Introduction

• Scope of Provisioning SAP HANA ready SUSE VMs

• VM Provisioning Environment and Setup

• IBM Power VC Resources

• Python Scripts

• Lessons Learned




IBM.


Live SAP HANA Workload Movement Using LPM Including Storage From One Physical System to Another

• Introduction

• Scope of Moving VM’s while SAP HANA workloads are Active

• Power System Configuration and Setup

• Moving SUSE VMs Using LPM (Excluding Storage)

• Manually Moving Storage

• Lessons Learned




IBM.

IBM Power VC Integration with Openstack for SAP HANA Deployments

Introduction

• Fully automated provisioning of computing resources and storage

• Fully customized SUSE Linux with Best Practices – Built Within Minutes

• SAP HANA SUSE VMs ready for HANA Installs

• Save time and effort in deploying SAP HANA

• Use of Openstack based deployment technology – Cinder, Neutron and Nova

• Easy re-use of compute capacity and storage

• Efficient Image Capturing and re-deployment

• End-to-end deployment of compute and storage ready for SAP HANA VM’s

• Images are secure, tuned and customized to SAP HANA Requirements

• Pre-installed all custom management and monitoring tools as a SOE image

• VM’s are customized based on SAP HANA specification and IBM TDI solution

• IBM Power VC 1.3.1 used with Openstack

• Storage fully configured as required by SAP HANA Database specification

• IBM Best Practices




IBM.

Scope of Provisioning SAP HANA VMs (1/2)

• Provisioning Compute Capacity

• Creation of POWER8 LPAR Profile

• Allocation of CPU Cores

• Allocation of Memory as Required

• Creation of Virtual Adapters for NPIV Network and Storage

• Setup of Virtualization Settings

• Generation of Virtual NPIV WWNs

• Provisioning Storage

• Creation of Storage LUNs based on the size of Memory

• Mapping of Storage LUNs to VM on Storage Subsystems

• Automated SAN switch Zoning as needed to Map LUNs to VM

CPU, Memory and

Adapters

LUNs and SANZoning




IBM.

Scope of Provisioning SAP HANA VMs (2/2)

• Provisioning OSI’s

• Installation of SUSE 11 SP4 as per SAP specification

• Application of Customer Security Policy

• Setting all SAP Specified OS Tuning

• Discovery of Storage Devices

• Configuration of Network Setup

• Setup of NTP and DNS Settings

• Setup of Multipath Configuration/Settings

• File System and Kernel Tuning

• Provisioning HANA File Systems

• Format LUNs for HANA LOGs, DATA and Shared

• Create LVM2 Volume Groups, Logical Volumes and File systems

• Configure to Mount File Systems Automatically

• Create and Place all HANA file systems on appropriate Disk types such as SSD for LOGs

• Placement of File systems as per Storage Best practices

SUSE11 SP4

XFS




IBM.

Hardware Configuration

• IBM Power8 - Compute:• 4 x E870 POWER8 Systems ( 2 per site) each

having;

• 80 x P8 Processor Cores

• 8TB of Memory

• 8 x 16Gbps HBA Ports

• 8 x 10Gbps Network Adapters

• 2 x HMCs per Site

• IBM Storage:• 4 x V7000 Enterprise Storage Subsystems -

Collocated with each E870 system rack

• 8 x Brocade SAN Switches – Two Collocated per E870)

• IBM Power VC:• 2 x RHEL OSIs with Power VC installed (Separate

to Power8 Systems) – one in each site




IBM.

Virtualization Setup – VIO Servers

• 4 x VIO Servers per E870 POWER8 Systems

• 2 x Production VIO Servers

• 2 x Non-Production VIO Servers

• 2 x vNICs per VM

• 1 x Data and Administration Network

• 1 x HSR Network

• 4x vFC adapters per VM

• Used for OS, Data and Logs




IBM.

Logical Construct of Provisioning Environment




IBM.

IBM Power VC and Openstack Integration




IBM.

IBM Power VC – Host, Storage and Network Providers

•




IBM.

IBM Power VC – Hardware Management Consoles(HMCs)

•




IBM.

IBM Power VC – Host Capacity Available for VM provisioning

•




IBM.

IBM Power VC – Compute Templates

•




IBM.

IBM Power VC – SLES11SP4 SOE Images

•




IBM.

IBM Power VC – VLAN/Network Resources

•




IBM.

IBM Power VC – SAN Fabric and Switches

•




IBM.

IBM Power VC – Storage Templates

•




IBM.

IBM Power VC – V7000 Storage Pools

•




IBM.

IBM Power VC – Storage Connectivity Groups

•




IBM.

Provisioning Scripts (1/2)

There are two main scripts that are used to provision VMs and to re-organize storage

once VMs are provisioned. In addition to these, there is an additional configuration text file

that provide details of the VM to be provisioned. This file is used by python scripts.

• Validate and Check configuration settings

• Authenticate and acquire PowerVC access tokens

• Create VM Profiles and Allocate resources on HMC

• Create and Map Storage for OS by Cloning in V7K

• Create and Map Storage for Hana Data, Shared and Logs from V7K

• Create SAN zoning on FC SAN Switches

• Startup VM using HMC

• Execute Cloudinit Customization

SAP HANA VM and Storage Provisioning – “build_hana_vm “




IBM.

Provisioning Scripts (2/2)

Once the SAP HANA VM is provisioned by cloning at storage layer, following post install

activities are performed by an embedded bash script passed to VM during the deployment

process.

• Format Storage LUNs

• Create HANA Volume Groups

• Create Logical Volumes

• Create HANA File systems

• Mount HANA Data, Shared and Log File systems

• Update “/etc/fstab” for automatic mounts

• Apply any SAP or OSI Tuning/Configuration settings

• Identify SAP HANA Related LOG LUNs

• Move LOG Related LUNs on to SSD Storage

Post Provisioning Customization – “cust_sap.sh”

Storage Re-organization – “move_ssd.sh”




IBM.

HANA VM Build Specification Configuration File

The Build is based on following parameters that need to be defined based on HANA VM Requirements

Name : <testlpar>ip_address : 192.168.1.1vlan : VLAN_100

target_host : 9119MME_99X99999

image : SLES11SP4_SOE_1

storage_connectivity_group : SCG_NPROD

storage_template : NPROD_1

virtual_processor_min : 1virtual_processor : 5virtual_processor_max : 8entitled_capacity_min : 1.0entitled_capacity : 5.0entitled_capacity_max : 8.0memory_min : 1024memory : 409600memory_max : 819200Data_lun_prefix : dataData_lun_size : 256Data_lun_count : 8Log_lun_prefix : logLog_lun_size : 128Log_lun_count : 4




IBM.

How to Run Build and Provisioning Script and Syntax

Run the build and provisioning script with parameters as needed as per below

Syntax:

./build_hana_vm –f <file> -p <Power VC Server> -u <User> -P <Password>

Parameters:• -f <file> : Build sheet populated with parameters• -p <Power VC Server> : Power VC Server IP or Hostname• -u <User> : Power VC Administrative User with deployment privileges• -P <Password> : Password for the <user>




IBM.

VM Deployment Checks and Validations

The Build sheet input values are validated as the one of the first steps

Checks and Validations:

• Valid Range for all numerical input values

• Validation of number of LUNs and size required for Hana Data

• Validation of number of LUNs and size for Hana Shared space

• Validation of number and size Hana Log space

• Hostname Validation




IBM.

Cloud-Init and Post Customization Script

After cloning OSI by the Deployment Process, first boot customisation takes

place with a boot process automatically. During this boot process, the cloud-initcode will perform following tasks.

First Boot and Customization Tasks

• Set IP addresses based on DNS resolution

(Hostname must exist on DNS prior to build)

• Cleanup RSCT/RMC subsystem with generating

new unique IDs

• Register with HMC for Dynamic Operations

• Set Hostname based on Provided name during

provisioning

• Discover all storage LUNs using “multipathd”

• Download post-customization script into VM

• Execute the post Customization Script

Cludinit is Responsible




IBM.

How to run Storage Re-Organization Script and Syntax

Script to move storage from SAS to SSD as per SAP HANA recommendations for

LUG Volumes

Syntax:

./move_ssd –s <lpar_hostname> -p <PowerVC Server> -u <Provisioning Admin ID> -P <Password>

Parameters:• -s <LPAR Hostname>• -p <Power VC Server>• -u <Provisioning user ID>• -P <Password for Provisioning User>




IBM.

Storage Re-Organization

Based on SAP Recommendations “move_ssd.sh” script will move all HANA Log

LUNs to SSD storage. In addition the same script will move OS Image to SSD.

Storage Re-Organization

• “move_ssd” script Identify

SAP Log LUNs on Non-SSD

disks

• Using ssh connections to

V7K CLI, All LOG Volumes

on SAS disks are moved to

SSD “mdisk” group within

respective V7K storage

subsystem

• OS Image will also be moved

to SSD “mdisk” group from

SAS disks

“move_ssd” script is responsible




IBM.

Post Install Customization Script – “cust_sap.sh”

The “cust_sap.sh” script is passed as a base64 encoded script during

provisioning of VM to the server that is built. The Cloudinit code will execute this script after the first boot of a VM until removed from cloud-init.

Syntax:

cust_sap.sh

Functions:• function logger {}

• function gen_mp_cfg {}

• function fix_fqdn {}

• function update_hostfile {}

• function restart_network {}

• function reconnect_to_rmc {}

• # Create VGs, LVs and XFS File Systems

• # Upate /etc/fstab

• # Create Mount point directories for new File Systems

• # Disable cloudinit on subsequent reboots

• # reboot




IBM.

PowerVC REST API Calls using Openstack

Following are some of the key IBM Power VC REST API Calls that are used in

provisioning scripts based on Openstack

REST APIs Used in Provisioning Script:

powervc_image="/powervc/openstack/compute/v2/<TENANT_ID>/images"

powervc_network="/powervc/openstack/network/v2.0/networks"

powervc_placement="/powervc/openstack/compute/v2/<TENANT_ID>/os-aggregates"

powervc_scg="/powervc/openstack/compute/v2/<TENANT_ID>/storage-connectivity-groups"

powervc_hypervisors="/powervc/openstack/compute/v2/<TENANT_ID>/os-hypervisors"

powervc_storage_template="/powervc/openstack/volume/v2/<TENANT_ID>/types"

powervc_tenant="/powervc/openstack/identity/v2.0/tenants"

powervc_server="/powervc/openstack/compute/v2/<TENANT_ID>/servers"

powervc_server_attvol="/powervc/openstack/compute/v2/<TENANT_ID>/servers"

powervc_volume_v1="/powervc/openstack/volume/v1/<TENANT_ID>/volumes"

powervc_volume_v2="/powervc/openstack/volume/v2/<TENANT_ID>/volumes"

powervc_image="/powervc/openstack/compute/v2/<TENANT_ID>/images"




IBM.

JSON Structures Used for Server and Storage (1/2)

Following json structures are used in populating and passing all parameters to

construct server arguments for openstack based REST API requestsServer json Construct:

# JSON Body for server creation

# New format for Server

# Attaching a multipale volumes during initialization of boot

json_body_create_vm={ "server":

{ "flavor":

{ "OS-FLV-EXT-DATA:ephemeral":0,

"disk": 0,

"extra_specs":

{

"powervm:proc_units": "<PROCU>",

"powervm:min_proc_units": "<MINPROCU>",

"powervm:max_proc_units": "<MAXPROCU>",

"powervm:min_vcpu": "<MINVCPUS>",

"powervm:max_vcpu": "<MAXVCPUS>",

"powervm:min_mem": "<MINMEM>",

"powervm:max_mem": "<MAXMEM>",

"powervm:storage_connectivity_group": "SCG"

},




IBM.

JSON Structures Used for Server and Storage (2/2)

Construct Continues….

Server json Construct: Continued......

"ram": "<EAM>",

"vcpus": "<VCPUS>"

},

"imageRef": "<IMAGE_ID>",

"max_count": 1,

"availability_zone": "<HYP>",

"name": "<NAME>",

"networkRef": "<NETWORK_ID>",

"networks": [ { "fixed_ip": "<IP>", "uuid": "<NETWORK_ID>" } ],

"block_device_mapping_v2": [],

"user_data": "<SCRIPT>"

}

}




IBM.

Json Script to Create LUNs – Logs or Data

Following is an extract of creating Data LUNs (similar for LOG LUNs) and these

LUNs are created ahead of VM creation.

Storage LUN Provisioning:

while lun <= int(f_Data_lun_count):

json_body_mkvol["volume"]["size"]=int(f_Data_lun_size)

json_body_mkvol["volume"]["name"]=str(f_name)+"_"+str(f_Data_lun_prefix)+"_"+str(lun)

all_volumes=call_api("POST",powervc_url,powervc_volume_v2_replaced,auth_headers_tok,json.dumps(json_

body_mkvol),200,0)

vol_id=all_volumes["volume"]["id"]

templ = { 'uuid': vol_id, 'source_type': 'volume','destination_type': 'volume', 'boot_index': '-

1' }

time.sleep(15)

wwn=call_api("GET",powervc_url,powervc_volume_v2_replaced + "/" +

str(vol_id),auth_headers_tok,"",200,0)

a_wwn=wwn["volume"]["metadata"]["volume_wwn"]

vol_uuid.append(vol_id)

vol_type.append(str(f_Data_lun_prefix))

vol_size.append(int(f_Data_lun_size))

vol_wwns.append(str(a_wwn))

json_body_create_vm["server"]["block_device_mapping_v2"].append(templ)

lun+=1




IBM.

Base64 Encoding of LUN IDs and “cust_sap.sh” Script to Pass via REST API to VM when Deploying a VM

Following extract illustrate json code for Base64 encoding required to pass a payload to

cloudinit to execute a script “cust_sap.sh” within VM upon VM’s first boot. This script will

also have LUN IDs and their size/purpose details at the end of the script.

Encode cust_sap.sh:

# Generate Post customisation script

f=open("/tmp/disk_details.txt","w")

i=0

while i < len(vol_uuid):

f.write(vol_type[i]+":"+str(vol_size[i])+":"+vol_wwns[i]+":"+f_mem+":\n")

i+=1

f.close()

cmd='cp /usr/local/scripts/cust_sap.sh /tmp/'

os.system( cmd )

cmd='cat /tmp/disk_details.txt >> /tmp/cust_sap.sh'

os.system( cmd )

base64encoded=""

f=open("/tmp/cust_sap.sh","r")

base64encoded=base64.b64encode(f.read())

f.close()

# -- Setting script

script=base64encoded

json_body_create_vm["server"]["user_data"]=script




IBM.

VM Deployment

Following extract illustrate json code to initiate the deployment of VM. Once deployment

started, it will display the Server ID as allocated by IBM PowerVC.

Deploying VM:

# Create VM based on input values together with Boot LUN and additional Data and Log Luns

create_vm=call_api("POST",powervc_url,powervc_server_replaced,auth_headers_tok,json.dumps

(json_body_create_vm),200,0)

# Set VM ID

vm_id=create_vm["server"]["id"]

print "[Info] Server ID: ",vm_id




IBM.

How To Request Scripts

Send an email to [email protected] requesting these scripts should you plan to try or use these scripts in your environment.

The scripts that are available to request are;

• “build_hana_vm.sh”

• “move_ssd.sh”

• “cust_sap.sh”

NOTES:• All scripts are provided “as-is” and no support from IBM.• Use it at you own risk and carry no warranty of any kind from IBM• Always validate these scripts and logic in these scripts are suitable for your environment.• There will be no enhancements to these scripts• IBM Carries no liabilities on these scripts

Scripts




IBM.

Lessons Learned

Main Issues experienced during deployment of HANA VMs using these scripts are provided below;

• IPV6 enablement on interface fails the Installation of IBM Power VC - APAR was created and afix is now in place to correct this issue.

• Default SCG create unwanted large number of storage paths – Specific SCGs must be definedto overcome this issue in IBM Power VC.

• Missing WWNs for Storage when VM Provisioned too quickly – Must delay provisioning VMsallowing IBM Power VC to populate WWNs into Database in IBM Power VC

• Cinder json constructs does not have a way to specify storage placement – V7K CLI alwaysuse default “mdisk” group and this lead to disk re-organization to move and place HANA Logs onSSDs

• Captured Customized SLES11SP4 SOE Image becomes outdated in Power VC – Due to SAPcontinuously releasing fixes/updates to apply on SLES11 Images need maintenance.

• Multiple Networks are needed by SAP HANA - Additional Networks such as HSR networks areadded post provisioning using IBM Power VC GUI

• Non-IBM Storage may not have Cinder Drivers - Validate supported storage systems are used

• Script works well for small deploys - Larger ones have issues where more error checking wouldbe needed

• Setup of storage POOLS vs “mdisks” - POOLS can be targeted in PowerVC however, specificmdisks are not possible to be targetted – yet.




IBM.


Live SAP HANA Workload Movement using LPM Including

Storage

• Introduction

• Scope of Moving VM’s while SAP HANA Databases workloads are Active

• Power System Configuration and Setup

• Moving SUSE VMs Using LPM (Excluding Storage)

• Moving Storage - Manually

• Lessons Learned




IBM.

Live HANA Workload Movement using LPM Including Storage From One Physical System to Another

Introduction

• Validate VM Move using LPM Tool in HMC

• Execute LPM to move from Source P8 System to Target P8 System

• Create Storage LUNs identical to Source System Using IBM Power VC on Target V7K

and Attach to VM

• Discover and Validate Storage on VM

• Extend SLES OS System Volume Group to New Target Disk

• Relocate all OS disk partitions From Source Disk to target OS System Disk

• Remove Source LUNs from Operating System volume groups within VM

• Extend HANA Data and HANA Log Volumes to Corresponding Target LUNs

• Relocate all Data and Log disk partitions to target LUNs

• Remove Source LUNs from HANA Data and HANA Log volume groups within VM

• Update “lilo.conf” with new boot disk device and re-create boot images

• Detach and Delete Source LUNs from VM using IBM Power VC

• Move OS disk and Log LUNs on to SSD Storage in Target V7K




IBM.

Live SAP HANA Workload Movement using LPM Including Storage

LPM

MANUAL




IBM.

Use LPM to Validate and Move Source System to Target POWER System




IBM.

Create Target LUNs

Create identical LUNs on Target V7000 using Power VC GUI

Details of Current Example LUN Configuration:

OS : 1 x 128G SSD

Data : 2 x 150G SAS

Log : 1 x 75G SSD

Original LUN IDs (V7K_#1) New LUNs on (V7K_#2)

OS : 60050764000000000000000000011111 60050000000000000000000000000011

Data : 60050764000000000000000000022222 60050000000000000000000000000022

60050764000000000000000000033333 60050000000000000000000000000033

Log : 60050764000000000000000000044444 60050000000000000000000000000044

Creating LUNs using V7K (SVC) CLI:

svctask mkvdisk -iogrp io_grp0 -mdiskgrp V7K_#1 -mdisk mdisk0:mdisk1:mdisk2 -name testlpar-boot-mirror

-size 137438953472 -unit b -syncrate 100 -easytier off

svctask mkvdisk -iogrp io_grp0 -mdiskgrp V7K_#1 -name testlpar_data_1_mirror -size

161061273600 -unit b -syncrate 100

svctask mkvdisk -iogrp io_grp0 -mdiskgrp V7K_#1 -name testlpar_data_2_mirror -size

161061273600 -unit b -syncrate 100

svctask mkvdisk -iogrp io_grp0 -mdiskgrp V7K_#1 -mdisk mdisk0:mdisk1:mdisk2 -name

testlpar_log_1_mirror -size 80530636800 -unit b -syncrate 100 -easytier off




IBM.

Create New Storage Identical to Existing Storage LUNs using IBM Power VC




IBM.

Discover and Validate Storage on VM (1/2)

Scan Virtual FC adapters to discover newly mapped storage

Before Attachment:

# pvs

PV VG Fmt Attr PSize PFree

/dev/mapper/mpatha_part2 system lvm2 a-- 127.80g 75.80g

/dev/mapper/mpathe hdbdatavg lvm2 a-- 150.00g 73.00g

/dev/mapper/mpathf hdbdatavg lvm2 a-- 150.00g 66.00g

/dev/mapper/mpathg hdblogvg lvm2 a-- 75.00g 48.00g

# multipath -ll|grep dm-

mpathe (60050764000000000000000000022222) dm-2 IBM,2145

mpatha (60050764000000000000000000011111) dm-3 IBM,2145

mpathg (60050764000000000000000000033333) dm-1 IBM,2145

mpathf (60050764000000000000000000044444) dm-0 IBM,2145

Discover new LUNs and Enable Multipaths:

# rescan-scsi-bus.sh

# service multipathd stop

# multipathd -F

# service multipathd start




IBM.

Discover and Validate Storage on VM (2/2)

After Scanning Virtual FC adapters to discover newly mapped

storage

After Attachment:

# pvs

PV VG Fmt Attr PSize PFree

/dev/mapper/mpatha_part2 system lvm2 a-- 127.80g 75.80g

/dev/mapper/mpathb lvm2 a-- 128.00g 128.00g

/dev/mapper/mpathc lvm2 a-- 75.00g 75.00g

/dev/mapper/mpathd lvm2 a-- 150.00g 150.00g

/dev/mapper/mpathe hdbdatavg lvm2 a-- 150.00g 73.00g

/dev/mapper/mpathf hdbdatavg lvm2 a-- 150.00g 66.00g

/dev/mapper/mpathg hdblogvg lvm2 a-- 75.00g 48.00g

/dev/mapper/mpathh lvm2 a-- 150.00g 150.00g

# multipath -ll|grep dm-

mpathe (360050000000100cd78000000000001ce) dm-2 IBM,2145

mpathd (360050000000100ce6800000000000076) dm-14 IBM,2145

mpathc (360050000000100ce6800000000000077) dm-13 IBM,2145

mpathb (360050000000100ce6800000000000074) dm-12 IBM,2145

mpatha (360050000000100cd78000000000001d1) dm-3 IBM,2145

mpathh (360050000000100ce6800000000000075) dm-15 IBM,2145

mpathg (360050000000100cd78000000000001d0) dm-1 IBM,2145

mpathf (360050000000100cd78000000000001cf) dm-0 IBM,2145




IBM.

Extend and Relocate SLES OS System Volume Group to New Target Disk

Following are the steps to extend and Relocate OSI from Source to target LUN

1. Create boot LV and a second partition as following for the root volume using “fdisk” tool:

# Make sure to match the partition configuration from Source LUN. For example:

Device Boot Start End Blocks Id System

/dev/mapper/mpathb_part1 2048 401407 199680 6 FAT16

/dev/mapper/mpathb_part2 401408 268414975 134006784 8e Linux LVM

2. Format /dev/mapper/mpathb_part2 and add to "system" volume group as a new disk:

pvcreate /dev/mapper/mpathb_part2

vgextend system /dev/mapper/mpathb_part2

3. Move all OS partitions to target:

# nohup pvmove /dev/mapper/mpatha_part2 /dev/mapper/mpathb_part2

4. Copy boot LV <Source_Boot_LUN>_part1 on boot disk to new system disk <Target_Booot_LUN>_part1 using

dd:

# dd if=/dev/mapper/mpatha_part1 of=/dev/mapper/mpathb_part1

5. Find UUID for "system-root" and "/system-swap“ and validate with /etc/fstab entires:

# blkid|grep system

/dev/mapper/system-root: UUID="0f17d4c1-8dfd-42c3-b7ec-c36551111111" TYPE="ext3"

/dev/mapper/system-swap: UUID="474bb6fa-bc0b-4a36-b4a2-16f551111111" TYPE="swap“

6. Reduce Source LUN:

# vgreduce /dev/mapper/mpatha_part2




IBM.

Extend HANA Data and HANA Log Volumes to Corresponding Target LUNs, Move Data and Remove Source LUNs

Use following sequence of activities to prepare storage, add to volume groups and then to

move all data and log partitions from source LUNs to Target LUNs. Finally remove source

LUNs from volume groups to cleanup source LUNs.

1. Format all new LUNs:

# pvcreate <New_LUN_1> <New_LUN_2> ... <New_LUN_N>

2. Extend Data Volume Group and Log Volume Groups to new LUNs:

# vgextend hdbdatavg <New_LUN_1> <New_LUN_2> ... <New_LUN_N>

# vgextend hdblogvg <New_LUN_A> <New_LUN_B> ... <New_LUN_M>

3. Relocate all Partitions from Source Data and Log LUNs to Target LUNs within each Volume Group:

# pvmove <Source_LUN> <Target_LUN>

4. Reduce Source LUNs from Data and Log Volume Groups when data is completely moved off:

# vgreduce hdbdatavg <New_LUN_11> <New_LUN_22> ... <New_LUN_NN>

# vgreduce hdblogvg <New_LUN_AA> <New_LUN_BB> ... <New_LUN_MM>




IBM.

Update lilo.conf and re-create boot images

Update the “/etc/lilo.conf” and re-create boot images as per following steps.

1. Update “/etc/lilo.conf” to reflect the new Boot LUN ID for boot entry:

boot = /dev/disk/by-id/scsi-360050764001111111100000000000074-part1

2. Update Lilo and generate boot images by running following commands:

# lilo

# mkinitrd

3. Validate /boot directory for latest boot images:




IBM.

Cleanup LUNs on IBM Power VC and V7K Storage System

Cleanup LUNs

• Detach All Source LUNs mapped to VM using IBM Power VC GUI

• Operating System LUN

• HANA DATA LUNs

• HANA Log LUNs

• Delete All source LUNs from IBM Power VC using GUI


• HANA DATA LUNs

• HANA Log LUNs

• Delete All Source LUNs from V7K Source Storage System – Using either V7K or

PVC GUI


• HANA DATA LUNs

• HANA Log LUNs




IBM.

Move OS disk and Log LUNs on to SSD Storage in Target V7K

Move OS and LOG LUNs to SSD “mdisk” group on Target V7K

• Identify Following list of LUNs that are required to be moved on to SSD storage in the

Target System


• All HANA Data LUNs

• All HANA Log LUNs

• Use V7K Subsystem capabilities in creating a mirror copy of each LUN to SSD and

remove the original copy once the synchronization is over.

• Remove all of the non-SSD LOG drives which are source for the mirror copy from V7K

Subsystem.




IBM.

Lessons Learned

Some of the main issues and concerns that were identified when moving live HANA

workloads:

• Leaving LOG and OSI volume olumes on non SSD disks unintentionally – Must plan to moveall HANA LOG disks and OSI disk to SSD

• Identical Storage Configuration is Created and Allocated as Target Storage to VM – It isimportant to make sure that all storage carved as Target storage are identical to number of LUNsand size of each LUNs to current existing LUNs.

• Updating /etc/lilo.conf and creating boot images – Must update lilo.conf and update lilo and bootimages.

• Make sure that all “pvmove” commands complete without interruptions from source to target arecompleted. Run with “nohup”




IBM.

Session summary

IBM Power VC with Openstack Based SAP HANA VM Provisioning:

• Efficient Rapid Deployment Capability

• Ease of Deployment in Compute and Storage Infrastructure

• Repeatable Workflows that can be Re-Used

• Production and Non-Production Deployment Capability

• SAP HANA and IBM Best Practices

• Secure End-User Deployment Capability

SAP HANA Live Workload Moves:

• Move on to Physically separate Compute and Storage Hardware

• Transparent moves - No interruptions or Outages during move to HANA Users or

Databases

• Uses the built in proven and tested LPM capability




IBM.

IBM Systems LAB Services

STG

LAB

Services

SAP HANA Migration

Services from Intel to

POWER Systems

SAP HANA Planning and

Implementation Services on

POWER Systems

[email protected]




IBM.

Session Evaluations

YOUR OPINION MATTERS!

Submit four or more session evaluations by 5:30pm Wednesday

to be eligible for drawings!

*Winners will be notified Thursday morning. Prizes must be picked up at

registration desk, during operating hours, by the conclusion of the event.

1 2 3 4




IBM.

Continue growing your IBM skills

ibm.com/training

provides a comprehensive portfolio of skills and careeraccelerators that are designed to meet all your training needs.

If you can’t find the training that is right for you with our

Global Training Providers, we can help.

Contact IBM Training at [email protected]

Global Skills Initiative




IBM.

Documentation and References

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102502




IBM.

SAP HANA on IBM Power Systems and IBM System Storage -Guides

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102502




IBM.

Acknowledgements and References

• Some Python Code extracts in this presentation may reference original code located in following link:http://chmod666.org/index.php/powervc-express-using-local-storage-overview-tips-and-tricks-and-lessons-learned-from-experience/

• There is a verygoodtutorialfromBenoit Creauthatexplainshowtocreate(Phyton-) scriptsthatcreatea newVirtualMachinefromthecommandline:

http://chmod666.org/index.php/powervc-express-using-local-storage-overview-tips-and-tricks-and-lessons-learned-from-experience/

• Cloudinit details for SUSE can be found in

http://download.opensuse.org/ports/ppc/distribution/13.2/repo/oss/suse/ and cloudinit package is on the latest PowerVC CD's (>=1.3.0 ) for most distro's

ibm power vc integration with openstack for sap hana

Documents