Building Virtualised CloudStack Test Environments

Geoff HigginbottomCTO ShapeBlue

geoff.higginbottom@shapeblue.comTwitter: @CloudStackGuru

www.shapeblue.com

@CloudStackGuru

Cloud Architect & ShapeBlue CTO Specialise in….

Designing & Building Clouds based on Apache CloudStack / Citrix CloudPlatform

Developing CloudStack training Blogging and sharing CloudStack knowledge

Involved with CloudStack before donation to Apache Designed Clouds for Cloudera, Everyware, SunGard, Ascenty,

BskyB, Trader Media, M5 Hosting, Team Cymru, Interoute, University of Pennsylvania.…

CloudStack Committer

About Me

www.shapeblue.com

@CloudStackGuru

Portable Test / Training / Demo Environment Windows, MAC and Linux Support Open Source Software Reflect Production CloudStack deployments

Basic Networking With and Without Security Groups

Advanced Networking VLANs for Traffic Isolation

Utilise XenServer Hypervisor(s) Shared / Local Primary Storage Dedicated Storage Networks for Shared Primary, and Secondary Storage 100% CloudStack Feature Support

Requirements

www.shapeblue.com

@CloudStackGuru

Flexible Architecture Remote access to system Manage external Hypervisors Use External Storage Test HA Failover Dual Management Servers with Load Balancing Dual MySQL Servers with Replication Dual DNS for ‘Internal DNS’ Configuration Management (Ansible)

Additional Requirements

www.shapeblue.com

@CloudStackGuru

Cross Platform Virtualisation Application Support to Windows, OSX and Linux OpenSource (unlike VMware Workstation / Fusion) Capable of running XenServer (but without HVM

support) Downloads available from

https://www.virtualbox.org/wiki/Downloads

VirtualBox

www.shapeblue.com

@CloudStackGuru

Management Server VM CentOS 6.5 VM CloudStack Management Server MySQL Server NFS Storage Server NGINX (for hosting templates)

XenServer VM XenServer 6.2

Sandbox Deployment

www.shapeblue.com

@CloudStackGuru

NAT

Network Design

Management

Public

Storage

Guest

XenServer VM

Host Only Network 0

Host Only Network 1

Host Only Network 2

Internal Network

NFS Storage

Templates

Management VM

www.shapeblue.com

@CloudStackGuru

Network Design

Management

Public

Storage

Guest

Management VM XenServer VM

192.168.56.0/24 - untagged

172.30.0.0/24 - untagged

10.10.100.0/24 – vlan 10010.10.101.0/24 – vlan 101

10.1.1.0/24vlans 600-699

NFS Storage

TemplatesNAT

10.0.2.0/24

www.shapeblue.com

@CloudStackGuru

2GB RAM Minimum required to install CloudStack Reduce to 1GB after CloudStack is deployed

1vCPU Enable I/O APIC, PAE/NX, VT-x/AMD-v 4 NICs 16 GB disc minimum – more if available on host

VirtualBox ConfigurationManagement VM

Resources

www.shapeblue.com

@CloudStackGuru

VirtualBox ConfigurationManagement VM Networks

Adapter 1:Management Traffic

www.shapeblue.com

@CloudStackGuru

VirtualBox ConfigurationManagement VM Networks

Adapter 2:NAT to enable accessto internet for install and updating etc

www.shapeblue.com

@CloudStackGuru

VirtualBox ConfigurationManagement VM Networks

Adapter 3:‘Public’ Traffic toallow the hostingof Templates vianginx

www.shapeblue.com

@CloudStackGuru

VirtualBox ConfigurationManagement VM Networks

Adapter 4:Storage Trafficacting as NFS Serverfor Primary and Secondary Storage

www.shapeblue.com

@CloudStackGuru

1.5GB (4GB RAM Host) 2vCPU Enable I/O APIC, PAE/NX, VT-x/AMD-v 4 NICs 16 GB disk minimum – more if testing local storage

VirtualBox ConfigurationXenServer VM Resources

www.shapeblue.com

@CloudStackGuru

VirtualBox ConfigurationXenServer VM Networks

Adapter 1:Management Traffic

www.shapeblue.com

@CloudStackGuru

VirtualBox ConfigurationXenServer VM Networks

Adapter 2:Guest Traffic

www.shapeblue.com

@CloudStackGuru

VirtualBox ConfigurationXenServer VM Networks

Adapter 3:Public Traffic

www.shapeblue.com

@CloudStackGuru

VirtualBox ConfigurationXenServer VM Networks

Adapter 4:Storage Traffic

www.shapeblue.com

@CloudStackGuru

DEVICE=eth0TYPE=EthernetIPADDR=192.168.56.11PREFIX=24ONBOOT=yesNM_CONTROLLED=noBOOTPROTO=noneIPV4_FAILURE_FATAL=yesIPV6INIT=noNAME=MGMT

Management Server Interfaceseth0 - Management

www.shapeblue.com

@CloudStackGuru

DEVICE=eth1TYPE=EthernetIPADDR=10.0.2.11GATEWAY=10.0.2.1PREFIX=24ONBOOT=yesNM_CONTROLLED=noBOOTPROTO=none

DEFROUTE=yesPEERROUTES=yesIPV4_FAILURE_FATAL=yesIPV6INIT=noNAME=NAT

Management Server Interfaceseth1 - NAT

www.shapeblue.com

@CloudStackGuru

Management Server Interfaceseth2 – Public

DDEVICE=eth2TYPE=EthernetIPADDR=172.30.0.11PREFIX=24ONBOOT=yesNM_CONTROLLED=noBOOTPROTO=noneIPV4_FAILURE_FATAL=yesIPV6INIT=noNAME=PUBLIC

www.shapeblue.com

@CloudStackGuru

Management Server Interfaceseth3 / eth3.100 / eth3.101 - Storage

DEVICE=eth3.100TYPE=EthernetIPADDR=10.10.100.11PREFIX=24ONBOOT=yesBOOTPROTO=noneNAME=PRI-STORVLAN=yesUSERCTL=noMTU=9000

DEVICE=eth3.101TYPE=EthernetIPADDR=10.10.101.11PREFIX=24ONBOOT=yesBOOTPROTO=noneNAME=SEC-STORVLAN=yesUSERCTL=noMTU=9000

DEVICE=eth3TYPE=EthernetBOOTPROTO=noneONBOOT=yesMTU=9000VLAN=yesUSERCTL=noMTU=9000

www.shapeblue.com

@CloudStackGuru

XenServer Networks

www.shapeblue.com

@CloudStackGuru

Install CentOS 6.5 Minimal and update Follow the ‘Quick Installation Guide for CentOS’

http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/latest/qig.html

Install and configure the following: MySQL Server NFS CloudStack-Management

Management Server Install

www.shapeblue.com

@CloudStackGuru

Install nginx so it can be used to host Templates Once installed download the default CentOS template and

place in the nginx web folder# cd /usr/share/nginx/html# wget -nc http://download.cloud.com/templates/builtin/centos56-x86_64.vhd.bz2

Update the CloudStack DB to change the location of the default template# mysql –p<password> cloud -e \ "UPDATE cloud.vm_template SET url='http://192.168.56.11/centos56-x86_64.vhd.bz2' WHERE unique_name='centos56-x86_64-xen';"

Configuration Tweaks

www.shapeblue.com

@CloudStackGuru

# mysql –p<password> cloud -e \ "INSERT INTO cloud.configuration (category, instance, component, name, value, description) VALUES ('Advanced', 'DEFAULT', 'management-server', 'xen.check.hvm', 'false', 'Shoud we allow only the XenServers support HVM');"

Configuration TweaksEnable use of Non-HVM XenServer

www.shapeblue.com

@CloudStackGuru

# mysql -u root –p<password> cloud -e \ "UPDATE cloud.service_offering SET ram_size='128', speed='128' WHERE vm_type='domainrouter';"# mysql -u root –p<password> cloud -e \ "UPDATE cloud.service_offering SET ram_size='128', speed='128' WHERE vm_type='elasticloadbalancervm';"# mysql -u root –p<password> cloud -e \ "UPDATE cloud.service_offering SET ram_size='128', speed='128' WHERE vm_type='secondarystoragevm';"# mysql -u root –p<password> cloud -e \ "UPDATE cloud.service_offering SET ram_size='128', speed='128' WHERE vm_type='internalloadbalancervm';"# mysql -u root –p<password> cloud -e \ "UPDATE cloud.service_offering SET ram_size='128', speed='128' WHERE vm_type='consoleproxy';"

Configuration TweaksReduce the RAM allocations for System VMs

www.shapeblue.com

@CloudStackGuru

expunge.delay = 60expunge.interval = 60account.cleanup.interval = 60capacity.skip.counting.hours = 60cluster.cpu.allocated.capacity.disablethreshold = 0.99cluster.memory.allocated.capacity.disablethreshold = 0.99capacity.check.period = 60000event.purge.delay = 1network.gc.interval = 60network.gc.wait = 60vm.op.cleanup.interval = 600

Configuration TweaksUpdate Global Settings for this special install

www.shapeblue.com

@CloudStackGuru

vm.op.cleanup.wait = 60vm.tranisition.wait.interval = 600vm.tranisition.wait.interval = 600vpc.cleanup.interval = 60cpu.overprovisioning.factor = 4storage.overprovisioning.factor = 4secstorage.allowed.internal.sites = 192.168.56.11/32management.network.cidr = 192.168.56.0/24host = 192.168.56.11enable.dynamic.scale.vm = truesecstorage.vm.mtu.size = 9000

Configuration TweaksUpdate Global Settings for this special install

www.shapeblue.com

@CloudStackGuru

Install XenServer 6.2 from ISO Apply latest patches in line with

http://support.citrix.com/cms/kc/cloud-troubleshooting/cloudplatform-compatibility-matrix/

XenServer Install

www.shapeblue.com

@CloudStackGuru

Zone Name – Test IPv4 DNS1 – 8.8.8.8 Internal DNS 1 – 192.168.56.11 Hypervisor – XenServer Guest CIDR – 10.1.1.0/24

CloudStack SettingsZone Setup

www.shapeblue.com

@CloudStackGuru

CloudStack SettingsPhysical Network

www.shapeblue.com

@CloudStackGuru

CloudStack SettingsNetwork Traffic Labels

www.shapeblue.com

@CloudStackGuru

Gateway – 172.30.0.1 Netmask – 255.255.255.0 VLAN - <blank>* Start IP – 172.30.0.21 End IP -172.30.0.30

* Leave VLAN blank so the Host Laptop can access the Public Network

CloudStack SettingsPublic Traffic

www.shapeblue.com

@CloudStackGuru

POD Name – POD1 Reserved System Gateway – 192.168.56.1 Reserved System Netmask – 255.255.255.0 Start Reserved System IP – 192.168.56.21 End Reserved System IP – 192.168.56.30

CloudStack SettingsPOD Settings

www.shapeblue.com

@CloudStackGuru

VLAN Range – 600 - 699

CloudStack SettingsGuest Traffic

www.shapeblue.com

@CloudStackGuru

Gateway – 10.10.101.1 Netmask – 255.255.255.0 VLAN – <blank>* Start IP – 10.10.101.21 End IP – 10.10.101.30

* Storage VLAN is left blank as it is being set by XenServer on its virtual network, if you also set it here it will not work

CloudStack SettingsStorage Traffic

www.shapeblue.com

@CloudStackGuru

Hypervisor – XenServer Cluster Name – CLU1

CloudStack SettingsCluster

www.shapeblue.com

@CloudStackGuru

Host Name - 192.168.56.101 Username – root Password – xxxxxxxx

CloudStack SettingsHost

www.shapeblue.com

@CloudStackGuru

Name – PRI1 Scope – Cluster Server – 10.10.100.11 Path - /exports/primary

CloudStack SettingsPrimary Storage

www.shapeblue.com

@CloudStackGuru

Provider – NFS Name – SEC1 Server – 10.10.101.11 Path - /exports/secondary

CloudStack SettingsSecondary Storage

www.shapeblue.com

@CloudStackGuru

Sandbox configuration limits access to only the Host laptop

Enhanced configuration allows external access to system: Management Server Secondary Storage VM and Console Proxy VM XenServer Management (via XenCenter)

Also enables: External Hypervisors (e.g. ESXi Cluster) External Storage

Enhanced Deployment

www.shapeblue.com

@CloudStackGuru

Replace ‘Host Only’ Networks with ‘Bridged’ Networks Requires Physical NIC as VirtualBox does not support

Promiscuous mode over Wifi Public IP Range should be compatible with Hosts Physical

Network Management Network

Replace 192.168.56.x with 192.168.57.x (prevents conflict with VB) Add 192.168.57.x IP to Host Laptop as secondary IP Configure external devices with 192.168.57.x IP to access UI

Will still operate as a ‘Sandbox’ config when not connected to Bridged Network

Changes to VirtualBox

www.shapeblue.com

@CloudStackGuru

2x CloudStack Management 2x MySQL - Master / Slave 2x HAProxy Keepalived 2x Bind Servers 2x Citrix XenServers

All on an 8GB RAM Laptop

Quick Demo (if time allows)

Get Ambitious!

Building Virtualised CloudStack Test Environments

Geoff HigginbottomCTO ShapeBlue

geoff.higginbottom@shapeblue.comTwitter: @CloudStackGuru