dell emc ready solution for vmware nfv platform...6 dell emc ready solution for vmware nfv platform...

A Dell EMC Deployment and Configuration Guide

Dell EMC Ready Solution for VMware NFV Platform Deployment and Configuration Guide for vCloud 2.0 with VMware Integrated OpenStack

Dell Engineering June 2018

2 Dell EMC Ready Solution for VMware NFV Platform

Revisions

Date Description

April 2017 Initial release

August 2017 Update to initial release

March 2018 Incorporate chapter for VMware Integrated OpenStack Manager

June 2018 Update to 14G

The information in this publication is provided “as is.” Dell Inc. makes no representations or warranties of any kind about the information in this

publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any software described in this publication requires an applicable software license.

Copyright © 2018 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Other

trademarks may be the property of their respective owners. Published in the USA June 2018.

Dell believes that the information in this document is accurate as of its publication date. The information is subject to change without notice.


Table of contents Revisions............................................................................................................................................................................. 2

Document scope ................................................................................................................................................................. 5

1 Physical wiring .............................................................................................................................................................. 6

2 Base physical network configuration without VTEP ..................................................................................................... 8

3 System updates ............................................................................................................................................................ 9

4 VMware vSphere install process ................................................................................................................................ 13

4.1 ESXi installation and configuration ................................................................................................................... 13

4.1.1 Installation of ESXi on Dell PowerEdge through iDRAC9 ................................................................................ 13

4.2 Steps for ESXi installation ................................................................................................................................ 14

4.3 ESXi customization ........................................................................................................................................... 19

5 Auxiliary components ................................................................................................................................................. 27

6 VMware vCenter install process ................................................................................................................................. 28

7 Configure virtual networking ....................................................................................................................................... 30

7.1 Create data center and add management hosts .............................................................................................. 30

7.2 Create required virtual distributed switches ..................................................................................................... 31

8 Deploy VMware Virtual SAN clusters ......................................................................................................................... 34

9 Deploy NSX for VMware vSphere Manager ............................................................................................................... 38

10 Add a transport zone .................................................................................................................................................. 40

11 VMware Integrated OpenStack Manager ................................................................................................................... 41

11.1 Deploy VMware Integrated OpenStack Manager ............................................................................................. 41

11.2 Deploy VMware Integrated OpenStack ............................................................................................................ 47

11.2.1 2-pod topology .............................................................................................................................................. 47

11.3 Deploy OpenStack ............................................................................................................................................ 47

11.3.1 Initiate OpenStack deployment .................................................................................................................... 47

11.3.2 Review current VIO deployment .................................................................................................................. 56

11.3.3 Manage or modify VIO settings .................................................................................................................... 57

11.3.4 Post deployment modifications for VIO ........................................................................................................ 58

11.4 Access VIO ....................................................................................................................................................... 59

11.4.1 Launch VIO Horizon dashboard ................................................................................................................... 59

11.4.2 Use CLI to access VIO components ............................................................................................................ 60

12 Install and Deploy VMware vRealize Operations Manager ........................................................................................ 66

13 Install and Deploy VMware vRealize Log Insight 4.3 ................................................................................................. 68


14 NSX Manager data backup and restore ..................................................................................................................... 71

A Switch configurations ................................................................................................................................................. 72

A.1 S6010 top configuration .................................................................................................................................... 72

A.2 S6010 bot configuration .................................................................................................................................... 82

A.3 S4048tmgmt configuration ................................................................................................................................ 92

B NUMA ....................................................................................................................................................................... 103

C Documentation resources ........................................................................................................................................ 104

D Technical support and resources ............................................................................................................................. 105


Document scope

The information in this document addresses the following:

Network Functions Virtualization Infrastructure (NFVI) being the entirety of all hardware and software

components to support virtual network Functions

Virtualized Infrastructure Management (VIM) to support hardware and software components, and to

manage NFVI

A greenfield deployment of the VMware vCloud NFV 2.0 platform using the following versions:

Note: For more VMware vCloud NFV 2.0 information, see the VMware vCloud NFV 2.0 Release Notes and

the VMware vCloud NFV 2.0 Reference Architecture links in Appendix C.

o VMware vCenter Server 6.5U1

o VMware vSphere Hypervisor (ESXi) 6.5.0U1

o VMware Virtual SAN (vSAN) 6.6

o VMware NSX for vSphere 6.3.1

o VMware vRealize Orchestrator Appliance 7.2.0

o VMware vRealize Operations Manager 6.5

o VMware vRealize Log Insight 4.3

o VMware Integrated OpenStack 4.0

Design considerations used to implement the NFVI platforms that accommodate the level of

requirements for the service providers, partners, technical staff, and Dell EMC technical architects

and engineers

VMware Integrated OpenStack (VIO) 4.0 is based on the Ocata release of OpenStack that

implements the vSphere architecture

Note: For more information on vCloud NFV architecture, see Dell EMC NFV Ready Bundle for VMware.

This document does not address:

Deployment of an environment using VMware vSphere Replication, VMware vSphere Data

Protection, or VMware Site Recovery Manager

Support escalations outside of the signed support agreement

Assumption of operations, maintenance, or communications outside of the Dell EMC signed support

agreements

Individuals using this document are considered to be Dell EMC technical architects, engineers, and

associates who participated in the series of architecture design, proof-of-concept, briefing sessions, or

technology partner programs.

http://en.community.dell.com/techcenter/service_provider_solutions/w/wiki/12384.dell-emc-nfv-ready-bundle-for-vmware


1 Physical wiring The following image demonstrates the physical wiring that supports the vCloud NFV 2.0 architecture. It is

based on the Dell EMC Networking S4048T-ON and S6000-ON switches and Dell PowerEdge R640 servers.

Management Cluster

Edge/Resource Cluster

QSFP+

29 31

30 32

25 27

26 28

21 23

22 24

17 19

18 20

13 15

14 16

9 11

10 12

5 7

6 8

1 3

2 4

Stack ID

QSFP+

29 31

30 32

25 27

26 28

21 23

22 24

17 19

18 20

13 15

14 16

9 11

10 12

5 7

6 8

1 3

2 4

Stack ID

33 34 35 3631 3229 3027 2825 26 45 46 47 4843 4441 4239 4037 389 10 11 127 85 63 41 2 21 22 23 2419 2017 1815 1613 14 50 52 54

49 51 53

Sta

ck

ID

SPINE 1 SPINE 2

LEAF1 LEAF2

Management

QSFP+

29 31

30 32

25 27

26 28

21 23

22 24

17 19

18 20

13 15

14 16

9 11

10 12

5 7

6 8

1 3

2 4

Stack ID

QSFP+

29 31

30 32

25 27

26 28

21 23

22 24

17 19

18 20

13 15

14 16

9 11

10 12

5 7

6 8

1 3

2 4

Stack ID

Management Cluster


Platform Services Controller vCenter NSX Manager vRealize Operations Manager vRealize Log Insight Integrated Openstack

DLR NSX Controllers 1-3 Edge Service Gateways VNF Services

Physical wiring for vCloud NFV 2.0


QSFP+

29 31

30 32

25 27

26 28

21 23

22 24

17 19

18 20

13 15

14 16

9 11

10 12

5 7

6 8

1 3

2 4

Stack ID

QSFP+

29 31

30 32

25 27

26 28

21 23

22 24

17 19

18 20

13 15

14 16

9 11

10 12

5 7

6 8

1 3

2 4

Stack ID

Management Cluster


40G QSFP+ Fanout -> 4 x 10Gb

33 34 35 3631 3229 3027 2825 26 45 46 47 4843 4441 4239 4037 389 10 11 127 85 63 41 2 21 22 23 2419 2017 1815 1613 14 50 52 54

49 51 53

Sta

ck

ID

1G iDrac

SPINE 1 SPINE 2

LEAF1 LEAF2

Management

VM VDS

Infrastructure VDS

Platform Services Controller vCenter NSX Manager vRealize Operations Manager vRealize Log Insight Integrated Openstack

40G QSFP+ (Fabric Interconnect)

DLR NSX Controllers 1-3 Edge Service Gateways VNF Services

iDrac

QSFP+

29 31

30 32

25 27

26 28

21 23

22 24

17 19

18 20

13 15

14 16

9 11

10 12

5 7

6 8

1 3

2 4

Stack ID

QSFP+

29 31

30 32

25 27

26 28

21 23

22 24

17 19

18 20

13 15

14 16

9 11

10 12

5 7

6 8

1 3

2 4

Stack ID

Proposed physical wiring configuration for vCloud NFV 2.0 architecture


2 Base physical network configuration without VTEP Each server has a total of 4 x 10Gb SFP+ ports along with 2 x1G Base-T interfaces. The ports are physically

connected to the top-of-rack (ToR) switches and to the management switch. Each switch port is tagged with

the VLAN required to connect to the network.

Note: See Appendix A for the required switch configuration details.

The vCloud NFV architecture assembles components into three different functions: Management, Edge, and

Resource. Previous VMware vCloud NFV reference architectures had these functions mapped directly to a

vSphere cluster/pod.

Note: Throughout the remainder of this document, the terms cluster and pod are used interchangeably.

The following image shows the vCloud NFV 2.0 is an alternative design that uses two pods, has been

introduced: The new design combines the Edge and Resource pods into a single pod and leaves the

management components in the Management pod. The Virtual Network Functions (VNF) and networking

functions such as Edge Service Gateway (ESG) and NSX controllers, are assigned to the Edge/Resource

pod. An overview of the clusters and networking is shown following:

vCloud NFV 2.0 two pod architecture

The information provided in this section is based on the two pod alternative.

https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/vmware-vcloud-nfv-reference-architecture-v2.0.pdf


3 System updates Each hardware element of the Network Functions Virtualization Infrastructure (NFVI) has the most recent

version of firmware.

Note: To view the hardware components and current version of firmware, see the System Inventory

section within the iDRAC user interface (UI).

Manual updates using the Virtual Console and Lifecycle Controller are necessary for the 12x host machines.

To run the updates, perform the following steps:

1. Log in to the iDRAC user interface (UI) and launch the Virtual Console.

Launch iDRAC Virtual Console

2. From the virtual console, click Next Boot and select Lifecycle Controller from the list provided.

3. Click Power then Power Cycle System (cold boot) on the Virtual Console, to reboot the server.

4. After the server has rebooted, select Firmware Update then Launch Firmware Update from the

Lifecycle Controller screen.


Launch Firmware Update from Lifecycle Controller

5. From the Select Update Repository screen, select the repository location for the catalog and update

packages. For this example, the FTP Server option is used.

Update repository server setting

6. Accept the default FTP address without credentials if the server has the Internet access, then click

Test Network Connection.


Network connection verification screen

Network Connection Test screen

7. Click the OK button, then click Next to continue.

The system takes a few minutes to check for latest versions on the repository and present the

available system updates.

8. Verify that all the required updates are selected, then click Apply.


Apply available updates via Lifecycle Controller

The following versions were used on the setup:

Sample setup configurations

Component Version

BIOS 1.3.7

R740xd HBA330+ or R640 HBA330 15.17.08.01

Lifecycle Controller 3.15.17.15

Dell 64-Bit uEFI Diagnostics 4301A13

Dell OS Driver Pack 17.10.13

SSD Disks L380

Spinning disks FJ23

BP13G+EXP 3.31

Intel® I350 and X710s 18.3.6

Intel® X710 18.3.6

Note: To update the network switches, see the Dell EMC Open Networking S6010-ON and Dell EMC Open

Networking S4048T-ON manuals in Appendix B.

Alternatively, Dell OpenManage Essentials can perform an automatic inventory of all Dell servers.

Note: OME can perform an automatic inventory of all Dell EMC servers and identify the servers to update.

Schedule updates directly from the GUI without going into the virtual console of each server.

http://www.dell.com/en-us/work/learn/openmanage-essentials


4 VMware vSphere install process The vSphere Installation and Setup Guide for VMware vSphere 6.5, VMware ESXi 6.5, and vCenter Server

6.5 document describes the setup process for small and large environments. In this example, a small option is

applied using two VMware vCenter Server appliances with an embedded Platform Services Controller (PSC).

One is deployed for the Management cluster, and the other is deployed for the Edge/Resource cluster.

Note: The vCenter Server appliance can be configured on High Availability (HA), however, a large-scale

production deployment should set up replicated pairs of vCenter and PSC for full-fault tolerance.

vSphere installation workflow

4.1 ESXi installation and configuration

4.1.1 Installation of ESXi on Dell PowerEdge through iDRAC9 Prerequisites:

ESXi Installer 6.5.0.U1 .iso file.

iDRAC interface with SD card (at least 16GB) enabled on it.

At least 2 CPU cores.

https://pubs.vmware.com/vsphere-65/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-65-installation-setup-guide.pdf



Hard disks: 30-36 GB (minimum).

Server iDRAC should be accessible.

The NX/XD bit to be enabled for the CPU in the BIOS.

A minimum of 4GB of physical RAM. It is recommended to provide at least 8 GB of RAM to run

virtual machines in typical production environments.

Support for hardware virtualization (Intel VT-x or AMD RVI) must be enabled on x64 CPUs.

One or more Gigabit or faster Ethernet controllers.

Cache: One SAS/SATA solid-state disk (SSD).

Storage controllers: One SAS or SATA Host Bus Adapter (HBA), or a RAID controller that is in non-

RAID (pass through) or RAID 0 mode.

4.2 Steps for ESXi installation 1. Connect to the iDRAC.

a. Connect to iDRAC card. (https://x.x.x.x) – use the IP for that particular blade server’s iDRAC

Card.

b. Use the iDRAC default user credentials (User name-root, Password- calvin) and click the ‘Submit’

button.

iDRAC 9 log in screen

2. Click the Launch tab to open Virtual console of iDRAC in System Summary page.


iDRAC virtual console page

Virtual console of iDRAC in HTML5 (Jave, ActiveX, or HTML5) displays.

iDRAC virtual console

3. Click the Virtual Media tab and click Connect Virtual Media. 4. Select the Virtual Media tab, choose Map CD/DVD for the usage of an image (ISO-file).

5. Click the Map CD/DVD tab and then browse the drive of local system for the image file path to install.


Browse to file path screen

6. Click the Map Device tab for finishing the source selection.

Virtual media selection screen

7. Select the Next Boot tab, click Local SD Card as boot device, then click YES to save the selection.

The ESXi installation starts.

8. From the Welcome to the VMware ESXi 6.5.0 Installation screen, press the Enter key to continue

the installation.


ESXi installation confirmation screen

9. Review the information presented in the End User License Agreement and to accept the terms,

press F11 to accept. 10. Select the Internal Dual SD storage device listing, press Enter to select, then press Enter again to

confirm the selection.

Disk selection screen

11. From the Keyboard layout selection screen, use the arrow keys to highlight the US Default listing,

then press Enter to continue.

Keyboard selection screen

12. Enter the root password then press Enter to start the installation. The Scanning system screen displays.

13. Press F11 to confirm the installation.

14. Press Enter to reboot the system and to complete the installation process.


Installation complete screen

The host reboot and displays the boot process.

Host reboot screen

Once the reboot process completes, the Direct Console User Interface, or DCUI, screen displays.


DCUI screen

4.3 ESXi customization 1. From the DCUI menu, press F2 then enter the crendtials for authentication, and press Enter.

Authentication screen

2. To customize the system, use the arrow keys to highlight any function to customize, then press

Enter.


System customization screen

3. Optionally, the password can be changed or the management network can be configured.

4. To configure the management network, use the arrow keys to highlight the listing, then press Enter.

5. From the options provided, click to select any of the network settings options to customize, then press

Enter.

6. To change the Network Adapter settings, highlight the listing then press Enter to view Network

Adapter cards.

Configure Management Network settings


7. Select the appropriate adapter card for the host’s default management network connection and use

two or more adapters for fault tolerance and load balancing.

Network Adapters screen

a. To configure static IPv4 address, select it and press Enter.

Configure Management Network screen

b. Select the static IPv4 and network configuration and provide IPv4 address, subnet mask and

default gateway then save the options selected.


IPv4 Configuration screen

c. To adjust the DNS Configuration, highlight the option then press Enter.

DNS Configuration screen

d. To assign a new DNS, provide DNS server details, save, then exit.


DNS configuration

e. To restart the management network, highlight the option provided then press Enter to restart.

Restart Management Network

f. Select Test management Network and press Enter to ping the configured default gateway,

primary and alternate DNS servers, and resolve the configured hostname.


Test Management Network screen

g. Optionally, you can select the Network Restore Options listing to restore the network.

Network Restore Options screen


h. Select Troubleshooting Options to enable ESXi Shell, SSH, and restart agents.

Troubleshooting Options screen

i. Use the arrow keys to highlight the Enabling ESXi Shell and SSH listing, then press Enter to

enable or disable.

Disable ESXi Shell option


j. Optionally, select to Modify ESXi Shell and SSH timeouts and press Enter.

Timeout setting screen

k. Highlight the DCUI idle timeout listing then press Enter.

DCUI idle timeout screen

l. Press Esc to go back to Main DCUI page.


5 Auxiliary components AD-DNS-NTP deployment and configuration:

ESXi only supports the UTC time zone. A secondary NTP server must be configured with UTC followed by

the ESXi hosts and VMs.

Primary and secondary scenario is supported for AD, DNS and NTP however secondary NTP will only sync

the time with Primary if it is configured with UTC time zone else secondary NTP server will act as a primary

Note: AD, DNS and NTP server must be available or installed first to configure Auxiliary components such

as AD, DNS, NTP.


6 VMware vCenter install process The Deploying the vCenter Server Appliance and Platform Services Controller Appliance section of the

vSphere Installation and Setup Guide for VMware vSphere 6.5 explains the installation process and

alternatives. The process of installing an embedded VMware vCenter with Platform Services Controller (PSC)

is used for both Management and Edge/Resource clusters. After it is deployed, High Availability (HA) is

enabled to protect catastrophic failures.

For a greenfield deployment, select Install on a new Virtual SAN cluster containing the target host in the

Select datastore step. This guides you through the vSAN setup before deploying vCenter.

vCenter datastore selection

vSAN datastore disk selection

Note: The vCenter deployment on a brand new ESXi host, uses the VM Network associated to vSwitch0

default port group. A port group that requires special configuration such as a VLAN ID, will not be accessible

after it is deployed. When this occurs, the following screen displays and recommends changes to guarantee

network connectivity to the vCenter deployment before completing the setup in Stage 2:



vCenter Appliance Management interface error

After the management vCenter appliance is set up, enter the appropriate program licenses.

Note: The VMware vCenter, VMware vSphere with Operations Management, and VSAN licenses must all

be entered before proceeding to the next step.

Installation of the resource vCenter should be complete after the management cluster is fully set up.


7 Configure virtual networking VMware recommends the separation of resources within a cluster (see VMware vCloud NFV Reference

Architecture Guide in Appendix B), with each containing at least four ESXi servers for the vSAN deployments

as shown in the Management Pod following image. The Management pod hosts VIM components for both the

Management and Edge/Resource pods.

Note: Management components shown in red and Edge/Resource components are shown in blue.

Management pod servers and components

After the installation of VMware vCenter and the application the appropriate licenses is complete, create the

management cluster to enable the vSAN, Distributed Resource Scheduler (DRS), and High Availability (HA),

after the VMware vSphere Distributed Switch (VDS) has been configured.

For a greenfield deployment the process, is as follows:

1. Create a data center (NFVI data center for example) by adding each of the hosts that are part of the

Management pod.

2. Create required distributed vSwitches.

3. Create a management pod to enable vSAN as indicated in the Deploy VMware Virtual SAN clusters

section.

7.1 Create data center and add management hosts After the data center has been created, add at least four hosts to create the management cluster.


Servers listed within management cluster

7.2 Create required virtual distributed switches Taken from the Architecting a vCloud NFV Platform RA v2.0 Guide, the following image shows the underlying

Virtual Distributed Switch (VDS) configuration suggested for the Management and Edge/Resource clusters:

Suggested Management and Edge/Resource pod configurations

While the number of NICs and VDSs to use is not suggested, the port groups and the VMkernel services

shown in the image below, are required. Teamed interfaces are optional for a proof of concept or test

deployment however, they are mandatory in a production environment with Fault Tolerance requirements.

To create the required virtual distributed switches, perform the following:

Note: See Appendix C for information about accessing the vSphere Networking VMware vSphere 6.5,

VMware ESXI 6.5, and vCenter Server 6.5 guide.

1. Select the data center, and create a distributed switch.

https://www.vmware.com/techpapers/2017/vmware-vcloud-nfv-2-reference-architecture.html


2. Select Distributed Switch 6.5.

3. Select the required number of uplinks.

Note: The number of require uplinks depends on the number of physical NICs assigned to the VDS.

Note: Do not create a default port group at this time.

4. Click Finish.

After the VDS is created, management hosts can be added to it.

5. Right click VDS and select Add and Manage Hosts.

Note: Template mode can be used if all the hosts have identical NIC layout.

Host selection in template mode

6. Select Manage physical network adapters to associate the uplinks to this DVS, then click Next.

7. Click to select the uplinks to be associated with the VDS.

8. Click Next.

The Manage physical network adapters, template mode, window displays.

Configure network adapters in template mode


Note: Select Apply to all to use the same configuration on each of the selected hosts.

9. Analyze the impact, click Next, and then click Finish.

10. For each of the networks created in this section, select the appropriate VLAN ID, and add port groups

to the VDS.

Port group configuration

11. Create more management VMkernel interfaces for the ESXi servers and add them to the

ESXi_Management port group.

12. Right click the VDS and select Manage Host Networking as the task.

13. Select the Manage VMkernel adapters as the network adapter task for the ESXi management,

VSAN, vMotion and the Replication options, and associate them with the appropriate distributed port

group.

View of configured VDS infrastructure

14. To add other DVS or port groups, repeat the steps preceding.


8 Deploy VMware Virtual SAN clusters Each server has a mix, or hybrid, of solid state and spinning drives for storage deployment. For VMware

vSAN, the solid-state disks are required for the cache tier, while the spinning disks make up the capacity tier.

Each Dell EMC server is configured for a host bus adapter (HBA) in non-RAID mode since the vSAN software

handles the redundancy and storage cluster information.

There are two types of clusters: Management and Edge/Resource. To make the hardware configuration

simple, the HY-4 Series vSAN Ready Nodes were selected. These modified systems accommodate more

performance in the resource cluster and limit disk capacity in the management and edge clusters.

Note: For more information about the vSAN Ready Node program, see Appendix C for information about

accessing the Administering VMware Virtual SAN document and the VMware vSAN Ready Node

Configurator.

To create a vSAN cluster using Management Servers, perform the following steps:

1. Right-click a data center in the vSphere Web Client and select New Cluster.

2. Enter a name for the cluster in the Name text box.

3. Verify that the DRS and vSphere HA check boxes are not selected.

4. Locate the vSAN option and click to place a check in the Turn ON check box, then click OK.

The cluster displays in the inventory.

Management cluster creation with vSAN enabled

5. In the left-navigation, click vSAN to expand the section.

6. Click General, then the Configure tab.

7. Verify that the Add disks to storage option is set to Manual, and that Networking mode is set to

Unicast.

Note: Unicast networking for vSAN is supported on the ESXi server version 6.5e and above.


Default vSAN configuration

8. Right click the cluster and assign the VSAN license to the cluster.

9. Add the servers to the cluster.

Note: Currently, only the node used for vCenter installation should have disks claimed for the vSAN.

vSAN disk management

10. Click Cluster, then click Configure, vSAN, then Disk Management to create a disk group for each

of the servers on the cluster.

Note: The following image shows the setup that supports the vSAN on the management cluster.


vSAN for Management cluster with disk groups configured

11. Click Cluster, Monitor, vSAN, Health, and then Retest to perform a vSAN health check.

Performing vSAN health check

Note: Persistent storage should be used for logs, stack traces, and memory dumps. It should not be part of

the vSAN datastore as a failure on the vSAN impacts accessibility to the log information. Either reserve a

disk on each server for this purpose or configure the ESXi dump and the Syslog collector on each host to

send logging information to vCenter.

12. After the vSAN clusters have been created and tested, use the System logs are stored on non-

persistent storage and Configuring syslog on ESXi knowledge base articles to assist with updating

the system logging location in each file.

13. Use the Enabling VMware High Availability and VMware Distributed Resource Scheduler in a cluster

knowledge base article to help you enable Distributed Resource Scheduler (DRS) and High

Availability (HA).

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2032823





Enabling DRS

Enabling HA

14. Click the Monitor tab again to verify vSphere DRS and vSphere HA health.


9 Deploy NSX for VMware vSphere Manager The following instructions outline the steps necessary to deploy the NSX managers for the Management and

Edge/Resource cluster. See the NSX Installation Guide link in Appendix B for more details.

Note: vCloud NFV 2.0 uses NSX release 6.3.1.

1. Deploy the OVF templates for NSX Manager 1 (Management Cluster) and NSX Manager 2

(Edge/Resource cluster).

2. After deployed, connect each NSX manager to the corresponding VMware vCenter by logging in to

the web interface of each manager.

3. Select Manage vCenter Registration.

4. Using the Install and Assign NSX for vSphere License instructions found in the NSX Installation

Guide (see Appendix B to access the document link), apply the NSX licenses to each vCenter.

5. Log out of the vSphere Web Client then log back in.

A new Networking and Security icon displays on the home screen.

6. Click Home, Networking & Security, and then Installation.

7. Select the NSX Manager, and then Add a new controller.

Note: This step can be repeated for controllers 2 and 3.

Adding controller to NSX Manager

8. Using the range for the controllers to connect over the management port group, make a new IP pool.

Note: The total time to deploy each controller is approximately 4 minutes.

9. Use the Exclude Virtual Machines from Firewall protection instructions found in the NSX Installation

Guide (see Appendix B to access the document link) to exclude vCenter Server from any future

firewall rules.

10. Following the information provided in the Prepare Host Clusters for NSX instructions provided in the

NSX Installation Guide (see Appendix B to access the document link), ensure that the prerequisites

outlined in that section, have been addressed.


11. After the prerequisites have been addressed, log in to the vSphere web Client.

12. Click Home, Networking & Security, Installation, and then click the Host Preparation tab.

13. Click Actions and then Install.

Installing NSX on ESXi servers

14. Repeat the same procedure for the management cluster on the management vCenter, which is

managed by a separate NSX Manager.

15. For the Resource cluster, configure VXLAN by selecting Not Configured in the VXLAN column and

edit the VXLAN Settings.

Note: The segment ID range is 5000 – 7999.

Addition of local networks


10 Add a transport zone A transport zone defines the span of a logical switch/network, and defines a collection of ESXi hosts that can

communicate with each other in the physical infrastructure. Communication between ESXi hosts in the

underlay happens with a VXLAN tunnel end point (VTEP) IP as source and destination. It is important to

understand the relationship between the VTEP, a VMware vSphere Distributed Switch (VDS), transport zone,

and logical switch to understand the VMware NSX VXLAN-based overlay networking.

Each ESXi host is identified by the NSX manager with the help of a unique VTEP IP assigned during the host

preparation process of the NSX installation. A VDS is a group of VTEPs and uplink ports, part of a given

cluster. A VDS can be centrally configured and managed through vCenter networking. A transport zone

combines compute and edge VDSs, with a logical switch typically associated with it. Broadcast domain of the

L2 logical switch is limited by the scope of the transport zone. By this definition, the scope of a logical switch

can extend across multiple clusters. Hosts in the management cluster never have to be part of the transport

zone, as logical networks should not span across management hosts.

Transport zone configuration in NSX Manager

To add a transport zone, perform the following steps:

1. Log in to the vSphere Web Client.

2. From the vSphere Web Client main screen, click Home, Networking & Security, and then

Installation.

3. Click the Logical Network Preparation tab.

4. Click Transport Zones, and then click the green New Transport Zone icon.

The New Transport Zone screen displays.

5. In the New Transport Zone dialog box, enter the desired name in the Name: field and a description

for the transport zone, in the Description: field.

6. Click to select one of the following control plane mode options:

- Multicast – Multicast used in the physical network are used for the VXLAN control plane

- Unicast – VXLAN control plate handled by the NSX Controller Cluster

- Hybrid – Offloads local traffic replication to a physical network

7. Click to place a check in the box next to each of the clusters to be part of the Transport Zone.

8. Click OK.


11 VMware Integrated OpenStack Manager The following image explains the implementation architecture of VMware vCloud NFV OpenStack:

VMware vCloud NFV OpenStack edition architecture

Using an integrated OpenStack Manager, the OpenStack deployment leverages the VMware vCenter vApp

process. The configure cluster, networking, computes, storages, and post-deployment processes are used for

scale. Depending on datacenter requirements, VIO can deploy in either compact, as a lab level, or in HA, as

production level, in either 2-pod design or 3-pod designs. This document covers the 2-pod design.

Note: Virtualized network function, or VNF, launches from VIO, and not from VMware vSphere.

VIO supports OpenStack image format types such as QCOW2, IMG, VMware VMDK, and the Heat

Orchestration Template, or HOT.

11.1 Deploy VMware Integrated OpenStack Manager To deploy the VIO manager:

1. From the Management cluster, initiate Deploy OVF Template from the cluster view.

2. Obtain from VMware-OpenStack--4.0.0.0-6437860_OVF10.ova from VMware and place in current

vSphere client view location.

3. Click Browse to location VMware-OpenStack--4.0.0.0-6437860_OVF10.ova file.


Deploy OVF template selection screen

4. In the Name field, enter the desired name for the VIO Manager.

Note: A default name is inherited from the OVA file name that is selected in step 3.

OVF template selection and deployment screen

5. Click Next.

6. Click to select the management cluster as the VIO manager’s cluster resource, then click Next.


Management cluster selection screen

7. Review the template details and if no other changes are necessary, click Next.

Review of selected template details

8. Review the information provided within the VMware End User License Agreement (EULA), and to

agree to the terms, click Accept, then click Next.


EULA review and acceptance screen

9. From the Select storage screen, select vsanDatastore as the VIO Manager storage resource, then

click Next.

vSAN storage selection screen

10. Select the destination network for each VIO resource network, then click Next.


VIO Manager Management network

11. From the Customize template screen, enter the required Network properties, Remote Syslog

Server, Time Synchronization, and VIO user password configuration in each of the fields

provided, then click Next.

Customize template screen

12. Review the vService bindings provided and select the vService that the OVF template is bound to,

then click Next.


vService binding selection

13. Review the configurations selected and if no changes are required, click Finish.

Review the settings

The VIO Manager deploys inside the Manager Cluster and within the extension to vCenter, which may take

approximately 20 minutes to complete.


11.2 Deploy VMware Integrated OpenStack To deploy the VMware Integrated OpenStack, or VIO, click the icon in the Inventories section, or in the left-

hand navigation panel.

VIO deployment icons

11.2.1 2-pod topology For 2-pod design:

1. SSH in to the VIO Manager (VIO OpenStack Management Service) using the VIO user credentials.

2. Switch to root user then enter the following commands:

sudo su –

cd /opt/vmware/vio/etc

Add “oms.allow_shared_edge_cluster = true” into omjs.properties service

oms restart

11.3 Deploy OpenStack VMware Integrated OpenStack, or VIO, provides services automate the deployment and management of an

OpenStack cloud.

11.3.1 Initiate OpenStack deployment To deploy VMware Integrated OpenStack:

1. From the VMware vSphere Web Client, click the VIO icon in the Inventories section, or in the left-



2. In the Getting Started tab, locate and click the Deploy OpenStack icon from the Basic Tasks

section.

VMware Integrated OpenStack deployment screen

3. From the Select a deployment method section within the Deploy OpenStack wizard, click the Use

this wizard to configure a new OpenStack instance option.

4. Click the Deployment type drop-down menu and select HA.

Select a deployment method screen

5. Click Next.

6. In the Review deployment process section, enter the desired name in the Deployment name: field.

Note: Verify that the Use management vCenter Server as Compute vCenter Server box is not selected.

7. In the Compute vCenter Server: field, enter the FQDN or IP that matches the certificate of the

Compute vCenter Server.

Note: If the certificate of the Compute vCenter Server does not match the management vCenter Server, the

deployment will fail.


8. Enter the Username, Password, and Availability Zone information in the fields provided, then click

Next.

Review deployment process screen

9. From the Select the management cluster screen, select the cluster to deploy the virtual machines,

then click Next.

Cluster selection for virtual machine deployment

10. In the Configure management networking screen, enter the network settings for both the

Management Network and OpenStack API Access Network Setting sections.


Configure management networking screen

11. Click Next.

12. In the Configure the load balancer section, enter the Public hostname and Public Virtual IP

information in the fields provided, then click Next.

Configure the load balancer screen

13. In the Add Nova cluster section, click to select the cluster to be used by the instance VMs that are

created through OpenStack, then click Next.


Add Nova cluster selection screen

14. From the Add Nova datastores screen, click to select the datastore to be used by Nova for creating

instances, then click Next.

Add Nova datastores selection screen

15. Within the Add Glance datastores screen, click to select the datastore to be used by the OpenStack

Glance service to store images, then click Next.


Add Glance datastores selection screen

16. From the Configure Neutron networking screen, click to select the Virtual Distributed Switch

Networking option.

17. Using the drop-down menu, select Tenant-vds for the Virtual Distributed Switch then click Next.

Configure Neutron networking screen

18. Within the Configure Neutron networking screen, locate and select the NSX Networking option.

Note: Use the NSX Manager IP instead of DNS. If the deployment with NSX fails, check NSX Manager in

Networking and Security (NSX Manager Managed), add VIO VMs to the exclusion list.

19. Enter the required Manager address, Username, and Password information in the fields provided

then click Next.


Configure Neutron networking screen

20. To configure the authentication source, click the Enable box to allow Active Directory as LADP

Backend.

21. In the fields provided, enter the Domain name, Bind user, Bind password, and the Domain

controllers information.

Note: For advanced configuration of DN Binding, consult AD Administrator to obtain detail DN.

Configure authentication source screen

22. To configure the syslog server, specify the IP/syslog server address, port, and protocol information in

the fields provided, then click Next.


Configure syslog server

23. Review the information regarding the Customer Experience Improvement Program, and click to select

the Join the VMware Customer Experience Program box to enroll, then click Next.

Customer Experience Improvement Program enrollment screen

24. Review the selected settings within the Ready to complete screen. If no additional changes are

required, click Finish.


Ready to complete screen

The OpenStack Deployment begins and displays the progress in the column provided.

OpenStack Deployment progress screen

25. When complete, the deployment changes from Provisioning in the Status column, to Running.

VIO deployment status


26. To view the nodes associated with the OpenStack deployment, double-click the VIO listing within the

OpenStack Deployment tab.

OpenStack Deployment listing

The Nodes associated with the OpenStack deployment, display.

OpenStack Deployment nodes

11.3.2 Review current VIO deployment To review the current VIO deployment, perform the following steps:



2. In the VMware Integrated OpenStack screen, click the Configuration tab.

The current VIO deployment information displays.


VIO deployment information

11.3.3 Manage or modify VIO settings When necessary, you can manage or modify the following VIO settings from the options provided:

Syslog

VIO Networking Access

Nova Compute

Nova Storage

Glance Storage

Updates

Upgrades

Horizon Region

Controller

Compute vCenter Server

To manage or modify any of these sections, perform the following steps:



2. In the VMware Integrated OpenStack screen, click the Manage tab.

3. Click to select the option to modify then click the Edit button.

4. After making the required changes, click Save.


VIO Manage options

11.3.4 Post deployment modifications for VIO Changes to an OpenStack Deployment can be made after the initial deployment is complete. The following

changes can be made to the VIO:

Note: Before editing any of the OpenStack Deployment services, the OpenStack service must first be

stopped. Select Stop OpenStack Services from the options provided.

Edit Deployment - From the OpenStack Deployment tab, right-click on the desired deployment and

select Edit OpenStack Deployment.

Export Configuration - Export configuration to create the back-up configuration as a JSON file.

Retry OpenStack Deployment - From the OpenStack Deployment tab, right-click on the desired

deployment and select Retry OpenStack Deployment.

Note: If the Retry OpenStack Deployment option is available for selection, then the initial deployment was

incomplete. Review the deployment's configuration settings then check to see if the option to Retry

OpenStack Deployment is grayed out.

Delete OpenStack Deployment – The OpenStack service must be stopped prior to deleting the

deployment

Stop OpenStack Services – Stops the OpenStack Services

Start OpenStack Services – Starts the OpenStack Services


Post deployment modification options for OpenStack Deployment

11.4 Access VIO This section details the various options available after completing the installation of VMware Integrated

OpenStack.

11.4.1 Launch VIO Horizon dashboard To launch the VMware Horizon dashboard, access the VMware Integrated OpenStack Virtual IP using a

browser, and log in using an Administrator account, embedded account, or Active Directory account.

VIO Horizon dashboard


From the VIO Horizon home screen, the following options are available:

Project – Option to create new tenants and adjust tenant limits

Admin – Manage VIO infrastructure

Identity – Modify user rules and manage domains, projects, or goals

VIO Horizon Projects screen

11.4.2 Use CLI to access VIO components Using SSH, log in to the VMware Integrated OpenStack Manager as viouser.

1. To display the OpenStack inventory path used for the current deployment:

sudo viocli show -p

/var/lib/vio/jarvis/47faf129-5dee-47d4-9a15-02e84044264f/keystone-backend-

req-e0bdd229-32d4-4a2e-b4b9-31898f0cd63b-hosts.ini

a. The output displayed is similar to the following by type in "more /var/lib/vio/jarvis/47faf129-5dee-

47d4-9a15-02e84044264f/keystone-backend-req-e0bdd229-32d4-4a2e-b4b9-31898f0cd63b-

hosts.ini"

[all:vars]

multi_vc = True

deployment_version = 4.0.0

deployment_type = large

region_name = nova

default_availability_zone = nova

cinder_volumes = "{\"nova:vcenter-edge-resource.r1r1.dellemc.local\":

{\"vcenter_ip\": \"vcenter-edge-resource.r1r1.dellemc.local\",

\"vcenter_insecure\": \"false\", \"vcenter_user\":

\"[email protected]\", \"cluster_name_list\": [\"r1r1-edge-

resource-cluster\"], \"availability_zone_name\": \"nova\"}}"

public_hostname = vio-lb.r1r1.dellemc.local

internal_vip = 172.17.4.150

public_vip = 192.168.100.2

syslog_server_protocol = udp

vcenter_certificate =


glance_datastores = r1r1-dellemc:vsanDatastore:100

vcenter_ip = vcenter-edge-resource.r1r1.dellemc.local

nsxv_username = admin

nsxv_external_network_moref = dvportgroup-64

nsxv_datacenter_moref = datacenter-2

vcenter_insecure = False

cinder_folder = Volumes

syslog_server = syslog.r1r1.dellemc.local

token_expiration_time = 7200

nsxv_metadata_proxy_netmask = 255.255.255.0

admin_tenant_name = admin

nsxv_management_network_moref = dvportgroup-63

nsxv_exclusive_router_appliance_size = large

nsxv_dvs_moref = dvs-61

vcenter_user = [email protected]

glance_folder = /images

nsxv_metadata_default_gateway = 172.17.1.254

nsxv_metadata_proxy_ips = 172.17.1.50,172.17.1.51

syslog_server_port = 514

nova_tenant_vdc = True

admin_user = administrator

management_default_gateway = 172.17.4.254

nsxv_certificate =

nsxv_manager = 172.17.4.205

horizon_regions = "AVAILABLE_REGIONS =

[('http://172.17.4.150:5000/v3','VMware_Openstack'),]"

nsxv_insecure = True

nsxv_resource_pool_moref = resgroup-132

nsxv_datastore_moref = datastore-47

neutron_backend = nsxv

management_network = 172.17.4.0

nsxv_metadata_network = 172.17.1.0

syslog_server_tag = VMware_Openstack

nsxv_vdn_scope_moref = vdnscope-2

nsxv_edge_ha = False

keystone_backend = sql

nova_passthrough = True

deployment_name = VMware_Openstack

[compute]

172.17.4.158 datastore_regex="vsanDatastore" vcenter_ip=vcenter-edge-

resource.r1r1.dellemc.local hostname=compute01 cluster_name=r1r1-edge-

resource-cluster [email protected]

cluster_moid=domain-c43 vcenter_insecure=False vcenter_uuid=e609187e-7f08-

4693-8449-1684dc0541e5

[lb]


172.17.4.156 hostname=loadbalancer01

172.17.4.157 hostname=loadbalancer02

[db]

172.17.4.151 hostname=database01



[mq]




[controller]

172.17.4.154 hostname=controller01


[memcache]



[storage]



b. Determine the host want to ssh into from the step above:

- ssh controller01

2. While still in sudo mode, edit the /root/cloudadmin.rc file.

3. Add OS_PASSWORD=<administrator password> into the file.

4. Enter the source ~/cloudadmin.rc command.

The following sections depict the options available from this point:

11.4.2.1 Project or tenant To list a project or tenant, use the following command:

openstack project list

To create a new project/tenant: use the following command

openstack project create <project/tenant name> --enable

11.4.2.2 Create object storage

openstack service create –name=swift \

object-store –description=”VIO Object Storage”


VIO Object Storage command

11.4.2.3 Neutron networking

neutron router-create –router_type=exclusive tenant_router –tenant-id

<project/tenant id>

Neutron router creation

neutron net-create tenant1_network –tenant-id <project/tenant id>

Neutron network creation

neutron subnet-create --name tenant1_subnet --tenant-id

8656195bfde54a858271e2c05c6e7b61 --allocation-pool

start=10.10.10.5,end=10.10.10.15 --ip-version 4 tenant1_network

10.10.10.0/24


Neutron subnetwork creation

11.4.2.4 Key pair To create key pair for Linux based VMs:

1. Generate an SSH public key from the host machine or from the source of the SSH session.

2. Upload id_rsa.pub into VIO manager.

Openstack keypair create –public-key id_rsa.pub <name of this key pair>

11.4.2.5 Glance image To create an image list:

Glance image-create --name cirros --file cirros-0.3.5-x86_64-disk.img --

disk-format iso --container-format bare

Glance image creation

11.4.2.6 Nova flavor To create a flavor:

openstack flavor create <flavor name> --ram <ram size> --vcpu <cpu> --disk

<size of disk> --id <id if need fixed id> --project <project id>


11.4.2.7 Heat stack To create a heat stack:

1. Verify that the following two file types are present:

> Heat environment file

> Heat template YAML files

2. Enter the following OpenStack Stack creation command:

openstack stack create -f yaml -e <environment.yaml> -t

<heat_deployment.yaml> sample-heat-name1


12 Install and Deploy VMware vRealize Operations Manager To install and deploy the VMware vRealize Operations Manager, perform the following steps:

1. Highlight the management cluster and deploy the vRealize Operations Manager OVA template.

2. Designate the name, location, and configuration size of the environment to be used for the appliance.

Note: For a POC environment, select a small size.

3. Click to highlight the vSAN datastore for the storage location.

4. Select the management network as the destination network for Network 1.

5. Customize the template according to the parameters of your network.

6. Review the settings then click Finish.

7. Power on the virtual appliance and use a web browser to configure the newly added node.

8. From the new cluster user interface (UI), select Express Installation to create a new vRealize

Operations Manager cluster.

Note: This node becomes the master.

9. Enter a password for the admin user, then complete the setup.

10. After a few minutes, the UI prompts for user name and password.

11. The application wizard prompts you to enter a license to complete the installation.

12. After the installation is complete, use the vRealize Operations Manager to add the Management and

Edge/Resource vCenter.

13. Click Administration, Solutions, VMware vSphere, and then Configure.

Addition of vCenter to vRealize Operations Manager

14. Click the + sign to enter the required credentials to the vCenter.


vCenter SSO credentials

15. Click Test Connection, then click Save Settings.

16. Repeat the steps for the Edge/Resource vCenter.

17. The vRealize Operations Manager begins collecting information about vCenter and its linked hosts

and services.

18. Click the Home icon to display information.

VMware vRealize Home screen with vCenter additions


13 Install and Deploy VMware vRealize Log Insight 4.3 The VMware vRealize Log Insight is a virtual appliance that must be deployed in the vSphere environment.

Before installing vRealize Log Insight, consider the following:

Verify that you have a copy of the vRealize Log Insight virtual appliance .ova file

You have the necessary authorization or permission to deploy OVF templates

Meet the minimum resource requirements to perform the installation

You are aware of the appliance sizing recommendations

Once these prerequisites have been addressed, proceed with the installation and deployment process:

1. Highlight the Management cluster and deploy the vRealize Log Insight Manager OVA template.

2. Designate the name and location to be used for the appliance.

3. Review the details of the template then click Next.

The license agreement displays.

4. Click Accept to approve the terms outlined in the license agreement, then select the size of the

environment to be configured.

Note: For a POC environment, select a small size.

5. Click to highlight the vSAN datastore for the storage location.

6. Select the management network as the destination network for Network 1.

7. Customize the template according to the parameters of your network.

8. Review the settings then click Finish to initiate the import and deployment of the OVF package.

9. Once the template is deployed, start the virtual machine (VM).

10. Open a web browser then navigate to the following URL through the wizard:

https://<vloginVM IP address>

The Welcome to VMware Log Insight window displays.

11. Click the Next button to continue.

VMware vRealize Log Insight Welcome message

The Choose Deployment Type window displays.

12. Click the Start New Deployment button to continue.


VMware vRealize Log Insight Choose Deployment window

The Admin Credentials window displays.

13. In the Admin Credentials window, enter the Email, New password, and Confirm new password

information in the required fields.

VMware vRealize Log Insight credentials

14. Click the Save and Continue button.

15. On the vSphere Integration screen, enter the Hostname, Username, and Password credentials in

the required fields as shown below:

VMware vSphere Integration license credentials

16. Repeat the previous step to add the Edge/Resource vCenter and the vRealize Operations server.


17. Log in to the vRealize Operations Manager and add the vRealize Log Insight server.

Note: The deployment may take several minutes to complete.

Addition of VMware vRealize Log Insight to VMware vRealize Operations

18. Click the Dashboards tab to access the event information from vCenter.

VMware vRealize Log Insight dashboard


14 NSX Manager data backup and restore VMware recommends that a scheduled backup of the NSX Manager data be performed immediately following

the installation of the NSX Manager.

Note: For complete instructions on performing a backup of the NSX manager, see the NSX Administration

Guide and follow the steps listed in the Backup and Restore section on page 345.

https://docs.vmware.com/en/VMware-NSX-for-vSphere/6.3/nsx_63_admin.pdf

https://docs.vmware.com/en/VMware-NSX-for-vSphere/6.3/nsx_63_admin.pdf


A Switch configurations

The following sections provide the necessary configurations required for the VLAN to connect to the network:

A.1 S6010 top configuration

Username: dellnfv

Target Name: S6010a

Password:

******

You are now connected to the target.

******

Leaf-1>

Leaf-1>Jan 23 23:30:17: %STKUNIT1-M:CP %SEC-5-LOGIN_SUCCESS: Login successful on

console

Leaf-1>

Leaf-1>en

The SupportAssist EULA acceptance option has not been selected. SupportAssist

can be enabled once the SupportAssist EULA has been accepted. Use the:

'support-assist activate' command to accept EULA and enable SupportAssist.

Leaf-1#Jan 23 23:30:20: %STKUNIT1-M:CP %SEC-4-ENABLE_PASSW_NOT_CONFIGURED:

Enable password is required for authentication but not configured (by default

from console)

Jan 23 23:30:20: %STKUNIT1-M:CP %SEC-5-AUTHENTICATION_ENABLE_SUCCESS: Enable

authentication success on console

Leaf-1#sh run

Current Configuration ...

! Version 9.10(0.1)

! Last configuration change at Thu Dec 8 09:32:32 2016 by default

!

boot system stack-unit 1 primary system: A:

boot system stack-unit 1 secondary system: B:

boot system stack-unit 1 default system: A:

!

hardware watchdog stack-unit 1







!

hostname Leaf-1

!

protocol lldp

!

redundancy auto-synchronize full

!

protocol spanning-tree rstp

no disable

hello-time 1

max-age 6

forward-delay 4

bridge-priority 0

!

vlt domain 1

peer-link port-channel 127

back-up destination 192.168.0.2

primary-priority 1

system-mac mac-address 02:11:22:33:44:01

unit-id 0

!

stack-unit 1 quad-port-profile

2,4,6,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,26,28,30,32

!

stack-unit 1 provision S6010-ON

!

stack-unit 1 port 11 portmode quad

!


!


!


!


!


!

interface fortyGigE 1/1

description To Mgmt-1 Fo1/53

no ip address

!

port-channel-protocol LACP

port-channel 126 mode active


no shutdown

!


description To Leaf-2 Fo1/2

no ip address

no shutdown

!



no ip address

no shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!

interface TenGigabitEthernet 1/11/1

no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!



no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport


no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216


portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!



no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown


!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!

interface ManagementEthernet 1/1

ip address 192.168.0.1/30

no shutdown

!


no shutdown

!


shutdown

!



shutdown

!


shutdown

!


shutdown

!

interface Port-channel 126

description To Mgmt-1 Po126

no ip address

portmode hybrid

switchport

vlt-peer-lag port-channel 126

no shutdown

!


description vlt interconnect

no ip address

channel-member fortyGigE 1/2,1/3

no shutdown

!

interface Vlan 1

!untagged Port-channel 126-127

!

interface Vlan 30

description VMware vMotion Network

no ip address

tagged TenGigabitEthernet 1/11/1-1/16/4

tagged Port-channel 126

no shutdown

!

interface Vlan 35

description VMware vMotion Resource

no ip address



no shutdown

!

interface Vlan 40

description Virtual SAN Network

ip address 192.168.1.2/24

mtu 9216



no shutdown


!

interface Vlan 45

description Virtual SAN Resource Network

no ip address



no shutdown

!

interface Vlan 50

description VMware vSphere Replication

no ip address



no shutdown

!

interface Vlan 60


no ip address



no shutdown

!

interface Vlan 2123

description External Network

no ip address



no shutdown

!

interface Vlan 2124

description ESXi Management

no ip address


untagged TenGigabitEthernet 1/11/1-1/16/4

no shutdown

!

line console 0

line vty 0

line vty 1

line vty 2

line vty 3

line vty 4

line vty 5

line vty 6

line vty 7

line vty 8

line vty 9


!

reload-type

boot-type normal-reload

config-scr-download enable

!

end

Leaf-1#

A.2 S6010 bot configuration

Username: dellnfv

Target Name: s6010b

Password:

Invalid username, target name or password.

Username: dellnfv

Target Name: S6010b

Password:

******


******

Leaf-2>Jan 24 00:28:03: %STKUNIT1-M:CP %SEC-5-LOGIN_SUCCESS: Login successful on

console

Leaf-2>en




Leaf-2#Jan 24 00:28:06: %STKUNIT1-M:CP %SEC-4-ENABLE_PASSW_NOT_CONFIGURED:


from console)



Leaf-2#sh run


! Version 9.10(0.1)

! Last configuration change at Wed Dec 7 09:16:30 2016 by default

! Startup-config last updated at Wed Dec 7 09:16:31 2016 by default

!

boot system stack-unit 1 primary system://A


boot system stack-unit 1 secondary system://B

boot system stack-unit 1 default system://A

!







!

hostname Leaf-2

!

protocol lldp

!


!

protocol spanning-tree rstp

no disable

hello-time 1

max-age 6

forward-delay 4

!

vlt domain 1

peer-link port-channel 127

back-up destination 192.168.0.1

system-mac mac-address 02:11:22:33:44:01

unit-id 1

!

stack-unit 1 quad-port-profile

2,4,6,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,26,28,30,32

!

stack-unit 1 provision S6010-ON

!


!


!


!


!


!


!


description To Mgmt-1 Fo1/54


no ip address

!



no shutdown

!



no ip address

no shutdown

!



no ip address

no shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

portmode hybrid


switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address


mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

mtu 9216

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid


switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address

portmode hybrid

switchport

no shutdown

!


no ip address


portmode hybrid

switchport

no shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown


!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


ip address 192.168.0.2/30

no shutdown

!


no shutdown

!


no shutdown

!


no shutdown

!


no shutdown

!


no shutdown

!


description To Mgmt-1 Po126

no ip address

portmode hybrid

switchport

vlt-peer-lag port-channel 126

no shutdown


!


description vlt interconnect

no ip address

channel-member fortyGigE 1/2,1/3

no shutdown

!

interface Vlan 1

!untagged Port-channel 126-127

!

interface Vlan 30

description VMware vMotion Network

no ip address



no shutdown

!

interface Vlan 35


no ip address



no shutdown

!

interface Vlan 40


no ip address

mtu 9216



no shutdown

!

interface Vlan 45


no ip address



no shutdown

!

interface Vlan 50


no ip address



no shutdown

!

interface Vlan 60



no ip address



no shutdown

!

interface Vlan 2123


no ip address



no shutdown

!

interface Vlan 2124


no ip address


untagged TenGigabitEthernet 1/11/1-1/16/4

no shutdown

!

line console 0

line vty 0

line vty 1

line vty 2

line vty 3

line vty 4

line vty 5

line vty 6

line vty 7

line vty 8

line vty 9

!

reload-type



!

end

Leaf-2#


A.3 S4048tmgmt configuration

Username: dellnfv

Target Name: S4048

Password:

******


******

Jan 23 23:59:31: %STKUNIT1-M:CP %SEC-5-LOGIN_SUCCESS: Login successful on

console

Mgmt-1>en




Mgmt-1#Jan 23 23:59:35: %STKUNIT1-M:CP %SEC-4-ENABLE_PASSW_NOT_CONFIGURED:


from console)



Mgmt-1#sh run


! Version 9.10(0.1)

!

boot system stack-unit 1 primary system: A:

boot system stack-unit 1 secondary system: B:

boot system stack-unit 1 default system: A:

!







!

hostname Mgmt-1

!

protocol lldp

advertise management-tlv management-address system-name

!



!

username dellnfv password 7 c631ec61d69f34f4c30557870d3323e7 privilege 15

username admin password 7 1d28e9f33f99cf5c privilege 15

!

stack-unit 1 provision S4048T-ON

!

interface TenGigabitEthernet 1/1

description idrac-F940SD2-172.16.124.11

no ip address

switchport

no shutdown

!


description idrac-F932SD2-172.16.1ion idrac-F933SD2-172.16.124.13

no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


description idrac-F931SD2-172.16.124.14

no ip address

switchport

no shutdown

!


description idrac-DPHCSD2-172.16.124.15

no ip address

switchport

no shutdown

!


description idrac-DPHBSD2-172.16.124.16

no ip address

switchport

no shutdown

!


description idrac-DPHDSD2-172.16.124.17

no ip address

switchport

no shutdown


!


description idrac-DPH9SD2-172.16.124.18

no ip address

switchport

no shutdown

!


description idrac-DPW7SD2-172.16.124.19

no ip address

switchport

no shutdown

!



no ip address

switchport

no shutdown

!



no ip address

switchport

no shutdown

!



no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!



no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport


no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!



no ip address

switchport

no shutdown

!


no ip address

switchport

no shutdown

!


no ip address

no shutdown

!


no ip address

no shutdown

!


no ip address

no shutdown

!


no ip address

no shutdown

!


no ip address

no shutdown

!


no ip address

no shutdown

!


no ip address

no shutdown

!


no ip address

no shutdown

!


no ip address

switchport

no shutdown

!



no ip address

switchport

no shutdown

!


no ip address

no shutdown

!


description To Core Lab Infra

no ip address

switchport

no shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!


no ip address

shutdown

!



no ip address

!



no shutdown

!



no ip address

!



no shutdown

!



no ip address

shutdown

!


no shutdown

!


no shutdown

!


no shutdown

!


shutdown

!


shutdown

!


description To Leaf-1 Leaf-2 Po126

no ip address

portmode hybrid

switchport

no shutdown

!

interface Vlan 1

!untagged Port-channel 126

!

interface Vlan 2

description VLT Heartbeat

no ip address

untagged TenGigabitEthernet 1/45-1/46

shutdown

!

interface Vlan 30

description VMware vMotion Mgmt&Edge

ip address 192.168.2.1/24


no shutdown

!

interface Vlan 35


ip address 192.168.4.1/24


no shutdown

!


interface Vlan 40


ip address 192.168.1.1/24


no shutdown

!

interface Vlan 45


ip address 192.168.3.1/24


no shutdown

!

interface Vlan 50

description Tenant Management

ip address 192.168.5.1/24


no shutdown

!

interface Vlan 60


ip address 192.168.6.1/24


no shutdown

!

interface Vlan 2123


ip address 172.16.123.2/24

tagged TenGigabitEthernet 1/48


no shutdown

!

interface Vlan 2124


ip address 172.16.124.2/24

tagged TenGigabitEthernet 1/48


untagged TenGigabitEthernet 1/1-1/36

no shutdown

!

management route 0.0.0.0/0 172.16.123.1

!

ip route 0.0.0.0/0 172.16.124.1

!

no ip telnet server enable

!

ip ssh server enable

!


line console 0

line vty 0

line vty 1

line vty 2

line vty 3

line vty 4

line vty 5

line vty 6

line vty 7

line vty 8

line vty 9

!

ip dhcp server

excluded-address 172.16.124.1 172.16.124.199

excluded-address 172.16.124.254

!

pool 1

network 172.16.124.0/24

default-router 172.16.124.1

domain-name nfv.local

dns-server 8.8.8.8 8.8.4.4

lease 0 0 1

!

pool 2

network 192.168.1.0/24


domain-name vsphere.local

dns-server 8.8.8.8 8.8.4.4

lease 0 6 0

!

pool 3

network 192.168.2.0/24



dns-server 8.8.8.8 8.8.4.4

lease 0 6 0

!

pool 4

network 192.168.3.0/24



dns-server 8.8.8.8 8.8.4.4

lease 0 6 0

!

pool 5

network 192.168.4.0/24




dns-server 8.8.8.8 8.8.4.4

lease 0 6 0

!

pool 6

network 172.16.123.0/24



dns-server 8.8.8.8 8.8.4.4

lease 0 0 1

!

pool 7

network 192.168.5.0/24



dns-server 8.8.8.8 8.8.4.4

lease 0 6 0

!

pool 8

network 192.168.6.0/24



dns-server 8.8.8.8 8.8.4.4

lease 0 6 0

no disable

!

reload-type



!

end

Mgmt-1#


B NUMA

The Dell EMC NFV Ready Bundle for VMware with VMware Integrated OpenStack provides the ability to

enable NUMA optimization and CPU pinning support on all Nova compute nodes in the solution.

Non-uniform memory access or NUMA allows multiple CPUs to share local memory, which improves

performance due to improved memory access times.

Determine Edge/Resource Cluster hosts physical network adapters for NUMA configuration

SSH into each ESXi host

vsish -e get /net/pNics/vmnic<id>/properties | grep 'Device NUMA Node'

For overlay networks, all VTEPs allocate physical NICs of VDS Uplink portgroup to associate NUMA

node as a team policy

For VLAN based network, from VIO Manager to access Neutron CLI,

neutron net-create --provider:network_type portgroup <numa_network_name>

Define NUMA in VIO flavor:

nova flavor-key <uuid> set vmware:extra_config='{"numa.nodeAffinity":

"node_ID"}'

openstack flavor set --property

vmware:extra_config='{"numa.nodeAffinity":"node_id"}' <flavor name>


C Documentation resources

The following documents that have been referenced throughout this document, are available to reference or

download:

VMware vCloud NFV 2.0 Release Notes

VMware vCloud NFV 2.0 Reference Architecture

Dell EMC Open Networking – S6010-ON

Dell EMC Open Networking – S4048T-ON

vSphere Installation and Setup for VMware vSphere 6.5, VMware ESXi 6.5, and vCenter Server 6.5

vSphere Networking VMware vSphere 6.5, VMware ESXi 6.5, and vCenter Server 6.5

Administering VMware Virtual SAN

VMware vSAN Ready Node Configurator

VMware NSX for vSphere 6.3.1 Release Notes

NSX Installation Guide

VMware Integrated OpenStack 4.0 Release Notes

VMware Integrated OpenStack 4.0 Installation Guide

VMware vRealize Operations Manager 6.5 Information Center

vRealize Log Insight Product Page

http://pubs.vmware.com/Release_Notes/en/vcloud-nfv/20/VMware-vCloud-NFV-20-Release-Notes.html

https://www.vmware.com/techpapers/2017/vmware-vcloud-nfv-2-reference-architecture.html

http://www.dell.com/support/home/us/en/04/product-support/product/networking-s6010-on/manuals

http://www.dell.com/support/home/us/en/04/product-support/product/networking-s4048t-on/manuals

https://docs.vmware.com/en/VMware-vSphere/6.5/vsphere-esxi-vcenter-server-65-installation-setup-guide.pdf

https://docs.vmware.com/en/VMware-vSphere/6.5/vsphere-esxi-vcenter-server-65-networking-guide.pdf

https://docs.vmware.com/en/VMware-vSphere/6.5/virtual-san-66-administration-guide.pdf

http://vsanreadynode.vmware.com/RN/RN

https://docs.vmware.com/en/VMware-NSX-for-vSphere/6.3/rn/releasenotes_nsx_vsphere_631.html

https://docs.vmware.com/en/VMware-NSX-for-vSphere/6.3/nsx_63_install.pdf

https://docs.vmware.com/en/VMware-Integrated-OpenStack/4.0/rn/vmware-integrated-openstack-40.html

https://docs.vmware.com/en/VMware-Integrated-OpenStack/4.0/integrated-openstack-40-install-config-guide.pdf

http://pubs.vmware.com/vrealizeoperationsmanager-65/topic/com.vmware.vcom.core.doc/GUID-7E6B5805-3D2F-41C4-ADFF-B7248386E7AC.html

https://www.vmware.com/products/vrealize-log-insight.html


D Technical support and resources

Dell.com/support is focused on meeting customer needs with proven services and support.

Dell TechCenter is an online technical community where IT professionals have access to numerous resources

for Dell EMC software, hardware and services.

Storage Solutions Technical Documents on Dell TechCenter provide expertise that helps to ensure customer

success on Dell EMC Storage platforms

http://www.dell.com/support

http://en.community.dell.com/techcenter/

http://en.community.dell.com/techcenter/storage/w/wiki/2631.storage-applications-engineering