vmware sd-wan by velocloud partner guide - vmware sd-wan ...€¦ · about vmware sd-wan by...

VMware SD-WAN by VeloCloud Partner Guide

2020VMware SD-WAN by VeloCloud 3.4

You can find the most up-to-date technical documentation on the VMware website at:

https://docs.vmware.com/

If you have comments about this documentation, submit your feedback to

[email protected]

VMware, Inc.3401 Hillview Ave.Palo Alto, CA 94304www.vmware.com

Copyright © 2020 VMware, Inc. All rights reserved. Copyright and trademark information.


VMware, Inc. 2

https://docs.vmware.com/

mailto:[email protected]

http://pubs.vmware.com/copyright-trademark.html

Contents

1 About VMware SD-WAN by VeloCloud Partner Guide 5

2 What's New 6

3 Introduction 7

4 Supported Browsers 8

5 Log in to SD-WAN Orchestrator using SSO for Partner User 9

6 Monitor Customers 10

7 Manage Customers 11Create New Customer 12

Clone a Customer 14

Configure Customers 16

Configure Partner Handoff 17

8 Monitor Events 22

9 Manage Partner Admin Users 23Create New Partner Admin 23

Configure Partner Admin Users 24

10 View Partner Information 28

11 Partner Settings 29Configure Partner Information 29

Configure Partner Authentication 30

Overview of Single Sign On 31

Configure Single Sign On for Partner User 32

Configure an IDP for Single Sign On 34

12 Edge Licensing 56Manage Edge Licenses for Customers 56

Generate an Edge Licensing Report 58

13 Manage Gateway Pools 59

VMware, Inc. 3

Create a Gateway Pool 60

14 Manage Gateways 61

15 Install VMware SD-WAN Partner Gateway 62Installation Overview 62

Hypervisor Minimum Hardware Requirements 63

SD-WAN Gateway Installation Procedures 65

Pre-Installation Considerations 65

Install SD-WAN Gateway 72

Post-Installation Tasks 87

Upgrade SD-WAN Gateway 90

Custom Configurations 91

NTP Configuration 91

Userdata 91

OAM Interface and Static Routes 92

OAM - SR-IOV with vmxnet3 or SR-IOV with VIRTIO 94

Special Consideration When Using 802.1ad Encapsulation 96

SNMP Integration 97

Custom Firewall Rules 98


VMware, Inc. 4

About VMware SD-WAN by VeloCloud Partner Guide 1The VMware SD-WAN™ by VeloCloud® Partner Guide provides information about VMware SD-WAN Orchestrator including how to configure and manage Customers who use the Orchestrator.

Intended AudienceThis guide is intended for IT Partners of SD-WAN Orchestrator, who are familiar with the Networking configurations and SD-WAN operations.

VMware, Inc. 5

What's New 2What's New in Version 3.4Feature Description

Clone Enterprises Clones the configurations from an existing customer and creates a new customer with the cloned settings. See Clone a Customer.

Support for X710/XL710 NIC with DPDK and SR-IOV

SR-IOV and DPDK support for the new Intel X710/XL710 NIC. See

n Hypervisor Minimum Hardware Requirements

n Install SD-WAN Gateway on VMware

n Install SD-WAN Gateway on KVM

Token Based Authentication The users can access the Orchestrator APIs using tokens instead of session-based authentication. As a Partner Super User, you can manage the API tokens for enterprise users. See API Tokens.

For a complete list of new and updated sections to the documentation for Administrators, VMware SD-WAN by VeloCloud Administration Guide.

Previous VMware SD-WAN by VeloCloud VersionsTo get product documentation for previous VMware SD-WAN by VeloCloud versions, contact your VMware SD-WAN representative.

VMware, Inc. 6

Introduction 3As a Partner user, you can configure and manage the following:

n Partner Admin Users

n Partner Events

n Partner Settings

n Partner Authentication

n Enterprise Customers

Refer to VMware SD-WAN by VeloCloud Administration Guide to become familiar with the core function of the VMware SD-WAN used by an Enterprise IT Administrator for a customer.

VMware, Inc. 7

Supported Browsers 4For the best experience, VMware SD-WAN recommends Google Chrome or Mozilla Firefox.

The SD-WAN Orchestrator supports the following browsers.

Browsers Qualified Browser Version

Google Chrome 77 – 79.0.3945.130

Firefox 69.0.2 - 72.0.2

Internet Explorer 11.765.17134.0 - 11.592.18362.0

Microsoft Edge 42.17134.1.0- 44.18362.449.0

Safari 12.1.2-13.0.3

VMware, Inc. 8

Log in to SD-WAN Orchestrator using SSO for Partner User 5Describes how to log in to SD-WAN Orchestrator using Single Sign On (SSO) as a Partner user.

Prerequisites

n Ensure you have configured SSO authentication in SD-WAN Orchestrator. For more information, see Configure Single Sign On for Partner User.

n Ensure you have set up roles, users, and OIDC application for SSO in your preferred IDPs. For more information, see Configure an IDP for Single Sign On.

Procedure

1 In a web browser, launch an SD-WAN Orchestrator application as Enterprise or Partner user.

The VMware SD-WAN Orchestrator by VeloCloud screen appears.

2 Click Sign In With Your Identity Provider.

3 In the Enter your Organization Domain text box, enter the domain name used for the SSO configuration and click Sign In.

The IDP configured for SSO will authenticate the user and redirect the user to the configured SD-WAN Orchestrator URL.

Note Once the users log in to the SD-WAN Orchestrator using SSO, they will not be allowed to login again as native users.

VMware, Inc. 9

Monitor Customers 6A partner can monitor customer status from the Monitor Customers link.

To monitor customers:

n In the navigation bar, click Monitor Customers.

The Monitor Customers page appears.

This screen shows the Edges and Links for all customers managed by this Partner. Selections can be made to control the interval for updating the information.

The major features of the Monitor Customer page include:

1 An aggregated summary of the status of all customers and their Edges.

2 A summary of the status of each customers and their Edges.

3 Interval selections that can be made to select a specific monitoring interval.

VMware, Inc. 10

Manage Customers 7The Manage Customers menu allows you to create new customers, configure the customer capabilities, clone the existing configuration, and to configure other customer settings.

In the Partner panel, click Manage Customers > Actions to perform the following activities.

n New Customer: Creates a new customer. See Create New Customer.

n Clone Customer: Creates a new customer, by cloning the existing configurations from the selected customer. See Clone a Customer.

n Modify Customer: Navigates to the System Settings in the Enterprise portal, where you can configure other settings corresponding to the selected customer. You can also click a customer name to navigate to the Enterprise portal. For more information see the VMware SD-WAN by VeloCloud Administration Guide.

n Delete Customer: Deletes the selected customers. Ensure that you have removed all the Edges associated to the selected customer, before deleting the customer.

n Release from Partner: Releases the selected customer from the partner.

n Support Email: Sends customer support messages to the selected customer.

n Update Pre-Notifications: Enables or disables the pre-notification alerts for the selected customers.

n Update Customer Alerts: Enables or disables the alerts for the selected customers.

n Export All Customers: Exports the details of all the customers in the Partner portal to a CSV file. The default separator used is comma (,) and you can choose to edit the separator to any other special character.

n Export Customer Edge Inventory: Exports the inventory details of all the Edges associated with all the customers to a CSV file. The default separator used is comma (,) and you can choose to edit the separator to any other special character.

This chapter includes the following topics:

n Create New Customer

n Clone a Customer

n Configure Customers

VMware, Inc. 11

Create New CustomerIn the Partner portal, you can create customers and configure the customer settings.

Only Partner Super Users and Partner Standard Admins can create a new customer.

Note An Operator Super User can temporarily disable creating new customers by setting the system property session.options.disableCreateEnterprise to True. If this property is set to True, the Partner Superusers and Partner Standard Admins cannot create new customers. If you are not able to create a customer, contact your Operator to enable the option.

In the Partner portal, navigate to Manage Customers.

1 In the Customers page, click New Partner Customer or click Actions > New Customer.

2 In the New Customer window, enter the following details. You can also choose the Clone from Customer option to clone the configurations from an existing customer. For more information, see Clone a Customer.

Customer Information

Option Description

Company Name Enter your company name

Domain Enter the domain name of your company


VMware, Inc. 12

Option Description

Account Number Enter a unique identifier for the customer

Partner Support Access This option is selected by default and grants access to the Partner's Support team to view, configure, and troubleshoot the Edges connected to the customer.

For security reasons, the Support cannot access or view the user identifiable information.

VeloCloud Support Access This option is selected by default and grants access to the VMware SD-WAN Support to view, configure, and troubleshoot the Edges connected to the customer.

For security reasons, the Support cannot access or view the user identifiable information.

VeloCloud User Management Access Select the checkbox to enable the VMware SD-WAN Support to assist in user management. The user management includes options to create users, reset password, and configure other settings. In this case, the Support has access to user identifiable information.

Street Address, City, State, Country, ZIP/Postcode Enter relevant address details in the respective fields.

Initial Admin Account

Option Description

Username Enter the user name in the [email protected] format.

Password Enter a password for the Administrator.

Confirm Re-enter the password.

First Name, Last Name, Phone, Mobile Phone Enter the details like name and phone number in the appropriate fields.

Contact Email Enter the Email address. The alerts on service status are sent to this Email address.

Customer Configuration

Option Description

Software Image Choose the Software Image to be updated in the Edges.

Gateway Pool Select an existing Gateway pool from the drop-down list. For more information on Gateway pools, see Chapter 13 Manage Gateway Pools.

3 Click Create.

The new customer name is displayed in the Customers page. You can click the customer name to navigate to the Enterprise portal and add configurations to the customer. For more information, see VMware SD-WAN by VeloCloud Administration Guide.


VMware, Inc. 13

Clone a CustomerYou can clone the configurations from an existing customer and create a new customer with the cloned settings.

Only Partner Super Users and Partner Standard Admins can clone a customer.

By default, the following configurations are cloned from the selected customer:

n Enterprise configuration profiles

n Enterprise network services and objects like:

n DNS services

n Private network names

n Network Segments

n Edge authentication scheme

n Address groups and Port groups

You cannot clone an enterprise if it consists of the following:

n Profile with edge references like hubs, clusters, and so on

n Cloud Security Service enabled

n Non-VMware SD-WAN Site

n VNF or VNF licenses

n Authentication services

n NetFlow objects like collectors or filters

In the Partner portal, navigate to Manage Customers.

1 In the Customers page, select the customer you want to clone, and then click Actions > Clone Customer.

2 In the New Customer window, enter the following details. You can also choose the New Customer option to create a new customer without cloning the configurations from the selected customer. See Create New Customer.


VMware, Inc. 14

Clone Configuration

Option Description

Template Customer By default, the selected customer is considered for the cloning purpose. If required, you can choose a different customer from the drop-down list.

If a customer or enterprise does not meet the appropriate cloning conditions, as listed at the beginning of this section, then it is not available in the drop-down list. This list displays only the name of customers that can be cloned.

Additional Clone Attributes In addition to the default cloned configurations, you can select the following settings to be cloned, as required:

n Security Policy

n Alert Configuration

n Global Routing Preferences

n IAAS Subscriptions

Enter the Customer Information and Initial Admin Account details, as described in Create New Customer.

In the Customer Configuration section, the Software Image and Gateway Pool details are cloned from the selected customer. If required, you can modify the settings.

3 Click Create.


VMware, Inc. 15

The new customer name is displayed in the Customers page. The customer is already configured with the cloned settings. You can click the customer name to navigate to the Enterprise portal and add or modify the configurations. For more information, see VMware SD-WAN by VeloCloud Administration Guide.

Configure CustomersAfter creating a customer, configure the options and settings that the customer can access. As a Partner Super User, you can choose the settings the customer or Enterprise can modify.

When you create a new customer, you are redirected to the Customer Configuration page, where you can configure the customer settings.

You can also navigate to the Configuration page from the Manage Customers page in the Partner portal. Select the customer and click Actions > Modify or click the link to the customer.

In the customer or Enterprise portal, click Configure > Customer, and you can configure the following settings.

Customer Capabilities


VMware, Inc. 16

Only an Operator can enable or disable the capabilities. You can view the status of the following capabilities. If you want to enable or disbale any of the capabilities, contact your Operator.

n Enable Enterprise Auth

n Enable Firewall logging to Orchestrator

n Enable Legacy Networks

n Enable Premium Service

n Enable Segmentation

n Enable Stateful Firewall.

n CoS Mapping

n Service Rate Limiting

Maximum Segments

Displays the maximum number of segments configured by the Operator.

OFC Cost Calculation

Displays whether Distributed Cost Calculation is enabled or not by the Operator. By default, the Orchestrator calculates the cost of the routes by receiving the learned routes from Edges and Gateways. The Operator can choose to distribute the cost calculation to the Edges and Gateways, which reduces the resource consumption and load of the Orchestrator.

Edge NFV

Displays whether the customers are allowed to deploy third party Virtual Network Functions (VNF) on service ready Edge platforms.

Software Image

The current Software Image associated with the selected customer is displayed. If required, you can choose a different Software Image from the available list.

Gateway Pool

The current Gateway pool associated with the selected customer is displayed. If required, you can choose a different Gateway pool from the available list.

If the Gateways available in the Gateway pool have been assigned with Partner Gateway role, you can handoff the Gateways to partners. Select the Enable Partner Handoff to configure the handoff options for the segments and Gateways. For more information, see Configure Partner Handoff.

After making changes to the configurations, click Save Changes.

Configure Partner HandoffYou can configure a Gateway to handoff to Partners. The Gateway acts as a Partner Gateway and you can configure the Hand off Interface, Static Routes, BGP, and other settings.


VMware, Inc. 17

Ensure that the Gateway to be handed off is assigned with Partner Gateway Role. In the Partner portal, click Gateways and click the link to an existing Gateway. In the Properties section of the selected Gateway, you can enable the Partner Gateway role.

To configure the handoff settings, go to the Customer Configuration page.

n In the Partner portal, click Manage Customers.

n Select the customer and click Actions > Modify or click the link to the customer.

n In the customer or Enterprise portal, click Configure > Customer.

n In the Customer Configuration, navigate to the Gateway Pool section and select the Enable Partner Handoff checkbox.

Configure the following settings:

Customer BGP Priority

n Select Enable Community Mapping to set the Community attributes, which would be tagged in the BGP advertised routes.

n The Community mapping is set to all the segments by default. If you want to configure the Community attributes for a specific segment, choose Per Segment, and select the Segment from the drop-down list.


VMware, Inc. 18

n Select Community Additive checkbox to enable the additive option associated with a particular auto community configuration. This option preserves the incoming community attributes for a prefix received from the overlay and appends the configured auto community to the prefix, on the Partner Gateway. As a result, the MPLS PE side receives prefixes with all the community attributes including the auto community attributes.

n Enter the Community attributes in the Community and Community 2 fields. Click the Plus(+) Icon to add more community attributes.

Configure Hand Off

n By default, the handoff configuration is applied to all the Gateways. If you want to configure a specific Gateway, choose Per Gateway and select the Gateway from the drop-down list.

n By default, the handoff configuration is applied to all the Segments. If you want to configure a specific Segment, select the Segment from the drop-down list.

n For configuring all the Gateways, click the Edit option. If you have selected a particular Gateway, click the Click here to configure link.

The Hand Off Details window appears and you can configure the following:


VMware, Inc. 19

Option Description

Hand Off Interface

Tag Type Choose the tag type which is the encapsulation in which the Gateway hands off customer traffic to the Router. The following are the types tags available:

n None– Untagged. Choose this during single tenant handoff or a handoff towards shared services VRF.

n 802.1q – Single VLAN tag.

n 802.1ad / QinQ(0x8100) / QinQ(0x9100) – Dual VLAN tag.

Transport LAN VLAN This option is available only when you choose the tag type as 802.1ad / QinQ(0x8100) / QinQ(0x9100). Choose the type of tag to configure the transport VLANs.

C-Tag (Customer tag) Enter the Customer VLAN tag

S-Tag (Service tag) Enter the service-provider-defined VLAN tag

Local IP Address Enter the Local IP address for the logical Handoff interface.

Use for Private Tunnels Select the checkbox so that private WAN links connect to the private IP address of the Partner Gateway. If private WAN connectivity is enabled on a Gateway, the Orchestrator audits to ensure that the local IP address is unique for each Gateway within an enterprise.

Advertise via BGP Select the checkbox to automatically advertise the private WAN IP of the Partner Gateway through BGP. The connectivity is provided using the existing Local IP address.

Static Routes – Click the plus(+) Icon to add more routes.

Subnets Enter the IP address of the Static Route Subnet that the Gateway should advertise to the Edge.

Cost Enter the cost to apply weightage on the routes. The range is from 0 to 255.

Encrypt Select the checkbox to encrypt the traffic between Edge and Gateway.

Hand off Select the handoff type as VLAN or NAT.

Description Optionally, enter a descriptive text for the static route.

BGP

Enable BGP Select the checkbox to enable BGP and set up the BGP configuration.

Customer ASN Enter the customer Autonomous System Number.

Neighbor IP Enter the IP address of the configured Neighbor network.

Neighbor-ASN Enter the ASN of the Neighbor network.

Secure BGP Routes Select the checkbox to enable encryption for data-forwarding over BGP routes.

BGP Inbound/Outbound Filters – Click the plus(+) Icon to add more Filters.


VMware, Inc. 20

Option Description

Type (Match) Choose the type of the BGP attribute to be considered for matching with the traffic flow. You can choose either Prefix or Community.

Value Enter the value according to the BGP attribute selected as Type.

Exact Match Select the checkbox for matching the attributes exactly.

Type (Action) Choose the action to be performed if the match is True. You can either Permit or Deny the traffic.

Set You can set the values of the attributes for the routes matching the filter criteria.

Choose from the following attributes, and enter the corresponding values to be set for the matching routes:

n None – The attributes of the matching routes remain the same.

n Local Preference

n Community – You can also enable the Community Additive option.

n Metric

n AS-Path-Prepend

BGP Optional Settings

Router ID Enter the Router ID to identify the BGP Router.

Keep Alive Enter the BGP Keep Alive time in seconds. The default timer is 60 seconds.

Hold Timers Enter the BGP Hold time in seconds. The default timer is 180 seconds.

Disable AS-PATH Carry Over Select the checkbox to disable AS-PATH carry over, which influences the outbound AS-PATH to make the L3-routers prefer a path towards a PE. If you select this option, ensure to tune your network to avoid routing loops. It is recommended not to select this checkbox.

Click Update to save the settings. In addition, click Save Changes in the Customer Configuration page to activate the settings.


VMware, Inc. 21

Monitor Events 8A partner can monitor operator events generated by VMware SD-WAN from the Events link.

To view events:

n In the navigation bar, click Events.

The Events page appears.

These events can help you determine the status of the VMware SD-WAN system. For some events, you can click a link in the event to display more information.

VMware, Inc. 22

Manage Partner Admin Users 9The Admins page displays the existing partner admin users. A Partner Super User can create new partner admin users with different role privileges and configure API tokens for each partner admin.

In the Partner portal, click Admins.

Click Actions to perform the following activities:

n New Admin: Creates new partner admin users. See Create New Partner Admin.

n Modify Admin: Modifies the properties of the selected admin user. You can also click the link to the username to modify the properties. See Configure Partner Admin Users.

n Password Reset: Sends an Email to the selected user with a link to reset the password.

n Delete Admin: Deletes the selected users.


n Create New Partner Admin

n Configure Partner Admin Users

Create New Partner AdminA Partner Super User can create new partner admin users.

In the Partner portal, click Admins.

Procedure

1 You can create new admin users by clicking either New Admin, or Actions > New Admin .

VMware, Inc. 23

2 In the New Admin window, enter the following details:

a Enter the user details like username, password, Name, Email, and Phone numbers.

b If you have chosen the authentication mode as Native in Configure Partner Authentication , then the type of the user is selected as Native. If you have chosen a different authentication mode, you can choose the type of the user. If you choose the user to be Non-Native, the password option is not available, as it is inherited from the authentication mode.

c Account Role: Choose the user role from the available options.

3 Click Create.

Results

The partner admin user details are displayed in the Admins page.

Configure Partner Admin UsersYou can configure additional properties and create API tokens for an Admin user.

In the Partner portal, click Admins. To configure an Admin user, click the link to a username or select the user and click Actions > Modify Admin.

The existing properties of the selected user are displayed and if required, you can add or modify the following:


VMware, Inc. 24

Status

By default, the status is in Enabled state. If you choose Disabled, the user is logged out of all the active sessions.

Type

If you have chosen the Partner authentication mode as Native in Configure Partner Authentication , then the type of the user is selected as Native. If you have chosen a different authentication mode, you can choose the type of the user. If you choose the user to be Non-Native, then you cannot reset the password or modify the user role.

Property

The existing contact details of the user are displayed. If required, you can modify the details and choose to reset the password. If you click Password Reset, an email is sent to the user with a link to reset the password.

User Role

The existing type of the user role is displayed. If required, you can choose a different role for the user. The role privileges change accordingly.

API Tokens


VMware, Inc. 25

The users can access the Orchestrator APIs using tokens instead of session-based authentication. As Partner Super User, you can manage the API tokens for your enterprise users. You can create multiple API tokens for a user.

Configure API Tokens

Any user can create tokens based on the privileges they have been assigned to their user roles, except the Business Specialist users.

The users can perform the following actions, based on their roles:

n Enterprise users can Create, Download, and Revoke tokens for them.

n Partner Super users can manage tokens of Enterprise users, if the Enterprise user has delegated user permissions to the Partner.

n Partner Super users can only create and revoke the tokens for other users.

n Users can download only their own tokens and cannot download other users' tokens.

To manage the API tokens:

n In the API Tokens section, click Actions > New API Token, to create a new token.

n In the New API Token window, enter a Name and Description for the token, and choose the Lifetime from the drop-down menu.

n Click Create and the new token is displayed in the API Tokens grid.

n Initially, the status of the token is displayed as Pending. To download the token, select the token, and click Actions > Download API Token. The status changes to Enabled, which means that the API token can be used for API access.

n To disable a token, select the token and click Actions > Revoke API Token. The status of the token is displayed as Revoked.

n When the Lifetime of the token is over, the status changes to Expired state.

Only the user who is associated with a token can download it and after downloading, the ID of the token alone is displayed. You can download a token only once.

After downloading the token, the user can send it as part of the Authorization Header of the request to access the Orchestrator API.


VMware, Inc. 26

The following example shows a sample snippet of the code to access an API.

curl -k -H "Authorization: Token <Token>"

-X POST https://vco/portal/

-d '{ "id": 1, "jsonrpc": "2.0", "method": "enterprise/getEnterpriseUsers", "params":

{ "enterpriseId": 1 }}'

After modifying the settings and API Tokens, click Save Changes.


VMware, Inc. 27

View Partner Information 10As a Partner super user, you can view the Software Images and Gateway Pools assigned to you by your Operator.

In the Partner portal, click Overview to view the following information.

Available Software Images

Displays the software images assigned to the partner by the Operator. You can assign the software images to your Enterprise customers from this list.

Gateway Pool

Displays the Gateway pools assigned to the partner by the Operator. You can assign the Gateway pools to your Enterprise customers from this list.

Note To assign the software images and Gateway pools to a customer, see Create New Customer and Configure Customers.

VMware, Inc. 28

Partner Settings 11The Settings option allows you to configure partner settings along with the authentication details.

In the Partner portal, click Settings to configure the following:

n General Information– Configure the user details, configure privacy settings, and enter the contact information. See Configure Partner Information.

n Authentication– Configure authentication mode and view the API tokens. See Configure Partner Authentication .


n Configure Partner Information

n Configure Partner Authentication

Configure Partner InformationYou can configure the partner information, privacy settings, and contact details for the partners using General Information.

In the Partner portal, click Settings. You can configure the following in the General Information tab.

VMware, Inc. 29

Privacy Settings

n Grant Access to VeloCloud Support – Select this option to grant access to the VMware SD-WAN Support to view, configure, and troubleshoot the events and settings.

General Information

Option Description

Name The existing username is displayed. If required, you can modify the name.

Domain The existing domain name is displayed and you can modify the domain, if required.

Description Enter a description for the customer.

Contact Information

The existing contact details are displayed in this section. If required, you can modify the details.

Configure Partner AuthenticationIn the Authentication tab, you can setup the authentication mode for the partners and view the existing API tokens.

In the Partner portal, click Settings > Authentication to configure the following:


VMware, Inc. 30

Partner Authentication

Choose one of the following from the Authentication Mode.

n NATIVE – This is the default authentication mode and you can login to the Partner portal with the native username and password. This mode does not require any configuration.

n SSO – Single Sign On (SSO) is a session and user authentication service that allows the users to log into the Partner portal with one set of login credentials to access multiple applications. For more information, see Configure Single Sign On for Partner User.

API Tokens

You can access the Orchestrator APIs using token-based authentication, irrespective of the authentication mode. You can view the existing API tokens in this section.

The Partner Super User or the User associated with an API token can revoke the token. Select the token and click Actions > Revoke . To create and download the API tokens, see API Tokens.

Overview of Single Sign OnThe SD-WAN Orchestrator supports a new type of user authentication called Single Sign On (SSO) for all Orchestrator user types: Operator, Partner, and Enterprise.

Single Sign On (SSO) is a session and user authentication service that allows SD-WAN Orchestrator users to log in to the SD-WAN Orchestrator with one set of login credentials to access multiple applications. Integrating the SSO service with SD-WAN Orchestrator improves the security of user authentication for SD-WAN Orchestrator users and enables SD-WAN Orchestrator to authenticate users from other OpenID Connect (OIDC)-based Identity Providers (IDPs). The following IDPs are currently supported:

n Okta

n OneLogin

n PingIdentity

n AzureAD


VMware, Inc. 31

n VMwareCSP

Configure Single Sign On for Partner UserTo setup Single Sign On (SSO) authentication for Partner user, perform the steps on this procedure.

Prerequisites

n Ensure you have the Partner super user permission.

n Before setting up the SSO authentication in SD-WAN Orchestrator, ensure you have set up roles, users, and OpenID connect (OIDC) application for SD-WAN Orchestrator in your preferred identity provider’s website. For more information, see Configure an IDP for Single Sign On.

Procedure

1 Log in to the SD-WAN Orchestrator application as Partner super user, with your login credentials.

2 Click Settings.

The Partner Settings screen appears.

3 Click the General Information tab and in the Domain text box, enter the domain name for your partner, if it is not already set.

Note To enable SSO authentication for the SD-WAN Orchestrator, you must set up the domain name for your partner.


VMware, Inc. 32

4 Click the Authentication tab and from the Authentication Mode drop-down menu, select SSO.

5 From the Identity Provider template drop-down menu, select your preferred Identity Provider (IDP) that you have configured for Single Sign On.

Note When you select VMwareCSP as your preferred IDP, ensure to provide your Organization ID in the following format: /csp/gateway/am/api/orgs/<full organization ID>.

When you sign in to VMware CSP console, you can view the organization ID you are logged into by clicking on your username. A shortened version of the ID is displayed under the organization name. Click the ID to display the full organization ID.

You can also manually configure your own IDPs by selecting Others from the Identity Provider template drop-down menu.

6 In the OIDC well-known config URL text box, enter the OpenID Connect (OIDC) configuration URL for your IDP. For example, the URL format for Okta will be: https://{oauth-provider-url}/.well-known/openid-configuration.

7 The SD-WAN Orchestrator application auto-populates endpoint details such as Issuer, Authorization Endpoint, Token Endpoint, and User Information Endpoint for your IDP.

8 In the Client Id text box, enter the client identifier provided by your IDP.

9 In the Client Secret text box, enter the client secret code provided by your IDP, that is used by the client to exchange an authorization code for a token.

10 To determine user’s role in SD-WAN Orchestrator, select one of the options:

n Use Default Role – Allows user to configure a static role as default by using the Default Role text box that appears on selecting this option. The supported roles are: MSP Superuser, MSP Standard Admin, MSP Support, and MSP Business.


VMware, Inc. 33

https://console-stg.cloud.vmware.com/

n Use Identity Provider Roles – Uses the roles set up in the IDP.

11 On selecting the Use Identity Provider Roles option, in the Role Attribute text box, enter the name of the attribute set in the IDP to return roles.

12 In the Role Map area, map the IDP-provided roles to each of the SD-WAN Orchestrator roles, separated by using commas.

Roles in VMware CSP will follow this format: external/<service definition uuid>/<service role name mentioned during service template creation>.

13 Update the allowed redirect URLs in OIDC provider website with SD-WAN Orchestrator URL (https://<vco>/login/ssologin/openidCallback).

14 Click Save Changes to save the SSO configuration.

15 Click Test Configuration to validate the entered OpenID Connect (OIDC) configuration.

The user is navigated to the IDP website and allowed to enter the credentials. On IDP verification and successful redirect to SD-WAN Orchestrator test call back, a successful validation message will be displayed.

Results

The SSO authentication setup is complete in SD-WAN Orchestrator.

What to do next

Chapter 5 Log in to SD-WAN Orchestrator using SSO for Partner User

Configure an IDP for Single Sign OnTo enable Single Sign On (SSO) for SD-WAN Orchestrator, you must configure an Identity Partner (IDP) with details of SD-WAN Orchestrator. Currently, the following IDPs are supported: Okta, OneLogin, PingIdentity, AzureAD, and VMware CSP.


VMware, Inc. 34

For step-by-step instructions to configure an OpenID Connect (OIDC) application for SD-WAN Orchestrator in various IDPs, see:

n Configure Okta for Single Sign On

n Configure OneLogin for Single Sign On

n Configure PingIdentity for Single Sign On

n Configure Azure Active Directory for Single Sign On

n Configure VMware CSP for Single Sign On

Configure Okta for Single Sign OnTo support OpenID Connect (OIDC)-based Single Sign On (SSO) from Okta, you must first set up an application in Okta. To set up an OIDC-based application in Okta for SSO, perform the steps on this procedure.

Prerequisites

Ensure you have an Okta account to sign in.

Procedure

1 Log in to your Okta account as an Admin user.

The Okta home screen appears.

Note If you are in the Developer Console view, then you must switch to the Classic UI view by selecting Classic UI from the Developer Console drop-down list.

2 To create a new application:

a In the upper navigation bar, click Applications > Add Application.

The Add Application screen appears.

b Click Create New App.

The Create a New Application Integration dialog box appears.


VMware, Inc. 35

https://login.okta.com/

c From the Platform drop-drop menu, select Web.

d Select OpenID Connect as the Sign on method and click Create.

The Create OpenID Connect Integration screen appears.

e Under the General Settings area, in the Application name text box, enter the name for your application.

f Under the CONFIGURE OPENID CONNECT area, in the Login redirect URIs text box, enter the redirect URL that your SD-WAN Orchestrator application uses as the callback endpoint.

In the SD-WAN Orchestrator application, at the bottom of the Configure Authentication screen, you can find the redirect URL link. Ideally, the SD-WAN Orchestrator redirect URL will be in this format: https://<Orchestrator URL>/login/ssologin/openidCallback.

g Click Save. The newly created application page appears.


VMware, Inc. 36

h On the General tab, click Edit and select Refresh Token for Allowed grant types, and click Save.

Note down the Client Credentials (Client ID and Client Secret) to be used during the SSO configuration in SD-WAN Orchestrator.

i Click the Sign On tab and under the OpenID Connect ID Token area, click Edit.

j From the Groups claim type drop-down menu, select Expression. By default, Groups claim type is set to Filter.


VMware, Inc. 37

k In the Groups claim expression textbox, enter the claim name that will be used in the token, and an Okta input expression statement that evaluates the token.

l Click Save.

The application is setup in IDP. You can assign user groups and users to your SD-WAN Orchestrator application.


VMware, Inc. 38

3 To assign groups and users to your SD-WAN Orchestrator application:

a Go to Application > Applications and click on your SD-WAN Orchestrator application link.

b On the Assignments tab, from the Assign drop-down menu, select Assign to Groups or Assign to People.

The Assign <Application Name> to Groups or Assign <Application Name> to People dialog box appears.

c Click Assign next to available user groups or users you want to assign the SD-WAN Orchestrator application and click Done.

The users or user groups assigned to the SD-WAN Orchestrator application will be displayed.

Results

You have completed setting up an OIDC-based application in Okta for SSO.

What to do next

Configure Single Sign On in SD-WAN Orchestrator.

Create a New User Group in Okta

To create a new user group, perform the steps on this procedure.

Procedure

1 Click Directory > Groups.

2 Click Add Group.

The Add Group dialog box appears.

3 Enter the group name and description for the group and click Save.


VMware, Inc. 39

Create a New User in Okta

To add a new user, perform the steps on this procedure.

Procedure

1 Click Directory > People.

2 Click Add Person.

The Add Person dialog box appears.

3 Enter all the mandatory details such as first name, last name, and email ID of the user.

4 If you want to set the password, select Set by user from the Password drop-down menu and enable Send user activation email now.

5 Click Save.

An activation link email will be sent your email ID. Click the link in the email to activate your Okta user account.

Configure OneLogin for Single Sign OnTo set up an OpenID Connect (OIDC)-based application in OneLogin for Single Sign On (SSO), perform the steps on this procedure.

Prerequisites

Ensure you have an OneLogin account to sign in.

Procedure

1 Log in to your OneLogin account as an Admin user.

The OneLogin home screen appears.


VMware, Inc. 40

https://app.onelogin.com/login


a In the upper navigation bar, click Apps > Add Apps.

b In the Find Applications text box, search for “OpenId Connect” or “oidc” and then select the OpenId Connect (OIDC) app.

The Add OpenId Connect (OIDC) screen appears.

c In the Display Name text box, enter the name for your application and click Save.

d On the Configuration tab, enter the redirect URI that SD-WAN Orchestrator uses as the callback endpoint and click Save.

In the SD-WAN Orchestrator application, at the bottom of the Authentication screen, you can find the redirect URL link. Ideally, the SD-WAN Orchestrator redirect URL will be in this format: https://<Orchestrator URL>/login/ssologin/openidCallback.


VMware, Inc. 41

e On the Parameters tab, under OpenId Connect (OIDC), double click Groups.

The Edit Field Groups popup appears.

f Configure User Roles with value “--No transform--(Single value output)” to be sent in groups attribute and click Save.

g On the SSO tab, from the Application Type drop-down menu, select Web.


VMware, Inc. 42

h From the Authentication Method drop-down menu, select POST as the Token Endpoint and click Save.

Also, note down the Client Credentials (Client ID and Client Secret) to be used during the SSO configuration in SD-WAN Orchestrator.

i On the Access tab, choose the roles that will be allowed to login and click Save.

3 To add roles and users to your SD-WAN Orchestrator application:

a Click Users > Users and select a user.

b On the Application tab, from the Roles drop-down menu, on the left, select a role to be mapped to the user.

c Click Save Users.

Results

You have completed setting up an OIDC-based application in OneLogin for SSO.

What to do next



VMware, Inc. 43

Create a New Role in OneLogin

To create a new role, perform the steps on this procedure.

Procedure

1 Click Users > Roles.

2 Click New Role.

3 Enter a name for the role.

When you first set up a role, the Applications tab displays all the apps in your company catalog.

4 Click an application to select it and click Save to add the selected apps to the role.

Create a New User in OneLogin

To create a new user, perform the steps on this procedure.

Procedure

1 Click Users > Users > New User.

The New User screen appears

2 Enter all the mandatory details such as first name, last name, and email ID of the user and click Save User.

Configure PingIdentity for Single Sign OnTo set up an OpenID Connect (OIDC)-based application in PingIdentity for Single Sign On (SSO), perform the steps on this procedure.

Prerequisites

Ensure you have a PingOne account to sign in.

Note Currently, SD-WAN Orchestrator supports PingOne as the Identity Partner (IDP); however, any PingIdentity product supporting OIDC can be easily configured.

Procedure

1 Log in to your PingOne account as an Admin user.

The PingOne home screen appears.


VMware, Inc. 44

https://admin.pingone.com/web-portal/login


a In the upper navigation bar, click Applications.

b On the My Applications tab, select OIDC and then click Add Application.

The Add OIDC Application pop-up window appears.

c Provide basic details such as name, short description, and category for the application and click Next.

d Under AUTHORIZATION SETTINGS, select Authorization Code as the allowed grant types and click Next.

Also, note down the Discovery URL and Client Credentials (Client ID and Client Secret) to be used during the SSO configuration in SD-WAN Orchestrator.


VMware, Inc. 45

e Under SSO FLOW AND AUTHENTICATION SETTINGS, provide valid values for Start SSO URL and Redirect URL and click Next.

In the SD-WAN Orchestrator application, at the bottom of the Configure Authentication screen, you can find the redirect URL link. Ideally, the SD-WAN Orchestrator redirect URL will be in this format: https://<Orchestrator URL>/login/ssologin/openidCallback. The Start SSO URL will be in this format: https://<vco>/<domain name>/login/doEnterpriseSsoLogin.

f Under DEFAULT USER PROFILE ATTRIBUTE CONTRACT, click Add Attribute to add additional user profile attributes.

g In the Attribute Name text box, enter group_membership and then select the Required checkbox, and select Next.

Note The group_membership attribute is required to retrieve roles from PingOne.

h Under CONNECT SCOPES, select the scopes that can be requested for your SD-WAN Orchestrator application during authentication and click Next.

i Under Attribute Mapping, map your identity repository attributes to the claims available to your SD-WAN Orchestrator application.

Note The minimum required mappings for the integration to work are email, given_name, family_name, phone_number, sub, and group_membership (mapped to memberOf).

j Under Group Access, select all user groups that should have access to your SD-WAN Orchestrator application and click Done.

The application will be added to your account and will be available in the My Application screen.

Results

You have completed setting up an OIDC-based application in PingOne for SSO.

What to do next


Create a New User Group in PingIdentity

To create a new user group, perform the steps on this procedure.

Procedure

1 Click Users > User Directory.

2 On the Groups tab, click Add Group

The New Group screen appears.

3 In the Name text box, enter a name for the group and click Save.

Create a New User in PingIdentity

To add a new user, perform the steps on this procedure.


VMware, Inc. 46

Procedure

1 Click Users > User Directory.

2 On the Users tab, click the Add Users drop-down menu and select Create New User.

The User screen appears.

3 Enter all the mandatory details such as username, password, and email ID of the user.

4 Under Group Memberships, click Add.

The Add Group Membership pop-up window appears.

5 Search and add the user to a group and click Save.

Configure Azure Active Directory for Single Sign OnTo set up an OpenID Connect (OIDC)-based application in Microsoft Azure Active Directory (AzureAD) for Single Sign On (SSO), perform the steps on this procedure.

Prerequisites

Ensure you have an AzureAD account to sign in.

Procedure

1 Log in to your Microsoft Azure account as an Admin user.

The Microsoft Azure home screen appears.


VMware, Inc. 47

https://portal.azure.com/


a Search and select the Azure Active Directory service.

b Go to App registration > New registration.

The Register an application screen appears.

c In the Name field, enter the name for your SD-WAN Orchestrator application.

d In the Redirect URL field, enter the redirect URL that your SD-WAN Orchestrator application uses as the callback endpoint.

In the SD-WAN Orchestrator application, at the bottom of the Configure Authentication screen, you can find the redirect URL link. Ideally, the SD-WAN Orchestrator redirect URL will be in this format: https://<Orchestrator URL>/login/ssologin/openidCallback.


VMware, Inc. 48

e Click Register.

Your SD-WAN Orchestrator application will be registered and displayed in the All applications and Owned applications tabs. Make sure to note down the Client ID/Application ID to be used during the SSO configuration in SD-WAN Orchestrator.

f Click Endpoints and copy the well-known OIDC configuration URL to be used during the SSO configuration in SD-WAN Orchestrator.

g To create a client secret for your SD-WAN Orchestrator application, on the Owned applications tab, click on your SD-WAN Orchestrator application.

h Go to Certificates & secrets > New client secret.

The Add a client secret screen appears.

i Provide details such as description and expiry value for the secret and click Add.

The client secret will be created for the application. Note down the new client secret value to be used during the SSO configuration in SD-WAN Orchestrator.

j To configure permissions for your SD-WAN Orchestrator application, click on your SD-WAN Orchestrator application and go to API permissions > Add a permission.

The Request API permissions screen appears.


VMware, Inc. 49

k Click Microsoft Graph and select Application permissions as the type of permission for your application.

l Under Select permissions, from the Directory drop-down menu, select Directory.Read.All and from the User drop-down menu, select User.Read.All.

m Click Add permissions.


VMware, Inc. 50

n To add and save roles in the manifest, click on your SD-WAN Orchestrator application and from the application Overview screen, click Manifest.

A web-based manifest editor opens, allowing you to edit the manifest within the portal. Optionally, you can select Download to edit the manifest locally, and then use Upload to reapply it to your application.

o In the manifest, search for the appRoles array and add one or more role objects as shown in the following example and click Save.

Sample role objects

{

"allowedMemberTypes": [

"User"

],

"description": "Standard Administrator who will have sufficient privilege to

manage resource",

"displayName": "Standard Admin",

"id": "18fcaa1a-853f-426d-9a25-ddd7ca7145c1",

"isEnabled": true,

"lang": null,

"origin": "Application",

"value": "standard"

},

{

"allowedMemberTypes": [

"User"

],

"description": "Super Admin who will have the full privilege on SD-WAN

Orchestrator",

"displayName": "Super Admin",

"id": "cd1d0438-56c8-4c22-adc5-2dcfbf6dee75",

"isEnabled": true,

"lang": null,

"origin": "Application",

"value": "super"

}


VMware, Inc. 51

Note Make sure to set id to a newly generated GUID value.

3 To assign groups and users to your SD-WAN Orchestrator application:

a Go to Azure Active Directory > Enterprise applications.

b Search and select your SD-WAN Orchestrator application.

c Click Users and groups and assign users and groups to the application.

d Click Submit.

Results

You have completed setting up an OIDC-based application in AzureAD for SSO.

What to do next


Create a New Guest User in AzureAD

To create a new guest user, perform the steps on this procedure.

Procedure

1 Go to Azure Active Directory > Users > All users.

2 Click New guest user.

The New Guest User pop-up window appears.

3 In the Email address text box, enter the email address of the guest user and click Invite.

The guest user immediately receives a customizable invitation that lets them to sign into their Access Panel.


VMware, Inc. 52

4 Guest users in the directory can be assigned to apps or groups.

Configure VMware CSP for Single Sign OnTo configure VMware Cloud Services Platform (CSP) for Single Sign On (SSO), perform the steps on this procedure.

Procedure

1 Sign in to VMware CSP console (staging or production environment) with your VMware account ID. If you are new to VMware Cloud and do not have a VMware account, you can create one as you sign up. For more information, see How do I Sign up for VMware CSP section in Using Vmware Cloud documentation.

2 Contact the VMware SD-WAN Support Provider for receiving an application onboarding invitation link to register your SD-WAN Orchestrator application to VMware CSP. For information on how to contact the Support Provider, see https://kb.vmware.com/s/article/53907 and https://www.vmware.com/support/contacts/us_support.html.

The VMware SD-WAN Support Provider will create and share a Service invitation URL that needs to be redeemed to your Customer organization.

3 Redeem the Service invitation URL to your existing Customer Organization or create a new Customer Organization by following the steps in the UI screen.

4 After redeeming the Service invitation, when you sign in to VMware CSP console, you can view your application tile under My Services area in the Vmware Cloud Services page.

The Organization you are logged into is displayed under your username on the menu bar. Make a note of the Organization ID by clicking on your username. A shortened version of the ID is displayed under the Organization name. Click the ID to display the full Organization ID.

5 Share the full Organization ID with the VMware SD-WAN Support Provider and request them to create an OAuth application.

The VMware SD-WAN Support Provider will create an OAuth application in VMware CSP console and share the IDP integration details such as Client ID, Client Secret, and OIDC well-known configuration URL.

6 Log in to your SD-WAN Orchestrator application as Super Admin user and configure SSO using the received IDP integration details as follows.

a Click Administration > System Settings

The System Settings screen appears.

b Click the General Information tab and in the Domain text box, enter the domain name for your enterprise, if it is not already set.

Note To enable SSO authentication for the SD-WAN Orchestrator, you must set up the domain name for your enterprise.

c Click the Authentication tab and from the Authentication Mode drop-down menu, select SSO.


VMware, Inc. 53

https://console.cloud.vmware.com/csp/gateway/discovery

https://docs.vmware.com/en/VMware-Cloud-services/services/Using-VMware-Cloud-Services.pdf

https://kb.vmware.com/s/article/53907

https://www.vmware.com/support/contacts/us_support.html

https://www.vmware.com/support/contacts/us_support.html

https://console.cloud.vmware.com/csp/gateway/discovery

d From the Identity Provider template drop-down menu, select VMwareCSP.

e In the Organization Id text box, enter the Organization ID in the following format: /csp/gateway/am/api/orgs/<full organization ID>

f In the OIDC well-known config URL text box, enter the OpenID Connect (OIDC) configuration URL (https://console.cloud.vmware.com/csp/gateway/am/api/.well-known/openid-configuration) for your IDP.

The SD-WAN Orchestrator application auto-populates endpoint details such as Issuer, Authorization Endpoint, Token Endpoint, and User Information Endpoint for your IDP.

g In the Client Id text box, enter the client identifier received from the Support Provider.

h In the Client Secret text box, enter the client secret code received from the Support Provider.

i To determine user’s role in SD-WAN Orchestrator, select either Use Default Role or Use Identity Provider Roles.

j On selecting the Use Identity Provider Roles option, in the Role Attribute text box, enter the name of the attribute set in the VMware CSP to return roles.

k In the Role Map area, map the VMwareCSP-provided roles to each of the SD-WAN Orchestrator roles, separated by using commas.

Roles in VMware CSP will follow this format: external/<service definition uuid>/<service role name mentioned during service template creation>.

7 Click Save Changes to save the SSO configuration.

8 Click Test Configuration to validate the entered OpenID Connect (OIDC) configuration.

The user is navigated to the VMware CSP website and allowed to enter the credentials. On IDP verification and successful redirect to SD-WAN Orchestrator test call back, a successful validation message will be displayed.


VMware, Inc. 54

https://console.cloud.vmware.com/csp/gateway/am/api/.well-known/openid-configuration

Results

You have completed integrating SD-WAN Orchestrator application in VMware CSP for SSO and can access the SD-WAN Orchestrator application logging in to the VMware CSP console.

What to do next

n Within the organization, manage users by adding new users and assigning appropriate role for the users. For more information, see Identity & Access Management section in in Using Vmware Cloud Documentation.


VMware, Inc. 55

https://docs.vmware.com/en/VMware-Cloud-services/services/Using-VMware-Cloud-Services.pdf

Edge Licensing 12SD-WAN Orchestrator provides different types of Licenses for the Edges. Partner users can manage and assign licenses to their Enterprise customers.

Only Operators can enable the Edge Licensing and assign the licenses to a Partner user. If the Edge Licensing is not enabled for you, contact your Operator.

The Edge licenses are available with the following components:

Component Supported Attributes

Bandwidth 10M, 30M, 50M, 100M, 200M, 500M, 1G, 2G, 5G, 10G

Editions Standard, Enterprise, Premium

Region North America, Europe Middle East and Africa, Latin America, Asia Pacific

Term 12 months, 36 months, 60 months

An Operator can assign different types of Edge licenses from the 270 types of licenses available with various combinations.

To manage the Edge licenses for Customers, see Manage Edge Licenses for Customers.

To view and generate a report of available License types, see Generate an Edge Licensing Report.


n Manage Edge Licenses for Customers

n Generate an Edge Licensing Report

Manage Edge Licenses for CustomersA Partner user can manage the Edge Licenses and assign them to customers.

To manage the Edge licenses for a customer:

Procedure

1 In the Partner portal, click Manage Customers.

2 Click the link to a customer name to navigate to the Enterprise portal.

3 In the Enterprise portal, click Administration > Edge Licensing.

VMware, Inc. 56

4 Click Manage Edge License.

5 In the Select Edge Licenses window, choose the relevant licenses based on the Bandwidth, Term, Edition, and Region.

Note While selecting the Licenses, you can choose either one of the following:

n Select only Standard Editions.

n Select Enterprise, Premium, or both. You cannot combine a Standard Edition with the other Editions.

6 Click OK.

Results

The selected licenses are displayed in the Edge Licensing window.

Click Report to generate a report of the licenses and the associated Edges in MS Excel format.

What to do next

You can assign a license to an Edge:

n In the Enterprise portal, click Configure > Edges.

n To assign license to each Edge, click the link to the Edge and select the License in the Edge Overview page. You can also select the Edge and click Actions > Assign Edge License to assign the License.


VMware, Inc. 57

n To assign a license to multiple Edges, select the appropriate Edges, click Actions > Assign Edge License, and select the License.

Generate an Edge Licensing ReportPartner Superusers, Partner Standard Administrators, Partner Business Specialist, and Partner Customer Support users can generate a report of the existing Edge licenses.

In the Partner portal, navigate to Edge Licensing.

Click Report to generate a report of the licenses, associated customers, and Edges in MS Excel format.


VMware, Inc. 58

Manage Gateway Pools 13Partners can create and manage Gateway Pools and Gateways if their Operator has enabled this functionality. Once enabled, partners can access this feature from the Gateway Pools and Gateway links, respectively.

If an Operator has granted a Partner access to create and manage Gateway Pools, the partner will see a check mark in the Managed Pool column associated with a Gateway Pool.

Partners cannot modify operator-owned Gateway Pools. These Gateway Pools will have a “x” associated with them under the Managed Pool column, and the settings in the Properties and Gateways In Pool areas are read-only.

VMware, Inc. 59


n Create a Gateway Pool

Create a Gateway PoolYou can create a new Gateway Pool if your Operator has granted you access to this feature. Contact your operator if you want to gain access. Gateways owned and created by your Operator are read-only.

To create a new Gateway Pool:

1 Click the New Gateway Pool button to create a new Gateway Pool.

2 In the New Gateway Pool dialog box:

a Enter a unique Name and a Description of the Gateway Pool.

b Choose an option from the Partner Gateway Hand Off drop-down menu.

3 Click the Create button to create your Gateway Pool.

You can modify any Gateway Pools that you own. However, Operator-owned Gateway Pools are read-only.

Note Partner-created Gateways will be visible only to that specific Partner and can only be used within the Partner Pools.


VMware, Inc. 60

Manage Gateways 14A partner can manage gateways from the Gateways link.

To manage Gateways:

n In the navigation bar, click Gateways.

The Gateways page appears.

VMware, Inc. 61

Install VMware SD-WAN Partner Gateway 15This document describes the steps needed to install and deploy VMware SD-WAN Gateway as a Partner Gateway. It also covers how to configure the VRF/VLAN and BGP configuration necessary on the SD-WAN Orchestrator.


n Installation Overview

n Hypervisor Minimum Hardware Requirements

n SD-WAN Gateway Installation Procedures

n Post-Installation Tasks

n Upgrade SD-WAN Gateway

n Custom Configurations

n SNMP Integration

n Custom Firewall Rules

Installation OverviewThis section provides an overview of VMware SD-WAN Partner Gateway installation.

About Partner GatewaysPartner Gateways are Gateways tailored to an on-premise operation in which the Gateway is installed and deployed with two interfaces.

n One interface is facing the private and/or public WAN network and is dedicated to receiving VCMP encapsulated traffic from the remote edges, as well as standard IPsec traffic from Non-VMware SD-WAN Sites.

n Another interface is facing the datacenter and provides access to resources or networks attached to a PE router, which the Partner Gateway is connected to. The PE router typically affords access to shared managed services that are extended to the branches, or access to a private (MPLS / IP-VPN) core network in which individual customers are segregated.

The following distributions are provided:

VMware, Inc. 62

Provided Description Example

VMware Gateway OVA package. velocloud-vcg-2.4.0-R24-20170428-GA.ova

KVM Gateway qcow2 disk image. velocloud-vcg-2.4.0-R24-20170428-GA.qcow2

Hypervisor Minimum Hardware RequirementsThe SD-WAN Gateway runs on a standard hypervisor (KVM or VMware ESXi).

Minimum Server RequirementsTo run the hypervisor:

n 10 Intel CPU's at 2.0 Ghz or higher. The CPU must support the AES-NI, SSSE3, SSE4, and RDTSC instruction sets.

n 20+ GB (16 GB is required for SD-WAN Gateway VM memory)

n 100 GB magnetic or SSD based, persistent disk volume

n 2 x 1 Gbps (or higher) network interface. The physical NIC card supported are Intel 82599/82599ES and Intel X710/XL710 chipsets (for SR-IOV & DPDK support).

Reference Hardware Specifications

NIC Chipset Hardware Specification

Intel 82599/82599ES HP DL380G9 http://www.hp.com/hpinfo/newsroom/press_kits/2014/ComputeEra/HP_ProLiantDL380_DataSheet.pdf

Intel X710/XL710 Dell PowerEdge R640

https://www.dell.com/en-us/work/shop/povw/poweredge-r640

n CPU Model and Cores - Dual Socket Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz with 16 cores each

n Memory - 384 GB RAM

Intel X710/XL710 Supermicro SYS-6018U-TRTP+

https://www.supermicro.com/en/products/system/1U/6018/SYS-6018U-TRTP_.cfm

n CPU Model and Cores - Dual Socket Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz with 10 Cores each

n Memory - 256 GB RAM

Supported NIC Specifications

Hardware Manufacturer Firmware VersionHost Driver for Ubuntu 16.04/18.04

Host Driver for ESXi 6.7

Dual Port Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+

6.80 2.7.11 1.7.17

Dual Port Intel Corporation Ethernet Controller X710 for 10GbE SFP+

6.80 2.7.11 1.7.17

Quad Port Intel Corporation Ethernet Controller X710 for 10GbE SFP+

6.80 2.7.11 1.7.17


VMware, Inc. 63

http://www.hp.com/hpinfo/newsroom/press_kits/2014/ComputeEra/HP_ProLiantDL380_DataSheet.pdf

http://www.hp.com/hpinfo/newsroom/press_kits/2014/ComputeEra/HP_ProLiantDL380_DataSheet.pdf

https://www.dell.com/en-us/work/shop/povw/poweredge-r640

https://www.supermicro.com/en/products/system/1U/6018/SYS-6018U-TRTP_.cfm

Supported Hypervisor Versions

Hypervisor Supported Versions

VMware n Intel 82599/82599ES - ESXi 5.5U3 or later. In order to use SR-IOV, the vCenter and the vSphere Enterprise Plus license are required.

n Intel X710/XL710 - ESXi 6.7 with VMware vSphere Web Client 6.7.0.

KVM n Intel 82599/82599ES - Ubuntu 14.04 LTS and 16.04 LTS

n Intel X710/XL710 - Ubuntu 16.04 LTS and 18.04 LTS

SD-WAN Gateway Virtual Hardware SpecificationFor VMware, the OVA already specifies the minimum virtual hardware specification. For KVM, an example XML file will be provided. The minimum virtual hardware specifications are:

n 8 vCPUs

n 8 GB of memory

n Minimum of 2 vNICs:

n One vNIC is the public (outside) interface, which must be an untagged interface.

n One vNIC is the private (inside) interface that must be tagged. This is the interface facing the PE router or L3 switch.

n Optional vNIC (if a separate management/OAM interface is required)

n 32 GB of virtual disk

Firewall/NAT Requirements

Note These requirements apply if the SD-WAN Gateway is deployed behind a Firewall and/or NAT device.

n The firewall needs to allow outbound traffic from the SD-WAN Gateway to TCP/443 (for communication with SD-WAN Orchestrator).

n The firewall needs to allow inbound traffic from the Internet to UDP/2426 (VCMP), UDP/4500, and UDP/500. If NAT is not used, then the firewall needs to also allow IP/50 (ESP).

n If NAT is used, the above ports must be translated to an externally reachable IP address. Both the 1:1 NAT and port translations are supported.

Git Repository with Templates and SamplesThe following Git repository contains templates and samples.

git clone https://bitbucket.org/velocloud/deployment.git


VMware, Inc. 64

SD-WAN Gateway Installation ProceduresThis section describes the SD-WAN Gateway installation procedures.

In general, installing the SD-WAN Gateway involves the following steps:

1 Create SD-WAN Gateway on SD-WAN Orchestrator and make a note of the activation key.

2 Configure SD-WAN Gateway on SD-WAN Orchestrator.

3 Create the cloud-init file.

4 Create the VM in VMware or KVM.

5 Boot the SD-WAN Gateway VM and ensure the SD-WAN Gateway cloud-init initializes properly. At this stage, the SD-WAN Gateway should already activate itself against the SD-WAN Orchestrator.

6 Verify connectivity and disable cloud-init.

Important SD-WAN Gateway supports both the virtual switch and SR-IOV. This guide specifies the SR-IOV as an optional configuration step. If SR-IOV is used, DPDK must be enabled for the partner handoff to work.

Pre-Installation ConsiderationsThe VMware SD-WAN Partner Gateway provides different configuration options. A worksheet should be prepared before the installation of the Gateway.

Worksheet

SD-WAN Gateway n Version

n OVA/QCOW2 file location

n Activation Key

n SD-WAN Orchestrator (IP ADDRESS/vco-fqdn-hostname)

n Hostname

Hypervisor Address/Cluster name

Storage Root volume datastore (>40GB recommended)

CPU Allocation CPU Allocation for KVM/VMware.

Installation Selections DPDK (YES/NO)

OAM Network ( Optional See Custom Configurations) n DHCP

n OAM IPv4 Address

n OAM IPv4 Netmask

n DNS server - primary

n DNS server - secondary

n Static Routes


VMware, Inc. 65

ETH0 – Internet Facing Network n IPv4 Address

n IPv4 Netmask

n IPv4 Default gateway



Handoff (ETH1) - Network n MGMT VRF IPv4 Address

n MGMT VRF IPv4 Netmask

n MGMT VRF IPv4 Default gateway



n Handoff ( QinQ (0x8100), QinQ (0x9100), none, 802.1Q, 802.1ad)

n C-TAG

n S-TAG

Console access n Console_Password

n SSH:

n Enabled (yes/no)

n SSH public key

NTP ( Optional see Custom Configuration Section) n Public NTP:

n server 0.ubuntu.pool.ntp.org




n Internal NTP server - 1

n Internal NTP server - 2

SD-WAN Gateway SectionMost of the SD-WAN Gateway section is self-explanatory.

SD-WAN Gateway n Version - Should be same or lower than SD-WAN Orchestrator

n OVA/QCOW2 file location - Plan ahead the file location and disk allocation

n Activation Key

n SD-WAN Orchestrator (IP ADDRESS/vco-fqdn-hostname)

n Hostname - Valid Linux Hostname “RFC 1123”

Creating a Gateway and Getting the Activation Key1 Go to Operator > Gateway Pool and create a new SD-WAN Gateway pool. For running SD-WAN

Gateway in the Service Provider network, check the Allow Partner Gateway checkbox. This will enable the option to include the partner gateway in this gateway pool.


VMware, Inc. 66

2 Go to Operator > Gateway and create a new gateway and assign it to the pool. The IP address of the gateway entered here must match the public IP address of the gateway. If unsure, you can run curl ipinfo.io/ip from the SD-WAN Gateway which will return the public IP of the SD-WAN Gateway.

3 Make a note of the activation key and add it to the worksheet.


VMware, Inc. 67

Enable Partner Gateway Mode1 Go to Operator > Gateways and select the SD-WAN Gateway. Check the Partner Gateway

checkbox to enable the Partner Gateway.

There are additional parameters that can be configured. The most common are the following:

Advertise 0.0.0.0/0 with no encrypt

This option will enable the Partner Gateway to advertise a path to Cloud traffic for the SAAS Application. Since the Encrypt Flag is off, it will be up to the customer configuration on the business policy to use this path or not.

The second recommend option is to advertise the SD-WAN Orchestrator IP as a /32 with encrypt.

This will force the traffic that is sent from the Edge to the SD-WAN Orchestrator to take the Gateway Path. This is recommended since it introduces predictability to the behavior that the SD-WAN Edge takes to reach the SD-WAN Orchestrator.

Installation Selection

Installation Selections DPDK (YES/NO)


VMware, Inc. 68

DPDK is optional but necessary for higher throughput. To Enable DPDK, you need to have SR-IOV support. Before starting your installation, decide if you are planning to enable DPDK.

Networking

Important The following procedure and screenshots focus on the most common deployment, which is the 2-ARM installation for the Gateway. The addition of an OAM network is considered in the section titled, OAM Interface and Static Routes.

The diagram above is a representation of the SD-WAN Gateway in a 2-ARM deployment. In this example, we assume eth0 is the interface facing the public network (Internet) and eth1 is the interface facing the internal network (handoff or VRF interface).

Note A Management VRF is created on the SD-WAN Gateway and is used to send a periodic ARP refresh to the default gateway IP to check that the handoff interface is physically up and speed ups the failover time. It is recommended that a dedicated VRF is set up on the PE router for this purpose. Optionally, the same management VRF can also be used by the PE router to send an IP SLA probe to the SD-WAN Gateway to check for SD-WAN Gateway status (SD-WAN Gateway has a stateful ICMP responder that will respond to ping only when its service is up).If a dedicated Management VRF is not set up, then you can use one of the customer VRFs as a Management VRF, although this is not recommended.

For the Internet Facing network, you only need the basic network configuration.

ETH0 – Internet Facing Network n IPv4_Address

n IPv4_Netmask

n IPv4_Default_gateway

n DNS_server_primary

n DNS_server_secondary

For the Handoff interface, you must know which type of handoff you want to configure and the Handoff configuration for the Management VRF.


VMware, Inc. 69

ETH1 – HANDOFF Network n MGMT_IPv4_Address

n MGMT_IPv4_Netmask

n MGMT_IPv4_Default gateway

n DNS_Server_Primary

n DNS_Server_Secondary

n Handoff (QinQ (0x8100), QinQ (0x9100), none, 802.1Q, 802.1ad)

n C_TAG_FOR_MGMT_VRF

n S_TAG_FOR_MGMT_VRF

Console Access

Console access n Console_Password

n SSH:

n Enabled (yes/no)

n SSH public key

In order to access the Gateway, a console password and/or an SSH public key must be created.

Cloud-Init CreationThe configuration options for the gateway that we defined in the worksheet are used in the cloud-init configuration. The cloud-init config is composed of two main configuration files, the metadata file and the user-data file. The meta-data contains the network configuration for the Gateway, and the user-data contains the Gateway Software configuration. This file provides information that identifies the instance of the SD-WAN Gateway being installed.

Below are the templates for both Meta_data and User_data files.

Fill the templates with the information in the worksheet. All #_VARIABLE_# need to be replaced, also check any #ACTION#

Important The template assumes you are using static configuration for the interfaces. It also assumes that you are either using SR-IOV for all interfaces or none. For more information, see OAM - SR-IOV with vmxnet3 or SR-IOV with VIRTIO. The templates are also available in the git repository at: git clone https://bitbucket.org/velocloud/deployment.git. It is recommended that you get the templates from repository instead of copying and pasting from this document. https://bitbucket.org/velocloud/deployment

meta-data (git ./vcg/samples/VCG_2ARM/meta-data)

instance-id: #_Hostname_#

local-hostname: #_Hostname_#

network-interfaces: |

auto eth0

iface eth0 inet static

address #_IPv4_Address_#

mac_address #_mac_Address_#

netmask #_IPv4_Netmask_#

gateway #_IPv4_Gateway_#

dns-nameservers #_DNS_server_primary_# #_DNS_server_secondary_#

auto eth1



VMware, Inc. 70

https://bitbucket.org/velocloud/deployment



metric '13'

address #_MGMT_IPv4_Address_#

mac_address #_MGMT_mac_Address_#

netmask #_MGMT_IPv4_Netmask_#

gateway #_MGMT_IPv4_Gateway_#


user-data (Git /deployment/vcg/samples/VCG_2ARM/user-data)

#cloud-config

hostname: #_Hostname_#

password: #_Console_Password_#

chpasswd: {expire: False}

ssh_pwauth: True

ssh_authorized_keys:

- #_SSH_public_Key_#

runcmd:

- 'echo "[]" > /opt/vc/etc/vc_blocked_subnets.json'

- 'sed -iorig "s/wan=\".*/wan=\"eth0 eth1\"/" /etc/config/gatewayd-tunnel'

- '/var/lib/cloud/scripts/per-boot/config_gateway'

- 'sleep 10'

- '/opt/vc/bin/vc_procmon restart'

write_files:

- path: "/var/lib/cloud/scripts/per-boot/config_gateway"

permissions: "0755"

content: |

#!/usr/bin/python

import json

import commands

is_activated = commands.getoutput("/opt/vc/bin/is_activated.py")

if "True" in str(is_activated):

print "Gateway already activated"

exit

commands.getoutput("/opt/vc/bin/activate.py -s #_VCO_# #_Activation_Key_# ")

### EDIT GATEWAYD ###

with open("/etc/config/gatewayd", "r") as jsonFile:

data = json.load(jsonFile)

data["global"]["vcmp.interfaces"] = ["eth0"]

data["global"]["wan"] = ["eth1"]

# NOTE FOR HAND OFF IT CAN BE "QinQ (0x8100)" "QinQ (0x9100)" "none" "802.1Q" "802.1ad”

data["vrf_vlan"]["tag_info"][0]["mode"] = "#_Handoff_"

data["vrf_vlan"]["tag_info"][0]["interface"] = "eth1"

data["vrf_vlan"]["tag_info"][0]["c_tag"] = "#_C_TAG_FOR_MGMT_VRF_#"

data["vrf_vlan"]["tag_info"][0]["s_tag"] = "#_S_TAG_FOR_MGMT_VRF_"

with open("/etc/config/gatewayd", "w") as jsonFile:

jsonFile.write(json.dumps(data,sort_keys=True,indent=4, separators=(",", ": ")))

### EDIT DPDK ###

with open("/opt/vc/etc/dpdk.json", "r") as jsonFile:


#SET 0 or 1 for enabled or DISABLED example data["dpdk_enabled"] = 0

data["dpdk_enabled"] = #_DKDP_ENABLED_(1)_OR_DISABLED_(0)_#

with open("/opt/vc/etc/dpdk.json", "w") as jsonFile:


final_message: "==== Cloud-init completed ===="

power_state:

delay: "+1"

mode: reboot


VMware, Inc. 71

message: Bye Bye

timeout: 30

condition: True

Important n VMware recommends to have a proper fully qualified domain name (FQDN) configured for all

production Orchestrators so proper TLS certificates may be issued for them.

n If activation using the Orchestrator’s IP address is the only option, use the following example which instructs the Edge to bypass TLS verification.

commands.getoutput("/opt/vc/bin/ activate.py -s myvco.example.com -i #_activation_key_#")

n This configuration is not recommended for production use and we highly encourage you to reactivate against the Orchestrator’s hostname at the soonest possible.

Note Always validate user-data and metadata, using http://www.yamllint.com/. The metadata should also be a valid network configuration under the network-interface (/etc/network/interfaces) section, once the cloud-init completes. Sometimes when working with the Windows/Mac copy paste feature, there is an issue of introducing Smart Quotes which can corrupt the files. Run the following command to make sure you are smart quote free.

sed s/[”“]/'"'/g /tmp/user-data > /tmp/user-data_new

Create ISO FileOnce you have completed your files, they need to be packaged into an ISO image. This ISO image is used as a virtual configuration CD with the virtual machine. This ISO image, called vcg01-cidata.iso, is created with the following command on a Linux system:

genisoimage -output vcg01-cidata.iso -volid cidata -joliet -rock user-data meta-data

If you are on a MAC OSX, use the command below instead:

mkisofs -output vcg01-cidata.iso -volid cidata -joliet -rock {user-data,meta-data}

This iso file which we will call #CLOUD_INIT_ISO_FILE# is going to be used in both OVA and VMware installations.

Install SD-WAN GatewayYou can install SD-WAN Gateway on VMware and KVM.

Prerequisites

KVM provides multiple ways to provide networking to virtual machines. VMware SD-WAN recommends the following options:

n SR-IOV


VMware, Inc. 72

http://www.yamllint.com/

n Linux Bridge

n OpenVSwitch Bridge

If you decide to use SR-IOV mode, enable SR-IOV on KVM and VMware. For steps, see:

n Enable SR-IOV on KVM

n Enable SR-IOV on VMware

To install SD-WAN Gateway:

n On KVM, see Install SD-WAN Gateway on KVM.

n On VMware, see Install SD-WAN Gateway on VMware.

Enable SR-IOV on VMwareEnabling SR-IOV on VMware is optional, but it is necessary to realize the full benefit of DPDK to improve packet processing performance.

Prerequisites

This requires a specific NIC card. The following chipsets are certified by VMware SD-WAN to work with the SD-WAN Gateway.

n Intel 82599/82599ES

n Intel X710/XL710

Note Before using the Intel X710/XL710 cards in SR-IOV mode on VMware, make sure the supported Firmware and Driver versions described in the Deployment Prerequisites section are installed correctly.

To enable SR-IOV on VMware:

1 Make sure that your NIC card supports SR-IOV. Check the VMware Hardware Compatibility List (HCL) at https://www.vmware.com/resources/compatibility/search.php?deviceCategory=io

Brand Name: Intel

I/O Device Type: Network

Features: SR-IOV

The following VMware KB article provides details of how to enable SR-IOV on the supported NIC: https://kb.vmware.com/s/article/2038739


VMware, Inc. 73

https://www.vmware.com/resources/compatibility/search.php?deviceCategory=io

https://kb.vmware.com/s/article/2038739

2 Once you have a support NIC card, go to the specific VMware host, select the Configure tab, and then choose Physical adapters.

3 Select Edit Settings. Change Status to Enabled and specify the number of virtual functions required. This number varies by the type of NIC card.

4 Reboot the hypervisor.

5 If SR-IOV is successfully enabled, the number of Virtual Functions (VFs) will show under the particular NIC after ESXi reboots.

Install SD-WAN Gateway on VMwareDescribes how to install the SD-WAN Gateway OVA on VMware.

Important When you are done with the OVA installation, DO NOT start the VM until you have the cloud-init iso file and mount as CD-ROM to the SD-WAN Gateway VM. Otherwise, you will need to re-deploy the VM again.

If you decide to use SR-IOV mode, then you can optionally enable SR-IOV on VMware. To enable the SR-IOV on VMware, see Enable SR-IOV on VMware.

To install the SD-WAN Gateway OVA on VMware:

1 Select the ESXi host, go to Actions, and then Deploy OVF Template. Select the SD-WAN Gateway OVA file provided by VMware SD-WAN and click Next.


VMware, Inc. 74

Review the template details in Step 4 (Review details) of the Deploy OVA/OVF Template wizard as shown in the image below.

2 For the Select networks step, the OVA comes with two pre-defined networks (vNICs).

vNIC Description

Inside This is the vNIC facing the PE router and is used for handoff traffic to the MPLS PE or L3 switch. This vNIC is normally bound to a portgroup that does a VLAN pass-through (VLAN=4095 in vswitch configuration).

Outside This is the vNIC facing the Internet. This vNIC expects a non-tagged L2 frame and is normally bound to a different portgroup from the Inside vNIC.

3 For the Customize template step, do not change anything. This is when you use vApp to configure the VM. We will not use vApp in this example. Click Next to continue with deploying the OVA.


VMware, Inc. 75

4 Once the VM is successfully deployed, return to the VM and click Edit Settings . Two vNICs are created with adapter type = vmxnet3.

5 (Optional for SR-IOV) This step is required only if you plan to use SR-IOV. Because the OVA by default creates the two vNICs as vmxnet3, we will need to remove the two vNICs and re-add them as SR-IOV.


VMware, Inc. 76

When adding the two new SR-IOV vNICs, use the same portgroup as the original two vmxnet3 vNICs. Make sure the Adapter Type is SR-IOV passthrough. Select the correct physical port to use and set the Guest OS MTU Change to Allow. After you add the two vNICs, click OK.

6 As SD-WAN Gateway is a real-time application, you need to configure the Latency Sensitivity to High. For more information about how to configure the VM for real-time application, see https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/latency-sensitive-perf-vsphere55-white-paper.pdf.


VMware, Inc. 77

https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/latency-sensitive-perf-vsphere55-white-paper.pdf



7 Refer to Cloud-init Creation. The Cloud-init file is packaged as a CD-ROM (iso) file. You need to mount this file as a CD-ROM.

Note You must upload this file to the datastore.

8 Start the VM.

Enable SR-IOV on KVMTo enable the SR-IOV mode on KVM, perform the following steps.

Prerequisites

This requires a specific NIC card. The following chipsets are certified by VMware SD-WAN to work with the SD-WAN Gateway and SD-WAN Edge.

n Intel 82599/82599ES


VMware, Inc. 78

n Intel X710/XL710

Note Before using the Intel X710/XL710 cards in SR-IOV mode on KVM, make sure the supported Firmware and Driver versions specified in the Deployment Prerequisites section are installed correctly.

To enable SR-IOV on KVM:

1 Enable SR-IOV in BIOS. This will be dependent on your BIOS. Login to the BIOS console and look for SR-IOV Support/DMA. You can verify support on the prompt by checking that Intel has the correct CPU flag.

cat /proc/cpuinfo | grep vmx

2 Add the options on Bboot (in /etc/default/grub).

GRUB_CMDLINE_LINUX="intel_iommu=on"

a Run the following commands: update-grub and update-initramfs -u.

b Reboot

c Make sure iommu is enabled.

velocloud@KVMperf3:~$ dmesg | grep -i IOMMU

[ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.13.0-107-generic root=/dev/mapper/qa--

multiboot--002--vg-root ro intel_iommu=on splash quiet vt.handoff=7

[ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.13.0-107-generic root=/dev/mapper/qa--

multiboot--002--vg-root ro intel_iommu=on splash quiet vt.handoff=7

[ 0.000000] Intel-IOMMU: enabled

….

velocloud@KVMperf3:~$

3 Based on the NIC chipset used, add a driver as follows:

n For the Intel 82599/82599ES cards in SR-IOV mode:

1 Download and install ixgbe driver from the Intel website.

2 Configure ixgbe config (tar and sudo make install).

velocloud@KVMperf1:~$ cat /etc/modprobe.d/ixgbe.conf

3 If the ixgbe config file does not exist, you have to create the file as follows.

options ixgbe max_vfs=32,32

options ixgbe allow_unsupported_sfp=1

options ixgbe MDD=0,0

blacklist ixgbevf

4 Run the update-initramfs -u command and reboot the Server.


VMware, Inc. 79

https://downloadcenter.intel.com

5 Use the modinfo command to verify if the installation is successful.

velocloud@KVMperf1:~$ modinfo ixgbe and ip link

filename: /lib/modules/4.4.0-62-generic/updates/drivers/net/ethernet/intel/ixgbe/ixgbe.ko

version: 5.0.4

license: GPL

description: Intel(R) 10GbE PCI Express Linux Network Driver

author: Intel Corporation, <[email protected]>

srcversion: BA7E024DFE57A92C4F1DC93

n For the Intel X710/XL710 cards in SR-IOV mode:

1 Download and install i40e driver from the Intel website.

2 Create the Virtual Functions (VFs).

echo 4 > /sys/class/net/device name/device/sriov_numvfs

3 To make the VFs persistent after a reboot, add the command from the previous step to the "/etc/rc.d/rc.local" file.

4 Black list the VF driver

echo “blacklist i40evf” >> /etc/modprobe.d/blacklist.conf

5 Run the update-initramfs -u command and reboot the Server.

Validating SR-IOV (Optional)

You can quickly verify if your host machine has SR-IOV enabled by using the following command:

lspci | grep -i Ethernet

Verify if you have Virtual Functions:

01:10.0 Ethernet controller: Intel Corporation 82599 Ethernet Controller Virtual Function(rev 01)

Install SD-WAN Gateway on KVMDescribes how to install the SD-WAN Gateway qcow on KVM.

Pre-Installation Considerations

KVM provides multiple ways to provide networking to virtual machines. The networking in libvirt should be provisioned before the VM configuration. There are multiple ways to configure networking in KVM. For a full configuration of options on how to configure Networks on libvirt, see the following link:

https://libvirt.org/formatnetwork.html

From the full list of options, VMware SD-WAN recommends the following modes:

n SR-IOV (This mode is required for the SD-WAN Gateway to deliver the maximum throughput specified by VMware SD-WAN)

n OpenVSwitch Bridge


VMware, Inc. 80

https://downloadcenter.intel.com

https://libvirt.org/formatnetwork.html

If you decide to use SR-IOV mode, enable SR-IOV on KVM. To enable the SR-IOV on KVM, see Enable SR-IOV on KVM.

SD-WAN Gateway Installation Steps on KVM

1 Copy the QCOW and the Cloud-init files created in the Cloud-Init Creation section to a new empty directory.

2 Create the Network interfaces that you are going to use for the device.

Using SR-IOV

The following is a sample network interface template specific to Intel X710/XL710 NIC cards using SR-IOV.

<interface type='hostdev' managed='yes'>

<mac address='52:54:00:79:19:3d'/>

<driver name='vfio'/>

<source>

<address type='pci' domain='0x0000' bus='0x83' slot='0x0a' function='0x0'/>

</source>

<model type='virtio'/>

</interface>

Using OpenVSwitch

The following are the sample templates of a network interface using OpenVSwitch.

git ./vcg/templates/KVM_NETWORKING_SAMPLES/template_outside_openvswitch.xml

<?xml version="1.0" encoding="UTF-8"?>

<network>

<name>public_interface</name>



<model type="virtio" />

<forward mode="bridge" />

<bridge name="publicinterface" />

<virtualport type="openvswitch" />

<vlan trunk="yes">

<tag id="50" />



<tag id="51" />



</vlan>

</network>

Create a network for inside_interface:

git ./vcg/templates/KVM_NETWORKING_SAMPLES/template_inside_openvswitch.xml

<network>

<name>inside_interface</name> 



VMware, Inc. 81

<forward mode="bridge"/>

<bridge name="insideinterface"/>

<virtualport type='openvswitch'></virtualport>

<vlan trunk='yes'></vlan>

<tag id='200'/> <!—Define all the VLANS for this Bridge -->



</network>

If you are using OpenVSwitch mode, then you have to verify if the basic networks are created and active before launching the VM.

Note This validation step is not applicable for SR-IOV mode as you do not create any network before the VM is launched.

3 Edit the VM XML file. There are multiple ways to create a Virtual Machine in KVM. You can define the VM in an XML file and create it using libvirt, using the sample VM XML template specific to OpenVSwitch mode and SR-IOV mode.

vi my_vm.xml

The following is a sample template of a VM which uses OpenVSwitch interfaces. Use this template by making edits, wherever applicable.


<domain type="kvm">

<name>#domain_name#</name>

<memory unit="KiB">8388608</memory>

<currentMemory unit="KiB">8388608</currentMemory>

<vcpu>8</vcpu>

<cputune>

<vcpupin vcpu="0" cpuset="0" />








</cputune>

<resource>

<partition>/machine</partition>

</resource>

<os>

<type>hvm</type>


VMware, Inc. 82

</os>

<features>

<acpi />

<apic />

<pae />

</features>

<cpu mode="host-passthrough" />

<clock offset="utc" />

<on_poweroff>destroy</on_poweroff>

<on_reboot>restart</on_reboot>

<on_crash>restart</on_crash>

<devices>

<emulator>/usr/bin/kvm-spice</emulator>

<disk type="file" device="disk">

<driver name="qemu" type="qcow2" />

<source file="#folder#/#qcow_root#" />

<target dev="hda" bus="ide" />

<alias name="ide0-0-0" />

<address type="drive" controller="0" bus="0" target="0" unit="0" />

</disk>

<disk type="file" device="cdrom">

<driver name="qemu" type="raw" />

<source file="#folder#/#Cloud_ INIT_ ISO#" />

<target dev="sdb" bus="sata" />

<readonly />

<alias name="sata1-0-0" />


</disk>

<controller type="usb" index="0">

<alias name="usb0" />

<address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x2" />

</controller>

<controller type="pci" index="0" model="pci-root">

<alias name="pci.0" />

</controller>

<controller type="ide" index="0">

<alias name="ide0" />


</controller>

<interface type="network">

<source network="public_interface" />

<vlan>

<tag id="#public_vlan#" />

</vlan>

<alias name="hostdev1" />


</interface>

<interface type="network">

<source network="inside_interface" />

<alias name="hostdev2" />


</interface>

<serial type="pty">

<source path="/dev/pts/3" />

<target port="0" />


VMware, Inc. 83

<alias name="serial0" />

</serial>

<console type="pty" tty="/dev/pts/3">


<target type="serial" port="0" />


</console>

<memballoon model="none" />

</devices>

<seclabel type="none" />

</domain>

The following is a sample template of a VM which uses SR-IOV interfaces. Use this template by making edits, wherever applicable.


<domain type="kvm">

<name>#domain_name#</name>

<memory unit="KiB">8388608</memory>

<currentMemory unit="KiB">8388608</currentMemory>

<vcpu>8</vcpu>

<cputune>









</cputune>

<resource>

<partition>/machine</partition>

</resource>

<os>

<type>hvm</type>

</os>

<features>

<acpi />

<apic />

<pae />

</features>

<cpu mode="host-passthrough" />

<clock offset="utc" />

<on_poweroff>destroy</on_poweroff>

<on_reboot>restart</on_reboot>

<on_crash>restart</on_crash>

<devices>

<emulator>/usr/bin/kvm-spice</emulator>

<disk type="file" device="disk">

<driver name="qemu" type="qcow2" />

<source file="#folder#/#qcow_root#" />

<target dev="hda" bus="ide" />

<alias name="ide0-0-0" />



VMware, Inc. 84

</disk>

<disk type="file" device="cdrom">

<driver name="qemu" type="raw" />

<source file="#folder#/#Cloud_ INIT_ ISO#" />

<target dev="sdb" bus="sata" />

<readonly />

<alias name="sata1-0-0" />


</disk>

<controller type="usb" index="0">

<alias name="usb0" />


</controller>

<controller type="pci" index="0" model="pci-root">

<alias name="pci.0" />

</controller>

<controller type="ide" index="0">

<alias name="ide0" />


</controller>


<mac address='52:54:00:79:19:3d'/>


<source>


</source>


</interface>


<mac address='52:54:00:74:69:4d'/>


<source>


</source>


</interface>

<serial type="pty">


<target port="0" />


</serial>

<console type="pty" tty="/dev/pts/3">


<target type="serial" port="0" />


</console>

<memballoon model="none" />

</devices>

<seclabel type="none" />

</domain>


VMware, Inc. 85

4 Launch the VM by performing the following steps:

a Ensure you have the following three files in your directory as shown in the following sample screenshot:

n qcow file - vcg-root

n cloud-init - vcg-test.iso

n Domain XML file that defines the VM - test_vcg.xml, where test_vcg is the domain name.)

b Define VM.

velocloud@KVMperf2:/tmp/VeloCloudGateway$ virsh define test_vcg.xml

Domain test_vcg defined from test_vcg.xml

c Set VM to autostart.

velocloud@KVMperf2:/tmp/VeloCloudGateway$ virsh autostart test_vcg

d Start VM.

velocloud@KVMperf2:/tmp/VeloCloudGateway$ virsh start test_vcg

5 If you are using SR-IOV mode, after launching the VM, set the following on the Virtual Functions (VFs) used:

a Set the spoofcheck off.

ip link set eth1 vf 0 spoofchk off

b Set the Trusted mode on.

ip link set dev eth1 vf 0 trust on

c Set the VLAN, if required.

ip link set eth1 vf 0 vlan 3500

Note The Virtual Functions configuration step is not applicable for OpenVSwitch (OVS) mode.

6 Console into the VM.

virsh list

Id Name State


VMware, Inc. 86

----------------------------------------------------

25 test_vcg running

velocloud@KVMperf2$ virsh console 25

Connected to domain test_vcg

Escape character is ^]

Special Consideration for KVM Host

n Disable GRO (Generic Receive Offload) on physical interfaces (to avoid unnecessary re-fragmentation in SD-WAN Gateway).

ethtool –K <interface> gro off tx off

n Disable CPU C-states (power states affect real-time performance). Typically, this can be done as part of kernel boot options by appending processor.max_cstate=1 or just disable in the BIOS. For more information, see https://docs.fedoraproject.org/en-US/Fedora/13/html/Virtualization_Guide/chap-Virtualization-KVM_guest_timing_management.html.

n For production deployment, vCPUs must be pinned to the instance. No oversubscription on the cores should be allowed to take place. For more information, see https://docs.fedoraproject.org/en-US/Fedora/13/html/Virtualization_Guide/ch25s06.html.

Post-Installation TasksThis section describes post-installation and installation verification steps.

If everything worked as expected in the installation, you can now login to the VM.

1 If everything works as expected, you should see the login prompt on the console. You should see the prompt name as specified in cloud-init.

2 You can also take a look at /var/log/cloud-init.log. If you see the message below, it is likely that the cloud init runs successfully.


VMware, Inc. 87

https://docs.fedoraproject.org/en-US/Fedora/13/html/Virtualization_Guide/chap-Virtualization-KVM_guest_timing_management.html

https://docs.fedoraproject.org/en-US/Fedora/13/html/Virtualization_Guide/chap-Virtualization-KVM_guest_timing_management.html

https://docs.fedoraproject.org/en-US/Fedora/13/html/Virtualization_Guide/ch25s06.html

https://docs.fedoraproject.org/en-US/Fedora/13/html/Virtualization_Guide/ch25s06.html

3 Verify that the SD-WAN Gateway is registered with SD-WAN Orchestrator.

4 Verify Outside Connectivity.

5 Verify that the MGMT VRF is responding to ARPs.

6 Remove cloud-init so it doesn’t run again.

apt-get purge cloud-init

7 Associate the new gateway pool, (created in the section titled, “ Creating a Gateway and getting the Activation Key”) with the customer.


VMware, Inc. 88

8 Associate the Gateway with an Edge.

9 Verify that the Edge is able to establish a tunnel with the Gateway on the Internet side. From the VMware SD-WAN Orchestrator, go to Monitor > Edges > Overview.

From the VMware SD-WAN Orchestrator, go to Test & Troubleshoot > Remote Diagnostics > [Edge] > List Paths, and click Run to view the list of active paths.


VMware, Inc. 89

10 Configure the Handoff interface.

11 Verify that the BGP session is up.

Upgrade SD-WAN GatewayThis section describes how to upgrade a SD-WAN Gateway installation.

To upgrade a SD-WAN Gateway installation:

1 Download the SD-WAN Gateway Update package.

2 Upload the image to the SD-WAN Gateway system (using, for example, the scp command). Copy the image to the following location on the system: /var/lib/velocloud/software_update/vcg_update.tar.

3 Connect to the SD-WAN Gateway console and run:

sudo /opt/vc/bin/vcg_software_update


VMware, Inc. 90

Custom ConfigurationsThis section describes custom configurations.

NTP ConfigurationNTP configuration involves editing the /etc/ntpd.conf file.

UserdataThis section describes userdata.

#cloud-config




ssh_pwauth: True



runcmd:




- 'sleep 10'


write_files:

- path: "/etc/ntp.conf"

permissions: '0644'

content: |

# Use servers from the NTP Pool Project. Approved by Ubuntu Technical Board

# on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for

# more information.

server #_NTP_SERVER_1_#

server #_NTP_SERVER_2_#

server 1.ubuntu.pool.ntp.org iburst




permissions: '0777'

content: |

#!/usr/bin/python

import json

import commands




exit







# NOTE FOR HAND OFF IT CAN BE "QinQ (0x8100)" "QinQ (0x9100)" "none" "802.1Q" "802.1ad"


VMware, Inc. 91







### EDIT DPDK ###




data["dpdk_enabled"] = #_DKDP_ENABLED_OR_DISABLED_#




power_state:

delay: "+1"

mode: reboot

message: Bye Bye

timeout: 30

condition: True

OAM Interface and Static RoutesIf Gateways are to be deployed with an OAM interface, complete the following steps.

1 Add an additional interface to the VM (ETH2).

VMware

If a dedicated VNIC for Management/OAM is desired, add another vNIC of type vmxnet3. You must repeat the previous step, which is to click OK and then Edit Settings again so you can make a note of the vNIC MAC address.


VMware, Inc. 92

KVM

If a dedicated VNIC for Management/OAM is desired, make sure you have a libvirt network named oam-network. Then add the following lines to your XML VM structure:

…..

</controller>

<interface type='network'>

<source network='public_interface'/>

<vlan><tag id='#public_vlan#'/></vlan>

<alias name='hostdev1'/>

<address type='pci' domain='0x0000' bus='0x00' slot='0x11' function='0x0'/>

</interface>


<source network='inside_interface'/>



</interface>


<source network='oam_interface'/>

<vlan><tag id='#oam_vlan#'/></vlan>



</interface>

<serial type='pty'>

<source path='/dev/pts/3'/>

<target port='0'/>

<alias name='serial0'/>

</serial>

2 Configure the meta-data file with the additional interface.

instance-id: #_Hostname_#

local-hostname: #_Hostname_#

network-interfaces: |

auto eth0


address #_IPv4_Address_#

netmask #_IPv4_Netmask_#

gateway #_IPv4_Gateway_#


auto eth1


metric '13'

address #_MGMT_IPv4_Address_#

netmask #_MGMT_IPv4_Netmask_#

gateway #_MGMT_IPv4_Gateway_#


auto eth2


address #_OAM_IPv4_Address_#

netmask #_OAM_IPv4_Netmask_#


VMware, Inc. 93

up route add -net 10.0.0.0 netmask 255.0.0.0 gw #_OAM_IPv4_Gateway_#

up route add -net 192.168.0.0 netmask 255.255.0.0 gw #_OAM_IPv4_Gateway_#

dns-nameservers # _DNS_server_primary_# #_DNS_server_secondary_#

OAM - SR-IOV with vmxnet3 or SR-IOV with VIRTIOIt is possible in some installations to mix and match and provide different interface types for the Gateway. This generally happens if you have an OAM without SR-IOV. This custom configuration requires additional steps since this causes the interfaces to come up out of order.

1 Record the MAC address of each interface.

VMWare

After creating the machine, go to Edit Settings and copy the Mac address.

KVM

After defining the VM, run the following command:

2 Edit the user-data and lock the mac address to the interface order as follows:

Userdata

#cloud-config



VMware, Inc. 94



ssh_pwauth: True



runcmd:




- 'sleep 10'


write_files:

- path: "/etc/udev/rules.d/70-persistent-net.rules"

permissions: '0644'

content: |

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="#_ETH0_MAC_ADDRESS_#",

ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"



# NOTE ETH2 IS OAM IF NO OAM PRESENT THEM REMOVE




permissions: "0777"

content: |

#!/usr/bin/python

import json

import commands




exit







# NOTE FOR HAND OFF IT CAN BE "QinQ (0x8100)" "QinQ (0x9100)" "none" "802.1Q" "802.1ad"







### EDIT DPDK ###









VMware, Inc. 95

power_state:

delay: "+1"

mode: reboot

message: Bye Bye

timeout: 30

condition: True

Special Consideration When Using 802.1ad EncapsulationIt seems certain that 802.1ad devices do not populate the outer tag EtherType with 0x88A8. Special change is required in user data to interoperate with these devices.

Assuming a Management VRF is configured with S-Tag: 20 and C-Tag: 100, edit the vrf_vlan section in / etc/ config/ gatewayd as follows. Also, define resp_mode to 1 so that the SD-WAN Gateway will relax its check to allow Ethernet frames that have incorrect EtherType of 0x8100 in the outer header.

#cloud-config




ssh_pwauth: True



runcmd:




- 'sleep 10'


write_files:


permissions: '0777'

content: |

#!/usr/bin/python

import json

import commands




exit







# NOTE FOR HAND OFF IT CAN BE “QinQ (0x8100)” “QinQ (0x9100)” “none” “802.1Q” “802.1ad”

data["vrf_vlan"]["tag_info"][0]["resp_mode"] = "1"







VMware, Inc. 96


### EDIT DPDK ###








power_state:

delay: "+1"

mode: reboot

message: Bye Bye

timeout: 30

condition: True

SNMP IntegrationThis section describes how to configure SNMP integration.

To configure SNMP integration:

1 Edit /etc/snmp/snmpd.conf. Add the following lines to the config with source IP of the systems that will be connecting to SNMP service.

The following example will configure access to all counters from localhost via community string vc-vcg and from 10.0.0.0/8 with community string myentprisecommunity using SNMPv2c version. For more information, see the Net-SNMP documentation.

agentAddress udp:161

# com2sec sec.name source community

com2sec local localhost vc-vcg

com2sec myenterprise 10.0.0.0/8 myentprisecommunity# group access.name sec.model sec.name

group rogroup v2c local

group rogroup v2c myenterpriseview all included .1 80

# access access.name context sec.model sec.level match read write notif

access rogroup "" any noauth exact all none none#sysLocation Sitting on the Dock of the Bay

#sysContact Me <[email protected]>sysServices 72master agentx#

# Process Monitoring

## At least one 'gwd' process

proc gwd

# At least one 'mgd' process

proc mgd#

# Disk Monitoring

#

# 100MBs required on root disk, 5% free on /var, 10% free on all other disks

disk / 100000

disk /var 5%

includeAllDisks 10%#


VMware, Inc. 97

# System Load

#

# Unacceptable 1-, 5-, and 15-minute load averages

load 12 10 5

2 Edit /etc/snmp/snmpd.conf. Add the following lines to the config with the source IP of the systems that will be connecting to SNMP service:

# WARNING: only add targeted rules for addresses and ports

# do not add blanket drop or accept rules since Gateway will append its own rules

# and that may prevent it from functioning properly

*filter

:INPUT ACCEPT [0:0]

-A INPUT -p udp -m udp --source 127.0.0.1 --dport 161 -m comment --comment "allow SNMP port" -j

ACCEPT

-A INPUT -p udp -m udp --source 10.0.0.0/8 --dport 161 -m comment --comment "allow SNMP port" -j

ACCEPT

:FORWARD ACCEPT [0:0]

:OUTPUT ACCEPT [0:0]

COMMIT

3 Restart snmp and iptables services:

service snmpd restart

service iptables-persistent restart

service vc_process_monitor restart

Custom Firewall RulesThis section describes how to modify custom firewall rules.

To modify local firewall rules, edit the following file: /etc/iptables/rules.v4

Important Add only targeted rules for addresses and ports. Do NOT add blanket drop or accept rules. SD-WAN Gateway will append its own rules to the table and, because the rules are evaluated in order, that may prevent Gateway software from functioning properly.

*filter

:INPUT ACCEPT [0:0]

-A INPUT -p udp -m udp --source 127.0.0.1 --dport 161 -m comment --comment "allow SNMP port" -j

ACCEPT

:FORWARD ACCEPT [0:0]

:OUTPUT ACCEPT [0:0]

COMMIT

Restart iptables service:

service iptables-persistent restart

service vc_process_monitor restart


VMware, Inc. 98

vmware sd-wan by velocloud partner guide - vmware sd-wan ...€¦ · about vmware sd-wan by...

Documents