the great azure networking tour - copenhagen | april 2019€¦ · • expressroute ipv6 for o365...
TRANSCRIPT
The Great Azure Networking Tour
Morgan Simonsen
Innofactor
About Your Speaker: Morgan Simonsen
• Cloud Evangelist@Innofactor
• P-TSP@Microsoft
• MCSE, MCSA, MCT
• MVP
• Twitter: @msimonsen
• Email: [email protected]
• Blog: morgansimonsen.com
Agenda
• Microsoft Hyper Scale Network
• Network Topology Architecture Patterns
• Network Performance
• Network Virtual Appliances
• Connectivity
• Availability
• Network Security
• Virtual Data Center (vDC)
Users
Internet
The Big (Network) Picture
Backend Connectivity
ExpressRouteVPN Gateways
Azure
Virtual Network
Front-End Access
Reserved Public IPs
ACLs for security
Load balancing
DNS services
DDoS protection (Basic/Standard)
Virtual Network
“Bring Your Own Network”
Segmentation with Subnets
Full control with Routes
and Security groups
vNet Service Endpoints
Backend Connectivity
Point-to-site for dev/test for
Mac/Windows
VPN Gateways for secure
site-to-site connectivity
ExpressRoute for private
enterprise grade connectivity
Infrastructure Services
Platform Services
• 42 Azure regions
• 8,000+ ISP sessions
• 100+ edge sites
• 44 ExpressRoute locations
• 53,000+ km of lit fiber
• 2 per DC (metro)
• 3 per Region (terrestrial)
• 3+ per Continent (subsea)
• Low cost 100G QSPF
• Optimized for metro networks
• Intelligent traffic management
• Automatic failure recovery
Microsoft’s Global Network
Connecting to Azure Services
Regional Networks
Azure Availability Zones
• Protects applications and data from datacenter failures
• Unique physical locations within an Azure region
• Zone: 1 or more datacenters with independent cooling, power and network
• Each region with AZs have at least 3 zones
• Zone-redundant services replicate applications and data across Availability Zones
• AZ give 99,99% SLA on VM uptime
• 2 categories of services that support AZ:
• Zonal service: pinned to a region (VM, managed disk, IP address)
• Zone-redundant service: platform replicates automatically across zones (zone redundant storage, SQL database, Standard LB)
• For best results: combine AZ with Azure Regions
Azure Availability ZonesRegions that support AZ:
• Central US
• France Central
• East US 2 (Preview)
• West Europe (Preview)
• Southeast Asia (Preview)
Services that support AZ:• Linux Virtual Machines• Windows Virtual Machines• Virtual Machine Scale Sets• Managed Disks• Load Balancer• Public IP address• Zone-redundant storage• SQL Database
Pricing
• No additional cost when service deployed across AZs
• AZ-to-AZ data charges
Your Network in Azure
Your Network in Azure
Your Network in Azure
Your Network in Azure
New VPN Gateway SKUs – 6x faster!
SKU Workload Throughput* S2S/V2V P2S SLA
VpnGw1 Production 650 Mbps Max. 30 128 99.95%
VpnGw2 Production 1 Gbps Max. 30 128 99.95%
VpnGw3 Production 1.25 Gbps Max. 30 128 99.95%
Basic Dev/Test 100 Mbps Max. 10 128 99.9%
Your Network in Azure
New VPN Gateway SKUs – 6x faster!
SKU Workload Throughput* S2S/V2V P2S SLA
VpnGw1 Production 650 Mbps Max. 30 128 99.95%
VpnGw2 Production 1 Gbps Max. 30 128 99.95%
VpnGw3 Production 1.25 Gbps Max. 30 128 99.95%
Basic Dev/Test 100 Mbps Max. 10 128 99.9%
Behind the Scenes
Management
Control
Data
Proprietary
applianceAzure Resource
Manager
Controller
Switch (Host)
Management
plane
Data plane
SDN
Control
plane
Key to flexibility and scale is Host SDN
Network Performance – Virtual Network
• SND: Network operations implemented in SW are much more variable than those implemented in HW
• Data center architecture is not homogenous and even varies within a DC; this will impact performance
• For some VMs encapsulation (NVGRE) is handled in SW by the host; for other VMs it can be offloaded to HW
• Latency within vNet: 1-2 ms
• Latency between peered vNets: 1-2 ms (except cross-region peerings)
• Occasional peaks of 3-4 ms due to network fabric reconfigs
• First ping might take 3-4 ms because SND routes need to be built
• Anything above 6 ms would warrant investigation• Unless otherwise explained
Global DDoS Protection
• 2016/12/17: R.I.U. Star Patrol boasts they will take down Xbox and PSN onChristmas Day: https://www.youtube.com/watch?v=L1R3rPdVl1Q
• Feb 2018: Github attacked via a reflection exploit in Memcached generating 1.35terabits of attack traffic, the largest DDoS attack ever recorded.
Azure DDoS Protection offerings
DDoS Protection Standard and WAF
Azure DNS - Private Zone Support
Azure DNS - Private Zone Support
Connectivity – Cross-Premises
Cross-Premises Connectivity
ExpressRoute
Cross-Premises Connectivity
ExpressRoute
S2S VPN
Cross-Premises Connectivity
ExpressRoute
S2S VPN
Cross-Premises Connectivity
Cross-Premises Connectivity
New Features – ExpressRoute & VPN
• 6x VPN throughput
• Improved UltraPerf ExpressRoute gateways (latency)
• Azure Monitor for ExpressRoute & VPN
• OMS Network Performance Monitoring for ExpressRoute
• Network Watcher for VPN
• Resource Health Check for ExpressRoute & VPN
Security & compliance• Custom IPsec/IKE policy for VPN
• Merging Microsoft & Azure Public Peering
• macOS support for P2S
• AD (Radius)-based authentication
• ExpressRoute IPv6 for O365 & Azure public services
• ExpressRoute route filter for Microsoft peering – selected services
• ExpressRoute connection weight
• Multi-site for policy-based firewalls
• VPN BGP routes & peering status
ExpressRoute
Customer’s Network
Azure Public Peering for Azure public IPs
Azure Private Peering for Virtual Networks
Microsoft Peering for Office 365 and Dynamics 365
Partner
Edge
✔
✔
✔
✔
Microsoft
Edge
ExpressRoute
Customer’s Network
Azure Private Peering for Virtual Networks
Microsoft Peering for Office 365 and Dynamics 365
Partner
Edge
✔
✔
✔
✔
Microsoft
Edge
Global VNet Peering
Network Virtual Appliances
• Integrated with Azure Marketplace
• Consistency between on premises & Azure
• Audit/filter traffic to/from VNet
• Re-direct traffic flow with custom routes
Shared services such as NVA can be placed in Central/Hub VNet
UDR routes to resources in peered VNet
10.0.0.0/16
10.2.0.0/16
10.1.0.0/16
10.0.0.1
NVA Transit with vNet Peering Internet
Shared services such as NVA can be placed in Central/Hub VNet
UDR routes to resources in peered VNet
Destination Next hop Next hop IP
0.0.0.0/0 NVA 10.0.0.1
Destination Next hop Next hop IP
0.0.0.0/0 NVA 10.0.0.1
10.0.0.0/16
10.2.0.0/16
10.1.0.0/16
10.0.0.1UDR
UDR
NVA Transit with vNet Peering Internet
Shared services such as NVA can be placed in Central/Hub VNet
UDR routes to resources in peered VNet
Destination Next hop Next hop IP
0.0.0.0/0 NVA 10.0.0.1
Destination Next hop Next hop IP
0.0.0.0/0 NVA 10.0.0.1
10.0.0.0/16
10.2.0.0/16
10.1.0.0/16
10.0.0.1UDR
UDR
NVA Transit with vNet Peering Internet
Shared services such as NVA can be placed in Central/Hub VNet
UDR routes to resources in peered VNet
VNet Peering is non transitive
Spoke can connect to spoke directly or through Forwarder in the Hub
Destination Next hop Next hop IP
0.0.0.0/0 NVA 10.0.0.1
Destination Next hop Next hop IP
0.0.0.0/0 NVA 10.0.0.1
10.0.0.0/16
10.2.0.0/16
10.1.0.0/16
10.0.0.1UDR
UDR
NVA Transit with vNet Peering Internet
FWD
10.1.0.0/16 NVA FWD
On-premises sites can be connected to VNets using shared Gateways
Gateway Transit is easy to setup and cost effective
10.0.0.0/16
10.2.0.0/16
10.1.0.0/16
Gateway Transit with vNet Peering
Gateway
On-premises sites can be connected to VNets using shared Gateways
Gateway Transit is easy to setup and cost effective
10.0.0.0/16
10.2.0.0/16
10.1.0.0/16
Gateway Transit with vNet Peering
GatewayAllowGWTransit
AllowGWTransit
UseRemoteGW
UseRemoteGW
Security for access to Azure services
Securing Access to PaaS Services
Securing Access to PaaS Services
Securing Access to PaaS Services
✔
✔
X
Securing Access to PaaS Services
✔
✔
X
VNet Service Endpoints
• Services in your VNet, managed by Azure!
• Directly extends your VNet to the service
• Secure your critical Azure resources to only your VNet
• Traffic remains on the Microsoft backbone
GA : Azure Storage, Azure SQL Database & Azure Cosmos DB
Allow VNet A
AccountA
Azure Storage
Vnet ARoute
Service Endpoints : Configuration
• Simple-click setup on an Azure subnet
• No NAT or GW devices!
• Network admins can set independently
Securing Azure resources to VNets
Integrated with service portal dashboards
Set for existing or new resources
Secure to VNets by name: Service endpoints is a pre-requisite
Configure once. Automatically applies to all access to child resources.
VNet Service Endpoints : Deep-Dive
Private IP: 10.0.0.6
Public IP1
Subnet 1
Azure Storage
Source: VM private IP (10.0.0.6)
Access is from VNet1: Subnet1
Before:
Source: Public IP1
Storage Diagnostics
SrcIP: 10.0.0.6, AcctA:GetBlob
SrcIP: 10.0.0.6, AcctB:PutBlob
Endpoints:
• Carry VNet identity to the service
• Source IPs switch to private VNet IPs
Service IPs and DNS entries remain as-is today
Enable service endpoint once. Secure resources as and when you want.
VNet1
Access from on-premises
• Secure Azure resources to on-premises via NAT IPs
• Add NAT IPs using IP firewall for services
• With endpoints, service IPs and DNS entries remain as is today
SERVICE ENDPOINT
On-premises
Allow VNet A
Allow Onprem:
NATIPs
AccountAVNetA
ExpressRoute Public Peering
or the Internet
ExpressRoute Private Peering
or Site-to-Site VPN
Internet
Service Tags in NSGs
• Restrict network access to just the Azure services you use.
• Maintenance of IP addresses for each tag provided by Azure
• Support for global and regional tags (varies by service)
Network Security Group (NSG)
Action Name Source Destination Port
Allow AllowStorage VirtualNetwork Storage Any
Allow AllowSQL VirtualNetwork Sql.EastUS Any
Deny DenyAllOutBound Any Any Any
Azure Services
Internet
Allow only Azure service traffic
Deny Internet outbound
Preview: Azure Storage, SQL, TrafficManager
Service Endpoints: Filter service traffic with appliances
Subnet 1 NVA Subnet
Allow NVA Subnet
SERVICE ENDPOINT
Route: 0/0->NVA
Filter: Allow myacct*.blob.core.windows.net
• Service endpoints bypass NVAs for service traffic, if registered on originating subnet
• Optionally, continue using NVAs for auditing/filtering service traffic
• Register endpoint on NVA subnet instead and secure resouces to that subnet
Stitching Azure services together
DEPLOYHDI Subnet
Allow HDI subnet InternetHDI Service
• Secure Azure resources to managed service subnets with endpoints
• Applies to all services directly deployed into VNet
Allow only Azure storage traffic
Services in a Virtual Network
Azure Storage
Azure SQL Database
Azure Cosmos DB
SQL DB Managed Instance
Azure Active Directory Domain Services for ARM
Azure Batch for ARM
Azure App Service V2
Azure API Management
Azure Batch for ASM VNets
HDInsight
Azure App Service V1
RedisCache
More services on the roadmap!
Deploy into Virtual Network VNet Service Endpoints
vNet Security
Simplifying the NSG Management
• Denying Outbound Internet access is common practice
• Automating NSG rules to whitelist Azure Datacenter IP ranges is tedious and error prone
• Introducing Augmented rules, comma separated list of IP address ranges
• Service Tags are named monikers for Azure Services that are usable in NSG
Network Security Group (NSG)
Action Name Source Destination Port
Allow AllowStorage VirtualNetwork IP1, IP2 … IP100 Any
Allow AllowSQL VirtualNetwork IP101, IP102 .. Any
Deny DenyAllOutBound Any Internet Any
Azure Services
Internet
Allow only Azure service traffic
Deny Internet outbound
Storage, SQL, AzureTrafficManager
Network Security Group (NSG)
Action Name Source Destination Port
Allow AllowStorage VirtualNetwork Storage Any
Allow AllowSQL VirtualNetwork Sql.EastUS Any
Deny DenyAllOutBound Any Any Any
Application Security Groups
Network security as natural extension of application architecture.
Security policies for dynamic workloads
Universal NSG:
Single policy for your virtual network
Network Security Group (NSG)
Action Name Source Destination Port
Deny BlockQuarantineVMs Any QuarantineVMs Any
Allow RemoteRDP VirtualNetwork WindowsVMs 3389 (RDP)
Allow RemoteSSH SysAdminVMs LinuxVMs 22 (SSH)
Allow AllowInternetToWeb Internet Web 80,8080 (HTTP)
Allow AllowWebToApp Web App 443 (HTTPS)
Allow AllowAppToDb App Db 3306 (MySQL)
Allow AllowMyOnPremises10.0.1.0/24, 192.168.2.1/25
13.68.1.64/28, 137.116.1.0/25,…
80,8080, 443
Deny DenyAllOutBound Any Any Any
Application Security Groups
Effortless scaling
Network security management simplified
Easy to manage, update and enforce new policies
Network Security Group (NSG)
Action Name Source Destination Port
Deny BlockQuarantineVMs Any QuarantineVMs Any
Allow AllowInternetToWeb Internet Web 80,8080 (HTTP)
Allow AllowWebToApp Web App 443 (HTTPS)
Allow AllowAppToDb App Db 3306 (MySQL)
Deny DenyAllOutBound Any Any Any
Web App Db
QuarantineVMs
Enhanced Networking Snapshot for VMs
Improved Auditing & Metrics
✓ NSG Data plane logs
✓ NSG Data plane analytics
✓ WAF Diagnostics logs
✓ DDoS metrics and logs
Availability
Outbound Connections in Azure
Scenario Method IP protocols Description
1. VM with an Instance Level Public IP address (with or without Load Balancer)
SNAT, port masquerading not used
TCP, UDP, ICMP, ESP Azure uses the public IP assigned to the IP configuration of the instance's NIC. The instance has all ephemeral ports available.
2. Public Load Balancer associated with a VM (no Instance Level Public IP address on the instance)
SNAT with port masquerading (PAT) using the Load Balancer frontends
TCP, UDP Azure shares the public IP address of the public Load Balancer frontends with multiple private IP addresses. Azure uses ephemeral ports of the frontends to PAT.
3. Standalone VM (no Load Balancer, no Instance Level Public IP address)
SNAT with port masquerading (PAT)
TCP, UDP Azure automatically designates a public IP address for SNAT, shares this public IP address with multiple private IP addresses of the availability set, and uses ephemeral ports of this public IP address. This is a fallback scenario for the preceding scenarios. We don't recommend it if you need visibility and control.
Why Load Balancing?
Load Balancer Application Gateway
aka.ms/lbpreview aka.ms/appgw
Traffic Manager
aka.ms/trafficmanager
Load Balancer
Public IP
Address
Private IP
Address
Outbound
connections
Public
Load Balancer
Internal
Load Balancer
Public IP
Address
Part of the Azure SDN stack
High performance & low latency for
all TCP & UDP applications
Flow-based Load Balancing
with Health Probing
Inbound NAT rules
Outbound Connections
Load balancing scenarios
DB DB
Scalable
Redundant
DB DB
Scalable
Redundant
+ Protection
DB DB DB DB
REGION A REGION B
Load Balancer SKU Choices
Metric Load Balancer Basic Load Balancer Standard
Scale Up to 100 Backend instances Up to 1000 backend instances
LB Scope Non-zonal Frontend IPs Zone redundant and Zonal Frontend IPs
Fault Tolerance Works in a Availability set Works in Availability set and Availability zones
Diagnostics Basic NAT and Probe health status Integrated Front end and Backend health metrics
NVA - Supports HA Ports
Cost Free Charged at GA
Http://aka.ms/lbpreview
Load BalancerNew Load Balancer, SKUs
Pool up to 1000 instances in VNet, incl. 1000 instance VMSS
Multiple VMSS
Zone-redundant data path with single IP
Zonal frontends
Cross-zone load balancing
Cross-zone VMSS
HA Ports for NVAs and more
Advanced analytics (Traffic Counters, Per Endpoint health probe status, Continuous in-band data plane health, Inbound connection attempts, Outbound connections)
Application GatewayWhat’s New?
Traffic Manager
Expands to entire VNet
• Standalone VMs without Availability Sets
• Standalone VMs with Availability Sets
• Virtual machine scale sets with up to 1000 instances
• Cross-zone VMSS https://aka.ms/xzonevmss
• Multiple VMSS
• Blending VMs and virtual machine scale sets
Load Balancer no longer uses Availability Set as a boundary for the backend pool
Network Security Groups are required for Public IP and all backend instances
Load Balancer Standard
VNet
VM scale set VM scale set VM scale set
Zone 1 Zone 2 Zone 3
Load Balancer Standard
VNet
VM Scale Set (coming soon)
Zone 1 Zone 2 Zone 3
Reference Architecture : Web App with AZs
Zone 1
Web
Ap
pSQ
L
Fro
nt
En
d
Su
bn
et
Zone 2
SQ
L
Zone 3
SQ
L
DB
Su
bn
et
VN
etW
eb
Ap
p
Web
Ap
p
Load Balancer Standard • Zone-redundant Load Balancer Standard balances across web frontends with single frontend IP
• Web frontend across 3 AZs
• Data layer across 3 AZs
• SQL on IaaS
• SQL Azure or CosmosDB
• NoSQL (Cassandra, MongoDB)
ARM Template—Cross-Zone VMSS and LB
ARM Template—Zonal VM
Takeaways
Standard
Up to 1000 backend instances
Zone-redundant frontendZonal frontend
Availability Sets not required and Availability Zones
Integrated Frontend and Backend health metrics
Supports HA Ports
NSG required
Charged at GA
Basic
Up to 100 backend instances
Non-zonal frontend
Availability Set (single)
Basic NAT and Probe health status
-
NSG optional
Free
vs.
You can use
Load Balancer Standard
for TCP & UDP scenarios with:
• Larger scale
• Greater flexibility
• HA Ports
• New metrics
• Availability zones
Preview GA
Network Topology Network
Architecture Pattern – Virtual
Datacenter
• Best practices vs. Best ideas• Scenario dependent designs
Thank you!
Please do not forget to evaluate the session before you leave!