Exchange Architecture

Pavel GolubinSolution Sales Professional

Agenda Client Access server role

Mailbox server role

Service Availability Changes

5 server roles

Tightly-coupledin terms of

versioning

functionality

user partitioning

geo-affinity

Previous Server Role Architecture

Internal Network Phone system (PBX or VOIP)

Web browse

r

Outlook (remote

user)

Mobile phone

Line of business application

MailboxStores mailbox

and public folder items

Unified MessagingVoice mail and

voice access

Client AccessClient connectivity

Web services

Outlook (local user)

Layer 7 LB

AD

ExternalSMTP

servers

Edge TransportRouting and

AV/AS

Hub TransportRouting and policy

Forefront Online

Protection for Exchange

Architecture overview

Enterprise networkInternet

Exchange building blocks

Client Access Server comprises of client protocols and SMTP

Mailbox Server hosts all components to process, render and store data

Laye

r 4 lo

ad

bal

ance

r

CAS

PBXLocal clientsRemote clients & devices

Edge

MBX

Functional Layering

AuthN, Proxy, Re-direct

Protocols, API, Biz-logic

Assistants, Store, CI

Exchange 2010

AuthN, Proxy, Re-direct

Store, CI

Protocols, Assistants, API, Biz-

logic

Exchange 2013

Client Access

Mailbox

Client AccessHub Transport,

Unified Messaging

Mailbox

HardwareLoad Balancer

L4 LBL7 LB

Client Access Server Role

Client Access Server role• Domain-joined machine in the internal Active Directory

forest Thin, stateless (protocol session) server

• Comprised of three components: Client access protocols (HTTP, IMAP, POP) SMTP UM Call Router

• Exchange-aware proxy server Understands requests from different protocols (OWA, EWS, etc.) Supports proxy and redirection logic for client protocols Capable of supporting legacy servers with redirect or proxy logic Contains logic to route specific protocol requests to their destination end-point

Client Access Array• A group of CAS organized in a load-balanced

configuration Designed to work with TCP affinity (aka, layer 4 LB) Does not require session affinity (aka, layer 7 LB)

• Provides a unified namespace and authentication Similar to Exchange 2010 in terms of providing a unified endpoint

for client connectivity and authentication

Load Balancer

MDB

HTTP Proxy

IISClient Acces

s

RPC CA

Mailbox

IIS

RPS OWA, EAS, EWS, ECP, OAB

POP, IMAP SMTP UM

POP IMAP

Transport UM

SMTPPOP, IMAPHTTP

MailQ

Client Protocol Architecture in Exchange 2013

RpcProxy

SMTP

SIP

Redirect

SIP + RTP

POP/IMAPOutlook Web App Outlook EAS EAC PowerShell

Outlook Connectivity in Exchange 2013• Exchange 2013 supports RPC/HTTP only; No

RPC/TCP Simplifies the protocol stack Provides an extremely reliable and stable connectivity model because RPC session is always on Mailbox server hosting active copy

Eliminates need for RPC CAS Array namespace(s) Eliminates end user interruptions like “The Exchange administrator has made a change that requires you to quit and restart Outlook” during mailbox moves or *overs

Namespace Simplification

• Exchange 2013 no longer requires multiple namespaces for site resilient solutions or site specific scenarios

• Easy to setup a single, worldwide client access namespace Can be used in coexistence with Exchange 2010

A Single Common Namespace ExampleGeographical DNS Solution

Sue (somewhere in

NA) DNS Resolution

DAG

VIP #1 VIP #2

Sue (traveling in APAC)DNS Resolution via Geo-

DNSRound-Robin between # of VIPs

DAG

VIP #3 VIP #4

mail.contoso.com

Round-Robin between # of VIPs

CAS 2013 Client Protocol Connectivity Flow

Layer 4 LB

CAS

IIS

HTTP Proxy

MBX

Protocol Head

DB

Layer 4 LB

CAS

IIS

HTTP Proxy

MBX

Protocol Head

DB

Site

Boundary

HTTP

Local Proxy Request OWA Cross-Site Redirect Request

MBX

Protocol Head

DB

Site

Boundary

Cross-Site Proxy Request

HTTP

HTTP

HTTPHTTP HTTP

13

CAS 2013 Client Protocol Connectivity FlowExchange 2010 Legacy Coexistence

Layer 4 LB

CAS 2013

IIS

HTTP Proxy

MBX2013

Protocol Head

DB

Exchange 2010 CAS

Protocol Head

MBX

Store

DB

Site

Boundary

E2010 CAS

Protocol Head

MBX

Store

DB

RPC RPC

Cross-Site Proxy Request

Layer 7 LBOWA

Cross-Site

Redirect Request

HTTP

14

Benefits of new architecture

Simplifies the network layer

Removes need for RPC CAS Array

Provides deployment flexibility

Mailbox Server Role

16

Mailbox Server Role• Server that hosts the components that process,

render and store Exchange data Includes components previously found in separate roles

• Only Client Access servers connect directly to the Mailbox server Clients connect to Client Access servers Note – one exception is UM with RTP

Connectivity to a mailbox is always provided by the protocol instance local to the active database copy

Database Availability Group• Collection of servers that form a unit

of high availability• Boundary for replication and *over• DAG members can be in different

sites• Can have a maximum of 16 Mailbox

servers

MBX1

MBX2

MBX16

Mailbox-related changes

Managed Store

IOPS reductions

Larger mailbox support

Modern public folders

New search infrastructure

Managed Store

Managed Store• Store service process

(Microsoft.Exchange.Store.Service.exe) Manages worker process lifetime based on mount/dismount Logs failure item when store worker process problems detected Terminates store worker process in response to “dirty” dismount

during failover

• Store worker process (Microsoft.Exchange.Store.Worker.exe) One process per database, RPC endpoint instance is database GUID Responsible for block-mode replication for passive databases Fast transition to active when mounted Transition from passive active increases ESE cache size 5X

Microsoft Exchange Replication service• MSExchangeRepl.exe

Detecting unexpected database failures Issues mount/dismount operations to Store

Provides administrative interface for management tasks

Initiates failovers on failures reported by ESE, Store and Responders

IOPS Reductions

E2010 vs. E2013 Performance Comparison* Results based on daily Outlook cached mode Load Generator simulations (10 databases, 1000 users) to measure key metrics used to identify performance improvements/regressions (Beta2 build 466, subject to change)

48-76% IOPS reduction (disk IOPS capacity not expected to change)

18-41% Average RPC Latency reduction

17-34% increase in CPU per RPC processed (offset by additional CPU cores)

~4X increase in store memory overhead (~4GB vs. ~1GB not including ESE cache)DB IOPS/Mailbox

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.65

0.16

E14SP1 E15 Build 466

RPC Average La-tency

Mcycles per RPC packet

Store Memory per Mailbox (MB)

0

0.5

1

1.5

2

2.5

3

3.5

43.99

3.09

0.736420927114487

2.35

3.75

3.16318300602913

E14SP1 E15 Build 46624

IOPS Reductions

Exchange 2003 Exchange 2007 Exchange 2010 Exchange 20130

0.2

0.4

0.6

0.8

1

DB IOPS/Mailbox

IOPS/Mailbox

~99.5% Reduction!

Larger Mailboxes

Support for Larger Mailboxes• Large Mailbox Size is 100

GB+ Aggregate Mailbox =

Primary Mailbox + Archive Mailbox + Recoverable Items

1-2 years of mail (minimum)

• Increase IW productivity• Eliminate or reduce PST

files• Eliminate or reduce third-

party archive solutions• OST size control with

Outlook 2013

Time Items Mailbox Size

1 Day 150 11 MB

1 Month 3300 242 MB

1 Year 39000 2.8 GB

2 Years 78000 5.6 GB

4 Years 156000 11.2 GB

Modern Public Folders

Modern Public Folders• Public folders based on the mailbox

architecture • Single-master model

Hierarchy is stored in a PF mailbox (one writeable) Content can be broken up and placed in multiple

mailboxes The hierarchy folder points to the target content

mailbox• Because it’s a mailbox, it’s in a mailbox

database…thus, High availability achieved through continuous

replication No separate replication mechanism

• Similar administrative features to current PFs No end-user changes

MBX2013

CAS2013

MBX2013

MBX2013

Public logon

Private logon

Public logon

Content Mailbox

Hierarchy Mailbox

Modern Public Folders• 1 - User connects to their home

Public Folder mailbox first, which should be located near their primary mailbox.

• 2- Folder contents live in one specific mailbox for that folder. All content operations are redirected to the mailbox for that folder

• 3 – Folder hierarchy changes are intercepted and written to writeable copy of Public Folder hierarchy

• 4 – All Public Folder mailboxes listen for hierarchy changes and update similar to Outlook clients

• 5 - When a Public Folder mailbox gets full, move some folders to a new mailbox

1

2 3 5

4

New Search Infrastructure

New Search Infrastructure

Uses Search Foundation

Significantly improved query performance

Significantly improved indexing performance

Search Foundation Primer

Core

Catalog

CTS

Incoming Documents

FilterWord Break

Content

XForm

MARS Write

r

Incoming Queries

“CTS Flow”

IMSContent XForm

Query

Parse

“IMS Flow”

Res

ults

Mailbox

DB

Idx

Passive

Exchange Search Infrastructure

TransportTransport CTS

MailboxStore

DB

Index Node

Idx

ExSearch

Loca

l Del

iver

y

Reliable

Event

CTS

Read Content

MBX2013

Log

MBX2013

Log

Transport-related Changes

Transport in MBX 2013 has been broken down into three componentsTransport Service – Stateful; handles SMTP mail flow for the organization and performs content inspection (Was previously referred to as “Hub Transport”)Mailbox Transport Delivery Service - Receives mail from the Transport Service and deliveries to the Mailbox DatabaseMailbox Transport Submission Service - Takes mail from the Mailbox Database and submits to the Transport Service.

Transport still has the following responsibilitiesReceives all inbound mail to the organization (proxied through CAS or direct)Submits all outbound mail from the organization (proxied through CAS or direct)Handles all internal message processing such as transport rules, content filtering, and antivirusPerforms mail-flow routingQueues messagesSupports SMTP extensibility

Transport Components

36

Mail Delivery Flow

DAGMBX2013-

1

MBX Transport

Transport

DB2DB1

MBX2013-2

MBX Transport

Transport

CAS 2013 or MBX 2013

DB2DB1DB1 DB1

MAPI MAPI

SMTP

37

Service Availability–Related Changes

All core Exchange functionality for a given mailbox is served by the MBX 2013 server where that mailbox’s database is currently activatedMailbox access fails over when a database fails over Protocols shift to the server hosting the active database copy

Managed availability: Internal monitoring and high availability are tied together and can be used to detect and recover from problems as they occur and are discovered

Best copy selection now includes health of services when selecting best copy

Failover time reductions

Service Availability Improvements

39

Managed Availability

DB2

Layer 4 LB

CAS-1

MBX-1

DB2

OWAMBX-2

DB1 DB2

OWAMBX-3

DB1 DB1DB1

CAS-2

OWAOWA

MA: Fa

ilove

r dat

abas

e

OW

A res

tart

com

ple

te

OW

A s

end

OW

A fai

lure

det

ecte

d

OW

A fai

lure

OW

A res

tart

serv

ice

OW

A v

erifi

ed a

s

heal

thy

OW

A res

tart

ser

vice

faile

d

OW

A s

end

OW

A fai

lure

det

ecte

d

OW

A fai

lure

OW

A res

tart

serv

ice

OW

A v

erifi

ed a

s

heal

thy

OW

A s

ervi

ce res

tart

s

time

Managed availability + retries …“Stuff breaks and the experience does not”

DB1

DAG

40

Every message is redundantly persisted before its receipt is acknowledged to the sender

Delivered messages are kept redundant in transport, similar to active messages

Every DAG represents a transport HA boundary and owns its HA implementationIf you have a stretched DAG, you also have transport site resilience

Resubmits due to transport DB loss or MDB *over are fully automatic and do not require any manual involvement

Transport High Availability Improvements

41

Same fundamental concept as in Exchange 2010, with new implementation in Exchange Preview

All mail is made redundant on a another server

Shadow messages are queued until Primary server successfully delivers the mail

Shadow server regularly heartbeats Primary server for status on the primary copy

On Primary server failure, Shadow server self-promotes itself as the Primary and delivers mail

Shadow Redundancy

42

New transport configuration – RejectOnShadowFailure ensures that no message is acknowledged and accepted unless a shadow copy was first created

Messages are made redundant on other servers within a DAG, stamp group, or site

Messages are tried for a configurable amount of time before giving up and rejecting the message

Guaranteed Redundancy

43

Messages are made redundant to DAGs, stamp group (in FFO), or site (in DAG-less environments)

Messages are ‘preferred’ to be made redundant in remote sites when DAGs are spread across sitesTransport HA can be configured to make a message redundant to local-only or remote-only sites

In DAG-less environments, messages are made redundant to a local site only to avoid shadow messages being spread across a resource forest

Scoped Redundancy

44

Introduced in Office 365 to redundantly store all mail for a configured time span to protect against mailbox irrecoverable failures

Now has a “shadow” equivalent and is no longer a Single Point of Failure (SPOF)

Consolidates and improves Exchange 2010 Transport Dumpster functionalityAlthough SafetyNet retains data for a set period of time, regardless of whether the message has been successfully replicated to all database copies or delivered to final destination

Processes replay requests from “primary” or “shadow” SafetyNet for lossy mailbox failovers

SafetyNet Enhancements

45

Transport HACAS2013

or MBX2013

1. Maintain a copy of the message in the queue database but don’t acknowledge the DATA verb

2. Generate a shadow copy on another MBX 2013 server in the DAG (remote site preferred)

3. Wait for acknowledgement from the shadow server4. Send acknowledgement to SMTP client5. Delete message from queue after SafetyNet threshold has

expired

MBX2

Transport

MBX Transport

Mail.que

StoreDB1 DB2

MBX3

Transport

MBX Transport

Mail.que

StoreDB1 DB2

MBX4

Transport

MBX Transport

Mail.que

StoreDB1 DB2

MBX5

Transport

MBX Transport

Mail.que

StoreDB3 DB4

MBX6

Transport

MBX Transport

Mail.que

StoreDB3 DB4

MBX7

Transport

MBX Transport

Mail.que

StoreDB3 DB4

MBX8

Transport

MBX Transport

Mail.que

StoreDB3 DB4

MBX1

Transport

MBX Transport

StoreDB1 DB2

Mail.que

SMTP

R1, R2, R3

R2

R1R3

Site

Bou

nd

ary

250 OK

R3

250 OK

R3

R1, R2, R3

250 OK

250 OK

Recipient StateR1 – ActiveR2 – ActiveR3 – Active

Recipient StateR1 – WaitingMDBReplR2 – ActiveR3 – Active

Recipient StateR1 – WaitingMDBReplR2 – WaitingMDBReplR3 – Active

Recipient StateR1 – WaitingMDBReplR2 – WaitingMDBReplR3 – Processed

Recipient StateR3 – Active

Recipient StateR3 – WaitingMDBRepl

Recipient StateR3 – Processed

Recipient StateR1 – ProcessedR2 – WaitingMDBReplR3 – Processed

Recipient StateR1 – ProcessedR2 – ProcessedR3 – Processed

LogLog Log LogLog Log

LogLog Log46

Q&A

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.