Going 64-bit with Microsoft Exchange Going 64-bit with Microsoft Exchange ServerServer  20072007

Published: November 2006

Messaging Trends and NeedsMessaging Trends and NeedsCategory

2002/2003 2003/2004

2004/2005

2005/2006 2006/2007

Mailboxes total 71,000 80,000 95,000 110,000 130,000

Exchange ActiveSync users per month

Not applicable

6,000 13,000 21,000 31,000

Outlook over remote procedure call (RPC)/Hypertext Transfer Protocol (HTTP) per month

Not applicable

20,000 25,000 60,000 60,000

Internet message submissions per day (unfiltered)

6,000,000 9,000,000 11,300,000 13,000,000 13,500,000

Blocked message submissions per day

2,500,000 7,500,000 10,000,000 10,500,000 11,000,000

Maximum message size 2 MB 5 MB 10 MB 10 MB 10 MB *

E-mail volume per user per day

10 MB 15 MB 15 MB 20 MB 25 MB

Number of mailbox servers 113 38 34 30 62

Typical mailbox quota 100 MB 200 MB 200 MB 200 MB 500 MB or 2 GB

Total mailbox data 7 terabytes 17 terabytes 19 terabytes 22 terabytes Up to 260 terabytes

* Size will increase after completion of Exchange Server 2007 migration

32-bit Platform Limitations32-bit Platform Limitations

● Physical and Virtual Memory LimitationsPhysical and Virtual Memory Limitations● Kernel-Mode Memory IssuesKernel-Mode Memory Issues● Virtual Memory Fragmentation Virtual Memory Fragmentation ● Input/Output Operations Input/Output Operations ● Input/Output and Storage RequirementsInput/Output and Storage Requirements● Backup DependenciesBackup Dependencies

Exchange Server 2007 Deployment Exchange Server 2007 Deployment GoalsGoals

● Increase mailbox quotasIncrease mailbox quotas● Migrate 130,000 users in three monthsMigrate 130,000 users in three months● Leverage new possibilities on the 64-bit Leverage new possibilities on the 64-bit

platformplatform● Lower IOPS per user (from 0.9 to .27 with Exchange Lower IOPS per user (from 0.9 to .27 with Exchange

Server 2007)Server 2007)● Implement Exchange Server 2007 server rolesImplement Exchange Server 2007 server roles● Implement new storage designImplement new storage design● Improve backup with CCRImprove backup with CCR

Server Role BenefitsServer Role Benefits

● Structured System Administration and Structured System Administration and MaintenanceMaintenance

● Optimized Server Hardware and Optimized Server Hardware and Software ComponentsSoftware Components

● Targeted Load-Balancing and Targeted Load-Balancing and ClusteringClustering

● Increased Flexibility in the Messaging Increased Flexibility in the Messaging EnvironmentEnvironment

Server Hardware UtilizedServer Hardware Utilized

Server roleProcessors Memor

yRaw Storage Capacity

Client Access Two dual-core AMD Opteron2.2 GHz

4 GB 50 GB for the operating system20 GB for miscellaneous data (tools, temp files, dumps, and so forth)70 GB for Exchange Server and Internet Information Services (IIS) files

Edge Transport Two dual-coreAMD Opteron 2.2 GHz

8 GB 50 GB for the operating system20 GB for transaction log files for queue database 275 GB for queue database files, tracking logs, protocol logs, and similar files

Hub Transport Two dual-coreAMD Opteron 2.2 GHz

8 GB 50 GB for the operating system20 GB for transaction log files for queue database275 GB for queue database files, tracking logs, protocol logs, and similar files

Unified Messaging One dual-coreAMD Opteron 2.2 GHz

4 GB 50 GB for the operating system20 GB for miscellaneous data70 GB for Exchange Server files

Mailbox Server Hardware UtilizedMailbox Server Hardware Utilized

Server roleProcessors Memor

yRaw Storage Capacity

Mailbox I(2,000 users with 500-MB quotas)

Two dual-coreIntel Xeon 2.66 GHz

12 GB 50 GB for the operating system96 GB for Exchange Server binaries730 GB for maintenance and recovery584 GB for transaction log files2,336 GB for database files5,000 GB for backup files

Mailbox II(2,400 users with 2-GB quotas)

Two dual-coreIntel Xeon 3.0 GHz

16 GB 50 GB for the operating system250 GB for Exchange Server binaries and logging825 GB for maintenance and recovery data1,600 GB for transaction log files6.6TB GB for database files12TB GB for backup files

Mailbox III(3,600 users with 2-GB quotas)

Four dual-coreAMD Opteron 2.6 GHz

24 GB 50 GB for the operating system225GB for Exchange Server binaries825 GB for miscellaneous data2.5TB for transaction log files10TB for database files22TB for backup files

Microprocessor ChoicesMicroprocessor Choices

● AMD Opteron 885 2.6 GHzAMD Opteron 885 2.6 GHz● Intel Xeon 5160 3.00 GHz Intel Xeon 5160 3.00 GHz ● Intel Xeon 5150 2.66 GHzIntel Xeon 5150 2.66 GHz

Random Access Memory ConfigurationRandom Access Memory Configuration

Server typeNumber of

usersQuota per mailbox

Minimum RAM

Microsoft IT RAM

Client Access Not applicable Not applicable 1 GB 4 GB

Edge Transport Not applicable Not applicable 1 GB 8 GB

Hub Transport Not applicable Not applicable 1 GB 8 GB

Mailbox I 2,000 500 MB 12 GB 12 GB

Mailbox II 2,400 2 GB 14 GB 16 GB

Mailbox III 3,600 2 GB 20 GB 24 GB

Unified Messaging Not applicable Not applicable 1 GB 4 GB

System Drive and Page FileSystem Drive and Page File

● 50 GB system drive on first logical 50 GB system drive on first logical diskdisk

● Page file size = RAM amount + Page file size = RAM amount + 1010  MBMB

Mailbox Server Storage DesignMailbox Server Storage Design

● Storage Capacity RequirementsStorage Capacity Requirements● Number of Mailboxes and Size LimitsNumber of Mailboxes and Size Limits● Deleted Items RetentionDeleted Items Retention● Context IndexesContext Indexes● Unexpected Database GrowthUnexpected Database Growth● Transaction LogsTransaction Logs

Mailbox Server Storage Design (cont)Mailbox Server Storage Design (cont)

● Capacity Needs for Database and Capacity Needs for Database and Transaction Log DrivesTransaction Log Drives

Server type

Number of

users

Mailbox

quotaUser data

Database

overhead

Content indexin

gGrowth reserve

Database drives ttotal

Transaction log drives

total

Mailbox I 2,000 500 MB 1 terabyte 200 GB 60 GB 120 GB 1.3 terabytes

350 GB

Mailbox II 2,400 2 GB 4.5 terabytes

960 GB 300 GB 600 GB 6.5 terabytes

1.6 terabytes

Mailbox III 3,600 2 GB 7 terabytes

1.4 terabytes

450 GB 900 GB 9.75 terabytes

2.5 terabytes

Mailbox Server Storage Design (cont.)Mailbox Server Storage Design (cont.)

● RAID Controller configurationRAID Controller configuration● DAS-based RAID controllers with a 256-DAS-based RAID controllers with a 256-

MB battery-backed cacheMB battery-backed cache● SAN-based controllers with a battery-SAN-based controllers with a battery-

backed cache of 16 GBbacked cache of 16 GB● Logical Storage ConfigurationLogical Storage Configuration

● Mailbox I serversMailbox I servers● Designed during Beta 1 Designed during Beta 1 ● Use DAS-based solutionUse DAS-based solution● Move from LFF to SFF disksMove from LFF to SFF disks

● Mailbox II and III serversMailbox II and III servers● Use SAN-based solutionUse SAN-based solution● Designed during Beta 2Designed during Beta 2

External Storage ConfigurationExternal Storage Configuration

● Universal Storage Building BlocksUniversal Storage Building Blocks● Consists of two storage enclosures and supports up to 1,200 usersConsists of two storage enclosures and supports up to 1,200 users● Mailbox I and II servers use two building blocks Mailbox I and II servers use two building blocks ● Mailbox III server types use three building blocks based on number of usersMailbox III server types use three building blocks based on number of users

External Storage Configuration (cont.)External Storage Configuration (cont.)

Server typeCapacity per

diskDisks per enclosure

RAID 10 drives

Mailbox I 146 GB 10 Drive 1: A1-A4, B1-B4Drive 2: A5-A8, B5-B8

Drive 3: A9-A10, B9-B10

Mailbox II and Mailbox III 300 GB 15 Drive 1: A1-A6, B1-B6Drive 2: A7-A12, B7-B12

Drive 3: A13-A15, B13-B15

● Mailbox Databases, and Storage GroupsMailbox Databases, and Storage Groups● Data distributed across seven mailbox databases per driveData distributed across seven mailbox databases per drive

● Fault Tolerance and ReliabilityFault Tolerance and Reliability● Storage enclosures connected to different controller ports per building blockStorage enclosures connected to different controller ports per building block

External Storage Configuration (cont.)External Storage Configuration (cont.)

Server DesignRAID

Controller Port 1

RAID Controller Port

2

Comments

Mailbox I and II A, C B, D The Mailbox I and II designs use two USBBs with the enclosure pairs A-B and C-D.

Mailbox III A, C, E B, D, F The Mailbox III design uses three USBBs with the enclosure pairs A-B, C-D, and E-F.

Exchange Server Backup SolutionExchange Server Backup Solution

● With Exchange Server 2003 on the 32-bit With Exchange Server 2003 on the 32-bit platformplatform● 1.2 terabytes per backup cycle1.2 terabytes per backup cycle● Full restore can take up to 24 hoursFull restore can take up to 24 hours

● Three redundancy levels with Exchange Three redundancy levels with Exchange Server 2007 on 64-bit platformServer 2007 on 64-bit platform● Mailbox storeMailbox store● Hub server transport dumpsterHub server transport dumpster● Online database backups Online database backups

Mailbox Store Backup FeaturesMailbox Store Backup Features

● Mailbox servers in CCR clustersMailbox servers in CCR clusters● Two-node Majority Node Set (MNS) server Two-node Majority Node Set (MNS) server

clusters with file share witnessclusters with file share witness● Requires a single mailbox store database per Requires a single mailbox store database per

storage group storage group

● All cluster nodes use DAS and separate All cluster nodes use DAS and separate storage subsystemsstorage subsystems

Transport Dumpster FeaturesTransport Dumpster Features

● Hub Transport servers retain a copy of all Hub Transport servers retain a copy of all messages delivered to clustered mailbox serversmessages delivered to clustered mailbox servers

● Messages remain for a specified period in Messages remain for a specified period in transporttransport dumpster queuedumpster queue● Queued messages available for redeliveryQueued messages available for redelivery● Necessary if a failover occurs to a passive node before Necessary if a failover occurs to a passive node before

CCR has copied the most recent messages CCR has copied the most recent messages ● Need to enable transport dumpster feature Need to enable transport dumpster feature

through through set-TransportConfigset-TransportConfig PowerShell PowerShell command command

● Transport dumpster uses additional disk spaceTransport dumpster uses additional disk space

Online Database BackupsOnline Database Backups

● Daily incremental online backups for redundancy Daily incremental online backups for redundancy and weekly full backupsand weekly full backups

● Separate RAID controller and RAID5 array backs Separate RAID controller and RAID5 array backs up each active node in the CCR clustersup each active node in the CCR clusters

● Streaming disk-to-disk backup with command-line Streaming disk-to-disk backup with command-line scripts for backup through NTBackup.exe scripts for backup through NTBackup.exe

Storage group Mon Tue Wed Thu Fri Sat Sun

SG 1 Full Inc Inc Inc Inc Inc Inc

SG 2 Inc Full Inc Inc Inc Inc Inc

SG 3 Inc Inc Full Inc Inc Inc Inc

SG 4 Inc Inc Inc Full Inc Inc Inc

SG 5 Inc Inc Inc Inc Full Inc Inc

SG 6 Inc Inc Inc Inc Inc Full Inc

SG 7 Inc Inc Inc Inc Inc Inc Full

The ResultsThe Results

● Lower storage costsLower storage costs● Eliminate tape backupsEliminate tape backups● Increased employee productivityIncreased employee productivity● Compliance with legal and regulatory Compliance with legal and regulatory

requirements requirements ● Improved messaging protection Improved messaging protection ● Establishment of foundation for new SLAs Establishment of foundation for new SLAs ● Simplified operational tasks Simplified operational tasks ● Demonstrated confidence in the product  Demonstrated confidence in the product 

BenefitsBenefits

For More InformationFor More Information

● Additional content on Microsoft IT Additional content on Microsoft IT deployments and best practices can be deployments and best practices can be found on found on http://www.microsoft.comhttp://www.microsoft.com● Microsoft IT Showcase on TechNetMicrosoft IT Showcase on TechNet

http://www.microsoft.com/technet/itshowcasehttp://www.microsoft.com/technet/itshowcase● Microsoft IT Showcase WebcastsMicrosoft IT Showcase Webcasts

http://www.microsoft.com/howmicrosoftdoesitwhttp://www.microsoft.com/howmicrosoftdoesitwebcastsebcasts

This document is provided for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS DOCUMENT.

© 2006 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY. Microsoft, Active Directory, ActiveSync, Internet Explorer, Outlook, Windows Mobile, and Windows Server are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. The names of actual companies and products mentioned herein may be the trademarks of their respective owners.