building the perfect sharepoint 2010 farm - sps sacramento

Welcome to the First Annual SharePoint

Saturday Sacramento!!

Building the ‘Perfect’ FarmBest Practices from the Field

Michael NoelConvergent ComputingTwitter: @MichaelTNoel

Michael Noel Author of SAMS Publishing titles “SharePoint 2010 Unleashed,” “SharePoint

2007 Unleashed,” “SharePoint 2003 Unleashed”, “Teach Yourself SharePoint 2003 in 10 Minutes,” “Windows Server 2008 R2 Unleashed,” “Exchange Server 2010 Unleashed”, “ISA Server 2006 Unleashed”, and many other titles .

Partner at Convergent Computing (www.cco.com / +1(510)444-5700) – San Francisco Bay Area based Infrastructure/Security specialists for SharePoint, AD, Exchange, Security

What we will cover

Examine various SharePoint 2010 farm architecture best practices that have developed over the past year

Examine SharePoint Best Practice Farm Architecture

Understand SharePoint Virtualization Options Explore SharePoint DR and HA strategies using

Database Mirroring Explore other common best practices (RBS, SSL,

NLB) Examine best practice security for SharePoint A large amount of best practices covered (i.e.

Drinking through a fire hose,) goal is for you to be able to take away at least 2-3 useful pieces of information that can be used in your environment

Architecting the Farm

Web

Service Apps

Data

Architecting the FarmUnderstanding the Three Tiers of SharePoint Infrastructure

‘All-in-One’ (Avoid)

DB and SP Roles Separate

Architecting the FarmSmall Farm Examples

2 SharePoint Servers running Web and Service Apps

2 Database Servers (Clustered or Mirrored)

1 or 2 Index Partitions with equivalent query components

Smallest farm size that is fully highly available

Architecting the FarmSmallest Highly Available Farm

2 Dedicated Web Servers (NLB)

2 Service Application Servers

2 Database Servers (Clustered or Mirrored)

1 or 2 Index Partitions with equivalent query components

Architecting the FarmBest Practice ‘Six Server Farm’

Multiple Dedicated Web Servers

Multiple Dedicated Service App Servers

Multiple Dedicated Query Servers

Multiple Dedicated Crawl Servers, with multiple Crawl DBs to increase parallelization of the crawl process

Multiple distributed Index partitions (max of 10 million items per index partition)

Two query components for each Index partition, spread among servers

Architecting the FarmScaling to Large Farms

Previously a third party product ($$$$)

More reasonable pricing now Highly tuned and specialised search

engine for SharePoint and also as an enterprise search platform

Replaces SharePoint 2010 Native Search if used

‘Net new’ features built-in.

Architecting the FarmFAST Search

Feature 

SharePoint Foundation

2010

Search Server 2010

Express

Search Server 2010

SharePoint Server 2010

FAST Search Server

2010 for SharePoint

Basic search X X X X XBest Bets   X X X XSearch Scopes   X X X XCrawled and Managed Properties

  X X X X

Query Federation   X X X XQuery Suggestions   X X X XRelevancy Tuning by Document or Site Promotions

  X X X X

Shallow Results Refinement   X X X XWindows 7 Federation   X X X X

Architecting the FarmFAST Search – Comparison Matrix – Slide 1 of 2

Feature 

SharePoint

Foundation 2010

Search Server 2010

Express

Search

Server 2010

SharePoint Server

2010

FAST Search Server

2010 for SharePoint

People Search       X XSocial Search       X XTaxonomy Integration       X XMulti-Tenant Hosting       X XVisual Best Bets         XSimilar Results         XDuplicate Results         XSearch Enhancement based on user context

        X

Sort Results on Managed Properties or Rank Profiles

        X

Deep Results Refinement         XDocument Preview         XRich Web Indexing Support         X

Architecting the FarmFAST Search – Comparison Matrix – Slide 2 of 2

Virtualization of SharePoint Servers

• Dedicated hosts for SharePoint Virtual Guests

• No Software on Host OS! (Except A/V or Backup)

• Don’t overallocate memory (ballooning) or Processor (2:1 ratio max)

Virtual Hosts

• Ensure proper amount of IO (0.75 IOPs / GB)

• Allocate Passthrough/RDM disk for best perf

• If using virtual disks, use fixed-sized, not dynamically expanding

Disk

• Aggregate multiple NICs on host for the guest networks

• Allocate Passthrough/RDM NICs for best perf

Network• Web Role is best candidate, but be

cautious if using multiple app pools (800MB/pool)

• Service App systems generally good candidates

• Use caution with the database role!

Virtual Guests

Virtualization of SharePoint ServersCaveats – Be Sure to Understand Virtualization Concepts

  vCPU RAM (Bare Minimum)

RAM (Recommend)

RAM (Ideal)

Web Only* 2 6GB 8GB 12GB

Service Application Roles Only

2 6GB 8GB 12GB

Dedicated Search Service App

2 8GB 10GB 16GB

Combined Web/Search/Service Apps

4 10GB 12GB 18GB

Database* 4 10GB 16GB 24GB

Virtualization of SharePoint ServersVirtual Guest Processor and Memory Guidelines

Allows organizations that wouldn’t normally be able to have a test environment to run one

Allows for separation of the database role onto a dedicated server Can be more easily scaled out in the future

Virtualization of SharePoint ServersSample 1: Small Single Server Environment / No HA

High-Availability across Hosts

All components Virtualized

Uses only two Windows Ent Edition Licenses

Virtualization of SharePoint ServersSample 2: Two Server Highly Available Farm

Highest transaction servers are physical

Multiple farm support, with DBs for all farms on the SQL cluster

Virtualization of SharePoint ServersSample 3: Mix of Physical and Virtual Servers – Best Perf

Virtualization of SharePoint ServersSample 4: Scaling to Large Virtual Environments

Processor (Host Only) <60% Utilization = Good 60%-90% = Caution >90% = Trouble

Available Memory 50% and above = Good 10%-50% = OK <10% = Trouble

Disk – Avg. Disk sec/Read or Avg. Disk sec/Write Up to 15ms = fine 15ms-25ms = Caution >25ms = Trouble

• Network Bandwidth – Bytes Total/sec– <40% Utilization =

Good– 41%-64% = Caution– >65% = Trouble

• Network Latency - Output Queue Length– 0 = Good– 1-2= OK– >2 = Trouble

Virtualization of SharePoint ServersVirtualization Performance Monitoring

1. Create new Virtual Guest (Windows Server 2008 R2)

2. Install SP2010 Binaries. Stop before running Config Wizard

3. Turn Virtual Guest into Template, modify template to allow it to be added into domain

4. Add PowerShell script to run on first login, allowing SP to be added into farm or to create new farm

End Result - 15 minute entire farm provisioning…quickly add servers into existing farms or create new farms (Test, Dev, Prod) on

demand

Virtualization of SharePoint ServersQuick Farm Provisioning using VMM/Virtual Center

Quick Farm Provisioning with VMM 2008 R2 - Demo

Data Management

Start with a distributed architecture of content databases from the beginning, within reason (more than 50 per SQL instance is not recommended)

Distribute content across Site Collections from the beginning as well, it is very difficult to extract content after the face

Allow your environment to scale and your users to ‘grow into’ their SharePoint site collections

Data ManagementDistribute Data Across Content DBs and Site Collections

BLOBs are unstructured content stored in SQL Includes all documents, pictures, and files

stored in SharePoint Excludes Metadata and Context, information

about the document, version #, etc. Until recently, could not be removed from

SharePoint Content Databases Classic problem of structured vs. unstructured

data – unstructured data doesn’t really belong in a SQL Server environment

Data ManagementBinary Large OBject (BLOB) Storage

Can reduce dramatically the size of Content DBs, as upwards of 80%-90% of space in content DBs is composed of BLOBs

Can move BLOB storage to more efficient/cheaper storage

Improve performance and scalability of your SharePoint deployment – But highly recommended to use third party

Data ManagementGetting your BLOBs out of the Content DBs

SQL Database Optimization

SQL Database OptimizationContent Databases Distributed Between Multiple Volumes

DB-AFile 1

DB-BFile 1

Volume #1

DB-AFile 2

DB-BFile 2

Volume #2

DB-AFile 3

DB-BFile 3

Volume #3

DB-AFile 4

DB-BFile 4

Volume #4

Tempdb File 1 Tempdb File 2 Tempdb File 3 Tempdb File 4

SQL Database OptimizationContent Databases Distributed Between Multiple Volumes

• Break Content Databases and TempDB into multiple files (MDF, NDF), total should equal number of physical processors (not cores) on SQL server.

• Pre-size Content DBs and TempDB to avoid fragmentation

• Separate files onto different drive spindles for best IO perf.

• Example: 100GB total Content DB on Four-way SQL Server would have four database files distributed across four sets of drive spindles = 25GB pre-sized for each file.

• TempDB is critical for performance• Pre-size to 20% of the size of the largest

content database.• Break into multiple files across spindles as

noted• Note there is a separate TempDB for each

physical instance• Note that if using SQL Transparent Data

Encryption (TDE) for any databases in an instance, the tempDB is encrypted.

SQL Database OptimizationTempDB Best practices

High Availability and Disaster Recovery

Clustering is Shared Storage, can’t survive storage failure, makes Mirroring more attractive

Clustering fails over more quickly Mirroring is not supported for all

databases, but Clustering is Both Clustering and Mirroring can be

used at the same time (Instance to Instance)

High Availability and Disaster RecoveryData Tier – Clustering vs. Mirroring

Introduced in SQL 2005 SP1 Greatly improved in SQL 2008 and now SQL 2008 R2 Available in Enterprise and Standard (Synchronous

only) editions Works by keeping a mirror copy of a database or

databases on two servers Can be used locally, or the mirror can be remote Can be set to use a two-phase commit process to

ensure integrity of data across both servers Can be combined with traditional shared storage

clustering to further improve redundancy SharePoint 2010 is now Mirroring aware!

High Availability and Disaster RecoveryData Tier – SQL Database Mirroring

Single Site Synchronous

Replication Uses a SQL Witness

Server to Failover Automatically

Mirror all SharePoint DBs in the Farm

Use a SQL Alias to switch to Mirror Instance

High Availability and Disaster RecoveryData Tier – Database Mirroring Model #1 – Single Site

Two Sites 1-10 ms

Latency max

1Gb Bandwidth minimum

Farm Servers in each location

Auto Failover

High Availability and Disaster RecoveryData Tier – Database Mirroring Model #2 – Cross-Site with HA

Two Sites Two Farms Mirror only

Content DBs

Failover is Manual

Read-only Mode possible

Must Re-Attach and Re-Index

High Availability and Disaster RecoveryData Tier – Database Mirroring Model #2 – Remote Farm

Synchronous Mirror Support

Asynchronous Mirror Support

Configuration XCentral Administration content XContent Databases X XUsage and Health Data Collection

Business Data Connectivity XApplication Registry service * (BDC Upgrade)

Subscription Settings service * (PowerShell Enabled)

X

Search – Search Administration XSearch - Crawl XSearch - Property X

High Availability and Disaster RecoveryData Tier – Database Support for Mirroring – Slide 1 of 2

Synchronous Mirror Support

Asynchronous Mirror Support

User Profile - Profile XUser Profile - Synchronization

User Profile – Social Tagging

Web Analytics - Staging

Web Analytics - Reporting XSecure Store X XStage XManaged Metadata XWord Automation Services XPerformancePoint X

High Availability and Disaster RecoveryData Tier – Database Support for Mirroring – Slide 2 of 2

High Availability and Disaster RecoveryTwo Node/Two Instance Cluster – Take Advantage of both servers

High Availability and Disaster RecoveryNetwork Load Balancing

Hardware Based Load Balancing (F5, Cisco, Citrix NetScaler – Best performance and scalability

Software Windows Network Load Balancing fully supported by MS, but requires Layer 2 VLAN (all packets must reach all hosts.) Layer 3 Switches must be configured to allow Layer 2 to the specific VLAN.

If using Unicast, use two NICs on the server, one for communications between nodes.

If using Multicast, be sure to configure routers appropriately

Set Affinity to Single (Sticky Sessions) If using VMware, note fix to NLB

RARP issue (http://tinyurl.com/vmwarenlbfix)

Best Practice – Create Multiple Web Apps with Load-balanced VIPs (Sample below) Web Role Servers

▪ sp1.companyabc.com (10.0.0.101) – Web Role Server #1▪ sp2.companyabc.com (10.0.0.102) – Web Role Server #2

Clustered VIPs shared between SP1 and SP2 (Create A records in DNS)▪ spnlb.companyabc.com (10.0.0.103) - Cluster▪ spca.companyabc.com (10.0.0.104) – SP Central Admin▪ spsmtp.companyabc.com (10.0.0.105) – Inbound Email VIP▪ home.companyabc.com (10.0.0.106) – Main SP Web App

(can be multiple)▪ mysite.companyabc.com (10.0.0.107) – Main MySites Web

App

High Availability and Disaster RecoveryWindows Software Network Load Balancing Recommendations

SharePoint Installation

Good to understand how to install SharePoint from the command-line, especially if setting up multiple servers.

Allows for options not available in the GUI, such as the option to rename databases to something easier to understand.

Use PowerShell with SharePoint 2010 Sample scripts available for download…

SharePoint InstallationScripted Installations

Function Configure-SPSearch {PARAM($AppPool, $FarmName, $SearchServiceAccount)

$searchServiceInstance = Get-SPEnterpriseSearchServiceInstance -localStart-SPEnterpriseSearchServiceInstance -Identity $searchServiceInstance

$dbName = $FarmName + "_SearchServiceApplication"

$searchApplication = New-SPEnterpriseSearchServiceApplication -Name "$FarmName Search Service Application" -ApplicationPool $AppPool -DatabaseName $dbName$searchApplicationProxy = New-SPEnterpriseSearchServiceApplicationProxy -name "$FarmName Search Service Application Proxy" -SearchApplication $searchApplication

Set-SPEnterpriseSearchAdministrationComponent -SearchApplication $searchApplication -SearchServiceInstance $searchServiceInstance

$crawlTopology = New-SPEnterpriseSearchCrawlTopology -SearchApplication $searchApplication$crawlDatabase = Get-SPEnterpriseSearchCrawlDatabase -SearchApplication $searchApplication

New-SPEnterpriseSearchCrawlComponent -CrawlTopology $crawlTopology -CrawlDatabase $crawlDatabase -SearchServiceInstance $searchServiceInstance

while($crawlTopology.State -ne "Active"){$crawlTopology | Set-SPEnterpriseSearchCrawlTopology -Active -ErrorAction SilentlyContinueif ($crawlTopology.State -ne "Active"){Start-Sleep -Seconds 10}}

$queryTopology = New-SPenterpriseSEarchQueryTopology -SearchApplication $searchApplication -partitions 1$searchIndexPartition = Get-SPEnterpriseSearchIndexPartition -QueryTopology $queryTopologyNew-SPEnterpriseSearchQueryComponent -indexpartition $searchIndexPartition -QueryTopology $queryTopology -SearchServiceInstance $searchServiceInstance

$propertyDB = Get-SPEnterpriseSearchPropertyDatabase -SearchApplication $searchApplication

Set-SPEnterpriseSearchIndexPartition $searchIndexPartition -PropertyDatabase $propertyDB

while ($queryTopology.State -ne "Active"){$queryTopology | Set-SPEnterpriseSearchQueryTopology -Active -ErrorAction SilentlyContinue

if ($queryTopology.State -ne "Active"){Start-Sleep -Seconds 10}}}

SharePoint InstallationSamples Scripts – http://tinyurl.com/SPFarm-Config

SharePoint InstallationSome Manual Service Apps Still Required

Due to complexity and/or bugs, certain Service Apps will need to be manually configured in most cases.

This includes the following: PerformancePoint Service Application User Profile Service Application Web Analytics Service Application

Security

SharePoint SecurityLayers of Security in a SharePoint Environment

Infrastructure Security and Best practices Physical Security Best Practice Service Account Setup Kerberos Authentication

Data Security Role Based Access Control (RBAC) Transparent Data Encryption (TDE) of SQL Databases Antivirus

Transport Security Secure Sockets Layer (SSL) from Server to Client IPSec from Server to Server

Edge Security Inbound Internet Security (Forefront UAG/TMG)

Rights Management

Service Account Name

Role of Service Account Special Permissions

COMPANYABC\SRV-SP-Setup

SharePoint Installation Account Local Admin on all SharePoint servers (for install of SP).

COMPANYABC\SRV-SP-SQL SQL Service Account(s) – Should be separate admin accounts from SP accounts.

Local Admin on Database Server(s) (Generally, some exceptions apply)

COMPANYABC\SRV-SP-Farm

SharePoint Farm Account(s) – Can also be standard admin accounts. RBAC principles apply ideally.

N/A

COMPANYABC\SRV-SP-Search

Search Account N/A

COMPANYABC\SRV-SP-Content

Default Content Access Account Read rights to any external data sources to be crawled

COMPANYABC\SRV-SP-Prof Default Profiles Access Account Member of Domain Users (to be able to read attributes from users in domain) and ‘Replicate Directory Changes’ rights in AD.

COMPANYABC\SRV-SP-AP-SPCA Application Pool Identity account for SharePoint Central Admin.

DBCreator and Security Admin on SQL. Create and Modify contacts rights in AD OU used for email.

COMPANYABC\SRV-SP-AP-Data

Application Pool Identity account for the Content related App Pool (Portal, MySites, etc.) Additional as needed for security.

N/A

SharePoint SecurityInfrastructure – Sample List of Service Accounts

When creating any Web Applications in Classic-mode, USE KERBEROS. It is much more secure and also faster with heavy loads as the SP server doesn’t have to keep asking for auth requests from AD.

Kerberos auth does require extra steps, which makes people shy away from it, but once configured, it improves security considerably and can improve performance on high-load sites.

Should also be configured on SPCA Site! (Best Practice = Configure SPCA for NLB, SSL, and Kerberos (i.e. https://spca.companyabc.com)

SharePoint SecurityInfrastructure – Enable Kerberos when using Classic-Auth

Role Groups defined within Active Directory (Universal Groups) – i.e. ‘Marketing,’ ‘Sales,’ ‘IT,’ etc.

Role Groups added directly into SharePoint ‘Access Groups’ such as ‘Contributors,’ ‘Authors,’ etc.

Simply by adding a user account into the associated Role Group, they gain access to whatever rights their role requires.

User1

User2

Role Grou

p

SharePoint Group

SharePoint SecurityData – Role Based Access Control (RBAC)

• New in SQL Server 2008

• Only Available with the Enterprise Edition

• Seamless Encryption of Individual Databases

• Transparent to Applications, including SharePoint

SharePoint SecurityData - Transparent Data Encryption (TDE)

SharePoint SecurityData - Use SharePoint-Aware Antivirus (3rd Party or FPS)

External or Internal Certs highly recommended

Protects Transport of content 20% overhead on Web Servers Can be offloaded via SSL

offloaders if needed Don’t forget for SPCA as well!

SharePoint SecurityTransport - Secure Sockets Layer (SSL) Encryption

By default, traffic between SharePoint Servers (i.e. Web and SQL) is unencrypted

IPSec encrypts all packets sent between servers in a farm

For very high security scenarios when all possible data breaches must be addressed

SharePoint SecurityTransport – IPSec from Server to Server

SharePoint SecurityEdge – Forefront Unified Access Gateway

AD RMS is a form of Digital Rights Management (DRM) technology, used in various forms to protect content

Used to restrict activities on files AFTER they have been accessed: Cut/Paste Print Save As…

Directly integrates with SharePoint DocLibs

SharePoint SecurityRights Management - Active Directory Rights Management Services

For More Information

SharePoint 2010 Unleashed from SAMS Publishing (http://www.samspublishing.com)

Windows Server 2008 R2 Unleashed and/or Hyper-V Unleashed (http://www.samspublishing.com)

Microsoft ‘Virtualizing SharePoint Infrastructure’ Whitepaper (http://tinyurl.com/virtualsp)

Microsoft SQL Mirroring Case Study (http://tinyurl.com/mirrorsp )

Failover Mirror PowerShell Script (http://tinyurl.com/failovermirrorsp )

SharePoint Kerberos Guidance (http://tinyurl.com/kerbsp)

SharePoint Installation Scripts (http://tinyurl.com/SPFarm-Config)

Contact us at CCO.com

Brisbane 2011

Don’t ForgetSharePint!!

Join us at 6:00 pm at BJs Brewhouse for a free drink,

kindly provided by Rackspace! A great opportunity to network with

other SharePoint Professionals!

Don’t forget to fill out your bingo cards and surveys to be eligible for GREAT end of day raffles!

Sponsored By…..

Brisbane 2011

Thanks to Our Sponsors

Thanks for attending!Questions?

Michael NoelTwitter: @MichaelTNoel

www.cco.comSlides: slideshare.net/michaeltnoel