1 © copyright 2012 emc corporation. all rights reserved. building the right infrastructure for...

1© Copyright 2012 EMC Corporation. All rights reserved.

Building the Right Infrastructure for Microsoft

SharePoint


EMC and Microsoft Alliance• EMC and Microsoft have been engaged as

strategic partners for over 16 years

• Shared view of importance of information

• Joint sharing of intellectual property

• EMC participates in Microsoft Technology Adopter Programs (TAPs) and Rapid Deployment Programs (RDP)

• CEO-sponsored worldwide go-to-market alliance

• Over 300 published technical solutions

• Over 100 data sheets on Microsoft technologies

• EMC has Hundreds of Microsoft Certified Professionals (MCPs)

• Microsoft is one of EMC’s Top 5 customers

16+ Microsoft Gold/Silver Competencies

21x Time Partner of Year

2,700+ consultants worldwide

Cooperative Support Agreement

Equipment in Redmond labs

Leading Microsoft consultancies


Affordable. Simple. Efficient. Powerful.

VNXe3100 VNX7500VNX5700VNXe3300 VNX5100 VNX5500VNX5300


Optimized for MicrosoftComplete Integration Across the Microsoft Stack

BusinessProductivity

ApplicationPlatform

CoreInfrastructure

On-Premise / Cloud

• ESI/SCOM/SCVMM Packs• SMB3.0 Support • Thin Provisioning • ODX Copy Offload

• FAST Cache for 9x increase in TPS

• VFCache for 87% reduction in RT

• Combined for 11x increase in TPS

• Manage Storage from SharePoint UI

• Shrink SQL ContentDB by 90-95% via RBS

• 30-40% reduced search RT using FAST Cache


Challenge: Relentless Growth of Unstructured Data

People / Business Processes:

1. Disconnected information silos

2. Information not available to those who need it in real-time

3. Governance

IT Related:

4. TCO / Asset Utilization

5. Backup Windows / Recovery Processes

6. Version Control, user mapping, permissionsSource : IDC, The Digital Universe

Decade


Why SharePoint? Why Now?

Why SharePoint?– Fastest growing platform in MSFT’s

history– $2B in 2011, 20,000 seats/DAY!– Accessible, extensible, unified

content platform

Why now?– 67% of customers report rolling out

SharePoint to the entire organization

– New release (Wave 15) coming in November

Gartner Magic Quadrant for Enterprise Content Management


Infrastructure Challenges with Current Sharepoint Deployments1. Unstructured Data Takeover

– Primary document types stored in SharePoint are PDF, Word, PowerPoint, and large Excel spreadsheets

– Documents typically well over 1MB– Files take over valuable server disk space and resources

2. Avalanche of large media– SharePoint often stores large files like media images and videos– SQL not optimized for file level access in general and streaming in

specific – Applications will not return control until file is fully loaded– File size and concurrency will increase the chance of browser timeouts


Infrastructure Challenges with Current Sharepoint Deployments3. Old and unused files hogging valuable SQL storage / CPU

– No means to intelligently tier off / archive older stale content– 60-80% of SQL BLOB corpus rarely used or not used at all– Equates to over 50% of the SQL Content DB Corpus if not externalized!

4. Lack of scalability – Proliferation of server resources to address performance bottlenecks– Poor economic scaling of SQL server licenses– Native backup / recovery tools too slow for sizable documents

▪ Only 600GB backed up in six hours using native tools▪ Only several TB’s backed up in six hours using MSFT DPM

– Lack of consistency in recovery of farm, ContentDB, and BLOB Store– Lack of tools to load-balance server resources


Building a SharePoint Architecture for Scale

Legacy ECM

File Shares

Future content growth

Today Centralize & Take Control Consolidate & Optimize

5%

95%

BLOBs

Metadata


Metalogix StoragePoint Librarian

ContosoFinance

AP

ARHR

Employees

Training Materials

Handbooks and Policies

Invoices

Purchase Orders

File Share LibrarianContent stored on file

system.

ContosoFinance

AP

ARHR

Employees

Training Materials

Handbooks and Policies

Invoices

Purchase Orders

SQL Content DB

List Items and

MetadataContent stays on file share

StoragePoint CatalogsFile share

StoragePoint Creates

SharePoint Structure and Items


The Challenge with SharePoint BLOBs

10 x 100GB SharePoint Content Databases

1TB of SQL Content Database Storage

PPT

PDF

XLS

DOC TIFF

90-95% unstructured, rest metadata

“Typically, as much as 80 percent of data for an enterprise-scale deployment of SharePoint Foundation consists of file-based data streams that are stored as BLOB data. These BLOB objects comprise data associated with SharePoint files. However, maintaining large quantities of BLOB data in a SQL Server database is a suboptimal use of SQL Server resources. You can achieve equal benefit at lower cost with equivalent efficiency by using an external data store to contain BLOB data.”

Source: http://msdn.microsoft.com/en-us/library/bb802976.aspx

• Performance for large files, especially write intensive • Large file upload time can be improved by as much

as 50% with RBS

• Can’t easily & economically scale

• Asset Utilization SQL Servers

• BLOBs don’t benefit from SQL query engine. Why are they there?

• Long backups / long restores

• Lack of tiered storage = Poor TCO

http://msdn.microsoft.com/en-us/library/bb802976.aspx

http://msdn.microsoft.com/en-us/library/bb802976.aspx


Metalogix StoragePoint RBS ArchitectureStoragePointWindows

SharePoint Services

.NET

SOAP

WebDAV

REST

SPRequest

SharePoint Object Model

Content Database

Metadata

Shared Services

Profile Management

Compression

Encryption

Logging

Notifications

Endpoint

Manager

Adapter

Adapter

Adapter

Filters (Size and Type)

Monitoring

Migration

Caching

Adapter Framework

EBS

RBSAtmos

VNX

Isilon


Metalogix StoragePoint Data Flow

SQL Server

RBSProvider

SP Web Application

Service

Document

BLOB write handed off to RBS Provider

Invoice #

Date

Amount

Vendor

Metadata

Document uploaded

Metadata uploaded

Meta

data

saved

to

SQ

L S

erv

er

Is there a storage profile?

BLOB

writ

ten

to

SQL

Serv

er

Determine Endpoint

Synchronous Asynchronous

BLOB Migrator Agent

YES

NO

Share Object Store / Cloud


StoragePoint – Using Aging Rules

VNX/NL-SAS VNX/NL-SAS Compressed Isilon

Active Inactive Archive

Invo

ice

Date

> 1 y

ear

Invo

ice

Date

> 3 y

ears

Older Content to Lower $/GB Tiers

Met

adat

a

BLOB

• On-Premise• Private Cloud• Public Cloud


StoragePoint - Advanced Filters

Storage Profile

FileSystem (Sync)

Filter: File Type = PDF, TIF, TIFF

Share1

FileSystem (Sync)Filter: List = HR DOCS

Share2

FileSystem (Sync)Filter: Site (Web) = Finance

BLOB Store 3


EMC / Metalogix Reference Architecture

Metadata5-10%

BLOBs90-95%

FC Network

CIFS Network

Share NL SAS

Share SAS

Share Deep Compressed

SQL Search DB

StoragePoint Rule moves BLOB from Tier 1 to Tier 2 At Invoice Date + 6 months

StoragePoint Rule moves BLOB from Tier 2 to Tier 3 At Invoice Date + 1 year

VNX FAST Cache enables faster indexing/crawl/search

Reference Architectures: http://www.emc.com/collateral/hardware/technical-documentation/h8185-sharepoint-vnx-metalogix-psg.pdf http://powerlink.emc.com/km/live1/en_US/Offering_Technical/Presentation/remote-blob-storage-sharepoint-tp.pptx

• Flash 1st 75% Cost Reduction via VNX FAST and smart use of SSDs and NL-SAS

• FAST Cache Up to 30-40% reduction in response times for Search

• FAST VP 80% less performance tuning for SQL

• Lower SQL Licensing Costs due to server virtualization and RBS

tempDBSearch Query

VNX FAST VP to reduce performance tuning

http://www.emc.com/collateral/hardware/technical-documentation/h8185-sharepoint-vnx-metalogix-psg.pdf




http://powerlink.emc.com/km/live1/en_US/Offering_Technical/Presentation/remote-blob-storage-sharepoint-tp.pptx



Applying StoragePoint Advanced Filters

Metadata FC Network

CIFS Network

Share FLR

Share NL SAS

LUNFAST Cache

Use Advanced Filters to move HR data to where retention /

compliance is ensured

Encrypt sensitive data before it is moved to storage endpoint

(up to 256-bit)

LUNFAST VP

• Set retention periods at file system creation • Default/minimum/maximum

• Automatic File Lock and Delete

VNX File Level Retention:

Archive / Big Data

Active Data


Business Continuity

Virtualizing SharePoint is Essential

Consolidation

Performance

Availability

Maintenance

Load Balancing

Achieve 2-10x consolidation ratio, especially for larger deploymentsLower SQL Licensing by consolidating vCPU onto physical CPU

Improved front end performance with more, smaller WFEs rather than few large WFEs

VM based protection for SharePoint provides homogeneous high availability (WSFC, VMware HA)

Simplified DR management (Geo-Clustering, vCenter Site Recovery Manager)

Live migration of virtual machines (Hyper-V Live Migration, VMware vMotion)

Maximized overall performance with balanced cross farm HW utilization (SCVMM PRO , VMware DRS)


Three Sources of Savings

1. Better Asset Utilization– File Share typically not well utilized – VNX Utilization ~70% on SAS / NL-SAS and up to 90% on SSD

2. More Efficiency via Flash 1st – Results in fewer drives, lower energy bills, and lower $/GB– Also lower $/IOPS via FAST Suite

3. Lower SQL licensing via RBS and virtualization– Reduce database size and increase VM density per physical server


FLASH 1st Data StrategyHot data on FAST Flash SSDs—cold data on dense disks

“Hot”high activity As data ages, activity falls,

triggering automatic movement to high-capacity disk drives for lowest cost

Highly active data is stored on Flash SSDs

for fastest response time

Movement Trigger

FlashSSD

“Cold”Low Activity

Data

A

ctiv

ity

High- capacity

HDD

Data Age


VNX5500

210 15K DRIVES

32 TB Usable. 50% Y/Y Growth. 60 Days Data Shelf-life.

FLASH 1st Example


VNX5300

20200GB SSDs

16600GB 15K

SAS

243TB

NL-SASVNX5500

210 15K DRIVES


FLASH 1st Example

60 DRIVES

FAST Cache & FAST VP



FLASH 1st Example

Footprint Power IOPS

33K

3,938W45RUVNX5300

20200GB SSDs

16600GB 15K

SAS

243TB

NL-SAS

60 DRIVES

FAST Cache & FAST VP18RU

990W

74K

5%FLASH


VNX FAST SuiteA FLASH 1st strategy enabler

FAST Cache continuously ensures that the hottest data is served from high-performance Flash SSDs

FAST VP* optimizes storage pools automatically, ensuring that active data is being served from SSDs, while cold data is moved to lower-cost disk tiers

Together they deliver a fully automated FLASH 1st storage strategy for optimal performance at the lowest cost attainable

Monitor and tune the whole system with the complementary Unisphere Quality of Service Manager and Unisphere Analyzer

* Not available for VNX5100

Real-time caching with FAST Cache

Scheduled optimizationwith FAST VP

FlashFlashSSD

15KHDD

High-

capacity HDD


VNX FAST Cache boosts SQL2012

Data Disk RAID FC

OLTP DB Files

600K SAS 5 ON

TempDBs 600K SAS 1/0 OFF

DB Logs 600K SAS 1/0 OFF

FAST Cache: 14 x 100GB SSDsWorking Set < FAST Cache

Results:• 5x Improvement of IOPS and TPS• 30 Minute Cache Warming

IOP

STP

S


Lower $/GB and $ / IOPS with RBS

Average

$/GB SAS–15K$2/GB

LOWEST

$/IOSSD

$.99/IOMetadata

5-10%

BLOBs90-95%M

eta

log

ixS

tora

geP

oin

t

Example:• Avg. Capacity VNX5300 = 50TB• Without RBS on SAS-15K: $100K• With RBS and 5% SSD and 95% NL-SAS: $40-50K

LOWEST

$/GB NL-SAS$.43/GB

No RBS With RBS


Lower SQL Licensing Costs

SQL Server is rarely CPU constrained– For SharePoint the performance

bottleneck tends to be the WFE– Idle CPU = Wasted licensing $$

SQL2012 Server Enterprise Licensing:

– Physical: per physical core – Virtual: per vCPU (all you can eat

per physical core)– No cost difference between a

physical core and vCPU

Virtualization = greater VM density and lower SQL licensing costs

– Savings determined by vCPU : Core Ratio (e.g. 4:1 ratio =75% savings)

RBS = greater VM density and lower SQL licensing costs

– Lower CPU load on SQL servers

Have to weigh against performance / CPU utilization

– SQL ContentDB recommended CPU limit 50%

– Perfmon helps determine the status quo


RBS + VNX = More efficient SQL IO

Overall more efficient handling of IO– Reduction in overall IO by 20-30%– IO coalescing on VNX

Most IO handled by VNX CIFS share– 70-80% of backend IO handled by

CIFS share– RBS reduces SQL CPU cycles by only

storing the varbinary file attribute in SQL

– Actual BLOB is stored in more efficient file system

Non-Externalized Externalixed0.00

100.00

200.00

300.00

400.00

500.00

600.00

BLOB StoreContentDB

Source: http://powerlink.emc.com/km/live1/en_US/Offering_Technical/Presentation/remote-blob-storage-sharepoint-tp.pptx





Results• Average page load time improved by 4x after RBS • Backup times from SQL and SharePoint reduced by a

factor of 13x• >90% moved from expensive SQL drives to NL-SAS

Challenge• Internally facing communications site, which included

large amounts of rapidly changing large objects• Page load times and backup / recovery times

Large Government Contractor EMC VNX and Metalogix StoragePoint Improves Operational Agility and Performance

“The dashboard reports showed that I was now saving >90% of my storage on the expensive SQL drives.”

Derek MartinCloud Solutions Architect


SharePoint Performance Considerations

Performance for SharePoint highly use case dependent– Focus on Enterprise Content Management*

SharePoint workload for content management– Can be characterized by 60/20/10/10 for Browse/Modify/Search/Upload– File size, distribution, and concurrent access impact performance

Symptoms vs. Remedy – Symptoms often occur during individual tasks (e.g. file upload by user)– Sizing however needs to factor in system level workloads for the entire farm– In addition need to size system for blended workload (e.g. Exchange, SQL,

SharePoint)


SharePoint Performance Considerations

The storage subsystem is rarely the bottleneck– Generally web front end (WFE) CPU represents the bottleneck – VNX FAST Suite enables very efficient configurations (majority NL-SAS, 2-5% SSDs)– Search functions are the workloads that happen to benefit the most

▪ Search critical as SharePoint environments and document repositories grow ▪ Microsoft Test Report for Very Large-Scale Document Repositories validates VNX performance

– VNX typically deployed with mixed workloads (e.g. Exchange, SQL, and SharePoint)

Case for RBS: SQL Database not efficient at handling files IO– Each write represents an update of the table and the log– Non-externalized storage formats use the SQL buffer pool when the data pages are

accessed– Large files and high concurrency can lead to buffer overruns compounding the issue– Other processes such as index rebuilds can also compete for the same server

resources

http://www.microsoft.com/en-us/download/details.aspx?id=27573


Best Use of VNX FAST Suite for SharePoint

SharePoint Component

FAST VP FAST Cache Workload

Search Index - ++• R:W Ratio of 90:10; 32K block read, 64K write; Small working set.• Search Index re-uses the same blocks on disk as working space to process

list items during indexing. Storage IO quiet between crawls.• Highly changing, throw-away data.

Search Query + +• R:W Ratio of 70:30; 32K read, 64K write; Small working set.• Query improves due to random large block reads/writes being serviced by

FAST Cache, but Query storage is large and costly in FAST Cache. • Highly-read data with small burst write changes.

tempdb + +• R:W Ratio of 50:50; 8K read, 32K write.• TempDB (database page reuse) is ideally suited to FAST Cache algorithms. • Same blocks are re-used on disk and performance of TEMPDB directly

affects SharePoint performance request.

Content DB + +• R:W Ratio of 95:5; 16K (read and write); Working set reduced via RBS.• Low IOPs requirements does not require FAST Cache.• DB Index operations see an improvement.

BLOB Store - - • Low IOPs requirements, large-block I/O.


Recovery Options / Tradeoffs

What are the recovery objectives?– Recovery Level: Database, Database + BLOBs, Database + BLOBs + Farm– Recovery Time: how long does it take to recover – Recovery Point: How much can I afford to lose vs. how much am I willing to spend– Distance: Local vs. Remote– Granularity: Item level vs. entire corpus – Management Overhead: how much time do I need to spend on backup / recovery

Challenges with native tools:– Limited in scale, scope, and performance (e.g. STSADM, Central Administration, Recycle Bin)– Configuration data not backed up– Low backup / recovery rate: rated at 600GB in 6 hours (TB’s with DPM)– Manual re-creation of configuration – STSADM / Central Administration not recommended above 85GB for site collections or Content DB

larger than 100GB


Availability & Protection Methods / Tradeoffs

Requirements (cumulative)

Backup / Recovery Replication High Availability

Database (incl. BLOBS) SQL native backupSTSADM / Central Admin.Avamar

SQL Availability Groups SQL Failover Clustering

DB+ Remote BLOB Store VNX Snaps / Replication Manager

SQL Failover Clustering (DB Only)

Item-Level Recovery Recycle BinAvamar / Ontrack (not RBS aware)

VNX Snaps / Replication Manager / Selective Restore Manager or Ontrack

SQL Failover Clustering (DB Only)

Farm Configuration / VM / OS VNX Snaps / Replication Manager

Hypervisor virtual machine clustering

Site-to-Site RecoverPoint / Cluster Enabler / SRM


Replication Manager / Item-Level Restore

RM supports recovery of Content DB and Remote BLOB Store via script

Order of restore:– Single ContentDB BLOB Store – StoragePoint DB– Singe ContentDB

For item-level restore:– Point Selective Restore Manager or

Ontrack PowerControls to mounted RM database files

– Browse to content and restore the required file(s)

or

Source: http://powerlink.emc.com/km/live1/en_US/Offering_Technical/Presentation/remote-blob-storage-sharepoint-tp.pptx

or





Replication Manager Protection

Data protected Start time End time Duration

SharePoint farm and search 7:45:26 7:49:04 3 minutes 38 seconds

StoragePoint 7:57:08 7:57:09 1 second

BLOB stores 7:57:21 7:58:12 51 seconds

Total time to protect entire farm content: 4 minutes 30 seconds

Data restored Start time End time Duration

BLOB store 8:48:02 8:48:18 16 seconds

StoragePoint 8:52:45 8:54:07 1 minute 22 seconds

Content Database 08:59:47 9:03:22 3 minutes 35 seconds

Total time to restore a Content Database: 5 minutes 13 seconds

Full farm protection

250 GB Single Content Database Restore


SharePoint 2010 Disaster Recovery:Virtualized Farm (Full Replication)

SharePoint Farm ASecondary SitePrimary Site

BLOB Store

VMs RecoverPoint

VMs

VMs

VMs

• Resource/Protection Group level granularity

• Active/Active (Sync distances) or Active/Passive (Async distances)

Failover automation: VMware Site Recovery

Manager (SRM) Protection Groups for all server roles

EMC Cluster Enabler with Hyper-V failover clustering

Databases

BLOB Store

VMs

VMs

VMs

VMs

Databases

WFE

Application Servers

Crawl

SQL Cluster


Automated DRConsistency Group (RP) VNX Volumes Grouping SRM CE

Web Front Ends Boot + Query (optional) Protection Group Cluster Group

Query Servers Boot + Query Component Protection Group Cluster Group

Index Servers Boot + Index Component Protection Group Cluster Group

Application Servers Boot + Application Volumes Protection Group Cluster Group

SQL Server(s)

Boot + Pagefile (optional)

Protection Group Cluster Group

SQL System Databases

Configuration Databases

Search Databases

Content Databases

RBS BLOB Store

Designing DR consistency for SharePoint

Search consistencyExternalized Content consistency


Provisioning SharePoint - ESI Open / browse local or remote

SharePoint farm

Provision SharePoint Web Application:

– Create disks– Create content database– Create Web Application – Attach the content database

Create content database only (no WFE)

– Web Application can be created later using SharePoint admin and the database created in ESI

Users can select any SQL server that is part of the farm and any instance in

that server.

leave unchecked


Health Monitoring in the Private CloudEMC System Center Operations Manager (SCOM) Management Packs:

• Array discovery & health monitoring

• Full hierarchical storage schema and relationship view

• Dashboards incl. physical view (SPs, DAEs, NICs, HBAs, Drives) and logical view (RAID Groups, Volumes/Pools, LUNs, Protocols…)

• Performance Monitoring• End to end application service

view• Map / visualize relationships

between storage and SharePoint farm

EMC System Center Orchestrator (SCO) Management Packs:

• Focal point for automation of heterogeneous datacenter management

• Service Manager for workflows and pre-defined run books• 21 storage activities and

counting• Example Drive failure:

1. Gather SP Collects 2. Post on pre-defined share 3. Send alerts to admin


Leveraging VSPEX

Storage

Network

x86 Server

Hypervisor

Application

*not committed

Proven Infrastructure

Partn

er B

undle

ISV


RBS Options & Feature ComparisonFeature RBS FILESTREAM StoragePoint SourceOne AvePoint

Catalogue Files into native SharePoint ribbon X √ X XBLOB Externalization (RBS) √ √ √* √

Offload BLOBs to remote storage platforms X

√ √ √BLOB store must be local to SQL Server

Outbound BLOB cache for off-premises (i.e. Cloud) operations X √ X X

Define multiple storage endpoints per content database X √ √ √

Filter endpoints by file type or scope (i.e. list or content type) X √ X X

Take endpoints offline (and/or send notifications) if they fill up or enter an error state

X √ √ X

Use SharePoint filename or folder on offloaded BLOBs X √ X X

Promote SharePoint properties to endpoint X √ X XSecurely delete (shred) BLOBs X √ X XReport and Health Monitor Dashboard X √ √ X

Administration User Interface X√

√ √Central Admin

PowerShell Support X √ √

Compression X √ X X

Encryption X√

X XTransmission and "at rest"

Tiered Storage (HSM) support X √ X X

Orphaned BLOB Garbage Collection √ √√ √ √

Basic Policy-based Basic

1 © copyright 2012 emc corporation. all rights reserved. building the right infrastructure for...

Documents

emc corporation

microsoft sharepoint

microsoft alliance emc

microsoft technologies

sharepoint ui

microsoft complete integration

current sharepoint deployments

fast cache slide