ebs and rbs in sharepoint 2010
DESCRIPTION
TRANSCRIPT
Long time IT guyUsing SharePoint since 2001Author of multiple books and articles.Regular Speaker at SharePoint Saturday and conferences
About Me
About BLOBs and storage in SharePointIntroduction to Content Externalization and why do it?EBS RBSPulling it all together
Agenda
Everything goes into the content database (SQL)MetadataFiles
Metadata is vital to SharePoint SuccessGreat for SQL small amounts of structured data
Easy for SQL pages architecture (Good I/O Profile)
Files Stored as a BLOB (Binary Large Object)
Basics of SharePoint Storage
Binary Large ObjectA file in a database
90% of a typical content database is made up of BLOB’s
What is a BLOB?
How does SQL handle a BLOBRows of data cannot exceed 8KPointer in a row to larger data made up of multiple pages.
Its all about the pages8K at a time50MB file = 6000 pagesDon’t forget about database fragmentation
BLOBS are a big cause of this
BLOB’s CAN BE problematicEspecially for files over 1MBAlso does not help in DB fragmentation
So what’s in your database
SharePoint Data Flow
Save Request
Get RequestContentDatabase
SQL Server
ConfigDatabase
SharePoint Web FE
SharePoint Object Model2.
Enforce Biz Logic
Important Note = The Blob Lifecycle = Static
Its Status QuoIts not different.No Change (who moved my cheese)
Transactional consistency1 stop shop
Why not keep things as is? Status Quo Benefits
Why BLOB Remoting? The simple case for Remote BLOB Storage
On average, 90-95% of a SharePoint content database’s storage overhead is comprised of content BLOBs.
BLOBs can be stored on less-expensive storage and SQL is no longer burdened with inefficient BLOB I/O.
Smaller and fewer content databases. Can leverage DFS, snapshot-ing, and/or mirroring for BLOB store recovery.
BLOBs can be remoted to WORM-compliant storage platforms like EMC Centera, Hitachi HCAP, or OSAR.
You can implement HSM where content can be moved to less expensive storage tiers as it becomes less relevant.
Database sizing guidelines become largely irrelevant.
(1)50GB SharePoint Content database
One Terabyte of Content
MetadataPPT
XLS
DOCTIFF
Content
Netw
ork
-A
dd
ress
ab
leC
on
ten
t-A
dd
ress
ab
leC
lou
d
950GB of file-based BLOB store(s)
Why BLOB Remoting? More benefits of BLOB remoting/offloadingGenerally faster content upload and retrieval.
Substantially for large (> 100MB) content and bulk operations.Implement multi-tiered content storage chargeback models
leveraging on-premise and/or Cloud-based platforms.Substantially increase the speed of upgrade/migration
processes from SharePoint 2007 to 2010.Content can be compressed or de-duplicated, adding to storage
cost savings.Content can be encrypted for greater transmission and storage
security.
(5) 100GB SharePoint Content Databases
Metadata
PPT
XLS
DOCTIFF
Content
NA
S
4500GB of file-based BLOBs(50) 100GB SharePoint Content Databases
PPT
XLS
DOCTIFF
Content & Metadata
SA
N o
r D
AS
SA
N o
r D
AS
SAN (RAID 10) = $12/GBNOTE: RAID 10 has 100% overhead
NAS (RAID 10) = $7/GBNOTE: RAID 10 has 100% overhead
5TB (100%) of content x 2 = 10TB10TB = 10,000 GB x $12/GB
4.5TB (90%)of content x 2 = 9TB9TB = 9,000 GB x $7/GB
= $63,000
.5TB (10%) of content x 2 = 1TB1TB = 1,000 GB x $12/GB
= $12,000
Total Cost $75,000
SAN (RAID 10) = $12/GBNOTE: RAID 10 has 100% overhead
Why BLOB Remoting? The simple case for Remote BLOB Storage
All Content in SQL
BLOBs Remoted to NAS
Total Cost $120,000
500GB in Content Databases
(5) 100GB SharePoint Content database
Metadata
PPT
XLS
DOCTIFF
Content
NAS (Tier 2) NAS (Tier 3) Cloud (Tier 4)
4500GB of file-based BLOBs
SA
N o
r D
AS
NAS (RAID 10) = $7/GBNOTE: RAID 10 has 100% overhead
.5TB of content x 2 = 1TB1TB = 1,000 GB x $7/GB
= $7,000
.5TB of content x 2 = 1TB1TB = 1,000 GB x $12/GB
= $12,000
Total Cost $35,200
SAN (RAID 10) = $12/GBNOTE: RAID 10 has 100% overhead
NAS (RAID 5) = $3/GBNOTE: RAID 5 has 25% overhead
2TB of content x 1.25 = 3TB3TB = 3,000 GB x $3/GB
= $9,000Cloud = $.10/GB/Month2TB of content x 1 = 2TB
2TB = 2,000 GB x $.10/GB/Month= $200 * 36 Months
= $7,200
Why BLOB Archiving? The simple case for Remote BLOB Archiving/Tiered Storage
500GB in Content Databases
BLOB Remoting and Archiving ROISummary
5TB of Content.Costs $120K to store in SQL on Tier 1 storage.Costs $75k to offload 90% to Tier 2 (NAS) storage.Costs $35k to archive BLOBs to less expensive tiers as it ages,
making room for new content in the more expensive tiers.The ultimate goal of archiving/tiered storage is to make
incremental investments in storage on the least expensive tiers only, move content to those tiers as quickly as possible, and make room for new content in the more expensive tiers.
EBSRBSThe other stuff
How to get the Blobs out
Introduced by SharePoint team WSS V3 SP1Farm ScopedCOM Interface, requires implementation by a provider
Implements Save and Load functions(Save binary, Retrieve Binary)
Once saved the provider returns a BLOB id to the system which is saved in place of the blob itself.
When retrieving SharePoint recognizes there is a blob id and not a blob and hands the ID to the EBS provider
Not deemed long term solutionNo planned for migration to future technologies
What is EBS
Provider basedNo default garbage collection
Provider must account for thisNo affect on existing content
Provider must account for thisExport –Import Options
No direct SQL integrationPurely implemented by SharePoint
EBS Misc
Remote Blob StorageImplemented entirely by SQL Server 2008 and laterUses a Provider Model
Its all about the ProviderImplemented by Managed Code (Not COM)Microsoft provides a default provider based on FILESTREAM
Only available for local disks because of the FILESTREAM limitations
No encryptionNo Mirroring Support
What is RBS
How does it work?
?
SQL RBS Client Library
RBS Provider
BLOB Store 1
RBS Provider
RBS Provider
BLOB Store 2
BLOB Store 3
RBS Maintainer
SharePoint Object Model
ContentDatabas
e
SQL Server
Provider API
RBS
A Typical RBS Workflow – Save document
SharePoint Web FE
SharePoint Object Model
Relational Access
1. Save Request
3. Save BLOB
4. Write BLOB
5. Return BLOB Id
6. Save Metadata & BLOB ID
7. Back to User
BLOB Store
ContentDatabase
SQL Server
ConfigDatabase
2. Enforce
Biz Logic
BLOB Store Provider Library
RBS Client Library
A Typical RBS Workflow – Read document
SharePoint Web FE
SharePoint Object Model
Relational Access
1. Open Document
4. Read BLOB
5. Read BLOB
6. Return BLOB
3. Get BLOB Id
7. BLOB Data to User
BLOB Store
ContentDatabase
SQL Server
ConfigDatabase
2. Enforce
Biz Logic
BLOB Store Provider Library
RBS Client Library
Provider based on “FileStream” functionalityAll blobs must be local to the SQL serverNo management interfaceNo monitoringRudimentary garbage collection
The “Out of the Box RBS Solution”
Console ApplicationTakes parameters to run per databaseCan be used with Task Scheduler
Reference ScanningFind orphans
Deletion PropagationDelete them
Orphan CleanupGet rid of the laggers
Keep in mind the true garbage time line (Recycle Bin)
RBS garbage collection (RBS Maintainer)
Maintainer Setup
Create File Stream group for content databaseInstall RBSActivate Provider (PowerShell)Test
RBS/File Stream installation steps
use [WSS_Content_Blob] if not exists (select * from sys.symmetric_keys where name =
N'##MS_DatabaseMasterKey##')create master key encryption by password = N'Admin Key Password !2#4'
use [WSS_Content_Blob] if not exists (select groupname from sysfilegroups where
groupname=N'RBSFilestreamProvider')alter database [WSS_Content_Blob] add filegroup RBSFilestreamProvider contains filestream use [WSS_Content_Blob] alter database [WSS_Content_Blob] add file (name = RBSFilestreamFile, filename =
'c:\Blobstore') to filegroup RBSFilestreamProvider
Create new database for use with Filestream
msiexec /qn /lvx* rbs_install_log.txt /i RBS_x64.msi TRUSTSERVERCERTIFICATE=true FILEGROUP=PRIMARY DBNAME="NoBlobs2" DBINSTANCE="local" FILESTREAMFILEGROUP=RBSFilestreamProvider2 FILESTREAMSTORENAME=RBSFilestreamFile2
Install RBS
$cdb = Get-SPContentDatabase -webapplication http://falcon$rbss =$cdb.RemoteBlobStorageSettings$rbss.Installed()
Retun should be True
$rbss.Enable()$rbss.GetProviderNames()
Return will be RBSFilestreamFile2
$rbss.SetACtiveProviderName($rbss.GetProviderNames()[0])
Enable it
When do I use RBS
Backup Restore Order of OperationsBackup:
Considerations
SQL
BLOB Store
Both Backups CompleteBackup Start
BLOB Store
SQL
Both Restores are CompleteRestore Start
Restore:
By default getting the content out of the database only solves a small percentage of the real problemYou must strive to drive real efficiency in your environment
Tiered Storage!Easier planning for RTO, RPO and Recovery Targets
EBS VS RBSRBS is said to be the path forwardRBS is application agnostic
EBS does not require SQL 2008EBS works in both 2007 and 2010
Weighing it all out
Questions to ask
Shopping for a 3rd party?
Reasons for Storage Optimizationhttp://nexus.realtimepublishers.com/irsc.phpArchitecture of External BLOB Storagehttp://msdn.microsoft.com/en-us/library/bb862195.aspxJie Li Blogshttp://blogs.msdn.com/b/opal/archive/2009/12/07/sharep
oint-2010-beta-with-filestream-rbs-provider.aspxBinary Large Objects: Externalizing BLOB storage
w RBShttp://www.lcbridge.nl/vision/2010/blob.htmhttp://nevertalkwhenyoucannod.typepad.com/nevertalk/
2008/11/sharepoint-archiving-1---rbs-vs-ebs-vs-content-transfer-vs-shortcuts.html
SELECT database_id,object_id, index_id, index_type_desc,avg_fragmentation_in_percent, page_count
FROM sys.dm_db_index_physical_stats (7, NULL, NULL, NULL, 'limited')