performance and capacity management
DESCRIPTION
Performance and Capacity Management. Nicolas Ménigon , SQLi et Dung T. Thoi, Microsoft Suisse. The Challenge SharePoint 2010 Performance Improvements The Capacity Management approach in 2010 Capacity Guidance. Agenda. SharePoint 2010 Performance Improvements. - PowerPoint PPT PresentationTRANSCRIPT
Nicolas Ménigon, SQLi etDung T. Thoi, Microsoft Suisse
Performance and Capacity Management
• The Challenge• SharePoint 2010 Performance Improvements• The Capacity Management approach in 2010• Capacity Guidance
Agenda
SQLI en bref :20 ans d’expérience, 20 agences dans le monde, 2000 collaborateursSQLI Suisse : 10 ans d’expérience, une 100ne de collaborateurs, 2 agences (Genève et Lausanne)
SQLI et MicrosoftMicrosoft Gold PartnerL’offre :
Consulting & ExpertiseRéalisation (projets forfait / assistance technique / TMA / Centre de services in shore - Near Shore)Formation et Accompagnement (AMOA, AMOE)
L’expertise technologique :Plateformes : SharePoint 2007 et 2010, SQL Serveur, Biztalk.Net : Architecture, Industrialisation (TFS/VSTS), framework et langages, RIA/RDA (Silverlight, WPF) & Multitouch
Les PLUS :Centre formation (CPLS / SDPS) certifié Microsoft : SQLI InstitutWeb Agency intégrée
SQLI en quelques mots
The Challenge
Project Server2007
Project Web Access
Shared Service Provider
SharePointSharePoint Server Client Apps
Browsers (IE)
InfoPath
Project
RSS Viewers
SharePoint Designer
OneNote
Groove
Project Server2007
Project Web
Access
Office Client
Outlook
Search Service
Excel Calc Service
Business Data Catalog
User Profiles
ContentConfig Workflow
2007
Windows SharePoint Services
InfoPath Service
More, More, More…
How is SharePoint 2010 Different
Project Server2007
Project Web Access
Shared Service Provider
SharePoint
Search Service
Excel Calc Service
Business Data Catalog
User Profiles
ContentConfig Workflow
SharePoint Server Client AppsBrowsers (IE)
InfoPath
Project
RSS Viewers
SharePoint Designer
OneNote
Groove
Project Server2007
Project Web
Access
Office Client
Outlook
Search Service
Excel Calc Service
Business Data Catalog
User Profiles
ContentConfig Workflow
2010 2007
Windows SharePoint Services
InfoPath Service
SharePoint Service Applications
Shared Service Provider
SharePoint 2010
Search Service
Excel Calc Service
Business Data Catalog
User Profiles
SharePoint Foundation
ContentConfig Workflow
SharePoint Server Client AppsProject Server
2010Project Web
Access
Windows SharePoint Services
2007 Browsers (IE)
InfoPath
Project
RSS Viewers
SharePoint Designer
OneNote
Office Client
Outlook
BusinessConnection
Service
GrooveSharePoint Workspace
SharePoint Service Applications
SharePoint 2010
Search Service
Excel Calc Service
BusinessConnection
Service
User Profiles
SharePoint Foundation
ContentConfig Workflow
SharePoint Server Client AppsProject Server
2010Project Web
Access
Sandboxed Code Service
Usage & Health Logging
Word Conversion
Service
PowerPoint Broadcast
ServiceWeb Analytics
Service
Visio Graphics Service
Access Service
InfoPath Service
Managed Metadata
PerformancePoint
Browsers (IE)
InfoPath
Project
RSS Viewers
SharePoint Designer
OneNote
SharePoint Workspace
Office Client
Outlook
Browsers (IE, Fire Fox)
InfoPath
Project
RSS Viewers
SharePoint Designer
OneNote
SharePoint Workspace
Office Client
Outlook
Browsers (IE, Fire Fox)
InfoPath
Project
RSS Viewers
SharePoint Designer
OneNote
SharePoint Workspace
Office Client
Outlook
Visio
Access
SharePoint 2010 is Scalable and Flexible
WFE and App Servers have more workSQL Servers have more workThe Client Browser has more work
SharePoint 2010 Perf Challenge
SharePoint 2010 Performance Improvements
End User Perceived Latency
Throughput and Scale
Out
Capacity and Data Scale
Reliability
Performance Terminology
Lighter and faster PagesEarly RenderingWAN OptimizedBetter First and Second Browse (PLT1 and PLT2)Focused on Popular ScenariosIE8 and similar new browser are better for WAN Dependent on Content and Customizations
Latency Improvements
WFE & App Servers
SQL
Incremental File Open/Save New protocol (Cobalt)Save is done in the background
File Open Save Latency Investments
Goodbye
Enabling new deployment scenarios in 2010Examples:
100 Million Items per Search Index (1 Billion with Fast 2010)Tens of Millions of Documents/Items in a single ListView/Query 5000 items at a time
Many recommendations/limits stay the sameSite Collections per WebApp (150,000)Site Collections per Content DB (50,000)Content DB Size (100 GB)
Data Scale Improvements
The Product has its limits!Caution: • Plan & Prototype & Measure ahead of
time• Don’t hit all limits simultaneously
Site Collection/Content DB Size Considerations
Mostly Read Write Intensive
100 GB
1 TB
Content DB Size
Read/Write Mix
100 GB and beyond…Repository / ArchiveChallenges:• Fast Storage for
performance• Backup & Restore • Upgrade
Team Collab
Frequent Writes
• Optimized Key Scenarios – reducing transaction cost • Optimized SQL to sustain higher load• Added Protection to sustain a stable deployment
Throughput & Reliability Improvements
WFE & App Servers
SQL
• Large List Throttling• Throttling Excessive
Client Load
New built in protection against latency spikes and unplanned downtimes
Large List Throttling
Demo
Throttling Excessive Client Load• The Server informs Clients on it health condition
• The 2010 Clients Tune down the sync frequency (Backoff) when server is in stress/overloaded
• The Server starts throttles “low-priority” requests if health drops below thresholds, and becomes more aggressive just before going down
Before Clients Backoff
After Clients Backoff
1 2503!
SharePoint 2007 Cap Planning
Analyze
Optimize
Initiate Change
Deploy and
MonitorModel
Study, Experiment and Model Pilot +Deploy
Stabilize
Over Sizing Under Sizing
$$$$
Proper Sizing
1 2
SharePoint 2010 Cap Management
Design
Pilot, Test and Optimize
DeployAnalysis
and Maintain
Model
Pilot +Deploy
Over Sizing Under Sizing
$$$$
Study, Experiment and Model
*Load Test Kit 2010 *SPDiag 2010
* These tools will ship as part of the SharePoint Administration Tool Kit
*SPDiag 2010
The Logging DB
The Dev Dashboard
Monitoring in SharePoint 2010
• SharePoint 2010 logs everything it does into the ‘Logging Database’
• Enabled by default on all deployments• Customizable retention policy• Schema is documented
You can read, query and build reports directly from the Logging DB3rd party apps can write their data to the Logging DB
• In SharePoint Server 2010, Web Analytics processes and uses this data for advanced reports
Logging Database
Capacity Guidance
Capacity Planning Tech Center - Table of ContentPlanning
Capacity Planning 101Plan for software boundaries
RecommendationsTopology recommendationsHardware recommendationsData Storage recommendations
Guidance per workloadCase Study documentation
Employee Portal, Company Portal, Team Collaboration, Social Networking, Document Management, Search…
Partners/Third party Case Studies - (Intel, HP, EMC…)Capacity Management Tools
SharePoint 2010 Load Test KitSPDiag 2010
Troubleshooting SharePoint 2010 Performance Monitor and troubleshooting SharePoint Scalability
Related ContentSQL capacity planning guidance links...
SharePoint 2010 "Capacity Planning Book"
Capacity Planning 101:
Your Star
No Cookbook
A universe of SharePoint Deployments
Chapter I: Overview
Standard Architectures
Single Server
Demos and Dev Boxes
Limited deployments minimum servicesup to 5000 users (~5 RPS)50-100 GB of data
Small Farm
WFE & App Servers
SQL
Schematic Diagram, not to be use as a recommendation for Server Counts
Limited deployments minimum servicesup to 5000 users (~5 RPS)50-100 GB of data
Demos and Dev Boxes
Common Enterprises10-50k users (~50 RPS)1-2 TB of data
Single Server
Small Farm
Medium
Farm
WFE & App Servers
SQL
App Servers
SQL
WFEbApp ServersSQL
WFE
Large EnterpriseUp to 500k users (~500 RPS)10-20 TB of data
Large Farm
Federated Services
Schematic Diagram, not to be use as a recommendation for Server Counts
Standard Architectures
What Differentiates SharePoint Deployments?Hardware, Setting and TopologyFacts: # of servers, HW spec, rolesTuning Options: Add WFE or App Server, tune settings
DatasetFacts: #of Site collections, DBs, Web Apps, Data SizeTuning Options: Split Site Collections, Balance Content DBs
WorkloadFacts: purpose, services, # of Total users, concurrency, RPSTuning Options: Split farms, disable services, block clients
Health And Performance Score Availability, Latency, Throughput, Responsiveness, Failure Rate…
8
.Your Star
Medium Farm: Logical Architecture
http://finance
Application pool
Web application—Finance Web
Application pool
Division 1
http://fabrikam
Division 2 Division 3
Web application—Company Web
http://my/personal/<user>
http://my
Web application—My Sites
Application pool
Managed Metadata
Secure Store Service
Default group Custom group
Access Services
Managed Metadata
http://hrweb
Application pool
Web application—HRWeb
Search
Custom group
Excel Calculation Services
Excel Calculation Services
User Profile
I I S Web Site—“SharePoint Web Services”
Business Data Connectivity
Business Data Connectivity
Medium Farm: Physical Topology
Excel ServicesCentral AdminUser Profiles
Metadata
Query Index
Excel ServicesUser Profiles
Metadata
1. Model: • Understand your 2007 facts - Workload and Dataset, Targets
2. Design: • Study the guidance on how your facts might change in 2010 • Pick initial Architecture - Hardware, Settings and Topology
3. Pilot, Test and Optimize • Load Test your 2010 farm, and analyze the Performance • Tune the Hardware, Settings and Topology• Split to multiple Farms if needed
4. Deploy • Implement in the data center and migrate users and content over
5. Monitor
• Track your deployment periodically, troubleshoot• Identify the bottlenecks and pain points, go to step 1.
Capacity Planning 101:Chapter II: Capacity Management Steps
Boundaries TypesHard limits enforced by system such as 2GB filesConfigurable Boundaries such as 5000 items in a queryNon-limited Boundaries - dependent on HW and configurations
Rules of Thumb 1 WFE for 10-20k usersEnabling 2010 features will require more power! (x1.5 to x2)Dedicate SQL power to Logging DB, Web Analytics, Project ServerWFE and Apps Servers – starting at quad processor, 8 GB RAMSQL - starting at quad processor, 16 GB RAM…
SharePoint 2010 Software Boundaries
The Product has its limits!Caution: • Plan & Prototype & Measure ahead of time• Don’t hit all limits simultaneously
• Early references for initial architecture• Understand the differences and extrapolate • Prototype, Measure and Tune.
MSITs Deployments/Case Studies
. Team
Collaboration
.Enterprise
Portal
.Social
Networking
Federated Search
federated Meta Data,
Web AnalyticsUser Profiles
.
..
Browser Re-
quests
Description: Company wide, Self provisioning, Team Collaboration, sites ~100k users
Team Collaboration
Farm’s Total Data Size 8.3 TB Largest Content Database 400 GB Number of Web Applications 2 Number of Content Databases 87 Total number of Site Collections 34,500 Total number of Sites 101,500 Search Index Size (# of items) 23,000,000
Workload: Total number of users per day 79,000 Concurrent users (Distinct Users per Minute) ~600 Total Requests per day ~17,000,000 Hourly Average RPS [Requests per Sec]: ~200 Average RPS at Peak Hours [Requests per Sec]: ~400
Data Set:
Search OneNote MSOffice Others0%
10%20%30%40%50%60% 57%
18% 17%9%
Requests Distribution Across Client Apps
Browser requests generating ~83%
Hardware: Hardware Load Balancer 2 SQL Servers [16 Core, 32 GB, SQL 2008]
5 WFE: [8 Core, 32 GB, Win 2008 R2]4 General + 1 crawl target
3 App Servers [8 Core, 32 GB, Win 2008 R2]WAC, Excel, PowerPoint
Services:Federated Services: User Profile, Web Analytics, BCS, Managed Metadata
*Logging DB on a separate SQL box
Availability: Average WFE Uptime 99.9% Server Side Latency: Responsiveness 0.3 Sec Server Side Latency: % of slower responds 4% Non scheduled IIS recycles WFE 0 SQL Average CPU 40% App Server Average CPU/RAM 10% / 5 GB WFE Average CPU/RAM 35% / 10 GB
Health & Performance Score:
Compared to O12: No SQL changes4 new App/WFE boxes + Scale up:
4 cores 8 cores, 16 GB 32 GB
* We plan to scale down now that things look healthy
Team Collaboration
• Load Test Kit (LTK) – V1 at RTM• SPDiag 2010 – (sometimes in 2010)
Additional investments:
• SharePoint 2010 is • Bigger, Richer, Scalable and Flexible• Faster, and more Reliable • Easier to Diagnose
• Plan, Prototype, Measure• Expect more guidance over time
Summary
Questions ?
Thank you for your Attention!For more Information please contact
Name Nicolas MENIGONTitle
Email [email protected] Tel Mobile
Company Name SQLiAddress LineZIP & Town
