sharepoint speedmetal admin 101 - spsden

SharePoint 2010 SpeedMetalAdmin 101

Chris McNultySharePoint Strategic Product Manager

Quest Software

3

©2011 Quest Software, Inc. All rights reserved..

Chris McNulty• SharePoint Strategic Product

Manager at Quest Software

• 10+ years with SharePoint

• 20 years consulting (led KMA SharePoint practice) and financial services technology (Santander, John Hancock/Manulife, GMO, State Street)

• MBA in Investment Management from Boston College

• Write and speak often on Microsoft IW technologies (blogs & books)

• MCSE MCTS MSA MVTSP MCC

• Hiking, cooking, playing guitar, colonial history, photography

• My family: Hayley, three kids (17, 7, 5) and my dog Stan

About Home

7


Welcome to Denver – Orange Crush

8


Objectives

• Out Of Scope• Deep Dives (e.g. PowerShell, BI, Upgrade, SQL DBA)

• Development

• Customization

• Design & Architecture

• Power User (e.g. Library Customization, Designer Workflows, etc.)

• Office 365

• Rules• Move fast, PowerPoint is shared

• Questions – time permitting during session

• Any time after session – email etc.

9


Agenda• The dilemma

• Architecture, Design & Planning

• Installation and Upgrade• Post Installation Best Practices

• Service/Feature Placement

• Support• Monitoring and Optimization

• Backup

• PowerShell

• Development Functions

• Optimization

• Patching

• SQL Maintenance

• Best Practices

10


Congratulations!oYou’re the new SharePoint Administrator!!!oBut…oYou’re still responsible for:• Exchange

• Active Directory

• SQL

• Desktop

• Help Desk

• Network/Firewall

• Cooking & Cleaning

• Etc.

11


The DilemmaoSharePoint administration is often an ‘add-on’ for other IT professionals (SQL DBAs, AD Admins, Exchange Engineers)

oTime and focus are scarce resources!

oCommon pain points include

• Upgrades are complex and hard to monitor

• Dispersed workforce, little control of browsers and Office versions

• Hard to understand and troubleshoot “behind the scenes” performance and capacity planning

• Best practices not always understood or compared to system health

• “All or nothing” administration means IT must be engaged for all admin responsibilities, even search

Microsoft SharePoint Server 2010 … the bright frontier

Eastern Long Island, July 4, 2010

Architecture and Design

15


Server Farm – Web Front End

• Typical Roles:

• http services

• Search query

• Scaling

• Add servers to load balanced cluster

• Performance Optimization

• RAM

• Easily virtualized

15

16


Server Farm - Application Server

• Typical Roles:

• Search index/crawl

• Excel calculation

• User profiles

• Managed Metadata

• Scaling

• Add search servers and partitions

• Move shared services to dedicated servers


• CPU

16

17


Server Farm - Database

• Typical Roles:

• Data storage

• SQL Reporting

• Scaling

• Add storage capacity


• Disk I/O

17

18


Sizing - Single Server

• Typical Roles:

• Small teams

• Small pools of documents

• Considerations

• Performance & fault tolerance less of a concern

• SQL & Web on same system

• Search not a core function

19


Sizing - Medium Farm

• Typical Roles:

• 100-10,000 users

• 10,000 – 1MM documents

• Scenarios

• Enterprise portal

• Large scale collaboration

• Broader applications platform

• Larger external search pool

• Mix and match internal external front end servers on common content databases

20


Sizing - Large Farm

• Typical Roles:

• Large distributed enterprise users (10000+)

• Large pools of documents (>1MM)

21


Top Level Logical Components• Farm level

• Web applications

• Independent top level URLs

• Run inside IIS pools

• Consume shared services and admin from the farm or other farms

• Site collections

• Security, branding, database frontier

• Contain single sites or site hierarchies

• Sites

• Group related SharePoint elements (lists, libraries, pages, web parts)

21

Web Applications

http://intranet

http://centadmin

Site Collections

Site Hierarchies

Single SItes (MySite)

Sites

Lists Liibraries Pages Web Parts

23


Logical Components

• High capacity!

• Maximums

• 250,000 sites per site collection

• 5,000 site collections per content DB

• 200GB max content DB (single site collection)

• >200GB post SP1

• 300 Content DBs per web application

• 30MM documents/library

• 2GB document size

• 2011 News• 14TB Demo

24


Disk SizingContent Search

Initial Content Size XXX GB External Crawl Size YYY GB

Initial User Pool U

User Collab Size .25GB

n YR Growth Rate – Archive Rate

G%

End Content Size XXX (1+G)n = ECS End Search Size YYY (1+G)n = ESS

End User Collab Size .25 * U * (1+G)n = EUCS

Content DBs ECS + EUCS

Search DBs .05 * (ECS + EUCS + ESS)

Search Index Files .05 * (ECS + EUCS + ESS)

• Inputs: Size of SharePoint content and non-SharePoint content included in search

• For DBs, don’t forget transaction logs, disk dumps (if used for backup) which can add 1-3X.

• In SAN or virtual environments, not all disk need be provisioned early

25


Memory Sizing

• Web Front End (WFE)• 8GB minimum

• 12GB larger

• 16GB max

• Application• 8-12GB

• SQL• 8-20GB

• HP Sizing Tool• http://

h71019.www7.hp.com/activeanswers/Secure/548230-0-0-0-121.html

• Don’t forget about the swap files (1-2X size of RAM)

http://h71019.www7.hp.com/activeanswers/Secure/548230-0-0-0-121.html

http://h71019.www7.hp.com/activeanswers/Secure/548230-0-0-0-121.html

28


Authentication Architecture

• Classic• Zones mapped

• Single URL for each authentication method:

• e.g.

• Default: Windows

• Forms (LDAP, SQL, ADAM)

• Generate an NTToken to represent SPUser

• Claims• Unified URL

• Multiple sources (Windows, FBA, SSO) combine to generate a single SAML token to represent SPUser

Regular label-callout text

Multi-AuthenticationMixed Authentication

SharePointFarm

Web Application

Extended Web Application




Zone: Custom

Zone: Extranet

Zone: Intranet

Zone: Internet

Zone: DefaultWindows Authentication

FBAAuthentication

...

...

...

SharePointFarm

Web Application





Zone: Custom

Zone: Extranet

Zone: Intranet

Zone: Internet

Zone: DefaultWindows AuthenticationFBA Authentication

SAML Based AuthenticationFBA Authentication

Windows Authentication

...

...

28

29


Authentication ScenariosMixed Mode

Remote Employe

es

Extranet

Zone

IntranetZone

EmployeesFBA

claims

Windows

claims

https://extranet.domain.com http://intranet

30


Authentication ScenariosMulti Authentication

IntranetZone

Employees

FBAclaims

Windows

claims

https://Corporate.domain.com

SAMLclaims

Vendors Partners

31


Internet Topology – Edge Firewall

• Traditional

• Inexpensive

• Simple

• Only one firewall

• External traffic comes inside internal network

WFE SQLEdge

Firewall AD

Internet

App

Internal Network

32


Internet Topology – Perimeter

• More complex

• Duplicative networks, backup, AD

• External traffic is reserved

• Larger server foot print (exposure) in perimeter

• Internal users need domain trusts

• Internal users access site across firewall

Router/Firewall

WFE SQLEdge

Firewall AD

Internet

App

Perimeter Network Internal Net

33


Internet Topology – Split Back to Back

• Most complex

• Intricate firewall rules

• App, AD and search roles optionally in perimeter

• Optional internal WFE or internal users always cross a firewall

• Crawl topologies important to avoid overtaxing the firewall

Router/FirewallWFE SQL

EdgeFirewall AD

Internet

App

Perimeter Network Internal Network

34


Internet Topology – Enhanced Techniques

• Multi-farm

• SSA farm

• Content publishing

35


Common Integration Touchpoints

InternalActive Directory

Exchange / File Shares

Index and integrate BCS data

External• LDAP

• Mail Relay

• Indexed search content

Other Systems• FAST (Search)

• Project Server / TFS

• BizTalk

• LoB/Dynamics

• Oracle (BCS)

• Notes (Search)

• Wikis and other indexed web sites

37


Platform Basics

• SharePoint 2010 is a 64 bit only platform. Direct upgrades from 32 bit to 64 bit requires prep work.

• Windows Server 2008 or Windows Server 2008 R2 X64

• SQL Server 2005 x64 SP3 CU3

Or

• SQL Server 2008 x64 SP1 CU2

Or

• SQL Server 2008 R2

38


Shared Service Applications• 2007 Shared Services

Provider has been broken up; each of its elements is now a Shared Service Application

• Mix and match them singly or in groups, to match farm’s needs.

• Crawl/index no longer a single server role

• In 2010, administration can be delegated• Key targets: Enterprise search,

metadata, user profiles

http://globalweb http://itportal

Visio

Search

Excel Calc

Metadata

User Profiles

39


Client/Browser Technology

• Internet Explorer 7/8/9, Firefox and Safari are all supported.

• Some support for Chrome

• IE6 is not supported

• Most other browsers are still supported for Internet configurations

• Office 2010 includes optimizations for the new platforms

• Offline Access• 2007: used Outlook 2007 and Groove

• SharePoint Workspace 2010 integrates offline documents and lists

40


Office Web Applications

• SharePoint 2010 provides a server version of Office applications – Office Web Access, or “OWA”.

• In part, this enables simultaneous multiuser editing of Office documents:

• Excel in OWA, not client

• Word/PowerPoint on client only if file opened from a shared document library

• OneNote client or OWA

Installation and Upgrade

42


Installation - Prerequisites• Servers:

• Windows 2008 R2 X64 Enterprise Edition

• SQL Server 2008 R2 x64

• Service Accounts• spfarm (Farm acct; local admin on the SharePoint servers and either sa or dbcreate, dbowner and

security admin on the SQL server.)

• svcsql (SQL Server service acct)

• sppool (IIS pool acct)

• spcrawl (Search accts)

• spadmin Interactive admin (install account; local, site collection and farm admin privileges)

• Install as SPAdmin

• Install Software Prerequisites - Checks for following elements:• Application Server Role, Web Server (IIS) Role, Microsoft SQL Server 2008 Native Client, Hotfix for

Microsoft Windows (KB976462), Windows Identity Foundation (KB974405), Microsoft Sync Framework Runtime v1.0 (x64), Microsoft Chart Controls for Microsoft .NET Framework 3.5, Microsoft Filter Pack 2.0, Microsoft SQL Server 2008 Analysis Services ADOMD.NET, Microsoft Server Speech Platform Runtime (x64), Microsoft Server Speech Recognition Language - TELE(en-US), SQL 2008 R2 Reporting Services SharePoint 2010 Add-in

43


Installation – Grey Wizard

• Initial• Product Key

• Type of installation - Always SERVER FARM

• Installation Type - Complete [Not Single Server]

• Accept default file locations – index files will stay on C:\Program Files\Microsoft Office Servers\14.0\Data

• At end NO Wizard

• Run OWA Setup

• Then, WIZARD! The wizard starts, and yes, it’s OK for IIS to reset during the wizard…

• Create a new farm• Set farm account

• Pick configuration database, Passphrase, CentralAdmin Port (Conventions)

• Final confirm and let the wizard run

44


Installation – White Wizard?

• Pros• Easy – shaken and stirred

• All SSAs Configured

• Saves time and PowerShell hand tooling of SSAs

• Cons• My Sites setup in same app and DB as

primary

• Database Names are default, GUID happy

• Kills kittens (ask SharePoint 911!)

• What it does• Sets up service acct for SSAs and other

services (sppool)

• Sets up a port 80 web app with a My Sites Host sub-site collection in WSS_Content database

45


Predictable Upgrade

• Three paths• In place

• Database upgrade

• Third party tools

• Process• Pre-upgrade checker

• Visual Upgrade

• Resumable upgrade

• Progress reports

• Parallel DB upgrades

46


Upgrade Preparation

Additional Prepwork• Content pruning

• Database alignment

• stsadm-o mergecontentdbs

DB Attach

• Preinstall Required Features

• Stsadm –o addcontentdb –databasename DBNAME –url URL –assignnewdatabaseid

• PowerShell Mount-ContentDatabase

• Test, test, test!

47


Pre-Upgrade Check• SharePoint 2007 SP2 minimum,

October 2009 CU best

• STSADM.exe –o preupgradecheck

• Documentation

• All servers and components in the farm, and whether the servers meet 64-bit hardware/OS requirements

• Alternate access mapping URLs

• A list of all site definitions, site templates, features, and language packs that are installed in the farm.

• Unsupported farm customizations (such as database schema modifications).

• Database or site orphans

• Missing or invalid configuration settings in the farm (missing Web.config file, invalid host names, invalid service accts).

• Whether the databases meet the requirements — for example, databases are set to read/write, and any databases stored in Windows Internal Database and larger than 4 GB.

48


Upgrade servers without changing the

user interface

Switch-on new UI across site collections

in a controlled manner

Preview new UI

IT Pro Investments – Visual Upgrade

49


DEMO• Upgrade

MonitoringMonitoring and Operations

51


Proactive Issue Resolution

• Developer Dashboard• Empower developers and

users

• Integrated Health Analyzer• Runs when necessary

• Alerts anomalies

• Fixes when it can

• Web Analytics• User usage

• Resource usage

52


Logging, Monitoring, and Alerts

• Unified Logging

• Out-of-the-box reports

• Richer Web Analytics

• Open Schema

• SCOM Integration

• PLUS• Developer Dashboard

• Health Analyzer

ULS Logs

Windows Events

Page requests

Feature Logging

Health data

Logging DB

http://images.google.com/imgres?imgurl=http://api.ning.com/files/MSkY5K50pB2mD02AgzOnIUnjFf7Du4hPd30u0JfqdoziCaWWJlO9PKOgVxtZtCdLuGNvfOOTqZzFrLh-mvFYsOKg91gxn0m7/Excel.jpg&imgrefurl=http://datashaping.ning.com/profile/VincentGranville&usg=__sTBSrEUG-RCSVa5bBuRBphZGCTU=&h=321&w=388&sz=13&hl=en&start=2&sig2=AvI5TaaWsHWb7tIz6Kqteg&um=1&tbnid=KNBphhuWmyfA5M:&tbnh=102&tbnw=123&prev=/images?q=excel&hl=en&sa=N&um=1&ei=JBEWSvbOJoTUswOOnNDhCA

53


Monitoring – General

• Monitoring• SCOM

• Central Admin

• Health Analyzer

• Site Collection Web Analytics

• Developer Dashboard

• stsadm –o setproperty –pn developer-dashboard –pv OnDemand

• (Get-SPFarm).PerformanceMonitor.DeveloperDashboardLevel = "OnDemand"

• Troubleshooting

• Correlation ID – One GUID to rule them all!

• ULS Logs, Event Logs, Performance Monitor

• OR

• WSS_Logging DB

54


Developer Dashboard Improve customized solutions with the Developer Dashboard

SQL Queries Performance

Memory Usage

Data-Request Trace

55


Monitoring – WSS_Logging

• Query Database Views Directly

• Requires Timer Jobs Enabled• Diagnostic Data Provider: Trace Log

• Diagnostic Data Provider: Event Log

• ULS Configuration Matters

• Database will GROW!

• Aggregates from ALL Servers

• Sample: • SELECT * FROM [WSS_Logging].

[dbo].[ULSTraceLog] WHERE CorrelationID = '04377DAE-C2FD-4DBE-A57E-101B3005059E'

56


Backup/Recovery

• Third Party Tools

• Recycle Bin

• Granular / Site Collection Backup (UI)• *.bak file

• Restore-SPSite

• Unattached Recovery• Browse unattached content database

• Account needs DB permissions

• Database need not be on the same server!

• No more granular than list or library!

• Browse Content• Export Site or List

• Export as a CMP file

• PowerShell restore• PS: Import-SPWeb http://msshome2010 –Path C:\

ListRecovery.cmp

• SQL Backup

• SharePoint Backup (UI or script)

http://msshome2010/

57


STSADM Backup@echo off

echo ==================================================

echo Backup the farm

echo ==================================================

@SET stsadm="C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\BIN\stsadm"

rmdir /S /Q "\\spsql08\spbackup\farmold"

ren "\\spsql08\spbackup\farm" "farmold"

md "\\spsql08\spbackup\farm"

%stsadm% -o backup -directory "\\spsql08\spbackup\farm" -backupmethod full

echo complete

58


PowerShell Backup

# NOT NEEDED write-output

write-host ==================================================

write-host Backup the farm

write-host ==================================================

Add-PSSnapIn Microsoft.SharePoint.Powershell

Remove-Item -Path "C:\PSBackup\farmold“ -recurse

Rename-Item -Path "C:\PSBackup\farm" -NewName "farmold"

New-Item -type directory -path C:\PSBackup\farm

Backup-SPFarm -directory "C:\PSBackup\farm" -backupmethod full –verbose –percentage 5

Write-host Backup complete

59


PowerShell

• SharePoint Shell vs. Base Shell• Add-PSSnapin Microsoft.Sharepoint.Powershell

• Command -?• Get-Help Command

• Get-Help Command –examples

• Pipe• Get-Command –Noun SP*

• Get-Command – Noun SP* | Select Name

• Get-Command – Noun SP* | Select Name | Out-File Commands.txt

• Get-SPSite –limit all | Get-SPWeb –limit all | Select URL, webtemplate | Out-GridView• WindowsPowerShell Integrated Scripting Environment to allow Out-

GridView

60


Some Useful PowerShell Snippets

• Visual Update a range of sites:

• Site Backup

• Add MMS Term

$webapp = Get-SPWebApplication http://sitenameforeach ($s in $webapp.sites){$s.VisualUpgradeWebs() }

$str = “SAMPLE”$site = new-object Microsoft.SharePoint.SPSite("http://MYSITE")$session = new-object Microsoft.SharePoint.Taxonomy.TaxonomySession($site)$termstore = $session.TermStores[“MYTERMSTORE"]

[…create group…][…create term set…]

$term = $termset.CreateTerm($str, 1033)

61


Some Useful PowerShell Snippets II

• Create and configure a library

#Load the Sharepoint .net Assembly[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint") #set the url of the site collection to a variable$siteurl = "http://msshome2010/"$subsitename = "Marketing"$newlibraryname = "NewLib"$newlibrarydesc = "NewLib Description" #create the new object passing the site collection URL, attach subsite$mysite=new-object Microsoft.SharePoint.SPSite($siteurl)$subsite = $mysite.openweb($subsitename) #make the new library - 101 is the generic for DocumentLibrary template$subsite.lists.add($newlibraryname ,$newlibrarydesc, 101) #open the new library and break inheritance$mylib = $subsite.lists[$newlibraryname]$mylib.BreakRoleInheritance($false)

67


Development Support – Three Regions

Development

•often internal to developers•problem reproduction that require advanced inspection tools (e.g. Visual Studio) are done here

•permissions can be looser, may have multiple environments for multiple developers

•sensitive data from production cannot be copied here without masking or customer signoff

•changes here can be deployed ad hoc

Staging/Test

•no Visual Studio, no MS Office•match/mirror production as closely as possible; match hardware/system performance as closely as practical•security permissions match production•any sensitive data copied here stays under production-grade controls•test accounts should be created in a separate OU if possible•changes here can only be delivered and deployed from source control and according to production release methods

Production

•optimized hardware configurations•highly secure•no use of user rotating password accounts as service accounts•changes here can only be delivered and deployed from source control and according to production release methods

68


demoMonitoringDeveloper DashboardHealth AnalyzerPowerShell

Optimization

Optimization

71


Optimization

• Disk-based BLOB Caching• Local store for audio/video, PDF other frequent read only files

• Edit in Web.config (C:\Inetpub\wwwroot\wss\virtualdirectories\...)

• <BlobCache location="" path="\.(gif|jpg|jpeg|jpe|jfif|bmp|dib|tif|tiff|ico|png|wdp|hdp|css|js|asf|avi|flv|m4v|mov|mp3|mp4|mpeg|mpg|rm|rmvb|wma|wmv)$" maxSize="10" enabled="false" />

• Location = Local Disk Location

• maxSize = GB

• Enabled = true

• Different from RBS/EBS!

• For publishing sites

72


Patching – Process through August 2011

Patch •SharePoint Foundation

Patch •SharePoint Server

Deploy •Run SharePoint Products and Technologies Wizard•(Or psconfig) •Sequential Application to Central Admin, Application Server(s), Web Front End Servers

73


Patching – Process After August 2011

Patch •Separate patch longer needed – single patch CU now available

Patch •SharePoint Foundation•OR SPF/Server•OR SPF/Server/Project Server

Deploy •Run SharePoint Products and Technologies Wizard•(Or psconfig) •Sequential Application to Central Admin, Application Server(s), Web Front End Servers

74


SQL Maintenance

• Backups• Local Disk – easy but storage intensive

• Agents – remote, requires extra software

• RBS Maintenance• BLOB Orphans

• Log Sizing• Full logged (default) generates huge t-logs

• Simple doesn’t but prevents point in time restore

• Maintenance Plans

Best Practices

77


Troubleshooting – Top Support Questions

• Users Receive “Cannot Connect to Configuration Database” Web Page• SharePoint farm account is locked out

• No one can upload anything but site is up• Database disk volume is full – check transaction logs, backups

• In virtualized environment, host file systems may be full

• I can’t find a document I think I should see; Someone can’t see a file I just uploaded• Security and permission variations

• Document “movement” (a/k/a ECM) try search by name or Document ID. Check ECM logs/audits

• Confirm permissions, and make sure document is checked in (Required properties may be missing)

78


Troubleshooting – More Support Questions

• Repeated requests to re-enter Windows credentials• Add to Local intranet zone, add site, custom level, automatic login with

current user name and password (it’s the last thing in the item list)

• OR Trusted sites

• My workflow didn’t start• Recycle timer service

• “FixSharePoint.exe” = IISReset & Timer Service Recycle

• I’m not seeing the right search results• Confirm that crawls are running and complete by checking crawl logs;

restart a full crawl if crawls finish OK

• I need a file back that I deleted• Recycle Bin Recovery

• Use Backup & Restore

80


Where Governance Begins?

Portal

Dept Sites

Project/Team

Sites

Personal / My Sites

Hig

her

Vis

ibili

ty

Lighte

r govern

ance

Seven Deadly Sins for the SharePoint Professional

83


Seven Deadly Sins

• No SQL maintenance plans

• Default names for every database (WSS_CONTENT_12345abc…)

• No patching

• One environment for everything

• One acct for everything

• Single server install with SQL Express

• Runaway content database size

Seven Virtues for the SharePoint Professional

85


Seven SharePoint Virtues

• Security Applied via AD Groups and SharePoint Groups

• Review System Health

• Test Restore and Recovery

• Monthly Web Analytics Review – Usage, Storage, Search

• PowerShell instead of STSADM

• Governance

• Use ECM, MMS, Clients, Archiving and Training to Keep Content in SharePoint, reduce accidental duplication and keep searching and browsing fresh

86


Congratulations!

• You’re the new SharePoint Administrator!!!

• And now

• You understand:• Design and Architecture

• Installation and Upgrade

• Support and Maintenance and Optimization

• PowerShell

• Customizations

• Troubleshooting

• Best Practices

• People from New York?

87


•Questions

•Evaluations

•Contact Me

88


Thank you…

• Contact• Email [email protected]

• Blog http://www.chrismcnulty.net/blog

• Also http://www.sharepointforall.com

• Twitter: @cmcnulty2000

• LinkedIn:http://www.linkedin.com/in/cmcnulty

• Upcoming:• Nov. 2011 – Microsoft New England (Best of SPC2011)

• Dec. 2011 – SPS South Florida, PSSPUG/Seattle, Gilbane Boston

• Jan. 2011 – SPS Austin TX (Business Intelligence)

http://www.chrismcnulty.net/blog

http://www.chrismcnulty.net/blog

http://www.sharepointforall.com/

http://www.linkedin.com/in/cmcnulty

http://www.linkedin.com/in/cmcnulty

89


Questions?Get more information at

www.Quest.com/ControlYourChaos