SharePoint 2010 SearchShikhar Thapa

Senior Consultant

11/05/2010

• Scale-out Architecture• End-user Enhancements• Social Search• Extensibility Enhancements• Deployment and Manageability• FAST Search for SharePoint

Agenda

Quick, easy, powerful search (for free!)

Complete intranet search

High-end search delivered through SharePoint

Basic search Intranet-wide searchPeople and expertise search

Visual experiences, extreme adaptability and advanced content processingPowerful platform to build on

• 360° customer views• Research portals• Reputation analysis• …

Products for Every Customer Need

Query Object Model

Concepts

Search Technology Concepts

Content Sources - Host the content we want to return in main results

ContentContentContent

OpenSearch Source

Crawling - Traverse URL space to record items in search catalog

Indexing - Extract information from items to enable efficient matching

Query Servers - Accept query requests from users and return results

Search Center - UI for users to issue queries and interact with results

Query Federation - Return results from non-SharePoint Indexes

Crawler

Indexer

Query Servers

Scaling

Index Partition

Connectors - Know how to process different content sources

Index Partition - Subset of the overall index

• Scale-out Architecture• End-user Enhancements• Social Search• Extensibility Enhancements• Deployment and Manageability• FAST Search for SharePoint

Agenda

MOSS 2007 Search Scale-out

Query

Query

Indexer“Single point of

failure”

“Bottleneck”

“The whole index”

“Bottleneck”

+ Query

SharePoint Server 2010 Search Scale-out

Query Query

Indexer“Single point of

failure”

“Bottleneck”

“The whole index”

“Bottleneck”

IndexerCrawler CrawlerCrawler Crawler

Query Query

Multiple Indexers

Crawl Distribution

Query MirroringIndex Partitioning

Stateless Crawlers

Multiple Prop DBs

AdminComponent

Admin Database

Admin Database +Admin Component

Content Distribution • Crawl Distribution

– In-built load balancer distributes hosts to crawl databases

– Crawlers crawl content that is covered by crawl database

– Default configuration can be overwritten using host distribution rules

• E.g. purchasing a new connector

• Query Distribution– Low query latency if all index partitions equal in size – Distribution by hash of documentId– Crawlers partition indexed data and propagate to query

servers

• Scale-out Architecture• End-user Enhancements• Social Search• Extensibility Enhancements• Deployment and Manageability• FAST Search for SharePoint

Agenda

End-User UI• Out-of-box refinement

– Refine over key results properties– Metadata, taxonomy and social tags based results

refinement– Easy to extend over custom properties

• One-stop Search Center– Scopes, web parts, best bets, top answers ,

advanced search– Query federation brings together results from all over

- native support for OpenSearch

• Core search experience– Improved did you mean suggestions– New pre-query and post related query suggestions– “View in browser” link (for most office docs)– Improved query syntax

End-User UI• Improved relevance ranking

– New ingredients for Relevance

– Improved low-noise snippets in summaries

• Enhanced multi-lingual support– Automatic detection of language of many

document types and part of documents– Compound word handling – Improved ranking of documents in multilingual

collections

• New form factors– Mobile search from Smartphone browsers– Desktop search integration in Windows 7

Search Center

• Site Template• Tabbed Interface• Multiple Entry-

points• Keywords,

Definitions, and Best Bets

• Refinement Panel• Extensible

Search Usability Features

Query Suggestion. One Point of Entry

Improved result layout and hit-

highlighting

Refine by query type, and many

other pivots

Best Bets/Did you Mean

Tabbed Interface

New Query Syntax

• Support for Boolean operators for FreeText queries and property queries– (“Search” OR “SharePoint Search”) AND

(title:“keyword syntax” OR title:“query syntax”)

• Prefix matching support for keywords and properties– Bench* jobtitle:Scien*

• Improved operator support for property restrictions– =, >, <, <=, >=– Can create range refinements

Demonstration

• Scale-out Architecture• End-user Enhancements• Social Search• Extensibility Enhancements• Deployment and Manageability• FAST Search for SharePoint

Agenda

Social Search• Entry from Core Results• Fuzzy name search• Self-Search• Expertise/Profile Search• Refinements

Overview of the social search system

Managed Metadata Service Application

(Taxonomy)

AD, LDAP, other

repository

Search Service Application

User Profile Service Application

Connect with People and Expertise

Phonetic and nickname matching

Improved result layout and hit-

highlightingRecently authored

content

Refine by query type, and many

other pivots

Sort by relevance, name

or social distance

Vanity search

Demonstration

• Scale-out Architecture• End-user Enhancements• Social Search• Extensibility Enhancements• Deployment and Manageability• FAST Search for SharePoint

Agenda

Do More with Search

Common Framework, APIs, Development Tools

Richer Capabilities enable search-

driven applications

Browser-Based Extensibility Options

• Search Web Parts– Search Box / Advanced Search– Search Summary– Search Statistics– Search Paging– Search Actions– Search Best Bets / High Confidence Results– Search Results– Search Refinement

• Search Tab Pages

Search Input Web Parts

• Search Box– Scope options– Additional query terms– Advanced Search Page– Search Results Page– Query Suggestion

• Advanced Search– Input types– Properties (metadata)– Search Results Page

Search Utility Web Parts

• Search Summary– “Did you mean”– “You searched for”

• Search Statistics• Search Paging

– Next / Prev Text– Next / Prev Images

• Search Action Links– Determine what links to display

Search Best Bets / High Confidence Results

• Same web part, different configuration• XSLT for display• Show Keywords?• Show Best Bets?• Show High Confidence Matches• Best Bet Limit

Search Core Results

• Location• Results per Page• Enable removal of duplicates• Enable stemming• Selected Columns• Configurable XSLT• Scope

Search Tabs

• Based off of lists in Search Center Site– Tabs in Search Pages– Tabs in Search Results

• Process:– Create Entry Page– Create Results Page– Configure Web Parts on Entry and Results

Page– Add Entry Page to Tabs in Search Pages– Add Results Page to Tabs in Search Results

Search Extensibility: End-User UI

• Change web part properties – no code– Modify XSLT – Modify config XML

• Refinement panel – control metadata available for refinement• Advanced search – control metadata available for advanced search

queries

• Extend OOB web parts programmatically– All OOB web parts are public– Extend web parts to change default behavior

• E.g. Extend core results web part to show result previews

• Extend Search Center with vertical experiences– All web parts communicate through public interface

(SharedQueryManager object)

Bringing it all together…

Search Extensibility - Federation

• Query Federation– Bring external search results into the

SharePoint experience– Based on Open Search standard– Exposed through UI + public federation

object model

Connector Architecture

Search Engine/Index

Search BDC Runtime

WCF Databases

.NET BDC

Connector

Execution call routing through BDCConnector using BDC model file

Custom BDC

Connector

Search user queries

BDC Profile Pages or

External Lists

Search results page – results based on

metadata

Search page points to Profile Page or custom page to show search result

Fetch data from repository

Protocol Handlers

Demonstration

• Scale-out Architecture• End-user Enhancements• Social Search• Extensibility Enhancements• Deployment and Manageability• FAST Search for SharePoint

Agenda

Richer Manageability

• Consolidated administration UI dashboard• Automated service password management

through “managed accounts”• PowerShell support for scripted

administration• Built-in system health monitoring, support

for SCOM monitoring and alerting• Built-in and extensible search analytics

reporting

Other Engine Enhancements

• Support for regular expressions in Crawl Rules

• Native support for crawling case sensitive repositories

• Ability to prioritize Content Sources so as to distribute crawler resources

• New ‘Crawl Policy’ to define how crawler treats error conditions

• Low indexing downtime Search Backups

Demonstration

• End-user Enhancements• Social Search• Scale-out Architecture• Extensibility Enhancements• Deployment and Manageability• FAST Search for SharePoint

Agenda

FAST Search for SharePoint

End Users Content +

Profiles

Architecture

Content search

People search

Unique Social Search from SharePoint Server 2010

Common connectors and content APIs

Common query and federation APIs

Best High-End Content Search based on FAST ESP

SharePoint Server2010

Visual, Conversational Search

Thumbnails

Sorting on any property

Similarity Search

Visual Best Bets

Scrolling PowerPointPreviewsRefinement with

countson any property

Beyond Just SharePoint Search

Highly capable and easy to manage

Same capabilities, plus:

Extreme scale-out

Content Processing pipeline

Entity extraction

Tunable relevance ranking

Easy setup of User Context,

Visual Best Bets, Promotion/

Demotion

Easy to configure sorting, and

refinement

 Enterprise class search system integrated with SharePoint

Enterprise Scale-out (to 100M

docs)

Full Fault Tolerance

Native 64 bit; Hyper-V support

Wizard - driven installation

Consolidated search dashboard

PowerShell support

SCOM support

Full search reporting

Full set of connectors OOB

Easy to add new sources via BDC

Content Processing and Schema

• Extracted document attributes reported as Crawled Properties

• Crawled Properties mapped to Managed Properties

• Characteristics are defined for Managed Properties, e.g.

– Refiners– Sorting– Queryable– Type

• Definition and mapping done via UI or Powershell

– Not via Central Admin

Admin UISchema CmdLets

Custom Client

Schema Object Model

Schema Service (hosted in IIS)

Property backend

bliss psctrlconfigserve

r

Update ToolsPersistence

Document Processing Pipeline

PropertiesMapper

PropertiesReporter

Update configuration

Alert pipelineof updatedschema

Report discovered crawled properties

Demonstration

Site Collection Scopes

• Similar interface to SSP Scope Management

• Organize by Display Group• Same flexibility for rules

Scope Visibility

• Define Display Groups• Groups Control Visibility on Search Web

Parts

Search Visibility

• Control if a site shows up in search results• Sites are already security trimmed• Example: Search Center Set to “No”

Keywords & Best Bets

• Use query reporting to help define keywords

• Use keywords to provide access to content inaccessible to the crawler

SharePoint Server 2010 Search

Major improvements over MOSS 2007Area New features and benefits MOSS

2007SharePoint

2010

Social Connect with people and expertise• Address-book style name lookup with phonetic matching• Improved expertise matching - mined e-mail inbox data• Social behavior improves search; search improves mySites

Refinement Find information faster• Metadata-based refiners provide drill-down• Related query suggestions show popular queries

Relevance Provide great search results• New ingredients, more experience -> better OOB search• Better wordbreakers in more languages• Social Distance, tagging, click-through influence relevance

Platform Industrial Strength Search at High Scale• Index partitioning -> sub-second latency at 100M items• Full failover support• Connector and BDC enhancements and tooling support

TCO Deploy and Operate with less effort• Virtualization and native 64-bit simplify deployment• Powershell, Admin web parts, SCOM provide easier admin

Extensibility Create powerful applications based on search• Web parts, query OMs are public -> easy to modify• Enhanced query syntax

Q & A

References:

http://www.mssharepointconference.com/Pages/default.aspx

http://www.microsoft.com/downloads/details.aspx?FamilyID=751fa0d1-356c-4002-9c60-d539896c66ce&displaylang=en

Presentation Material:<MNSPUG> slide link

Contact:Shikhar Thapa (sthapa@inetium.com)

A key element to your success.