1

AgendaAgenda

Data and System TrendsData and System Trends

ChallengesChallenges

ObservationsObservations

ConclusionConclusion

2

19731973Winchester Winchester

DriveDrive

The Big TrendsThe Big Trends

1986198650 50

MegabyteMegabytes s

198219823 ½ Floppy Disk3 ½ Floppy Disk

19831983CD ROMsCD ROMs

19671967First Floppy DiskFirst Floppy Disk

197219725 ¼ Floppy5 ¼ Floppy

StorageStorage

1984198410 MB HD10 MB HD

2000’s2000’sTerabytesTerabytes

1990’s1990’sGigabytesGigabytes

20052005

3

The Big TrendsThe Big Trends

Digitally Born DataDigitally Born Data

20052005

4

Increased Data Increased Data Under ManagementUnder Management

MasterMasterFilesFiles

SalesSalesDataData

IWIWFilesFiles PicturesPictures

MusicMusic

Product Product Design Design

DataData

ConversationsConversations

MasterMasterTapesTapes

1960’s1960’s 1970’s1970’s 1980’s1980’s 1990’s1990’s 2000’s2000’s

ElectronicElectronicTradesTrades

GISGIS

LocationsLocations

5

TransactionTransactionProcessingProcessing

OLTPOLTP

InternetInternet

XMLXML

ReportingReporting

DataDataProcessingProcessing

1960’s1960’s 1970’s1970’s 1980’s1980’s 1990’s1990’s 2000’s2000’s

CommunicationsCommunications

Reengineering IT: Reengineering IT: EfficiencyEfficiencyWhere have the gains come from?Where have the gains come from?

B2B ServicesB2B Services

Knowledge RetentionKnowledge Retentionand Retrievaland Retrieval

Reduce LatencyReduce LatencyIn Human ProcessesIn Human Processes

6

Data Moving UpstreamData Moving Upstream

Post Post TransactionalTransactional

SummarySummaryAfter the After the factfact

TransactionalTransactional

Record of factRecord of factEnables Enables AnalyticsAnalytics

Pre-Pre-TransactionalTransactional

Record of Record of fact and fact and intentintentEnables Enables deeper deeper insightinsight

7

The Data MonolithThe Data Monolith

Previous 30 years Previous 30 years trended towards trended towards data consolidationdata consolidation

SOA, connected SOA, connected systems, common systems, common schema, pulling schema, pulling data apartdata apart

Data ProcessingData Processing

OLTPOLTP

EAEA

InternetInternet

B2B ServicesB2B Services

DevicesDevices

SOASOA

Dat

a C

onve

rgen

ce

Dat

a C

onve

rgen

ce

Data D

istribution

Data D

istribution

8

Trends Recap…Trends Recap…Huge Gains in PSIHuge Gains in PSI

ProcessingProcessingStorageStorageI/OI/O

Exponential Improvement in all of Exponential Improvement in all of the abovethe aboveData now online and available with Data now online and available with schemaschema

Need to derive value from itNeed to derive value from it

Human costs now dominate Human costs now dominate solutions!solutions!The data monolith is getting broken The data monolith is getting broken apartapart

9

ChallengesChallenges

Data Challenges:Data Challenges:ProgrammingProgramming

DistributingDistributing

Getting ValueGetting Value

Relating and OrganizingRelating and Organizing

SharingSharing

10

Challenge: Programming Challenge: Programming DataDataDifferent EnvironmentsDifferent Environments

Data query environment distinct from Data query environment distinct from rest of programrest of program

No Intellisense, late bound, verbose…No Intellisense, late bound, verbose…

T-SQL versus .NET LanguagesT-SQL versus .NET Languages

““Impedance Mismatch” between Impedance Mismatch” between code and databasecode and database

TransactionsTransactions

Nulls - 3-value logicNulls - 3-value logic

Normalized DataNormalized Data

Declarative QueriesDeclarative Queries

TransparencyTransparency

Exception HandlingException Handling

““Different” nullsDifferent” nulls

ObjectsObjects

Imperative operationsImperative operations

EncapsulationEncapsulation

Database WorldDatabase World Programming WorldProgramming World

The LINQ ProjectThe LINQ Project

StandardStandardQueryQuery

OperatorsOperators

ObjectsObjects

DLinqDLinq(ADO.NET)(ADO.NET)

XLinqXLinq(System.Xml)(System.Xml)

<book> <title/> <author/> <year/> <price/></book>

XMLXML

.NET Language Integrated Query.NET Language Integrated Query

C#C# VBVB Others…Others…

SQLSQL WinFWinFSS

TLN306 – Wed, 1:45PMTLN306 – Wed, 1:45PMDAT323 – Thu, 2:15PMDAT323 – Thu, 2:15PMDAT324 – Fri, 10:30AMDAT324 – Fri, 10:30AM

12

SqlConnection c = new SqlConnection(…);SqlConnection c = new SqlConnection(…); c.Open(); c.Open(); SqlCommand cmd = new SqlCommand(SqlCommand cmd = new SqlCommand( @"SELECT c.Name, c.Phone@"SELECT c.Name, c.Phone

FROM Customers cFROM Customers c WHERE c.City = @p0"WHERE c.City = @p0"

);); cmd.Parameters.AddWithValue("@po", cmd.Parameters.AddWithValue("@po", ""LondonLondon""); ); DataReader dr = c.Execute(cmd); DataReader dr = c.Execute(cmd); while (dr.Read()) {while (dr.Read()) { string name = dr.GetString(0);string name = dr.GetString(0); string phone = dr.GetString(1);string phone = dr.GetString(1); DateTime date = dr.GetDateTime(2);DateTime date = dr.GetDateTime(2); }} dr.Close();dr.Close();

Data Access Data Access in APIs Todayin APIs Today

Queries in Queries in quotesquotes

Arguments Arguments loosely loosely boundbound

Results Results loosely typedloosely typed

Compiler Compiler cannot help cannot help

catch catch mistakesmistakes

13

Data Access Data Access in DLinqin DLinq

public class Customerpublic class Customer{{ public int Id;public int Id; public string Name;public string Name; public string Phone;public string Phone; … …}}

Table<Customer> customers = db.Customers;Table<Customer> customers = db.Customers;

var contacts =var contacts = from c in customersfrom c in customers where c.City == "London"where c.City == "London" select new { c.Name, c.Phone };select new { c.Name, c.Phone };

Classes Classes describe describe

datadataTables are Tables are collectionscollections

Query is Query is natural part natural part

of the of the languagelanguage

The compiler The compiler helps you helps you

outout

14

ArchitectureArchitecture

LINQ Query Objects SubmitChanges()

SQL Query Rows SQL or Stored Procs

DLinq(ADO.NET)

SQLServer

from c in db.Customersfrom c in db.Customerswhere c.City == "London"where c.City == "London"selectselect new { c.Name, c.Phone } new { c.Name, c.Phone }

select Name, Phoneselect Name, Phonefrom customersfrom customerswhere city = 'London'where city = 'London'

Application

Services:Services:- Change tracking- Change tracking- Concurrency control- Concurrency control- Object identity- Object identity

15

Hello World “DLinq” StyleHello World “DLinq” Style

Dinesh KulkarniDinesh KulkarniProgram ManagerProgram ManagerC#C#

16

ADO.NET vNextADO.NET vNext

Pablo CastroPablo CastroProgram ManagerProgram ManagerDataworksDataworks

17

Challenge: Distributing Challenge: Distributing DataData

Need data everywhereNeed data everywherePDA → Desktop → App Server → EnterprisePDA → Desktop → App Server → Enterprise

Need synchronization/replicationNeed synchronization/replication

Need reliable messaging between Need reliable messaging between storesstores

Need reference data everywhereNeed reference data everywhereContactsContacts

CatalogsCatalogs

Price listsPrice lists

18

Challenge: Value From Challenge: Value From DataData

Search versus QuerySearch versus Query

Information from existing dataInformation from existing dataAnalyticsAnalytics

ReportingReporting

Managing the data lifecycleManaging the data lifecycle

19

Search Versus QuerySearch Versus QueryShow pictures of Show pictures of

people who attended people who attended Eric’sEric’s

2004 Birthday Party2004 Birthday Party

QueryQuery

SearchSearch

Eric Eric andand Birthday Birthday

20

PortalPortal(Office Server)(Office Server)

Integrated OfferingIntegrated OfferingOfficeOffice

SQLSQL

Data WarehouseData Warehouse(SQL RDBMS)(SQL RDBMS)

IntegrateIntegrate(SSIS)(SSIS)

AnalyzeAnalyze(SSAS)(SSAS)

ReportReport(SSRS)(SSRS)

Business ScorecardingBusiness Scorecarding(BSM)(BSM)

End-user AnalysisEnd-user Analysis(Excel + Excel (Excel + Excel

Server)Server)

BIBIPlatformPlatform

AnalyticAnalyticApplicationsApplications

End-userEnd-userToolsTools

DAT315 – Thu, 10:00AMDAT315 – Thu, 10:00AMOFF 323 – Fri, 10:30AMOFF 323 – Fri, 10:30AM

21

Challenge: Exploiting Challenge: Exploiting Rich DataRich Data

How to get more value from rich dataHow to get more value from rich data

AsAsOffice documents content more Office documents content more accessibleaccessible

New XML formats for Office “12”New XML formats for Office “12”

More intrinsic metadata becomes More intrinsic metadata becomes availableavailable

We enable data sharingWe enable data sharing

22

Challenges: Exploiting Rich Challenges: Exploiting Rich DataData

UnifyUnifyIt’s not just about filesIt’s not just about files

OrganizeOrganizeIt’s not just about folders (or tables)It’s not just about folders (or tables)

ExploreExploreIt’s not just about searchIt’s not just about search

InnovateInnovateIt’s about a platformIt’s about a platform

23

DAT 209 – Wed, 1:45PMDAT 209 – Wed, 1:45PMDAT 209 – Thu, 5:15PMDAT 209 – Thu, 5:15PMDAT 310 – Wed, 3:15PMDAT 310 – Wed, 3:15PMDAT 312 – Wed, 5:00PMDAT 312 – Wed, 5:00PMADO.NETADO.NET

WinFSWinFS

Item Data Model

Query

Services

File Services Metadata Handlers,…

OperationsBackup, AV,…

Item Association Extension

Schemas

Tasks Media

Docs …

SyncAdapters, …

People Mail

File Services Metadata Handlers,…

OperationsBackup, AV,…

SyncAdapters, …

Update

Logic

“DLinq”

Unified APIUnified APIADO.NET support – query, update, app ADO.NET support – query, update, app logic, language integrated query, …logic, language integrated query, …

It’s Still a File SystemIt’s Still a File SystemFS Semantics, Security, Manageability, FS Semantics, Security, Manageability, Backwards Compatible with Win32 Backwards Compatible with Win32

OperationsOperationsBackup/Restore, AV, …Backup/Restore, AV, …

SyncSyncMulti-master SyncMulti-master Sync

New Windows TypesNew Windows Types “ “Everyday info” schemas + ISV Everyday info” schemas + ISV extensionsextensions

Structured “Item” Store Structured “Item” Store Files & Folders Files & Folders Items & Associations Items & Associations

Integrated storage componentsIntegrated storage componentsLeverage investments in DB Leverage investments in DB technologies, CLR, NTFS supporttechnologies, CLR, NTFS support

Unified APIUnified APIADO.NET support – query, update, app ADO.NET support – query, update, app logic, language integrated query, …logic, language integrated query, …

It’s Still a File SystemIt’s Still a File SystemFS Semantics, Security, Manageability, FS Semantics, Security, Manageability, Backwards Compatible with Win32 Backwards Compatible with Win32

OperationsOperationsBackup/Restore, AV, …Backup/Restore, AV, …

SyncSyncMulti-master SyncMulti-master Sync

New Windows TypesNew Windows Types “ “Everyday info” schemas + ISV Everyday info” schemas + ISV extensionsextensions

Structured “Item” Store Structured “Item” Store Files & Folders Files & Folders Items & Associations Items & Associations

Integrated storage componentsIntegrated storage componentsLeverage investments in DB Leverage investments in DB technologies, CLR, NTFS supporttechnologies, CLR, NTFS support

What is WinFS?What is WinFS?

File Services Metadata Handlers,…

OperationsBackup, AV,…

SyncAdapters, …

Database Engine

NTFS CLR

24

WinFSWinFS

Shishir MehrotraShishir MehrotraLead Program ManagerLead Program ManagerWinFSWinFS

25

ConclusionConclusion

Data Management value moving -Data Management value moving -From: base services – store, query, From: base services – store, query, retainretain

To: semantic services – mine, analyze, To: semantic services – mine, analyze, repurposerepurpose

We can make it We can make it muchmuch easier to easier to program against dataprogram against data

We will make it much easier for you We will make it much easier for you to:to:

Interact with dataInteract with data

Program against dataProgram against data

Derive value from dataDerive value from data

26

© 2005 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.