Use OData to get more from Office & Power BI Mark StaffordProgram ManagerMicrosoft

DEV-B388

Shanghai, China

Help me design easier data access

Here’s the bad news

We have more slides than minutes

Proof that I work for Microsoft

O365 Power BI

Corporate Data Sources

IT Admin

Data Management Gateway

Corporate Data Catalog

Power BI Admin Center

Data Management Portal

SPO team sites, BI Sites, Q&A

Cloud Data Sources

PublicData Catalog

Data Steward

IW (Author)

IW (Consumer)

Manage Data

Publish raw data sources or expose them as OData feeds

Publish queries

Data Refresh

Data Refresh

Power QueryPower PivotPower ViewPower Map

Publish reports & create featured questions, configure scheduled refresh

Import data, Publish queries

Excel,Tablet/Mobile,Browser – Silverlight, HTML5

Consume interactive reports, ask questions

Configure

Import Data

Import Data

Import Data

Import Data

Public Cloud Data Sources

Index

Here’s the good news

We’re going to spend most of our time on demos instead

How we’ll spend our time

So what?0:05

Demo: Import OData in Excel

0:05Demo: Import

OData in Power Query0:10

Demo: POJSEC read-only

0:10Demo: POJSEC mad lib

mail0:10

Customer story: OData HQ

0:15

Tips & tricks0:10

So what?

Source: TechEd NA 2014 Keynote

You need to work with lots of data stored in many different places

OData provides a consistent API for data exchange

Demo time

Eric Jones @erjjonesDylan Vester

@odatahq

+

IoT

WEBMICROSOFT

MOBILE

THIS IS A CLIENTSUCCESSSTORY

WITH POWER QUERY

INDIANASTATEMUSEUM

INDIANASTATEMUSEUM

REALTIME FEEDBACK

WIFI DEVICE ODATA SERVICE POWER QUERY

The solution…

STORYTIMEwhere our journey begins…

VISITOR ENGAGEMENTWENT REAL TIME

DATA GREWAND EVOLVED

CURATORCURIOSITY GREW

MOREWIFI DEVICES

+

EXHIBIT USAGEPATTERNS EMERGED

LEARNED BASED

D A T A

C U R A T O RS&MARKETING ON

EXHIBITSEVOLVEDBASED ON DATA

INNOVATES

INDIANASTATEMUSEUM

with Power Query…

POWER QUERYshow me the how…

[demo time]POWER QUERY

@ODATAHQ #ODATAHQODATAHQ.COM

Demo time

1. Invest some time in learning about HTTP, JSON, etc2. Know where to find the protocol documentation3. Know the difference between OData v3 and v44. Use the reference services at services.odata.org5. Use LinqPad + code gen to learn query format6. Use an existing stack if possible7. Use Fiddler to trace8. Use Fiddler or Postman as “poor-man’s” consoles9. Know when to use gzip and when not to10.Provide feedback to make OData better

10 tips for getting started with OData

Dev. .comOffice

One stop shop for Office Developer Platform

Explore our MSDN Libraryhttp://dev.office.com

Calls to action

Give Feedback Drive our roadmap http://aka.ms/OfficeDevFeedback

Get Answershttp://aka.ms/AskSharePointhttp://aka.ms/AskOffice

Play with our code sampleshttp://aka.ms/OfficeDevSamplesFollow our Patterns & Practiceshttp://aka.ms/OfficeDevPnP

Patterns and practicesProviding App Model Patterns for common Full Trust Code scenarios

30+ Visual Studio projectsCommon scenarios• Branding• Site provisioning• Remote event receivers • Large file support• Taxonomy driven navigation• And much more…

ContributeOpen source coming soon!

aka.ms/OfficeAMS

Related content – BreakoutsOFC-B254 Integrating Yammer and Microsoft SharePoint Using .NETMonday, May 12 1:15 PM - 2:30 PM Room: 350D

DEV-B230 Most Commonly Asked for On-Premises Customizations Reimagined as Applications for SharePointMonday, May 12 3:00 PM - 4:15 PM Room: 350D

DEV-B319 Get Started Developing Applications for Microsoft Office and SharePoint Server 2013 Monday, May 12 4:45 PM - 6:00 PM Room: 332A

DEV-B231 Office Power Hour: New Developer APIs and Features for Applications for Office Tuesday, May 13 8:30 AM - 9:45 AM Room: 381A

DEV-B227 Anyone Can Build a SharePoint Application with Microsoft AccessTuesday, May 13 8:30 AM - 9:45 AM Room: 332A

OFC-B274 Implementing Microsoft SharePoint 2013 Hybrid for Search, Business Connectivity Services, Microsoft OneDrive for Business and YammerTuesday, May 13 1:30 PM - 2:45 PM Room: 360A

DEV-B232 Creating Cloud Hosted Line-of-Business Applications with Apps for Office, Microsoft Office 365, Microsoft Azure, and Windows Phone 8 Wednesday, May 14 8:30 AM - 9:45 AM Room: 340

OFC-B311 A Practical Use of External Data SourcesWednesday, May 14 8:30 AM - 9:45 AM Room: 360A

DEV-B357 Developing Office 365 Cloud Business Applications Wednesday, May 14 5:00 PM - 6:15 PM Room: Hilton L2 Ballrm C

DEV-B387 Deep Dive into Mail Compose Applications APIs Wednesday, May 14 5:00 PM - 6:15 PM Room: 352D

DEV-B386 Setting Up Your On-Premises Environment for App Development Thursday, May 15 10:15 AM - 11:30 AM Room: 320A

DEV-B228 Build Connected Productivity Apps for SharePoint and Office Thursday, May 15 10:15 AM - 11:30 AM Room: Hilton L2 Ballrm C

DEV-B390 SharePoint Power Hour: New Developer APIs and Features for Apps for SharePoint Thursday, May 15 1:00 PM - 2:15 PM Room: 360A

DEV-B389 Who Are You and What Do You Want? Working with OAuth in Microsoft SharePoint 2013 Thursday, May 15 2:45 PM - 4:00 PM Room: 310A

Related content – LabsDEV-H207 Developing Applications for Microsoft SharePoint 2013 with Visual Studio 2013OFC-H308 Building Task Pane Applications for Microsoft Office 2013 OFC-H309 Building Visual Studio-Based Workflows in Microsoft SharePoint 2013 OFC-H310 Building Windows 8 Applications with Microsoft SharePoint 2013 OFC-H311 Business Connectivity Services (BCS) for Microsoft SharePoint 2013

Related content – Exam Prep EXM04 Exam Prep: 70-331 and 70-332 MCSE: SharePoint (Microsoft SharePoint Server 2013)Tuesday, May 13 8:30 AM - 9:45 AM Room: Hilton L2 Ballrm F

Visit the Developer Platform & Tools BoothHaving a friend buy your coffee?Yea, it’s kind of like that.

MSDN Subscribers get up to $150/mo in Azure credits.

Stop by the Developer Platform and Tools booth and visit the MSDN Subscriptions station to activate your benefits and receive a gift!

http://aka.ms/msdn_teched

3 Steps to New Gear! With Application Insights

1. Create a Visual Studio Online account http://visualstudio.com

2. Install Application Insights Tools for Visual Studio Online http://aka.ms/aivsix

3. Come to our booth for a t-shirt and a chance to win!

VSIP QR Tag Contests Visit our booth to join the hunt for cool prizes!

Alex Park

ResourcesMicrosoft Engineering Stories

How Microsoft Builds Softwarehttp://aka.ms/EngineeringStories

Visual Studio Industry Partner Program

Meet Our New Visual Studio Online Partners or Join Now.http://vsipprogram.com

Visual Studio | Integrate

Create Your Own Dev Environmenthttp://integrate.visualstudio.com

Development tools & services for teams of all sizeshttp://www.visualstudio.com

Complete an evaluation and enter to win!

Evaluate this session

Scan this QR code to evaluate this session.

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Appendix

Click icon to add picture

What’s so special about OData?mastaffo@Microsoft.com

OData sales pitch

Standardized format, custom feel

Reduced cost for design & development

Broad reach with existing ecosystem

What does OData enable?ConventionsConsistent patterns to work with data reduces ramp-up timeQuery conventions enable generic clients to compose queries that execute on the server

InteroperabilityDiverse ecosystem of servers, clients and servicesMany of Microsoft’s flagship products have an OData API

Data virtualizationA way to pass queries to an unspecified data sourceServices can “fold queries” through to the data store

OData sweet spotAPIs with ad-hoc queryNo need to design how to query in APIsConsistency means generic clients can build queries

Structured (not necessarily schematized) dataQueries work better over regular structuresData source does not have to be schematized or a relational database

Interoperability requirementsEasy to get your data into Excel, Power BI and other Microsoft productsStandardized protocol, open sourced frameworks

How does OData help Office/Power BI?Get data outBuild custom apps that access data from Exchange, SharePoint, Project, Power BI and more

Get data inPublish custom OData services and data sources through Power BIConsume any OData service in ExcelModify data in Exchange, SharePoint, Project

Get more valueSave time for innovation, not reinventing the wheelDiscover new and unique ways of interacting with your company’s most valuable information

Click icon to add picture

OData overview

An overview of OData conventionsWell-known endpointsOne entry point to the service for hypermedia clientsData model is publicly accessible at <service root>/$metadataPublic endpoint for batch requests

Powerful set of URL conventions for filtering, paging, shaping, and morePayload conventions for simple and consistent serialization

The service document{ @odata.context: "http://.../TripPinService/$metadata", value: [ {name: "People", kind: "EntitySet", url: "People"}, {name: "Airlines", kind: "EntitySet", url: "Airlines"}, {name: "Airports", kind: "EntitySet", url: "Airports"}, {name: "Me", kind: "Singleton", url: "Me"}]}

$metadata<Schema xmlns="http://..." Namespace="TripPin"> <EntityType Name="Airport" BaseType="TripPin.Location"> <Property Name="IcaoCode" Type="Edm.String" Nullable="false"/> <Property Name="IataCode" Type="Edm.String" Nullable="false"/> <Property Name="Address" Type="TripPin.AirportAddress"/> </EntityType> <EnumType Name="PersonGender"> <Member Name="Male" Value="0"/> <Member Name="Female" Value="1"/> <Member Name="Unknown" Value="2"/> </EnumType> <EntityType Name="Person" OpenType="true"> <Key> <PropertyRef Name="UserName"/> </Key> ...

OData URL componentshttp://.../TripPinService/Me/Trips(‘TechEd NA 2014')/Plans/Flight?$top=5&$skip=5

URL path identifies a resourcehttp://.../TripPinService/Me/Trips('TechEd NA 2014')/Plans/Flight

Query string allows selective filtering from a feedhttp://.../TripPinService/Me/Friends?$top=5http://.../TripPinService/Me/Friends?$filter=DisplayName eq 'Miles Rhoden'

OData URL conventions

http://.../TripPinService/

$filterA basic property matchhttp://.../Me/Friends?$filter=FirstName eq 'Scott'Logical operators: eq, ne, gt, ge, lt, le, and, or, notArithmetic operators: add, sub, mul, div, modString functions: contains, startswith, concat, length, substring, trim, etc.Other functions: round, floor, ceiling, year, month, day, isof, etc.

Any/allhttp://.../Me/Friends?$filter=Friends/any(f:f/FirstName eq 'Scott')

Compound conditions (can be arbitrarily complex)http://.../People?$filter=EmailAddresses/any(ea:endswith(ea,'@example.com')) and Trips/any(t:year(t/StartsAt) gt 2014)

$orderbySimple orderinghttp://.../Me/Friends?$orderby=LastName

More advanced orderinghttp://.../Me/Friends?$orderby=Trips/$count desc

Multiple field orderinghttp://.../Me/Friends?$orderby=Trips/$count desc,LastName

$top and $skipAllow client-driven paginghttp://.../Me/Friends/?$top=2&$skip=2

Defaults to ordering by IDServer-driven paging uses nextLink as an opaque token

$select and $expandColumn filtering (not full projections)http://.../Me/Friends/?$select=FirstName,Trips/$count

Inclusion of related entitieshttp://.../Me/?$expand=Friends,Trips

New in OData v4http://.../Me/?$expand=Trips($filter=Plans/Flight/$count ge 3;$orderby=StartsAt desc;$top=10)http://.../Me/?$expand=Friends($levels=2)

Combining URL Conventionshttp://.../Me/Friends

?$top=2&$skip=2&$filter=EmailAddresses/any(ea: endswith(ea,'@contoso.com')&$orderby=EmailAddresses/$count desc&$select=FirstName,LastName,EmailAddresses&$count=true&$format=json

Payload ConventionsSuccess and error codes200 OK, 201 Created, 415 Unsupported Media Type, etc

Overall formatJSONAtomPub

NuancesPATCH, Null payloads, etc

JSON FormatFriendly to JavaScript90% more compact than AtomPubReads like a custom APIVery little OData “gunk”

Sample JSON payload{ @odata.context: "http://odatasampleservices.azurewebsites.net/V4/TripPinService/$metadata#People", value: [ { UserName: "russellwhyte", FirstName: "Russell", LastName: "Whyte", EmailAddress: [ "russell@example.com", "russell@contoso.com"], Gender: "Male" }, { UserName: "scottketchum", FirstName: "Scott", LastName: "Ketchum", EmailAddress: ["scott@example.com"], Gender: "Male " }, ...

What’s so great about conventions?Frameworks can codify interactionsGeneric tooling can produce and consume servicesDevelopers get up to speed fasterPredictability enables a more compact payload

What else can I do with OData?Open types for unschematized data sourcesActions and functions allow RPC-style method invocationFirst-class extensions for aggregation and temporalExtensible with custom annotations (bring your own ontology)

What is Power BI?Discover & combine data with Power QueryModel & analyze data with Power PivotVisualize data with Power View and Power MapShare insights with Power BI sitesGet answers using natural language Q&A

Proof that I work for Microsoft

O365 Power BI

Corporate Data Sources

IT Admin

Data Management Gateway

Corporate Data Catalog

Power BI Admin Center

Data Management Portal

SPO team sites, BI Sites, Q&A

Cloud Data Sources

PublicData Catalog

Data Steward

IW (Author)

IW (Consumer)

Manage Data

Publish raw data sources or expose them as OData feeds

Publish queries

Data Refresh

Data Refresh

Power QueryPower PivotPower ViewPower Map

Publish reports & create featured questions, configure scheduled refresh

Import data, Publish queries

Excel,Tablet/Mobile,Browser – Silverlight, HTML5

Consume interactive reports, ask questions

Configure

Import Data

Import Data

Import Data

Import Data

Public Cloud Data Sources

Index

Data Management GatewayPoint-and-click to create OData feedsSQL ServerOracleOther data sources

Simple way to get corporate data into Power BIEnterprise-grade access control and scalabilityMakes OData feeds searchable in Power Query

SharePoint ♥ ODataSharePoint has been doing OData for yearsSharePoint 2010 exposed list data via ODataSharePoint 2013 exposes much more data via OData

SharePoint OData feeds are read/writeSharePoint OData works best with CSOMSharePoint model uses many actionsNext-gen OData will reduce need for CSOM

Exchange ♥ ODataExchange just announced OData supportOData will replace legacy Exchange Web ServicesFirst major product in the world to publicly support OData v4!

Mail, calendar, contactsConsume and create the most important data types in ExchangeMore to come (hopefully)!

Project Server

Dynamics CRM

Azure Active Directory

Azure Table Storage

PowerShell

…

Other examples

Power QueryVery helpful add-in to ExcelConsumption & mashup of many different data sourcesMakes the ingest process repeatableNot surprisingly, it has great OData support

Microsoft product families that use ODataAzureDynamicsOfficeSQLWindows