The Case For AtomNot All REST APIs Are CreatedEqual

DRAFT

Officeof theChief

Scientist

Murray Spork, Juergen SchmerderSeptember 23, 2009Version 1

An Investigation: Atom-basedAPIs For The Enterprise

Using Atom With ERP andGoogle Spreadsheets

Officeof theChief

Scientist

Murray Spork, Juergen SchmerderOctober 6, 2009Version 1

2

Agenda

Background (5 mins)

little bit of history

Atom adopters

Atom overview (5 mins)

Conceptual model

Some examples

ERP and Atom (5 mins)

ERP and GData (5 mins)

Lessons learned (2 mins)

Open Issues (3 mins)

3

The (not-so) Hidden Agenda:Evangelizing Support for Atom Within SAP

Atom as a standard platform access mechanism

ReST as the next “Tower of Babel”?

Instead - leverage network effects by adopting an existing and widely

supported protocol

Atom is in already in our ecosystem: Microsoft, IBM, Google, Oracle ....

Atom as exemplar for REST best practices

Understanding ReST can be difficult

Sometimes the benefits appear too abstract

- “What the heck is hypermedia as the engine of application state”??

ReSTless POX-over-HTTP misses many of the benefits of the Web Architecture

Atom embodies many ReST best practices

How Did We Get Here?

5

A Quiz To Start

Which of these technologies got its start in the world of Blogging?

1.RSS/ Atom

2.XML-RPC

3.SOAP

6

A Quiz To Start

Which of these technologies got its start in the world of Blogging?

1.RSS/ Atom

2.XML-RPC

3.SOAP

“Like XML-RPC and SOAP before, feeds andpublishing protocols were born in the blogopshere

and quickly moved beyond blogging.”

Answer = All of the above!

7

Where Did The Idea of Atom as a GeneralPurpose Web Syndication Format Come From?

Blog Syndication was a mess:at least 4 different formats (RSS)poor extensibilitymany ambiguities... let’s clean it up!

Atom Atom Pub

“If we do a better job at separatingform from content intermediariescan do a lot more with feeds”--> Aggregators--> Filters--> Mashups--> repurpose content

“Feeds are basically lists of things...what other kinds of information canwe share via feeds besides blogs?”

“SOAP seems too heavy-weightfor the Web - but do I have todefine my own protocol forsharing data over HTTP?”

Gdata (Google)Lotus (IBM)Live Mesh (Microsoft)… many others

8

Why Is Atom Used in APIs?

So what’s the big idea?- Most structured data can naturally exposed as simple lists of things

(i.e. collections of entries)- Collections may contain other nested collections- Entries (within a collection) can link to other entries or other collections- Entries (Resources) can represent anything: documents, business objects,

events, processes…- Atom can do all of this in a “web-friendly” way (URLs for everything)- Alternative representations of the same resource is possible (HTML, JSON, XML,

media files…)

9

Atom Feeds: From Web 2.0 to Business Apps

Europa build system (Eclipse)

Yahoo Pipes

Google Docs

Web 2.0 Enterprise

Producer

Consumer

Live Mesh Lotus Connections

CICS

WebCenter

Bitsy & AtomDB

Blog Servers

Feed Readers

Del.ic.ious

Browsers

Outlook

Open Cloud

Sharepoint

Open Gov - eGov.org

10

Google Data (Gdata):A Family of Atom-based REST APIs

The APIs that Google currently offer based on Atom and Atom Publishing Protocol

Source: Google Data Protocol http://code.google.com/apis/gdata/

Atom Data Model

12

What is Atom?

IETF Atom WG Charter: “Atom defines a feed format for representing and a protocol for editing Web

resources such as Weblogs, online journals, Wikis and similar content”

Things to note: A syndication format: RFC-4287 (published December 2005) A publishing protocol: RFC 5023 (published October 2007) A “Web resource” can represent anything

13

Feed Data Model

Source:Beyond Blogging: Feeds in Action, Dave Johnson, 2007 Java One Conference

14

Collection

Workspace

Service Document for Feed Discovery

Service

Workspace

1..*

‣ Discovery - from a “well knownlocation”‣ Service Document structures

Feeds (= Collections) intoWorkspaces

Collection1..*

<?xml version="1.0" encoding='utf-8'?> <service xmlns="http://www.w3.org/2007/app" xmlns:atom="http://www.w3.org/2005/Atom"> <workspace> <atom:title>Address Book of User SCHMERDER</atom:title> <collection href="http://nspah223.pal.sap.corp:50000/sap/sruby_rack/sinatra/address_book/contacts" > <atom:title>My Contacts</atom:title> </collection> </workspace> </service>

Atom and Enterprise Software

16

Atom and Enterprise Software - Scenarios

Scenario 1: Enterprise Application as Atom Publisher Worklists (POWL) as Atom Feeds Monitoring / Log Feeds that show recorded events & exceptions “Business Objects” (BOR, AP, …) as Atom Collections and Entries *

⇒ Display data on any device that supports Atom/RSS

Scenario 2: Enterprise Application as Atom Consumer Feed SAP data into non-SAP applications (and load it back in) *

Google Docs River / Coghead

Scenario 3: Enterprise Application as Atom Publisher & Consumer Peer-to-peer data synchronization Distributed Data Management

FeedSync PubSubHubBub

* = Demo

Demo 1 - Atom Feeds and ERP

18

Demo Scenario - Contact Feed

Disclaimer: We used Blue Ruby (a Ruby VM running on the ABAP Server) tobuild the demo scenarios. This was for rapid prototyping purposesonly. Everything we show could be built in plain ABAP.

Address book feed powered by SAP Business Partner:Contact persons for the current user (= BusinessPartnersof type “Person” that have a Relationship of type “IsContact Person for” to a BusinessPartner mapped to SY-UNAME) are exposed as Atom Feed

The feed can be shown in any client that supports AtomClients can publish data to the feed

Demo 2 - Google Spreadsheets and ERP

20

Demo Scenario - Contact Data Cleansing

A simple Data Cleansing Application using a Google Spreadsheet Contact data (the feed from Demo 1) is uploaded to a Google Spreadsheet Data is maintained in the Google Spreadsheed Changed data is pulled back into the ABAP backend

Records that have not changed are ignored (using Etags) Data is not re-imported if it has changed in the backend (using Timestamps)

Extra: Workflow linkage for event BusinessPartner.changed updates spreadsheetwhen data changes in backend

Upload contact data

Update changed data

Download changed data

Atom Lessons Learned

22

Lessons 1: ERP is a Rich Source for Feeds!

Most Documents (we often call them “Business Objects”) fit the Atom paradigmsof collections and entries

Collections are lists of objects, Entries are single objects or parts of objects It’s always been good practice to record creation/change times and users

A lot of events are recorded as documents in the ABAP Backend Syndication and Aggregation of different resources could help end-users

(e.g. Universal Work List as aggregation of Object Work Lists) Publish / Subscribe totally underused Etags and Timestamps allow for optimistic locking

Plus: many more examples here @ REST Summit 2009 If you have any more ideas - please talk to us!

23

Lessons 2: “Designing protocols is hard”

When Building Your REST API - Did You Think About.....

‣ encoding & escaping‣ content-negotiation‣ empty v not-present‣ required v optional‣ extensions & new formats‣ timestamps‣ bi-directional‣ caching & versioning

....didn't think so ;).

Sources:1. Bill deHora, “Snowflake APIs”, http://www.dehora.net/journal/2009/01/09/snowflake-apis/

24

Lessons 3: Links in Content is Good!

Atom is modular - at a minimum you should consider using atom:link inyour APIs (there are examples of APIs that do this - can we find any?)

<entry gd:etag='"YDwqeyI."'> <id>http://spreadsheets.google.com/feeds/worksheets/key/private/full/worksheetId</id> <updated>2006-11-17T18:23:45.173Z</updated> <title type="text">Sheet1</title> <content type="text">Sheet1</content> <link rel="http://schemas.google.com/spreadsheets/2006#listfeed" type="application/atom+xml" href="http://spreadsheets.google.com/feeds/list/key/worksheetId/private/full"/> <link rel="http://schemas.google.com/spreadsheets/2006#cellsfeed" type="application/atom+xml" href="http://spreadsheets.google.com/feeds/cells/key/worksheetId/private/full"/> <link rel="self" type="application/atom+xml" href="http://spreadsheets.google.com/feeds/worksheets/key/private/full/worksheetId"/> <link rel="edit" type="application/atom+xml" href="http://spreadsheets.google.com/feeds/worksheets/key/private/full/worksheetId/version"/> <gs:rowCount>100</gs:rowCount> <gs:colCount>20</gs:colCount> </entry>

Atom Open Issues

26

Atom is Far From Perfect or Even Complete

Missing from Atom Filtering/ Searching/ Paging - use OpenSearch protocol Support for hierarchies

– In-lining extensions for Atom (draft proposal from Oracle)– Atom threading extensions– Microformats– Hierarchy IDs

Handling partial updates POST or PATCH? Diff/delta format needed?

Best practices for incorporating foreign markup and other data inside Atom feed As children of atom:entry (Gdata model) As children of atom:content with media type text/xml Define custom media type Microformats

How to handle batch updates Adopt the google batch model?

Transaction/sync models Is optimistic concurrency good enough? (we think so - for most things) Can we make it easier to resolve conflicts?

– Microsofts FeedSync spec. How do we tell clients how to add or update entries?

What replaces html:form? Xform, RDF-Forms, WADL …?

27

But It Still Gets You 80% of the Way There

Gdata and the 80/20 Rule

“Google has definitively provided a simple but powerful enough API foraccessing their services. They do not want to solve 100% of all use casesbut rather provide a simple and uniform API for the majority (80%)”

Our takeaway:

Even considering the various open questions, Atom is good enough for alot of things already today.

Final Thoughts

29

Adam Bosworth Was One of the EarliestVoices Pushing ATOM for Business Data

Adam Bosworth in 2005 - Lessons of last 10 years:

Open up your front end to wire formats and data that are easy open andextensible (create network effects)

There is a serious change in computing happening - similar to the advent of theweb - around data

We need a standard simple protocol for data feeds

It needs to be sloppy (“democratize data”)

Needs to support updates

Has to scale - scale needs “stupidity” - complex things break - simple thingswork

We have and answer in RSS 2.0/Atom - does for data what HTML did for content

Source: 2005 Keynote at MySQL

Who Are We?

31

Murray SporkOffice ofChief Scientist(now in IP&NW- Solution Management)

Murray joined SAP Research in 2003 where hehas lead several projects in diverse areas suchas model-driven engineering, Semantic Web,Web 2.0, introducing scripting languages intothe ABAP stack and network-enabled multi-enterprise business apps. He is passionateabout any technology that facilitatescollaboration and decentralization andempowers “innovation at the edges” of anorganization.

EMail: murray.spork@sap.com

Contact

Juergen SchmerderOffice ofChief Scientist

Juergen joined SAP in 1999 and worked invarious development projects in CRM,Netweaver and SAP Business ByDesign - bothin ABAP and Java. In 2007 he moved to PaloAlto, California, where he joined the Office ofthe CTO to lead the research project BlueRuby - a Ruby VM, implemented in ABAP. Heworks part time as a project manager, parttime as an evangelist for scripting languagesand part time as a developer.

Email: juergen.schmerder@sap.com

32

About the Office of the Chief Scientist

The Office of the Chief Scientist (oCS) isresponsible for assuring SAP's awareness andplanning with respect to critical technologies,especially those that are externally driven fromindustry, academic institutions and customers. Thegroup, led by SAP Chief Scientist, Ike Nassi, is part ofthe Office of the CTO (oCTO). oCS is not onlyexpected to think, project, and experiment, but also togrow deep technical competencies ensuring anongoing ability to keep SAP at the forefront of theindustry.

The four main areas of work performed by oCSinclude: Knowledge and Technology Scouting

Technology Vision and Guidance

Engineering Services

Experimentation and Prototyping

For more information on the group and its work:http://ocs.pal.sap.corp:1080