Pragmatic REST APIs

Andre Mesarovic7 March 2013

Overview

• Basics of REST

• Beyond bare-bones REST, what additional features are necessary for a robust API?– Data model

– Error handling

– Paging

– Querying

– Batch/Bulk

• API Manageability– Security

– Rate limiting

– Analytics

– Monitoring

Types of APIs

• Private Internal API– Within the firewall

– API provider and consumer own both ends of pipe

• Partner API– B2B integration such as BML and Reporting

– Limited access and users

• Public API– Open to entire internet

– API managability is major concern

• A private API can evolve into a public API

• An API may be both private and partner

APIS are Everywhere

Roy Fielding

• “Father” of REST

• Co-author of HTTP spec

• PHD thesis describes original REST vision

• Thesis is very theoretical with little practical guidance

• Fielding speaks in very abstract and obtuse language

• No Fielding-sponsored REST reference implementation

• Is there any REST API that Fielding approves of?

• Hypermedia is important

API Definition

• What is an API?

• Application Programmering Interface == Service

• REST API is a service comprised of service operations rooted by a base URL

• Service operation == resource URL + HTTP method– GET rps/api/profile/{ID}

– PUT rps/api/profile/{ID}

• An API is not a single service operation

• Not all APIs are RESTful. There are SOAP, POX APIs, etc.

REST API• What is a REST API? What actually comprises a REST contract?• An API is a collection of resources and HTTP methods rooted under a

common base URL• Resource/Method contract:

– Resource URL– HTTP Method: GET, POST, PUT or DELETE– Input

• Query parameters• Request body definition• Request headers such as accept, token, etc.

– Output• HTTP status code• Response body. Two possible variants:

– Happy path where we return the expected business object– Unhappy path where the body is empty or returns an error object

• Response headers

• Data Format– How do define our data formats for the request and response bodies?– RPS has rich and complex data model for profiles, identities, and frequency

capping– For JSON discussion, see JSON Schema below

Web Services Taxonomy

http://www.crummy.com/writing/RESTful-Web-Services/

Non-REST-ful APIs

• Not all successful APIs are “RESTful”

• Case in point: Amazon EC2 Query

• Can’t argue with success ;)

• eBay API

• Google AdWords: still rocking with SOAP! https://developers.google.com/adwords/api/

Richardson REST Maturity Model

• Maturity Heuristic by Restful Web Services book author– http://www.crummy.com/writing/speaking/2008-QCon/act3.html

– http://martinfowler.com/articles/richardsonMaturityModel.html

• Level 0 : HTTP Tunneling– One URI, one HTTP method

– XML-RPC and SOAP

• Level 1: Resources– Many URIs, one HTTP method

– Flickr, del.icio.us, Amazon's ECS

• Level 2: HTTP – Many URIs each with many HTTP methods

– Amazon S3

• Level 3: Hypermedia controls– HATEOS and web linking

REST vs SOAP (RPC)

• REST is a style, SOAP is a standard

• REST has a fixed set of verbs and infinite number of resources

• SOAP has infinite number of verbs and one endpoint

• REST limited verb paradigm is analagous to SQL limited verbs

• REST presents a radically different programming model than classical procedural style that programmers are familiar with

• REST is a Kuhn-ian paradigm shift

APIs of Note and Interest

• Google

• Amazon

• eBay

• Twitter

• Netflix

• Facebook

• OpenStack

• Stripe API

• Balanced API

Avoid REST-ful Tail Chasing

• Business needs can be much more complex than HTTP/REST model• Avoid tail-chasing REST purity debates and be pragmatic• At the same time do not undermine core HTTP/REST tenets

HTTP Basics

• Uniform Interface– Resource URL

– Content-type of both request and response representation

– HTTP method/verb

• REST services have an infinite number of resources (endpoint) and only four methods

• RPC services have one endpoint and an infinite number of operations

• Resources represent things or objects

• Major distinction between resource and representation

• A resource can have one or more representations (JSON, XML, Atom, binary)

• Representation selection is governed by HTTP content negotiation using standard accept and content-type headers

Idempotence and Safety

• RFC Spec - Safe and Idempotent Methods -http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html

• Safe Methods– No side effects

– Methods: GET and HEAD

• Idempotent Methods– Result for 1+ identical requests is the same as for a single request

– Methods: GET, HEAD, PUT and DELETE

HTTP Methods

Method URL Template Note

GET api/profile/{ID} Returns resource representation.

POST api/profile Creates new resource. New URL is returned in Location header. Response body is optional.Often used for business operations not easily mapped other HTTP methods.

PUT api/profile/{ID} Updates or creates new resource. Response body isoptional.

DELETE api/profile/{ID} Deletes resource.

HEAD Same as GET but no body is returned.

OPTIONS Information on what methods are supported.

HTTP Method Basics

• Resource URL

• The four standard HTTP methods are roughly analogous to SQL CRUD operations• GET – select

• POST – insert

• PUT – upsert - update or insert

• DELETE - delete

• Not all complex business requirements can easily be translated into the REST paradigm.

• Avoid breaking the HTTP/REST protocol

• For complex operations, noun-ify the intended operation and create a resource.

• If all else fails, use POST. For example for RPS, the complexity involving identity linking probably requires a POST.

Canonical Resource Pattern

Method URL Template Note

GET api/profile?{QP} Collective GET. Query parameters to select data. Returns list of matching resources.

GET api/profile/{ID} Returns resource representation.

POST api/profile Creates new resource. New URL is returned in Location header. Response body is optional.

PUT api/profile/{ID} Updates or creates new resource. Response body isoptional.

DELETE api/profile/{ID} Deletes resource.

GET

• Returns the representation of a resource

• GET never modifies the state

• GET is both safe and idempotent

• This is not OK: GET flickr/api/photo?method=delete

• Two types of GET resources:

– Singleton GET• Retrieves one resource specified by last path component (ID) in the URL

• Query parameters can be used to select a partial representation

• Example: rps/api/profile/71dc982-0ac3-4aad-8747-9e0a7d763e04

– Collection GET• Retrieves a collection of resources filtered by query parameters

• Example: rps/api/profile?keys=ps:def;es1=qwerty

• Can either be links (by reference) or expanded full data

GET Gotchas

• Input to GET is specified in query parameters

• Heretical question: any inherent reason GET could not support a request body?

• Why cram everything into a limited structure such as QP?

• Accidental complexity: server/browser limitations on QP size

• Non-trivial structured complex queries are awkward to represent in QP

• Google BigQuery– Analytics queries on big data

– Uses SQL dialect for queries and POST

– Asynchronous and asynchronous queries

• Subbu Recipe 8.3: How to Support Query Requests with Large Inputs

POST

• Creates a new resource and returns system-generated ID

• Request body contains new data

• Returns the new resource URL in Location header

• Response body can optionally return a more expanded version of the new resource

• POST is not idempotent. Repeating the operation will result in a new resource

• Often used for “tunneling” protocols on top of HTTP

PUT

• Two purposes:– Update a resource by ID

– Creates a new resource if you want to specify the ID

• Request body contains new data. Response body can optionally return a more expanded version of the new resource.

• PUT is idempotent: repeatedly calling PUT on same resource will yield the same result.

PUT - Partial Updates

• PUT is often mistakenly thought of solely as an update

• Not quite analagous to SQL update

• Since PUT specifies resource ID, then it can be a create if resource does not exist

• Therefore PUT is an upsert

• PUT only as update– Some services only permit server-generated IDs

– In this case PUT can only be an update

– PUT as create is disallowed by application semantics• Mismatch between application needs and HTTP/REST constraints

• What to return as error code? 404?

• Two types of updates:– Overwrite existing resource with input

– Merge input with existing resource – PATCH

PATCH HTTP Method

• PATCH Method for HTTP: https://tools.ietf.org/html/rfc5789

• AbstractSeveral applications extending the Hypertext Transfer Protocol (HTTP) require a feature to do partial resource modification. The existing HTTP PUT method only allows a complete replacement of a document. This proposal adds a new HTTP method, PATCH, to modify an existing HTTP resource.

DELETE

• Delete a resource by ID

• Note that typically a 404 will not be returned if the resource doesn’t exist since most data stores do not return this information.

• DELETE is idempotent

Data Access

• How do we decide what granularity of data to return?

• Purist REST approach can result in chatty service. Major issue with older Netflix API.

• Two dimensions: which objects and which properties (scalar or collections)

• Selection: Which objects to return?

• Projection: Which properties to return?

• Do collections get represented by reference or by value?

– Client-specified or fixed by server?

• Partial Responses

• Object graphs – nested collections: both selection and projection needed

Querying

• Querying syntax - how do we do complex boolean searches (AND, OR, NOT)?

• Query syntax for GET and potentially for POST when we do profile merges

• Some current practices:

– Google GData Queries

– OpenSearch

– YogaWiki

• Yoga framework for REST-like partial resource access -Vambenepe - July 2011

– FIQL: The Feed Item Query Language - Dec. 2007

– Google Base Data API vs. Astoria: Two Approaches to SQL-like Queries in a RESTful Protocol - Obasanjo - 2007 -

Paging

• Feed Paging and Archiving: RFC 5005– http://www.ietf.org/rfc/rfc5005.txt

• Paged feed documents MUST have at least one of these link relations present:– "first" - A URI that refers to the furthest preceding document in a series

of documents.

– "last" - A URI that refers to the furthest following document in a series of documents.

– "previous" - A URI that refers to the immediately preceding document in a series of documents.

– "next" - A URI that refers to the immediately following document in a series of documents.

• Other paging links of interest to an application– Count – items in current page

– MaxCount – items in entire result – optional since this can often be very expensive or impossible for the server to implement

RFC 5005 Paging Example

<feed xmlns="http://www.w3.org/2005/Atom">

<title>ExampleFeed</title>

<linkhref="http://example.org/"/>

<linkrel="self" href="http://example.org/index.atom"/>

<linkrel="next" href="http://example.org/index.atom?page=2"/>

<updated>2003-12-13T18:30:02Z</updated>

<author>

<name>JohnDoe</name>

</author>

<id>urn:uuid:60a76c80-d399-11d9-b93C-0003939e0af6</id>

<entry>

<title>Atom-PoweredRobots Run Amok</title>

<linkhref="http://example.org/2003/12/13/atom03"/>

<id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>

<updated>2003-12-13T18:30:02Z</updated>

<summary>Sometext.</summary>

</entry>

</feed>

Paging Examples

• Google + Pagination example:– maxResults

– nextPageToken

– pageToken

• CM2– page

– results_per_page

– total_results fields

• Instagram{ ...

"pagination": {

"next_url": "https://api.instagram.com/v1/tags/puppy/media/recent?access_token=qwerty&max_id=13872296",

"next_max_id": "13872296"

}

}

Query Parameters

• Naming convention – be consistent

• Choices:– Underscores: max_count

– Dashes – max-count (youtube)

– Camel case: maxCount – preferred

• Share same query parameters across methods

• Query parameter validation:– HTTP status code

– Application error code

– Type constraints – integer, string

– Required constraints

– Complex format constraints

Data Modeling

• How to model a large number of objects with complex operations and constraints?

• JSON is currently popular but has no clear modeling conventions or standards

• XML has a number of schemas: XSD, Schematron, RelaxNG

• XML also has a large number of ontologies

Data Contract

• How do specify our data contract in an unambiguous manner?

• In SOAP/WSDL world, for the better or worse, XSD rules

• New fashion is to use JSON: easier to read, less noise

• But JSON lacks the rigor of a canonical contract

• OK for simple objects

• Things get complicated with types, structures and references

• An object can have multiple valid representations and one example does not describe all these variants

• Mere instances are often ambiguous

• Lack of machine-readable contract leads to ad-hoc validation logic buried inside application code

• Nice to have: independent validators

Content Negotiation – Conneg

• HTTP feature to specify different representation of resource

• Request Accept header is used to specify list of desired MIME types

• Example: Accept: application/json; q=1.0, application/xml;

• Analagous response header is content-type

• Conneg is a unique feature of HTTP/REST having no analogues in classical programming paradigms

• Conneg also extends to language dimension: Accept-Languageheader

• Very powerful concept that can be leveraged in different ways

• Mime-types can also be used for fine-grained versioning

• Example: application/vnd.acme.rps-v2+json

Google JSON Specification• Not all Google API data formats use same data specification

• Some Google APIs support only XML, others only JSON and others both

• Several conventions for describing JSON objects

• Google initially solely used instance examples to specify data formats

• Current Google data specification styles:– Mere examples

– Structured “ad-hoc”

– JSON Schema

Google Structured JSON Specification Samplehttps://developers.google.com/+/api/latest/people#resource

{"kind": "plus#person","etag": etag,"emails": [ {

"value": string,"type": string,"primary": boolean

}],"urls": [ {

"value": string,"type": string,"primary": boolean

}],"objectType": string,"id": string,"name": {"formatted": string,"familyName": string,

},"tagline": string,"relationshipStatus": string,"url": string

}

JSON Schema

• JSON Schema Home- http://json-schema.org

• The latest IETF published draft is v3 (Nov. 2010): http://tools.ietf.org/html/draft-zyp-json-schema-03

• Abstract:JSON (JavaScript Object Notation) Schema defines the media type "application/schema+json", a JSON based format for defining the structure of JSON data. JSON Schema provides a contract for what JSON data is required for a given application and how to interact with it. JSON Schema is intended to define validation, documentation, hyperlink navigation, and interaction control of JSON data.

• Multiple sub-specifications:– JSON Schema Core: defines basic foundation of JSON Schema

– JSON Schema Validation: defines validation keywords of JSON Schema

– JSON Hyper-Schema: defines hyper-media keywords of JSON Schema

JSON Schema Overview

• Human and machine readable data contract specification

• Type constraints

• Structural validation

• Optionality

• Ignored fields – unlike XSD

• Qualified refined constraints

• Multiple language validator implementations: Java, Ruby, Python, .NET, JavaScript

• Google Discovery uses JSON Schema

• How does it play with JAXB validations?

JSON Structure Specification• How do we specify the format of our JSON objects?

– How are data types defined?– How are cardinalities defined?– How is optionality defined?

• Typically JSON examples of payloads are provided with ad-hoc text documentation

• The problem is that the documentation is non-standard and it is very difficult to define a contract solely from examples.

• Problem is that the data validation is split between the code and the documentation. It is very easy for these to get out of sync. In addition, if a client wants to validate objects, then this validation code is duplicated from the server.

• Validation is buried in the code and will differ between serves and clients (multiple languages).

• Investigate the applicability of JSON schema– New standard in progress– I have done some hacking around and it seems promising– JAX-RS JSON validation uses JAXB which is based on XML XSD schema– It is very easy to get into semantic mismatch anomalies, e.g. a XSD sequence is

ordered but JSON has no concept of ordered lists.

JSON Schema – Standard Schemas

• Geographic Coordinate: a location as longitude and latitude

• Card: a microformat-style representation of a person, company, organization, or place

• Calendar: a microformat-style representation of an event

• Address: a microformat-style representation of a street address

JSON Schema – Standard Schemas – DetailsGeographic Coordinate

{"description": "A geographical coordinate","type": "object","properties": {

"latitude": { "type": "number" },"longitude": { "type": "number" }

}}

Address{

"description": "An Address following the convention of http://microformats.org/wiki/hcard","type": "object","properties": {

"post-office-box": { "type": "string" },"extended-address": { "type": "string" },"street-address": { "type": "string" },"locality":{ "type": "string", "required": true },"region": { "type": "string", "required": true },"postal-code": { "type": "string" },"country-name": { "type": "string", "required": true}

},"dependencies": {

"post-office-box": "street-address","extended-address": "street-address"

JSON Schema – Google Discovery Schema{"id": "Url","type": "object","properties": {

"analytics": {"$ref": "AnalyticsSummary",

}, "id": {

"type": "string","description": "..."

},"kind": {"type": "string","description": "...","default": "urlshortener#url"

},"longUrl": {

"type": "string","description": "..."

}}

}

Instance Sample{

"kind": "urlshortener#url","id": "http://goo.gl/fbsS","longUrl": "http://www.google.com/"

}

JSON Schema – Payments

{

"type":"object",

"$schema":"http://json-schema.org/draft-03/hyper-schema",

"name":"Address",

"id":"address:v1",

"properties":{

"line1":{ "type":"string", "required":true },

"line2":{ "type":"string", "required":false },

"city":{ "type":"string", "required":true },

"country_code":{ "type":"string", "required":true },

"postal_code":{ "type":"string", "required":false },

"state":{ "type":"string", "required":true },

"phone":{ "type":"string", "format":"phone", "required":false }

}

}

JSON Schema - Store

{

"description" : "Store JSON Schema",

"type" : "object",

"properties" : {

"name" : { "type" : "string", "required" : true },

"phone" : { "type" : "string", "required" : true },

"isCheckPermanent" : { "type" : "boolean", "required" : true },

"location" : { "$ref": "Location.json", "required" : true }

},

"additionalProperties" : false

}

JSON Schema – Location {

"description" : "Geographic Location","type" : "object","properties" : {

"lat" : {"title" : "Latitude","required" : "true","type" : "number","minimum" : -90.00,"maximum" : 90.00

},"lng" : {

"title" : "Longitude","required" : "true","type" : "number","minimum" : -180.00,"maximum" : 180.00

}},"additionalProperties" : false

}

JSON Schema - Address

{

"description" : "Address JSON Schema",

"type" : "object",

"properties" : {

"streetLine1" : { "type" : "string", "required" : true },

"streetLine2" : { "type" : "string", "required" : true },

"neighborhood" : { "type" : "string", "required" : false },

"city" : { "type" : "string", "required" : true },

"state" : { "type" : "string", "required" : true },

"country" : { "type" : "string", "required" : true },

"zip" : { "type" : "string", "required" : true }

},

"additionalProperties" : false

}

Error Handling

• Familiarize yourself with RFC 2616 Status codes:– http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10

• Necessary but not sufficient: every response must return an appropriate HTTP status code

• The HTTP spec is ambiguous and not sufficient for complex REST APIs

• HTTP status code can be too coarse-grained

• Often application-specific error is also needed

• Two options (not necessarily mututally exclusive):

– Return an error object in response body

– Return custom headers

• Response body is much more common

• Implementation note: results in polymorphic response which complicates both server and client code

• Never return 200 when an error has happened

Error Handling – Reuse Possibilities

• It would be nice to standardize format of the error message

• Any inherent reason for arbitrary plethora of error formats?

• Perfect example of accidental complexity

• Reuse idea:– Create langugage-specific artifacts with error-handling client logic –

Java JARs, Ruby Gems, etc.

– Need developer buy-in

– Need some governance structure

– Need to create an independent “commons” group responsible for development and maintenance of artifact lifecycle

– Reuse is great – minimizes DRY but introduces coupling

Sample Errors – Facebook

Facebook{

"type": "OAuthException",

"message": "Authenticate"

}

Facebook Batch[

{ "code": 403,

"headers": [

{"name":"WWW-Authenticate", "value":"OAuth…"},

{"name":"Content-Type", "value":"text/javascript; charset=UTF-8"} ],

"body": "{\"error\":{\"type\":\"OAuthException\", … }}"

}

]

SOAP Fault – Nice StandardXSD Schema

<xs:complexType name="Fault" final="extension">

<xs:sequence>

<xs:element name="faultcode" type="xs:QName"/>

<xs:element name="faultstring" type="xs:string"/>

<xs:element name="faultactor" type="xs:anyURI" minOccurs="0"/>

<xs:element name="detail" type="tns:detail" minOccurs="0"/>

</xs:sequence>

</xs:complexType>

Instance Example

<SOAP-ENV:Envelope

<SOAP-ENV:Body>

<SOAP-ENV:Fault>

<faultcode xsi:type="xsd:string">SOAP-ENV:Client</faultcode>

<faultstring xsi:type="xsd:string">

Missing key value

</faultstring>

</SOAP-ENV:Fault>

</SOAP-ENV:Body>

</SOAP-ENV:Envelope>

JAX-RS Error Handling Recomendations

• JAX-RS layer should be thin pass-through layer that delegates all business logic to the injected service object

• JAX-RS layer only re-packages input/output of service layer

• No explicit exception catching

• Exception are handled by JAX-RS exception mappers

• Exceptions bubble up and are appropriately mapped to HTTP status codes and application errors

• Service layer is responsible for all business logic and exceptions

• HTTP-specific distinction between client 4xx and server 5xx introduces new complexity into lower layers

• Abstraction leakage – inevitable engineering trade-offs ;)

RPS Application Errors

public enum RpsApplicationError {RpsDao_PutError,RpsDao_GetError,RpsDao_DeleteError,RpsDao_ServerBusy,RpsService_NoValidKeyPassed,RpsService_PubIdForPubUserMissing,RpsService_MultipleKeyValuesForSameKeyNotAllowed,RpsService_NullValuesForKeyNotAllowed,RpsService_ProviderPrefixNotFoundInSegmentName,RpsService_SegmentNamePostProviderPrefixCannotBeNull,RpsService_ProfileNotFound,RpsService_MultipleProfilesFound,RpsService_ProfileIdRequired,RpsService_InvalidProviderCodeInSegmentName,RpsService_ProfileDifferent

}

Batch/Bulk Operations

• Naïve REST can lead to inefficient chatty protocol

• Create a resource that can operate on several resources

• If we need to add a million objects to API, it would be better to chunk the calls rather than make a million calls

• Can lead to some interesting response status formats

• Subu recipe 11.13: How to Support Batch Requests

Asynchronous Operations

• HTTP status code 202– The request has been accepted for processing, but the processing has not

been completed. The request might or might not eventually be acted upon, as it might be disallowed when processing actually takes place.

• Client does not want to block for long-running server tasks

• Subbu Recipe 1.10: How to use POST for Asynchronous Tasks– Return a 202 and a URL that client can query for status

– When client queries status URL three possibilities:• Still processing - 200 and respone with status object

• Success - 303 and Location header with new resource URL

• Failure - 200 and response with error object

• Can also use 202 for DELETE or PUT

Atom

• Atom Syndication Format - http://tools.ietf.org/html/rfc4287Atom is an XML-based document format that describes lists of related information known as "feeds". Feeds are composed of a number of items, known as "entries", each with an extensible set of attached metadata. For example, each entry has a title.

The primary use case that Atom addresses is the syndication of Web content such as weblogs and news headlines to Web sites as well as directly to user agents.

• Atom is a better alternative to the plethora of RSS variants

• Feed metaphor – many applications can work with this model –but many cannot

• Google has leveraged Atom for Youtube API

• Atom data model has powerful extension mechanism that allows you to embed your own schemas

AtomPub

• Atom Publishing Protocol - http://www.ietf.org/rfc/rfc5023.txtApplication-level protocol for publishing and editing Web resources. The protocol is based on HTTP transfer of Atom-formatted representations. The Atom format is documented in the Atom Syndication Format

• Enhances read-only AtomPub with mutable operations: PUT, DELETE

• Is XML-based as Atom is

• Definitely worth understanding – less acceptance than Atom

• Gregorio: Atom Publishing Protocol is a failure:– http://bitworking.org/news/425/atompub-is-a-failure

• Obasanjo: Joe Gregorio on why the AtomPub is a failure:– http://www.25hoursaday.com/weblog/2009/04/18/JoeGregorioOnWhyT

heAtomPublishingProtocolAtomPubIsAFailure.aspx

Web Linking

• Web Linking RFC: http://tools.ietf.org/html/rfc5988

• Links should not be specific to content-type

• Enumerate link relations

• Relative links:– Full URLs can lead to bloated payloads

– Alternative is to use relative URLs plus a base URL

• Bill Burke: Link headers vs. Custom header– bill.burkecentral.com/2009/10/14/link-headers-vs-custom-headers

• JSON Link Initiatives:– JSON-HAL

– Collection+JSON

HATEOS -

• Hypermedia as the Engine of Application State

• Fielding: REST APIs must be hypertext-driven– http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven

• Decouple client from server endpoints

• HTML with links is prime example– Generic browser engine understands and follows links

• Instead of many public fixed resources have few well-known endpoints

• Clients call these resources and then follow links describing known relations

• Is HATEOS applicable to all business problems?

• Shifts complexity to client links processor

• Bookmarking doesn’t work

Versioning and Extensibility

• Versioning– Not specific to REST but REST introduces new wrinkles

– Backward Compatability

– What part of the REST contract does version refer to?• Data model?

• URLs?

• Semantic meaning changes of existing URL

• Extensibility– How an we extend parts of the contract without requiring versioning?

– Certain changes (additive) do not break contract• For example, a new resource is OK

• Removing a resource breaks client expectations

– In Atom there well-known strategies to do this

– JSON hasn’t arrived there yet

Versioning • Important to think about versioning up-front.• Should develop a versioning strategy or at least be aware of the thorny issues

surrounding REST versioning.• One common way is to version the entire API by using a version identifier in

the URL.– But this is coarse-grained, since one minor change may require rolling out a new

version– Requires support of multiple APIS and sunsetting strategy

• What features comprise a new version?– Removing a resource does.– Changing the data format or query parameter contract?

• Once an API is public with many clients it has, change is difficult.• Use content negotiation to specify data format versions?• Need to develop a version life-cycle strategy to sunset API versions.• API extensibility. Extending the API does not require a new version. Backward

compatible changes such as:– Adding a new resource.– Adding optional query parameters– Adding optional JSON properties in data payloads

• Curated REST Versioning links

Client

• Raw HTTP vs client libraries• JAX-RS 2.0 introduces client spec• By far most developers interact with client libraries and not

the raw API• The service definition is the REST API contract, but users

interact through client packages• Does API provider support language-specific clients?• By definition we will already have a Java-specific client that

we use for our extensive tests, so with minimal effort we can make it public.

• We might want to provide a few certified client libraries for popular languages, e.g. Java.

• Investigate annotation-based client generators – see CXF or RESTEasy.

• Can also use WADL to auto-generate client stubs.

GData - Google Data Protocol

• REST-based protocol for reading, writing, and modifying data

• Internal Google technology

• Recently “deprecated” – not clear why – Apparently Google is favoring Google APIs Discovery

• Supports Atom and JSON

• Serves as base data model for many Google APIs– https://developers.google.com/gdata/docs/directory

– Youtube API: https://developers.google.com/youtube/

OData

• Web “protocol” for querying and updating data

• Microsoft-initiated and submitted to OASIS

• Similar in nature to GData but is an open spec

• OData builds on top of AtomPub

• Focus is on rich API data access to a rich data objects

• Spec is an extremely difficult read

• Adds a whole new conceptual layer on top of REST:– EDM - Entity Data Model

– CSDL - Conceptual Schema Definition Language

• Some implementations– eBay OData: http://ebayodata.cloudapp.net/docs

– Netflix OData: http://developer.netflix.com/docs/oData_Catalog

Security

• Authentication

• Authorization

• Oauth 2.0

• User management and provisions

• Depends on API type: private, partner or public

API Manageability

• Develop infrastructure for common API needs– Metrics

– Rate Limiting

– Security

– Monitoring

– Analytics

• Solution is either a proxy gateway or a plugin

• Apigee provides turnkey system that can be hosted by client

API Manageability Vendors

• Apigee

• Mashability

• 3scale

• Programmable Web

• API Evangelist

Documentation

• How do we document our API?

• Important to keep documentation in sync with the actual API

• Some ideas– Google APIs Discovery Service

• Awesomeif Google open-source metadata mechanism to autogenerate documentation!

– API Consoles

– WADL + XSLT, Enunciate

• Without machine readable contract, documentation is the contract

• As API grows, it is difficult to consistently document API manually

• Conversely, how do we add non-trivial business logic to document generated from code?

Documentation Tools

• WADL - https://wadl.java.net/

• Swagger - http://swagger.io/

• API Blueprint - http://apiblueprint.org/

• Enunciate - http://enunciate.codehaus.org/

• Tools to generate beautiful web API documentation

WADL

• Web Application Description Language

• Machine-readable XML description of API

• With XSLT can generate HTML documentation

• Clients libraries can be automatically generated from XML

• Light-weight REST analogue of WSDL

• Java REST vendors auto-generate WADL from JAX-RS annotations– Requires editing Java files to update documentation!

– Also for richer documentation requires JAX-RS provider-specific annotations

• Doesn’t address payload description very well

WADL Example<resource path="/profile">

<method name="GET"><request>

<param name="keys" style="query" type="xs:string"/></request><response>

<representation mediaType="application/json"/></response>

</method><method name="POST">

<request><representation mediaType="application/json"/>

</request><response>

<representation mediaType="application/json"/></response>

</method><resource path="/search">

<method name="POST"><request>

<representation mediaType="application/json"/></request><response>

<representation mediaType="application/json"/></response>

</method></resource>

Google APIs Discovery Service

• Exciting new API management API from Google

• Exposed machine-readable metadata about Google APIs

• Uses JSON Schema!

• API console explorer

• Documentation for resources and reprentations

• Use for client libraries, IDE plugins

• Would be great if it was open-sourced

• How does Google implement the service internally?– Is metadata added to source?

– What languages are supported

– What is the tooling?

API Aggregation

• Service composition

• Applications and APIs need to access several back-end APIs

• ql-io – Declarative, data-retrieval and aggregation gateway for quickly consuming HTTP APIs– Open source developed at eBay

– SQL and JSON based DSL

– SQL-like aggregation syntax atop multiple APIs

– http://www.slideshare.net/sallamar/qlio-consuming-http-at-scale

Books

• RESTful Web Services Cookbook - Subbu Allamaraju - Feb. 2010 -Excellent collection of pragmatic recipes by eBay's own (but then of Yahoo) - a must read for any REST API developer

• RESTful Web Services – Ruby, Richardson - 2007 – Classic book on REST

• RESTful Java with JAX-RS - O'Reilly - Burke - author of. RESTEasy -2009. Good exposition of JAX-RS

• APIs: A Strategy Guide - Jacobson of Netflix - O'Reilly - Business perspective - 2011