Android and Cloud Data

A Streamlined Approach

Laird Dornin

About the Presenter: Laird Dornin • Co-author of two books on Android Development:

• Programming Android from O’Reilly Media

• Enterprise Android from Wiley Media (Wrox)

• Extensive experience in the Mobile development industry:

• Helped build a comprehensive Java based operating system SavaJe

OS, an “Android before Android”

• Ported the WebKit browser library to this OS

• Diverse Backend development experience from Sun Microsystems to

Harvard Law School, Wellington Financial

Two Books about Android:

Books (cont.)

• Programming Android has broken the top 2000 on Amazon.com’s best

seller list.

• Enterprise Android teaches cutting edge Mobile and Scalable Cloud data

concepts

• This presentation roughly follows the organization of Enterprise Android

• If you read Enterprise Android you’ll be able to build dynamic robust

Android applications and scalable cloud services to support those

applications - the book covers an end to end picture of mobile

development.

The State of Android • Full featured modern mobile OS

• Roughy a million applications

• A wide variety of form factors:

• Phones, Tablets, TVs, Toasters

• Over 1 BILLION activations

• The most popular world-wide OS

• 4 times the popularity of iOS

Overview: A Tour of Enterprise

Android • Android MVC

• Networking

• Data management

• Tying it all together: Project-Migrate

• Project-Migrate in Detail

• Cloud Service Development

• What are cloud services?

• REST

• Sync

What is MVC?

• Modern UI frameworks use a pattern called: Model View

Controller (MVC)

• A Model holds the data of an application: (e.g. a contact

type backed by a database)

• A View enables a user to interact with data:

• Examples: An HTML form, an Android layout file.

• A Controller receives user action to implement

application behavior and modify data.

• These components use callbacks to communicate.

• Ch1. Enterprise Android discusses UI in detail.

MVC Supports Dynamic UIs

• Changes in data

automatically propagate to

the view and controller

• The view does not need to

“poll” the model to learn

about modifications.

Relationship between

View, Data, Controller

Android MVC

• Like all modern UI frameworks - follows an MVC architecture:

Digression: Angular MVC

AngularJS is a modern JavaScript MVC

framework that uses HTML as its templating

language: https://angularjs.org/

Create dynamic UIs entirely in html that

automatically bind to JavaScript objects

(models) with extremely simple notation.

Digression: Angular Example <table>

<tr ng-if=”contacts.Length > 0">

<td >Name</td>

<td>Phone Number</td>

</tr>

<tr data-ng-repeat=”contact in contacts">

<td>

<a href="" ng-click=

"selectContact(contact)">

{{contact.Name}}

</a>

</td>

</tr>

</table>

• Adds, Deletes from the contacts list automatically change the size of the contact table

Android Views

• The Activity and Fragment classes provide the

main unit of Android user interfaces.

• View objects

• The Android platform supports high level XML layouts

• It’s possible (and often easy) for developers to build

modern and sophisticated mobile User Interfaces.

• Programming Android and Enterprise Android both

contain detailed tutorials for best practices in UI design

Activity, Fragments Life Cycle

• Activity Life cycle:

• Activities support the following life cycle methods

• Resumed, Started, Paused, Created, Stopped, Destroyed

• Android process life cycle can be tricky

• Activities can span different processes by Intent execution

• Takes some expertise to write a stable application

Programming Android and Enterprise Android Ch1 have examples that show

how life cycle works in practice

Android Life

Cycle in Detail

Contact

List Layout

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" xmlns:tools="http://schemas.android.com/tools"

android:layout_width="match_parent"

android:layout_height="match_parent"

android:orientation="vertical"

tools:context=".ContactsActivity" >

<ListView android:id="@+id/activity_contacts_list"

android:layout_width="wrap_content"

android:layout_height="0dp"

android:layout_weight="1" />

<Button android:id="@+id/activity_contacts_add"

android:layout_width="wrap_content"

android:layout_height="wrap_content"

android:layout_gravity="right"

android:text="@string/contacts_add" />

</LinearLayout>

Android List Activity Example

Android Models • Android applications rely on SQLite as a robust lightweight

database for persistent storage.

• A query on an SQLite database returns its results in a Cursor object (like JDBCResultSet)

• Cursors are observable holders of application data - they

are the model.

• Changes in underlying SQLite data propagate automatically

to Android views using: • Cursor.registerContentObserver(

ContentObserver contentObserver);

Cursor Example A query on an SQLite database returns its results in a Cursor

object (like JDBCResultSet):

Cursor contactCursor =

db.query(contactTable,

new String[]

{ContactsColumn.DISPLAY_NAME},

null, null, null, null, null, null);

String displayName =

contactCursor.getString(displayNameIndex);

Model Objects Optional

Contact {

String getFirstName();

String getLastName();

String getPhoneNumber();

String getEmail();

}

• Android supports full SQL with the widespread SQLite

• SQLite does not lose data in the face of failure

• SQLite supports transactions with ACID semantics for

concurrent access:

• (Atomic, Consistent, Isolated, Durable)

• Android SQLite API visible throughout persistence support

Enterprise Android Ch. 2 provides a highly detailed discussion

of SQLite – Ch. 3 explains how this database integrates with

Android APIs.

SQLite

Android Controllers Java code that registers to listen to Android Views ((Button) findViewById(R.id.activity_contacts_add)).setOnClickListener(

new View.OnClickListener() {

@Override

public void onClick(View v) {

showDetails(getUri(v)); // WARNING: calls on the UI thread must NOT block!

}});

void showDetails(Uri uri) {

if (BuildConfig.DEBUG) { Log.d(TAG, "adding contact"); }

Intent intent = new Intent();

intent.setClass(this, ContactDetailActivity.class);

if (null != uri) {

intent.putExtra(ContactDetailActivity.KEY_URI, uri.toString());

}

startActivity(intent);

}

A Blocked Android Application

Networking

• Network requests enable loading of network data,

usually from RESTful services

• A broken invocation in Android runs on the event thread

• A naive invocation, uses Handler and AsyncTask

REST • REST is RElational State Transfer

• Popular, possibly de facto standard for web APIs

• GET, PUT, POST, DELETE unique resources

• Examples: POST /laird/cars {“toyota”, 2008, “rav4”}

GET /laird/cars/1

Networking: Apache client request

HttpPost req = new HttpPost( uri.toString()); StringEntity s = new StringEntity(payload);

s.setContentType(MIME_JSON);

req.setEntity( s);

req.setHeader(HEADER_ACCEPT, MIME_JSON);

HttpParams httpParams = new BasicHttpParams();

HttpConnectionParams.setConnectionTimeout(httpParams, TIMEOUT);

HttpConnectionParams.setSoTimeout(httpParams, TIMEOUT);

DefaultHttpClient client = new DefaultHttpClient(httpParams);

HttpResponse resp = client.execute( req); Reader in = new InputStreamReader(resp.getEntity(). getContent());

hdlr.handleResponse(in);

Async Task Based Request

A common

(naive) scenario,

but what if the

OS interrupts a

long running

request?

Content Provider

● An interprocess data service that wraps a SQLite

database

● Content provider methods are RESTLike: query(uri), insert(uri),

update(uri), delete(uri)

● Query returns a cursor just like a database query.

● Most Android services use content providers.

● Ch. 4: Enterprise Android

Recall Android MVC:

Cursor queryCursor =

getContentResolver().query(queryUri, ...);

Can you make a REST service look

like a content provider?

• Yes, write your own content provider that

delegates to a RESTful service

• REST, Content Provider APIs both CRUD

• Ch.13 of Programming Android: “Content

Provider as facade for a RESTful web service.”

• Ch. 5 of Enterprise Android adds sync adapter

Why write a content provider this way?

• Protect UI thread

• Insulate app from network instability

• Transparent caching

• Capitalize on Android idioms, API, MVC

• Use the network efficiently

A Content Provider as a Facade for a

RESTful service

Content Provider as Cache for

Network Data

Synchronization

• Batched integration of changes between two parties

• Add changes since last sync into local database

• Enables offline editing

• Conflicts are possible

• Easy to run over HTTP, but not RESTful

• A natural fit for Sync adapters and Google cloud

messaging

Sync Adapters • The place for synchronized network communication -

possibly all network communication • onPerformSync provides placeholder for custom

sync protocols

• Works well with Google Cloud Messaging and “send-to-

sync”

• Send messages to listening devices to avoid polling

• Ch. 5: Enterprise Android

Sync Adapter + Content Provider

More Android Networking Patterns

See Virgil Dobjanschi on YouTube:

http:// www.youtube.com/ watch?v=xHXn3Kg2IQE

http://bit.ly/UYnhnn

Optimistic Sync based on Time

Deltas

• Clients track all locally “dirty” contacts

• On sync with service, send modifications

• Conflicting changes rejected

• Client resends after resolution

• Server sends all changes after last sync,

relative to server time

• Ch. 5, 6: Enterprise Android synchronized

contacts example, service and sync adapter

REST and mobile applications

Simple

Robust

Many libraries

Widely applicable

Same architecture as Web apps

Same implementation skills as Web apps

REST client libraries are good for…

Simple RPC-like use of a REST API

Populating a UI on-demand

E.g. fetch search results

What's not to like?

• Incremental access to data

• Relies on good connectivity

• High performance can be complex to code

and difficult to reach

• Doesn’t use ContentProvider-based

persistence and MVC idioms supported in

Android

Incremental access to data

• What's the problem?

• Inappropriate use of incremental access

means the user waits for every access

• Unavoidable in some cases, e.g. search

results

• But, many use cases can use synchronized

data

Sync Vs REST

• REST usually requires polling to know when

resources are up to date

• Sync can merge offline changes

• Concurrent Data editing on multiple devices

requires conflict resolution not typically

found with REST

Sync vs REST (cont)

• Sync avoids polling by using notifications

• Only send aggregate changes (e.g. if client

edits the same data multiple times)

• Generic networking layer

• Can every request fit into a Sync paradigm?

Requires some thought…

OK, why isn’t everyone sync’ing?

Some apps, like email and PIM apps,

benefitted from sync built-in to protocols

IMAP for email

CardDAV and SyncML for contacts

ActiveSync for Windows PIM and email data

Google Play services leverage sync heavily

Is it too hard to work with?

Email and PIM apps benefit from

sync’ed data

Read and write email anywhere

Access calendar data anywhere

Access contact data anywhere

Good characteristics for mobile apps

Why email and PIM apps?

A big head start

Email is store-and-forward

PIM apps defined the pre-iPhone “smart

mobile device” e.g. Symbian, Windows

Phone

They were sync’ing before REST was even a

glimmer in W3C’s eye

But wait! Isn’t part of being RESTful

being cacheable?

• Cache improves REST performance by

reducing redundant downloads

• Conceptually similar to caching in Web

browsers

• Only works in one direction, does not deal

with conflicts

• Not a substitute for sync

Writing a Scalable backend service

• What is the cloud? • You don’t know where or how many servers will run

your backend service.

• An Amorphous “cloud” of compute resources.

• Too many choices: • Java, Ruby, Python, Go?

• AWS, Google App Engine, Mongo DB (JavaScript)?

• Ch. 6, 7: Enterprise Android

Single Database as Bottleneck

Horizontal Scaling

Avoid Hotspots in Key Distribution

Popular Cloud Service Providers

• AWS (EC2, Dynamo DB)

• Google App Engine (Big Table)

• Mongo DB

• All scale sufficiently for most apps. Pick

based on price, flexibility

• Ch. 8 Enterprise Android: AWS and App

Engine examples

Alot to learn...

• It would be easier if app developers did not

have to worry about this stuff:

• MVC, content provider, sync adapter, rest

vs. sync, cloud scalability.

• Can we make it easy?

Project-Migrate Simplifies App

Development

Current Project-Migrate Features

• Generic Content Provider and Sync Adapter

• Generic Back-end Sync service

• Schema Injection

• Conflict Resolution Support

• Conflict Resolution User Interface

• Simple web based UI for editing schema

• Generic Browser Application

• Sample Contacts Application

• A long wish list…

Its easy to try Project-Migrate:

• Chapter 10 of Enterprise Android

• Download the SDK: https://github.com/wileyenterpriseandroid/migrate/wiki

http://goo.gl/eXfKFt

http://projectmigrate.com/

• Post a backend schema to define client,

service Communication

Using the Project-Migrate SDK Register or Login to a

ProjectMigrate Cloud

Service

Settings ->

Add Account

Configure ProjectMigrate

SyncAdapter

Using the Project-Migrate SDK (cont) Make sure to activate

Sync

Select Project-Migrate

Service Endpoint

Account Added

View/Edit Schema Data Before you

have an App

Develop the App

• Most of the work in developing the app should

be in writing the UI

• The Project-Migrate SDK includes tools for

authoring and POSTing your schema

• Its possible to log into the Project-Migrate

service and edit schema using a simple web

interface

Project-Migrate implementation details

• Chapter 9 of Enterprise Android

• Generic Data Synchronization

• How does the system solve common data

problems in a generic way?

WebData Protocol

• Use schema to define data

• Uses time-based synchronization deltas

• Supports generic object storage on client, cloud

• Lets look at an example Contacts application

Post Schema Request

Post Schema Data Fields (cont)

WebData tracking

fields

Specialized Contacts Application

Create contact Edit contact fields, submit

POST A New Contact

GET a Contact

Example Conflict Resolution

Two clients have edited the same contact, Mark

Johnson, between syncs with the service

Contact has id: a7329678-4bdc-536a-b234-78236431026a

Each client changes the contact phone number: Client 1: 781-201-4567

Client 2: 781-223-1234

Listings of the Same Contacts Generic Browser Listing Contact App Listing

Two Clients Edit Same Contact

Contacts Client Resolves Conflict

Select Change, Resolve

Browser Shows Version Update

Prior to resolve

version is 4

With resolve,

increments to 5

Conflict Resolution Protocol

Client Sends Changes in Sync

Server Sync Response

Client Resolution Sync

Points to Note about the WebData

Protocol

• Detection of changed items is quick – based on

simple time compare

• Relies on optimistic concurrency – client attempts

sync and service easily detects conflicts

• The protocol is robust, if there is an error in

transit, client can simply repeat sync with no fear

of duplicates

• Light-weight protocol

Future Directions for Project-

Migrate? • Automated schema evolution

• Digression: Do for data what angular does

for the UI – make it declarative

• Digression: Integrate Project-Migrate models

with Angular.

• Would be nice to have the same generic API

across all platforms…

Summary

This presentation provided introductions to: • Android MVC

• Handling Data with Content Providers and REST

Services

• Service Development

• Tying it together with Project-Migrate

• Project-Migrate in detail

Laird Dornin

ldornin@gmail.com

Download the SDK:

http://goo.gl/eXfKFt

http://projectmigrate.com/