google back to front: from gears to app engine and beyond

Google Back to FrontFrom Gears to App Engine, and beyond

Interesting, passionate, times

The Web is the platform of today, and of the future

WeBelieve...

Act OneThe Client

Top Grossing Film of 1957 Top Grossing Film of 2007

“Gimme Offline!”

It takes too long to update the Web

What is the philosophy?Do for offline what XMLHttpRequest did for web apps

Open Web

XMLHttpRequest

Ajax LibrariesDojo, jQuery, Prototype, GWT

Open Web

Gears

Gears LibrariesDojo Offline, GWT

DatabaseEmbedded using SQLite

Contributed Full Text Search

var db = google.gears.factory.create('beta.database', '1.0');db.open('database-demo');

db.execute('create table if not exists Demo (Phrase varchar(255), Timestamp int)');db.execute('insert into Demo values (?, ?)', [phrase, currTime]);

GearsDBAbstract over the API

var bob = {id: 3, name: 'Bob', url: 'http://bob.com', description: 'whee'};db.insertRow('person', bob);db.insertRow('person', bob, 'name = ?', ['Bob']);

db.selectAll('select * from person', null, function(person) { document.getElementById('selectAll').innerHTML += ' ' + person.name;});

var person = db.selectRow('person', 'id = 1');

// updateperson.name = 'Harry';db.updateRow('person', person);person = db.selectRow('person', 'id = 1');

// forceperson.name = 'Sally';db.forceRow('person', person);person = db.selectRow('person', 'id = 1');

db.deleteRow('person', bob);

GearsORMAre we going to get a GearsHibernate?

var Person = new GearsOrm.Model("Person", { firstName: GearsOrm.Fields.String({maxLength:25}), lastName: GearsOrm.Fields.String({maxLength:25})

});

GearsORM.Transaction(function() { new Person({name:"John"}).save(); new Person({name:"Doe"}).save();

});

Person.select("firstName = 'Uriel'");Person.count("lastName = ?",["Katz"])

“While developing transaction support for GearsORM i had to write a test, in that test it

execute 100 inserts and 100 updates, this test take about 15 seconds for the inserts and about 10

seconds for the updates without transactions,when using transactions for each set it takes about

377ms for the inserts and 200ms for the updates that is about

39 times faster!”

GearShiftDB Migrations for Gears

Gearshift.rules[1] = { // create the demo table up: function() { return this.e("CREATE TABLE demo_table ( id INTEGER PRIMARY KEY AUTOINCREMENT, name VARCHAR(30), movie VARCHAR(30) )").success; }, down: function() { return this.e("DROP TABLE demo_table").success; }};

Database Tools

Buxfer

Full Text Search•Gears added FTS2 to SQLite

•Create the databasedb.execute('CREATE VIRTUAL TABLE recipe USING fts2(dish, ingredients)');

•Search the databasedb.execute('SELECT dish FROM recipe WHERE recipe MATCH ?', ['tomatoes']);

Fun queries: dish:stew tomatoes Find rows with 'stew' in the dish field, and 'tomatoes' in any field.

The Digg Oracle

Local ServerA mini-web server that groks 200 and 304

ResourceStoreManually Capturing

var pageFiles = [ location.pathname, 'gears_base.js', '../scripts/gears_db.js', ‘foo.html’];

try { localServer = google.gears.factory.create('beta.localserver', '1.0');} catch (e) { alert('Could not create local server: ' + e.message); return;}

var store = localServer.openStore(this.storeName) || localServer.createStore(this.storeName);

store.capture(pageFiles, function(url, success, captureId) { console.log(url + ' capture ' + (success ? 'succeeded' : 'failed'));});

ManagedResourceStoreJSON Me

{ // version of the manifest file format "betaManifestVersion": 1,

// version of the set of resources described in this manifest file "version": "my_version_string",

// optional // If the store specifies a requiredCookie, when a request would hit // an entry contained in the manifest except the requiredCookie is // not present, the local server responds with a redirect to this URL. "redirectUrl": "login.html",

// URLs to be cached (URLs are given relative to the manifest URL) "entries": [ { "url": "main.html", "src": "main_offline.html" }, { "url": ".", "redirect": "main.html" }, { "url": "main.js" } { "url": "formHandler.html", "ignoreQuery": true }, ]}

HTML 5Offline in General

<html application=”manifest-of-urls.txt”>

<html application>“There’s a concept of an application cache. An application cache is a groupof resources, the group being identified by a URI (which typically happensto resolve to a manifest). Resources in a cache are either top-level ornot; top-level resources are those that are HTML or XML and when parsedwith scripting disabled have with the value ofthe attribute pointing to the same URI as identifies the cache.

When you visit a page you first check to see if you have that page in acache as a known top-level page.”

HTML 5 Gears

• Standards• Long term• All browsers• No plugin

• Implementation• Battle hardened• A place for innovation• Cross browser• Plugin

Past

Present

Future

A bleeding edge version of HTML 5!

Operating System

Event Queue

Mouse Moved

Mouse Pressed

Mouse Released

Key Pressed

Key Released

JavaScript Web Browsing

Potential

Bottleneck

Browser

User Interface

WorkerPool

Browser

1

WorkerPool

2 3

Message Passing

Worker Pool Code

function nextPrime(n) { // TODO: New top-secret prime-finding algorithm goes here. google.gears.workerPool.sendMessage(result);}

var pool = google.gears.factory.create('beta.workerpool', '1.0');pool.onmessage = function(message) { alert('next prime is: ' + message);}

var worker = pool.createWorker(String(nextPrime) + '; nextPrime()');

JavaScript Read Write APIs

DesktopShortcuts

var desktop = google.gears.factory.create('beta.desktop');desktop.createShortcut("Test Application", "An application at http://www.test.com/index.html", "http://www.test.com/index.html", {"16x16": "http://www.test.com/icon16x16.png", "32x32": "http://www.test.com/icon32x32.png", "48x48": "http://www.test.com/icon48x48.png", "128x128": "http://www.test.com/icon128x128.png"});

Notification APIGrowl for the Web

var notifier = google.gears.factory.create('beta.notifier', '1.0'); notifier.notify({ application: "My App", title: 'warning', description: 'some text', priority: 2, sticky: 'True', password: 'Really Secure',});

Location APIEven on a laptop?

// Getting the objectvar location = google.gears.factory.create( "beta.location", "1.0" ); // Setting up a callback to handle "location changed" eventslocation.onlocationstatechanged = function() { if (this.state == COMPLETE) { SetStatusText("Location accuracy:", this.accuracy); MoveMap(this.latitude, this.longitude); }}

location.startLocationUpdates();

Audio APIPlay and Record

// playvar audio = google.gears.factory.create('beta.audio');audio.src = 'http://blahblahblob.com/sampleaudio.wav';audio.load();audio.play();

// recordvar recorder = google.gears.factory.create('beta.audiorecorder');recorder.destination = <http post url>recorder.autoStream = true;recorder.record(); //asynchronous call

Don’t you want a better File Upload?

Resumable HTTP

Blob API

File System API

Gears: Open Source Update Mechanism for the Web

Don't forget, RIA's have rich internet back-ends (RIBs?)

Jonathan SchwartzCEO, Sun Microsystems

Act TwoThe Server

Google App EngineRunning Web Apps on Google’s Infrastructure

• Fully-integratedapplication environment

• Language agnostic runtimePython for now

• Free quota of 5Mpageviews per month

code.google.com/appengine

Google App EngineChallenges

Google App EngineEasy to use, Easy to scale, Free to start

Develop locally. Deploy to Google. Launch.

Develop locally. Deploy to Google. Launch.

Deploy

Develop locally. Deploy to Google. Launch.

Develop locally.

• Google App Engine SDK is open source

• Simulates production environment

•dev_appserver.py myapp•appcfg.py update myapp

# helloworld.py print "Content-Type: text/html" print print "Hello, world!"

# app.yaml application: dalmaer-helloworld version: 1 runtime: python api_version: 1 handlers: - url: /.* script: helloworld.py

Hello WorldSimplest output, simplest config

Web Services

loadcontacts

var contacts = [ { id: 1, name: ‘Dion Almaer’, ... }, { id: 2, name: ‘Ben Galbraith’, ... },]savecontact

name=’Dion A Lamer’email=’dion@almaer.com’

the snake is almost there

threadssockets

filesforeground

from django import v0_96 as django

Wow, that's abig table!

Store dataScalable, and not like a RDBMS

from google.appengine.ext import db

class Story(db.Model): title = db.StringProperty() body = db.TextProperty(required=True) author = db.UserProperty(required=True) created = db.DateTimeProperty(auto_now_add=True) rating = db.RatingProperty()

# Many other types: BlobProperty, ReferenceProperty, EmailProperty, PhoneProperty, IMProperty, PostallAddressProperty, etc.

Data ModelJust an object

SELECT *FROM StoryWHERE title = 'App Engine Launch'AND author = :current_userAND rating >= 10ORDER BY rating, created DESC

GQLOur query language

Story.gql(GQL_FROM_BEFORE, current_user = users.get_current_user())

query = Story.all()

query.filter('title =', 'Foo') .order('-date') .ancestor(key)

Run the queriesBeyond GQL

story = Story( title = 'Morning Glory', body = '....', author = users.get_current_user(),)

story.put() # save or update

story.delete() # delete if saved

InsertsManipulating the data

Call Web Services

from google.appengine.api import urlfetch

some_feed = urlfetch.fetch('http://somesite.com/rss')

if some_feed.status_code == 200: self.response.out.write(some_feed.content)

URL Fetch APIAint no urllib2

54321

Authenticate to GoogleOpenID provider.... available

from google.appengine.api import users

current_user = users.get_current_user()

if not current_user: self.redirect(users.create_login_url('/current_url'))

nickname = current_user.nickname()email = current_user.email()

if users.is_current_user_admin(): ...

Users APIBut you can do your own thing too of course...

Send Email

from google.appengine.api import mail

def post(self): email_body = self.request.get('email_body') sender = users.get_current_user().email mail.send_mail(sender=sender, to='marce@google.com', subject='Wiki Page', body=email_body)

self.response.out.write('Email Sent')

Email APINo SMTP config required

Manipulate Images

# in modelavatar = db.BlobProperty()

# in handleravatar = images.resize(self.request.get("img"), 32, 32)

# availableresizecroprotatehorizontal_flipvertical_flipim_feeling_lucky :)

Image Manipulation APINo SMTP config required

Memcache SupportIn-memory distributed cache

from google.appengine.api import memcache

def get_data(): data = memcache.get("key") if data is not None: return data else: data = self.query_for_data() memcache.add("key", data, 60) return data

# Set several values, overwriting any existing values for these keys.memcache.set_multi({ "USA_98105": "raining", "USA_94105": "foggy", "USA_94043": "sunny" }, key_prefix="weather_", time=3600)

# Atomically increment an integer value.memcache.set(key="counter", 0)memcache.incr("counter")memcache.incr("counter")memcache.incr("counter")

Memcache APIIn-memory distributed cache

74

Google App EngineAreas of Work, Including…

• Offline Processing

• Rich Media Support

– e.g., large file upload / download

• Add’l Infrastructure Services

• What would you like to see?

Use it as you willNo need to go whole hog

Web Services

Your Application

Memcache API

Email API

Static Files

PythonRuntime

Configuration

URL Fetch API

Users API

Image API

Web Applications

Google App Engine

Act ThreeAnd there’s more

Now hosting open source JavaScript libraries at GoogleStarting with: Prototype, Script.aculo.us, jQuery, Dojo, MootoolsAccepting requests for other open source librariesCan access directly:

1

What if popular JavaScript libraries were available and shared in the browser?

ajax.googleapis.com/ajax/lib/prototype?v=1.6.0.2&packed=false

Other featuresAutomatic compressionMinification of libraries

2

Can access via AJAX API Loader: google.load(“prototype”, “1.6”);

Not tied to Google Code