google back to front: from gears to app engine and beyond
DESCRIPTION
I had the privilege of giving a Yahoo! Tech Talk at their HQ in Sunnyvale. I spoke on Gears, App Engine, and other technologies such as the Ajax Libraries API and Doctype.TRANSCRIPT
Google Back to FrontFrom Gears to App Engine, and beyond
Interesting, passionate, times
The Web is the platform of today, and of the future
WeBelieve...
Act OneThe Client
Top Grossing Film of 1957 Top Grossing Film of 2007
“Gimme Offline!”
It takes too long to update the Web
What is the philosophy?Do for offline what XMLHttpRequest did for web apps
Open Web
XMLHttpRequest
Ajax LibrariesDojo, jQuery, Prototype, GWT
Open Web
Gears
Gears LibrariesDojo Offline, GWT
DatabaseEmbedded using SQLite
Contributed Full Text Search
var db = google.gears.factory.create('beta.database', '1.0');db.open('database-demo');
db.execute('create table if not exists Demo (Phrase varchar(255), Timestamp int)');db.execute('insert into Demo values (?, ?)', [phrase, currTime]);
GearsDBAbstract over the API
var bob = {id: 3, name: 'Bob', url: 'http://bob.com', description: 'whee'};db.insertRow('person', bob);db.insertRow('person', bob, 'name = ?', ['Bob']);
db.selectAll('select * from person', null, function(person) { document.getElementById('selectAll').innerHTML += ' ' + person.name;});
var person = db.selectRow('person', 'id = 1');
// updateperson.name = 'Harry';db.updateRow('person', person);person = db.selectRow('person', 'id = 1');
// forceperson.name = 'Sally';db.forceRow('person', person);person = db.selectRow('person', 'id = 1');
db.deleteRow('person', bob);
GearsORMAre we going to get a GearsHibernate?
var Person = new GearsOrm.Model("Person", { firstName: GearsOrm.Fields.String({maxLength:25}), lastName: GearsOrm.Fields.String({maxLength:25})
});
GearsORM.Transaction(function() { new Person({name:"John"}).save(); new Person({name:"Doe"}).save();
});
Person.select("firstName = 'Uriel'");Person.count("lastName = ?",["Katz"])
“While developing transaction support for GearsORM i had to write a test, in that test it
execute 100 inserts and 100 updates, this test take about 15 seconds for the inserts and about 10
seconds for the updates without transactions,when using transactions for each set it takes about
377ms for the inserts and 200ms for the updates that is about
39 times faster!”
GearShiftDB Migrations for Gears
Gearshift.rules[1] = { // create the demo table up: function() { return this.e("CREATE TABLE demo_table ( id INTEGER PRIMARY KEY AUTOINCREMENT, name VARCHAR(30), movie VARCHAR(30) )").success; }, down: function() { return this.e("DROP TABLE demo_table").success; }};
Database Tools
Buxfer
Full Text Search•Gears added FTS2 to SQLite
•Create the databasedb.execute('CREATE VIRTUAL TABLE recipe USING fts2(dish, ingredients)');
•Search the databasedb.execute('SELECT dish FROM recipe WHERE recipe MATCH ?', ['tomatoes']);
Fun queries: dish:stew tomatoes Find rows with 'stew' in the dish field, and 'tomatoes' in any field.
The Digg Oracle
Local ServerA mini-web server that groks 200 and 304
ResourceStoreManually Capturing
var pageFiles = [ location.pathname, 'gears_base.js', '../scripts/gears_db.js', ‘foo.html’];
try { localServer = google.gears.factory.create('beta.localserver', '1.0');} catch (e) { alert('Could not create local server: ' + e.message); return;}
var store = localServer.openStore(this.storeName) || localServer.createStore(this.storeName);
store.capture(pageFiles, function(url, success, captureId) { console.log(url + ' capture ' + (success ? 'succeeded' : 'failed'));});
ManagedResourceStoreJSON Me
{ // version of the manifest file format "betaManifestVersion": 1,
// version of the set of resources described in this manifest file "version": "my_version_string",
// optional // If the store specifies a requiredCookie, when a request would hit // an entry contained in the manifest except the requiredCookie is // not present, the local server responds with a redirect to this URL. "redirectUrl": "login.html",
// URLs to be cached (URLs are given relative to the manifest URL) "entries": [ { "url": "main.html", "src": "main_offline.html" }, { "url": ".", "redirect": "main.html" }, { "url": "main.js" } { "url": "formHandler.html", "ignoreQuery": true }, ]}
HTML 5Offline in General
<html application=”manifest-of-urls.txt”>
<html application>“There’s a concept of an application cache. An application cache is a groupof resources, the group being identified by a URI (which typically happensto resolve to a manifest). Resources in a cache are either top-level ornot; top-level resources are those that are HTML or XML and when parsedwith scripting disabled have with the value ofthe attribute pointing to the same URI as identifies the cache.
When you visit a page you first check to see if you have that page in acache as a known top-level page.”
HTML 5 Gears
• Standards• Long term• All browsers• No plugin
• Implementation• Battle hardened• A place for innovation• Cross browser• Plugin
Past
Present
Future
A bleeding edge version of HTML 5!
Operating System
Event Queue
Mouse Moved
Mouse Pressed
Mouse Released
Key Pressed
Key Released
JavaScript Web Browsing
Potential
Bottleneck
Browser
User Interface
WorkerPool
Browser
1
WorkerPool
2 3
Message Passing
Worker Pool Code
function nextPrime(n) { // TODO: New top-secret prime-finding algorithm goes here. google.gears.workerPool.sendMessage(result);}
var pool = google.gears.factory.create('beta.workerpool', '1.0');pool.onmessage = function(message) { alert('next prime is: ' + message);}
var worker = pool.createWorker(String(nextPrime) + '; nextPrime()');
JavaScript Read Write APIs
DesktopShortcuts
var desktop = google.gears.factory.create('beta.desktop');desktop.createShortcut("Test Application", "An application at http://www.test.com/index.html", "http://www.test.com/index.html", {"16x16": "http://www.test.com/icon16x16.png", "32x32": "http://www.test.com/icon32x32.png", "48x48": "http://www.test.com/icon48x48.png", "128x128": "http://www.test.com/icon128x128.png"});
Notification APIGrowl for the Web
var notifier = google.gears.factory.create('beta.notifier', '1.0'); notifier.notify({ application: "My App", title: 'warning', description: 'some text', priority: 2, sticky: 'True', password: 'Really Secure',});
Location APIEven on a laptop?
// Getting the objectvar location = google.gears.factory.create( "beta.location", "1.0" ); // Setting up a callback to handle "location changed" eventslocation.onlocationstatechanged = function() { if (this.state == COMPLETE) { SetStatusText("Location accuracy:", this.accuracy); MoveMap(this.latitude, this.longitude); }}
location.startLocationUpdates();
Audio APIPlay and Record
// playvar audio = google.gears.factory.create('beta.audio');audio.src = 'http://blahblahblob.com/sampleaudio.wav';audio.load();audio.play();
// recordvar recorder = google.gears.factory.create('beta.audiorecorder');recorder.destination = <http post url>recorder.autoStream = true;recorder.record(); //asynchronous call
Don’t you want a better File Upload?
Resumable HTTP
Blob API
File System API
Gears: Open Source Update Mechanism for the Web
Don't forget, RIA's have rich internet back-ends (RIBs?)
Jonathan SchwartzCEO, Sun Microsystems
Act TwoThe Server
Google App EngineRunning Web Apps on Google’s Infrastructure
• Fully-integratedapplication environment
• Language agnostic runtimePython for now
• Free quota of 5Mpageviews per month
code.google.com/appengine
Google App EngineChallenges
Google App EngineEasy to use, Easy to scale, Free to start
Develop locally. Deploy to Google. Launch.
Develop locally. Deploy to Google. Launch.
Deploy
Develop locally. Deploy to Google. Launch.
Develop locally.
• Google App Engine SDK is open source
• Simulates production environment
•dev_appserver.py myapp•appcfg.py update myapp
# helloworld.py print "Content-Type: text/html" print print "Hello, world!"
# app.yaml application: dalmaer-helloworld version: 1 runtime: python api_version: 1 handlers: - url: /.* script: helloworld.py
Hello WorldSimplest output, simplest config
Web Services
loadcontacts
var contacts = [ { id: 1, name: ‘Dion Almaer’, ... }, { id: 2, name: ‘Ben Galbraith’, ... },]savecontact
name=’Dion A Lamer’email=’[email protected]’
the snake is almost there
threadssockets
filesforeground
from django import v0_96 as django
Wow, that's abig table!
Store dataScalable, and not like a RDBMS
from google.appengine.ext import db
class Story(db.Model): title = db.StringProperty() body = db.TextProperty(required=True) author = db.UserProperty(required=True) created = db.DateTimeProperty(auto_now_add=True) rating = db.RatingProperty()
# Many other types: BlobProperty, ReferenceProperty, EmailProperty, PhoneProperty, IMProperty, PostallAddressProperty, etc.
Data ModelJust an object
SELECT *FROM StoryWHERE title = 'App Engine Launch'AND author = :current_userAND rating >= 10ORDER BY rating, created DESC
GQLOur query language
Story.gql(GQL_FROM_BEFORE, current_user = users.get_current_user())
query = Story.all()
query.filter('title =', 'Foo') .order('-date') .ancestor(key)
Run the queriesBeyond GQL
story = Story( title = 'Morning Glory', body = '....', author = users.get_current_user(),)
story.put() # save or update
story.delete() # delete if saved
InsertsManipulating the data
Call Web Services
from google.appengine.api import urlfetch
some_feed = urlfetch.fetch('http://somesite.com/rss')
if some_feed.status_code == 200: self.response.out.write(some_feed.content)
URL Fetch APIAint no urllib2
54321
Authenticate to GoogleOpenID provider.... available
from google.appengine.api import users
current_user = users.get_current_user()
if not current_user: self.redirect(users.create_login_url('/current_url'))
nickname = current_user.nickname()email = current_user.email()
if users.is_current_user_admin(): ...
Users APIBut you can do your own thing too of course...
Send Email
from google.appengine.api import mail
def post(self): email_body = self.request.get('email_body') sender = users.get_current_user().email mail.send_mail(sender=sender, to='[email protected]', subject='Wiki Page', body=email_body)
self.response.out.write('Email Sent')
Email APINo SMTP config required
Manipulate Images
# in modelavatar = db.BlobProperty()
# in handleravatar = images.resize(self.request.get("img"), 32, 32)
# availableresizecroprotatehorizontal_flipvertical_flipim_feeling_lucky :)
Image Manipulation APINo SMTP config required
Memcache SupportIn-memory distributed cache
from google.appengine.api import memcache
def get_data(): data = memcache.get("key") if data is not None: return data else: data = self.query_for_data() memcache.add("key", data, 60) return data
# Set several values, overwriting any existing values for these keys.memcache.set_multi({ "USA_98105": "raining", "USA_94105": "foggy", "USA_94043": "sunny" }, key_prefix="weather_", time=3600)
# Atomically increment an integer value.memcache.set(key="counter", 0)memcache.incr("counter")memcache.incr("counter")memcache.incr("counter")
Memcache APIIn-memory distributed cache
74
Google App EngineAreas of Work, Including…
• Offline Processing
• Rich Media Support
– e.g., large file upload / download
• Add’l Infrastructure Services
• What would you like to see?
Use it as you willNo need to go whole hog
Web Services
Your Application
Memcache API
Email API
Static Files
PythonRuntime
Configuration
URL Fetch API
Users API
Image API
Web Applications
Google App Engine
Act ThreeAnd there’s more
Now hosting open source JavaScript libraries at GoogleStarting with: Prototype, Script.aculo.us, jQuery, Dojo, MootoolsAccepting requests for other open source librariesCan access directly:
1
What if popular JavaScript libraries were available and shared in the browser?
ajax.googleapis.com/ajax/lib/prototype?v=1.6.0.2&packed=false
Other featuresAutomatic compressionMinification of libraries
2
Can access via AJAX API Loader: google.load(“prototype”, “1.6”);
Not tied to Google Code