![Page 1: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/1.jpg)
I’ve (probably) been using
Google App Enginefor a week longer than you have
Simon Willison - http://simonwillison.net/BarCamp London 431st May 2008
![Page 2: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/2.jpg)
Except you have to re-writeyour whole application
If you totally rethink the way you use a database
![Page 3: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/3.jpg)
What it can do
• Serve static files
• Serve dynamic requests
• Store data
• Call web services (sort of)
• Authenticate against Google’s user database
• Send e-mail, process images, use memcache
![Page 4: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/4.jpg)
The dev environment
• Download the (open source) SDK
• a full simulation of the App Engine environment
• dev_appserver.py myapp for a local webserver
• appcfg.py update myapp to deploy to the cloud
is really, really nice
![Page 5: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/5.jpg)
Options
• You have to use Python
• You can choose how you use it:
• CGI-style scripts
• WSGI applications
• Google’s webapp framework
• Django (0.96 provided, or install your own)
![Page 6: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/6.jpg)
Hello World# helloworld.pyprint "Content-Type: text/html"printprint "Hello, world!"
# app.yamlapplication: simonwillison-helloworldversion: 1runtime: pythonapi_version: 1
handlers:- url: /.* script: helloworld.py
![Page 7: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/7.jpg)
With webapp and WSGIimport wsgiref.handlersfrom google.appengine.ext import webapp
class MainPage(webapp.RequestHandler): def get(self): self.response.headers['Content-Type'] = 'text/html' self.response.out.write('Hello, webapp World!')
def main(): application = webapp.WSGIApplication( [('/', MainPage)], debug=True) wsgiref.handlers.CGIHandler().run(application)
if __name__ == "__main__": main()
![Page 8: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/8.jpg)
With Django
from django.conf.urls.defaults import *from django.http import HttpResponse
def hello(request): return HttpResponse("Hello, World!")
urlpatterns = patterns('', ('^$', hello),)
(And django_dispatch.py for boilerplate)
![Page 9: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/9.jpg)
• Don't use CGI: it requires reloading for every hit
• Why use Django over webapp?
• Django has easy cookies and custom 500 errors
• Django is less verbose
• Django middleware is really handy
• You can use other WSGI frameworks if you like
![Page 10: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/10.jpg)
# in app.yaml
handlers:- url: /css static_dir: css- url: /img static_dir: img- url: /favicon.ico static_files: img/favicon.ico upload: img/favicon.ico mime_type: image/x-icon
Static files
![Page 11: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/11.jpg)
The Datastore API
![Page 12: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/12.jpg)
“Bigtable is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of
data across thousands of commodity servers. Many projects at Google store data in Bigtable, including web indexing, Google
Earth, and Google Finance.”
![Page 13: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/13.jpg)
The App Engine datastore
• Apparently based on BigTable
• Absolutely not a relational database
• No joins (they do have “reference fields”)
• No aggregate queries - not even count()!
• Hierarchy affects sharding and transactions
• All queries must run against an existing index
![Page 14: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/14.jpg)
Models and entities
• Data is stored as entities
• Entities have properties - key/value pairs
• An entity has a unique key
• Entities live in a hierarchy, and siblings exist in the same entity group - these are actually really important for transactions and performance
• A model is kind of like a class; it lets you define a type of entity
![Page 15: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/15.jpg)
AppEngine Models
from google.appengine.ext import db
class Account(db.Model): slug = db.StringProperty(required=True) owner = db.UserProperty() onlyme = db.BooleanProperty() referrers = db.StringListProperty()
(There is a ReferenceProperty, but I haven’t used it yet)
![Page 16: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/16.jpg)
Inserting dataaccount = Account( key_name = slug, slug = slug, referrers = ['...', '...'], onlyme = False, owner = users.get_current_user())db.put(account) # Or account.put()
Browser.get_or_insert(key_name, parent = account, slug = browser_slug)
![Page 17: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/17.jpg)
Running queries
Browser.all().ancestor(account)
Account.gql("WHERE slug = :1", slug))
Story.all().filter( 'title =', 'Foo').order('-date')
![Page 18: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/18.jpg)
BUT...• All queries must run against an existing index
• Filtering or sorting on a property requires that the property exists
• Inequality filters are allowed on one property only
• Properties in inequality filters must be sorted before other sort orders
• ... and various other rules
• Thankfully the dev server creates most indexes for you automatically based on usage
![Page 19: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/19.jpg)
How indexes are used1. The datastore identifies the index that
corresponds with the query’s kind, filter properties, filter operators, and sort orders.
2. The datastore starts scanning the index at the first entity that meets all of the filter conditions using the query’s filter values.
3. The datastore continues to scan the index, returning each entity, until it finds the next entity that does not meet the filter conditions, or until it reaches the end of the index.
![Page 20: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/20.jpg)
Further limitations
• If you create a new index and push it live, you have to wait for it to rebuilt
• This can take hours, and apparently can go wrong
• You can’t safely grab more than about 500 records at once - App Engine times out
• You can’t delete in bulk
![Page 21: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/21.jpg)
Other random notes
• You have to use the URL Fetch API to do HTTP requests (e.g. for web services) - and it times out agressively at about 5 seconds
• The Google accounts Users API is ridiculously easy to use, but...
• no permanent unique identifier; if the user changes their e-mail address you’re screwed
• The new image and memcache APIs are neat
![Page 22: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/22.jpg)
Final thoughts
• It’s really nice not to have to worry about hosting
• But... the lack of aggregate queries and ad-hoc queries really hurts
• Perfect for small projects you don’t want to worry about and big things which you’re sure will have to scale
• Pricing is comparable to S3 - i.e. stupidly cheap
![Page 23: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/23.jpg)
Pricing
• $0.10 - $0.12 per CPU core-hour
• $0.15 - $0.18 per GB-month of storage
• $0.11 - $0.13 per GB outgoing bandwidth
• $0.09 - $0.11 per GB incoming bandwidth
![Page 24: I've (probably) been using Google App Engine for a week longer than you have](https://reader033.vdocuments.site/reader033/viewer/2022052618/554be7bcb4c90556328b4cb6/html5/thumbnails/24.jpg)
Thank you!