life of an app engine request

8/14/2019 Life of an App Engine Request

1/53


2/53

From Spark Plug to Drive Train:Life of an App Engine Request

Alon Levi05/27/09Post your questions for this talk on Google Moderator:code.google.com/events/io/questions


3/53

Designing for Scale and Reliability

App Engine: Design Motivations

Life of a Request

Request for static contentRequest for dynamic content

Requests that use APIs

App Engine: Design Motivations, Recap

App Engine: The Numbers

Agenda


4/53

Designing for Scale and Reliability


5/53

Google App Engine


6/53

LiveJournal circa 2007From Brad Fitzpatrick's USENIX '07 talk:"LiveJournal: Behind the Scenes"


7/53

From Brad Fitzpatrick's USENIX '07 talk:"LiveJournal: Behind the Scenes"

LiveJournal circa 2007

Frontends StorageApplicationServers

Memcache

Static File Servers


8/53

Basic LAMP

Linux, Apache, MySQL,Programming Language

Scalable?Shared machine for databaseand webserver

Reliable?Single point of failure (SPOF)


9/53

Database running on aseparate server

RequirementsAnother machine plusadditional management

Scalable?Up to one web serverReliable?

Two single points of failure

Dedicated Database


10/53

Benefits:

Grow traffic beyond the capacity of onewebserverRequirements:More machinesSet up load balancing

Multiple Web Servers


11/53

Load Balancing: DNS Round Robin


Register list of IPs with DNS

Statistical load balancingDNS record is cached with Time To Live (TTL)

TTL may not be respected


12/53

Register list of IPs with DNS

Statistical load balancingDNS record is cached with Time To Live (TTL)

TTL may not be respected



Now wait for DNS changes to propagate :-(


13/53

Scalable?

Add more webservers as necessaryStill I/O bound on one databaseReliable?

Cannot redirect traffic quicklyDatabase still SPOF




14/53

Benefits:

Custom RoutingSpecializationApplication-level load balancing

Requirements:More machines

Configuration and code for reverse proxies

Reverse Proxy


15/53

Scalable?

Add more web serversSpecializationBound by

Routing capacity of reverse proxyOne database server

Reliable?Agile application-level routingSpecialized components are more robustMultiple reverse proxies requires network-level routing

DNS Round Robin (again)Fancy network routing hardware

Database is still SPOF

Reverse Proxy


16/53

Master-Slave Database

Benefits:

Better read throughputInvisible to application

Requirements:

Even more machinesChanges to MySQL


17/53


Scalable?Scales read rate with # of servers

But not writes

What happens eventually?


18/53


Reliable?Master is SPOF for writesMaster may die before replication


19/53

Partitioned Database

Benefits:Increase in both readand write throughput

Requirements:Even more machinesLots of managementRe-architect data model

Rewrite queries


20/53

The App Engine Stack


21/53

App Engine:Design Motivations


22/53

Design Motivations

Build on Existing Google Technology

Provide an Integrated Environment

Encourage Small Per-Request Footprints

Encourage Fast Requests

Maintain Isolation Between Applications

Encourage Statelessness and Specialization

Require Partitioned Data Model


23/53

Life of a Request:Request for Static Content


24/53

Request for Static Content on Google Network

Routed to the nearest Google datacenterTravels over Google's network

Same infrastructure other Google products useLots of advantages for free


25/53

Request for Static Content

Google App Engine Front Ends

Load balancingRouting

Frontends route static requests to specialized servinginfrastructure

Routing at the Front End


26/53


Google Static Content Serving

Built on shared Google Infrastructure

Static files are physically separate from code files

How are static files defined?

Static Content Servers


27/53

What content is static?


...


28/53

Request For Static Content

Back to the Front End and out to the user

Specialized infrastructure

App runtimes don't serve static content

Response to the user


29/53

Life of a Request:Request for Dynamic Content


30/53

Request for Dynamic Content: New Components

App Servers

Serve dynamic requestsWhere your code runs

App MasterSchedules applicationsInforms Front Ends

App Servers and App Master


31/53

Request for Dynamic Content: Appservers

Many applications

Many concurrent requestsSmaller app footprint + fast requests = more apps

Enforce IsolationKeeps apps safe from each other

Enforce statelessnessAllows for scheduling flexibilityService API requests

What do they do?


32/53

Request For Dynamic Content

Front Ends route dynamicrequests to App Servers

Routing at the Frontend


33/53

1. Checks for cached runtimeIf it exists, no initialization

2. Execute request3. Cache the runtime

System is designed to maximize caching

Slow first request, faster subsequent requestsOptimistically cache data in your runtime!

Request for Dynamic Content

App Server


34/53

Life of a Request:Requests accessing APIs


35/53

App Server

1. App issues API call2. App Server accepts3. App Server blocks runtime4. App Server issues call5. Returns the response

Use APIs to do things youdon't want to do in yourruntime, such as...

API Requests


36/53

APIs


37/53

Distributed in-memory cachememcacheg

Also written by Brad Fitzpatrickadds: set_multi, get_multi, add_multi

Optimistically cache for optimizationVery stable, robust and specialized

Memcacheg

A more persistent in-memory cache


38/53

The App Engine Datastore

Persistent storage


39/53

The App Engine Datastore

Your data is already partitionedUse Entity Groups

Explicit Indexes make for fast readsBut slower writes

Replicated and fault tolerantOn commit: 3 machinesGeographically distributed

Bonus: Keep globally unique IDs for free

Persistent storage


40/53

GMail

GoogleAccounts

Picasaweb

Other APIs

Gadget API

On roadmap

Google Talk


41/53

App Engine:

Design Motivations, Recap


42/53

creative commons licensed photograph: http://www.flickr.com/photos/cote/54408562/

Build on Existing Google Technology


43/53

Provide an Integrated Environment

Why?

Manage all apps togetherWhat it means for you:

Follow best practicesSome restrictionsUse our tools

Benefits:Use our toolsAdmin ConsoleAll of your logs in one place

No machines to configure or manageEasy deployment


44/53

Encourage Small Per-Request Footprints

Why?

Better utilization of App ServersFairness

What it means for your app:Less Memory UsageLimited CPU

Benefits:Better use of resources


45/53

Encourage Fast Requests

Why?

Better utilization of appserversFairness between applicationsRouting and scheduling agility

What it means for your app:Runtime cachingRequest deadlines

Benefits:Optimistically share state between requestsBetter throughput

Fault toleranceBetter use of resources


46/53

Maintain Isolation Between Apps

Why?

SafetyPredictability

What it means for your app:Certain system calls unavailable

Benefits:

SecurityPerformance


47/53

Encourage Statelessness and Specialization

Why?

App Server performanceLoad balancingFault tolerance

What this means for you app:Use API calls

Benefits:Automatic load balancingFault toleranceLess code for you to write

Better use of resources


48/53

Require Partitioned Data Model

Why?

The DatastoreWhat this means for your app:

Data model + IndexesReads are fast, writes are slower

Benefits:

Design your schema onceNo need to re-architect for scalability

More efficient use of cpu and memory


49/53

Google App Engine:

The Numbers


50/53

Google App Engine

Currently, over80K applications

Serving over140M pageviews per day

Written by over200K developers


51/53

Open For Questions

The White House's "Open For Questions" application

accepted 100K questions and 3.6M votes in under48 hours


52/53

Questions?

Post yourquestions for this talk on Google Moderator:code.google.com/events/io/questions


53/53

life of an app engine request

Documents