strategies for maintaining app engine availability during read only periods
DESCRIPTION
Discusses how you can use the capabilities API and/or the CapabilityDisabled exceptions to catch and handle errors during planned or unplanned downtimes and help maintain a good user experience.TRANSCRIPT
Strategies for maintainingduring read-only periods
Jason CooperGoogle Developer [email protected]
April 6, 2010
Strategies
Capabilities APIException handlingRead-only versions
Strategies
Capabilities APIException handlingRead-only versions
Capabilities API - datastore
is_enabled method can be used to test whether a certain capability is currently enabled.
datastore_writes = CapabilitySet( 'datastore_v3', capabilities=['write'])
if not datastore_writes.is_enabled(): # ...render form with form elements disabled # or, if a form is submitted, push a new task # to the task queueelse: # ...render form normally
Capabilities API - datastore
will_remain_enabled_for method can be used to learn whether a component will be disabled within a certain number of seconds
datastore_writes = CapabilitySet( 'datastore_v3', capabilities=['write'])
if datastore_writes.will_remain_enabled_for(60): # ...render form normallyelse: # ...render form with form elements disabled
Capabilities API - memcache
The capabilities API can be used with other services including memcache...
memcache_set = CapabilitySet( 'memcache', methods=['set'])
if memcache_set.is_enabled(): # ...fetch from cacheelse: # ...bypass cache
Capabilities API - images
... as well as the Images service.
images_capability = CapabilitySet('images')
if images_capability.is_enabled(): my_image = images.resize(my_image, 64, 64)
Capabilities API
Pros:Automated -- no changes needed when read-only mode begins or endsAllows for datastore writes to be "deferred" by pushing a new task to the task queue instead of writing immediately; the task will be continually re-tried until the task succeeds, so the write should occur eventually.
Cons:Python-only (for now)Undocumented (for now)
see google/appengine/api/capabilities SDK directory
Strategies
Capabilities APIException handlingRead-only versions
Exception handling - Python
Python: http://code.google.com/appengine/docs/python/howto/maintenance.html
from google.appengine.ext import dbimport google.appengine.runtime.apiproxy_errors
myModel = db.Model()try: myModel.put()except apiproxy_errors.CapabilityDisabledError: # fail gracefully here or add a new task to the # task queue that writes the new entity when # the datastore is available
Datastore exception handling - Java
Java:http://code.google.com/appengine/docs/java/howto/maintenance.html
import com.google.apphosting.api.ApiProxy.CapabilityDisabledException;
try { // JDO: pm.makePersistent(entity); // JPA: em.persist(entity); // low-level: ds.put(entity);} catch (CapabilityDisabledException e) { // fail gracefully here} finally { // ...}
Memcache exception handling - Java
Java:http://code.google.com/appengine/docs/java/howto/maintenance.html
As with Python, memcache is unavailable during read-only mode and exceptions aren't thrown by default.
You can use a StrictErrorHandler if you want an exception thrown when get and put aren't available.
Memcache exception handling - Java
Java:http://code.google.com/appengine/docs/java/howto/maintenance.html
import com.google.appengine.api.memcache.MemcacheServiceException;
// ...ms.setErrorHandler(new StrictErrorHandler());
try { ms.put(key, value);} catch (MemcacheServiceException e) { // degrade gracefully}
Exception handling
Pros:Automated -- no changes needed when read-only mode begins or endsAllows for datastore writes to be "deferred" by pushing a new task to the task queue instead of writing immediately; the task will be continually re-tried until the task succeeds, so the write should occur eventually.
Cons:Reactive -- exception is caught only after the read or write call is processed, unlike the first solution.Complicates code slightly -- the exception needs to be caught for every write attempt.
Note on using tasks to defer writes
The task queue allows you defer writes when the datastore is unavailable -- you can add a new task instead, which the system will automatically retry in the background until it succeeds.
If you choose this approach, keep the following in mind:
The number of tasks that you can add per day is currently limited to 1,000,000. If your app receives a lot of write traffic (e.g. > 350 QPS), you could exceed this quota unless the period of unavailability is short.Consider adding a timestamp field to your entities plus extra logic so you don't accidentally overwrite a later update when the tasks are applied.
Strategies
Capabilities APIException handlingRead-only versions
Read-only versions
User
App Engine
Frontend
Version 1.1
Version 2.1
Datastore
http://2.1.fredsa.appspot.com/
http://fredsa.appspot.com/
Read-only versions
Python: app.yaml
application: helloworldversion: readonlyruntime: pythonapi_version: 1
handlers:- url: .* script: main.py
Read-only versions
Java: appengine-web.xml
<?xml version="1.0" encoding="utf-8"?><appengine-web-app> <application>helloworld</application> <version>readonly</version>
<!-- ... --></appengine-web-app>
Read-only versions
Pros:Proactive -- e.g. users see a grayed-out form instead of an error on save
Cons:Manual -- it's your responsibility to be aware of and switch versions before read-only mode starts and after it endsOnly useful for planned downtimes -- unless other mitigation strategies are in place, your app will still go down during unplanned downtimes.Maintenance