forward: declarative ajax web applications through sql++ on a unified application state yupeng fu...
TRANSCRIPT
1
FORWARD:Declarative Ajax Web
Applications through SQL++ on aUnified Application State
Yupeng Fu Kian Win Ong
Yannis Papakonstantinou
UC San Diego
2
• Web application framework
• Single language: SQL++
• Zero frictions and automatic optimizations– Enable IT personnel to create Ajax apps and interactive
visualizations– Improve development productivity
• Alpha release– http://forward.ucsd.edu– 8 applications (commercial + academic)
FORWARD
3
FORWARD Website
http://forward.ucsd.edu
4
Simple Web Application
5
Simple Web Application
6
Simple Web Application
7
Mundane Low-Level Programming for Ajax Applications
Browser
ApplicationServer
Database
DOM (XHTML),JavaScript (JSON)
Java, Pythonetc.(Objects)
SQL (Tables)
Event Side-effect
Convert JSON to objects
Convert objects to tables
Convert tables to objects
Convert objects to JSON
Challenge #1: Impedance mismatch due to different languages and data models
Requires using multiple languages, tedious data modeling and coding
8
Mundane Low-Level Programming for Ajax Applications
Browser
ApplicationServer
Database
DOM (XHTML),JavaScript (JSON)
Java, Pythonetc.(Objects)
SQL (Tables)
Event Side-effect
Retrieve user input
Serialize and parse request
Check access rights
Parameterize SQL query
INSERT INTO ...
Challenge #2: Distributed data access and
programming due to multiple tiers and machines
Requires partitioning code manually, and ad-hoc
coordination across multiple machines
9
Mundane Low-Level Programming for Ajax Applications
Browser
ApplicationServer
Database
DOM (XHTML),JavaScript (JSON)
Java, Pythonetc.(Objects)
SQL (Tables)
Event Side-effect
SELECT *FROM ...WHERE id = ...
Instantiate partial HTML and JSON template
Incrementally update old state to new state
Incrementally re-render DOM and JavaScript components
Challenge #3: Incrementally modifying old page to new page due to Ajax programming model
Requires identifying data flow dependencies, correctly
transitioning from one consistent state to another.
10
Incremental Updates are Error-Prone
• New requirements:• Libraries with saved
comments are green instead of red
• Multiple code paths to change:• Code that loads the page• Code that saves the
comment
• Code complexity increases with number of events/actions
11
FORWARD Architecture
Actions
Unified Application State (UAS)
Pages
• SQL++ as single language (inspired by Hilda and Links)
• SQL extensions for semi-structured features (nesting, heterogeneity)
• State machine conceptual model• No need for multiple tiers, machines, languages
12
Runtime Interpreter
FORWARD Architecture
Actions
Unified Application State (UAS)
Pages
read writesideeffects next_page
http://localhost/libraries
• Virtual database of different data sources
• Unifying data model
• PL/SQL++
• INSERT, UPDATE, DELETE statements on UAS
• Control flow (conditionals, loops, functions)
13
Runtime Interpreter
FORWARD Architecture
Actions
Unified Application State (UAS)
Pages
read
Page instance
rendering• Declarative page as rendered queries
• Data of page is a SQL++ query over UAS
• Rendering of page is template markup
• Template markup for both HTML and rich JavaScript components
• No side-effects: can only read from UAS
14
Runtime Interpreter
FORWARD Architecture
Actions
Unified Application State (UAS)
Pages
read writesideeffects next_page
User Input
Page instance
renderingAjax (XHR)
read
• Action only saves comment
• Action does not specify how to replace the textarea and Save button on the page
15
FORWARD: Single Language, Location Transparent, Declarative
ActionSpecifications
Unified Application State (UAS)Specification
Page Specifications
• SQL++ as single language
• Challenge #1: Impedance mismatch
• Declarative page as rendered queries
• Challenge #3: Incrementally modifying old page to new page
• Virtual database
• Challenge #2: Distributed data access and programming
16
• Motivation
• Architecture
• Syntax and Semantics– Unified Application State– SQL++– Page– Action
• Future Work
Outline
17
UAS Specification
Unified Application State (UAS)
RDBMS(Tables)
Session(Objects)
URL Params(KV Pairs)
User Input(JSON)
define sql source db options { driver : 'postgresql', host : 'book_db_host', port : 5432, database : 'libraries', user : 'postgres', password : '9xk8NToA'}
Source Specification
SQL Source Wrapper
• Developer provides source specification
• Framework provides capability-based source wrappers that execute different subsets of SQL++
• SQL++ queries over UAS virtual database• Run by FORWARD distributed query processor
• Current sources: • SQL database• In-memory SQL++ values
• Future sources: • JSON databases (MongoDB, CouchDB)• Large-scale data services (BigQuery, Redshift)
select ...from db.librariesleft join session.notes on ...where ...
18
SQL++ Data Model
value = null | scalar | tuple | table
scalar = primitive | type "(" primitive ")"
tuple = "{" ( name : value "," )* "}”
table = "[" ( key? tuple "," )* "]"key = "(" primitive ")"
{ lat : 0.0 , lng : 12.3 , ... }
[ (1) { lat : 0.0 , lng : 12.3 } , (2) { lat : 0.0 , lng : 45.6 }]
12.3
date('2013-08-30')
• Extended JSON syntax with rich types and keys
• Opposite of Math conventions!
• Data model designed to capture both SQL tables and JSON for the need of pages• Captures subset of JSON used in
~30 JavaScript components
19
SQL++ Data Model
• Extensions over SQL tables: • Optional type checking• Root value• Nesting• Heterogeneity• Optional keys
• Extensions over JSON• Optional keys• Rich scalar types (e.g. dates)
• Work in progress• Order• Table of scalars/tables
• Analogous SQL++ extensions to query language
value = null | scalar | tuple | table
scalar = primitive | type "(" primitive ")"
tuple = "{" ( name : value "," )* "}”
table = "[" ( key? tuple "," )* "]"key = "(" primitive ")"
20
Page Instance
_1_html : { template : '<body>...', children : {
_2_google_map_Maps : { markers : [ { position : { lat : 32.7, lng : -117.2 },
infowindow : _3_html : { template : '<div>...', children : ... } } ... ] } }}
SQL++ Value
State of JavaScript component
HTML string template with placeholders for children
21
Page Instance
_1_html : { template : '<body>...', children : {
_2_google_map_Maps : { markers : [ { position : { lat : 32.7, lng : -117.2 },
infowindow : _3_html : { template : '<div>...', children : ... } } ... ] } }}
SQL++ Value
<% html %><body> ..
<% unit google.map.Maps %> { markers : [ { position : { lat : 32.7, lng : -117.2 },
infowindow : <% html %> <div> ... </div> <% end html %> } ... ] } <% end unit %></body><% end html %>
Syntactic sugar: inline HTML
Template Markup
Syntactic sugar: visual units
Also SQL++ query Also Page Specification
22
• Motivation
• Architecture
• Syntax and Semantics– Unified Application State– SQL++– Page– Action
• Future Work
Outline
23
Page Specification – Static Page
<% html %>...
<% unit google.map.Maps %> { markers : [ { position : { lat : 32.7, lng : -117.2 }, infowindow : <% html %> Library: Del Mar Library <br /> Comment: <% unit html.TextArea %> { value : Great! } <% end unit %> <br /> <button value="Save"> <% end html %> } ... ] } <% end unit %><% end html %>
• Declarative static page using only template markup• HTML• Visual unit wrappers for
JavaScript components
• Visual unit wrappers translate between state and behavior• Translation via renderers
and collectors• No JavaScript code is
needed on the page specification
24
Page Specification – Dynamic Visualization
<% with libraries as select * from db.libraries left join session.notes on ... where ...%>
<% unit google.map.Maps %>{ markers : [ <% for l in libraries primary key (library_id) %> { position : { lat : <%= l.lat %> lng : <%= l.lng %> }, ... } <% end for %> ]}<% end unit %>
• Declarative visualization page using only template markup and SQL++ queries
• Dynamic data of page specified by SQL++ queries over UAS
View
Syntactic sugar: iteration
Substitution
25
Page Specification – Dynamic Visualization
<% with libraries as select * from db.libraries left join session.notes on ... where ...%>
<% unit google.map.Maps %>{ markers : [ <% for l in libraries primary key (library_id) %> { position : { lat : <%= l.lat %> lng : <%= l.lng %> }, ... } <% end for %> ]}<% end unit %>
with libraries as select ... from db.libraries left join session.notes on ... where ...
{ markers : select position : { l.lat as lat, l.lng as lng } from libraries as l primary key (library_id)}
• Incremental updates are delegated to FORWARD’s• Incremental renderers• Incremental view maintenance
26
Page Specification – User Input
<% with libraries as select * from db.libraries left join session.notes on ... where ...%>
<% unit google.map.Maps %>{ markers : [ <% for l in libraries primary key (library_id) %> { position : { lat : <%= l.lat %> lng : <%= l.lng %> }, ... } <% end for %> ]}<% end unit %>
<% define comment string %>infowindow : <% html %> Library: <%= l.library_name %> <br /> Comment: <% if l.comment is null %>
<% unit html.TextArea %> { value : <% bind comment %> } <% end unit %> <br /> <button value="Save" <% event click ajax /save_note %>>
<% else %>
<%= l.comment %> <br /> <button value="Delete" <% event click ajax /delete_note %>>
<% end if %> <% end html %>
• Declarative page for visualization and user input
• User input is bound into the UAS
• Events trigger an action with its context
Variable in user input source of UAS
Binds to variable
Event / Action
27
Action Specification
define action /save_note function save_note;
define function save_note() asbegin insert into session.notes (library_ref, comment) values ( context.l.library_ref, context.comment ); next_page('map');end;
• An action is a URL mapped to a function (procedure) that causes side-effects
• Action can access all variables in the event’s context• define (mutable)• with (immutable)• for ... in (immutable)
• insert, update and delete statements writes to UAS
• next_page() chooses the next page to be evaluated and displayed
28
• Incremental view maintenance (IVM) of pages– Incremental update of pages is delegated to FORWARD’s optimizations– First IVM prototype by reduction to delta-based, relational IVM
• Y. Fu, K. Kowalczykowski, K. W. Ong, Y. Papakonstantinou and K. K. Zhao. Ajax-based Report Pages as Incrementally Rendered Views. SIGMOD 2010.
– Work-in-progress based on algebraic rewrite rules that propagate predicates (as opposed to data)
• Distributed query processor– Virtual database is enabled by FORWARD’s distributed query processor– Recent work led to distributor that is tuned for data of web applications: large
database and small in-memory objects
• Lightweight JavaScript components– Visual unit wrappers for components with 50++ attributes / methods require
upfront time and effort to translate between state and behavior– Work in progress: enable translation for subset of attributes / methods that
matter to a page
Related and Future Work in FORWARD
29
Cloud-based IDE
30
FORWARD Website
http://forward.ucsd.edu