making database applications perform using program analysis

Making Database Applications Perform Using Program Analysis

Alvin CheungSamuel MaddenArmando Solar-

LezamaMIT

Owen ArdenAndrew C. Myers

Cornell

Developing Database Applications

2/1/13 NEDB '13 2

Application Server SQL Database

JavaApplication Logic

SQLQuery


2/1/13 NEDB '13 3

SQL Database


PL/SQLStoredProcedures

SQLQuery

Application Server


2/1/13 NEDB '13 4

SQL Database



SQLQuery

Application Server

Language Choice for Application Logic

Application Distribution

Program analysis to the rescue!

StatusQuo

• Express application logic in ways that programmers are comfortable with

• Job of compiler & runtime to determine the most efficient implementation

2/1/13 NEDB '13 5

Two Key Technologies

• Infer queries from imperative code

• Migrate computation between servers for optimal performance

2/1/13 NEDB '13 6

Relational Operations in Imperative Code

2/1/13 NEDB '13 7

List getUsersWithRoles () { List users = getUsersFromDB(); List roles = getRolesFromDB(); List results = new ArrayList(); for (User u : users) {

for (Role r : roles) { if (u.roleId == r.id) results.add(u); }} return results; }

SELECT * FROM userSELECT * FROM role

List getUsersWithRoles () { return executeQuery( “SELECT u FROM users u, roles r WHERE u.roleId == r.id

ORDER BY u.roleId, r.id”; }

convert to

Relational Operations in Imperative Code

2/1/13 NEDB '13 8

List getUsersWithRoles () { List users = getUsersFromDB(); List roles = getRolesFromDB(); List results = new ArrayList(); for (User u : users) {

for (Role r : roles) { if (u.roleId == r.id) results.add(u); }} return results; }

List getUsersWithRoles () { return executeQuery( “SELECT u FROM users u, roles r WHERE u.roleId == r.id

ORDER BY u.roleId, r.id”; }

convert to

Goal Find a variable that

we can rewrite into a SQL expression

output variableresults

Query By Synthesis (QBS)• Identify potential code fragments– i.e., regions of code that fetches

persistent data and return values

• Find SQL expressions for output variables

• Try to prove that those expressions preserve program semantics– if so, convert the code!

2/1/13 NEDB '13 9

Search for Output Expressions

2/1/13 NEDB '13 11

List getUsersWithRoles () { List users = query(select * from users); List roles = query(select * from roles); List results = []; for (User u : users) {

for (Role r : roles) { if (u.roleId == r.id) results = results : [] }} return results; }Relations involved:users, roles

Possible expressions to consider for results: σf(users) topf(users) πf(users ⨝g roles)πf(σg(users) ⨝h roles) other expressions involving users, roles

Infinite search space size!

usersroles

results

Constraints for Output Expressions

2/1/13 NEDB '13 12

List getUsersWithRoles () { List users = query(select * from users); List roles = query(select * from roles); List results = []; for (User u : users) {

for (Role r : roles) { if (u.roleId == r.id) results = results : [] }} return results; }

usersroles

results

Hoare-style program verification

If

output expression

outer loop invariant

outer loop invariant outer loop terminatesthenoutput expression

is true andis true

Still need a smarter way to search

results = πuser( users ⨝roleId = id roles )

results = πuser( users[0 .. i] ⨝roleId = id roles )

Search for Output Expressions

and Invariants• Use program synthesis as search

engine

2/1/13 NEDB '13 13

Program synthesizer

Symbolic desc. of search space

Solutionconstraints

Expression thatsatisfies all theconstraints

Symbolic manipulation Counter-example driven

search

Experiments

2/1/13 NEDB '13 14

Real-world Evaluation

2/1/13 NEDB '13 15

Wilos (project management application) – 62k LOC

Operation type # Fragments found

# Fragments converted

Projection 1 1Selection 13 10Join 7 7Aggregation 11 10Total 33 28

0 20K 40K 60K 80K 100K100

1K

10K

100K

1000Koriginal (lazy)

Number of roles / users in DB

Exec

utio

n ti

me

(ms)

Performance Evaluation:Join Query

2/1/13 NEDB '13 16

Nested-loop join Hash join! O(n2) O(n)


2/1/13 NEDB '13 17

SQL Database



SQLQuery

Application ServerApplication Distribution

Running Examplediscount = executeQuery("select discount from

customers where id = " + cid);

totalAmount = orderTotal * (1 – discount);

credit = executeQuery("select credit from customers where id = " +

cid);

if (credit < totalAmount)printToConsole("Only " + credit + " in account!");

elseexecuteUpdate("update customer set credit = " +

(credit – totalAmount) + " where id = " + cid);

2/1/13 18NEDB '13

Actual Executiondiscount = executeQuery("select discount from




cid);




2/1/13 19NEDB '13

DB

APP

APP

DB

DB

Actual Executiondiscount = executeQuery("select discount from




cid);




2/1/13 20NEDB '13

network communication




DB

APP

APP

DB

DB

Speeding up Executiondiscount = executeQuery("select discount from customers

where id = " + cid);


credit = executeQuery("select credit from customers where id = " + cid);


elseexecuteUpdate("update customer set credit = " + (credit – totalAmount) + " where id = " + cid);

2/1/13 21NEDB '13

DB

APP

DB







2/1/13 22NEDB '13

data dependencyDB

APP

DB

control dependency







2/1/13 23NEDB '13

DB ServerDB

APP

DB

control dependency

data dependency

Introducing Pyxis• “Store-procedurizes” DB apps and

pushes computation to the DB

• Adaptively controls the amount of computation pushed to DB for optimal performance

• No programmer intervention required

2/1/13 24NEDB '13

Using Pyxis

2/1/13 NEDB '13 25

How Pyxis Works

2/1/13 NEDB '13 26

InstrumentPartition

Monitor

App Server DB Server

Deploy

JavaSQLJavaSQLJava

Java

JavaSQLJavaSQL

SQLJava

JavaSQLJavaSQLJavaSQLJava

SQL

Java

control transfer

How Pyxis Works

2/1/13 NEDB '13 27

Monitor

App Server DB Server

DeployJava

Java

SQLJava

SQL

SQLJava

control transfer

InstrumentPartition



Generating Program Partitions

• Deploy and profile application as-is• Construct a dependence graph of

program statements– captures both control and data flow

• Formulate linear program from profile data and dependence graph– solution gives a partitioning of the

source code

2/1/13 NEDB '13 28

Experiments

2/1/13 NEDB '13 30

100 300 500 700 900 1100 13005

10

15

20

25JDBCManualPyxis

Average Thruput (xact / s)

Ave

rage

Lat

ency

(m

s)TPC-C on 16-core DB

machine

2/1/13 32NEDB '13

Pyxis generated implementation:

3x latency reduction1.7x thruput increase

StatusQuoEase DB application development

Convert imperative program statements into declarative SQL

Fully automatic code partitioning using application and server characteristics

db.csail.mit.edu/statusquo

2/1/13 NEDB '13 33

Backup

2/1/13 NEDB '13 34

making database applications perform using program analysis

Documents