case-erp

6
A Case Study on Code Generation of an ERP System from Event-B estor Cata˜ no Madeira Interactive Technologies Institute M-ITI Funchal, Caminho da Penteada, 9020-105, Portugal Email: [email protected] Tim Wahls Department of Mathematics and Computer Science Dickinson College Carlisle, Pennsylvania USA 17013 Email: [email protected] Abstract—Most code generation tools for Event-B are de- signed for generating small, in-memory applications such as embedded controllers. In this work, we investigate whether the EventB2SQL tool (Wang and Wahls, LNCS 8702) can generate satisfactory code for the OpenBravo POS ERP (Enterprise Resource Planning) system by replacing the database core of the system with code generated from an Event-B model. We describe our methodology for generating code with EventB2SQL and enhancements to EventB2SQL that improve the performance of the generated code, and present empirical results and a user study comparing the performance of OpenBravo POS as is and with its core replaced by code generated by EventB2SQL. Index Terms—Event-B, code generation, ERP, database I. I NTRODUCTION Event-B [1] and formal methods in general have seen rela- tively widespread use in the development of reactive systems such as railway controllers [2]. Such systems share a number of characteristics: they are relatively small, they do not manipulate large volumes of data, they do not need to have persistent state. That is, data manipulated by the system need not be saved to persistent storage. As such, most code generators for Event-B (as described in Section II-D) do not support generating applications that man- age large volumes of data in a persistent fashion. In this work, we conducted a case study designed to determine whether a new Event-B code generation tool – EventB2SQL [3] – can generate satisfactory code for such applications, and if not, whether the tool can be improved to do so. We do not expect that generated code can be made to run as fast as a good quality hand-coded implementation, so we set a tentative goal of achieving running times for the generated code that were within one order of magnitude of the times for hand-written code for the same database tasks. We chose to work with an Enterprise Resource Planning (ERP) system called OpenBravo POS (Point of Sale) [4] for a number of reasons. OpenBravo POS has a significant database component. OpenBravo POS is an open-source system, providing us with the access needed for comparing performance in an easily automated way OpenBravo POS is widely used, with over 274,000 down- loads of the current version (2.30.2) as of Jan. 2015 [4]. perhaps most importantly, the authors had already written an Event-B model (the Inventory Model) for the core OpenBravo POS functionality, and discharged the asso- ciated proof obligations with the Rodin platform. While we added a few events for generating reports during the case study, most of the model was written before the development of EventB2SQL, and so is not biased towards executability. To demonstrate the applicability of Event-B in reasoning about business systems, we added an invariant to the Inventory Model that requires the resale price of each product to be greater than the purchase price of that product. We then adjusted the guards of the appropriate events, used the Rodin provers to verify that no state of the model violated the invariant, generated code for the model using EventB2SQL, and then wrote JUnit tests to confirm that the generated code maintained the invariant. II. BACKGROUND A. Event-B Event-B [1] is a method for modeling and analysing soft- ware systems based on Refinement Calculus. Event-B relies on Action Systems [5], a formalism describing the behaviour of a system by the atomic actions that the system carries out. An Event-B model is comprised of a static and a dynamic part. The static part defines the context of the system (constants and carrier sets and their properties) and the state space (variables, their types and properties). Carrier sets introduce types (sets of values) that are used by the model, but are not further defined. The dynamic part defines all possible events, including one computing the initial state. An event is comprised of a guard and a set of actions. The guard represents conditions that must hold in a state for the event to trigger. The actions compute new values for state variables, thus performing an observable state transition. B. OpenBravo POS OpenBravo [4] is a Point of Sale application (POS) that enables retailers to buy and to sell goods, to create purchase and sales orders, and to generate inventory and many other reports. OpenBravo POS is an open source system that runs under Mac OS, Linux, and Windows. The implementation

Upload: phulam146

Post on 15-Jul-2016

216 views

Category:

Documents


0 download

DESCRIPTION

Case ERP paper

TRANSCRIPT

Page 1: case-ERP

A Case Study on Code Generation of an ERPSystem from Event-B

Nestor CatanoMadeira Interactive Technologies Institute

M-ITIFunchal, Caminho da Penteada, 9020-105, Portugal

Email: [email protected]

Tim WahlsDepartment of Mathematics and Computer Science

Dickinson CollegeCarlisle, Pennsylvania USA 17013

Email: [email protected]

Abstract—Most code generation tools for Event-B are de-signed for generating small, in-memory applications such asembedded controllers. In this work, we investigate whether theEventB2SQL tool (Wang and Wahls, LNCS 8702) can generatesatisfactory code for the OpenBravo POS ERP (EnterpriseResource Planning) system by replacing the database core ofthe system with code generated from an Event-B model. Wedescribe our methodology for generating code with EventB2SQLand enhancements to EventB2SQL that improve the performanceof the generated code, and present empirical results and a userstudy comparing the performance of OpenBravo POS as is andwith its core replaced by code generated by EventB2SQL.

Index Terms—Event-B, code generation, ERP, database

I. INTRODUCTION

Event-B [1] and formal methods in general have seen rela-tively widespread use in the development of reactive systemssuch as railway controllers [2]. Such systems share a numberof characteristics:

• they are relatively small,• they do not manipulate large volumes of data,• they do not need to have persistent state. That is, data

manipulated by the system need not be saved to persistentstorage.

As such, most code generators for Event-B (as described inSection II-D) do not support generating applications that man-age large volumes of data in a persistent fashion. In this work,we conducted a case study designed to determine whether anew Event-B code generation tool – EventB2SQL [3] – cangenerate satisfactory code for such applications, and if not,whether the tool can be improved to do so. We do not expectthat generated code can be made to run as fast as a goodquality hand-coded implementation, so we set a tentative goalof achieving running times for the generated code that werewithin one order of magnitude of the times for hand-writtencode for the same database tasks. We chose to work with anEnterprise Resource Planning (ERP) system called OpenBravoPOS (Point of Sale) [4] for a number of reasons.

• OpenBravo POS has a significant database component.• OpenBravo POS is an open-source system, providing us

with the access needed for comparing performance in aneasily automated way

• OpenBravo POS is widely used, with over 274,000 down-loads of the current version (2.30.2) as of Jan. 2015 [4].

• perhaps most importantly, the authors had already writtenan Event-B model (the Inventory Model) for the coreOpenBravo POS functionality, and discharged the asso-ciated proof obligations with the Rodin platform. Whilewe added a few events for generating reports duringthe case study, most of the model was written beforethe development of EventB2SQL, and so is not biasedtowards executability.

To demonstrate the applicability of Event-B in reasoningabout business systems, we added an invariant to the InventoryModel that requires the resale price of each product to begreater than the purchase price of that product. We thenadjusted the guards of the appropriate events, used the Rodinprovers to verify that no state of the model violated theinvariant, generated code for the model using EventB2SQL,and then wrote JUnit tests to confirm that the generated codemaintained the invariant.

II. BACKGROUND

A. Event-B

Event-B [1] is a method for modeling and analysing soft-ware systems based on Refinement Calculus. Event-B relieson Action Systems [5], a formalism describing the behaviourof a system by the atomic actions that the system carries out.An Event-B model is comprised of a static and a dynamic part.The static part defines the context of the system (constants andcarrier sets and their properties) and the state space (variables,their types and properties). Carrier sets introduce types (sets ofvalues) that are used by the model, but are not further defined.The dynamic part defines all possible events, including onecomputing the initial state. An event is comprised of a guardand a set of actions. The guard represents conditions that musthold in a state for the event to trigger. The actions computenew values for state variables, thus performing an observablestate transition.

B. OpenBravo POS

OpenBravo [4] is a Point of Sale application (POS) thatenables retailers to buy and to sell goods, to create purchaseand sales orders, and to generate inventory and many otherreports. OpenBravo POS is an open source system that runsunder Mac OS, Linux, and Windows. The implementation

Page 2: case-ERP

follows the MVC (Model-View-Controller) software designpattern [6], so it was convenient for us to replace OpenBravoPOS’s Model with the code generated by EventB2SQL fromthe Inventory Model of Section III.

C. EventB2SQL

EventB2SQL [3], [7] is a Rodin plugin [8] for generatingJava database applications (using JDBC) from Event-B mod-els. The entire state of the model is stored in the database,making it persistent. With a few exceptions, Event-B opera-tions are translated to SQL queries, which can then be executeddirectly over the database representation of the model’s state.Each event is translated as a database transaction, making thegenerated code suitable for use in multi-threaded and client-server applications. EventB2SQL can generate code for boththe MySQL and SQLite database engines.

An Event-B machine (abstract or refinement) and the carriersets, constants and axioms of any contexts that the machinesees are translated to a single Java class. Carrier sets aretranslated as both generic type parameters of the generatedclass and tables in the database, allowing the generated codeto manipulate and store Java objects. As described in thefollowing paragraphs, EventB2SQL uses integer identifiers torefer to tuples, sets and relations. The value of each constantand machine variable is stored in a column of type int in adatabase table that always has exactly one row. Each eventis translated to a Java method that executes as a databasetransaction. The method has return type boolean, returningtrue if the translation of the event guard was satisfied andthe method was able to execute, or false otherwise. If theguards are satisfied, the method executes database updatescorresponding to the actions in the body of the event.

The original version of EventB2SQL (as described in [3])used a highly normalized database structure that guaranteedthat no object was ever stored in the database more than once.To achieve this, all tuples, sets and relations (that is, any Event-B structure more complex than an integer or boolean) wereassigned unique integer identifiers and were always referredto indirectly via this identifier.

The fact that objects were always referred to via identifiersmeant that all intermediate results had to be stored in thedatabase, as otherwise the identifier would have nothing torefer to. As Event-B operations are frequently composed,this resulted in significant database overhead and negated anyadvantage gained from the lack of redundancy in the databasedesign. Additionally, checking whether each new tuple, setand relation object already existed in the database imposed asignificant performance overhead, as did the frequent retrievalsof the value of a tuple, set or relation by identifier. As such,we were concerned that the code generated by EventB2SQLwould run too slowly to be useful in practice, particularly forlarger and more complex systems.

D. Related Work

Of the many code generators for B and Event-B, we areaware of only three that can handle models written at a similar

level of abstraction as those translatable by EventB2SQL. TheEventB2Java tool [9] can translate abstract Event-B machinesto JML-annotated Java implementations. The JeB tool [10]translates Event-B models to JavaScript for animation andsimulation. Neither of these tools provide support for datapersistence. The UB2SQL tool [11], [12] uses B modeling andrefinement in the development of database applications. Theapplication is specified in IS-UML – a UML variant in whichoperations are specified via B-annotated UML diagrams. Thetool translates the data model from UML to B and then toSQL tables. An automated refinement process translates theabstract B model to a relatively concrete one. Finally, the userimplements the concrete model using JDBC code generated bythe tool. One significant difference from EventB2SQL is thatwhen using UB2SQL, the user essentially defines the structureof the database in UML, while EventB2SQL generates thestructure from the Event-B model. Further, UB2SQL does not(yet) produce full Java/JDBC code for the model automatically.

Numerous case studies on the use of Event-B in developingindustrial software systems have been completed. For example,the authors of [13] use Event-B to develop and verify avehicle control system for the platooning problem (one ormore autonomous vehicles following a lead vehicle). Theystress the importance of proper tool support in applying formalmethods. The authors of [14] used Event-B to redevelop thecore of a commerical air traffic information system. Neitherof these case studies employed a code generator, and neitherconsidered systems with persistent state.

III. THE INVENTORY MODEL

The Inventory Model is an Event-B model of thedatabase core of OpenBravo POS. In particular, the InventoryModel stores information about customers, products, productproviders, product inventory, purchase and sales orders, prod-uct taxes, payments, and provides a series of reports withstatistics about inventory, payments and the like.

The Inventory Model is organised into one abstract machine(Event-B model) and four machine refinements. The abstractmachine includes products, customers and providers. Productsare organised into three categories:

1) resale products with associated purchase and sale prices2) manufactured products with a sale price only3) consumable products with a purchase price onlyThe invariant on the sale and purchase price of resale

products mentioned at the end of Section I is shown inFigure 1. pprice is a function mapping a product and provider(supplier) pair to the purchase price, and sprice is a functionmapping a product to the sale price. As the original version ofOpenBravo POS does not require the sale price of a productto be greater than the purchase price, we did not includethis invariant in the version of the Inventory Model used inbenchmarking and the user study.

The first machine refinement introduces sales and purchaseorders with the aid of sets sorder and porder , respectively,and additional relations defined over these sets. Purchaseorders are defined over products and providers, and sales

Page 3: case-ERP

inv29 ∀prod · prod ∈ resale ⇒(∀prov · (prod 7→ prov) ∈ dom(pprice) ∧

prod ∈ dom(sprice)⇒pprice(prod 7→ prov) < sprice(prod))

Fig. 1. An Event-B invariant stating that the sale price of each resale productis greater than the purchase price from any supplier.

event get ticketany ordwhere grd1 ord ∈ sorderthen act1 the ticket := {oi ·oi ∈ sorderitem sorder ∼ [{ord}] |(sorderitem product(oi) 7→ ((sprice(sorderitem product(oi)) 7→sorderitem quantity(oi)) 7→sprice(sorderitem product(oi))∗sorderitem quantity(oi)))}

end

Fig. 2. An Event-B event for retrieving a ticket.

orders over products and customers. The second refinementorganises products and customers into different categories andintroduces product taxes based on those categories. The thirdrefinement features payment for sale and purchase orders,using total boolean functions cashin and cashout from sorderand porder , respectively. The fourth refinement adds reportson the stock (inventory of products) to the Inventory Model.

Event get ticket (Figure 2) from the last machine refine-ment uses a set comprehension to assign the price, numberof units and total price for each product in a sales order tomachine variable the ticket . sorderitem sorder is a functionfrom order items (ticket lines) to sales orders, so its inverse(∼) is used to find all order items for the order. Func-tions sorderitem product and sorderitem quantity returnthe product and quantity for an order item.

IV. THE CASE STUDY

A. Methods

Our first step in conducting the case study was to use theoriginal version of EventB2SQL to translate the InventoryModel to a database application, and then to write JUnit testsfor each method in the generated code. This checked that thecode generated by the tool was correct (which was the casefor our test suite) and allowed us to measure the performanceof the generated code. Unfortunately, the generated code wasquite slow – it took about 10 minutes to add 100 customersto the database on our test machine.

In order to measure our progress as we implemented variousoptimizations and enhancements to EventB2SQL (describedin Section IV-B), we chose 5 OpenBravo POS operations thatseemed likely to be frequently used in practice: adding a new

customer, adding a new product, adding a ticket (sales order),retrieving a ticket by ticket identifier (Figure 2), and retrievingthe inventory of a product by name and location.

Next, we implemented code for randomly generating cus-tomers, products and tickets, with each ticket randomly con-taining between 1 and 10 ticket lines (products, each witha quantity and unit price). Then, we implemented a tim-ing framework for executing the chosen operations for boththe existing OpenBravo POS implementation and the codegenerated by EventB2SQL from the Inventory Model. InSection V-A, we present these timing results for various datasizes for the existing OpenBravo POS implementation, thecode generated by the original version of EventB2SQL, andthe code generated by the current version of EventB2SQL.

For an interactive system such as OpenBravo POS, userperceptions of the responsiveness of the system are moreimportant than running times for batch processing. Hence, wealso conducted a small user study in which human subjectswere asked to use both the hand-coded OpenBravo POSsystem and a version of OpenBravo POS with its core databasehandling code replaced by code generated by EventB2SQL toperform a small set of tasks. See Section V-B for details.

B. Enhancements to EventB2SQL

Over the course of the case study, we made significantperformance and functionality enhancements to EventB2SQL.After completing each enhancement, we re-ran our JUnit testsuite as a quick check of correctness.

1) Performance Enhancements to EventB2SQL: Our firstperformance improvement was achieved by eliminating stor-age of intermediate results and most checks for duplicateobjects. As described in Section II-C, the original versionof EventB2SQL always referenced sets, relations and tuplesvia an integer identifier. The code generated for each Event-B operation used these identifiers to retrieve the values ofoperands, and “packaged” its result as an identifier. If the resultalready existed in the database, the generated code returnedthe identifier of the existing object. If not, the generated codeadded the object to the database with a new identifier.

In the current version of EventB2SQL, the code generatedfor each Event-B operation assumes that the query generatedfor each operand returns the value of that operand. Queryresults need only be “packaged” as identifiers when the resultis assigned to an Event-B machine variable, or when the resultis to be stored as part of some larger object (i.e. an elementof a powerset). After this modification, the time needed toadd 100 customers using the generated code dropped from 10minutes down to about 2 minutes.

Our next step was to eliminate checks for whether an objectassigned to a machine variable already existed in the database.This introduced the possibility of some database redundancy(for example, if some set is both the value of a machinevariable and an element of a powerset), but this disadvantage isoutweighed by running time improvements. This change alsoallows objects assigned to machine variables to be mutated,permitting further optimizations. In the code generated by the

Page 4: case-ERP

previous version of EventB2SQL, all objects were immutable,and an object identifier always referred to the same immutableobject. Objects were never duplicated, so the same objectidentifier could occur (for example) as the value of a machinevariable and as an element of a powerset – thus creating analias. In this situation, mutating the object referred to by themachine variable would (incorrectly) change the powerset.

The optimizations made possible by mutation include:• v := v ∪ s is now executed by inserting elements of s

that do not already occur in v into the value of v.• v := v \ s by deleting elements of s from v• v := v ∩ s by deleting elements of v \ s from v• v := sC− v (domain subtraction) by deleting tuples from

v with first component equal to some value in s• v := sCv (domain restriction) by deleting tuples from v

with first component equal to some value in dom(v) \ s• similar optimizations for C− (relational override), B−

(range subtraction) and B (range restriction).The primary benefit of all 8 of these optimizations is that theymake the running time depend on the number of elementsto be inserted or deleted, rather than the overall size of theset or relation. For example, without the optimization for ∪,executing v := v ∪ s requires constructing a completely newobject for v ∪ s in the database, even if this object containsonly a few more tuples than the value of v. To correctlyimplement the simultaneous assignment semantics of Event-Bthe generated code calculates the set of elements to be insertedinto or deleted from the value of each machine variable beforeperforming any database modifications.

At this point, profiling the generated code revealed thatthe remaining running time was dominated by executing setextensions (literal set expressions). The code generated by theoriginal version of EventB2SQL inserts all elements of a setextension into a temporary (in-memory) table, and creating atemporary table takes approximately 100 milliseconds on ourtest machine. To avoid the creation of temporary tables, thequery generated for small set extensions (9 elements or less)is now the union of the individual queries for each element.As set extensions in Event-B models typically contain only afew elements, the generated code now rarely needs to createa temporary table when executing a set extension.

The time needed to add 100 customers to the database wasnow down to just under 9 seconds. Full benchmarking resultsare presented in Section V-A.

2) Other Enhancements to EventB2SQL: The current ver-sion of EventB2SQL features an option that allows a carrierset called STRING to be translated automatically as typejava.lang.String, and a constant nullstring of typeSTRING to be translated as "". This does not quite havethe effect of making String a built-in type in Event-B (asstring operations such as concatenation and substring are notavailable for use in modeling), but it does significantly simplifywriting client code for code generated from Event-B modelsthat work with text (such as the Inventory Model).

The get ticket event in the Inventory Model (Figure 2) iswritten using a set comprehension over the lines of the ticket.

Thus, we implemented code generation for set comprehensionsin EventB2SQL (for cases where the bound variable rangesover a known and finite set). The generated code:

• evaluates the query for the domain of the bound variable,• uses a loop to execute the query for the expression

over the bound variable for each element of the domain,storing the results in a temporary table, and

• returns a query that selects all elements of that table.Hence, EventB2SQL can now generate code for the get ticketevent, although this code is quite expensive to execute. SeeSection V-A for timing results.

C. Lessons Learned

Although all proof obligations for the Inventory Model hadpreviously been discharged, we found and corrected a numberof errors and omissions in the model over the course of thecase study:

• the event for adding a manufactured product did not addthe product to a warehouse, so there was no way toincrease the inventory of manufactured products

• taxes were not being calculated correctly• resale and consumable products could have only one

provider, which did not match the behavior of OpenBravo• there was no way to add multiple order items (lines) to

a sales order (ticket).These kinds of errors emphasize the need for tools such asEventB2SQL that facilitate validation (checking that the modelcaptures user requirements) in addition to verification tools.

Another important lesson is the need to try tools on real-world examples in addition to the kinds of toy examples oftenemployed in academic work. Although we were concernedabout the performance of the code generated by EventB2SQL,we had not actually observed any performance issues in ourtest suites. After this case study, it is clear that the performanceof code generated by the original version of EventB2SQL isnot acceptable for most applications.

V. RESULTS

A. Benchmarking

We benchmarked the 5 chosen OpenBravo POS opera-tions using the hand-coded OpenBravo POS implementation(OpenBravo), the code generated by the original version ofEventB2SQL (EventB2SQL(O)), and the code generated bythe current version of EventB2SQL (EventB2SQL(C)). For theOpenBravo and EventB2SQL(C), each operation was executedfor data sizes 1000, 2000, 3000 . . . 20000. For adding andgetting tickets, the number of customers was set to 20%of the number of tickets and the number of products wasset to 10% of the number of tickets. Running times weremeasured using Java’s System.currentTimeMillis()method. Each operation was repeated 5 times for each datasize, and the times reported below are averages over these 5runs. The code generated by EventB2SQL(O) is much slower,so it was not feasible to execute the benchmarks over thesame range of data sizes. Hence, timing results for this code

Page 5: case-ERP

0

100

200

300

400

500

600

20 30 40 50 60 70 80 90 100

OpenBravoEventB2SQL(C)EventB2SQL(O)

Fig. 3. Timing results for adding customers for smaller data sizes.

0

200

400

600

800

1000

1200

1400

1600

1800

0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000

OpenBravoEventB2SQL(C)

Fig. 4. Timing results for adding customers for larger data sizes.

are for data sizes 20, 40, 60, . . . 100. All timing data wascollected on an otherwise idle Dell OptiPlex 745 with 4.8GB of memory and a dual-core Intel Pentium D 3.4 GHzprocessor. The machine was running Ubuntu 14.04 (64 bit),Java 1.8, MySQL 5.6 and OpenBravo POS 2.30.2.

In each of the following graphs, the X-axis is the data sizeand the Y-axis is the average running time in seconds. Dueto space limitations, we present only the graphs for addingcustomers and getting tickets.

Figures 3 and 4 give running times for adding customers.Because the times for EventB2SQL(O) are so large, Open-Bravo and EventB2SQL(C) are indistinguishable in Figure 3.In Figure 4, the time for EventB2SQL(C) is about 2.4 timesgreater than for OpenBravo for all data sizes. Adding acustomer in OpenBravo adds a row to one database table,while adding a customer using EventB2SQL(C) adds rows to11 tables (the table holding the variable customer , the carrierset CUSTOMER, and the tables holding the 9 functions thatreturn some attribute of a customer. Hence, running moreslowly by a factor of 2.4 is relatively good performance.

Running times for adding products follow a similar pattern.EventB2SQL(C) runs more slowly than OpenBravo by a factorof about 4.2 for 1000 products, and about 4.8 for 20000products. For retrieving product inventories, EventB2SQL(C)was slower by a factor of about 4.5 for all data sizes.

For adding and getting tickets, once again the poor perfor-mance of EventB2SQL(O) makes EventB2SQL(C) and Open-Bravo POS indistinguishable for small data sizes (Figure 5).

0

500

1000

1500

2000

2500

3000

3500

20 30 40 50 60 70 80 90 100

OpenBravoEventB2SQL(C)EventB2SQL(O)

Fig. 5. Timing results for getting tickets for smaller data sizes.

0

1000

2000

3000

4000

5000

6000

7000

8000

0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000

OpenBravoEventB2SQL(C)

Fig. 6. Timing results for getting tickets for larger data sizes.

For adding tickets, running times for EventB2SQL(C) startat about 9.7 times greater than for OpenBravo POS, but thisfactor becomes smaller as the data size increases. OpenBravoPOS stores all ticket lines for a ticket as a BLOB (BinaryLarge OBject), so the time needed for adding a ticket doesnot depend on how many lines the ticket has. This is not thecase for EventB2SQL(C). Additionally, the code generated byEventB2SQL(C) stores each ticket line twice – once in thetable for the carrier set for ticket lines and once in the tablerepresenting the set of all lines that are included in tickets.

As shown in Figure 6, getting tickets is the one operationfor which we did not achieve goal of having code generatedby EventB2SQL(C) perform within an order of magnitude ofOpenBravo. EventB2SQL(C) is slower than OpenBravo bya factor of 7.1 for retrieving 1000 tickets, and by a factorof 22.1 for 20000 tickets. Retrieving each ticket line withEventB2SQL(C) requires a separate database query, and thetables storing ticket lines grow quickly with an average of 5.5lines per ticket. Additionally, the set comprehension used ingetting tickets (as shown in Figure 2) is expensive to execute.

B. User Study

We conducted a small user study to gain some intuition onthe usability of the generated code. Nine volunteers were askedto complete three tasks (adding a product, adding a customer,adding a sales order) with two versions of OpenBravo POS– the original hand-coded implementation, and a versionwith the database core replaced with the code generated by

Page 6: case-ERP

Task 0 1 2 3 4 5 Avg. RatingCustomer 2 5 2 0 0 0 1Product 2 6 1 0 0 0 0.89Sale 5 1 2 1 0 0 1

TABLE IUSER STUDY RESULTS FOR ADDING A CUSTOMER, ADDING A PRODUCT

AND REGISTERING A SALE.

EventB2SQL(C). Users retrieved all products and customersin the process of adding a sales order. The user study wasperformed on a dual-monitor system with one version ofOpenBravo POS on each monitor, so users could switchfreely between the two versions. Both versions used the sameMySQL installation, with a separate logical database (schema)for each. Each schema contained the same collection of 500customers, 100 products and 5000 tickets, and the schemaswere reset to this state for each user session. As they workedwith both versions, users were asked to rate the performancedifference for each task on a scale from 0 (no difference) to5 (extreme difference, slower version is unusable).

The results of the user study are presented in Table I. Thecolumn labeled 0 contains the number of users who did notnotice a performance difference for each task, column 1 thenumber of users who found a barely noticeable difference, andso on. The last column is the average rating for each task. Ineach case where a user noticed a difference, they identified theversion of OpenBravo POS that used generated code as beingslower. Hence, the users in the study found the version usinggenerated code to be slower for each task than the hand-codedversion, but only by a barely noticeable amount overall.

VI. FUTURE WORK AND CONCLUSION

A. Future Work

While we have made progress in improving the performanceof the code generated by EventB2SQL, more work in this areais needed. The most critical area for improvement identifiedduring this case study is in executing set comprehensions.

Code generation tools for Event-B must be sound, so thatall properties verified to hold for the Event-B model alsohold for code generated from that model. An important firststep in proving the soundess of the translation performed byEventB2SQL is proving that the SQL query generated for eachEvent-B operation is equivalent to that operation. We havecompleted a preliminary version of this proof by hand, usingthe formal SQL semantics given in [15].

We also plan a case study on using EventB2SQL to developan Android application, leveraging EventB2SQL’s ability togenerate code that uses SQLite databases.

B. Conclusion

Although the code generated by EventB2SQL for the Inven-tory Model is still slower than the corresponding hand-codedfunctionality in OpenBravo POS, our benchmarks show highlysignificant improvements in the performance of the generatedcode over the course of the case study. For four of the five

tasks chosen for benchmarking (all except retrieving tickets),we were able to achieve our goal of generating code that wasno more than an order of magnitude slower than the hand-coded implementation for all data sizes tested. As noted inSection V-A, the generated code is in general inserting intomany more tables than is the hand-coded implementation, soachieving significantly better performance for adding tickets,customers and products and for getting the inventory of aproduct does not seem feasible without radical changes tothe translation performed by the EventB2SQL tool. In ouruser study, a number of users were unable to detect anyperformance differences between the generated code and hand-coded implementation for modest data sizes, and on averagerated the performance differences as barely noticeable.

Applying formal methods in the development of systemsthat manage volumes of persistent data has advantages overtraditional software development – notably the ability to verifycorrectness and security properties of such systems. Businessrules can be verified in a similar manner. The ability toautomatically generate database designs and back-end codefor such systems saves considerable time in the developmentprocess, mitigating the costs of applying formal methods. Ofcourse, the generated code must exhibit acceptable perfor-mance. We have demonstrated that formal methods can beapplied successfully in the development of an ERP system,including the generation of code that is usable in practice.

REFERENCES

[1] J.-R. Abrial, Modeling in Event-B: System and Software Design. NewYork, NY, USA: Cambridge University Press, 2010.

[2] “Saet meteo: new driverless metro line 14 in paris (ratp),” SiemensTransportation Systems, 1998.

[3] Q. Wang and T. Wahls, “Translating Event-B machines to databaseapplications,” in Software Engineering and Formal Methods, ser. LNCS,D. Giannakopoulou and G. Salaun, Eds., vol. 8702, 2014, pp. 265–270.

[4] “OpenBravo POS,” http://sourceforge.net/projects/openbravopos/, 2015.[5] R. Back and K. Sere, “Stepwise Refinement of Action Systems,”

Structured Programming, vol. 12, pp. 17–30, 1991.[6] E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design patterns:

elements of reusable object-oriented software. Boston, MA, USA:Addison-Wesley Longman Publishing Co., Inc., 1995.

[7] Q. Wang and T. Wahls, “The EventB2SQL tool homepage,” 2015, http://users.dickinson.edu/∼wahlst/eventb2sql/eventb2sql.html.

[8] M. J. Butler, C. B. Jones, A. Romanovsky, and E. Troubitsyna, Eds.,Rigorous Development of Complex Fault-Tolerant Systems [FP6 IST-511599 RODIN project], ser. LNCS. Springer, 2006.

[9] V. Rivera, N. Catano, T. Wahls, and C. Rueda, “Code generation forEvent-B,” Software Tools for Technology Transfer, pp. 1–22, May 2015.

[10] F. Yang, J.-P. Jacquot, and J. Souquieres, “JeB: Safe simulation of Event-B models in JavaScript,” 2013 20th Asia-Pacific Software EngineeringConference (APSEC), vol. 1, pp. 571–576, 2013.

[11] A. Mammar and R. Laleau, “UB2SQL: A tool for building databaseapplications using UML and B formal method.” Journal of DatabaseManagement, vol. 17, no. 4, pp. 70–89, 2006.

[12] ——, “From a B formal specification to an executable code: Applicationto the relational database domain,” Inf. Softw. Technol., vol. 48, no. 4,pp. 253–279, Apr. 2006.

[13] F. Yang and J.-P. Jacquot, “Scaling up with Event-B: A case study,” inThird NASA Formal Methods Symposium, M. Bobaru et al., Eds., 2011.

[14] A. Rezazadeh, N. Evans, and M. Butler, “Redevelopment of an industrialcase study using Event-B and Rodin,” in Proceedings of the 2007International Conference on Formal Methods in Industry, ser. FACS-FMI’07. British Computer Society, 2007.

[15] M. Gogolla, “Formal semantics of SQL,” in An Extended Entity-Relationship Model, ser. LNCS. Springer, 1994, vol. 767, pp. 99–120.