providing a web view for monitoring erlang systems
TRANSCRIPT
IT 12 009
Examensarbete 30 hpApril 2012
Providing a web view for monitoring Erlang systems
Nicklas Nordenmark
Institutionen för informationsteknologiDepartment of Information Technology
Teknisk- naturvetenskaplig fakultet UTH-enheten Besöksadress: Ångströmlaboratoriet Lägerhyddsvägen 1 Hus 4, Plan 0 Postadress: Box 536 751 21 Uppsala Telefon: 018 – 471 30 03 Telefax: 018 – 471 30 00 Hemsida: http://www.teknat.uu.se/student
Abstract
Providing a web view for monitoring Erlang systems
Nicklas Nordenmark
Tools for monitoring Erlang nodes exist but they have not aged well. With the help of new web technologies such as HTML5, JavaScript and CSS3 more modern interfaces can be developed. However, to combine an Erlang backend with a web front-end requires a very broad skill set. This thesis aims to present a prototype framework to simplify the construction of web based interfaces for Erlang developers. The framework consists of multiple layers to achieve modularity (such as supporting multiple Erlang web servers). Functionality is implemented in so called components (with similarities to an MVC structure) that each consists of an Erlang module, HTML templates, a CSS file, a JavaScript file and a config file. While the main purpose was to implement monitoring functionality, the framework is general enough to implement custom functionality.
Evaluation was conducted using usability testing and this helped to identify problems which led to a change in the framework structure.
Tryckt av: Reprocentralen ITCIT 12 009Examinator: Anders BerglundÄmnesgranskare: Justin PearsonHandledare: Adam Lindberg
Popularvetenskaplig sammanfattning
Kompletta datorsystem bestar idag av ett flertal program och kan i sin tur avenvara uppbyggda av flera mindre datorsystem. De olika programmeringssprakenger ofta tillgang till statusinformation om systemets halsa och belastning. Utoverstatusinformation finns ofta ocksa information om vilka delsystem som kors ochvad de har for uppgift just nu. Denna information presenteras pa traditionellt satti textbaserad form och hamtas fran ett antal olika platser. Detta gor det svart attmed traditionella verktyg fa en overblick over statusen pa storre system. Maletmed detta examensarbete ar att rada bot pa detta genom att foresla ett mod-ernare webbaserat granssnitt for att presentera statusinformation om datorsystemgrafiskt.
Utveckling pa webben blir allt mer populart och idag ar det mojligt att produc-era bade interaktiva och eleganta granssnitt som samtidigt ar oberoende av oper-ativsystem. I samband med att nya teknologier for utveckling av webbgranssnittslapps sa okar dock aven kraven pa utvecklarna. Fram tills for nagra ar sedan vardet inte sallan samma personer som arbetade bade pa server-sidan och klientsidanmedan det idag kan vara svart att hitta personer med sa pass bred kompetens.Det ar idag inte ovanligt att olika utvecklare fokuserar pa granssnitt och presen-tation respektive det underliggande systemet med databas och leverans/hanter-ing av data. Att folja med i de senaste trenderna och framstegen inom webb(JavaScript/jQuery, HTML5 och CSS3) kan vara svart att kombinera med enspetskompetens inom databaser och transportformat (till exempel JSON). Dettaexamensarbete syftar till att presentera ett ramverk for att forenkla utvecklingenav sadana granssnitt (framst) for anvandare av programmeringsspraket Erlang. Iteorin kan detta ramverk dock appliceras aven pa andra programmeringssprak.
Olika metoder for att utvardera ramverket undersoktes och i slutandan anvandessa kallad usability testing. Denna metod gar ut pa att lata testpersoner utfora ettantal givna scenarier pa produkten som onskas testas. De scenarier som togs frambaserades pa riktlinjer som var relevanta for utvarderingar av produkter sasomdetta ramverk. Dessa scenarier var vardefulla stod for testarna och hjalpte till attidentifiera och belysa en del problem som ledde till en forandring av ramverketsstruktur. Tanken var att utfora testning pa det har sattet i iterationer men pagrund av tidsbrist sa hanns bara en iteration med.
Tack vare utvarderingen sa kunde en protyp av ett generellt ramverk tas framsom stodjer utvecklingen av fristaende och mer specifika komponenter. Ramverketblev val mottaget hos testarna och har potential att kunna utvecklas till ett systematt monitorera Erlang-system.
3
Contents
1 Background 81.1 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.2 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2 Methodology 92.1 Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3 System Overview 123.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.2 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4 Implementation 134.1 JSFW Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.2 JSFW Webserver API . . . . . . . . . . . . . . . . . . . . . . . . . 144.3 JSFW JavaScript API . . . . . . . . . . . . . . . . . . . . . . . . . 154.4 JSFW Erlang API . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.5 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164.6 JSFW Webserver Plugins . . . . . . . . . . . . . . . . . . . . . . . . 174.7 JSFW Components . . . . . . . . . . . . . . . . . . . . . . . . . . . 174.8 Erlang module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174.9 JavaScript file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.10 CSS file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.11 Constructing a Component . . . . . . . . . . . . . . . . . . . . . . . 19
5 Evaluation 205.1 Evaluation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 215.2 Evaluating JSFW . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235.4 Resolution of Problems . . . . . . . . . . . . . . . . . . . . . . . . . 24
6 Conclusion and discussion 256.1 Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
7 Future Work 267.1 WebSockets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267.2 Support for Additional Webservers . . . . . . . . . . . . . . . . . . 267.3 Integration of a template engine . . . . . . . . . . . . . . . . . . . . 26
8 References 27
5
A Scenarios 29A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29A.2 Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29A.3 Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
B Questions 30
C Framework Documenation 31C.1 Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31C.2 URL Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32C.3 Component structure . . . . . . . . . . . . . . . . . . . . . . . . . . 32C.4 JavaScript API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35C.5 Erlang API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
D Source Code 36D.1 Cowboy Web server plugin . . . . . . . . . . . . . . . . . . . . . . . 36D.2 Misultin Web server plugin . . . . . . . . . . . . . . . . . . . . . . . 37D.3 Example Components . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6
1 Background
As systems grow more and more complex the need for easy-to-use monitoring toolsincreases. The size and number of components in a system grow and gaining anoverview becomes more di�cult. Traditionally, Erlang nodes are usually moni-tored via the console. This means querying the system using Erlang functions orparsing log files to diagnose the system. Graphical monitoring applications (e.g.WebTool [1] and Appmon [2]) are available but their interfaces have not aged welland most will not work remotely. The maturing front end technologies includingHTML5, CSS3 and JavaScript are becoming increasingly popular as they providea multitude of features that allow developers to build interactive and attractiveinterfaces.
1.1 Related work
Functionality wise, the most relevant related works are Appmon, WebTool andBigWig. Appmon (short for application monitor) is a tool that allows users tomonitor processes running on the local machine. Process information, such asmemory usage, is displayed in a graphical interface. WebTool allows users to startweb based Erlang tools such as a web version of Appmon. WebTool also acts asa framework for developing new web based tools. Tools developed using WebToolsimply output raw HTML code from Erlang modules. This thesis takes some ofits inspiration from an entry in the Spawnfest competition [3] called BigWig [4].It is a web-based monitoring tool for Erlang nodes where the user can get basicinformation about the node it runs on. The information presented is accessiblethrough the Erlang shell but is here presented in a much more user-friendly way,without the need for scrolling through terminal output in order to find what isbeing sought after. The ambition of BigWig was to supersede monitoring toolssuch as WebTool [1] and Appmon [2].
1.2 Problem Description
As front-end development becomes more and more sophisticated, the need fordedicated interface developers increases. In many cases it gets harder to implementfront end systems with an existing team since they need a knowledge of moderntechnologies in both the front end and back end. This paper presents a prototypeimplementation of a library that will aid the Erlang developer seeking to developfairly advanced web based graphical user interfaces where back-end data is to bepresented. The ambition is to allow these developers to present the same type ofinformation as Appmon does but in a web based interface. This is essentially whatWebTool does but here we try to take this further by utilizing modern technologies
7
such as HTML5, jQuery, CSS3 and JSON. This is a somewhat apt description ofthe BigWig project. However, unlike the BigWig project, we aim to produce aframework that is so general that people can develop their own custom componentsto suit their needs.
2 Methodology
This project was mostly conducted in an iterative manner with the developmentof individual components as foundation. The initial idea was to try to focus onthe realization of components similar to those prototyped by the BigWig project,since they cover most basic monitoring setups.
2.1 Tools
2.1.1 Erlang
Erlang is a general-purpose programming language and runtime environment de-veloped by Ericsson. It has built-in support for concurrency, distribution andfault tolerance [5, 6]. In 1998 Ericsson decided to release Erlang as open sourceand today it is available online1.
2.1.2 JavaScript
JavaScript2 is a widely used scripting language primarily used in front-end webdevelopment. Lately, JavaScript has also gained ground on the server-side thanksto frameworks such as Node.js3.
2.1.3 jQuery
jQuery4 is a JavaScript library that claims to make many JavaScript operations(e.g. document traversal and Ajax functionality) easier. jQuery was used to allowchanges in the interface layout without reloading the web page.
2.1.4 JSON
There was a need to transform data to a uniform format that could be under-stood by both Erlang and JavaScript. JSON5 was chosen as this data transport
1http://www.erlang.org/2https://developer.mozilla.org/en/JavaScript3http://nodejs.org/4http://jquery.com/5http://json.org/
8
format since it is so widely used and implementations exist for many di↵erentprogramming languages (including Erlang).
2.1.5 JSX
JSX [7] was chosen as the JSON encoding / decoding library since there exists aforked version of it [8] which includes support for additional Erlang terms such asprocess ids (PIDs), ports, functions and tuples. A comparative benchmark suitefor comparing di↵erent Erlang to JSON libraries exists [9] and although JSX isnot the top scorer when looking at the benchmark results [10] we decided to useit since it supports the Erlang terms mentioned above (that are commonly usedin monitoring systems) as well as being standalone (unlike Mochijson from theMochiweb web server [11]).
2.1.6 HTML
HTML use used to describe the basic layout of elements on any website. Apartfrom using basic HTML elements for presentation, HTML56 adds many new andinteresting features such as more semantic markup elements (header, footer, articleetc.), client-side validation and a History/State API for manipulating the browserhistory.
2.1.7 History.js
History.js7 is a JavaScript library that aims to supersede jQuery History8. Bothof these JavaScript libraries make it easier to make use of the History/State APIintroduced with HTML5 which is advantageous to do when writing asynchronousweb front ends. This was used to make sure that navigation of the browsing historyworked as intended while the website was changed/updated using JavaScript.
2.1.8 CSS
Cascading Style Sheets (CSS)9 is a mechanism for adding custom styling to webdocuments. A new version (CSS3) is currently under development10. A small frac-tion of styling for common elements (such as tables) was included in the framework.
6http://www.w3.org/TR/html5/7https://github.com/balupton/History.js/8https://github.com/balupton/jquery-history9http://www.w3.org/Style/CSS/
10http://www.w3.org/TR/CSS/
9
2.1.9 Web server
A web server listens and responds to requests from clients (usually end users usingbrowsers). Relevant content (usually HTML documents) is then sent back tothe client. Below the two applications that were used as web servers are brieflydescribed.
2.1.10 Cowboy
Cowboy11 is described as nothing more than a TCP acceptor pool, forwardingrequests to specified handlers. Included is an HTTP handler so Cowboy can easilybe set up as a web server. This was a target web server early on because of itsweb sockets support, which we initially planned on implementing.
2.1.11 Misultin
Misultin12 is an Erlang library for building fast lightweight HTTP servers. For thesame reason as Cowboy, this was also a target web server.
3 System Overview
BigWig is a good system and can manage the monitoring of the most basic sections(processes, applications, system load, etc.). However, if a user wants to managea specific part of his or her system (a list of online users in a chat system forexample), then he or she is at a loss with BigWig. Thus, a system that extendsthe functionality of BigWig was to be implemented. Based on this, a list of sys-tem requirements was constructed to guide the implementation and help identifycritical features.
3.1 Requirements
The requirements were categorized in the following four categories and by fulfill-ing these requirements we would have achieved a quite powerful framework formonitoring Erlang systems.
Components System
First of all, the system has to allow for specific functionality like the case describedabove with the list of users. By definition, we call the part that takes care of a
11https://github.com/extend/cowboy12https://github.com/ostinelli/misultin
10
specific functionality a component. This can quite naturally lead to a multitudeof components in a large system and therefore we wanted independence amongcomponents as well as the ability to easily insert or remove individual componentsin the system. This also means that the framework itself must come with a set ofexternally accessible API functions for the components to use.
User Interaction
Another interesting feature that opens up a lot of di↵erent implementations is theability for users to interact with the system. We want the framework to allowusers to send messages to Erlang processes as well as input to Erlang modules.
Integration
Since this is supposed to be a monitoring framework, we want it to integrate easilywith currently running systems that need to be monitored. Since we are exposinga web interface we need to be able to run on any (within reason) Erlang webserver that the current system might be using. Therefore, we need a web serverabstraction layer and plugins for the di↵erent web servers that can account forthis.
Modern Technology
Just like BigWig, this framework should be built using modern web technologiesincluding HTML5, CSS and JavaScript/jQuery.
3.2 System Architecture
Based on the requirements above, an abstract sketch of the system architecturewas constructed (see Figure 1).
Figure 1: Flow chart of the JSFW system on an abstract level.
11
4 Implementation
The framework is called JSFW (JavaScript FrameWork). The framework wasdeveloped in multiple layers to support flexibility and modularity. The first layeris a web server abstraction layer that is supposed to rid the core framework ofany web server dependency. Next layer contains the general functionality of theframework and takes care of forwarding requests to the correct component in thefinal layer. This final layer contains the components which in turn contain specificisolated functionality. These di↵erent layers are described in more detail below.
To allow for easy integration without database setup, persistent data (such ascomponent meta data) was stored in configuration files and a list of componentswas maintained by searching a path.
4.1 JSFW Core
This is the core system that takes care of HTTP request routing, component andplugin setup and provides an API for constructing the Webserver plugins and thecomponents. This part contains both JavaScript and Erlang since it does workboth client-side and server-side. In the core part you can also find a pre-builtcomponent that deals with displaying process information based on PIDs and alsoan interface for managing your available components. What is also included issome pre-defined CSS styling for common elements.
4.2 JSFW Webserver API
This is the web server abstraction layer where HTTP requests are translated toa common format (implemented as an Erlang record, #jsfw http). This requestthen propagates to JSFW core where it is handled and a reply is constructed(#jsfw reply). This reply is translated to a format that is understood by thecorresponding web server and then returned to the client via the web server. Themotivation for this translation was to enable support for multiple web servers.
4.2.1 #jsfw http
This record is filled with selected information extracted from the HTTP requestcoming from the web server. Only information that is required by JSFW core isextracted to avoid information duplication.
• The request method13 (only GET and POST are allowed at this stage).
13http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html
12
• The request path to allow us to respond to the request based on what resourceis being requested.
• Request query (the part after the ? in URLs) to include arguments with therequest. This can be used to specify in detail what information is requested(for example details=name,age).
An example record of this type could look something like this:
#jsfw_http{method="GET",path="/users/1",query="details=name,age"
}
4.2.2 #jsfw reply
The record holding the reply information is, just like the request record, straightforward and contains the following information.
• An HTTP status code14 to tell the web server how the request went.
• HTTP headers to tell the content type15 of the returned data.
• A body that contains the response from the HTTP request.
An example record of this type could look something like this:
#jsfw_reply{status=200,headers="Content-Type: application/json",body="This is a sample JSON string"
}
4.3 JSFW JavaScript API
The JSFW JavaScript API provides the components with an object oriented wayof constructing front-ends. At the top level a Body object is created to whichelements are added.
var body = new Body();
14http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html15http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
13
This object is actually just an abstraction layer on top of a regular <div> elementselected by jQuery which acts as a container for the added elements. The elementsavailable are common HTML elements such as <table>, <ul>, <span> and <div>and they are created and added to the Body element in a standardized manner.Here a <table> with column headers TableHeader1 and TableHeader2 is createdand added to the Body.
var table = new Table(["TableHeader1, TableHeader2"]);body.add(table);
The Body object also provides a function for filling those HTML elements withJSON data from a given source in the form of the fill json(source, element)function. Based on what type of element is filled, the JSON data is traversedaccordingly. This functionality is demonstrated with an example below, where the<table> element table is filled with JSON from "/json/users" (which would bethe output of the get json() function of the users component’s Erlang module).
fill_json("/json/users", table);
4.4 JSFW Erlang API
Since the Erlang part of components only need to transfer data from the back-endto the front-end the Erlang API is very limited and only consists of a single func-tion, arg to pid/1, that is used to convert binary arguments to existing Erlangprocess ids. In one of the example components, information about processes is dis-played. A user can request information for a specific process based on its processid by clicking on a link to that process in the list of processes. As an example, itcould be the process with process id 3. The Erlang backend would then be sent<<"3">> as input to its get json/1 function.
arg_to_pid(<<"3">>).
The above example would then return the process id <<3>> which can be used bythe Erlang module to retrieve additional information about the specified process.
4.5 Documentation
A rather extensive documentation on how to use JSFW and what API functionsare available is included in the package itself. This was actually implemented as aJSFW component and is accessible through the web interface.
14
4.6 JSFW Webserver Plugins
One of the goals with this framework is to have it easily integrate with di↵erentweb servers, and thus the need for a general web server layer arose. These pluginsare not actually too complex (see Appendix D) and they are supposed to translatethe incoming HTTP request to a uniform format that is supported and specifiedby JSFW Core. When the framework has dealt with the incoming request, a replyis returned to the plugin and it is up to the plugin to translate the response andreturn it to the client. These plugins are written in Erlang.
4.7 JSFW Components
The components contain the actual functionality and this is where the user imple-ments the specific functionalities. A component could for example list the onlineusers in a chat service. The components contain an Erlang module, a JavaScriptfile and a CSS file. See Figure 2 for a simple illustration of the component dataflow.
Figure 2: Inner workings of JSFW components.
4.8 Erlang module
The role of the component’s Erlang module is to provide data that in turn willbe presented by the component’s JavaScript. Two standard functions have to beimplemented, namely get json/0 and get json/1. The former one should returna tuple according to {ok, JSON} where JSON is a JSON encoding of the data thatthe user wishes to provide to the front-end (e.g. a list of usernames and user id’s inthe case of a component that lists users). The second function has the same purposeas the first one except that it also accepts an argument extracted from the trailingpart of the request URL (e.g. in the case http://hostname/json/users/3, theErlang binary <<"3">> will be sent as argument to the get json/1 function of theusers component).
15
4.9 JavaScript file
The JavaScript file contains a JavaScript object which consists of three standard-ized functions: init(), handle(argument) and refresh(). Below is an exampleof a simple component.
var users = new function() {var table;
this.init = function() {var body = new Body();table = new Table(["TableHeader1", "TableHeader2"]);fill_json("/json/users", table);body.add(table);
}this.handle = function(argument) {
var body = new Body();var span = new Span("Let’s display the argument: " + arg);body.add(span);
}this.refresh = function() {
fill_json("/json/users", table);}
}
init() is called when the component is clicked in the menu and handles theinitial display of data on the front-end. Typically (and as illustrated by the ex-ample above), this function sets up the initial appearance of the component bycreating suitable elements and fetching the JSON output from the Erlang module.handle(argument) is called when a link printed by a component is clicked. If thelink leads the user to http://hostname/users/3 then 3 is the argument that issent to the handle(argument) function. Generally speaking, handle(argument)takes care of setting up the front-end when an argument is given. Automaticrefreshing of components is supported by JSFW Core and upon refreshing, it isrefresh() that is called.
4.10 CSS file
A CSS file was used to include any custom styling that the component may wishto use.
16
4.11 Constructing a Component
This section will walk you through the process of constructing a simple componentthat displays a list of usernames when we navigate to http://hostname/users.Having constructed the users component also leads to exposing the web resourceshttp://hostname/json/users and http://hostname/json/users/Argument, whichgives us the data returned from users:get json() and users:get json(Argument)respectively.
users.erl
The Erlang module’s role is to JSON encode and output the data that we want todisplay, in this case the list of users: ["John", "Jackie", "Michael", "Sarah"].We ignore the get json(Arg) function for this simple example.
-module(users ).-export([get_json/0, get_json /1]).
get_json () ->Data = ["John", "Jackie", "Michael", "Sarah"],jsfw_json:encode(Data).
get_json(Arg) -> jsfw_json:encode(Arg).
users.js
The role of the JavaScript file is to take the JSON data output by the Erlang mod-ule and use the JavaScript API to put that data in an HTML element. We ignorethe handle and refresh functions for this simple example. For this example, thismeans:
• Creating the Body and List elements
• Fetching the data from the JSON resource http://hostname/json/users
• Calling the API function fill json(source, target) to fill the list withthe users data
• Adding the list element to the body
var users = new function () {
this.init = function () {var body = new Body ();var list = new List ();fill_json("/json/users", list);body.add(list);
17
}this.handle = function(argument) { }
this.refresh = function () { }}
Result
This will result in an <ul> element being populated by the four names output bythe Erlang module above.
<ul><li>John </li><li>Jackie </li><li>Michael </li><li>Sarah </li>
</ul >
5 Evaluation
When evaluating a framework like this, one cannot expect to get any raw bench-mark results but instead has to consider a di↵erent approach. Since the goal isan easy to use framework one should try to evaluate its user-friendliness. Howdoes one do this in a reasonable manner? Although this framework is not a userinterface per se, but rather a tool to aid developers in creating user interfaces, westill believe some of the principles from user interface evaluation are applicable.Selected methods from a few papers on the topic [12, 13] are described below.
5.1 Evaluation Methods
5.1.1 Heuristic Evaluation
In this type of evaluation, a number of evaluators (three to five people [14]) arepresented with an interface design and are asked to comment on it, based on a listof usability principles (heuristics) [14]. It is up to the evaluators how they want toproceed with the evaluation, as long as it is conducted individually. The resultsof the di↵erent evaluations are then aggregated to a single final evaluation. It issuggested that this type of evaluation performs better when UI experts are usedas evaluators [14] and this is confirmed by results of research papers comparingdi↵erent evaluation methods [15, 13]. An advantage of heuristic evaluation is thatthe evaluators are not performing real tasks per se and it can therefore be appliedat a very early stage of implementation.
18
5.1.2 Usability Testing
A di↵erent approach is to present evaluators (UI experts or not) with pre-defineduser scenarios and let them perform the tasks and report any errors encounteredor strange behavior by the system. Usability testing usually cannot be applieduntil at a later stage in product development since one needs to have the featuresin place required for the evaluators to perform their tasks.
5.1.3 Empirical Evaluation
In empirical evaluation, potential users test the interface. In contrast to usabilitytesting, the users are free to use the interface in the way that they normally wouldand comment on it accordingly. This type of evaluation su↵ers from the sameproblem as usability testing, it cannot be applied early on in the developmentcycle.
5.2 Evaluating JSFW
When evaluating JSFW, as previously mentioned, one does not evaluate a stan-dard user interface but instead looks at an interface made up of API functionsand an API documentation that are used to construct customized user interfaces.Since UI experts are not available for this evaluation, letting potential users (i.e.Erlang developers) conduct the evaluation seems to be the most feasible alter-native. Heuristic evaluation focuses on design flaws in user interfaces and sincewe are more interested in the functionality of the framework, this method is notsuitable. Since we want users of this framework to be able to implement specificfunctionality that they need in order to monitor their system, providing the testerswith scenarios containing general tasks (such as creating a new component) seemsreasonable. The advantage of usability testing over empirical evaluation is thatusers are guaranteed to test particular tasks that we find interesting, while alsoallowing them to get a general impression of the framework.
Clarke [16] describes the evaluation of APIs as finding the di↵erences betweenwhat users expect from the API and what the API really does. It is mentionedperforming this evaluation with the help of the cognitive dimensions framework [17]which lists 12 dimensions/factors that are important when it comes to API design.These dimensions are good pointers towards things to consider when constructingAPIs and therefore they will be useful in the evaluation as well.
5.2.1 Cognitive Dimensions
• Abstraction level. The minimum and maximum levels of abstraction exposedby the API, and the minimum and maximum levels usable by a targeted
19
developer.
• Learning style. The learning requirements posed by the API, and the learningstyles available to a targeted developer.
• Working framework. The size of the conceptual chunk (developer workingset) needed to work e↵ectively.
• Work-step unit. How much of a programming task must/can be completed ina single step.
• Progressive evaluation. To what extent partially completed code can be exe-cuted to obtain feedback on code behavior.
• Premature commitment. The amount of decisions that developers have tomake when writing code for a given scenario and the consequences of thosedecisions.
• Penetrability. How the API facilitates exploration, analysis, and understand-ing of its components, and how targeted developers go about retrieving whatis needed.
• API elaboration. The extent to which the API must be adapted to meet theneeds of targeted developers.
• API viscosity. The barriers to change inherent in the API, and how muche↵ort a targeted developer needs to expend to make a change.
• Consistency. How much of the rest of an API can be inferred once part of itis learned.
• Role expressiveness. How apparent the relationship is between each compo-nent exposed by an API and the program as a whole.
• Domain correspondence. How clearly the API components map to the domainand any special tricks that the developer needs to be aware of to accomplishsome functionality.
5.2.2 Method
Usability tests combined with the help of cognitive dimensions framework wereused to conduct user testing from people within the Erlang community. This isthe most relevant group of people since they are the potential users of JSFW. Inpractice, JSFW was shipped to developers together with an API documentation,
20
a set of tasks that we wanted to be tested and a link to an online evaluationform consisting of questions based on the cognitive dimensions framework. Detailsabout the user scenarios and the questions asked can be found in Appendix A andAppendix B respectively.
5.3 Results
The evaluation package was sent out to seven developers, all working as Erlangdevelopers.
5.3.1 Identified Problems
The testers found the following critical problems.
1. Not enough feedback was given to the user when constructing components.
2. The API documentation did not contain enough examples.
3. It was not obvious to tell what data structure to return from the Erlangmodules of components.
4. A need for updating only parts of components was presented.
5. Reloading of components did not function every time.
6. The ability to send input to components was sought after.
5.4 Resolution of Problems
After having researched methods for resolving the problems identified by the de-velopers, the framework was changed in the ways described below.
Not enough feedback was given to the user when constructing compo-
nents
This problem belongs to the progressive evaluation dimension and makes it di�cultfor users to to test partially completed code while implementing a new component.In order to resolve this, the framework was changed to report component compi-lation and runtime errors to the component front end.
21
The API documentation did not contain enough examples
An insu�cient documentation is a type of problem that belongs to the role expres-siveness dimension since one purpose of the documentation is to tell the user howeach API part fits into the program as a whole. Including examples is a good wayto make a documentation better. Thus, examples were added to the frameworkdocumentation.
It was not obvious to tell what data structure to return from the Erlang
modules of components
Once again, this is a problem related to an insu�cient documentation and thedocumentation was updated to account for this accordingly.
A need for updating only parts of components was presented
This problem does not fit accurately into any of the dimensions but is clearly aproblem concerning usage limitation. In order to be more flexible, the frameworkwas changed to use jQuery selectors16 as internal selectors when specifying targetsfor data to be put in.
Reloading of components did not function every time
The old way of recompiling components (via the settings interface) was changedinto an ”on demand” method that recompiles the components when the user nav-igates to them. A timestamp check was implemented to make sure compilationonly occurs if the .erl file is newer than the .beam file. This was an improvementover the old implementation which had a problem with purging old code causingit to sometimes not pick up changes.
The ability to send input to components was sought after
In response to this a new template tag was introduced that prints out a text inputto allow users to send arbitrary Erlang terms as input to components.
Other changes
In addition to resolving problems identified by the testers, new features were im-plemented. The component structure was changed to also include two HTMLtemplates to allow for more flexible and less painful front-end presentation. This
16http://api.jquery.com/category/selectors/
22
was implemented statically at this stage with ideas to specify any number of tem-plates in a future version.
6 Conclusion and discussion
A fairly competent and complete framework prototype was implemented and waswell received by the evaluators. It also became obvious, with the help of theweb server abstraction layer, that supporting multiple web servers was straightforward and did not require much code at all since di↵erent web servers have alot in common. The evaluation performed was very helpful to identify usabilityproblems and thanks to that, it became clear what changes were necessary to makethe framework more usable.
6.1 Challenges
While usability testing really helped the implementation of the framework it alsoshowed how long it can take to get a hold of people and actually get a response.The original idea was to perform the evaluation in several iterations with imple-mentation adjustments in between, but this proved to take too much time.
7 Future Work
7.1 WebSockets
WebSockets17 is an interesting new concept that allows for a two-way communica-tion between client and server, resulting in the ability for the server to push datato the front-end without the client needing to request it. With the help of thiswe can for example update graphs only when new data is available on the serverwithout having the client poll the server at regular intervals. An implementationof this would include adding a web socket client abstraction layer for componentsto use.
7.2 Support for Additional Webservers
While developing your own JSFW Web server plugin is fairly simple it wouldcertainly make JSFW more accessible if support for additional web server wasincluded. We could even imagine the scenario where the JavaScript front-end is
17http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-17
23
separated from the back-end, making it possible to run JSFW on non-Erlang serversoftware (e.g. Apache18).
7.3 Integration of a template engine
Many powerful template engines exist that allow a lot of flexibility and customiz-ability of dynamic data in HTML templates. One good example is ErlyDTL19
which is an Erlang implementation of the Django Template Language20.
8 References
[1] Ericsson AB. WebTool. http://www.erlang.org/doc/man/webtool.html.Accessed: November 2011.
[2] Ericsson AB. Appmon. http://www.erlang.org/doc/man/appmon.html.Accessed: November 2011.
[3] Spawnfest: 48 hour Erlang development competition. http://spawnfest.com/, July 2011. Accessed: November 2011.
[4] Smells like BEAM spirit. BigWig: A better Erlang webtool. http://www.metabrew.com/article/bigwig-erlang-webtool-spawnfest, July2011. Accessed: November 2011.
[5] J. Armstrong. Concurrency Oriented Programming in Erlang. Challenge,2002.
[6] J. Armstrong. Making Reliable Distributed Systems in the Presence of Soft-ware Errors. http://www.erlang.org/download/armstrong_thesis_2003.pdf, 2003.
[7] A. Sullivan. JSX - JSON library for Erlang. https://github.com/talentdeficit/jsx, October 2010. Accessed: November 2011.
[8] Smells like BEAM spirit. JSX - JSON library for Erlang. https://github.com/spawnfest/jsx, July 2011. Accessed: November 2011.
18http://httpd.apache.org/19https://github.com/evanmiller/erlydtl20https://docs.djangoproject.com/en/dev/ref/templates/builtins
24
[9] M. Troug. Erlang JSON libraries benchmarks. https://github.com/okeuday/erlbench, November 2011. Accessed: November 2011.
[10] M. Troug. Erlang JSON libraries benchmark results. http://okeuday.livejournal.com/18353.html, March 2011. Accessed: November 2011.
[11] B. Ippolito. Mochiweb, an Erlang library for building lightweight HTTPservers. https://github.com/mochi/MochiWeb, November 2011. Accessed:November 2011.
[12] Jakob Nielsen. Usability inspection methods. In Conference companion onHuman factors in computing systems, CHI ’95, pages 377–378, New York,NY, USA, 1995. ACM.
[13] Robin Je↵ries, James R. Miller, Cathleen Wharton, and Kathy Uyeda. Userinterface evaluation in the real world: a comparison of four techniques. InProceedings of the SIGCHI conference on Human factors in computing sys-tems: Reaching through technology, CHI ’91, pages 119–124, New York, NY,USA, 1991. ACM.
[14] Jakob Nielsen and Rolf Molich. Heuristic evaluation of user interfaces. InProceedings of the SIGCHI conference on Human factors in computing sys-tems: Empowering people, CHI ’90, pages 249–256, New York, NY, USA,1990. ACM.
[15] Jakob Nielsen. Finding usability problems through heuristic evaluation. InProceedings of the SIGCHI conference on Human factors in computing sys-tems, CHI ’92, pages 373–380, New York, NY, USA, 1992. ACM.
[16] S. Clarke. Measuring API Usability. Dr. Dobbs Journal, May 2004.
[17] T. R. G. Green and M. Petre. Usability analysis of visual programmingenvironments: a ‘cognitive dimensions’ framework. JOURNAL OF VISUALLANGUAGES AND COMPUTING, 7:131–174, 1996.
25
Appendix
A Scenarios
This is a short introduction followed by the scenarios sent out to the developersthat tested the framework.
A.1 Introduction
This framework aims to help with the development of web views that can helpwhen monitoring Erlang nodes. Users have the possibility to create custom madecomponents in the framework that display relevant information. A JavaScriptAPI exists that helps with the presentation of JSON data sent from Erlang mod-ules. This evaluation presents a number of scenarios and questions that wouldbe valuable to us when improving/evaluating the framework. Documentation canbe found at http://localhost:8080/jsfw_docs once the web server is started.Along with the examples found in the framework, this should be enough informa-tion to develop fairly advanced components.
In order to evaluate we would like you to try out the scenarios listed belowand after having completed them (or failed them for that matter), answering thequestions below.
A.2 Scenarios
A.2.1 List component
• Create a new component (mylist) through the settings interface
• Confirm that the component was created successfully by refreshing the pageand visiting the newly created component
• Modify the component so that it prints a list of names in an unordered list(<ul>)
A.2.2 Table component
• Create a new component (mytable) through the settings interface
• Confirm that the component was created successfully by refreshing the pageand visiting the newly created component
• Modify the component so that it prints some tabular data to a table (<table>)
26
A.2.3 Refresh
• Modify the mytable component so that the displayed data gets refreshed
• Add a button to pause/resume the refreshing
A.2.4 Custom
• Create a new component (mycomponent) in the settings interface
• Modify this component into a component of your own choice
A.3 Questions
Questions are found online here: http://goo.gl/bUUn8
B Questions
• What is your knowledge of Erlang? 1-5
• What is your knowledge of JavaScript/HTML? 1-5
• What browser are you using (and version)?
• What operating system are you running?
• What version of Erlang are you running?
• Could you complete the listed scenarios? If no, what hindered you?
• Did you feel that you could test your component even if it wasn’t finished?If no, what more feedback would you want?
• Did you feel that you could implement your component in an iterative man-ner?
• When using the API, is it is easy to know what methods to use?
• Does the object oriented way of building the JavaScript part of componentsmake sense?
• Is the amount of code needed to construct a TABLE element suitable?
• Once you managed to create the LIST, was it easy to construct the TABLE?
• Was there something in the framework that made you feel limited?
27
• Did you miss some API methods?
• Was it di�cult to know what to output from the Erlang module?
• What component did you try to implement for the Custom scenario?
• Were you successful in this? Did you think of another idea and consideredit too di�cult to implement using this framework?
C Framework Documenation
C.1 Structure
In the root of the program where you want to integrate JSFW, there must exist afile named jsfw.config, which must contain tuples according to:
{webserver_plugins_dir, PATH}
PATH - string, the path to the directory holding the webserver plugins
{components_dir, PATH}
PATH - string, the path to the directory holding the components
C.2 URL Scheme
The framework relies on a few static URLs and a URL system that are describedbelow.
/static/RESOURCEPATHReturns pre-shipped resource file found in priv/RESOURCEPATH
/json/componentsReturns a JSON array containing available components and their attributes fromtheir respective component.config files
/json/COMPONENTReturns JSON encoding of COMPONENT:get()
/json/COMPONENT/ARGReturns JSON encoding of COMPONENT:get(ARG)
28
C.3 Component structure
Components are placed in the components dir directory as specified in the configfile mentioned above, in a directory named component. This here (jsfw docs)is actually a component just like any other! Individual components consist of aJavaScript file, an Erlang module, a HTML template, another HTML templateto display arguments, a CSS file and are structured and named in the followingmanner:
C.3.1 HTML template (COMPONENT.html)
1| <h2>%%COMPONENT%%</h2>2| Click this example argument link: <span id="jsoncontainer"></span>
This is the template used when creating a new component. All occurrences of%%COMPONENT%% are then replaced by the name of the new component.
Support exists for a few special template tags and those are described below.
• {{toggle button}} - will be converted to a toggle button to pause/resumethe automatic refreshing of a component (refreshing only happens if thejavascript file implements the refresh function).
• {{input}} - will be converted to a text input that allows you to send ar-bitrary Erlang terms to your component. These terms will eventually behandled by the component’s Erlang module, more specifically, the input/1function. This text input should allow most Erlang terms.
C.3.2 Handle template (handletemplate.html)
1| <h1>Handle template for component: %%COMPONENT%%</h1>2| Argument given: <span id="argcontainer"></span>
This is the template shown when displaying a component with an argument (i.e.path /COMPONENT/ARG)
C.3.3 JavaScript file (COMPONENT.js)
var %%COMPONENT%% = new function() {// This function is called when the component link is// clicked in the menuthis.init = function() {
fill_json("/json/%%COMPONENT%%", $("#jsoncontainer"));
29
}// This function is called when the component is// automatically refreshedthis.refresh = function() {}// This function is called when an argument is// clicked on the component’s pagethis.handle = function(arg) {
fill_json("/json/%%COMPONENT%%/" + arg, $("#argcontainer"));}
}
This is the template used when creating a new component. All occurrences of%%COMPONENT%% are then replaced by the name of the new component. TheJavaScript file needs to uphold this basic structure, with the variable declara-tion on the top level as well as the init function declaration and the (optional)refresh function declaration for automatic refreshing of the component. See howjQuery selectors are used to define where to put the JSON data?
C.3.4 Erlang module (COMPONENT.erl)
-module(%%COMPONENT%%).-export([get/0, get/1]).
% -export([init/0]).% -export([input/1]).
get() ->[{’$link’,<<"This is an example argument">>,<<"/%%COMPONENT%%/3">>}].
get(Arg) ->Arg.
% init() ->% ok.%% input(Input) ->% io:format("Component ~p received input: ~p~n",
[%%COMPONENT%%, Input]).
This is the template used when creating a new component. All occurrences of%%COMPONENT%% are then replaced by the name of the new component. Note how
30
get/0 and get/1 should return data that can be encoded to JSON (see moreinformation further down). init/0 and input/1 are optional. init/0 is calledwhen the module is loaded the first time or recompiled and is used for setup(such as registering a spawned process etc.). input/1 is called whenever inputis sent to the module with the help of the {{input}} template tag (see HTMLdocumentation).
C.3.5 Config file (component.config)
[{enabled,"true"},{displayname,DISPLAYNAME}]
This file contains the properties specified for the component in question. The valueof displayname is a string that is displayed in the component menu and enableddecides whether it should show in the menu or not.
C.3.6 CSS file (COMPONENT.css)
This is optional and contains any custom styling wanted for the component.
Components can be created and enabled/disabled through the /jsfw settings webinterface. Go ahead and create your first example component right away!
C.4 JavaScript API
There is actually only a single JavaScript API function but that should help youcomplete most of your implementations if you’ve constructed a clever HTML tem-plate.
fill_json(source, element);Example: fill_json("/json/users", $("#usertable"));
Used to fill element (which is an element selected using jQuery selectors (moreon those here: http://api.jquery.com/category/selectors/) with the JSONreturned from the URL specified by source (typically ”/json/COMPONENT”).Make sure that you format your output Erlang data so that it suits the elementthat you wish to put it in (see more on that under Erlang API). In the aboveexample, the data returned from /json/users is inserted into the html ¡table¿element with id ’usertable’. Thus, the Erlang data output from users:get()must be in tabular format.
31
C.5 Erlang API
C.5.1 jsfw utility
arg_to_pid(Arg)Given a binary Arg (e.g. <<"0.1.0">>), returns a PID (<<0.1.0>>)
C.5.2 Data structures
Below is a list of how Erlang data should be structured. This is what shouldbe returned from your component’s get() and get(Arg) functions. When usingfill json(), JSFW will look at the target element type to decide how to iterateover the data given to insert it.
C.5.3 Tabular data (output in a <table>)
[[row1col1, row1col2, ...], [row2col1, row2col2, ...], ...]
C.5.4 List data (output in a <ul>)
[element1, element2, element3, ...]
C.5.5 Links
{’$link’, TEXT, URL}.Example: {’$link’,<<"3">>,<<"/users/3">>}
This will be printed as:
<a href="/users/3">3</a>
Clicking on that link when navigating the users component will call the op-tional handle(argument) javascript function of the component with ”3” as argu-ment (in this particular case). Fetching the JSON from /json/users/3 will callusers:get(<<”3”>>) similarily.
C.5.6 PIDs
PIDs are treated with special care and taken care of by the included jsfw pidcomponent. All PIDs printed will be links to /jsfw pid/PID and when clicked, bedisplayed by the jsfw pid component.
32
D Source Code
D.1 Cowboy Web server plugin
-module(jsfw_cowboy ).-include_lib("jsfw/include/jsfw.hrl").-include_lib("cowboy/include/http.hrl").-behaviour(cowboy_http_handler ).
-export([start /0]).-export([init/3, handle/2, terminate /2]).
init({_Any , http}, Req , []) ->{ok, Req , undefined}.
handle(Req , State) ->JSFWreq = to_jsfw_request(Req),{ok, JSFWreply} = jsfw_request:handle_request(JSFWreq),StatusCode = jsfw_reply:get(status , JSFWreply),Headers = jsfw_reply:get(headers , JSFWreply),Body = jsfw_reply:get(body , JSFWreply),{ok, Req2} = cowboy_http_req:reply(StatusCode , Headers , Body , Req),{ok, Req2 , State}.
%% "Translates" a cowboy http request%% to a jsfw_req as defined in jsfw.hrl-spec to_jsfw_request (# http_req{}) -> #jsfw_req{}.to_jsfw_request(Req) ->
%% Extract the relevant information from the Cowboy HTTP request{Method , Req} = cowboy_http_req:method(Req),{Raw_Path , Req} = cowboy_http_req:raw_path(Req),{Args , _Req} = cowboy_http_req:qs_vals(Req),%% Constructs the jsfw_req record#jsfw_req{
%% HTTP Method usedmethod = Method ,%% Turn the binary path into a list of binaries ,%% strip empty binariespath = [ B || B <- re:split(Raw_Path , "/"), B /= <<>> ],%% Query string argumentsargs = [{list_to_atom(binary_to_list(Key)),Value} ||
{Key ,Value} <- Args]}.
terminate(_Req , _State) ->ok.
start () ->ok.
D.2 Misultin Web server plugin
-module(jsfw_misultin ).
33
-include_lib("jsfw/include/jsfw.hrl").-include_lib("misultin/include/misultin.hrl").-export([handle_http /1]).
handle_http(Req) ->%% Translate the misultin requestJSFWreq = to_jsfw_request(Req),{ok, JSFWreply} = jsfw_request:handle_request(JSFWreq),Status = jsfw_reply:get(status , JSFWreply),Headers = jsfw_reply:get(headers , JSFWreply),Body = jsfw_reply:get(body , JSFWreply),Req:respond(Status , Headers , Body).
%% Extracts the relevant information from the misultin http request%% and returns a corresponding record structure as defined by jsfw.hrl-spec to_jsfw_request (#req{}) -> #jsfw_req{}.to_jsfw_request(Req) ->
Method = Req:get(method),{abs_path , Raw_Path} = Req:get(uri),Args = Req:parse_qs(),#jsfw_req{
method = Method ,%% Turn the string list into a list of binariespath = [list_to_binary(P) || P <- string:tokens(Raw_Path , "/")],args = [{list_to_atom(Key), list_to_binary(Value)} ||
{Key ,Value} <- Args]}.
D.3 Example Components
D.3.1 List Processes
-module(lister ).-export([get/0, get/1]).
get() ->PIDs = erlang:processes(),Key = fun(K,PID) -> proplists:get_value(K, erlang:process_info(PID))
end ,[[
%% Return Process ID, registered name , current functionPID ,proplists:get_value(registered_name , PID),proplists:get_value(current_function , PID)
] || PID <- PIDs ].
get(Arg) ->PID = jsfw_utility:arg_to_pid(Arg),Info = erlang:process_info(PID),[ [Key , Value] || {Key , Value} <- Info ].
34