ousia weaver - a tool for creating and publishing mashups as impressive web pages-
DESCRIPTION
This presentation describes Ousia Weaver, a tool for creating and publishing mashups as impressive Web pages.This will be presented in MEM2010(3rd Workshop on Mashups, Enterprise Mashups and Lightweight Composition on the Web) held on April 26th.TRANSCRIPT
![Page 1: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/1.jpg)
Ousia WeaverA tool for creating and publishing mashups
as impressive Web pages
Ikuya Yamada, Wataru Yamaki, Hirotaka Nakajima, Yoshiyasu Takefuji
Studio Ousia Inc.Keio University
3rd Workshop on Mashups, Enterprise Mashupsand Lightweight Composition on the Web (MEM 2010)
![Page 2: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/2.jpg)
Mashup is a technique which combines data and functions from multiple Web sources into new Web contents.
Mashups have become very popular. Numerous mashups have been created on the Web.◦ There are over 4500 mashups registered on
ProgrammableWeb (http://www.programmableweb.com/).
Mashups in a nutshell
![Page 3: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/3.jpg)
Recently, software tools called “mashup editors” have been attracting attention.◦ Yahoo! Pipes◦ Microsoft Popfly◦ Intel MashMaker
These editors are typically targeted at non-expert users and allow them to create mashups without writing any code.
Mashup Editors
![Page 4: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/4.jpg)
However, existing mashup editors tend to focus only on creating mashups and lack in publishing and visualizing mashups.
If users want to publish mashup results on the Web, they eventually have to do complex programming and/or settings.
Issues
![Page 5: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/5.jpg)
Yahoo! Pipes only supports three simple visualizations (List, Image, Map).
Users cannot customize these visualizations. Pipes doesn’t support publishing its mashups as independent Web
pages.
Example
Visualized mashup result by Yahoo! Pipes
![Page 6: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/6.jpg)
We propose a novel mashup editor named Ousia Weaver.
Ousia Weaver has distinguished functions that strongly support publishing and visualizing mashups.
Ousia Weaver provides a visual editor which enables users to create mashup without writing any code.
Users can create, visualize, and publish mashups without any complex programming and settings.
Ousia Weaver
![Page 7: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/7.jpg)
With Ousia Weaver, users can create mashups using three steps:1. Creating the mashup with Visual Editor2. Visualizing the mashup with Visualization
Widgets3. Publishing the mashup with Mashup Server
Three Steps to Publish Mashups with Ousia Weaver
Creating the mashup
Visualizing the mashup
Publishing the mashup
![Page 8: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/8.jpg)
Creating Mashups
![Page 9: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/9.jpg)
Ousia Weaver provides a visual editor which enables users to create mashups by visually stringing together components.
Mashups are represented using directed graph structure.◦ Operators (nodes) represent data operations.◦ Connections (edges) connect two operators
and define data paths between these.
Programming Model
Connections
Operators
Creating mashup data-
flow
Visualizing the mashup
Publishing the mashup
![Page 10: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/10.jpg)
Creating a mashup with Ousia Weaver is an iterative process:◦ Adding desired operators◦ Stringing operators together using
connections.
Mashups are executed in a data-flow manner. 1. InputString operator receives a keyword
from a user.2. AmazonAllByKeyword operator performs a
search on Amazon.com with the keyword.3. OutputItemList operator returns the search
result.
Programming Model
Operators
Connections
Creating mashup data-
flow
Visualizing the mashup
Publishing the mashup
![Page 11: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/11.jpg)
The most fundamental component of Ousia Weaver is operators.
Operators represent all data operations in Ousia Weaver.
Currently, Ousia Weaver provides over 300 operators.
An operator consists of input terminals, argument name/value pairs, and output terminals.
OperatorsInput terminal
Argument value
Argument name
Output terminal
Creating mashup data-
flow
Visualizing the mashup
Publishing the mashup
![Page 12: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/12.jpg)
All operators can be classified into three groups:
◦ Base operators provide basic programming operations. Arithmetic operation Data type conversion Standard selection operation (If /Switch) Loop operation (While/Foreach)
◦ Data source operators represent Web data sources. Web API Web data extraction (RSS / CSV)
◦ Data transformation operators transform data into a visualizable form. AddImage operator adds thumbnails to the data AddLocation operator attaches geographical locations to the data AddDate and AddDateRange operator add dates and ranges of dates to the
data respectively. AddCoordinate2d operator adds two dimensional coordinates to the data.
Operators
![Page 13: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/13.jpg)
Operators are extensible. Users can add and update arbitrary operators whenever they need.
An operator consists of two files:◦ Simple metadata file (JavaScript)◦ Program code file (Python)
Ousia Weaver provides code templates. Users can create new operators by writing just a few code fragments.
Extending Operators ousiaweaver.Operators.Append = function(){ this.addInputTerminal("string", "Input"); this.addOutputTerminal("string", "Output");}ousiaweaver.Operators.Append.packageName = "base.string";ousiaweaver.Operators.Append.arguments = { "text": { label: "Text", type: "string", required: true, }}
Append.js
class AppendTerminal(OperatorBaseTerminal): def __init__(self,args): OperatorBaseTerminal.__init__(self, args) Input_terminal = {} input_terminal["Input"] = None self.set_input_terminals(input_terminal)class AppendBody(OperatorBaseBody): def __init__(self, args, manager_args, worker): OperatorBaseBody.__init__(self, args, manager_args, worker) def run(self, input_terminals): output_terminals = {} output_terminals["Output"] = str(input_terminals["Input"]) + str(self.get_args()["text"]) return output_terminals
Append.py
Creating mashup data-
flow
Visualizing the mashup
Publishing the mashup
![Page 14: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/14.jpg)
Ousia Weaver implements a simple data typing system.
There are some unique data types:◦ any can represent any data
including other data types.◦ itemlist represents a
mashup result and contains multiple item.
◦ data-flow represents a mashup.
Data TypesName Description
any Can represent any data.
string Represents a string.
boolean Represents a boolean.
number Represents a number.
coordinate Represents a coordinate.
datetime Represents a date and a time.
daterange Represents a range of dates.
location Represents a geographical location.
image Represents an image.
uri Represents a URI.
array Represents an array.
object Represents an object (hash-table).
item Represents a constituent of mashup result.
itemlist Represents a mashup result.
data-flow Represents a mashup.Creating
mashup data-flow
Visualizing the mashup
Publishing the mashup
![Page 15: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/15.jpg)
A terminal has a data type property.
Two terminals, which have different data type properties, cannot be connected unless either or both properties are any.
When users start dragging one terminal, other terminals will automatically be disabled and turn gray if they cannot be connected with the dragged terminal.
Data Types
Terminals which cannot be connected with the dragged terminal will automatically turn gray.
Creating mashup data-
flow
Visualizing the mashup
Publishing the mashup
![Page 16: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/16.jpg)
Visualizing Mashups
![Page 17: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/17.jpg)
With Ousia Weaver, the user can visualize the mashup result by choosing the desired visualization widgets from the list.
Visualizing Mashups
Creating mashup data-flow
Visualizing the mashup
Publishing the mashup
![Page 18: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/18.jpg)
Visualization widgets are components which visualize mashup results.◦ Catalog widget ◦ Plot widget◦ Table widget ◦ Map widget◦ Timeline widget
Visualization Widgets
Catalog widget Plot widget Table widget
Map widget Timeline widget
Creating mashup data-flow
Visualizing the mashup
Publishing the mashup
![Page 19: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/19.jpg)
Catalog widget visualizes the mashup results like a catalog. It arranges mashup results on the screen using their
thumbnails, titles, and descriptions.
Catalog Widget
Catalog widget Plot widget Table widget
Map widget Timeline widget
Creating mashup data-flow
Visualizing the mashup
Publishing the mashup
![Page 20: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/20.jpg)
Plot widget plots the mashup results on a two dimensional space.
Plot Widget
Catalog widget Plot widget Table widget
Map widget Timeline widget
Creating mashup data-flow
Visualizing the mashup
Publishing the mashup
![Page 21: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/21.jpg)
Table widget presents all titles and descriptions of the mashup results in a table.
Table Widget
Catalog widget Plot widget Table widget
Map widget Timeline widget
Creating mashup data-flow
Visualizing the mashup
Publishing the mashup
![Page 22: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/22.jpg)
Map widget depicts the mashup results visually on a map.
Map Widget
Catalog widget Plot widget Table widget
Map widget Timeline widget
Creating mashup data-flow
Visualizing the mashup
Publishing the mashup
![Page 23: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/23.jpg)
Timeline widget puts the mashup results into a graphical timeline.
Timeline Widget
Catalog widget Plot widget Table widget
Map widget Timeline widget
Creating mashup data-flow
Visualizing the mashup
Publishing the mashup
![Page 24: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/24.jpg)
Ousia Weaver provides a function which automatically selects appropriate visualization widgets for the mashup.
The function scans the mashup result and detects if the result has all of the data which need to be visualized.
Also, users can choose visualization widgets manually.
Automatic Widgets Selection
Creating mashup data-flow
Visualizing the mashup
Publishing the mashup
![Page 25: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/25.jpg)
Publishing Mashups
![Page 26: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/26.jpg)
Ousia Weaver provides a lightweight server named Mashup Server.
Mashup server executes mashups dynamically and makes the mashup results accessible to external users.
The server automatically assigns a URL to each mashup and dynamically executes the mashup when an external user accesses it. ◦ The URL of a mashup named “search_videos” in the
“foo” server would be “http://foo/search_videos/”
Mashup Server
Creating mashup data-flow
Visualizing the mashup
Publishing the mashup
![Page 27: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/27.jpg)
Implementation
![Page 28: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/28.jpg)
Ousia Weaver consists of five main components:◦ Operator manager◦ Run-time engine◦ Visualizer◦ Visual Editor◦ Mashup server
Architecture
Run-timeengine
Visual editor Mashup server
Operator manager
Visualizer
Results of mashups
Metadata ofoperators
Results ofvisualization
Program codes of operators
Sources of mashups
![Page 29: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/29.jpg)
Operator manager is a simple component implemented using Python.
It manages the list of all operators and delivers their program codes and metadata to other components.
Operator Manager
Run-timeengine
Visual editor Mashup server
Operator manager
Visualizer
Results of mashups
Metadata ofoperators
Results ofvisualization
Program codes of operators
Sources of mashups
![Page 30: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/30.jpg)
Run-time engine is a component responsible for executing mashups.
It is implemented as a multi-process server daemon using Python and Python’s multiprocessing module.
Run-time Engine
Run-timeengine
Visual editor Mashup server
Operator manager
Visualizer
Results of mashups
Metadata ofoperators
Results ofvisualization
Program codes of operators
Sources of mashups
![Page 31: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/31.jpg)
Visualizer is a component which visualizes mashups.
It passes the mashup result to each of the selected visualization widgets.
It is implemented in JavaScript and uses jQuery (jquery.com) and Simile Widgets (simile-widgets.org).
Visualizer
Run-timeengine
Visual editor Mashup server
Operator manager
Visualizer
Results of mashups
Metadata ofoperators
Results ofvisualization
Program codes of operators
Sources of mashups
![Page 32: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/32.jpg)
Visual editor provides a rich visual editing interface for creating and publishing mashups.
It is implemented using JavaScript and HTML, and can be run on most commonly used Web browsers.
It can also be executed as a stand-alone application on Windows, Mac OS X, and Linux using Mozilla’s XUL technology.
Visual Editor
Run-timeengine
Visual editor
Mashup server
Operator manager
Visualizer
Results of mashups
Metadata ofoperators
Results ofvisualization
Program codes of operators
Sources of mashups
![Page 33: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/33.jpg)
Mashup server makes mashups accessible to external users.
It provides a simple Web server which returns mashup results to external users.
The mashup server is written in Python and uses Apache CouchDB as a datastore.
Mashup Server
Run-timeengine
Visual editor Mashup server
Operator manager
Visualizer
Results of mashups
Metadata ofoperators
Results ofvisualization
Program codes of operators
Sources of mashups
![Page 34: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/34.jpg)
Execution Model
![Page 35: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/35.jpg)
Mashup server executes mashups as follows:1. An external user accesses a mashup.2. It executes the mashup using the runtime engine.3. It visualizes the result using the visualizer.4. It returns the visualized result to the user.
Mashup Execution Model(Mashup Server)
(2) (1
)
(4)
Run-timeengine
Visualizer
External user
(3)
Mashup server
The basic sequence of operations of the mashup server.
![Page 36: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/36.jpg)
Run-time engine executes mashups in parallel.◦ It divides a mashup into multiple, independent tasks and
executes them using worker processes.
The manager processes and the worker processes are recycled across mashup executions.
Mashup Execution Model(Run-time Engine)
Manager processes
Task queue
task
Workerprocesses
Add tasks
Watch progress
Take tasks
Add/getresults
Result buffer
task task
result result result
![Page 37: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/37.jpg)
Examples
![Page 38: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/38.jpg)
“Book Mashup” provides a simple book search function.1. Receiving a keyword from the user.2. Performing a book search on Amazon.com.3. Attaching coordinates, which consists of price
and the number of pages, to the result.4. Returning the result to the user.
Book Mashup
![Page 39: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/39.jpg)
“Event mashup” provides an event search function.1. Receiving a keyword from the user.2. Retrieving event information from
Upcoming and Eventful.3. Adding ranges of dates of the events
to the result.4. Attaching geographic locations of the
events to the result.5. Returning the result to the user.
Event mashup
![Page 40: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/40.jpg)
Conclusion and Future Work
![Page 41: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/41.jpg)
We proposed a mashup editor called Ousia Weaver. ◦ Ousia Weaver provides a sophisticated visual editor which
enables users to create and publish mashups without writing any code.
◦ Ousia Weaver offers a variety of rich visualization widgets which visualize mashups in an impressive way.
◦ Ousia Weaver includes a simple Web server which makes the mashup accessible to external users.
With Ousia Weaver, users can easily create and publish mashups as impressive Web pages.
Conclusion
![Page 42: Ousia Weaver - A tool for creating and publishing mashups as impressive Web pages-](https://reader033.vdocuments.site/reader033/viewer/2022052821/553e01374a79599f2a8b488a/html5/thumbnails/42.jpg)
We plan to do an extensive user evaluation which compares Ousia Weaver to existing mashup editors.
We will continue to develop Ousia Weaver.
We are planning to release Ousia Weaver as an open source product.
Future Work