maristella matera, matteo picozzi, cinzia...

End-User Development of Mobile Mashups Maristella Matera, Matteo Picozzi, Cinzia Cappiello

Introduction and motivations

Mobile

User

Web

Development of “ad hoc”

mobile applications

spread of mobile devices

powerful communication

networks

huge number of services

easy access to data

situational needs

data Integration modularity and reuse

desire of participating to content

and apps creation

application integration

user interface integration

limited expertise

in technologies

web mashup

improved HW and SW

technologies

2

Introduction and motivations

(M)EUD Mobile End-User

Development

MobiMash

2

Mobile

User

Web

Platform for lightweight,

user-driven service

composition

spread of mobile devices

powerful communication

networks

huge number of services

easy access to data

situational needs

data integration modularity and reuse

desire of participating to content

and apps creation

application integration

user interface integration

limited expertise

in technologies

web mashup

improved HW and SW

technologies

Mobile Mashups

Web Mashups: • modular and composable

• flexibile

• vertical

• contingent

Mobile Apps: access to

• OS (agenda, calendar,..)

• HW (GPS, camera, ..)

(M)EUD

Mobile Mashups • native mobile applications

• data integration

• remote/local services

• UI component synchronization

3

MobiMash

• Design Environment (DE)

• Execution Engine (EE)

• Configuration Server

5

Data integration

Process for data integration [1]:

‣ data samples retrieved from sources and

local schema representation

[1] J. Bleiholder and F. Naumann. Data fusion. ACM

Comput. Surv., 41(1), 2008.

7

Visual

Source

Data

Sampling

Rules file

generation

Visual

mapping

Visual template

Selection

Classic

Source schema

reverse

engineering

Mapping

between

sources and

global schema

Global

schema

definition

Data views

generation

task for expert users

task for non expert users

automatic task

Data integration

‣ data samples retrieved from sources and


‣ implicit definition of the global schema

through the selection of a Visual Template


Comput. Surv., 41(1), 2008.


7

Visual

Source

Data

Sampling

Rules file

generation

Visual

mapping

Visual template

Selection

Classic

Source schema

reverse

engineering

Mapping

between

sources and

global schema

Global

schema

definition

Data views

generation



automatic task

Data integration

‣data samples retrieved from sources and




‣ mapping between the source local

schema and the global schema through

XPath queries on the source result sets


Comput. Surv., 41(1), 2008.


7

Visual

Source

Data

Sampling

Rules file

generation

Visual

mapping

Visual template

Selection

Classic

Source schema

reverse

engineering

Mapping

between

sources and

global schema

Global

schema

definition

Data views

generation



automatic task

Data integration



‣data samples retrieved from sources and


‣ mapping between the source local

schema and the global schema through

XPath queries on the source result sets

‣ generation of the rules for the automatic

instantiation of the mobile application


Comput. Surv., 41(1), 2008.


7

Visual

Source

Data

Sampling

Rules

generation

Visual

mapping

Visual template

Selection

Classic

Source schema

reverse

engineering

Mapping

between

sources and

global schema

Global

schema

definition

Data views

generation



automatic task

Data integration process

Source Data

Sampling

Visual Template

Selection Visual Mapping

Rules File

Generation

8

Selection of data sources from the

platform repository

Data sources: RESTful service returning

an XML document (data.xml)

Tree based representation of the source

local schema and a sample of the service

data instances.


Source Data

Sampling

Visual Template


Rules File

Generation

data value

tag name

attribute name

Tree Legend

data.xml

Tree-based

rapresentation

http://free.worldweatheronline.com/weather.ashx?

q=Chicago&

format=xml&

num_of_days=4&

key=5ada3838de115125111210

9

http://free.worldweatheronline.com/feed/weather.ashx?q=milan&format=xml&num_of_days=4&key=5ada3838de115125111210














Source Data

Sampling

Visual Template


Rules File

Generation

Visual Template (VT): a UI-based representation of the way data can be rendered in the

application under construction.

VT= <vr1,vr2,...,vrn>

vrk : visual renderer, it renders data items extracted

by data sources

a) union sub-template: {vrk} k=1,...,m

b) merge sub-template: {vrk} k=m+1,...,n

b) a)

title

subtitle

image

detail

10


Source Data

Sampling

Visual Template


Rules File

Generation

• Visual mapping, to fill out:

‣ union sub-template

‣ merge sub-template

• Synchronization with additional

UI components

11


Source Data

Sampling

Visual Template


Rules File

Generation

Drag&drop actions

map attributes on

the visual renderers

in the union sub-

template Visual mapping, to fill out



•Synchronization with

additional UI components

11


Source Data

Sampling

Visual Template


Rules File

Generation

Drag&Drop actions

map attributes on

the visual renderers

in the merge sub-

template

• Visual mapping, to fill out



•Synchronization with additional

•UI components

11


Source Data

Sampling

Visual Template


Rules File

Generation

• Visual mapping, to fill out



• Synchronization with

additional UI components

UI Component: a component equipped

with their own UI, through which the

user interacts with the underlying

service (e.g., Google Maps).

Binding: synchronization of the unified

data view (merge sub-template) with

UI components.

- Remote binding: it involves remote

services

- Local binding: it involves services

local to the mobile device (e.g.,

Android applications)

Definition of

synchronization

binding, by dragging

service icon on merge

visual renderer

11


Source Data

Sampling

Visual Template


Rules File

Generation

11

S= {s1,s2,s3,..,sj}: set of available data sources

M(si.h) ➝ vrk: mapping function

si.h: attribute of the i-th data source

ls(si): local schema, i.e., reduction of the

source schema according to the selection of

attributes operated through the visual mapping

- ∀ si.h ∈ uls(si) : si.h ➝ uvrk

- ∀ si.h ∈ mls(si) : si.h ➝ mvrk


Source Data

Sampling

Visual Template


Rules File

Generation

Application Schema ‣ Service Data-Layer

‣ Presentation Layer

‣ Service-Binding-Layer

Mobile Mashup

Model

application-schema.xml

12


Source Data

Sampling

Visual Template


Rules File

Generation




Mobile Mashup

Model


<sources>

<category name=”Concerts” icon=”..”>

<source name=”Last.fm” url=”..”>

<operation>

<param name=”location”

label=”Luogo”

type=”input”>

Chicago</param>

..

</operation>

</source>

<source name=”Upcoming” .... > ... </source>

</category>

..

</sources>

12

Source Data

Sampling

Visual Template


Rules File

Generation




Mobile Mashup

Model


<union>

<vr name=”Title” type=”Text”>

<data source=”Last.Fm” query=”/events/title” />

<data source=”Upcoming” query=”/@name” />

</vr>

<vr name=”Subtitle” type=”Text”>

<data source=”Last.Fm” query=”/venue/name” />

<data source=”Upcoming” query=”/@venue_name” />

</vr>

<vr name=”Image” type=”Image”>

<data source=”Last.Fm” query=”/venue/image” />

<data source=”Upcoming” query=”/@photo_url” />

</vr>

..

</union>


12

Source Data

Sampling

Visual Template


Rules File

Generation



‣ Service-Binding Layer

Mobile Mashup

Model


<merge>

<vr name=”Stato” priority=”0” type=”Text”

source=”Last.Fm” query=”..”

binding=”twitter | flickr | wiki” />

<vr name=”Città” priority=”1” type=”Text”



<vr name=”Telefono” priority=”2” type=”Text”


binding=”dialer | addressbook” />

<vr name=”Città” priority=”0” type=”Text”

source=”Upcoming” query=”..”


<vr name=”Indirizzo” priority=”1” type=”Text”

source=”Upcoming” query=”..”

binding=”map” />

...

</merge>


12

Source Data

Sampling

Visual Template


Rules File

Generation



‣ Service Binding-Layer

Mobile Mashup

Model


<filters>

<list-size> true </list-size>

<search> true </search>

</filters>


12

Data fusion “on-demand”

13

(ON) • Fusion at “runtime” of the data instances deriving from different services and mapped

on the same visual template.

• Global Schema: built “on demand” (at runtime) using the set of attributes in the union

sub-template, uls(si), plus the union of all the attributes associated in the different merge

sub-templates, mls(si)

(OFF) • The union sub-template

includes the union of all the

instances extracted through

the mapping query by the

involved services.

• When a specific instance

is selected from the list, the

user can explore the data in

the corresponding merge

sub-template – no fusion is

applied for similar instances

Upcoming

Last.Fm

b) Merge sub-template

a) Union sub-template

c) Merge sub-template



c) Union sub-template

after duplicate elimination and

conflict resolution

b)Merge sub-template

after data-fusion

Duplicate

identification

14

Data Fusion “on-demand”:

• duplicate detection

• data fusion

• duplicate elimination


Data Fusion “on-demand:


• data fusion





conflict resolution


after data fusion

Duplicate

identification

- Similarity evaluation: Chapman’s Soundex metrics

Sim(si.x, sj.y) → s:

s >= 0.965

- mls(si.x) e mls(sj.y) “fused”

s < 0.965

- Renderin of the only mls(si.x)

14

si.x

sj.y





conflict resolution


after data-fusion

Duplicate

identification

- The merge sub-template of the selected

instance will collect all the details of the

instances found as “similar” at step (1)

(duplicate detection)

14

si.x

sj.y

Data Fusion-on-Demand:


• data fusion






conflict resolution


after data-fusion

Duplicate

identification

- Elimination of the duplicate instances

after conflict resolution:

➡ image not present

➡ different “sub-titles”

- conflict resolution (and completion):

based on a priority assigned to the

services when the application is

designed (data from high priority

services are privileged)

si.x

sj.y

14

Data Fusion-on-Demand:


• data fusion


Architecture: Execution Engine

16

Architecture: Design Environment

15

Execution of the Mobile Mashup

• Mashup selection

• Data sources selection

• Modification of data source parameters

• Enabling data fusion “on-demand”

• List (union sub-template)

• Details (merge sub-template)

• Binding

- Remote bindings: Wikipedia, Youtube,

GoogleMaps, Flickr, Twitter

- Local bindings: dialer

17


17







• Binding








• Enabling Data Fusion “On-Demand”



• Binding




17








• Binding




17


• Mashup Selection






• Binding




17








• Binding




17

Execution of the mobile mashup







• Binding




17

Validation: Performance of the mobile application

Test performed on a Samsung Galaxy SII, dual-core processor (1.2 GHz ARM Cortex-

A9) and 1GB RAM

Data fusion execution time:

T_avg(100,2) = 4,9 sec

T_avg(100,3) = 4,02 sec

T_avg(200,2) = 9,31 sec

T_avg(200,3) = 6,71 sec

Average time for data fusion with varying

numbers of data items

0

1

2

3

4

5

6

7

8

9

10

10

20

30

40

50

60

70

80

90

100

110

120

130

140

150

160

170

180

190

200

No of returned data items

Ave

rag

e T

ime

(se

c)

data fusion-on-demand, 3 services data fusion-on-demand, 2 services

18

Validation: Performance of the mobile application

Downloaded Items in the List

40 80 120

3G 7,79 11,70 16,15

Wi-Fi 2,77 4,43 6,15


A9) and 1GB RAM

Download time

List download

7,79

11,70

16,15

2,77

4,43

6,15

0

2

4

6

8

10

12

14

16

18

1 web service (40

items)

2 web services (80

items)

3 web services (120

items)

Ave

rag

e tim

e

(se

c)

3G Wi-fi

18

Validation: performance of the mobile application

Download Item List

40 80 120

3G 7,79 11,70 16,15

Wi-Fi 2,77 4,43 6,15

n° of Remote Binding Views

1 2 3 4

5,70 6,67 9,16 10,54

2,20 2,88 3,54 5,96

Download for the Remote Binding View

2,202,88

3,54

5,965,70

6,67

9,16

10,54

0

2

4

6

8

10

12

1 views 2 views 3 views 4 views

Ave

rag

e tim

e

(se

c)

3G Wi-fi

18


A9) and 1GB RAM

Download time

Validation: formative evaluation with users

User sample: 10 user divided into two groups: expert users (eu) and non-expert users (neu).

Test Methodology: 2 tasks (A e B) with an incremental complexity (time and number of steps measured).

HP: our method enables both expert users and non-expert users to achieve a good performance as

app creators

• Efficiency

Task A:

- Average time [min] ( neu = 3.83; eu = 4.22)

- Wilcoxon-Mann (U = 15.5, p = .55)

Task B:

- Average time [min] (neu = 8.61; eu = 7.99)

- Wilcoxon-Mann (U = 17, p = .42)

19

Time to tackle design sub-activities

0

20

40

60

80

100

120

140

160

180

service

selection

union merge binding saving running

Sub-activityT

ime

[se

c]

Task A: non expert user Task A: expert user

Task B: non expert user Task B: expert user


Steps for tackling the design sub-activities

0

2

4

6

8

10

12

service

selection

union merge binding saving running

Sub-avtivity

Ste

pTask A: non expert user Task A: expert user

Task B: non expert user Task B: expert user

• Efficiency

Task A:

- Average time [min] ( neu = 3.83; eu = 4.22)


Task B:



Task A:

- Average no of steps [step] (neu = 24; eu = 21.6)

- Wilcoxon-Mann (U = 17,5, p = .31)

Task B:

- Average no of steps [step] (neu = 41.4; eu = 39.2)


Tim

e

No

of S

teps

19


There is no empirical

evidence against our

hp: the behaviour of

non-expert users is

comparable to the

one of expert users

• Efficiency

Task A:



Task B:



Task A:

- Average no of steps [step] (neu = 24; eu = 21.6)

- Wilcoxon-Mann (U = 17,5, p = .31)

Task B:

- Average no of steps[step] (neu = 41.4; eu = 39.2)


Tim

e

No o

f S

teps

19

Validation: post-test satisfaction questionnaire

• Ease of use

- Internal reliability: α = 0.81 (0.8<α<0.9 - very good)

- Average = 3.7, minS.E. = 0.39 (values between 0 and 5)

- No significant difference for the two user groups (t(46) = .875, p = .39)

• User satisfaction

- Internal reliability: α = 0.73 (0.7<α<0.8 - good)

- Average = 4.28, minS.E.= 0.46

- Tool evaluation (1-10) : average = 8.3, minS.E.= 0.86

- Self-confidence on the achieved results = 92%, , minS.E = 0.79

- Satisfaction as composers of “Mobile Apps” (1-4) : average = 3.4, minS.E.= 0.52

20

A satisfaction questionnaire was filled out by each user after the experimental tasks

Tasks were accomplished

easily;

Users felt very satisfied

maristella matera, matteo picozzi, cinzia...

Documents