Analyzing and Specifying Concerns for

DaaS

Hong-Linh Truong

Distributed Systems Group,

Vienna University of Technology

truong@dsg.tuwien.ac.athttp://dsg.tuwien.ac.at/staff/truong

1ASE Summer 2014

Advanced Services Engineering,

Summer 2014, Lecture 4

Advanced Services Engineering,

Summer 2014, Lecture 4

Outline

What are data concerns and why their are

important

Issues in DaaS concerns

Analysis and specification of DaaS concerns

Broad issues in DaaS concerns provisioning

ASE Summer 2014 2

........

What are data concerns?

datadata DaaSDaaS.... data assetsdata assets

APIs, Querying, Data Management, etc.

Located

in US?

free?

price?

redistribution?Service

quality?

3ASE Summer 2014

Quality of data? Privacy

problem?

........

DaaS concerns

ASE Summer 2014 4

datadata DaaSDaaS.... data assetsdata assets

Data

concerns

Quality of

dataOwnership

PriceLicense ....

APIs, Querying, Data Management, etc.

DaaS concerns include QoS, quality of data (QoD),

service licensing, data licensing, data governance, etc.

DaaS concerns include QoS, quality of data (QoD),

service licensing, data licensing, data governance, etc.

Why DaaS/data concerns are

important?

Too much data returned to the

consumer/integrator are not good

Results are returned without a clear usage and

ownership causing data compliance problems

Consumers want to deal with dynamic changes

5

Ultimate goal: to provide relevant data with

acceptable constraints on data concerns in

different provisioning models

Ultimate goal: to provide relevant data with

acceptable constraints on data concerns in

different provisioning models

ASE Summer 2014

Example: mashup (1)

Composition of Yahoo! Boss News Search,

Google News Search , and Flickr

recent news and high-qualified images, but free-

of charge, related to "Haiti earthquake"

6

Hong Linh Truong, Marco Comerio, Andrea Maurino, Schahram Dustdar, Flavio De Paoli, Luca Panziera: On

Identifying and Reducing Irrelevant Information in Service Composition and Execution. WISE 2010: 52-66

Hong Linh Truong, Marco Comerio, Andrea Maurino, Schahram Dustdar, Flavio De Paoli, Luca Panziera: On

Identifying and Reducing Irrelevant Information in Service Composition and Execution. WISE 2010: 52-66

ASE Summer 2014

7

Example: mashup (2)

ASE Summer 2014

8

If the composer is aware of context

and quality parameters

Possible mappings of context and quality

requirements

but it is a tedious task and hard to be automated and we

are not sure we have a correct mapping.

but it is a tedious task and hard to be automated and we

are not sure we have a correct mapping.

ASE Summer 2014

Example: open data (1)

ASE Summer 2014 9

10

Example: open data (2)

Retrieve big

datasets from

services for further

extraction,

transform or data

composition

activities

ASE Summer 2014

11

Example: open data (3)

Example: study the population growth and

literacy rate from 1990-2009 for all countries in

the world

Without QoD: get datasets and perform mashup

ASE Summer 2014

12

Example: open data (4)

With QoD support:

Population annual growth rate (percent):

dataelementcompleteness= 0.8654708520179372,

datasetcompleteness=0.7356502242152466;

Adult literacy rate (percent):

dataelementcompleteness=0.5874439461883408

datasetcompleteness=0.04349775784753363

Should we retrieve the data and perform data

composition?

Should we retrieve the data and perform data

composition?

ASE Summer 2014

Countries\Year 1990 ... 2009

1

...

223

223

elements

13

Example: smart environments

Smart environments with several low level sensors:

Recognize human activities: idle, relaxing, and cleaning

up,

Provide context information for adaptive service

discovery and execution

E.g., FP7 SM4All, FP7 EU OPPORTUNITY

Virtual Sensor-as-a-Service provides human activities

Sensors in smart cities for sustainability governance

E.g., Xively, Pacific Controls

ASE Summer 2014

14

Example: smart environments (2)

PoC: Probability of Correctness

QoC: Quality of Context

VSS: Virtual Sensor Service

CMS: Context Management Service

CCS: Context Consumer Service

AC: Appliances Control (AC)

AM: Ambiance Management

Atif Manzoor, Hong Linh Truong, Christoph

Dorn, Schahram Dustdar: Service-centric

Inference and Utilization of Confidence on

Context. APSCC 2010: 11-18

Atif Manzoor, Hong Linh Truong, Christoph

Dorn, Schahram Dustdar: Service-centric

Inference and Utilization of Confidence on

Context. APSCC 2010: 11-18

ASE Summer 2014

WHAT ARE OTHER CASES

WHERE DAAS CONCERNS

ARE IMPORTANT FOR?

Discussion time

ASE Summer 2014 15

Issues on DaaS concerns (1)

DaaS concern models

Unstructured description of context, QoS and

quality of data (QoD)

Different specifications and terminologies

Mismatching semantics of information about

services and data concerns

16ASE Summer 2014

Issues on DaaS concerns (2)

DaaS APIs

No/Limited description of data and service

usage

No API for retrieving quality and context

information

No quality and context information associated

with requested data

17ASE Summer 2014

Issues on DaaS concerns (3)

Evaluation techniques

Missing evaluation of compatibility of context

and concerns for multiple DaaS and data

assets

Missing evaluation techniques to filter

large/irrelevant data quantity

18

Require a „holistic integration“ of information models,

APIs and evaluation techniques for DaaS concerns!

Require a „holistic integration“ of information models,

APIs and evaluation techniques for DaaS concerns!

ASE Summer 2014

19

Solutions needed

ASE Summer 2014

Developing techniques for context and DaaS concerns evaluation

On-the-fly data concerns evaluation

Concerns compatibility evaluation and composition

Developing context and DaaS concerns that can be accessed via open APIs

APIs extension External DaaS information service

Developing meta-model and domain-dependent semantic representations for quality and context information specifications

Reconciliation of DaaS concern terms

Linked DaaS concerns models

WHY CONTEXT IS

IMPORTANT?

Discussion time

ASE Summer 2014 20

DaaS concerns analysis and

specification

Which concerns are important in which

situations?

How to specify concerns?

21ASE Summer 2014

Hong Linh Truong, Schahram Dustdar On analyzing and specifying concerns for data as a service. APSCC 2009: 87-

94

Hong Linh Truong, Schahram Dustdar On analyzing and specifying concerns for data as a service. APSCC 2009: 87-

94

The importance of concerns in

DaaS consumer‘s view – data

governance

ASE Summer 2014 22

Important factor, for example, the security and

privacy compliance, data distribution, and auditing

Storage/Database

-as-a-Service

Storage/Database

-as-a-Servicedatadata DaaSDaaS

Data governance

The importance of concerns in DaaS

consumer‘s view – quality of data

Read-only DaaS

Important factor for the

selection of DaaS.

For example, the

accurary and

compleness of the data,

whether the data is up-to-

date

CRUD DaaS

Expected some support

to control the quality of

the data in case the data

is offered to other

consumers

23 23ASE Summer 2014

The importance of concerns in

DaaS consumer‘s view– data and

service usage

Read-only DaaS

Important factor, in

particular, price, data

and service APIs

licensing, law

enforcement, and

Intellectual Property

rights

CRUD DaaS

Important factor, in

paricular, price, service

APIs licensing, and law

enforcement

ASE Summer 2014 24

The importance of concerns in

DaaS consumer‘s view – quality of

service

Read-only DaaS

Important factor, in

particular availability and

response time

CRUD Daas

Important factor, in

particular, availability,

response time,

dependability, and security

ASE Summer 2014 25

The importance of concerns in DaaS

consumer‘s view– service context

Read-only DaaS

Useful factor, such as

classification and service

type (REST, SOAP),

location

CRUD DaaS

Important factor, e.g.

location (for regulation

compliance) and versioning

ASE Summer 2014 26

WHAT ARE OTHER

IMPORTANT ISSUES? ADD

YOUR FINDING!

Discussion time

ASE Summer 2014 27

Conceptual model for DaaS

concerns and contracts

28ASE Summer 2014

Capability concerns

Data Quality capabilitiesBased on well-established research on data quality

Timelineness, uptodate, free-of-error, cleaning, consistency,

completeness, domain-specific metrics, etc.

We mainly support the specification of QoD metrics for the whole

DaaS but possible to extend to the service operation level

Data Security/Privacy capabilitiesData protection within DaaS, e.g. encryption, sensitive data

filtering, and data privacy

Many terms are based on the W3C P3P

29ASE Summer 2014

Capability concerns (2)

Auditing capabilities

Logging, reporting (e.g., daily, weekly, and monthly),

and warning

Support system maintenance, SLA monitoring, billing,

and taxation

Data lifecycle

Backup/recovery, distribution (e.g., a service is in

Europe but data is stored in US), and disposition

Support system maintenance but also regulation on

data

30ASE Summer 2014

Capability concerns (3)

Data and service license

Usage permission: for data (distribution, transfer,

personal use, etc.) and for service APIs (adaptation,

composition, derivation, etc.)

We utilize some terms from ODRL/ODRL-S

Copyrights

Liability: e.g., who is reponsible for the loss due to a

network disruption?

Law enforcement (e.g., US or European court)

Domain specific Intellectural property rights

31ASE Summer 2014

Data source concerns

A DaaS may utilize data from many sources.

Similar DaaSs may utilize data from the same source

Data source properties

Name, e.g. ddfFlus or DataFlux

Size

Timespan: the duration of collected data

Update Frequency: how offen the data is updated

etc

32ASE Summer 2014

Service context concerns

Location:

Selecting a DaaS in Amazon US Zone or European Zone?

Service Type: REST or SOAP?

Level of Service

Service Classification

Based on UNSPSC Code Classification Services

Data Classification

Service/data versioning

33ASE Summer 2014

34

XML Diagram for the DaaS

capability specification

34ASE Summer 2014

35

XML Diagram for DaaS specification

ASE Summer 2014

36

Implementation (1)

Check http://www.infosys.tuwien.ac.at/prototyp/SOD1/dataconcernsCheck http://www.infosys.tuwien.ac.at/prototyp/SOD1/dataconcerns

ASE Summer 2014

37

Implementation (2)

Data privacy concerns are annotated with WSDL

and MicroWSMO

ASE Summer 2014

38

Implementation (3)

Joint work with

Michael Mrissa, Salah-Eddine Tbahriti, Hong Linh

Truong: Privacy Model and Annotation for

DaaS. ECOWS 2010: 3-10

Michael Mrissa, Salah-Eddine Tbahriti, Hong Linh

Truong: Privacy Model and Annotation for

DaaS. ECOWS 2010: 3-10

ASE Summer 2014

HOW TO MODEL DOMAIN-

SPECIFIC DATA CONCERNS?

HOW TO DEAL WITH THE BIG

SCALE (BIG DATA)?

Discussion time

ASE Summer 2014 39

Recall -- stakeholders in data

provisioning

ASE Summer 2014 40

Data

Data Provider

• People (individual/crowds/organization)

• Software, Things

Data Provider

• People (individual/crowds/organization)

• Software, Things

Service Provider

• Software and people

Service Provider

• Software and people

Data Consumer

• People, Software, Things

Data Consumer

• People, Software, Things

Data Aggregator/Integrator

• Software

• People + software

Data Aggregator/Integrator

• Software

• People + software

Data Assessment

• Software and people

Data Assessment

• Software and people

41

Populating DaaS concerns

DaaS

Concerns

evaluate, specify,

publish and manage

specify, select,

monitor, evaluate

monitor and

evaluate

The role of stakeholders in the most trivial view

Data Aggregator/Integrator

Data Consumer

Data Assessment

Service Provider

Data Provider

ASE Summer 2014

Support DaaS concerns selection

42

Service Information

Management

Service

service/data

information, including

concerns

DeXIN

Data

Consumer

External

sources

SECO2

1. Muhammad Intizar Ali, Reinhard Pichler, Hong Linh Truong, Schahram Dustdar: Data Concern Aware Querying

for the Integration of Data Services. ICEIS (1) 2011: 111-119

2. Marco Comerio, Hong Linh Truong, Flavio De Paoli, Schahram Dustdar: Evaluating Contract Compatibility for

Service Composition in the SeCO2 Framework. ICSOC/ServiceWave 2009: 221-236

1. Muhammad Intizar Ali, Reinhard Pichler, Hong Linh Truong, Schahram Dustdar: Data Concern Aware Querying

for the Integration of Data Services. ICEIS (1) 2011: 111-119

2. Marco Comerio, Hong Linh Truong, Flavio De Paoli, Schahram Dustdar: Evaluating Contract Compatibility for

Service Composition in the SeCO2 Framework. ICSOC/ServiceWave 2009: 221-236

ASE Summer 2014 42

43

From capability/context to

DaaS contract

43

Search

properties of

DaaSs

Define and

negotiate contract

terms

Contracts

DaaS Capabilities,

Context, Data

Source

Consumer-specific

concerns

A DaaS contract includes a set of generic, data-

specific and service-specific conditions established

based on concerns (see Lecture 6)

A DaaS contract includes a set of generic, data-

specific and service-specific conditions established

based on concerns (see Lecture 6)

ASE Summer 2014

WHAT CAN WE DO MORE

WITH INFORMATION ABOUT

DAAS CONCERNS?

Discussion time

ASE Summer 2014 44

Providing data concerns

Importance issue for DaaS service/data

providers

How do data concerns provisioning models influence

service provisioning models, service execution and

management?

Some complex tradeoffs

Increase quality of data

increase computational effort ?

reduce the performance ?

Providing data concerns

Need more resources, Need more data

ASE Summer 2014 45

Data concerns in multi-dimensional

elasticity

Simple

dependency

flows (increase nr. of services)

(increase) (increase response time)

(increase cost)

How do we maintain

our systems to deal

with such complex

dependencies?

How do we maintain

our systems to deal

with such complex

dependencies?

ASE Summer 2014 46

Exercises

Read mentioned papers

Visit DaaS mentioned in previous lectures

Analyze existing DaaS concerns

Examine how they specify and publish concerns

Investigate possible concerns when merging

data from different types of DaaS

Open government data and near-realtime data from

sensors

Work on some DaaS concern dependency

scenarios

ASE Summer 2014 47

48

Thanks for your attention

Hong-Linh Truong

Distributed Systems Group

Vienna University of Technology

truong@dsg.tuwien.ac.at

http://dsg.tuwien.ac.at/staff/truong

ASE Summer 2014