slua: towards semantic linking of users with actions in crowdsourcing

www.insight-‐centre.org

SLUA: TOWARDS SEMANTIC LINKING OF USERS WITH ACTIONS IN CROWDSOURCING

Umair ul Hassan, Sean O’Riain, Edward Curry INSIGHT Centre for Data Analy4cs

Na4onal University of Ireland, Galway

1st International Workshop on Crowdsourcing the Semantic Web, CrowdSem’13, Sydney, Australia


Paper Overview

•  MoJvaJon –  MulJple crowdsourcing plaBorms –  Lack of tools for finding tasks –  Need to query across plaPorms for skills and knowledge of workers

•  Problem –  Enabling interoperability across crowdsourcing plaBorms –  Suppor4ng users in their search for tasks –  Enable task and user matching services

•  ContribuJon –  An ini4al lightweight ontology for describing crowd sourced tasks and

users with regard to human capabili4es, ac4ons, and rewards

2


Agenda

•  Mo4va4on – Crowdsourcing Landscape – Seman4c Heterogeneity

•  Challenges •  SLUA Ontology •  Examples •  Summary

3


4

Crowdsourcing Landscape


Crowdsourcing Landscape

5


Challenges

•  Difficult to interoperate across crowdsourcing systems and plaPorms –  e.g. searching for appropriate workers on StackExchange for Wiki edi4ng tasks

•  VariaJons of data semanJcs across systems and plaPorms – Different APIs used by current marketplaces

•  ExisJng Taxonomies plaPorm-‐centric –  Categorize crowdsourcing plaBorms instead of tasks –  LiPkle considera4on human of factors such as ac4ons, capabili4es, mo4va4on

6


Heterogeneous Crowds

•  Mul4ple requesters, tasks, workers, plaBorm

7

Collaborative Data Curation

Tasks Workers

Cyber Physical Social System

Platforms


PlaBorm Heterogeneity

Tasks Human AcJons Required CapabiliJes Rewards

Wikipedia Create Content Edit Content Moderate Content

Write text Include references Highlight mistakes

Domain Knowledge Wri4ng Research

Social Good

Quora Ask Ques4ons Answer Ques4ons

Write text Domain Knowledge

Reputa4on

Amazon Mechanical Turk

Micro Tasks Transcribe, Translate, Categorize, etc.

Various Capabili4es Money

TaskRabbit Physical Tasks Collect Item Deliver Item Shop Item, etc.

Various Capabili4es Money

Microtask Form Filling Scan Correc4on Data Verifica4on

Play games Online gaming Fun

8


Proposed Solu4on •  A common model for describing tasks in crowdsourcing

•  Seman4cally Linked Users and Ac4ons (SLUA)

9

– Lightweight semanJc descripJon of crowdsourcing tasks in terms of human capabiliJes, acJons, and rewards

•  SLUA – Gaelic word for… …“Crowd”


SLUA Design Medthdology

1.  Enumerate similar terms on crowdsourcing plaBorms

2.  Define the main concept in each group of terms

3.  Compare with exis4ng ontologies 4.  Define core classes and their rela4onships 5.  Extend core classes with subclasses 6.  Create example instances

10


•  Terms used for similar concepts in crowdsourcing plaBorms

Crowdsourcing Terminology

11

Amazon Mechanical

Turk Mobileworks ShorXask CrowdFlower

Task HIT Task ShortTask Microtask

User Worker Requester

Worker Developer

Solver Seeker

Contributor Customer

Capability Qualifica4on Filter

Reward Payment Payment Reward Payment


Conceptual Requirements

•  Task: A unit of work to be performed by people in the crowd

•  AcJon: The cogni4ve or psychomotor ac4vity that leads towards the comple4on of a task

•  User: The human par4cipant, commonly described as “worker” in crowdsourcing marketplaces.

•  Capability: The human ability, knowledge, or skill that allows a user to perform the necessary ac4ons for task comple4on.

•  Reward: A core concept to the mo4va4on of people in the crowd

12


Exis4ng Ontologies Concept PIMO TMO HRM-‐O FOAF SIOC

Task Task Task

AcJon

User Person Job Seeker Person UserAccount

Reward Compensa4on

Reputa4on

Money Salary

Fun

Altruism

Learning

Capability

Loca4on Loca4on

Skill Skill

Knowledge

Ability Ability

Availability Interval

13

Personal Information Management Ontology (PIMO) Task Management Ontology (TMO) Human Resource Management Ontology (HRM-O) Friend of a Friend (FOAF) Semantically Interlinked Online Communities (SIOC)


SLUA Core Classes and Proper4es

14

Reward

Action

Capability

User Task

offers earns

includes performs

requires possesses


SLUA Sub-‐classes

•  Capability –  The ability of people to do things -‐ both the capacity and the opportuni4es to do things.

– Main capabili4es in literature •  Knowledge, Skill, Ability, and Others (e.g. Loca4on, Availability)

•  Reward –  The benefit generated from the use of capability in both labour market and non-‐labour market ac4vi4es.

– Main rewards in literature •  Reputa4on, Money, Fun, Learning, Altruism or Social Good

15


SLUA Ontology

16

Reward

Action

Capability

User Task

offers earns

includes performs

requires possesses

Location Skill Knowledge Ability Availability

Reputation Money Fun Altruism Learning

subClassOf

subClassOf


Example Task

17

hPp://www.wikipedia.org/wiki/A3_road/tasks/1

slua:Task Please consider adding full cita4ons to

the Wikipedia ar4cle

:loc1

slua:Loca4on

hPp://live.dbpedia.org/resource/London

:knw1

slua:Knowledge

hPp://live.dbpedia.org/resource/London

:rw1

slua:Reward

slua:Reputa4on

:ac1

slua:Ac4on Wiki page edit

rdfs:label

rdfs:label

a

a

a

a

a

a

slua:offers slua:includes slua:requires

slua:requires


SLUA in Ac4on

SLUA Mediated Task Routing

18

Worker Profiling

Task Modelling

Task Routing

Matching

Cold Start

Ordering

SLUA Mediated Infrastructure Services Application Interface, User Interface, Identity Management,

Notification Services

Task Model

Capability Model

Capability Models Worker

Profiless

Worker Profiles

Worker Profiles Collaborative

Data Curation

Tasks

Cyber Physical Social System

Workers


Collabora4ve Data Cura4on

19

DQ Rules & Algorithms

Entity Linking Data Fusion

Relation Extraction

Human Computation

Relevance Judgment

Data Verification Disambiguation

Clean Data Internal Community - Domain Knowledge - High Quality Responses - Trustable

Web of Data

Databases

Textual Content

Programmers Managers

External Crowd - High Availability - Large Scale - Expertise Variety


Monitored Physical Environment

Cyber-‐Physical Social Systems

20

Sensor Data Collection

Human Actuation

Energy Decision Models

Temperature of Room 202e

Close Window Room 202e

Located near Room 202e Located near Room 202e


Summary & Future Work

•  SLUA is an ini4al step towards defining a light-‐weight ontology for describing tasks, acJons, users, rewards, and capabiliJes in crowdsourcing plaBorms

•  Future Plans –  Enumerate design with addi4onal crowdsourcing plaBorms

–  Prototype SLUA-‐based for cross plaBorm query –  Task rou4ng system used match between tasks and users with SLUA descriptors

21


Further Reading

Ontology Available at: hXp://vocab.deri.ie/slua U. Ul Hassan, S. O’Riain, and E. Curry, “SLUA: Towards SemanJc Linking of Users with AcJons in Crowdsourcing,” in 1st InternaJonal Workshop on Crowdsourcing the SemanJc Web, 2013. hXp://deri.ie/users/umair-‐ul-‐hassan

22

International Workshop on “Crowdsourcing the Semantic Web” Sydney, 21 October 2013

slua: towards semantic linking of users with actions in crowdsourcing

Technology