Mining Minds

Mr. Amjad Usman

19-July-2014

KHU

High-level Context Awareness

2/Presentation Outline

• Introduction

• Motivation

• Related Work

• Limitations

• Proposed Architecture

• Tools and Technologies

• Development Timeline

• Current Status

3/Introduction

• According to Dey, Abowd and Salber (2001)• Context is any information that can be used to characterize the situation of

entities (i.e. whether a person, place or subject) that are considered relevant to the interaction between a user and an application, including the user and the application themselves

• Means Context is very dynamic and transient

• Low-level Context is the raw data coming from sensors / source

• High-level Context is the abstract information extracted from logically relevant low-level context

4/Motivation

• End-users are interested in high-level context, not raw data• For example“It’s raining” (high-level) is preferred rather than “humidity: 77%“ (low-level)

• To discover Higher-level Context Information• Context fusion is required

• Find the semantic relationship

• By integrating all the information relevant to one’s life• It can give us a real sense of Life logging

5/Motivation

• Context Information• Sensor data from sensors• Activity data from smartphones• Nutrition data• Social media data

• Recommending Services• Lifestyle & Lifecare• Exercise and nutrition • Social interaction• Behavior analysis &

prediction

Behavior Modeling

Lifestyle Analysis,

Prediction, andRecommendati

ons

Context Informatio

n

Context Informatio

n

Context Informatio

n

6/

User Profi

le

Social

Media

Life log

Environmen

t

Proposed Idea

IntegrationSocial MediaInformation

Diet Information

Activity Information

ProfileInformation

Environment Information

Beh

avio

r

Mod

elin

g

Activity

Context Information Independent logs

Life log Information Semantically related

7/Related Work

Ontology/ Subdomain

CoBrA-Ont[2003]

CoDAMoS[2004]

SOUPA[2005]

Delivery Context [2009]

Device X X X

Environment X X X

Location X X X X

Network X

Role X X

Time X X X

User X X X

Implementation Available

X X X

8/Related Work

Ontology/ Subdomain

CONON[2003]

Situation Ont[2004]

mIO[2005]

PiVOn[2009]

PalSPOT[2011]

Device X X X X X

Environment X X X X X

Location X X X X X

Network X X

Role X X

Time X X X X X

User X X X X X

Implementation Available

X X

9/Limitations

• Domain Specific Context Models

• Information exist in different types of Logs but are stand alone

• No integrated system

• Very few systems consider social interaction

• None of the existing systems use any behavioral model

• High level contexts are extracted using context interpretation and aggregation techniques regardless of

• Context verification, validation and fusion

10/Proposed Architecture

Long/Short-term Behavior AnalysisContext Awareness

Mapper and Transformer

Context Interpreter

Context Analyzer

Rulebase

Parser

Decision Making

DTD2OWL OWL Ontology XML2OWL

QueryGenerator

ActivityRetrieval

Match Making

Rule-basedFiltering

Decision PropagationSituation Analyzer

Prediction and ReasonerRecom

mendation M

anager

Low Level Context-awareness

HDFS Data Interface

Intermediate Data

Life log Modeling

Context Receiver

Context Representation

Context Verification

Context Fusion Context Logging

Behavior Modeling

ParserLife Log

Repository

Feature Selection

Pattern Classification

Pattern Identification

Life log Extractor

11/ScenarioContext Awareness | Long/Short-term Behavior Analysis

Prediction and Reasoner Recommendation Manager

Low Level context-awareness

HDFS Data Interface

Intermediate Data

Long/short termBehavior Analysis

Mapper and Transformer

Context Interpreter

Context AnalyzerParser

Decision Making

DTD2OWL OWL Ontology

QueryGenerator

ActivityRetrieval

Match Making

Rule-basedFiltering

Decision PropagationSituation Analyzer

7

4

2

11

6

7

3

Rulebase

5

7

Activities detected: walkingLocation detected: KitchenTime noticed: 12:00:00

Ali Exercising Sentiment: tired

Rule 1 (activity=eating AND time<8:00:00) Taking BreakfastRule 2 (activity=eating AND time=12:00:00) Taking LunchRule 3 (activity=eating AND time=18:00:00) Taking Dinner

<activity>eating</activity><activityLocation>Kitchen </activityLocation><activityTime>12:00:00 </activityTime><performedBy>Ali </performedBy>

"SELECT ?activityName ?hasConsequentAction ?type ?performedBy ?time WHERE { <" + strNS + strActivity + "> <" + strNS + "hasName> ?activityName ." +"<" + strNS + strActivity + "> };

In Kitchen Eating 12:00:00

Taking Lunch in Kitchen

Rule 2 Matched(activity=eating AND time=12:00:00) Taking Lunch

XML2OWL

12/ScenarioContext Awareness | Long/Short-term Behavior Analysis

Context Receiver

Context Representation

Context Fusion

Context Verification and LoggingLife Log Repository

Behavior Analysis

Life Log Extractor

Parser

Context ConverterContext Representation Context Mapper

Horizontal Context Fusion

Vertical Context Fusion

DataExtractor

Data Logger

Query Formulation

Data Fetcher Data Processing

Pattern Classification Pattern Identification Feature Selection

Consistency Verification

SemanticStructural

Log ContextExistence Verification

Low Level context-awareness

High Level Context-awareness

1 1 HDFS Data Interface

Intermediate Data

1

Prediction and Reasoner Recommendation Manager

3

10

9

8

7

6

5

4

10

2

Verified Data

<activity>eating</activity><activityLocation>Kitchen</activityLocation><activityTime>12:30:00</activityTime><performedBy>Ali</performedBy>

Lunch, Exercising

Complete Information of User like profile, activities performed when and where, tweets, etc

Behavior AnalysisLunch No proper timingExercising Regular

High Level Context: Exercising Structured Data

Ali,Lunch,Hotel,2014-05-02, 12:00:00

walking, sitting, eating, tweet: tired

Activity detected: walking, eatingLocation detected: KitchenTime: 12:30:00

13/Tools and Technologies

• Knowledge Representation–RDF / RDFS / OWL Ontologies–Protégé as IDE

• Programming Language & API– Java– Jena, Twitter

• Querying Language–SPARQL

• Reasoner–Racer Pro / Pellet / FaCt++

14/Development Time LineFour Year Work Break Down

Literature Study:Context Modeling

Analytical Study:Context Conversion

Analytical Study:Context Interpreta-

tion

Literature Study:Life log System

Context logging in Life log

Context Integration& Verification

Literature Study:Behavior Representa-

tion

Study:Behavior Modeling

Algorithmic Study:Behavior Analysis

System Develop-ment

Unit Testing

Integration Testing

Context Model

Context MappingTechnique

Context Analyzer

Life log Model

Life log Parser

Fusion + VerificationTechniques

Features SelectionBehavioral Model Cre-

ation

Model SelectionPattern Identification

Algorithm: Long-term /

Short-term Analysis

High-levelContext Awareness

Test Report

Test Report

First Year Second Year Third Year Fourth Year

Context awareness Life log Management Behavioral Analysis Documentation

Prototype Develop-ment

Prototype Develop-ment

Prototype Develop-ment Technical Guide

15/Current Status

• Context Awareness– Literature Survey regarding

– Context Modeling Techniques– Context Conversion & Mapping Algorithms

• Long/Short-term Behavior Analysis – Literature Survey regarding Lifelog Design and Development

• LifeLog Model: OWL+Dynamic/Static• Storage: Ont-RDB / JenaTDB / RDF• Nature & type of Logs

16/Context Modeling Work Flow

17/What to store in Life log?

18/Behavior Analysis & Prediction Work Flow

QuestionsThank You!

Email address: amjad.usman@oslab.khu.ac.kr

Cell Number: +82-10-4769-8867

20/Long/Short-term Behavior AnalysisDeployment Diagram

21/Proposed Architecture

Prediction and Reasoner Recommendation Manager

Low Level context-awareness

HDFS Data Interface

Intermediate Data

Long/Short-termBehavior Analysis

Context Awareness

22/Proposed Architecture

Prediction and Reasoner Recommendation Manager

Low Level context-awareness

HDFS Data Interface

Intermediate Data

Long/Short-termBehavior Analysis

Mapper and Transformer

Context Interpreter

Context Analyzer

Rulebase

Parser

Decision Making

DTD2OWL OWL Ontology XML2OWL

Query Generator

Activity Retrieval

Match Making

Rule-based Filtering

Decision PropagationSituation Analyzer

Context Awareness | Long/Short-term Behavior Analysis

23/Proposed Architecture

Low Level context-awareness

HDFS Data Interface

Intermediate Data

Context-Awareness Context Receiver

Context Representation

Context Fusion(Vertical / Horizontal)

Context Verification and Logging

Life Log Repository

Behavior Analysis

Life Log Extractor

Parser

Prediction and Reasoner Recommendation Manager

Context Awareness | Long/Short-term Behavior Analysis

24/Proposed Architecture

Long/Short-term Behavior AnalysisContext Awareness

Mapper and Transformer

Context Interpreter

Context Analyzer

Rulebase

Parser

Decision Making

DTD2OWL OWL Ontology XML2OWL

QueryGenerator

ActivityRetrieval

Match Making

Rule-basedFiltering

Decision PropagationSituation Analyzer

Prediction and ReasonerRecom

mendation M

anager

Low Level Context-awareness

HDFS Data Interface

Intermediate Data

Lifelog Modeling

Context Receiver

Context Representation

Context Verification

Context Fusion Context Logging

Behavior Modeling

ParserLife Log

Repository

Life Log Extractor

Behavior Checker

Model CreaterBehavior Descriptor

Model Validator

Behavior Analyzer

25/Motivation

• To integrate the different context information emerging from diverse sources to identify user’s behavior in order to analyze the user’s lifestyle and provide recommendations to promote active lifestyle

Social MediaInformation

Diet Information

Activity Information

ProfileInformation

Environment Information

Lon

g / S

hort-te

rm

Beh

avio

r Mod

elin

g

Life log

26/Motivation

IntegrationSocial MediaInformation

Diet Information

Activity Information

ProfileInformation

Environment Information

Social MediaInformation

Diet Information

Activity Information

Environment Information

ProfileInformation

Short-term

Behavior

Modeling

Long-term

Behavior

Modeling