An Intelligent Broker Architecture for Context-Aware Systems

Harry ChenDept. of CSEE, UMBC

PhD Dissertation Proposal Defense

January 2003

Outline

I. IntroductionII. Research ReviewIII. Context Broker ArchitectureIV. Research Plan & Summary

Part I. Introduction

Yesterday: Gadgets are Everything

Cool toys…

Too bad they can’t talk to each other…

Today: Communication is Everything

Sync. Download. Done.

Configuration? Too much

work…

Tomorrow: Service is Everything

Thank God! Everything is done for me!

One Step Towards Tomorrow

Context-aware Computing Brings us one step closer to the

Pervasive Computing vision Enables computer systems to anticipate

users’ needs and to act in advance An emerging paradigm to free everyday

users from manually configuring and instructing computer systems

Life is Not Perfect Building context-aware systems for

Pervasive Computing is often difficult and costly: User privacy issues when sharing personal

information Supporting resource-poor mobile devices How to reason about sophisticated contexts in a

dynamic environment Inconsistent and ambiguous contextual

knowledge Security, trust, ... (goes on and on)

My Research Objective

To develop and prototype a broker-centric agent architecture to support context-aware systems

To demonstrate this architecture can be used to reduce the difficulty and cost of building context-aware systems in an Intelligent Meeting Room environment

Context Broker Architecture

The “broker” of this architecture will Sense and reason about contexts on

the behalf of capability-limited agents Enable agents to share contextual

knowledge Protect the privacy of users Maintain consistent contextual

knowledge

Let’s talk about …

Agent Broker Context Context-aware systems

About “Agent”

Study context-aware systems using intelligent agents (context-aware agents) Autonomous and Proactive Can communicate, not just connect Have beliefs about the World Have desires and intentions

About “Broker”

Broker, an overloaded term for agents with a specialized role: Mediator: mediate communication

messages in a Multi-agent system (MAS) Facilitator: facilitate task execution

between agents to achieve cooperation Match-maker: match/recommend

service advertisements

“Broker” in My Architecture

Broker is a server agent that controls and manages the contextual knowledge of a Pervasive Computing environment: Enables agents to contribute to and

access a shared model of context Allows users to control the access of

their personal information in a context-aware environment B

About “Context”

In Merriam-Webster Dictionary [1] the parts of a discourse that

surround a word or passage and can throw light on its meaning

[2] the interrelated condition in which something exists or occurs

We are interested in [2]

Definitions of Context

In context-aware computing No unified definition of context Most of the definitions agree that

context has something to do with the interactions between the users and the computing systems

My Definition of Context Context is information that can be used

to characterize the situation of a person or an object in a Pervasive Computing environment. The identities and attributes of people and

devices The locations of people and devices The activities that people are participating in The roles and intentions of people when

participating in the activities

Context-Aware Systems (1 of 2)

A user has left his office

The system forwards

the call to a nearby phone

The system detectshis current location

Calls are forwarded to

his voice mailbox

A Call-forwarding System

The system detects the

user is in an meeting

The phone rings in his office

Context-Aware Systems (1 of 2)

Two types of contexts are used:

The system detectshis current location

The system detects the user is in an meeting

Location Context

Activity Context

Context-Aware Systems (2 of 2)

Shopping Assistant

A user enters a store Turns on his PDA

+

=

+

PDA displays the info of a store item

As the user wonders around in the store

PDA analyzes user’s personal profile

PDA recommends store items to the

user

Context-Aware Systems (2 of 2)

Three types of contexts are used:

+

=

+

PDA displays info of the store items

PDA analyzes user’s personal profile

Location Context

Identity Context&

Attribute Context

System Characteristics Context-aware agents often run on

mobile devices For agents to become context-aware,

context sensing and context reasoning mechanisms are required

Context-aware systems often exploit user information (e.g. personal profile, user location & social activity)

Research Problem

Building context-aware systems can be difficult and costly because (1) limited resources in mobile

devices (2) lack of reusable context-aware

mechanisms (3) privacy issues in accessing user

information

(1) Limited Resources in Mobile Devices

Battery Power Constraint Small devices => limited built-in sensors Big devices => too many external

sensors can be awkward Information Storage Constraint

Historic knowledge saves computation Limited storage => must intelligently

choose what to save and delete (hard problem!)

(1) Limited Resources in Mobile Devices

Computing Power Constraint To process contexts needs CPU power Limited CPU power => primitive

contexts ONLY (limited intelligence)! Communication Constraint

Contexts are “hidden” heterogeneous sources

Not knowing where are the sources and how to communicate => limited context-awareness

(2) Lack of Reusable Context-Aware Mechanisms

2 Essential Mechanisms: Context Sensing: acquiring

information from the physical environment

Context Reasoning: interpreting the information that have been acquired

In the existing systems, both are built from the scratch every time => no reuse!

Privacy is about control of information

In context-aware systems, users may not have full control of their information Sensors are hidden in the environment Information are collected without

explicit consent from the users

(3) Privacy Issues in Accessing User Information

What about sharing of information? You tell an agent something about

you because you want its service, and later it tells someone else... (you are in trouble!)

The downstream consequences of information are unknown or unspecified.

(3) Privacy Issues in Accessing User Information

Proposed Solution

Context Broker Architecture Philosophy: agents can’t do

“everything” by themselves, let’s provide a powerful server entity to help them

Rationale: the Moore’s Law for mobile computing is likely to hold; developing a centralized solution is much easier than any P2P solutions.

Context Broker Architecture

The key features: Sense and reason about contexts on

the behalf of capability-limited agents Enable agents to share contextual

knowledge Protect the privacy of users Maintain consistent contextual

knowledge

Part II. Research Review

The Purpose of Research Review

(1) To argue building context-aware systems can be difficult and costly using the existing architecture

(2) To put Context Broker Architecture research into perspective (so that we can compare and contrast it with the existing systems)

Look From Two Different Angles

Approaches to Support Context-Aware Systems

Types of Context-Aware Research

Types of Context-Aware Research

Types of Context-Aware Research

Enhancing User Interfaces

Guiding the Adaptation of System Behavior

Building PervasiveComputing Service

Enhancing User Interfaces (1 of 2)

Problem: the user interface of the existing mobile devices demand too much user attentions (i.e. cognitive and visual).

Solution: to replace the traditional user interface by enabling devices to become context-aware.

Enhancing User Interfaces (2 of 2)

Microsoft research has developed a Cassiopeia E-105 that can Active voice recording application when

detects the user is holding the device like a cell phone or microphone

Automatically reformat the screen display (landscape portrait) depending how the user holds the device

Guiding the Adaptation of System Behavior (1 of 2)

Problem: Environment changes can affect the performance of applications (e.g. using wireless PDA while walking on the street)

Solution: enabling applications to adapt their behavior in according to condition changes

A video streaming application can dynamically adjust the streaming quality of video without interrupting the viewer’s attention (Odyssey)

Widely used contexts: network bandwidth, error rate, connection setup time, and usage costs XeroxPARC Active Badge and PARC Tabs

Applications (the 1st context-aware system)

Guiding the Adaptation of System Behavior (2 of 2)

Building Pervasive Computing Services (1 of 3)

Problem: Complex computer systems are drawing humans into the world of computing Think Oxygen & Mark Weiser’s vision

Solution: enabling computers to reason and act in according to the situation of users as they carry out their every activities

Building Pervasive Computing Services (2 of 3)

In MIT’s Intelligent Room, the open/close of window curtains are automated by detecting the body position of a user in a couch

In HP’s Cooltown museum, the visits of Cooltown users are automatically documented based on what they have seen.

Building Pervasive Computing Services (3 of 3)

Different Types of Context-Aware Research

Enhancing User Interface

Guiding Behavior Adaptation

Building Pervasive

Computing Srv.

MicrosoftCassiopeia E-

105EX

Video Streaming App. (Odyssey) X

XeroxPARC Active Badge Apps X X

Cooltown Museum X

MIT Intelligent Room X

Context Broker Architecture X

Approaches to Supports Context-Aware Systems

Approaches to Support Context-Aware Systems

Directly Sensors Access

Facilitated by Middle-ware

Acquiring Context Directly from Sensors

Host Device

Context Reasoning

Contexts in the Environment

Agent

SensorSensor Sensor

Host Device

Context Reasoning

Agent

SensorSensor Sensor

Facilitated by a Middle-ware Infrastructure

Host Device

Contexts in the Environment

Agent

SensorSensor Sensor

Context Reasoning

Middle-ware

Agent

Host Device

Agent

SensorSensor Sensor

Context Reasoning

Middle-ware

Agent

Different Approaches to Support Context-Aware Systems

Direct Sensors Access

Facilitated by Middle-ware

MicrosoftCassiopeia E-

105EX

XeroxPARC Active Badge

AppsX

Video Streaming App. (Odyssey) XContext Toolkit XMIT Intelligent

Room X XContext Broker

Architecture

The Result of Research Review

Conclusion: building context-aware systems can be difficult and costly using the existing architecture No or very minimal information sharing No or limited reuse of context reasoning No explicit support for privacy protection

Context Broker Architecture belongs to Building Pervasive Computing Services Neither “direct sensors access” or

“facilitated by middle-ware”

Part III. Context Broker Architecture

Research Problems (again)

(1) Limited resources in mobile devices

(2) Lack of reusable context-aware mechanisms

(3) Privacy issues in accessing user information

What’s the Implication?

A single agent has limited capability to acquire contexts

Building context sensing and reasoning mechanisms from the scratch can be expensive

Protecting user privacy in a context-aware environment is critical

Context Broker Architecture

Host Device

Contexts in the Environment

Agent

Agent

Host Device

Agent

Agent

Middle-ware

Context Reasoning

Domain Context Broker

……

SemanticWeb DB

EnterpriseServers

Middle-ware

… …

Domain Context Broker

The architecture structures contexts in a collection of micro-worlds, called domains. E.g. a meeting room domain, a school

domain, a home domain etc. In each domain, there is a

Domain Context Broker (broker)

B

A Broker’s Job is to…

1. Acquire contexts from heterogeneous sources on the behalf of context-aware agents

2. Enable agents to contribute to and access a shared model of context

3. Allow users to use policy to control the access of their personal information

4. Detect and resolve inconsistent contextual knowledge

A Conceptual Design of the Broker

Knowledge Base

Broker Behavior

InferenceEngine

ContextAcquisitionComponent

(4)

(2)(3) (1)

(1) Knowledge Base

Ontology: domain ontology, domain heuristics and privacy ontology

Context model: information that can be used to characterize the situation of a person or an object in the domain i.e. identity, attribute, location, activity,

role and intention context

(4)

(2)(3)(1)

(2) Inference Engine

Ontology Reasoning: deduce facts that can be inferred from ontology knowledge (not context)

Context Reasoning: deduce facts that are parts of the context model (context)

Knowledge Maintenance: detect and resolve knowledge inconsistency in the context model

(4)(2)(3)

(1)

(2) Inference Engine

Hybrid Reasoning Mechanism The existing systems use ad-hoc procedures

with deductive reasoning at core to reasoning about context

I will attempt to develop a hybrid composition of logic reasoning (e.g. deduction and abduction), fuzz logic and statistical analysis (e.g. decision tree, Bayesian networks etc.) to reason about context

(4)(2)(3)

(1)

(3) Context AcquisitionComponent

Context Sensors: physical sensors and virtual sensors

Context Interpreter: procedures/rules that translates sensed data into knowledge that be processed by the Inference Engine

(4)(2)(3)

(1)

(4) Broker Behavior

A collection of protocols that the broker follows when interact with context-aware agents Privacy Policy Negotiation: the

broker forms agreement with the users before disseminating their personal info.

Knowledge Sharing: the broker enables agents to acquire contexts that are otherwise not accessible

(4)

(2)(3)(1)

(4) Broker Behavior (4)

(2)(3)(1)

Privacy Negotiation Protocol

Request for user contexts

Agent Broker User

Propose privacy policy

Reject proposal

Accept proposal

Propose modification

Propose privacy policy

Inform user contexts

Propose privacy policy

(4) Broker Behavior (4)

(2)(3)(1)

Broker Agent

Inform contexts

Accept

Disagree

Request for contexts

Inform contexts

Broker Agent

Request for contexts

Inform contexts

Refuse

Failure

Knowledge Sharing Protocols

Informing the broker of some contextual knowledge

Querying the broker for some contextual knowledge

(4) Broker Behavior (4)

(2)(3)(1)

Broker Agent

Subscribe to some context changes

Agree

[pre-conditions holds]

Refuse

Inform context changes

Knowledge Sharing Protocols

Subscribe to the broker to be notified when context changes

Part IV. Research Plan and Summary

Expected Research Contribution

To show the difficulty and cost of building context-aware systems can be greatly reduced by using the Context Broker Architecture Prototype this architecture (COBRA) Evaluate the feasibility of COBRA

through a series of experiments in an Intelligent Meeting Room environment

How Will I Know When I am Done?

(1) If I can show a working prototype of COBRA that exhibits certain system characteristics

(2) If COBRA can help to build an extensible Intelligent Meeting Room system

(3) If I can “qualitatively” compare COBRA with the existing architecture

A working prototype means…

COBRA can support: Context acquisitions (sensing and

reasoning) from heterogeneous sources Knowledge sharing between distributed

agents User privacy protection using policy rules Knowledge maintenance to resolve

inconsistent and ambiguous contexts

Help to build an extensible Intelligent Meeting Room means…

The end system is not “hard-wired” and “ad-hoc” Is built using well-defined protocols,

ontologies, APIs etc. Within a reasonable amount of effort,

other developers can use COBRA to enhance or extend the system functionalities

Comparing COBRA with the existing architecture means …

I can show why COBRA is better/worse than other system architectures For example, MIT’s Intelligent Room,

GTI’s Context Toolkit etc. But this may not substantial because

No comprehensive Pervasive Computing architecture has yet been developed

What I am proposing to work on is still relative new research

Scenario: Intelligent Meeting Room

Alice enters a conference room

The broker negotiatesprivacy policy with Alice

The broker detects Alice’s presence

B

Policy says, “can share with any agents in the room”

AB

The broker buildsthe context model

Web

1 of 2

The broker knows Alice’s role and

intention

+

Scenario: Intelligent Meeting Room

2 of 2

The projector agent wants to help Alice

The broker informsthe subscribed agents

B A

The projector agentasks slide show info.

B

The broker acquires the slide show info.

BWeb

The broker informs the

projector agent

B

The projector agentsetup the presentation

Critical Technologies

Semantic Web Languages & Tools Web Ontology Language (OWL),

RDF/RDFS and XML JESS (Java Expert System Shell)

Rule-based reasoning engine in Java FIPA Standards & Framework

Java agent libraries: JADE & LEAP

Semantic Web in COBRA

Benefit: a new source of context! Vast information: web services,

personal websites, public announcements, news etc.

Useful => determining context, resolving inconsistent knowledge, predicting the future context etc.

COBRA implementation: ontologies & privacy policies

JESS in COBRA Benefit: practical, efficient & Java-

compatible. Supports both forward & backward-

chaining Good experience in building HP’s

CoolAgent RS (context-aware system) COBRA implementation: inference

engine, heuristic rules & knowledge maintenance

FIPA Standards in COBRA Benefit: standards for programming

distributed agents; readily available Java framework FIPA: agent management,

communication, life-cycle and more. JADE/LEAP: VERY Good experience; run

agents on GSM phones and Pocket PC COBRA implementation: broker

behavior, Intelligent Meeting Room

Research Plan (for the next 12-18 months)

Stage 1Est. 1-2

mo.

Develop design spec. for the Intelligent Meeting Room System & COBRA

Stage 2Est. 3-5

mo.Prototype Domain Context Broker

Stage 3Est. 3-5

mo.

Prototype a demo of the Intelligent Room System (midterm evaluation)

Stage 4Est. 2-3

mo.

Evaluate the feasibility of COBRA by conducting experiments

Stage 5Est. 2-3

mo.

Complete PhD. dissertation writing

Summary The initial design of Context Broker

Architecture shows great promise in reducing the difficult and cost of building context-aware systems. COBRA will enable resource-limited agents

to contribute to and access a shared context model

COBRA will allow users to control the access of their personal information in a context-aware environment

Question?

Related material

http://users.ebiquity.org/~hchen4/phd/