january 2008 ucsb 1 the vision and reality of ubiquitous computing prof. henning schulzrinne dept....

January 2008 UCSB 1

The Vision and Reality of Ubiquitous Computing

Prof. Henning Schulzrinne

Dept. of Computer Science

Columbia University(with Arezu Moghadam, Ron Shacham, Suman Srinivasan,

Xiaotao Wu and other IRT members as well as the IETF GEOPRIV and ECRIT WGs)

January 2008 UCSB 2

Overview

• The original vision of ubiquitous computing

• User challenges

• Beyond terminal mobility

• Location as new core service

• Beyond connectivity: 7DS

• CS technology into reality

January 2008 UCSB 3

Ubiquitous computing ubiquitous communications

• “It is invisible, everywhere computing that does not live on a personal device of any sort, but is in the woodwork everywhere.”

• Weiser’s original vision (“Nomadic Issues in Ubiquitous Computing”, 1996)– “one person, many computers”– many computers embedded in environment– dynamic ownership– PC phonebooth– “IR use will grow rapidly”

• Updated version, 2007– not physically invisible, but transparent– emphasis on communications, not computing– most devices are mobile (or nomadic)– cheap electronics personal devices– radio (channelized and UWB)

January 2008 UCSB 4

Overview


• User challenges• Beyond terminal mobility• Location as new core service• Universality: 7DS• CS technology into reality

January 2008 UCSB 5

User challenges vs. research challenges

• Are we addressing real user needs?– Engineering vs. sports scoring

• My guesses

reliability

ease of use

cost

no manual

integration

limited risk

phishingdata loss

no re-entryno duplication

January 2008 UCSB 6

Example: Email configuration

• Application configuration for (mobile) devices painful

• SMTP port 25 vs. 587• IMAP vs. POP• TLS vs. SSL vs. “secure

authentication”• Worse for SIP...

January 2008 UCSB 7

Example: SIP configuration

• highly technical parameters, with differing names• inconsistent conventions for user and realm• made worse by limited end systems (configure by multi-tap)• usually fails with some cryptic error message and no

indication which parameter• out-of-box experience not good

partially explains

January 2008 UCSB 8

Mobile why’s

• Not research, but examples of real annoyances• Why does each mobile device need its own power supply?• Why do I have to adjust the clock on my camera each time I travel?• Why do I have to know what my IMAP server is and whether it uses

TLS or SSL?• Why do I have to type in my address book?• Why do I have to “synchronize” my PDA?• Why do I have to manually update software?• Why is connecting a laptop to a projector a gamble?• Why do we use USB memory sticks when all laptops have 802.11b?

January 2008 UCSB 9

Consumer wireless & mobile devices

MSN Direct weather

Prius key Garage door opener TAN display

Water leak alarm

wireless door bell

SPOT watch

January 2008 UCSB 10

Mobile systems - reality

• idea: special purpose (phone) --> universal communicator

– idea is easy...

• mobile equipment: laptop + phone– sufficiently different UI and capabilities

• we all know the ideal (converged) cell phone

• difficulty is not technology, but integration and programmability

– (almost) each phone has a different flavor of OS

– doesn’t implement all functionality in Java APIs

– no dominant vendor (see UNIX/Linux vs. Microsoft)

– external interfaces crippled or unavailable• e.g., phone book access

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.











GPS

location data


Example: displays and speakers


The mobile ubiquitous challenge

Mobile phone

Mobile Internet access

Interconnected devices“Internet of things”


What do we need?

• Standards, not new technology• Radio connectivity

– 802.11a/b/g/n, 802.15.4 – better discovery of networks

• Location information everywhere• Discovery: devices & services

– network-local discovery via Bonjour (mDNS) – missing: location-based discovery

• Advanced mobility: session, personal, service• Event notification• Data formats

– location, sensor events, ...


Examples of “invisible” behavior

• MP3 player in car automatically picks up new files in home server• A new email with vcard attachment automatically updates my cell

phone address book• The display of my laptop appears on the local projector

– without cable or configuration

• I can call people I just met at UCSB– without exchanging business cards

• My car key opens my front door• My cell phone serves as a TAN (one-time password) generator• My cell phone automatically turns itself off during a lecture• My camera knows where the picture was taken


An interconnected system

any weather serviceschool closings

opens doors

incoming call

generates TAN

acoustic alerts

updates location

time, location

alert, events

address book


Thinking beyond 802.11 and UMTS

• Many interesting networks beyond those covered in conferences– ease of access by researchers vs. importance

– 90% of papers on 802.11b and maybe GPRS, BlueTooth

• New wireless networks– broadcast instead of unicast -- useful for many ubiquitous applications

– S5 for low-rate sensors (city scale)

– Zigbee (802.15.4) for local sensors (20 - 250 kb/s)

– FM subcarrier (not really new) -- MSN Direct

– FM Radio Data System -- TMC

– Sirius / XM

– HD radio

– paging




Overview


• User challenges

• Beyond terminal mobility

• Location as new core service

• Universality: 7DS

• CS technology into reality


Application-layer mobility

• terminal mobility– one terminal, multiple network addresses

• Personal mobility– one person, multiple terminals– e.g., Grandcentral

• session mobility– one user, multiple terminals in sequence or in parallel

• service mobility– services move with user


Session mobility

• Walk into office, switch from cell phone to desk phone

– call transfer problem SIP REFER

• related problem: split session across end devices

– e.g., wall display + desk phone + PC for collaborative application

– assume devices (or stand-ins) are SIP-enabled

– third-party call control


How to find services?

• Two complementary developments:– smaller devices carried on user instead of stationary

devices– devices that can be time-shared

• large plasma displays• projector• hi-res cameras• echo-canceling speaker systems• wide-area network access

• Need to discover services in local environment– SLP (Service Location Protocol) allows querying for

services• “find all color displays with at least XGA resolution”• slp://example.com/SrvRqst?public?type=printer

– SLP in multicast mode– SLP in DA mode– Apple Bonjour

• Need to discover services before getting to environment

– “is there a camera in the meeting room?”– SLP extension: find remote DA via DNS SRV– LoST to find services by geographic location


Internet

CorrespondentNode (CN)

SIP UA

SLP UA

SIP SM

Local Devices

SLP SA SLP UA

SIP SM SIP UA

SLP DA

Mobile Node (MN)

SLPSIPRTP

SIP UA

Transcoder

Session mobility


Overview




Context-aware communication

• context = “the interrelated conditions in which something exists or occurs”

• anything known about the participants in the (potential) communication relationship

• both at caller and callee

time chronology, uniqueness

capabilities caller preferences

location location-based call routing

location events (emergency alerts)

security (access control)

service discovery

activity/availability presence

sensor data (mood, bio) privacy issues similar to location data


GEOPRIV and SIMPLE architectures

targetlocationserver

locationrecipient

rulemaker

presentity

caller

presenceagent

watcher

callee

GEOPRIV

SIPpresence

SIPcall

PUBLISHNOTIFY

SUBSCRIBE

INVITE

publicationinterface

notificationinterface

XCAP(rules)

INVITE

DHCP


Presence data architecture

rawpresencedocument

createview

(compose)

privacyfiltering

draft-ietf-simple-presence-data-model

compositionpolicy

privacypolicy

presence sources

XCAP XCAP

(not defined yet)

depends on watcherselect best sourceresolve contradictions

PUBLISH


Presence data architecture

candidatepresencedocument

watcherfilter

rawpresencedocument

post-processingcomposition(merging)

finalpresencedocument

differenceto previous notification

SUBSCRIBE

NOTIFY

remove data not of interest

watcher


Presence data model

“calendar” “cell” “manual”

[email protected], video, text

[email protected]

person(presentity)

(views)

services

devices


RPID = rich presence

• Provide watchers with better information about the what, where, how of presentities

• facilitate appropriate communications:– “wait until end of meeting”– “use text messaging instead of phone call”– “make quick call before flight takes off”

• designed to be derivable from calendar information– or provided by sensors in the environment

• allow filtering by “sphere” – the parts of our life– don’t show recreation details to colleagues


Rich presence

• More information: for (authorized) people and applications• automatically derived from

– sensors: physical presence, movement– electronic activity: calendars

• Rich information:– multiple contacts per presentity

• device (cell, PDA, phone, …)• service (“audio”)

– activities, current and planned– sphere (home vs. work)– current user mood– surroundings (noise, privacy, vehicle, …)– contact information– composing (typing, recording audio/video IM, …)


Presence and privacy

• All presence data, particularly location, is highly sensitive

• Basic location object (PIDF-LO) describes

– distribution (binary)– retention duration

• Policy rules for more detailed access control

– who can subscribe to my presence

– who can see what when

<tuple id="sg89ae">

<status>

<gp:geopriv>

<gp:location-info>

<gml:location>

<gml:Point gml:id="point1“

srsName="epsg:4326">

<gml:coordinates>37:46:30N 122:25:10W

</gml:coordinates>

</gml:Point>

</gml:location>

</gp:location-info>

<gp:usage-rules>

<gp:retransmission-allowed>no

</gp:retransmission-allowed>

<gp:retention-expiry>2003-06-23T04:57:29Z

</gp:retention-expiry>

</gp:usage-rules>

</gp:geopriv>

</status>

<timestamp>2003-06-22T20:57:29Z</timestamp>

</tuple>


Events as missing Internet capability

• aka PUB/SUB• Used across applications, e.g.,

– email and voicemail notification

– presence

– replace RSS (= poll!)

– web service completion

– emergency alerts (“reverse 9-1-1”)

– network management

– home control

– data synchronization

• Rich research history– but too complex, optimize the wrong thing

• XMPP and SIP as likely transport candidates


Local Switch

Automatic Number Identification

Automatic Location Identification

Collaboration between local phone providers and local public safety agencies


911 technology failures

• NY Times (“An S O S for 911 Systems in Age of High-Tech”), 4/6/07:– “40% of … counties, most of them rural or small-town …, cannot yet pinpoint the

location of the cellphone callers, though the technology to do so has been available for at least five years.”“In … Okmulgee, Okla., last November, 4-year-old Graciella Mathews-Tiger died in a house fire after a 911 operator who lacked the technology to pinpoint the call misheard the address.”

• Phase II wireless; billions of dollars spent

• In Mississippi, only 1 of out 5 counties

– “As it ages, it is cracking, with problems like system overload, understaffing, misrouted calls and bug-ridden databases leading to unanswered calls and dangerous errors.”

• operator (CAMA) trunks, with 8-digit number delivery

• MSAG and ALI databases


Location delivery

DHCP

HTTP

GPS

HELD

LLDP-MED

wiremap


Location, location, location, ...

Voice Service Provider (VSP)sees emergency call

but does not know caller location

ISP/IAP knows user locationbut does not handle call


Options for location delivery

• Wireless– GPS– S5 wireless (active sensors + triangulation)– Skyhook (802.11 in urban areas)– HDTV

• L2: LLDP-MED (standardized version of CDP + location data)– periodic per-port broadcast of configuration information

• L3: DHCP for– geospatial (RFC 3825)– civic (RFC 4676)

• L7: proposals for retrievals: HELD, SIP, …– for own IP address or by third party (e.g., ISP to infrastructure provider)– by IP address– by MAC address– by identifier (conveyed by DHCP or PPP)– HTTP-based


Locating Caller using LLDP-MED

CALLER EQUIPMENT

LLDP-MED SWITCH

LLDP-MED stands for: *Link Layer Discovery Protocol “a vendor-neutral Layer 2 protocol that allows a network device to advertise its identity and capabilities on the local network.”Media Endpoint Discovery “an enhancement to the LLDP that allows discovery of other things including location “

“I am LLDP-MED Capable.I can process location information.”

“Your location is:500 W 120TH st. New York NY 10027”

* From Wikipedia


Location determination options

Method CDP or LLDP-MED

DHCP HELD GPS manual entry

Layer L2 L3 L7 (HTTP) - user

advantages • simple to implement

• built into switch• direct port/room

mapping

• simple to implement

• network locality

• traverses NATs

• can be operated by L2 provider

• accurate• mobile

devices• no carrier

cooperation

• no infrastructure changes

• no carrier cooperation

problems may be hard to automate for large enterprises

mapping MAC address to location?

mapping IP address to switch port?

• indoor coverage

• acquisition time

• fails for mobile devices

• unreliable for nomadic

Use Ethernet LANs Enterprise LANs

Some ISPs

DSL, cable mobile devices fall back


INVITE urn:service:sos SIP/2.0

To: urn:service:sosCall-ID: [email protected]: SIP/2.0/TCP 192.168.1.106:4064;rportContent-Type: multipart/mixed; boundaryFrom: sip:[email protected]: <sip:[email protected]:5060>CSeq: 1 INVITEContent-Length: 1379

------ =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=MIME-Version: 1.0content-Type: application/sdpContent-Transfer-Encoding: 8bit

v=0o=eddie 1127764654 1127764654 IN IP4 192.168.1.106s=SIPC Callc=IN IP4 160.39.54.70t=0 0m=audio 10000 RTP/AVP 0 3m=video 20000 RTP 31

SDP

header fields

request line ------- =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=MIME-Version: 1.0Content-Type: application/pidf+xmlContent-Transfer-Encoding: 8bit

<?xml version="1.0" encoding="ISO-8859-1"?><presence xmlns="urn:ietf:params:xml:ns:pidf" xmlns:gp="urn:ietf:params:xml:ns:pidf:geopriv10" xmlns:cl=" urn:ietf:params:xml:ns:pidf:geopriv10:civilLoc" xmlns:gml="urn:opengis:specification:gml:schema-xsd:feature:v3.0" entity="sip:[email protected]"> <tuple id="28185"> <status> <gp:geopriv> <gp:location-info> <cl:civilAddress> <cl:country>us</cl:country> <cl:A1>ny</cl:A1> <cl:A2>new york</cl:A2> <cl:A3>new york</cl:A3> <cl:A6>amsterdam</cl:A6> <cl:HNO>1214</cl:HNO> </cl:civilAddress> </gp:location-info> <gp:method>Manual</gp:method> </gp:geopriv> </status> <contact priority="0.8">sip:[email protected]:5060</contact> <timestamp>2005-09-26T15:57:34-04:00</timestamp> </tuple></presence>------- =_ZGY1NTFlZDJkMDkxY2FkMTIxMWI2MzIzNjE1M2U0OTY=--

PIDF-LO

SIP message for Location Info.


Tracking


Data formats: location

• Civic (street)– jurisdictional & postal

• Geo (longitude & latitude)– point, polygon, circle, …

• see GeoRSS for simple example


ECRIT: LoST Functionality

• Civic as well as geospatial queries– civic address validation

• Recursive and iterative resolution• Fully distributed and hierarchical

deployment– can be split by any geographic or civic

boundary– same civic region can span multiple

LoST servers

<findService xmlns="urn:…:lost1"><location profile="basic-civic"> <civicAddress> <country>Germany</country> <A1>Bavaria</A1> <A3>Munich</A3> <A6>Neu Perlach</A6> <HNO>96</HNO> </civicAddress> </location> <service>urn:service:sos.police</service></findService>

• Indicates errors in civic location data debugging

– but provides best-effort resolution• Can be used for non-emergency services:

– directory and information services– pizza delivery services, towing companies, …


LoST: Location-to-URL Mapping

clusterserves VSP2

NYUS

NJUS

Bergen CountyNJ US

123 Broad AveLeoniaBergen CountyNJ US

cluster serving VSP1

replicateroot information

searchreferral

rootnodes

LeoniaNJ US

sip:[email protected]

VSP1

LoST


LoST Architecture

T1

(.us)

T2

(.de) T3

(.dk)

G

G

GG

G broadcast (gossip)T1: .us

T2: .de

resolver

seeker313 Westview

Leonia, NJ US

Leonia, NJ sip:[email protected]

tree guide


Left to do: event notification

• notify (small) group of users when something of interest happens– presence = change of communications state– email, voicemail alerts– environmental conditions– vehicle status– emergency alerts

• kludges– HTTP with pending response– inverse HTTP --> doesn’t work with NATs

• Lots of research (e.g., SIENA)

• IETF efforts starting– SIP-based– XMPP


Overview




Problems with Wide Area Wireless

• 802.11 currently hard to deploy across city or large area• Problem: How can mobile devices / gadgets get information

while on the move?• Use local peer-to-peer wireless networks to exchange

information– Peers can get information they do not have from another peer

Solution: 7DS!


How 7DS Works

1. When devices are in the same BSS (Basic service set) of 802.11 ad-hoc network, they discover each other using service discovery of Zeroconf

zeroconf


How 7DS Works

Internet

2. If there is no Internet connection, the devices can communicatewith each other to exchange information


Web Delivery Model

• 7DS core functionality: Emulation of web content access and e-mail delivery


Design

• Peer-to-peer network set up using zeroconf– Protocol enables devices to communicate with each other

without a DHCP server, a DNS server and a Directory server

• Proxy server serves content• Search engine searches for local data• MTA store and forward• In progress: File synchronization, BBS


Store and Forward

• Forwarding e-mail in the ad-hoc network• Acts as an MTA


Search Engine

• Provides ability to query self for results

• Searches the cache index using Swish-e library

• Presents results in any of three formats: HTML, XML and plain text

• Similar in concept to Google Desktop


Query Multicast Engine

• Used to actually exchange information among peers

• Requesting peer broadcasts a query to the network

• Responding peers reply if they have information

– Send encoded string with list of matching items

• Requesting peer retrieves suitable information


File Synchronization

SRV : 7ds-fs1.filesync._7ds._udp.local.

7ds-device1.local:2525

TXT : file1.xml

TXT : file2.xml

SRV : 7ds-fs2.filesync._7ds._udp.local.

7ds-device2.local:2525

TXT : word.doc

TXT : presentation.pptFile1.xmlFile2.xml

Word.docPresentation.ppt

“I wantWord.doc and presentation.ppt”

“I wantFile1.xml and file2.xml”

Word.docPresentation.ppt

File1.xmlFile2.xml

SERVICE ANNOUNCEMENTSERVICE RESOLUTIONFILE SYNCHRONIZATION USING RSYNC PROTOCOL


Overview




CS research to reality

CS as science

CS as engineering


CS tech transfer, mode 1


CS tech transfer, mode 2


The standards route


Role of standards

• Widespread use of technology requires standards– railroad gauges, nuts & bolts, Morse code, phone system

• industrial age = interchangeable parts

– CD ROM, DVD, HD-DVD/BlueRay– JPEG, MPEG, ...– SQL– C, Java

• Particularly for network technology– many mid-size actors– “network effect” = utility increases with number of users

• e.g., cars have inverse network effects

• chess game vs. email and word processor

– DECnet, SNA, ARCnet Ethernet, IP– “hypermedia” HTML, HTTP


Standards

• Tanenbaum: “The nice thing about standards is that you have so many to choose from.”

VHS + BetaEthernet + TokenringATM + IP

!

component technologies


Standards: technology translator

• Similar in some ways to text books• “accepted technology”

– lower/known risks (“vetted”)– infrastructure (“eco system”)

• requires expertise and broader training– many CS standards don’t have either

• example: HTTP/1.0, HTML 1.0

• thus, way to provide technology leadership


Examples of standards needed

• each with multi-B$ impact:– medical records– energy control– transportation

• inter-vehicle safety systems, traffic alerts, ...

– academic records– business transactions

• e.g., your travel expense report

• requires both CS and domain expertise


Conclusion

• Motivate mobile ubiquitous research by user problems• From stovepipe mobile & wireless systems to personal

and shareable wireless networks• Thinking beyond single applications

– presence event notification– 9-1-1 location time & location as infrastructure

• Need new models of creating services– domain-specific languages, not Java APIs

january 2008 ucsb 1 the vision and reality of ubiquitous computing prof. henning schulzrinne dept....

Documents

ubiquitous communicationsit

mobile equipment

dscs technology

right time

new core serviceuniversality

real user needs

special purpose phone

right media