research issues in adapting computing to small devices

Research Issues in Adapting Research Issues in Adapting Computing to Small DevicesComputing to Small Devices

Francis C.M. Lau

Department of Computer Science

The University of Hong Kong

APPT 2005 • Hong Kong

fcmlau@cs.hku.hk

APPT 05 (27/10/05, HK) 2

Popular ComputingPopular Computing

• 70’s: one computer, many users• 80’s: one computer, one user• 90’s: many (connected) computers,

many users – distributed computing• Now: many computers, one user

• What next? Ubiquitous computing? Grid computing? Service-oriented computing? …– Do we need that many computers?– Do we need a “computer”?

APPT 05 (27/10/05, HK) 3

Windows PanicWindows Panic

• Continue or quit or kill the window or details?

• How would my mother know!?

• The best computing is without the computer

APPT 05 (27/10/05, HK) 4

Imagine …Imagine …

• APPT 2015 (Disneyland, Hong Kong) …

APPT 2015APPT 2015

APPT 05 (27/10/05, HK) 5

My Future HandheldMy Future Handheld

• All in one: phone, PDA, wallet, game console, remote terminal, ad-hoc networking …

• That means a lot of software – big and fat!?

• But what I really want is lean and thin and mean– Lean – simple software which does only what I want

– Thin – that’s all the space in my pocket, or around my neck!

– Mean – affordable

LTMLTM

APPT 05 (27/10/05, HK) 6

LTM ComputingLTM Computing

• The (thin-) client-server paradigm, works but in limited ways

• Too thin: a browser-only device, a remote terminal …

• Future devices are more powerful than just that– But how powerful is

powerful?

APPT 05 (27/10/05, HK) 7Subramanian, 1999

O my god!

Software complexity

APPT 05 (27/10/05, HK) 8

Rich ComputingRich Computing

• My hard disk and VM can be as big as the Internet

• Not just contents, but software come and go on demand

• Pay-per-use subscription model• Browser plug-in’s and Java applets –

not lean enough, and some require just too much resources

• Software in terms of “functionalities”

The Networkis the Computer ™

APPT 05 (27/10/05, HK) 9

The Network Behind Every DeviceThe Network Behind Every Device

• Future digicams– Snap and go

• The network is the memory• The network is the photo album• (The network is NOT the power)

– Context aware

APPT 05 (27/10/05, HK) 10

Edge ComputingEdge Computing

• Person .. device .. middleware (proxies) .. Internet

• The abstract cloud moves with the client – personalized “cuddleware”, nomadic computing

Internetproxies united

client

metropolis

APPT 05 (27/10/05, HK) 11

DEVICEEDGE

GRID

The continuum: WHAT SHOULD BE DONE The continuum: WHAT SHOULD BE DONE WHERE?WHERE?

APPT 05 (27/10/05, HK) 12

A ContinuumA Continuum

• Your device .. other devices (peers) in the vicinity .. server(s) nearby .. the grid– “Neighborhood” is still important

• network bandwidth can’t be infinite; latency will never be 0; network will never be trouble-free

• What should be done where?– The end-device: completely passive (HAS) to c

ompletely active (DIY)

APPT 05 (27/10/05, HK) 13

““Hire a Server”Hire a Server”

• Completely passive (remote terminal)– A waste of bandwidth– Responsiveness and interactivity (esp. for

games)• Does it make sense?

– MP3, camera, PDA, …

• Something must be done locally– Adaptation

APPT 05 (27/10/05, HK) 14

Limits You Can’t BeatLimits You Can’t Beat

• Power?• Screen size: foldable, rollable, we

arable, projectable? Or do without the screen (audible?)– Convenience and privacy; and po

wer consumption– Resolution helps a little bit

• Why I don’t use a PDA (I just ordered an 24” LCD; but I still look at my watch)– Ergonomics is very important– Scrolling is a nuisance (now)– User experience!

APPT 05 (27/10/05, HK) 15

Wearable Wearable ComputersComputers

server

display

APPT 05 (27/10/05, HK) 16

AdaptationAdaptation

• Data– = not code

• Content– A special kind of data– Text, paged, non-text– Pixel level and/or higher

• UI– Higher level, structural, interactive– Buttons, icons, menus, panels and panes … content

• Code

APPT 05 (27/10/05, HK) 17

Componentized SoftwareComponentized Software

• Executable code as puzzle pieces

• Fast start, like contents• Software download

pipelining – first-come-first-execute, the most needed functionalities first

APPT 05 (27/10/05, HK) 18

The Need for Code On DemandThe Need for Code On Demand

• How many functions have you ever used?

A simplified view of Microsoft Word

APPT 05 (27/10/05, HK) 19

Remember UNIX?Remember UNIX?

• Small is beautiful• Make each program do one

thing well• Choose portability over

efficiency• Store data in flat ASCII files• Avoid captive user interfaces• Make every program a filter• …

APPT 05 (27/10/05, HK) 20

UNIX Philosophy AppliedUNIX Philosophy Applied

• Small software components implementing single functionalities

• Bigger functions runtime-composable from small components

• Download on demand, and disposable

• Standard formats and interfaces (eg. XML)

• Shared GUI

APPT 05 (27/10/05, HK) 21

Service-Oriented ComputingService-Oriented Computing

Can Web Services Scale Down?

- for small device apps

APPT 05 (27/10/05, HK) 22

Adapting X to YAdapting X to Y

• Many issues– Psychology– Aesthetics– Readability, convenience, privacy …– Economics

• Context awareness– Because small devices are likely mobile– The user moves around

APPT 05 (27/10/05, HK) 23

Awareness and AdaptationAwareness and Adaptation

• An intelligent supporting architecture

• Aware of the come and go of mobile users

• Choices of functions, depending on situation: the location, the device, the user, …

• Fast reacting nearby proxy• Proxies united, within certain

locality

APPT 05 (27/10/05, HK) 24

Summary 1: Summary 1: BasicBasic Problems (I Problems (I think) Worth Pursuingthink) Worth Pursuing

• Edge computing → “seamless”– New protocols for the edge

• The continuum → the network is the computer– Collaborative models and mechanisms, esp. at the edge

• The global grid → invisible, “PC” disappearing• The device

– Adaptation• On-demand code composition

– The SOC approach?

• Content– HTML

• UI description languages

– New paradigms for user interaction in small devices (input and output)

APPT 05 (27/10/05, HK) 25

Some HKU ProjectsSome HKU Projects

• The Sparkle Project – “code adaptation”• PDF document content adaptation system

– “content adaptation”• HTML adaptation – “UI/content

adaptation”

APPT 05 (27/10/05, HK) 26

SparkleSparkle

• Mobile code on demand– Proxy to device, device to device

• Features– Code On Demand– Context Based Adaptation– Proxy Caching on Code– Code & State Migration– Ontology– Sparkle View & Universal Browser

Application = “facet” composition

APPT 05 (27/10/05, HK) 27

Universal BrowserUniversal Browser

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

Time (ms)

Add Noise AddShadow

FindEdges

FlipHorizontal

FlipVertical

GaussianBlur

OpenImage

Save asJpeg

Sharpen

APPT 05 (27/10/05, HK) 28

PDF Document AdaptationPDF Document Adaptation

APPT 05 (27/10/05, HK) 29

APPT 05 (27/10/05, HK) 30

APPT 05 (27/10/05, HK) 31

APPT 05 (27/10/05, HK) 32

Web Document AdaptationWeb Document Adaptation

APPT 05 (27/10/05, HK) 33

(a) The original content

(b) Only the bottom level is “compressed”

(c) The two lowest levels are “compressed”

APPT 05 (27/10/05, HK) 34

Handling SegmentsHandling Segments

APPT 05 (27/10/05, HK) 35

Summary 2: When I Grow OlderSummary 2: When I Grow Older

• I want a simpler life (others can enjoy all the complexities)– One device, LTM– No computer please; terminal is fine; plug and

play– Single data storage by hire, access anywhere– Single sign-on, single environment (adaptive)

APPT 05 (27/10/05, HK) 36