midas multi-device integrated dynamic activity spaces unmil p. karadkar ph.d. candidate dept. of...
TRANSCRIPT
![Page 1: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/1.jpg)
MIDAS Multi-device Integrated Dynamic
Activity SpacesUnmil P. Karadkar
Ph.D. CandidateDept. of Computer Science
Texas A&M University
![Page 2: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/2.jpg)
• Theory– Which aspects of information content do humans find valuable?
• Techniques– How to adapt content while optimizing for human perception? – How to enable heterogeneous device co-use?
• Frameworks– What are the attributes of multi-device systems? – How options are available to system designers for each attribute?
• System– Design a flexible architecture to embody multi-device system behaviors– Proof-of-concept implementation and analysis
• Practice– Harness device multiplicity to accomplish significant tasks
Research Questions
![Page 3: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/3.jpg)
• Motivation• Related Work• 12C Framework• Approach• MIDAS Architecture• Content Transformation• Demonstration• Analysis• Contributions• Future Work
Outline
![Page 4: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/4.jpg)
• Motivation• Related Work• 12C Framework• Approach• MIDAS Architecture• Content Transformation• Demonstration• Analysis• Contributions• Future Work
Outline
![Page 5: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/5.jpg)
14” 17” 40”- 60”21-24”3.5”
Custom
13”- 17”
75”
10”
4.5” 6” 7”
– Display characteristics– Network bandwidth
– Processing power– Interaction modalities
![Page 6: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/6.jpg)
Societal Changes
• User demographics• Nature of activity• Access location• Affordability– Free phones with contract– $150 e-readers
• Multiplicity – iPhone, iPad, MacBookPro
Heterogeneous contexts of information access
![Page 7: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/7.jpg)
MIDAS
Multi-deviceIntegratedDynamicActivitySpaces
MIDAS
![Page 8: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/8.jpg)
1. Deliver information elements to multiple devices simultaneously– Content– Interaction
2. Rely on widely available infrastructure– Wired and wireless computer networks– Mobile phones, e-readers, tablets, desktop computers– Sensors embedded in the environment
3. Support content transformation– Within form (image scaling, text summarization)– Across forms (text to speech, audio extraction from
video)
Design Goals
![Page 9: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/9.jpg)
4. Strive to retain content integrity from a human perspective– Select transformations that a user perceives to be
closest to the original form
5. Direct content elements to devices that can present them optimally
6. Reconfigure presentation dynamically in response to changes in device availability– New devices become available– Current devices become unavailable
Design Goals
![Page 10: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/10.jpg)
• Motivation• Related Work• 12C Framework• Approach• MIDAS Architecture• Content Transformation• Demonstration• Analysis• Contributions• Future Work
Outline
![Page 11: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/11.jpg)
Background
Individual devices
device-based content adaptation
Ubiquitous Computing
Infrastructure-enabled device co-use
• Service separation– Yahoo!, CNN, Weather Channel, UT Mobile
• Image distillation– Digestor (Fox 1996)
• Text and Web form summarization– Pythia (Bickmore 1997)– Power Browser (Buyukkokten 2002)– Proteus (Anderson 2001)
• Web page content extraction– Baseline SVPs + device-based PVPs (Chua 2007)
• Location-aware services– Computer for the 21st century (Weiser1991)– Java Ring (Olsen 2001)– IntelliBadgeTM (Cox 2003)
• Mobile devices in interactive spaces– Personal Server (Want 2002)– InfoStick (Kohtake 1999)– Elope (Pering 2005)
Opportunistic device co-use
![Page 12: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/12.jpg)
Background—Device co-use• Pebbles (Myers 1997)
– Desktop and PDA co-use– Extend Windows applications to PDAs– PDAs as input and control devices
• WebSplitter (Han 2000)
– Split Web content across a user’s devices– Support for multiple views
• ANMoLE (Haneef 2004)
– Dynamic content reformulation (splitting video)– Integration of phone and computer networks
![Page 13: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/13.jpg)
Background—Device co-use• Ubiquitous Display System (Aizawa 2002)
– Large public displays and mobile phones– Control from mobile phone
• Multibrowsing (Johanson 2001)
– Desktops, notebooks, PDAs, and large displays– Specially encoded links for content routing
• Interface Distribution Daemon (Luyten 2005)
– Web interface as well as stand-alone applications– Multiple content allocation modes– Usability analysis - Completeness and Continuity
![Page 14: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/14.jpg)
• Motivation• Related Work• 12C Framework• Approach• MIDAS Architecture• Content Transformation• Demonstration• Analysis• Contributions• Future Work
Outline
![Page 15: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/15.jpg)
• Concurrency– Simultaneous multi-device content presentation
• Control– Interaction support
• Comity– Permissible diversity in device characteristics
• Completeness– Presentation of relevant information elements
• Coverage– Distribution of content to optimize device usage
• Conversion– Content transformation to suit target device characteristics
12C Framework
![Page 16: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/16.jpg)
• Composition– Relationship of components to the interface
• Coherence– Consistency in presentation of content elements
• Coordination– Reallocation of elements when devices change
• Continuity– Support for users to interpret and evaluate system state
• Constancy– Reuse of content instances that have been shown before
• Confidence– Security and trustworthiness in communication
12C Framework
![Page 17: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/17.jpg)
• Motivation• Related Work• 12C Framework• Approach• MIDAS Architecture• Content Transformation• Demonstration• Analysis• Contributions• Future Work
Outline
![Page 18: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/18.jpg)
Development Platform
Web or something else?
![Page 19: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/19.jpg)
• WWW– Dominant infrastructure– World-wide audience
• Stateless server– Client-initiated communication– Content embedded within structure
• <p>, <h1>
– Changing devices means restarting the activity
• Standardized browsers– Little support for custom behavior
WorldWide Web
![Page 20: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/20.jpg)
context-aware Trellis (caT)• Petri net-based hypertext system (Furuta, Stotts - 1989)
– Formal, graph-based model– Browsing semantics (user privileges, time of day, location)
• Decouple specification from information content– Content not subsumed within structure– <img>, <a>– Separation allows devices to select suitable content
• Decouple content from presentation– Browsers present content autonomously
• Stateful server– Support for multiple clients– Easy propagation of user actions and content to multiple
devices
![Page 21: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/21.jpg)
• Motivation• Related Work• 12C Framework• Approach• MIDAS Architecture• Content Transformation• Demonstration• Analysis• Contributions• Future Work
Outline
![Page 22: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/22.jpg)
Device 3
Browser 4Browser 3
Browser 2
Device 2
Browser 4Browser 3
Browser 2Device 1
Browser 4Browser 3
Browser 2
MIDAS
Browser
ArchitectureSpecification
Information ServicecaT
Users
![Page 23: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/23.jpg)
Device 3
Browser 4Browser 3
Browser 2
ApplicationCoordinator
Device 2
Browser 4Browser 3
Browser 2
ApplicationCoordinator
Device 1
Browser 4Browser 3
Browser 2
MIDAS
Browser
Architecture
Device Manager
BrowserCoordinator
ResourceRealizer
Specification
Information Service
Users
![Page 24: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/24.jpg)
Device 3
Browser 4Browser 3
Browser 2
ApplicationCoordinator
Device 2
Browser 4Browser 3
Browser 2
ApplicationCoordinator
Device 1
Browser 4Browser 3
Browser 2
MIDAS
Architecture
instance propertiescontent handle
user actions
content handles
Specification
Information Service
Users
instance handle, actions
status,user actions
instance handle
content
Browser
user actions content
ResourceRealizer
ResourceRepository
BrowserCoordinator
Device Manager
content handles instance
properties
(instances, devices)
![Page 25: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/25.jpg)
Presentation onlyInput only Interactive
Device Manager
Instance properties
Device Selection Scheme
Device properties, interaction modeInstance Scorer
Content-Device Map
Content Handles, properties
Information Service
ResourceRealizer
BrowserCoordinator
Fully ReplicatedInteraction Replicated
Instance OptimalMIDAS Optimal
![Page 26: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/26.jpg)
• Motivation• Related Work• 12C Framework• Approach• MIDAS Architecture• Content Transformation• Demonstration• Analysis• Contributions• Future Work
Outline
![Page 27: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/27.jpg)
Content Transformation• Images– Scaling (automatic)– Color reduction (automatic)
• Text– Text extraction from formatted documents (automatic)
• HTML, Word, PDF– Summarization (semi-automatic)– Formatted data such as tables (semi-automatic)– Visual or audio rendering (automatic)
• Video– Audio extraction, down-sampling (automatic)– Video summarization (semi-automatic)
![Page 28: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/28.jpg)
Resource Manager• Authoring support
• Device dependence for content instances
• Automatic transformation of images (ImageMagick)
• Author validation
![Page 29: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/29.jpg)
Perception-based Image Transformation
• Images– Changes to attributes of photographs– Fastest growing information content on the Web
• Goal of user study– Understand human perception of nearness• Scaling of images• Color reduction (and gray scaling)
– Obtain actionable metrics to design rules for providing content in its most suitable form
![Page 30: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/30.jpg)
Study Design• Two stages– Stage 1: Image classification– Stage 2: Similarity of image pairs
• Subject demographics– Five subjects per stage– Graduate students and staff– Different disciplines
![Page 31: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/31.jpg)
Stage 1• Subjects classified 100 photographs– Taken by four photographers– Everyday and travel pictures– Prints of digital photos to make piles
• No guidelines regarding classification– Subjects created between 12 and 40 categories
• I coalesced categories based on image content– Four final categories
![Page 32: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/32.jpg)
Image Types
People
Structures
Nature
Text
![Page 33: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/33.jpg)
Stage 2• Subjects viewed image pairs on identical displays– No overlap or switching windows
• Each pair differed in size or color– Variables manipulated independently
• 20 image pairs for each variable
• Answered three questions– Similarity of images (9-point scale)– Suitability of automatic substitution (yes/no)– Acceptance of informed substitution (9-point scale)
![Page 34: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/34.jpg)
Scaling
![Page 35: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/35.jpg)
Color Reduction
![Page 36: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/36.jpg)
Data Analysis
160x120
320x240
640x480
800x600
1024x768
2 (1-bit)
4 (2-bit)
16 (4-bit)
256 (8-bit)
16 Million (24-bit)
Size Colors1
2
3
4
![Page 37: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/37.jpg)
Effect of Information Content Loss
Distance
Score Content Loss
0 6.89 67%
1 6.03 54 %
2 4.57 79 %
3 4.93 90 %
4 4.14 96 %
Distance Score Content Loss
1 7.12 56 %
2 7.11 80 %
3 6.87 93 %
4 6.14 98 %
Scores on a 9 pt. Scale
320 X 240 X 24 vs. 320 X 240 X 1
1024 X 768 X 24 vs. 160 X 120 X 24
(no loss) (98 %)
(no loss) (96 %)
Scaling Color depth change
![Page 38: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/38.jpg)
Similarity by image type
• Images of nature and structures scale well• Human faces• Legibility of textual elements• Algorithms for face and text detection in images exist
Distance nature structures people text
1 6.57 7.67 7.17 6.78
2 7.33 6.78 7.09 7.25
3 7.75 7.00 7.00 6.29
4 7.00 7.50 5.71 5.00
Scaling
![Page 39: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/39.jpg)
Distance Automatic substitution
Acceptability Informed Substitution
Acceptability
1 44% 7.66 56% 6.05
2 49% 7.42 51% 6.00
3 50% 7.13 50% 6.50
4 21% 7.67 79% 4.18
Scaling– Image Substitution
• No clear trends in terms of distance• Subjects allowed automatic substitution
– Corresponded with higher replacement suitability
![Page 40: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/40.jpg)
Distance Automatic substitution
Acceptability Informed Substitution
Acceptability
0 44% 7.50 56% 5.10
1 36% 7.43 64% 4.44
2 10% 8.00 90% 3.89
3 29% 8.75 71% 4.00
4 29% 8.00 71% 3.80
Color Reduction – Image Substitution
• Acceptance of automatic substitution is lower than that for scaled images
• The suitability ratings are correspondingly higher
![Page 41: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/41.jpg)
Perception-informed Rules • Principles
– Prefer scaling over color reduction– Avoid scaling of images that contain people and text– Gray scaling is the best option when used by itself
• Normalized score: Original image = 0 (best score)
+ Wcolor
(9 - Scoredist, color )
9x
Image Rating = Wsize
(9 - Scoredist, size)9
x Wtypex0.4
0.6
Wtogray = 1.0 (grayscale)Wtogray = 0.0 (otherwise)
Wtype
+ Wtogray
(9 - Scoredist, color )
9x
Wtype (people, text) = 0.53Wtype (nature, structures) = 0.47
![Page 42: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/42.jpg)
• Motivation• Related Work• 12C Framework• Approach• MIDAS Architecture• Content Transformation• Demonstration• Analysis• Contributions• Future Work
Outline
![Page 43: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/43.jpg)
• Motivation• Related Work• 12C Framework• Approach• MIDAS Architecture• Content Transformation• Demonstration• Analysis• Contributions• Future Work
Outline
![Page 44: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/44.jpg)
1. Deliver information elements to multiple devices simultaneously– Content– Interaction
2. Rely on widely available infrastructure– Wired and wireless computer networks– Mobile phones, e-readers, tablets, desktop computers– Sensors embedded in the environment
3. Support content transformation– Within form (image scaling, text summarization)– Across forms (text to speech, audio extraction from
video)
Analysis - Design Goals
✔
✔
✔
![Page 45: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/45.jpg)
4. Strive to retain content integrity from a human perspective– Select transformations that a user perceives to be
closest to the original form
5. Direct content elements to devices that can present them optimally
6. Reconfigure presentation dynamically in response to changes in device availability– New devices become available– Current devices become unavailable
Analysis - Design Goals
✔
✔
✔
![Page 46: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/46.jpg)
Analysis - 12C FrameworkConcurrency ComityControl
ConversionCoverageCompleteness
Coordination
CoherenceComposition
ConfidenceConstancy Continuity
Present information elements on multiple devices
• Inclusion of devices in MIDAS is optional• WebSplitter, ANMoLE
• A user can choose the interaction mode for each device
• Inclusion of public devices is not supported• Ubiquitous Display System
• Content is directed to devices automatically• Multibrowsing, Ubiquitous Display System
![Page 47: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/47.jpg)
12C FrameworkConcurrency ComityControl
ConversionCoverageCompleteness
Coordination
CoherenceComposition
ConfidenceConstancy Continuity
Interaction support
• Flexible – dependent on device choices• Most architectures support centralized or distributed
• It is possible to have no input devices at all
• May change during a session
![Page 48: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/48.jpg)
12C FrameworkConcurrency ComityControl
ConversionCoverageCompleteness
Coordination
CoherenceComposition
ConfidenceConstancy Continuity
Permissible diversity in device characteristics
• MIDAS places few restrictions on devices• Pebbles, Ubiquitous display system
• Device must run the Browser Coordinator
• No requirement of proximity to a user• Remote devices may be included - printers
![Page 49: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/49.jpg)
12C FrameworkConcurrency ComityControl
ConversionCoverageCompleteness
Coordination
CoherenceComposition
ConfidenceConstancy Continuity
Presentation of relevant information elements
• Ensures completeness by presenting all resource ids
• Resource Manager aids content authors• Support includes textual description as fallback
• IDD assertion is based on fixed content instances• ANMoLE and MIDAS convert content
![Page 50: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/50.jpg)
12C FrameworkConcurrency ComityControl
ConversionCoverageCompleteness
Coordination
CoherenceComposition
ConfidenceConstancy Continuity
Distribution of content to optimize device usage
• Flexible – dependent on Device Manager mode• Fully replicated - redundancy• Interaction replicated • Instance optimal - coverage for individual instances• MIDAS optimal – system-wide coverage
• Coverage may not apply to some systems• Ubiquitous Display System, Multibrowsing
![Page 51: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/51.jpg)
12C FrameworkConcurrency ComityControl
ConversionCoverageCompleteness
Coordination
CoherenceComposition
ConfidenceConstancy Continuity
Content transformation to suit target device characteristics
• Resource Manager• Authoring-time syntactic conversion with user validation
• Device• Presentation-time implicit conversion
![Page 52: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/52.jpg)
12C FrameworkConcurrency ComityControl
ConversionCoverageCompleteness
Coordination
CoherenceComposition
ConfidenceConstancy Continuity
Relationship of components to the interface
• Generative approach• Instances and actions together form the interface
• Many Web-based systems take a degenerative approach• Content filtering
![Page 53: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/53.jpg)
12C FrameworkConcurrency ComityControl
ConversionCoverageCompleteness
Coordination
CoherenceComposition
ConfidenceConstancy Continuity
Consistency in the presentation of content elements
• Content presented on all devices reflects the current state
• Device Manager mode dependent• Replication modes may present inconsistent information if the content
is not vetted by the author
![Page 54: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/54.jpg)
12C FrameworkConcurrency ComityControl
ConversionCoverageCompleteness
Coordination
CoherenceComposition
ConfidenceConstancy Continuity
Reallocation of elements when devices change
• In response to • Device changes• User actions• Environmental properties changes (inherited from caT)
• Potential for optimization• Not all events threaten the integrity of a presentation
![Page 55: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/55.jpg)
12C FrameworkConcurrency ComityControl
ConversionCoverageCompleteness
Coordination
CoherenceComposition
ConfidenceConstancy Continuity
Reuse of instances that have been shown before• Not supported (yet)
• MIDAS always picks the most optimal instance
• Systems that use fixed instances support constancy better• Web-based presentations
![Page 56: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/56.jpg)
12C FrameworkConcurrency ComityControl
ConversionCoverageCompleteness
Coordination
CoherenceComposition
ConfidenceConstancy Continuity
Support for users to interpret and evaluate system state
• Supported with user-initiated state changes• Device changes• User actions
• Presentation changes due to environmental properties• Causes may be subtle – off-peak hours in another time zone
![Page 57: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/57.jpg)
12C FrameworkConcurrency ComityControl
ConversionCoverageCompleteness
Coordination
CoherenceComposition
ConfidenceConstancy Continuity
Security and trustworthiness in communication
• Reliance on the underlying communication infrastructure• Connection-oriented sockets
• Security or encryption is not supported
![Page 58: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/58.jpg)
• Motivation• Related Work• 12C Framework• Approach• MIDAS Architecture• Content Transformation• Demonstration• Analysis• Contributions• Future Work
Outline
![Page 59: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/59.jpg)
Contributions• Theory– 12C Framework for characterizing multi-device systems– Human perception of nearness in image adaptation
• Nature of content loss is important• Not all images scale equally gracefully• Gray scaled images are perceived to be the closest
• Techniques– Perception-informed metrics for image adaptation
• System– MIDAS architecture and prototype
• Flexible infrastructure • Minimal requirements for including diverse devices• Expression of various server- and client-side behaviors• Coordination in response to users, environment, and devices
![Page 60: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/60.jpg)
• Motivation• Related Work• 12C Framework• Approach• MIDAS Architecture• Content Transformation• Demonstration• Analysis• Contributions• Future Work
Outline
![Page 61: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/61.jpg)
Future Work• Infrastructure
– WebSockets API (HTML5), CSS3 (device-specific templates)– Clients for Android, iOS, and Windows– Inclusion of large-screen displays — GoogleTV(?)
• Interaction and Usability– Realistic setting, contemporary devices– Attention, cognition, privacy– Controlled studies, data from long-term use– Exploration of collaborative use
• Content transformation– Other image adaptations– Suitability of lessons learned from presentation for disabled
populations– Cross-form adaptation– Normalization of scores across different content forms
![Page 62: MIDAS Multi-device Integrated Dynamic Activity Spaces Unmil P. Karadkar Ph.D. Candidate Dept. of Computer Science Texas A&M University](https://reader030.vdocuments.site/reader030/viewer/2022032706/56649e015503460f94aeb622/html5/thumbnails/62.jpg)
The most profound technologies are those that disappear.
- Weiser 1991.