Accommodating Global Policies andUser Preferences in Computer Supported Collaborative Systems
Student:
Helmuth Trefftz
Advisors:
Prof. Rick Mammone
Prof. Ivan Marsic
Rutgers University
April, 2001
Agenda
• Motivation• Related Work• Thesis Statement• Experimental Setup• Mathematical Model• Preliminary Results• Future Work• Conclusions
Motivation (1)
• We are multi-modal
Motivation (1)
• We are multi-modal• We acquire information
through multiple modalities (sensory channels):– Sight– Sound– Smell– ..
Motivation (1)
• We are multi-modal• We acquire information through
multiple modalities (sensory channels): :– Sight– Sound– Smell– ..
• Most intense experiences involve multiple modalities (love)
Motivation (2)
• Computer Systems become more and more multimodal:
Motivation (2)
• Computer Systems become more and more multimodal:– e-mail, chat
Motivation (2)
• Computer Systems become more and more multimodal:– e-mail, chat– chat with cartoons (Microsoft)
Motivation (2)
• Computer Systems become more and more multimodal:– e-mail, chat– chat with cartoons (Microsoft)– CuSeeMe
Motivation (2)
• Computer Systems become more and more multimodal:– e-mail, chat– chat with cartoons (Microsoft)– CuSeeMe– NetMeeting, Face Mail– …
• Integration of text, voice, video
Motivation (2)
• Face Mail:• Input
– Text + “:- ):
• Output– Synthesized voice– Animated character
• Multiple Info. Rep.!
Motivation (3)
• Multiple levels of information representation
Motivation (3)
• Multiple levels of information representation– Some years ago: browsers showed
a low-resolution image first, then refined
Motivation (3)
• Multiple levels of information representation– Some years ago: browsers showed
a low-resolution image first, then refined
– In Virtual Environments: distant objects are represented with lower Level Of Detail
Motivation (3)
• Multiple levels of information representation– Some years ago: browsers showed
a low-resolution image first, then refined
– In Virtual Environments: distant objects are represented with lower Level Of Detail
– “FaceMail”: you type text, output: voice + avatar
Motivation (4)
• Multiple dimensions of information representation
Motivation (4)
• Multiple dimensions of information representation– Text: easier to store, easier to
search
Motivation (4)
• Multiple dimensions of information representation– Text: easier to store, easier to
search– Sound: hands-free, ‘warmer’
interaction
Motivation (4)
• Multiple dimensions of information representation– Text: easier to store, easier to
search– Sound: hands-free, ‘warmer’
interaction– Text -> Sound: MS Whisper– Sound -> Text: IBM Via-voice
Motivation (6)
• Multiple dimensions of information representation (VSI):
• Company I used to work for.• Product: multi-modal
representation of information for distance learning.
• Channels: video + sound + text + animations.
Motivation (6)
• Multiple levels of information representation (VSI)– Video + Sound (engaging)
Motivation (6)
• Multiple levels of information representation (VSI)– Video + Sound (engaging)– Graphics + Animations (concepts)
Motivation (6)
• Multiple levels of information representation (VSI)– Video + Sound (engaging)– Graphics + Animations (concepts)– Text (search)
Motivation (6)
• After months of separate work in video, text, graphics
Motivation (6)
• After months of separate work in video, text, graphics
• Integrated product was… NOT SMOOTH!
Motivation (6)
• After months of separate work in video, text, graphics
• Integrated product was… NOT SMOOTH!
• Elaborate animations made video choke!
Motivation (6)
• Integration: find the appropriate space in a multi-dimensional space:
Video + Audio
Graphics
Motivation (7)
• Multiple values in each dimension:
Motivation (7)
• Multiple values in each dimension:– Video: resolution, frames per
second
Motivation (7)
• Multiple values in each dimension:– Video: resolution, frames per
second– Sound: sample resolution, sample
rate
Motivation (7)
• Multiple values in each dimension:– Video: resolution, frames per
second– Sound: sample resolution, sample
rate– In Networked Virtual
Environments: update rate
Motivation (7)
• Multiple values in each dimension:– Video: resolution, frames per
second– Sound: sample resolution, sample
rate– In Networked Virtual
Environments: update rate– Text: ?
Motivation (8)
• Higher values in each dimension:
Motivation (8)
• Higher values in each dimension:– More space to store
Motivation (8)
• Higher values in each dimension:– More space to store– More CPU cycles to process
Motivation (8)
• Higher values in each dimension:– More space to store– More CPU cycles to process– More bandwidth to transmit
Motivation (9)
• Computers in a Collaborative System
Adapted from: http://www.intel.com/intel/museum/25anniv/hof/moore.htm
generation
performance
i i+1i - 1
Motivation (9)
• Computers in a Collaborative System:
• Disparities among consecutive generations will grow larger in:– Processor speed– Memory– Bandwidth
Motivation (9)
• How to cope with these disparities?
Slower computers Faster computers
Motivation - Summary
• Multiple modalities - multiple dimensions
Motivation - Summary
• Multiple modalities - multiple dimensions
• Multiple qualities of information representation - multiple points
Motivation - Summary
• Multiple modalities - multiple dimensions
• Multiple qualities of information representation - multiple points
• Performance - limits the “valid” hyper space
Related Work (1)
• CVEs in Distance Education in Colombia
Related Work (1)
• CVEs in Distance Education in Colombia– Collaborative Virtual Environment
as learning experience
Related Work (1)
• CVEs in Distance Education in Colombia– Collaborative Virtual Environment
as learning experience– AVALON: (Carlos Correa)
• Multiple modalities:– Avatars, VRML worlds, OpenGL– Voice: multicast version of “Speak
freely” (Francisco Cardona, now at Swiss Federal Institute of Technology (EPFL) )
Related Work (1)
• CVEs in Distance Education in Colombia– Used in a real learning
environment (Environmental Issues class at Eafit University).
– Improvement in learning measured with the “Teaching for Understanding” model (Harvard)
Related Work (2)
• Michael Macedonia - Ph.D. thesis (1995)– Partition the space in hexagonal
spaces– Each hexagonal space is a
multicast group– Implemented in DIS (Distributed
Interactive Simulation) and SIMNET (Simulator Networking)
Read Thesis Statement
Related Work (2)
• Thesis Statement:“Virtual environment software
architectures can exploit wide area multicasting communications and
entity relationships to partition the virtual world and enable the
development of scalable distributed interactive simulations for military
projects”.(Michael Macedonia, 1995)
Related Work (3)
• Michael Capps - Ph.D. thesis (2000)– Have multiple representations for
each object– Define which representation to
use based on:• Quality• Importance• Cost
Our Thesis Statement
“It is possible to express global policies and individual users’ preferences in
collaborative multimodal systems as a formal mathematical model.
Solving this mathematical model, which includes performance measurements taken at each
participating node, guarantees enforcement of the policies while
allowing individual users to adjust their interaction with the system.”
Thesis Statement
Assumptions
• Collaborative System• Diverse degrees of computing
power• Nodes have limited computing
power• Shared information: varying
levels of fidelity in time/space dimensions
Experimental Setup (1)
• Distributed visualization system• Users share a Visualization Data
Set• “Video” of the other
participants • Telepointers• One user can manipulate the
object at a time
Experimental Setup (2)
• User Interface
Experimental Setup (3)
• Information Dimensions (variables):
Experimental Setup (3)
• Information Dimensions (variables):– Video
Experimental Setup (3)
• Information Dimensions (variables):– Video– Object Movements
Experimental Setup (3)
• Information Dimensions (variables):– Video– Object Movements– Telepointer Movements
Experimental Setup (3)
• Information Dimensions (variables):– Video– Object Movements– Telepointer Movements– Graphic complexity used locally
for the visualized data set
Experimental Setup (3)
• Information Dimensions (variables):– Video– Object Movements– Telepointer Movements– Graphic complexity used locally
for the visualized data set– Rendered object frames per
second
Experimental Setup (4)
• Independent variables:– Video– Object Movements– Telepointer Movements– Graphic complexity used locally
for the visualized data set
• Dependent variable:– Rendered object frames per
second
Experimental Setup (5)
• Mapping from independent to dependent variable:
Experimental Setup (5)
• Mapping from independent to dependent variable:
• “Performance Mapping”
Experimental Setup (5)
• Mapping from independent to dependent variable:
• “Performance Mapping”
• Individual for each machine• Determined before the
collaborative session running a simulated session
Experimental Setup (6)
• Architecture:
Server
Client 1 Client 2 Client n
Experimental Setup (6)
• Switchboard analogy
Mathematical Model (1)
• Measurable way to express:– User preferences– Minimum performance levels – Maximum level of service that can
be offered• Server processing cycles• Available bandwidth
Mathematical Model (1)
• User preferences: (i)
• Objective Function.
})30/50(.)20/25(.)20/0()3/25(.)20/0{( FMTGVMax
Mathematical Model (1)
• Minimum performance levels: (ii)– V >= 1– G >= 1– T >= 5– M >= 1– F >= 2
Mathematical Model (1)
• Maximum level of service: (iii)– Sum (Vi) <= 100
– Sum (Ti) <= 150
– Sum (Mi) <= 120
– Gi and Fi have only local impact
Mathematical Model (1)
• Can be solved locally (at each machine):– Individual Objective Function +
type (ii) restrictions
• Require a global solution:– Type (iii) restrictions + Global
Objective Function
• Possibility to solve the first part in parallel.
Mathematical Model (2)
• Combining the three types of equations
• A “LINEAR PROGRAMMING” OPTIMIZATION PROBLEM
Mathematical Model (2)
• Combining the three types of equations
• A “LINEAR PROGRAMMING PROBLEM”
• Each local solution:– Valid search space enforcing
global policies– Optimize local Objective Function
allowing each user to adjust her interaction
Mathematical Model (2)
• Global Solution:– Determine (and enforce) top
limit on service handled by the server and the network
Mathematical Model (3)
• Challenges:– In classic Linear Programming
variables take continuous values.– In our problem: discrete values.– Parallel processing of partial
solutions– Once a solution is found, if a
condition changes, the next solution should be found without restarting the computation.
Mathematical Model (3)
• Possible solutions:• Use integer programming• Use piece-wise linear
programming• Non-linear optimization
techniques
Preliminary results (1)
• Server:– Accepts TCP connections for meta-
information – A predefined multicast group for
each “socket” in the switchboard
TCP
UDP multicastTimer
UDP unicast
Preliminary results (2)
• Client:– Establishes a TCP connection for
subscription messages– Receives simulated updates from
the server
TCP
UDP (mcast)
TimerSS
UDP (unicast)
Preliminary results(3)
• Computers:– Dragonfire: Pentium II @ 350 MHz,
64MB main memory– Morlak: Pentium II @ 400 MHz,
256 MB main memory
Preliminary results(3)
• 3D Models:– Guts: 27,202 vertices– Skull: 1,210 vertices
Preliminary results(4)
• “Idle” performance:
SkullGuts
Dragonfire
Morlak0
20
40
60
80
100
No remote updates
Preliminary results(5)
• Effect of video-events frequency:
No
upda
tes
1000
ms
500
ms
200
ms
100
ms
Morlak GutsMorlak Skull0
1020304050607080
Video Updates
Preliminary results(5)
• Effect of object-movement-events frequency:
No
upda
tes
1000
ms
500
ms
200
ms
100
ms
Morlak GutsMorlak Skull0
102030405060708090
Object Updates
Preliminary results(5)
• Note that video events have more impact on the local performance than object movement events.
Preliminary results(6)
• Resulting performance mapping (Skull):
1
2
3
4
5
1
2
3
4
5
40
50
60
70
80
90
Object Updates
Object x Video Morlak Skull
Video Updates
Avera
ge F
PS
Preliminary results(6)
• Resulting performance mapping (Guts):
1
2
3
4
5
1
2
3
4
5
4
4.5
5
5.5
6
6.5
7
Object
Object x Video Morlak Guts
Video
Avera
ge F
PS
Future Work (1)
• Server: – Receive and cache (not queue)
messages.
Future Work (1)
• Server: – Receive and cache (not queue)
messages.
• Client:– Implement frame grabbing– Send updates
Future Work (2)
• Math:– Local solution for type (i) and type
(ii) equations– Global solution involving type (iii)
equations– Choose solver:
• Initially: “brute force” (search through the finite search space)
• Linear programming?• Non-linear programming?
Conclusions (1)
• Mathematical model provides objective solution for two types of conflicting situations:
• Diverse Computing power of the nodes
• Global policies and users’ preferences
Conclusions (2)
• Assigning different multicast groups to the different values the variables can take make the solution scalable.
Thank you!