liz marai 07/08/08 data management and visualization at pitt cs liz marai pitt computer science cmu...
TRANSCRIPT
Liz Marai 07/08/08
Data Management and Visualization at Pitt CS
Liz Marai Pitt Computer Science
CMU Robotics Institute (adj)
LSST, 08/07/2008
Liz Marai 07/08/08
Pitt Computer Science• Data Management
• Graphics and Visualization
• Artificial Intelligence
• Core Systems
• Theory and Algorithms
• Small: 12 active faculty (avg. tier I: 25)
• Strong: tier I (2 CAREER awards this summer only)
• Excellent record of interdisciplinary collaboration
Liz Marai 07/08/08
Current Collaborations• Pitt School of Medicine
– Orthopedics & Bioengineering
– Center for Modeling Pulmonary Immunity
– Center for Biomedical Informatics
• Center for Computational Biology and Bioinformatics
• Pitt Center for Modeling and Simulations (Chemistry)
• CMU Robotics Institute
• Center for Parallel, Distributed and Intelligent Systems
• Pittsburgh Supercomputing Center
Liz Marai 07/08/08
Visualization Research Lab
Department of Computer Science
University of Pittsburgh
Liz Marai 07/08/08
Image Processing, Modeling & Simulation of Biological Structures
w/ UPMC Orthopedics
Medical measurements (images, motion, forces etc)Half of parameters not measurable, yet inferrableUncertaintyMultiple sources of data
Predictive models and simulations
6Liz Marai 07/08/08
Exploratory Visualization and Analysis
Anomaly detection (exploration)
Quantitative measures, incorporated into the modeling process (analysis)
7Liz Marai 07/08/08
Annotations for Interdisciplinary Collaboration
Miscalibration? Or valid observation?
Liz Marai 07/08/08
Software Tools for Visual Mining
ADMT Lab / University of Pittsburgh July 8, 2008
Advanced Data Management Technologies Laboratory
July 2008
Department of Computer ScienceUniversity of Pittsburgh
ADMT Lab / University of Pittsburgh July 8, 2008
Graduate Students
• Lory Al Moakar
• Roxana Gheorgiou
• Shenoda Guirguis
• Qinglan Li
• Panickos Neophytou
• Jie Xu
Staff:
• Alex Connor
Advanced Data Management Technologies Lab Department of Computer Science, University of Pittsburgh
• Stream Data Management
• Web & Real-time Data Management
• Scientific Data Management
• Sensor Data Management
• Mobile Data Management
People Research
Panos Chrysanthis
AlexLabrinidis
User-centric Data Management for Scalable Network Centric Applications
http://db.cs.pitt.edu
ADMT Lab / University of Pittsburgh July 8, 2008
M1
Q1 Q2
1 1
M2
2 2
33
4 5
Oy
Oz
Ox
Ol
Operator Segment Ex
Q3Or
Shared Operators
Data Acquisition
Data Stream Processing
Web Data Management
Data Dissemination
ADMT Lab / University of Pittsburgh July 8, 2008
Data Stream Management Systems
• Alerting/Monitoring Service – Register query (filter) ahead of time– “Match” against incoming data stream– Generate “events” & notify users
• Examples: – Stock market monitoring– Transient alerts– Google alerts– Detection of outbreak of diseases
ADMT Lab / University of Pittsburgh July 8, 2008
Efficient Query Scheduling (Results)
Utilization
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Avg
. Res
pons
e T
ime
(S
ec)
0.0
5.0e+5
1.0e+6
1.5e+6
2.0e+6
2.5e+6 RRFCFSSRPTHR
Avg
. R
espo
nse
Tim
e (
Sec
)
65%
73%
ADMT Lab / University of Pittsburgh July 8, 2008
User-centric Web-data Management
• Given an option, would you prefer slightly-stale results fast OR fresh results, slightly delayed?
4:26 AM ET
ADMT Lab / University of Pittsburgh July 8, 2008
How to capture user preferences?
Proposed Quality Contracts Framework Micro-economic paradigm Combination of quality functions
Consider Quality of Service (response time)Consider Quality of Data (freshness)
Basic idea: Convert performance on individual metric into “worth” to users Use Quality Contracts to guide resource allocation
ADMT Lab / University of Pittsburgh July 8, 2008
Center for Modeling Pulmonary Immunity
+ =
ADMT Lab / University of Pittsburgh July 8, 2008
Scientific Data Management
• Biological Data Management
• Center for Modeling Pulmonary Immunity– NIH-funded (2005 - 2009)– 4 centers in the US– Build mathematical models of immune response– http://cmpi.cs.pitt.edu
• Data exchange server– Platform to record all experimental information– Enable sharing & interoperability across centers
ADMT Lab / University of Pittsburgh July 8, 2008
Data Exchange Server
• Platform to exchange experimental data– Goal 1: organize data already online– Goal 2: capture data from notebooks– Goal 3: allow for new data to be recorded
• Provenance (data lineage)• Annotations
– Goal 4: export / import capability– Goal 5: make repository user-friendly– Goal 6: minimize need for data cleaning – Goal 7: make repository active (alerts)
• Publish / Subscribe
ADMT Lab / University of Pittsburgh July 8, 2008
Advanced Data Management Technologies Lab Department of Computer Science, University of Pittsburgh
• Stream Data Management– Efficient Query Scheduling – AQSIOS project
• Web & Real-time Data Management– Quality Contracts – Admission Control – Transaction Scheduling
• Biological Data Management– Data Exchange Server– Annotation (Metadata) Management – Publish/subscribe
• Sensor Data Management• Mobile Data Management
Directors
Panos Chrysanthis
AlexLabrinidis
Research
User-centric Data Management for Network Centric Applications
http://db.cs.pitt.edu
ADMT Lab / University of Pittsburgh July 8, 2008
Acknowledgments• National Science Foundation
– CAREER: User-Centric Data Management (IIS-0746696)Jul 2008 - Jun 2013
– STREAMS: Algorithms and Metrics for New Generation Data Stream Management Systems (IIS-0534531)Mar 2006 - Feb 2009
– S-CITI: A Secure Critical Information Technology Infrastructure for Disaster Management (ANI-0325353)Oct 2003 - Sep 2008
• National Institutes of Health– CMPI: Center for Modeling Pulmonary Immunity
Sep 2005 - Sep 2010
Advanced Data Management Technologies Lab Department of Computer Science, University of Pittsburgh
Liz Marai 07/08/08