coast to coast march 2013
DESCRIPTION
General talk telecast to universities across Canada. A bit more of a focus on mixed-initiative systems than some earlier talksTRANSCRIPT
Visual Analytics as a Cognitive Science
Brian Fisher
SFU School of Interactive Arts & Technology and Program in Cognitive Science
UBC Media & Graphics Interdisciplinary Centre (MAGIC)
• In 2011, data expected to be about 1.8 zettabytes (1021).
• In 2013, Internet traffic to reach 667 exabytes (1018)/yr.
• Comparison: US Library of Congress is ~10 petabytes (1015).
© 2011 VIVA All rights reserved.
“Big Data: Volume, Velocity & Variety
• “By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.” (McKinsey Global Institute 2011)
Challenges for model-based approaches
• “3 Vs” challenge• Relevance, validity, reliability of data uncertain• Insufficient time to reach solution
• Model challenges• Multiple models to chose from• Assumptions may or may not hold
• “Wicked problems” challenge (Rittel)• Lack criteria to evaluate solution• Each problem is unique (no population)• Problem is not understood until solution is found
The Information needed to understand the problem depends upon one’s idea for solving it. -- Rittel & Webber 1973
Visualization history• NSF: Visualization in
Scientific Computing: McCormick, DeFanti, & Brown, Computer Graphics, Nov. 1987
• IEEE Visualization conference 1990
“The purpose of [scientific] computing is insight, not numbers.” Richard Hamming
“Visualization is a method of computing.” Authors of report
Information visualization • 1990 Conference
on diagrammatic reasoning
• 1995 InfoVis Conference• “Information
Visualization: Wings for the Mind” Keynote by Stuart Card
Stuart Card’s view
• Increase the memory & processing resources available to users
•Reduce the search for information by using visual representations to enhance the detection of patterns
•Engage perceptual inference operations•Use perceptual attention mechanisms for
monitoring•Support manipulation of information
Human/Computer cognitive system
•Mixed-initiative system composed of•Computer: Math, logic, machine learning etc. •Human: Perception, cognition, & collaboration
•Need an interactive visual interface that can effectively blend the two•Computer: steerable algorithms•Human: complementary actions
* Meyer J., Thomas, J., Diehl, S., Fisher, B., Keim, D., Laidlaw, D. Miksch S., Mueller, K. Ribarsky, W., Preim, B., & Ynnerman, A. (2010) From Visualization to Visually Enabled Reasoning. In “Scientific Visualization: Advanced Concepts”. vol. 1 pp. 227-245. Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, Germany. I978-3-939897-19-4
http://www.visual-literacy.org/periodic_table/periodic_table.html
On the Death of Visualization (Lorensen 2004)
Can It Survive Without Customers?
• Visualization, alone, is not a solution.
• Visualization is a critical part of many applications.
• Visualization, the Community, lacks application domain knowledge.
• Visualization has become a commodity.
• Visualization is not having an impact in applications.
Visual analytics origins• National Visualization & Analytics Center (NVAC)
• Battelle/PNNL 2004 R&D Agenda panel• University: Brown, GMU, Georgia Tech, OSU, Penn State, Purdue, SFU ,
Stanford, UC, UI, UM, UNC, UU, WPI
• Industry: Boeing, Microsoft, PARC, Sandia Labs
• Gov: CIA, DHS, FBI, NIST, NSA, unspecified
– Industry Consortium – Regional Visualization Centres – Centre of Excellence
•Ccicada (Rutgers DIMACS)•VACCINE (Purdue et al)
“This science must be built on integrated perceptual and cognitive theories that embrace the dynamic interaction between cognition, perception, and action. It must provide insight on fundamental cognitive concepts such as attention and memory. It must build basic knowledge about the psychological foundations of concepts such as ‘meaning,’ ‘flow,’ ‘confidence,’ and ‘abstraction.’ “
“Illuminating the Path” (IEEE Press)
“The science of analytical reasoning facilitated by interactive visual interfaces”
Visual analytics
How are VA Information systems different?
• Development based on understanding of expert cognition in situ• Informed by current cognitive & social science• Engagement with community of experts • Emergent cognitive science of expert reasoning
• Obvious support for analytical processes-- collaboration and interaction as well as observation• Graphical analog for analytic processes• Support “Human-information discourse”• Integrated across roles in the community
Visual analytics disciplines • Statistics, data representation and statistical graphics• Geospatial and Temporal Sciences• Applied Mathematics• Knowledge representation, management and
discovery• Ontology, semantics, Natural Language Processing, extraction,
synthesis, …
• Cognitive and Perceptual Sciences• Communication: Capture, Illustrate and present a
message• Decision sciences• Information and Scientific Visualization
Jim Thomas slide
Cognitive &PerceptualSciences
VisualInformationSystems
Graphic & Interaction Design
Mathematical & Statistical Methods
EffectiveSituated
R&D
http://www.vacommunity.org/HomePage
My claims
• VA becomes a science when its disciplines develop scientific methods that advance the goals of visually-enabled reasoning
• VA itself must build translational research methods, an application-focused “trade language”
Fisher, B., Green, T.M., Arias-Hernández, R. (2011) "Visual analytics as a translational cognitive science," Topics in Cognitive Science 3,3 609–625.
What kind of science?• Natural science approach to reasoning
• Experimental psychology, decision sciences
• Social science approach to coordination • Distributed cognition, social psychology
• Focus both on phenomena associated with rich interaction with visual information-- a “cyberpsychology”
• Add technology bits as needed
Fisher, B., Green, T.M., Arias-Hernández, R. (2011) "Visual analytics as a translational cognitive science," Topics in Cognitive Science 3,3 609–625.
"Il n'existe pas une catégorie de sciences auxquelles on puisse donner le nom de sciences appliquées. Il y a la science et les applications de la science, liées entre elles comme le fruit à l'arbre qui l'a porté"
Louis Pasteur
Pure Basic Research(Bohr)
Use-inspired Basic Research (Pasteur)
Sampling,Description,Taxonomy(Audubon)
Pure Applied Research (Edison)
Quest for Fundamental Understanding?
No
Yes
Consideration of Use ?
No
Yes
(1822–95)
Pasteur’s Quadrant
(Stokes)
The study of thought, learning, and mental organization, which draws on
aspects of psychology, linguistics, philosophy, and computer modelling.
(OED)
Cognitive Science Society founded in 1979
Cognitive Science
• Daniel Bobrow - AI• Eugene Charniak - AI• Allan Collins - Psychology• Edward Feigenbaum - AI• Charles Fillmore - Linguistics• Jerry Fodor - Philosophy• Walter Kintsch - Psychology• Donald Norman - Psychology• Zenon Pylyshyn - Psychology• Raj Reddy - AI• Eleanor Rosch - Psychology• Roger Schank - AI
Cognitive science origins
Key figures not at DallasAI Linguistics Neurosci Philosophy Psychology
Chomsky
Miller
Minsky
Newell
Simon
✓ ✓
✓ ✓ ✓ ✓
✓ ✓ ✓
✓ ✓
✓ ✓ ✓
Cogsci society logo
• A division of labour?•Simon’s“nearly decomposable problems”
• Conceptual & methodological “trading zone”?•Image and Logic: a Material Culture of Microphysics (Galison)
•Trading Zones in Cognitive Science (Thagard)
• A translational science of cognition?
What happens in the maze?
Translational cognitive science: Distributed cognition
• Expert perception and projection • Images may be static
• Active engagement with artifact• Objects are responsive, not autonomous• Complementary action (Kirsh)• Soft constraints (Gray)
• Agent-agent interaction• Socially-distributed cognition (Hutchins)• Mixed-initiative interaction
Claims• Since analytical reasoning is cognition, a
science of analytical reasoning ought to be a cognitive science
• Current cognitive science (psychology, visions research, social science) research methods and foci are useful but not sufficient for a visual analytics
• The multidisciplinary field approach taken by the Cognitive Science Society is a useful example for visual analytics
Core problem for VA science• Visual analytics is not a natural kind
• We design environments and their characteristics• We adapt to those environments
• While psychology can concentrate on typical behaviour, VA will be a moving target-- its capabilities may well change in response to the environments we create• e.g. recalibration by pairing-- statistical regularities
of the natural environment determine how sensory channels recalibrate each other
• Cognitive architecture from psychology• Extend to expert human performance
• Cognitive expertise• Visual expertise• Visuomotor expertise• Multimodality & modularity
• Test human capabilities in dynamic display environments
Distributed cognition in air traffic control
Air traffic control • NextGen ATC
“fishtank” projection • Change camera
position for better view
• How will global motion affect tracking?
Liu, G. Austen, E. L., Booth, K.S. Fisher, B., Argue, R. Rempel, M.I., & Enns, J. (2005) Multiple Object Tracking Is Based On Scene, Not Retinal, Coordinates. Journal of Experimental Psychology: Human Perception and Performance. 31(2), Apr 2005, 235-247.
http://www.youtube.com/watch?v=tKJVB4id_TY
with John Dill
FINST theory of spatial indexing
Multiple object tracking (Pylyshyn)
3-D Projected display
Test at different speeds
Fit human tracking function
... Then add display motion
Tracking vs object speed
Tracking in warped space
Tracking in warped space
Conclusion: Humans track in allocentric space
• Retinal speed of targets does not determine performance
• Motion of targets relative to each other does
• But only if motion preserves good metric characteristics of space
• Explanation is at the level of a human - display cognitive system
CognitionPerceptual
ScienceMethods
SocialScienceMethods
Computation and
Visualization Methods
Graphic & Interaction
DesignMethods
Visual analysis
• Base on cognitive architecture & D-Cog perspectives
• Focus on specific situations: •Cognition supported by interactive visualization•Socially distributed cognition via technology
• Incorporate these in •Design & evaluation of technologies•Selection, training, organizational change
Visual analytics methods
• “Pair analytics” sessions•Student visual analyst &
trained domain expert collaborate on analytic task
•Student “drives”, expert “navigates”
•Video session & capture screen
Socially distributed cognition
Arias-Hernandez, R, Kaastra, L.T., and Fisher, B. (2011) Joint Action Theory and Pair Analytics: In-vivo Studies of Cognition and Social Interaction in Collaborative Visual Analytics. In L. Carlson, C. Hoelscher, and T. Shipley (Eds.), Proceedings of the 33rd Annual Conference of the Cognitive Science Society (pp. 3244-3249). Austin TX: Cognitive Science Society.
SCIENCElab research• Emergency Management (NSERC, DHS)
• Mobile analytics / sensor analytics• “Virtual EOC” visual analytic environment
• Aircraft Safety, Reliability (Boeing/MITACS)• “Pair analytics” of complex quant and text data
• Economics and finance (MITACS, NSF)• Behavioural economics (portfolios)
• Healthcare Monitoring & Management (DHS)• Complex data in health research (CFRI)• Public health monitoring & management (BC Injury
Research and Prevention Unit)
SCIENCElab• Dr Richard Arias-
Hernández• Dr. Nathalie
Prevost• Dr. Linda Kaastra• Samar Al-Hajj• Nadya Calderón
• Tera Marie Green• Sabrina Hauser• Ali Khalili• Barry Po• Aaron Smith• Andrew Wade
Decision Analytics, Mobile Services and Service Science
Visualization and Analytics for Decision Support, Operational Management, and
Scientific Discovery Submissions due June 15, 2013http://www.hicss.hawaii.edu/hicss_47/