visualization and analysis of text
DESCRIPTION
Visualization and Analysis of Text. Remco Chang, PhD Assistant Professor Department of Computer Science Tufts University December 17, 2010 Cologne, Germany. Introduction. Information Visualization Novel visual representations Storytelling User-Driven Visual Analysis Data exploration - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/1.jpg)
1/41
Visualization and Analysis of Text
Remco Chang, PhDAssistant Professor
Department of Computer ScienceTufts University
December 17, 2010Cologne, Germany
![Page 2: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/2.jpg)
2/41
CMVVis Examples P Topics
Introduction
• Information Visualization– Novel visual representations– Storytelling– User-Driven
• Visual Analysis– Data exploration– Hypotheses generation– Interactive visualization + Computation
![Page 3: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/3.jpg)
3/41
CMVVis Examples P Topics
Visualization
• Pre-attentive Processing
Examples courtesy of Chris Healey
![Page 4: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/4.jpg)
4/41
CMVVis Examples P Topics
Visualization• This is helpful
because:
– It allows us to process more information quickly
– We can see trends and patterns
![Page 5: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/5.jpg)
5/41
CMVVis Examples P Topics
Storytelling
• US Budget from 1961 - 2008
![Page 6: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/6.jpg)
6/41
CMVVis Examples P Topics
Storytelling
• Minard’s Map:
• Napolean’s March to Moscow
![Page 7: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/7.jpg)
7/41
CMVVis Examples P Topics
Visualization
• Influences the thought…
Images courtesy of Barbara Tversky
![Page 8: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/8.jpg)
8/41
CMVVis Examples P Topics
Visual Encoding
• Affects the:
– Types of possible operations
– The user’s thinking process
Zhang and Norman. The Representation Of Numbers. Cognition. (1995)
![Page 9: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/9.jpg)
9/41
CMVVis Examples P Topics
Classifying Numeric Systems
![Page 10: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/10.jpg)
10/41
CMVVis Examples P Topics
Example: Arithmetic
Slide courtesy of Pat Hanrahan
![Page 11: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/11.jpg)
11/41
CMVVis Examples P Topics
Example: Arithmetic
![Page 12: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/12.jpg)
12/41
CMVVis Examples P Topics
Example: Arithmetic
![Page 13: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/13.jpg)
13/41
CMVVis Examples P Topics
Example: Arithmetic
![Page 14: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/14.jpg)
14/41
CMVVis Examples P Topics
Examples of Text Visualization
• Wordle
Images Courtesy of Many Eyes
![Page 15: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/15.jpg)
15/41
CMVVis Examples P Topics
Examples of Text Visualization
• WordTree
![Page 16: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/16.jpg)
16/41
CMVVis Examples P Topics
Examples of Text Visualization
• WordTree
![Page 17: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/17.jpg)
17/41
CMVVis Examples P Topics
Examples of Text Visualization
• Phrase Net
![Page 18: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/18.jpg)
18/41
CMVVis Examples P Topics
Examples of Text Visualization
• Google Auto-Complete
![Page 19: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/19.jpg)
19/41
CMVVis Examples P Topics
Examples of Text Visualization
• Visualizing changes in Wikipedia
Images Courtesy of Info.fm
![Page 20: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/20.jpg)
20/4120/37
CMVVis Examples P Topics
Examples of Text Visualization
• ThemeRiver
![Page 21: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/21.jpg)
21/41
CMVVis Examples P Topics
Visual Exploration
• Coordinated Multi-Views (CMV)
Where
When
Who
What
Original Data
EvidenceBox
![Page 22: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/22.jpg)
22/41
CMVVis Examples P Topics
WHY?
This group’s attacks are not bounded by geo-locations but instead, religious beliefs.
Its attack patterns changed with its developments.
Coordinated Multi-Views
![Page 23: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/23.jpg)
23/41
CMVVis Examples P Topics
LIDAR Linked Feature Space
23/37
![Page 24: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/24.jpg)
24/41
CMVVis Examples P Topics
LIDAR Change Detection
24/37
![Page 25: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/25.jpg)
25/41
CMVVis Examples P Topics
Urban Model
25/37
![Page 26: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/26.jpg)
26/41
CMVVis Examples P Topics
Urban Visualization
26/37
![Page 27: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/27.jpg)
27/41
CMVVis Examples P Topics
Coordinated Multi-Views• Financial Wire Fraud
– With Bank of America– Discover suspicious
international wire transactions
• Bridge Maintenance – With US DOT– Exploring subjective
inspection reports
• Biomechanical Motion– With U. Minnesota and
Brown– Interactive motion
comparison methods
![Page 28: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/28.jpg)
28/41
CMVVis Examples P Topics
Coordinated Multi-Views• Financial Wire Fraud
– With Bank of America– Discover suspicious
international wire transactions
• Bridge Maintenance – With US DOT– Exploring subjective
inspection reports
• Biomechanical Motion– With U. Minnesota and
Brown– Interactive motion
comparison methods
![Page 29: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/29.jpg)
29/41
CMVVis Examples P Topics
Coordinated Multi-Views• Financial Wire Fraud
– With Bank of America– Discover suspicious
international wire transactions
• Bridge Maintenance – With US DOT– Exploring subjective
inspection reports
• Biomechanical Motion– With U. Minnesota and
Brown– Interactive motion
comparison methods
![Page 30: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/30.jpg)
30/41
CMVVis Examples P Topics
CMV + Text Analysis
![Page 31: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/31.jpg)
31/41
CMVVis Examples P Topics
Parallel Topics
• Task: Given the proposals submitted to the National Science Foundation (NSF), identify:
– Proposals that are interdisciplinary– Proposals that are potentially transformative– Proposals that are focused
![Page 32: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/32.jpg)
32/41
CMVVis Examples P Topics
Parallel Topics
• Approach:
– Apply topic modeling algorithms to identify latent topics (David Blei, “Latent dirichlet allocation”, 2003)
– Visualize the distribution of proposals based on the topics
![Page 33: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/33.jpg)
33/41
CMVVis Examples P Topics
Topic Modeling
• Given a set of k documents, find n number of topics– Each document then is described as:• (W1 * Topic1, W2 * Topic2, W3 * Topic3, …, Wn * Topicn)
• W1 + W2 + W3 + … + Wn = 1
Topic 1 Topic 2 … Topic NDocument 1 0.12 0.68 … 0.005Document 2 0.3 0.06 … 0.01…Document K
∑ = 1
∑ = 1...
![Page 34: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/34.jpg)
34/41
CMVVis Examples P Topics
Topic Modeling
• A topic is a combination of keywords
![Page 35: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/35.jpg)
35/41
CMVVis Examples P Topics
Parallel Topics
• Based on “Parallel Coordinates”– Each vertical axis is a topic– Each set of horizontal connected lines is a
document
![Page 36: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/36.jpg)
36/41
CMVVis Examples P Topics
Visual Signatures
Single topic Bi-topic
No salient topic
• We identify different signatures for proposals:– Single Topic – focused research– Bi-Topic – Interdisciplinary research– No-Topic – Potentially transformative research
![Page 37: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/37.jpg)
37/41
CMVVis Examples P Topics
Selecting Single Topic Proposals
Topic 1 Topic 2 … Topic N
Document 1 0.12 0.68 … 0.005
Document 2 0.3 0.06 … 0.01
…
Document K
SD = 0.14SD = 0.06
Max
SD
![Page 38: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/38.jpg)
38/41
CMVVis Examples P Topics
Selecting Multi-Topic ProposalseducationtechnologyInteractive
environment
![Page 39: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/39.jpg)
39/41
CMVVis Examples P Topics
Selecting No-Topic Proposals
![Page 40: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/40.jpg)
40/41
CMVVis Examples P Topics
Recap
• Objective: To discover interdisciplinary and potentially innovative research proposals
• Parallel Topics – data-centric approach
• Approach: To support interactive selection of proposals based on their number of topics
![Page 41: Visualization and Analysis of Text](https://reader035.vdocuments.site/reader035/viewer/2022062519/56815170550346895dbfa635/html5/thumbnails/41.jpg)
41/41
CMVVis Examples P Topics
Questions and Comments?
Thank you!!
[email protected]://www.cs.tufts.edu/~remco