april morton gulcan can rui hu - unifr.ch morton gulcan can rui hu ... develop statistical shape...
TRANSCRIPT
MAAYA: Multimedia Analysis and Access for Documentation and
Decipherment of Maya Epigraphy
April Morton
Gulcan Can
Rui Hu
February 27th, 2013
The MAAYA Project
1
Main goal: Advance the state of the art in image description and visualization techniques, while providing support to epigraphers
Dr. Daniel Gatica-Perez
Gulcan Can
Dr. Jean-Marc Odobez
Dr Rui Hu
Prof. Stéphane Marchand-Maillet
Dr. Edgar Roman-Rangel
April Morton
Oscar Dabrowski
Prof. Nikolai Grube
Carlos Pallan
Guido Krempel
Outline
• Mayan Overview
• Project Overview
• Data Preparation/Storage
• Computer Vision Algorithms
• Using Context to Improve Glyph Recognition
• Visualization
• Conclusion
2
Mayan Overview
3
Ancient Mayan Culture
Flourished from BC 2000 - 1500 AD
4
Mayan Writing System
Mayan Overview
Logographs
Syllabographs
Iconography
Take Photographs
Obtain Digital Line Drawings
Identify Glyphs Annotate Glyphs
Documentation Process
5
Mayan Overview
Need better digital preservation and automatic analysis techniques
6
The CODICES Project
Project Overview
Statistical Shape Descriptors
CBIR System
Main goal: Develop statistical shape descriptors and a CBIR system
The MAAYA Project
Project Overview
Data Preparation/Storage
Content-Based Data Processing/Computer
Vision Algorithms High Level Analysis Visualization
Main Goal: Advance the state of the art in image description and visualization techniques, while providing support to epigraphers
7
8
Data Preparation/Storage Process
Segment glyphs and blocks
Upload segmented page into database
Annotate and store glyphs
Computer Vision Algorithms
9
Glyph Representation
Identification
(Classification
- Retrieval)
Attribute Learning
Where is the
data?
How to recognize the data?
Localization
Segmentation
Decomposition
Sparse Representation
Shape Descriptors
Part-based Representation
Computer Vision Algorithms
10
Localization
Computer Vision Algorithms
Localization
11
Computer Vision Algorithms
12
Segmentation
Hierarchical Decomposition
Codices → Page → T'ol → Row of Blocks → Block → Glyph
Computer Vision Algorithms
13
• Glyph representation + Identification • Can we utilize/capture diagnostic features?
Computer Vision Algorithms
14
Recognition
Computer Vision Algorithms
15
Diagnostic Features
Computer Vision Algorithms
16
Histogram-of-Orientations Shape Context
17
Learning Features with Autoencoders
Computer Vision Algorithms
Computer Vision Algorithms
18
Compute pivots and HOOSC descriptors
k-means clustering & bag-of-visual-words
representation
Ranking of “bov”s of each glyph
Identification – Glyph Retrieval
Computer Vision Algorithms
16
Compute pivots and HOOSC descriptors
Classify each pivot
Apply voting for glyph-level prediction
Identification - Classification
19
Using context to improve glyph recognition
20
Using context to improve glyph recognition
21
Using context to improve glyph recognition
22
Category:T668.T102 Cha-ki Meaning: Rain god
Category:T227:T561.T23 xib?-ka’an-na Meaning: man, sky, ?
Using context to improve glyph recognition
23
Data preparation
Using context to improve glyph recognition
24
Glyph matching
Using context to improve glyph recognition
25
Glyph matching
Using context to improve glyph recognition
26
● Statistic language model
● Icon
● Translation
Category:T668.T102 Cha-ki Meaning: Rain god
Category:T227:T561.T23 xib?-ka’an-na Meaning: man, sky, ?
Using context to improve glyph recognition
27
Context Information
Using context to improve glyph recognition
28
● Improve recognition accuracy
● Help with noisy and partially missing examples
● High-level translation
Using context to improve glyph recognition
29
A final goal is to advance state-of-the art in effective interactive visualization tools
Visualization
Input Data
Apply Data Mining
Technique
Display Visualization
Accept User Input
Update Parameters
30
MDS projection based on weighted Euclidean distance function
Interactive Clustering
Visualization
Data from UCI wine data set
Eli T. Brown, Jingjing Liu, Carla E. Brodley, Remco Chang: Dis-function: Learning distance functions interactively. IEEE VAST 2012: 83-92
31
Interactive Clustering
Visualization
Re-projection
Eli T. Brown, Jingjing Liu, Carla E. Brodley, Remco Chang: Dis-function: Learning distance functions interactively. IEEE VAST 2012: 83-92
32
Interactive Clustering
Visualization
Updated Weights
Re-projection Could think of data points as glyphs
Eli T. Brown, Jingjing Liu, Carla E. Brodley, Remco Chang: Dis-function: Learning distance functions interactively. IEEE VAST 2012: 83-92
33
Distribution of values among attributes
Visualization
Can filter and sort words based on probability of co-occurrence
Area encodes probabilities
Topic Model Visualization
Jason Chuang, Christopher D. Manning, Jeffrey Heer: Termite: visualization techniques for assessing textual topic models. AVI 2012: 74-77
34
Topic Model Visualization
Visualization
Area encodes probabilities
Can filter and sort words based on probability of co-occurrence
Can think of terms as glyphs
Can think of documents as codices (add) Jason Chuang, Christopher D. Manning, Jeffrey Heer: Termite: visualization techniques for assessing textual topic models. AVI 2012: 74-77
35
Conclusion
The MAAYA project aims to improve the entire process of digital glyph analysis
Data Preparation/Storage
Content-Based Data Processing/Computer
Vision Algorithms High Level Analysis Visualization
36
Main Goal: Advance the state of the art in image description and visualization techniques, while providing support to epigraphers
And a special thanks to the Swiss National Science Foundation: Project CR21I2L_144238
Acknowledgements
37
Questions
38
Backup Slides
The webpage includes a user friendly upload wizard for segmented files
The Online Wizard
As well as a standardized data annotation page
The Data Annotation Page
5
Mayan Overview
Mayan Writing System
Mayan Writing System
6
Mayan Overview