timo honkela: from patterns of movement to subjectivity of understanding
DESCRIPTION
Human visual system interprets information obtained through eyes to build a model of the surrounding world. This channel is our main source for understanding the world. Walking on a street, reading a book, or watching a movie all rely on our visual system. The relationship between movement, visual perception and language is complex. Movement is a specific focus of this presentation for several reasons. It is a fundamental part of human activities that ground our understanding of the world. Abstract meanings are often constructed as metaphoric extensions of movement schemas. As there is an increasing amount of video and motion tracking data available, formation of semantic models based on movement using computational methods is becoming feasible. In addition to movement, multilinguality and subjectivity of understanding are also addressed.TRANSCRIPT
![Page 1: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/1.jpg)
From Patterns of Movement toSubjectivity of Understanding
Timo Honkela
Aalto University(former Helsinki University of Technology)
Department of Information and Computer ScienceCognitive Systems research group
Finland
UC Santa Barbara, 4 Dec 2012
![Page 2: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/2.jpg)
0Background and
(un)Related Topics
![Page 3: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/3.jpg)
Natural language database interfacewith dependency-based compositional semantics
● H. Jäppinen, T. Honkela, H. Hyötyniemi & A. Lehtola (1988):A Multilevel Natural Language Processing Model. Nordic Journal of Linguistics 11:69-87.
What is the turnover of the ten largest stock exchange companies in forestry?
Morphological analysis
Dependency parsing
Logical analysis
Database query formation
Result from the SQL database
![Page 4: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/4.jpg)
Classical example: Learning meaning from context:
Maps of words in Grimm fairy tales
Honkela, Pulkki & Kohonen 1995
![Page 5: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/5.jpg)
Map of Finnish Science
Chemistry
Physics andengineering
Biosciences
Medicine
Culture and society
![Page 6: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/6.jpg)
WordICA
Timo Honkela, Aapo Hyvärinen, and Jaakko Väyrynen. WordICA - Emergence of linguistic representations for words by independent component analysis. Natural Language Engineering, 16(3):277–308, 2010.
Jaakko J. Väyrynen, Lasse Lindqvist, and Timo Honkela. Sparse distributed representations for words with thresholded independent component analysis. In Proceedings of IJCNN'07, pages 1031–1036, 2007.
![Page 7: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/7.jpg)
Learning taxonomies
Mari-Sanna Paukkeri, Alberto Pérez García-Plaza, Víctor Fresno, Raquel Martínez Unanue and Timo Honkela (2012). Learning a taxonomy from a set of text documents. Applied Soft Computing, 12(3), pp. 1138--1148.
![Page 8: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/8.jpg)
Text mining for peer support
TOPIC ANALYSIS SENTIMENT ANALYSIS
Discussion forum postings, etc.
Selected stories
STYLEANALYSIS
MULTICRITERIA SELECTION PROCESS
User modelingand analysis of
feedback
EVALUATION
User'sinput
(Hon
kela
, Iz
zatd
ust,
Lag
us 2
012)
![Page 9: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/9.jpg)
ICA of wellbeing-related termsin Reddit texts
(Honkela, Izzatdust, Lagus 2012)
![Page 10: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/10.jpg)
Analyzing Complexity of Languages
Markus Sadeniemi, Kimmo Kettunen, Tiina Lindh-Knuutila, and Timo Honkela. Complexity of European Union languages: A comparative approach. Journal of Quantitative Linguistics, 15(2):185–211, 2008.
![Page 11: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/11.jpg)
META-NETNetwork of Excellence
![Page 12: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/12.jpg)
MultilingualWeb
![Page 13: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/13.jpg)
Concept Formation andCommunication - General Theory
Timo Honkela, Ville Könönen, Tiina Lindh-Knuutila, and Mari-Sanna Paukkeri. Simulating processes of concept formation and communication. Journal of Economic Methodology, 15(3):245–259, 2008.
λ : Ci × Cj → R, i ≠ jA distance between two points in the concept spaces of different agents
S: symbol space,The vocabulary of anagent that consists of discrete symbols
: sξ i S∈ i → CAn individual mapping function from symbols to concepts
φi: Si D→An individual mapping from agent i's vocabulary to the signal space D andan inverse mapping φ
1 i from the signal
space to the symbol space
Ci: Ndimensional metric concept space
Observing f1 and after symbol selection process, agent 1 communicates a symbol s*to agent 2 as signal d. When agent 2 observes d, it maps it to some s2
S∈ 2 by using the function φ 11.
Then it maps the symbol to some point in its concept space by using ξ2. If this point is close to its observation f2 in the sense of λ, the communication process has succeeded.
![Page 14: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/14.jpg)
1Patterns of
Movement
![Page 15: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/15.jpg)
Why brains?
● What are the central differences between plants and animals?
“The original need for a nervous system was to coordinate movement, so an organism could go find food, instead of waiting for the food to come to it.”
● An extreme example: A sea squirt transformsfrom an “animal” to a “plant”. It absorbs its own cerebral ganglion that it used to swim about and find its attachment place.
http://goodheartextremescience.wordpress.com/2010/01/27/meet-the-creature-that-eats-its-own-brain/
http://www.fi.edu/learn/brain/
![Page 16: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/16.jpg)
Human movement
![Page 17: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/17.jpg)
![Page 18: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/18.jpg)
David Bailey's thesis (1997):
Verbs related to hand movement
![Page 19: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/19.jpg)
Point of view fromcognitive linguistics
● The meaning of linguistic symbols in the mind of the language users derives from the users' sensory perceptions, their actions with the world and with each other.
● For example: the meaning of the word 'walk' involves● what walking looks like● what it feels like to walk and after having walked● how the world looks when walking
(e.g. objects approach at a certain speed, etc.). ● ...
![Page 20: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/20.jpg)
Abstract vs concrete grounding
Ronald Langacker
![Page 21: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/21.jpg)
Motion capture
AnimationImage analysis
Video analysis
Robotics
Machine learning
Language learning
Socio-cognitive modelingSymbol grounding
Jorma Laaksonen
Tapio Takala
Klaus Förger
Harri Valpola
Oskar Kohonen
Timo Honkela
Reinforcementlearning
Paul WagnerMarkus Koskela
Xi Chen
Learning relations
Kinect
OptiTrack
![Page 22: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/22.jpg)
Multimodally Grounded Language Technology
A project funded by Academy of Finland2011-2014
Timo Honkela as the Principal Investigator
A collaboration betweendepartments of
* Information and Computer Science, and
* Media Technology
![Page 23: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/23.jpg)
Turk on Gesture Interaction
Matthew Turk, UCSB
Potential uses of gesture interaction technologies:
● mouse replacement for user interaction
● video game control● navigation for
visualization● sign language
recognition● automatic transcription
of communication● medical diagnosis and
rehabilitation● sculpting● conducting and playing
music● interactive art
Earlier today:
![Page 24: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/24.jpg)
Labeling movements
From an unpublished manuscript. Experiments by Klaus Förger.
![Page 25: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/25.jpg)
Linking between modalities
![Page 26: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/26.jpg)
Potential uses of the emerging technologies
● Multimodally grounded natural language interaction and machine translation
● Animation based on linguistic instruction● Automated skill instruction
(playing an instrument, learning some sports, etc.)
● Video annotation● Addressing some of the fundamental issues
in traditional AI, cognitive science and philosophy
![Page 27: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/27.jpg)
2Contextuality and
Subjectivity of
Understanding
![Page 28: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/28.jpg)
Meaning is contextual
red winered skinred shirt
Gärdenfors: Conceptual Spaces
Hardin: Color for Philosophers
![Page 29: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/29.jpg)
Meaning is contextual
SNOW -WHITE?
WHITE
![Page 30: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/30.jpg)
Meaning is contextual
● “Small”, “big”● “White house”● “Get”● “Every” - “Every Swede is tall/blond”● etc. etc.
Another comment:
Strict compositionality cannot be assumed
Fuzziness
![Page 31: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/31.jpg)
Learning meaning from context
● Self-Organizing Semantic Maps● Latent Semantic Analysis● Latent Dirichlet Allocation● WordICA● etc. etc.
![Page 32: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/32.jpg)
Classical example: Learning meaning from context:
Maps of words in Grimm fairy tales
Honkela, Pulkki & Kohonen 1995
![Page 33: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/33.jpg)
Meaning is subjective
![Page 34: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/34.jpg)
Meaning is subjective
● Good● Fair● Useful● Scientific● Democratic● Sustainable● etc.
A proper theory ofmeaning has to takethis into account
![Page 35: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/35.jpg)
Gary B. Fogel11th of June, 2012
WCCI 2012
![Page 36: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/36.jpg)
2bMeasuring
Subjectivity of
Understanding
![Page 37: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/37.jpg)
User-specificdifficulty
assessment
Basic architecture of the method
![Page 38: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/38.jpg)
![Page 39: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/39.jpg)
GICA:Grounded
Intersubjective Concept Analysis
Description of the method
![Page 40: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/40.jpg)
Timo Honkela, Juha Raitio, Krista Lagus, Ilari T. Nieminen, Nina Honkela, and Mika Pantzar.
Subjects on objects in contexts: Using GICA method to quantify epistemological subjectivity.
Proceedings of IJCNN 2012, International Joint Conference on Neural Networks, pp. 2875-2883, 2012.
Publication:
![Page 41: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/41.jpg)
Subjectifying: adding subjective views into object-context matrices
Outcome: Subject-Object-Context (SOC) Tensors
![Page 42: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/42.jpg)
Potential sources for subjectification
● Conceptual surveys: ● individual assessment of contextual
appropriateness
● Text mining:● statistics of word/phrase-context patterns
● Empirical psychology:● reaction times, etc.
● Brain research
![Page 43: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/43.jpg)
Flattening: unfolding 3-way tensorfor traditional 2-way analysis
![Page 44: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/44.jpg)
GICA:Grounded
Intersubjective Concept Analysis
Examples of use
![Page 45: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/45.jpg)
OBJECTS:
Relaxation
Happiness
Fitness
Wellbeing
CONTEXTS:
SUBJECTS: Event participants
Data collection
![Page 46: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/46.jpg)
MDS: Objects x Subjects
Fitness
![Page 47: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/47.jpg)
NeRV: Objects x Subjects
Fitness
J. Venna, J. Peltonen, K. Nybo, H. Aidos, and S. Kaski. Information Retrieval Perspective to Nonlinear Dimensionality Reduction for Data Visualization. Journal of Machine Learning Research, 11:451-490, 2010.
NeRV:
![Page 48: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/48.jpg)
SOM: Objects x Subjects
![Page 49: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/49.jpg)
Case 2: State of the Union Addresses
● In this case, text mining is used for populating the Subject-Object-Context tensor
● This took place by calculating the frequencies on how often a subject uses an object word in the context of a context word● Context window of 30 words
![Page 50: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/50.jpg)
Analysis of the word 'health'
![Page 51: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/51.jpg)
Timo Honkela, Ville Könönen, Tiina Lindh-Knuutila, and Mari-Sanna Paukkeri. Simulating processes of concept formation and communication. Journal of Economic Methodology, 15(3):245–259, 2008.
Conclusions (1)
● Languages, including formal languages, should be considered as tools for coordination, storing and sharing knowledge in a compressed form – approximate and relative to the point of view taken
● Constructing a language or symbol system (such as an ontology) is an investment and spreading the language into use in a community is even a larger one
![Page 52: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/52.jpg)
Conclusions (2)
● Making people aware of the differences in the conceptual systems among them may have different applications, e.g., ● Helping in conflict resolution● Promoting interdisciplinary communication● Enhancing participatory processes and
democracy
![Page 53: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/53.jpg)
From TEDxAALTO presentation “Measuring Subjectivity of Meaning – and How it may change our life” with illustrations by Nelli Honkela
![Page 54: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/54.jpg)
1+2Movement and
Subjectivity
![Page 55: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/55.jpg)
goo.gl / UZnvH
![Page 56: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/56.jpg)
goo.gl / UZnvHSurvey address:
![Page 57: Timo Honkela: From Patterns of Movement to Subjectivity of Understanding](https://reader030.vdocuments.site/reader030/viewer/2022020116/55618c6bd8b42a91778b4cee/html5/thumbnails/57.jpg)
Thank y o u !