Letting structure emerge:

Connectionist and dynamical systems approaches to cognition (McClelland, et al.,

2010)

Jennifer D’SouzaDonald Kretz November 17, 2010

Definitions and main concepts Connectionist and probabilistic models General arguments against probabilistic

models Cognitive areas where models are useful

◦ Language◦ Development◦ Semantics

Conclusions Questions and Discussion

Outline

Some definitions

Connectionism - neurons are the basic information processing structures in the brain, and every sort of information the brain processes occurs in networks of interconnected neurons. Models knowledge and knowledge acquisition as adjusting strengths on connections of networks of “neuron-like processing units”

Neural Networks

Structured probabilistic models

Semantic Cognition

Universal grammar

Explore the claim

Arguments against probabilistic models

Cognitive Domain – Emergent Phenomena Models

Conclusion

Main idea: How low-level, highly localized, non-cognitive processes can combine to produce cognition

Claim: only models that account for how neurons connect (“mechanism”) can fully explain cognition

Main concepts

Explore the claim

Do the ants possess a blueprint for creating this structure?

Ant hill analogy

“Human thoughts, language and behavior have a rich and complex structure that is the emergent consequence of a large number of simpler processes.”

Emergent phenomena

Reaching task

Looking + discriminating locations + posture control + motor planning

Reaching Action

Emergent Consequence

Structured Probabilistic Approach to

CognitionReading 15Griffiths, T.L. et al. (2010) Probabilistic models of cognition: Exploring the laws of thought

Structured Probabilistic Approach

Cognizing Agents

Hypothesis Space with a prior probability

distribution

ObservationsResult with

highest posterior probability

InferenceInput

Evaluation

Arguments against probabilistic models

Why is consideration of the structured probabilistic approach to cognition dangerous?

Computational Level – what does the system do?

Algorithmic Level – how does the system do what it does?

Implementational Level – how is the system physically realized?

Marr’s Tri-Level Hypothesis to understanding vision as an information processing system

Structured Probabilistic Inference Model

Chomsky’s competence-based approach to linguistics

Probabilistic Inference Problem Goal is to characterize language user’s knowledge

Select the correct knowledge structure

Grammar as representation that explain the facts

Abstraction from cognitive tasks – Marr’s computational level

Theory is pitched at a competence level

Box 1: Chomsky’s structured probabilistic approach to linguistics

Cognitive performance remains as a “promissory note”.

Problem formulation is not neutral.

Is there a generalized knowledge representation technique?◦ Propositional Logic

◦ First-order Logic

Why is it not entirely relevant?

“There is someone who can be fooled every time.”

Treating levels of analysis as independent is counter-productive.

Level of description and competence / performance approaches also introduce a comfortable extra degree of freedom w.r.t. data.

Why is it not entirely relevant?

High level computational theories Behavior

Explicit inferences in contingency learning task

Structured Probabilistic approaches fall short

Ample time to make response

Quick response – Time constrained

Exploit the causal framing scenario to make normatively correct, explicit inferences.

Learning process – simple connection weight adjustments.

How can the statistical structure or the computational-level analysis of what would be optimal be the same?

NOT about probabilities – both approaches emphasize statistics

NOT about bottom-up over top-down – both are important

IS about cognition being a choice of statistical models

What is the disagreement about?

The utility of the structured probabilistic approach depends in part on the validity of the units as descriptions of linguistic structure. Herein lies the problem.

◦ Hypothesis space in the case of language would be characterized by discrete units such as phonemes, morphemes and sentences.

Case in point: The units problem to language and cognition

From citation 37 ◦ As far as phonemes are concerned:-

◦ High frequency words – just, went or don’t all have the final /t/ or /d/ deleted than,

◦ Low frequency words – innocent, interest or attract

◦ High frequency words - Every – 2 syllable word

◦ Lower frequency words – Mammary, Summary – 3 syllable words Memory, family – anywhere between 2 to 3 syllables

Why can’t we generalize the use of phonemes or morphemes?

From citation 36 ◦ As far as morphemes are concerned:-

◦ Derived forms that are more frequent than their base should be less decomposable, than derived forms that are less frequent than their base.

◦ Conclusion – Relative frequency matters more than absolute frequency.

Why can’t we generalize the use of phonemes or morphemes?

Arguments against probabilistic approaches No real basis – not representative of the actual processes –

use of probabilistic models is unnecessary and dangerous Don’t account for (explain) the development of cognitive

abilities What if the high-level models are wrong?

Source of disagreement NOT about probabilities – both approaches emphasize

statistics NOT about bottom-up over top-down – both are important IS about cognition being a choice of statistical models

General Arguments Against Probabilistic Models

Language◦ Tense, word reading, sentence processing

Development◦ Stage transitions, walking

Semantics◦ Representing living vs. nonliving things

Cognitive Areas for Modeling

For each cognitive area, the authors present:

Their interpretation of probabilistic modeling approaches

Examples of probabilistic models gone awry

An explanation of how a connectionist model better accounts for cognitive development and activity

Cognitive Areas for Modeling

Probabilistic: ◦ Problem formulation◦ A priori assignment of outcomes and probabilities◦ Abstraction from mechanistic details

Chomsky’s universal grammar◦ Formulation: characterizing knowledge of

language user assuming that is the user’s goal◦ Commitment: selecting grammar that explains

such knowledge but may not select the grammar that convergence would

◦ Abstracting at the competence level but may not map directly to behavior or neurophysiological details

Cognitive Models: Language

Probabilistic: ◦ Characterized in discrete units

Elemental structure◦ Phonemes, morphemes, sentences but these are

matters of degree and may be misleading approximations

Cognitive Models: Language

Connectionist:

◦ No fixed vocabulary of representational units

◦ Graded patterns of distinctness, compositionality, and context sensitivity

Cognitive Models: Language

Probabilistic: ◦ Stages of development (Piaget)◦ Object permanence

A-not-B task◦ Objects exist independent of one’s own action◦ Not an explicit focus of research in probabilistic

modeling

Cognitive Models: Development

Connectionist: ◦ Dynamic Field Theory – integrates multiple

sources of relevant information◦ Situation (events, past reaches, object positions)◦ Motor planning (direction of next reach)

Cognitive Models: Development

Probabilistic: ◦ Acquiring semantic knowledge represents a

choice among alternatives◦ Requires knowledge of hypothesis space, space of

possible choices, prior distributions

Taxonomic hierarchy of nature◦ Separate branches such as birds, fishes, and

mammals do not account for partial homologies◦

Cognitive Models: Semantics

Connectionist:◦ Learn a set of weights◦ Discrepancies between predicted and observed

outcomes serve as feedback to weight adjustment◦ Related items evoke similar but differentiated

internal representations

Cognitive Models: Semantics

Cognition depends fundamentally on underlying mechanism – abstract models will miss important aspects

Connectionist modeling efforts have led to advances in cognitive theories

Authors advocate an integrated approach where high-level models are informed by knowledge about underlying neural mechanisms

Conclusions

1. Why are we interested in modeling cognition?

2. Is there always a need to choose one modeling approach over the other?

3. Did the authors convince you that higher cognitive abilities can be modeled at the connectionist level?

4. “Understanding how each and every neuron functions still tells us absolutely nothing about how the brain manufactures a mental state.” (Gazzaniga, 2010)

5. Are there other types of symbolic models (other than probabilistic) that may be appropriate for cognitive modeling?

Questions & Discussion

Emergence as patterns observed from activations and inhibitions across connections of neurons. Note that the term "emergent" was coined by the pioneer

psychologist G. H. Lewes, who wrote:"Every resultant is either a sum or a difference of the co-operant forces; their sum, when their directions are the same -- their difference, when their directions are contrary. Further, every resultant is clearly traceable in its components, because these are homogeneous and commensurable. It is otherwise with emergents, when, instead of adding measurable motion to measurable motion, or things of one kind to other individuals of their kind, there is a co-operation of things of unlike kinds. The emergent is unlike its components insofar as these are incommensurable, and it cannot be reduced to their sum or their difference." (Lewes, G. H. (1875), Problems of Life and Mind (First Series), 2, London: Trübner)

Questions & Discussion