III. Brain Function I: Evidence from Linguistics
The Neurological Basis of Language and Thought
Brain, Mind, and Belief: The Quest for Truth
Language, for all its seeming complexity, is more amenable to analysis than other cognitive structures, so that investigation of language is one of the best ways to proceed to an understanding of human mind and nature.
Tim Pulju, 1990
Agenda for TodayThe Problem: How does the brain work?
History of the study of brain function Common errors to be avoided
• Tool-driven inquiry• The misapplied metaphor
Help from the study of linguistic structure• Claim: As language works in the brain, so the
brain works in general• Therefore, If we can understand how language
works, we will know how the brain works• Relational networks
Cortical columns of neurons
2
History of the study of brain function I
Early investigators in the 19th century came up with the idea of locationism: Local areas of the cortex have specific functions• Franz Joseph Gall (1758-1828)
promoted the idea His followers took it too far As a result, the idea of locationism was widely discredited
3
History of the study of brain function II
Locationism: Local areas of the cortex have specific functions• Paul Pierre Broca (1824-1880)
Stroke patient with impaired speech Autopsy after patient’s death Damage in lower left frontal lobe
• Area now known as Broca’s area• Responsible for articulation of speech
4
History of the study of brain function III
The origin of Connectionism Like locationism but more sophisticated:
Local areas of the cortex have highly specific simple functions, and complex functions are carried out by multiple interconnected areas• Carl Wernicke (1848 - 1906)
Stroke patient unable to comprehend speech
Damage in upper left temporal lobe• Area now known as Wernicke’s area• Responsible for speech comprehension
5
Connectionism
Connectionism includes a version of locationism• But is more sophisticated
Each local area performs a very specific simple function
Complex functions require multiple local areas acting together
• They can act together because they are interconnected
» CONNECTIONS RULE!
6
Two basic language areas
Primary Somato-sensory Area
Primary Motor Area
Primary AuditoryArea
PrimaryVisual Area
Mouth
HandFingers
Arm
Trunk
Leg
PhonologicalRecognition
PhonologicalProduction
Broca’s area
Wernicke’s area
7
Arcuate Fasciculus (from langbrain website)
Connects Wernicke’s area to Broca’s area
8
History of the study of brain function IV
The decline and revival of Connectionism• As with Gall, followers of Wernicke were
too speculative and went too far, and the whole idea was discredited for several decades
• Finally revived in the 1960’s by Norman Geschwind (1926-1984)
• Now widely accepted by neurologists (but criticized by some psychologists)
9
History of the study of brain function V
Two major methods of investigation① Lesion studies
E.g., Broca and Wernicke and Geschwind If area A is damaged and function F is
impaired, then A must have function F (or at least contribute to F)
② Functional Brain Imaging• A recent innovation• Made possible by technological advances• Now very widely used• Location-based but sometimes without
the sophistication of connectionism
10
Functional Brain Imaging
Electro-Encephalography (EEG)• Excellent temporal resolution• Very poor spatial resolution
Positron Emission Tomography (PET)• Poor spatial resolution• Very poor temporal resolution
Functional Magnetic Resonance Imaging (fMRI)• Spatial resolution better than PET• Temporal resolution a little better than PET
Magneto-Encephalography (MEG)• Excellent temporal resolution• Spatial resolution not so good
11
Positron emission tomography (PET)
PET shows areas of increased cortical metabolism Spatial resolution: 5-10 mm
• How good is that? Under one sq mm of cortical surface, 130,000 neurons
Temporal resolution: “…on the order of minutes…” (A. Papanicolaou, Fundamentals of Functional Brain Imaging (1998), p. 14)
12
Functional Magnetic Resonance Imaging (fMRI)
Measures the amount of oxygenated blood supplied to different areas of the brain• Common abbreviation: rCBF (regional cerebral blood flow)
When a group of neurons increases its signaling rate, its metabolic rate increases
13
An fMRI example
www.firstscience.com/ SITE/ARTICLES/love.asp
Areas of the brain used in working memory
14
Properties of fMRI
Temporal resolution: Not very good Image reflects the increase in oxygenated blood
that occurs 5 to 8 seconds after the neurons fire Spatial resolution:
• Better than PET• But it is unclear whether the imaged area is
precisely the area involved in the activity The flow of oxygenated blood into the depleted
area may also flow into neighboring vessels in areas where neural firing did not occur
15
Magnetoencephalography (MEG)
MEG (MagnetoEncephaloGraphy) measures the magnetic field around the head
Magnetoencephalography
magnetic brain
pictureproduction of
16
How MEG works
An electric current is always accompanied by a magnetic field perpendicular to its direction
MEG records the magnetic flux or the magnetic fields that arise from electric currents in neurons
Magnetic flux lines are not distorted as they pass through the brain tissue because biological tissues offer practically no resistance to them • Therefore, MEG is more accurate than EEG
17
Magnetic flux from source currents
Source current
Magnetic flux Magnetometer
18
Recording of Magnetic Signals
19
Temporal Resolution of MEG
Excellent – unlike fMRI and PET Therefore, it is possible to discern the temporal order
of activation of cortical areas MEG has potential to detect the activation of several
brain regions as they become active from moment to moment during a complex function such as recognition
20
A major challenge of MEG
The cortex is a parallel processor• Hundreds or thousands of dipoles can be
active simultaneously Multiple dipoles make comprehensive
inverse dipole modeling virtually impossible Hence, compromises are necessary
• Sample larger time spans (up to 500 ms)• Sample larger areas (up to several sq cm)
21
Some MEG results: Speech recognition
Hemispheric Asymmetry Wernicke's Area
22
Wernicke’s area in Spanish-English bilinguals
From MEG lab, UT Houston23
Spatial Resolution
How accurately is location determined?• EEG: Poor• PET: Fair – 4-5 mm• fMRI: Fair – 4-5 mm• MEG: Fairly good – 3-4 mm or less
Under good conditions How good are these figures?
• under 1 sq mm of cortical surface,140,000 neurons
24
Temporal resolution
Temporal resolution• Key neural events can occur within 5 ms• Terrible: PET
40 seconds and up• Pretty bad: fMRI
10 seconds or more• Excellent: MEG and EEG
Instantaneous Theoretically it is possible to do ms by ms tracking,
to follow time course of activation But such tracking is usually difficult or impossible
• The inverse problem• Too many dipoles at each point in time
25
Sensitivity of Imaging Methods
All of the methods have limited sensitivity MEG
• 10,000 dendrites in close proximity have to be active to detect signal
PET and fMRI• Similar limitations
Any activation that involves fewer numbers goes undetected
26
Faulty thinking in neuroscience I:Tool-driven inquiry
Tool-driven inquiry: letting the available tools shape the investigation• Like looking for the lost car keys under the street
light instead of where they got lost The available tools: Brain imaging machines What they are good for: determining locations
of brain activity Therefore, what do they investigate?
• Locations of brain activity The question being asked: Where? The questions not being asked: What?, How?
• What is going on?• How does the brain work?
27
More on the lack of interest in what/how
There are no available machines for investigating the what/how question
Experiments have not been devised for investigating the what/how question
It is necessary to rely on thinking Scientists believe that doing science is conducting
experiments and using high-tech machinery• Thinking is done by theoreticians
Akin to philosophers and poets
28
A mitigating circumstance?
Many have not realized that the what/how question is important
They may assume it is already known:• The brain is assumed to work like a computer
A symbol-manipulating device• This assumption is unwarranted
29
Faulty thinking in neuroscience II:The misapplied metaphor
The brain is assumed to work like a computer This assumption is unwarranted
The computer is a symbol-manipulating device• Not a connectionist system
An example of faulty thinking:• The misapplied metaphor
The brain works by means of connectivity and operations upon its connectivity
30
The Cortex is a NetworkEntirely different structure than that of computers
Connectivity as key property of brain structure Symbol-manipulation is the key property of computers The cortex operates by means of connections
• Transmission of activation along neural pathways• Changes in connection strengths
31
Computers and Brains: Different Structures, Different Skills
Computers• Exact, literal• Rapid calculation• Rapid sorting• Rapid searching• Faultless memory• Do what they are told• Predictable
Brains• Flexible, fault tolerant• Slow processing• Association• Intuition• Adaptability, plasticity• Self-driven activity• Unpredictable • Self-driven learning
32
Things that brains but not computers can do
Acquire information to varying degrees• “Entrenchment” • How does it work?
Variable connection strength Connections get stronger with repeated use
Perform at varying skill levels• Degrees of alertness, attentiveness• Variation in reaction time• Mechanisms:
Global neurotransmitters Variation in blood flow Variation in available nutrients Presence or absence of fatigue Presence or absence of intoxication
33
How to study the what/how question
You have to think harder Also, we can make use of findings from structural
linguistics Language as the key to unlock mysteries of the mind
• If cortical structures for language are like those for other high-level skills
• Then if we figure out language, we also have the answer to how other high-level intellectual processing works
34
Thinking harder
Avoid metaphorical thinking• The brain is not a computer• Not like a human being with paper & pencil & books• In fact it is not like anything else
It is itself: the brain
35
How does your brain tell your fingers what to do?
Question to daughter (age 6):• How does your brain tell your finger to hit that
key on the piano? Sarah:
• Well my brain writes a little note and sends it down to my finger, …
What really happens? Neurons in the motor cortex send activation
(nerve impulses) down (through subcortical structures and the spinal cord) to neurons that activate the muscles that make my finger move
36
Auditory Imagery
Auditory images of words, music, etc.• We can hear things in our heads• What is an auditory image?
What does it consist of?• Sound?
» There is no air inside the head to vibrate• What hears it?
» There are no little ears inside the head
37
Visual Imagery
Visual images of people, buildings, etc. What is a visual image? What does it consist of?
• Is it a little picture? If so, where are the eyes to see it? What is it drawn on? Where is the visual perception system
to interpret it?• If not, what?
38
Vision
When you see something.. A picture on your retina?
• Something in your brain looks at it?• Are there a couple of little eyes inside?
A picture somewhere inside your brain?• Same problems: no eyes inside
And if there were, they would have to be supported by a visual perception system
39
Compare the TV set
Does it have little people inside? Similarly, no pictures inside the brain No sounds inside the brain No words or other symbols inside the brain
40
How vision really works
There is no picture on the retina, just neurons that get activated by light• Some of them get activated by color
The visual perception system learns during early childhood to integrate configurations of these little dots into larger units
Next higher level: larger configurations Many levels up, recognition of objects
• The brain goes through a long process of learning, to build these many levels
• More on this next week!
41
The Nature of Language
Some history• Louis Hjelmslev• Prolegomena to a Theory of Language (1943/60)
Linguistic structure is a system of relationships
42
”The postulation of objects as something different from the terms of relationships is a superfluous axiom and con-sequently a metaphysical hypothesis from which linguistic science will have to be freed.”
Understanding linguistic units as purely relational I
dog
d - o - g
Symbols?Objects?
Understanding linguistic units as purely relational II
dog Seems to be one unit
Three phonemes (or graphemes), in sequence
d o g
Understanding linguistic units as purely relational III
DOG Noun
d o g
dog The object we are considering
The meaning of dog – a concept
Grammatical properties
Understanding linguistic units as purely relational IV
DOG Noun
d o g
dog
Understanding linguistic units as purely relational V
DOG Noun
d o g
We can remove the symbol with no loss of information. Therefore, it is a connection, not an object
Another way of looking at it
DOG Noun
d o g
dog
Another way of looking at it
DOG Noun
d o g
The phonological (or graphemic) segments
DOG Noun
d o g
What about these segments? Are they objects?
The phonological (or graphemic) segments
d o g
What about these segments? Are they objects?
/d/ as a phoneme has components: Articulation: closure of mouth (so also /g/) Position: Tongue tip against alveolar ridge Voiced (compare /t/) (Also auditory components: more complex)
A closer look at the segments
b
boy
y
Phonologicalfeatures
o The phonological segments also are just locations in the network – not objects
(Bob) (toy)
Relations all the way
Perhaps all of linguistic structure is relational It’s not relationships among linguistic items; it
is relations to other relations to other relations, all the way to the top – at one end – and to the bottom – at the other
In that case the linguistic system is a network of interconnected nodes
What is at the bottom?
In the system of the speaker, we have relational network structure all the way down to the points at which muscles of the speech-producing mechanism are activated• At that interface we leave the purely relational
system and send activation to a different kind of physical system
For the hearer, the bottom is the cochlea, which receives activation from the sound waves of the speech hitting the ear
What is at the top?
Somehow at the top there must be meaning
What are meanings?
DOGC
Perceptual
properties
of dogsAll those dogs
out there and
their properties
In the Mind
The World Outside
For example, DOG
What are meanings?
Perceptual
properties
of dogsAll those dogs
out there and
their properties
In the Mind
The World Outside
For example, DOG
Conceptual
properties
of dogs
What are meanings?
Perceptual
properties
of dogsAll those dogs
out there and
their properties
In the Mind
The World Outside
For example, DOG
Conceptual
properties
of dogs
Also relational network structure
The concept DOG
We know what a dog looks like• A visual subnetwork, in occipital lobe
We know what its bark sounds like• An auditory subnetwork, in temporal lobe
We know what its fur feels like• A somatosensory subnetwork, in parietal lobe
All of the above..• constitute perceptual information• are subnetworks with many nodes each• Are interconnected into a larger network
59
The concept of DOG as a network
V
PA
M
T
C
A – Auditory C – ConceptualM – MemoriesP – PhonologicalT – TactileV - Visual
Each node in this diagram connects to a subnetwork of properties
60
Objects in the mind?
When the relationships are fully identified, the objects as such disappear, as they have no existence apart from those relationships
“The postulation of objects as some- thing different from the terms of relationships is a superfluous axiom and consequently a metaphysical hypothesis from which linguistic science will have to be freed.”
Louis Hjelmslev (1943/61)
How the mind operatesExample: language
People are able to use their languages• Speaking• Writing• Comprehension
Such operation takes the form of activation of lines and nodes• Activation travels from node to node• Along connecting lines
Compare the highway system• A network• Operation: vehicles move along the roads
Two different network notations
Narrow notation
ab
a b
b
a b
Abstract notation Bidirectional
ab
a b f
Upward Downward
Narrow relational network notation
Represents network structures in greater detail internal structures of the lines and nodes of the more
abstract notation Closer to neurological structure Each node represents a bundle of neurons Links represent neural fibers (or bundles of fibers)
Abstract and narrow network notation
The lines and nodes of the abstract notation are abbreviations for more complex structures
Compare the representation of a divided highway on a highway map• In a more abstract notation it is shown
as a single line• In a narrow notation it is shown as two
parallel lines of opposite direction
AND vs. OR
AND
OR
twenty seven
27
12
twelve dozen
AND vs. OR: Internal Structure (narrow notation)
AND
OR
2
1
Thresholds in Narrow Notation
1 2 3 4
OR AND
– You can have intermediate degrees, between AND and OR
– The AND/OR distinction was a simplification anyway — doesn’t always work!
Levels of precision: Add another
Abstract relational network notation Narrow relational network notation Neural structures
Narrow RN notation and neural structures
Question: Are relational networks related in any way to neural networks?
We can find out Relational networks were devised as a means
of accounting for linguistic structure• Their properties depend on properties of language• Evidence for them comes from language, not from
the brain Narrow RN notation can be viewed as a set of
hypotheses about brain structure and function• Properties of narrow RN notation can be tested for
neurological plausibility
Some properties of narrow RN notation
Lines have direction (they are one-way)
But they tend to come in pairs of opposite direction (“upward” and “downward”)
Connections are either excitatory or inhibitory
Nerve fibers carry activation in just one direction
Cortico-cortical connections are generally reciprocal
Connections are either excitatory or inhibitory (from different types of neurons, with two different neurotransmitters)
More properties as hypotheses
Nodes have differing thresholds of activation
Inhibitory connections are of two kinds
Additional properties – (too technical for this presentation)
Neurons have different thresholds of activation
Inhibitory connections are of two kinds • (Type 2: “axo-axonal”)
All are verified
Type 1
Type 2
The node of narrow RN notationvis-à-vis neural structures
The node (of narrow RN notation) corresponds (not to a single neuron but) to a bundle of neurons• The cortical column• A column consists of 70-100 neurons stacked
on top of one another• More on this next week!
T h a t ‘ s i t f o r t o d a y !
74