a well grounded education: the role of perception in...

A Well Grounded Education 1

A Well Grounded Education: The role of perception in

science and mathematics

Robert Goldstone

David Landy

Ji Y. Son

Draft prepared for the Symbols, Embodiment, and Meaning Debate, held

December 16-18, 2005

Draft prepared 3/28/07

Indiana University

Psychology Department, Program in Cognitive Science

Running head: A Well Grounded Education


Abstract

One of the most important applications of grounded cognition theories is to science and mathematics education

where the primary goal is to foster knowledge and skills that are widely transportable to new situations. This

presents a challenge to those grounded cognition theories that tightly tie knowledge to the specifics of a single

situation. In this chapter, we develop a theory learning that is grounded in perception and interaction, yet also

supports transferable knowledge. A first series of studies explores the transfer of complex systems principles

across two superficially dissimilar scenarios. The results indicate that students most effectively show transfer by

applying previously learned perceptual and interpretational processes to new situations. A second series shows

that even when students are solving formal algebra problems, they are greatly influenced by non-symbolic,

perceptual grouping factors. We interpret both results as showing that high-level cognition that might seem to

involve purely symbolic reasoning is actually driven by perceptual processes. The educational implication is that

instruction in science and mathematics should involve not only teaching abstract rules and equations but also

training students to perceive and interact with their world.


Scientific progress has been progressing at a dizzying pace. In contrast, natural human biology changes rather

sluggishly, and we are using the essentially the same kinds of brains to understand advanced modern science that

have been used for millennia. Further exacerbating this tension between the different rates of scientific and

neuro-evolutionary progress is that our techniques for teaching mathematics and science are not keeping up with

the pace of science (Bialek & Botstein, 2004). Politicians, media, and pundits have all expressed frustration with

the poor state of mathematics and science education in the United States and world-wide.

There will not be any easy or singular solution to the problem of how to improve mathematics and science

education. However, we believe that the consequences of improved science and mathematics education are

sufficiently important1 that it behooves cognitive scientists to apply their state-of-the-science techniques and

results to informing the discourse on educational reform. Cognitive scientists are in a uniquely qualified position

to provide expert suggestions on knowledge representation, learning, problem solving, and symbolic reasoning.

These topics are core to understanding how people utilize mathematical and scientific principles. We also

believe that the recent developments in embodied and grounded cognition have direct relevance to mathematics

and science education, offering a promising new perspective on what we should be teaching and how students

could be learning.

We will describe two separate lines of research on college students’ performance on scientific and

mathematical reasoning tasks. The first research line studies how students transfer scientific principles

governing complex systems across superficially dissimilar domains. The second line studies how people solve

algebra problems. Consistent with an embodied perspective on cognition, both lines show strong influences of

perception on cognitive acts that are often associated with amodal, symbolic thought - namely cross-domain

transfer and mathematical manipulation,

1 Eric Hunushek, a senior fellow at the Hoover Institution at Stanford University, estimates that improving U.S. math and science to the levels of Western Europe within a decade would increase our gross domestic product by 4% in 2025 and by 10% by 2035.


Transfer of Complex Systems Principles

Scientific understanding frequently involves comprehending a system at an abstract rather than superficial

level. Biology teachers want their students to understand the genetic mechanisms underlying heredity, not

simply how pea plants look. Physics teachers want their students to understand fundamental laws of physics

such as conservation of energy, not simply how a particular spring uncoils when weighted down (Chi, Feltovich,

& Glaser, 1981). This focus on acquiring abstract principles is well justified. Science often progresses when

researchers find deep principles shared by superficially dissimilar phenomena and can describe situations in

terms of mathematical or formal abstractions. Finding biological laws that govern the appearance of both snails

and humans (Darwin, 1859), physical laws that govern both electromagnetic and gravitational acceleration

(Einstein, 1989), and psychological laws that underlie transfer of learning across species and stimuli (Shepard,

1987) are undeniably important enterprises.

Although transcending superficial appearances to extract deep principles has inherent value, it has proven

difficult to achieve (Carreher & Schliemann, 2002). Considerable research suggests that learners in many

domains do not spontaneously transfer what they have learned, at least not to superficially dissimilar domains

(Detterman, 1993; Gick & Holyoak, 1980; 1983). This lack of transfer based on shared deep principles has led

to a major theoretical position in the learning sciences called “situated learning.” This community argues that

learning takes place in specific contexts, and these contexts are essential to what is learned (Lave, 1988; Lave &

Wenger, 1991). Traditional models of transfer are criticized as treating knowledge as a static property of an

individual (Hatano & Greeno, 1999), rather than as contextualized or situated, both in a real-world environment

and a social community. According to situated learning theorists, one problem of traditional theorizing is that

knowledge is viewed as tools for thinking that can be transported from one situation to another because they are

independent of the situation in which they are used. In fact, a person’s performance on formal tasks is often

worse than their performance in more familiar contexts even though by some analyses, the same abstract tools

are required (Nunes, 1999; Wason & Shapiro, 1971).


Providing a basis for the transfer of principles across domains is a challenge for embodied cognition

approaches. Simply put, if cognition is tied to perceiving and interacting with particular scenarios, then how can

we hope to have transfer from one scenario to another scenario that looks quite different (for an elaboration of

this question, see Sanford, this volume)? One response, given by several researchers in the situated learning

community, is to give up on the prospects of transfer. The very notion of transfer is suspect in that community

because of their focus on contextualized knowledge. In one often-reported study, Brazilian children who sell

candy may be quite competent at using currency even though they have considerable difficulty solving word

problems requiring calculations similar to the ones they use on the street (Nunes, 1999). Transfer of

mathematical knowledge from candy selling on the streets to formal algebra in the classroom is neither found nor

expected by situated learning theorists. So in order to understand success in the classroom, situated learning

theorists place the emphasis on the context of the classroom and to understand success in the streets, the street

context is studied.

We share with the situated learning community an emphasis on grounded knowledge. However, we mean

something quite different by “grounding.” For the situated learning community, knowledge is contextualized in

the actions, social goals, and physical details of particular concrete scenarios such as selling candy on a Brazilian

street. Situated learning theorists equate knowledge with problem solving activities that are cued, in some sense

“grounded”, by the features of these concrete scenarios. Generalization is thus a problem solving behavior

exhibited over a set of scenarios (Greeno, 1997). Typically situated learning theorists criticize cognitive theorists

for abstracting away from these scenarios to explain generalization. However, for us, knowledge is not simply in

the extracted verbal or formal description of a situation, but rather in the perceptual interpretations and motoric

interactions involving a concrete scenario. While we look to embodied experiences to ground learning, we still

believe in the possibility and power of transfer across contexts. It is possible to learn principles in a grounded

way that enables the principles to be recognized in the myriad of concrete forms that they can take.

Typically, cognitive theories of transfer are expressed in terms of the acquisition of abstracted formalisms

that lead to the direct perception of mathematical structures and patterns. Instead, we believe that learning that


transfers to new scenarios and transports across domains most often proceeds not through acquiring and applying

symbolic formalisms, but rather through modifying automatically perceived similarities between scenarios by

training one’s perceptual interpretations. In the following two sections we will 1) argue for the desirability of

cross-domain transfer of scientific principles, and 2) show how this transfer is compatible with a perceptually

grounded understanding of science.

Complex Systems and Transfer

Our claim for the desirability and possibility of transfer of scientific principles across domains is based on

the power of complex systems theories. A complex systems perspective provides a unifying force, bringing

increasingly fragmented scientific communities. Journals, conferences, and academic departmental structures are

becoming increasingly specialized and myopic (Csermely, 1999). One possible response to this fragmentation of

science is to simply view it as inevitable. Horgan (1996) argues that the age of fundamental scientific theorizing

and discoveries has passed, and that all that is left to be done is refining the details of theories already laid down

by the likes of Einstein, Darwin, and Newton.

Complex systems researchers offer an alternative to increasing specialization. They have pursued

principles that apply to many scientific domains, from physics to biology to social sciences. For example,

reaction-diffusion equations that explain how cheetahs develop spots can be used to account for geospatial

patterns of Democrats and Republicans in America. The same abstract schema underlies both phenomena – two

kinds of elements (e.g. two skin cell colors, or two political parties) both diffuse outwards to neighboring regions

but also inhibit one another. The process of Diffusion-Limited Aggregation is another complex adaptive systems

explanation that unifies diverse phenomena: individual elements enter a system at different points, moving

randomly. If a moving element touches another element, they become attached. The emergent result is fractally

connected branching aggregates that have almost identical statistical properties (Ball, 1999). This process has

been implicated in the growth of human lungs (Garcia-Ruiz et al., 1993), snowflakes (Bentley & Humphreys,

1962), and cities (Batty, 2005). These examples all describe principles that can be instantiated with highly


dissimilar sets of individual elements, but with interactions between the elements that are captured by very

similar algorithmic rule sets (Bar-Yam, 1997).

Generally speaking, complex systems are systems made up of many units (often times called agents),

whose simple interactions give rise to higher-order emergent behavior. Typically, the units all obey the same

simple rules but because they interact, the units that start off homogenous and undifferentiated may become

specialized and individualized (O’Reilly, 2001). Despite the lack of a centralized control, leader, recipe, or

instruction set, these systems naturally self-organize (Resnick, 1994; Resnick & Wilensky, 1993). Many real-

world phenomena can be explained by the formalisms of complex adaptive systems, including the foraging

behavior of ants, the development of the human nervous system, the growth of cities, growth in the world wide

web, the perception of apparent motion, mammalian skin patterns, pine cone seed configurations, and the shape

of shells (Casti, 1994; Flake, 1998).

In the following studies, we will focus on one particular complex systems principle, “Competitive

Specialization,” because we have taught this principle in our own undergraduate “Complex Adaptive Systems”

courses, and because we have conducted controlled laboratory studies on students’ appreciation and use of the

principle (Goldstone & Sakamoto, 2003; Goldstone & Son, 2005). A well-worked out example of Competitive

Specialization is the development of neurons in the primary visual cortex that start off homogenous and become

specialized to respond to visually presented lines with specific spatial orientations (von der Malsburg, 1973).

Another application is the optimal allocation of agents to specialize to different regions in order to cover a

territory. In these situations, a good solution is found if every region has an agent reasonably close to it. For

example, an oil company may desire to place oil drills such that they are well spaced and cover their territory. If

the oil drills are too close, they will redundantly access the same oil deposit. If the oil drills do not cover the

entire territory, then some oil reserves will not be used.


Ants and Food. The first example of competitive specialization involves ants foraging food resources

drawn by a user. The ants follow exactly the following three rules of competitive specialization. At each time

step, 1) a piece (pixel) of food is randomly selected from all of the food drawn by a user, 2) the ant closest to the

piece moves with one rate, and 3) all of the other ants move with another rate. In interacting with the simulation,

a learner can reset the ants' positions, clear the screen of food, draw new food, place new ants, move ants,

start/stop the ants' movements, and set a number of simulation parameters. The two most critical user-controlled

parameters determine the movement speed for the ant that is closest to the selected food (called "closest rate" in

Figure 1) and the movement speed for all other ants ("Not closest rate"). Starting with the initial configuration of

three ants and three food piles shown in Figure 1, several important types of final configuration are possible and

are shown in Figure 3. If only the closest ant moves toward a selected piece of food, then this ant will be the

closest ant to every patch of food. This ant will continually move to new locations on every time step as

I n fo Reset Adju st Qu iz

P a in t E ra se P la ce Move

C le a rGraph Updating

Number of ants Pen Size

Closest Rate Not Closest Rate

Time

Dis

tance

of

Food

Patc

h

to

Clo

sest

Ant

Figure 1. A screen-dump of an initial configuration for the "Ants and food" computer simulation. At

each time step, a patch of food is randomly selected, and the ant closest to the patch moves toward the

patch with one speed (specified by the slider "Closest Rate") and the other ants move toward the patch

with another speed ("Not Closest Rate").


different patches are sampled, but will tend to hover around the center-of-mass of the food patches. The other

two ants will never move at all because they will never be the closest ant to a food patch. This configuration is

sub-optimal because the average distance between a food patch and the closest ant (a quantity that is continually

graphed) is not as small as it would be if each of the ants specialized for a different food pile. On the other hand,

if all of the ants move equally quickly, then they will quickly converge to the same screen location. This also

results in a sub-optimal solution because the ants do not cover the entire set of resources well – there will be

patches that do no have any nearby ant. Finally, if the closest ant moves more quickly than the other ants but the

other ants move too, then a nearly optimal configuration is achieved. Although one ant will initially move more

quickly toward all selected food patches than the other ants, eventually it will specialize to one patch and the

other ants will then be the closer to the other patches allowing for specialization.

An important, subtle aspect of this simulation is that poor patterns of resource covering are self-correcting

so the ants will almost always self-organize themselves in a 1-to-1 relationship to the resources regardless of the

lopsidedness of their original arrangement if good parameter values are used.


Pattern Learning. The second example of competitive specialization involves sensors responding to

patterns drawn by the user. Just like the Ants/Food scenario, the Pattern Learning simulation also follows the

three rules of competitive specialization. At the beginning of the simulation, the sensors respond to random

noise. But at each time step, a pattern is randomly selected from all of the patterns drawn by a user, and the

sensor most similar to that pattern adapts to become more similar to that pattern at a particular rate. All of the

other sensors adapt towards the selected pattern at another rate. Users can reset sensors to become random again,

draw patterns, erase patterns, copy patterns, add noise, start/stop pattern learning, and change a number of

parameters. The most important parameters are the rates of adaptation for the most similar and not most similar

Figure 2. A screen-dump for the simulation "Pattern Learning." Users draw pictures, and prior to learning, a set of categories are given random appearances. During learning, a picture is selected at random, and the most similar category to the picture adapts its appearance toward the picture at one rate (specified by the slider "Most similar") while the other categories adapt toward the picture at another rate ("Not most similar").


sensors.

Using these two case studies of the “competitive specialization” principle, we now in a good position to

state our challenge for grounded educational practices. Our desiderata is a teaching method that will promote

transfer from one example of competitive specialization to another. How can students who learn about

competitive specialization with the ants and food scenario spontaneously apply what they have learned to the

pattern learning situation? Given the lack of superficial perceptual features shared by Figures 1 and 2, can the

transfer be due to cognitive processes grounded in perception and interaction? The next section sketches out our

affirmative answer.

How to Teach Complex Systems Principles

Powerful complex systems principles cut across scientific disciplines. We are therefore inherently

interested in transportable knowledge - knowledge that can be applied to domains significantly beyond those

originally presented. In setting the stage for our grounded approach to cognition, we will first describe a

traditional alternative approach to transfer.


Transfer Via Formalisms. It is understandable why so many educators have been drawn to couch their textbook

and classroom teaching in terms of formalisms. The formalizations established by algebra, set theory, and logic

are powerful because they are domain-general. The same equation for probability can apply equally well to

shoes, ships, sealing-wax, cabbages, and kings. Not only is no “customization” of equations to a content domain

required – it is forbidden. The formalism sanctions certain transformations that are provably valid, and once a

domain has been translated to the formalism, one can be assured that the deductions drawn from the

transformations will be valid. Logic and math are the best examples that we can think of for the kind of

symbolic processing that Glenberg, De Vega, & Graesser (this volume) have in mind when they describe

symbols as abstract, amodal, and manipulated by using explicit rules. This is not to imply that we believe that

engaging in mathematics is a purely symbolic activity. Crucially, we do not (see the last section of this chapter),

but it is the best candidate domain for symbolic reasoning, because the rules for mathematics are relatively

noncontentious, compared to, for example, the rules for language. Furthermore, students receive a significant

Figure 3. The basis for the isomorphism between the ants and Pattern Learning simulations. If only the most similar agent to a resource adapts, then often a single agent will move toward the average of all of the resources. If all agents adapt equally quickly, then they will all move toward the average position. If the agent closest to a resource patch moves much faster than the other agents but all agents move a bit, then each of the agents typically becomes specialized for one resource type.


amount of training with the generic, abstract form of mathematical rules, and explicitly learn how to transform

symbolic representations.

Mathematical formalisms are the epitome of representations that abstract over domains. Given this, there

is some reason to believe that they will promote free transfer of knowledge from one domain to another. This

notion is endemic in high-school mathematics curricula, which often feature abstract formalisms that, once

presented, are only subsequently fleshed out by examples. Systematic analyses of mathematics textbooks have

shown that formalisms tend to be presented before worked-out examples, and that this tendency increases with

grade level (Nathan, Long, & Alibali, 2002).

Although formalisms may seem plausible candidates for producing transportable knowledge, the literatures

from cognitive science and education give three grounds for skepticism. First, the mismatch between how

mathematics is presented and how it is conceived by practitioners has been often noted (Lakoff & Nuñez, 2000;

Thurston, 1994; Wilensky, 1991). As teachers’ mathematical expertise increases, they increasingly believe that

using formalisms is a requirement for solving mathematical problems, even though this kind of strong

association is not found in actual students’ performance (Nathan & Petrosino, 2003). Hadamard has complained

that the true heart of a mathematical proof, the intuitive conceptualization, is ignored in the formal description of

the proof steps themselves (Hadamard, 1949). The scholarly articles contain the step-by-step, formally

sanctioned steps, but if one wishes to understand where the idea for these steps comes from, then one must

attempt to generate the visuo-spatial inspiration oneself, without much insight from the published report. In

mathematics textbooks, formalisms are either treated as givens, or when they are derived, they tend to be

formally derived from other formalisms.

The second reason for skepticism is that formalism-based transfer can only work if people spontaneously

recognize when an equation can be applied to a situation. In fact, the connection between equations and

scenarios is typically indirect and difficult to see (Chi, 2005; Hmelo-Silver & Pfeffer, 2004; Nathan, Kintsch, &

Young, 1992; Penner, 2000). Students often have difficulty finding the right equation to fit a scenario even when

they know both the equation and the major elements of the scenario (Ross, 1987; 1989).


The third reason for skepticism is that transfer potential is almost certainly not maximized by creating the

most efficient minimalist representation possible, one that leaves out irrelevant content information and captures

only relevant structural information. Formalisms are maximally content-independent, but this is the precise

property that often leads them to be cognitively inert. They offer little by way of scaffolding for understanding,

and may not generalize well because cues to resemblance between situations have been stripped away. Research

has shown that the content-dependent semantics of a scenario promotes transfer, and also gives valuable clues

about the kinds of formalism that are likely to be useful (Bassok, 1996, 2001; Bassok, Chase, & Martin, 1998).

Grounded Transfer. We would like to propose an alternative, intermediate position, to the extreme

positions that transfer should proceed via formalisms and that remote transfer cannot be achieved at all

(Detterman, 1993). Our position is that transfer can be achieved, not by teaching students abstract formalisms,

but through the interpretation of grounded situations. One motivation for this method comes from work with

students solving story problems with and without access to physical manipulatives that can be used to act out the

story (Glenberg, Gutierrez, Levin, Japuntich, & Kaschak, 2004). The physical manipulatives helped students

understand and solve the problems, and these benefits transferred to conditions in which students simply

imagined using the physical objects (Glenberg, Jaworski, Rischall, & Levin, in press). The process of

interpreting actual or imagined objects was instrumental in getting students to properly understand the

underlying mathematical issues involved in a situation.

Another motivation for this method comes from our observation of students learning principles of complex

systems in our classes and laboratory experiments. We have observed that students often interact with our

pedagogical simulations by actively interpreting the elements and their interactions. Their interpretations are

grounded in the particular simulation with which they are interacting. Furrther, because the interpretations may

be highly selective, perspectival, and idealized, the same interpretation can be given to two apparently dissimilar

situations. The process of interpreting physical situations can thus provide understandings that are grounded yet

transportable. Practicing what we preach, we now will ground our notion of interpretive generalization with the


earlier described “Competitive Specialization” principle example from our “Complex Adaptive Systems” course.

The relevant simulations can be accessed at http://cognitrn.psych.indiana.edu/rgoldsto/complex/.

Our laboratory and classroom investigations with these two demonstrations of competitive specialization

have shown that students can, under some circumstances, transfer what they learn from one simulation to another

(Goldstone & Sakamoto, 2003; Goldstone & Son, 2005). In our experiments, we first give students a period of

focused exploration with the “Ants and food” simulation because it embodies competitive specialization in a

relatively literal and spatial manner. Then, we let students explore the “Pattern Learning” simulation. We probe

their understanding of the latter simulation through both multiple choice questions designed to measure their

appreciation of the principle of competitive specialization in the pattern learning context, as well as a

performance-based measure of how quickly students can create parameter settings whereby categories

automatically adapt so as to represent the major classes of input patterns.

Using this method, we find that students show better understanding of the Pattern Learning simulation

when it has been preceded by Ants and Food than by a simulation governed by a different principle. Student

interviews indicate that a major cause of the positive transfer is training perceptual interpretations of grounded

Figure 4. A standard visualization of competitive specialization, adapted from an illustration by

Rumelhart and Zipser (1985). Panel A shows the coordinates of seven objects, represented by Xs, in a

three-dimensional space. In Panel B, the starting, random coordinates for two categories are shown by

shaded circles. Panel C shows the resulting coordinates for the categories after several iterations of

learning by competitive specialization.

XX

X

X

X X

X

XX

X

X

X X

X

XX

X

X

X X

X

A B C


situations. After exposure to Ants and Food, several students spontaneously applied the same perceptual

representation to Pattern Learning. The visuo-spatial dynamics of Ants and Food are aptly applied. In fact,

when originally presenting their general competitive learning algorithm, Rumelhart and Zipser use a

visualization along the lines of Ants and Food to give the reader a solid intuition for how their algorithm works.

This visualization, shown in Figure 4, depicts the adaptation of categories in a high-dimensional space as the

movement of those categories in three-dimensional space. Our students’ spontaneous visualizations have

elements in common with this professional visualization. Typical elements in our students’ visualizations are

shown in Figure 5. A student was asked to visually describe what would happen when there are two categories,

and four input pictures that fell into two clusters: variants of As and variants of Bs. The student drew the

illustration in Panel A. In this illustration, adaptation and similarity are both represented in terms of space, much

as they are in Figure 4. The two categories (“Cat1” and “Cat2”) are depicted as moving spatially toward the

spatially defined clusters of “A”s and “B”s, and the “A” and “B” pictures are represented as spatially separated.

The context for Panel B was a student who was asked what problem might occur if the most similar detector to a

selected picture moved quickly to the picture, while the other detectors did not move at all. The student showed

a single category (shown as the box with a “1”) moving toward, and eventually oscillating between, the two

pictures. Again, similarity is represented by proximity and adaptation by movement.


Students’ verbal descriptions also give evidence of the spatial diagrams that they use to explicate Pattern

Learning. Students frequently talk about a category “moving over to a clump of similar pictures.” Another

student says that “this category is being pulled in two directions – toward each of these pictures.” A third

student also uses spatial language when saying, “These two squares are close to each other, so they will tend to

attract the same category to them.” In this final example, the two square pictures were not physically close to

each other on the screen – they were separated by a picture of a circle. All three of these reports show that

students are understanding visual similarity in terms of spatial proximity. The squares are close in terms of their

visual appearance, not their spatial distance. Categories do not change their position on the screen, but only in a

high-dimensional space that describes pictures. Ants move in visual space, while categories adapt in description

space. In doing so, both ants and categories are adapting so that they cover their resources (food or pictures)

well. Our students frequently find making the connections between adaptation and motion, and between

similarity and proximity to be natural, and much more so after they have had experience with Ants and Food.

How should we understand these connections? One common approach is to claim that Ants and Food uses

space literally, while students’ understanding of Pattern Learning treats space metaphorically or figuratively.

A B Figure 5. Typical visualizations of students expressing their knowledge of Pattern Learning. In Panel A, the student represents the adaptation of categories by moving them through space towards two clusters of stimuli. The similarity of the two variants of “A” is represented by their spatial proximity. In Panel B, a student was asked to illustrate a problem that arises when the closest category to a pattern adapts, but the others do not adapt at all: a single category is shown oscillating between two patterns. Note the similarity to the top panel of Figure 3 with the Ants and Food simulation.


Movement in space is viewed as an apt analogy for adaptation. The problem with describing space as figurative

in Pattern Learning is that space is very concrete and grounded for students who draw diagrams like those shown

in Figure 5. Our alternative proposal is that students are using the literal, spatial models that they learned while

exploring Ants and Food to understand and predict behavior in Pattern Learning. There is a process where visual

similarity is thought of in terms of spatial proximity, but once that mapping has been made, students conduct the

same kinds of mental simulations that they perform when predicting what will happen in new Ants and Food

situations. So, we do not believe that students abstract a formal structural description that unifies the two

simulations. Instead, they simply apply to a new domain the same perceptual routines that they have previously

acquired. By one account, the domains are visually dissimilar. However, once construed appropriately, the

domains are highly similar visually. In fact, students seem to use superficial spatial properties to construe Ants

and Food. An interesting aspect of both illustrations in Figure 5 is that although space is being used to represent

similarity, there are still vestiges of space being used to represent space. In these, and other, student illustrations,

the categories are placed below the input pictures, just as they are in the interface shown in Figure 2. Even

though students have difficulty explicitly describing the general principle that governs both simulations, the

perceptual trace left by Ants and Food can prime an effective perceptual construal of Pattern Learning.

What is most striking about the students’ descriptions of Pattern Learning is the extent of knowledge-

driven perceptual interpretation. These interpretations are not simply formalisms grafted onto situations. Rather,

the interpretations affect the perception of the simulation elements. Students see single categories trying to

“cover” multiple pictures, pictures being “close” or “far” in terms of their appearances, and categories “moving”

toward pictures. To the experienced eye, identical perceptual configurations are visible in the two simulations.

Figure 3 shows equivalent configurations. The critical point is that these pairs are obviously not perceptually

similar under all construals. It is only to the student who has understood the principle of competitive

specialization as applied to Ants and Food that the two situations appear perceptually similar. One moral is: just

because a reminding is perceptually-based does not make it necessarily superficial. A superficial construal of the


left and right columns of Figure 3 would not reveal much commonality. However, a dynamic, visual, and spatial

construal of Pattern Learning can be formed that makes the solutions of Ants and Food directly relevant.

In opposition to the traditional schism between perceptual and conceptual processes, our work on

transferring complex systems principles suggests that the perceptual interpretation process is key to generating

transportable conceptual understandings. Perceptual interpretation requires both a physically present situation

and construals of the elements of the situation and their interactions. Simply giving the interpretation is not

adequate. Like equations, standalone descriptions are unlikely to foster transfer because of their lack of contact

to applicable situations. When we simply give students the rules of Pattern Learning, they are seldom reminded

of Ants and Food. Physically grounding a description is one of the most effective ways of assuring that it is

conceptually meaningful. However, giving the grounded situation but no interpretation is also inadequate. It is

only when the interpretation is added to the presentation of two superficially dissimilar situations that the

resemblance becomes apparent. Students are not just seeing events, they are seeing events as instantiating

principles. This act of interpretation, an act of “seeing something as X” rather than simply seeing it

(Wittgenstein, 1953), is the key to cultivating transportable knowledge.


Perceptual Learning. An important plank of our proposal is that the similarity between situations

governed by the same complex systems principle can be used to promote transfer even if the situations are

dissimilar to the untutored eye, and even if the similarity is not explicitly noticed. This claim apparently

contradicts the empirical evidence for very limited transfer between remote situations (Detterman, 1993; Reed,

Ernst, & Banerji, 1974, but see also Barnett & Ceci, 2002 for a balanced evaluation of the evidence). Our claim

is that the perceived similarity of situations is malleable, not fixed by objective properties of the situations

themselves. It may well be that remotely related situations rarely facilitate each other. However, well-designed

Figure 6. The design of an experiment how concrete visual simulations should be to promote transfer (Goldstone & Sakamoto, 2003). Better transfer performance was found for the idealized, relative to concrete, training condition, particularly for students with poor comprehension of the training simulation.


activities can alter the perceived similarity of situations, and what were once dissimilar situations can become

similar to one another with learning (Goldstone, 1998; Goldstone & Barsalou, 1998). Our hope, then, is not to

have students transfer by connecting remotely related situations, but rather to have students warp their

psychological space so that formerly remote situations become similar (Harnad, Hanson, & Lubin, 1995). There

is already good evidence for this kind of warping of perception due to experience and task requirements

(Goldstone, 1994; Goldstone, Lippa, & Shiffrin, 2001; Livingston, Andrews, & Harnad, 1998; Özgen, 2004;

Roberson, Davies, & Davidoff, 2000).

Graphical, interactive computer simulations (Goldstone & Son, 2005; Jacobson, 2001; Jacobson &

Wilensky 2005; Resnick, 1994; Wilensky & Reisman, 1999, 2006) offer attractive opportunities for promoting

generalization. Principles are not couched in equations, but rather in dynamic interactions among elements

(Nathan, this volume; Nathan et al., 1992). Students who interact with the simulations actively interpret the

resulting patterns, particularly if guided by goals abetted by knowledge of the principle. Their interpretations are

grounded in the particular simulation, but once a student has practiced building an interpretation, it is more likely

to be used for future situations. In contrast to explicit equation-based transfer, perceptually-based priming is

automatic. For example, an ambiguous man/rat drawing is automatically interpreted as a man when preceded by

a man and as a rat when preceded by a rat (Leeper, 1935). This phenomenon, replicated in countless subsequent

experiments on priming (see Goldstone, 2003 for a theoretical integration), is not ordinarily thought of as

transfer, but it is an example of a powerful influence on perception due to prior experiences. This kind of

automatic shift of perceptual interpretation accompanies engaged interaction with complex system simulations.

In these cases, generalization arises, not just from the explicit and effortful application of abstract formalisms,

but crticially from the simple act of “rigging up” a perceptual system to interpret a situation according to a

principle, and leaving this rigging in place for subsequently encountered situations.

Perception and Idealization. In arguing for an embodied basis for transfer complex systems principles, it is

important to clarify what we mean to entail by “embodied.” To us, embodiment is compatible with idealization.

Following Glenberg, De Vega, and Graesser (this volume), we consider our students’ understanding of complex


systems principles to be embodied when it depends on activity in systems used for perception and action. Both

our computer simulations, and the mental simulations of those simulations are perceptual in that they incorporate

spatial and temporal information, and presumably do so by using brain regions that are dedicated to perceptual

processing. Arnheim (1970), Barsalou (1999; Barsalou et al., 2003; Simmons & Barsalou, 2003), Glenberg (this

volume; Glenberg, Jaworski, & Rischall, in press), Schwartz (Schwartz & Black, 1999), Roy (this volume) and

others have argued that our concepts are not amodal and abstract symbolic representations, but rather are

grounded in the external world via our perceptual systems.

Embodied knowledge has a major advantage over amodal representations – they preserve aspects of the

external world in a direct manner so that the mental simulations and the simulated world automatically stay

coordinated even without explicit machinery to assure correspondence. Dimensions in the model naturally

correspond to dimensions of the modeled world. Given this characterization, it is clear that embodied

representations need not superficially resemble the modeled world nor preserve all of the raw, detailed

information of that world (Barsalou, 1999; Shepard, 1984). Our experience with students’ understanding of

complex systems computer simulations indicates that simulations lead to the best transfer when they are

relatively idealized. Goldstone and Sakamoto (2003) gave students experience with two simulations

exemplifying the principle of competitive specialization -- Ants and Food, followed by Pattern Learning. Figure

6 shows the design for the experiments. Ants and Food was either presented using line drawings of ants and

food, or simplified geometric forms. Overall, students showed greater transfer to the second scenario when the

elements were graphically idealized rather than realistic. Interestingly, the benefit of idealized graphical elements

was largest for our students who had relatively poor understanding of the initial simulation. It might be thought

that strong contextualization and realism would be of benefit to those students with weak comprehension of the

abstract principle. Instead, it seems that these poor comprehenders are particularly at risk for interpreting

situations at a superficial level, and using realistic elements only encourages this tendency. Smith (2003) has

found consistent results with rich versus simple geometric objects, and Sloutsky, Kaminski, and Heckler (2005)


have found similar benefits of idealized over concrete symbols for learning about algebraic groups (see also

Wiley, 2003 for a review of the benefits and hazards of concretization).

Judy DeLoache’s model room paradigm (DeLoache, 1991, 1995; DeLoache & Burns, 1994; DeLoache &

Marzolf, 1992) has also found that idealization can promote symbolic understanding. A child around the age of

2.5 years is shown a model of a room, the child watches as a miniature toy is hidden behind or under a miniature

item of furniture in the model, and the child is told that a larger version of the toy is hidden at the corresponding

piece of furniture in the room. Children were better able to use the model to find the toy in the actual room when

the model was a two-dimensional picture rather than a three-dimensional scale model (DeLoache, 1991;

DeLoache & Marzolf, 1992). Placing the scale model behind a window also allowed children to more effectively

use it as a model (Deloache, 2000). DeLoache and her colleagues (DeLoache, 1995; Uttal, Scudder, &

DeLoache, 1997; Uttal, Liu, & Deloache, 1999) explain these results in terms of the difficulty in understanding

an object as both a concrete, physical thing, and as a symbol standing for something else.

A natural question to ask is, “How can we tell whether a particular perceptual detail will be beneficial

because it provides grounding or detrimental because it distracts students from appreciating the underlying

principle?” The answer depends both on the nature of complex systems principles, the learner’s cognitive

development, and what is easily implemented in the mental simulations. Complex systems models are typically

characterized by simple, similarly configured elements that each follow the same rules of interaction. For this

reason, idiosyncratic element details can often be eliminated, and information about any element only needs be

included to the extent that it affects its interactions with other elements. Mental simulations are efficient at

representing spatial and temporal information, but are highly capacity-limited (Hegarty, 2004a). Under the

assumption that a student’s mental model will be shaped by the computational model that informs it, the

following prescriptions are suggested for building computer simulations of complex systems: 1) eliminate

irrelevant variation in elements’ appearances, 2) incorporate spatial-temporal properties, 3) do not incorporate

realism just because it is technologically possible, 4) strive to make the element interactions visually salient, and

5) be sensitive to peoples’ capacity limits in tracking several rich, multi-faceted objects. Another empirically


supported suggestion for compromising between grounded and idealized presentations is to begin with relatively

rich, detailed representations, and gradually idealize them over time (Goldstone & Son, 2005). This regime of

“Concreteness fading” was proposed as a promising pedagogical method because it allows simulation elements

to be both intuitively connected to their intended interpretations, but also eventually freed from their initial

context in a manner that promotes transfer.

Perceptual Grounding in Mathematics

In the previous section, we contrasted the benefits of presenting scientific principles using perceptually

grounded and interactive simulations, rather than traditional algebraic equations. One reason why complex

system computer simulations are so pedagogically effective is that they mesh well with human mental models

(Gentner & Stevens, 1983; Graesser & Jackson, this volume; Graesser & Olde, 2003; Hegarty, 2004b). These

computer models enact the same kind of step-by-step simulation of elements and interactions that effective

mental models do. However, even though we advocate the perceptual grounding provided by simulations for

pedagogical use, our position is not that algebraic equations are ungrounded or that they somehow transcend

perception. In fact, some of our recent experiments indicate surprisingly strong influences of apparently

superficial perceptual grouping factors on people’s algebraic reasoning.

To study the influence of perceptual grouping on mathematics, we gave undergraduate participants a task

to judge whether an algebraic equality was necessarily true (Landy & Goldstone, under review). The equalities

were designed to test their ability to apply the order of precedence of operations rules. Our participants have

learned the rule that multiplication precedes addition. Our instructions and post-experiment interviews confirm

that participants know this rule. However, we were interested in whether perceptual and form-based groupings

would be able to override their general knowledge of the order of precedence rules. We tested this by having

grouping factors either consistent or inconsistent with order of precedence. For example, if shown the stimuli in

the top row of Figure 7, participants would be asked to judge whether R * E + L * W is necessarily equal to L *

W + R * E. These terms are necessarily equal, and so the correct response would be “Yes” for this trial. On

some trials, the physical spacing between the operators was consistent with the order of precedence – large


spacings separated the terms related by addition and small spacings separated terms related by multiplication. A

powerful principle of Gestalt perception (Koffka, 1935) is that close objects are seen as grouping together.

Accordingly, small spaces between terms to be multiplied together would be expected to facilitate grouping them

together early, consistent with the order of precedence. On other trials, the physical spacing was inconsistent

with the order of precedence, with larger spaces around “*”s than “+”s.


The influence of spacing was profound, and is shown in Figure 8. When physical spacing was inconsistent

with order of precedence rules, six times as many errors were made relative to when the spacing was consistent.

However, for problem types where the order of operations did not influence the validity of the equation

(insensitive trials) accuracy was relatively spared, indicating that the deployment of order of operations

knowledge was selectively affected by the grouping pressures. Several aspects of the experiment make this

influence of perception on algebra striking. First, they demonstrate a genuine cognitive illusion in the domain of

mathematics. The criteria for cognitive illusions in reasoning are that people systematically show an influence of

Figure 7. Samples from five experiments reported by Landy and Goldstone (under review). Participants were asked to verify whether an equation is necessarily true. Grouping suggested by factors such as physical spacing, regions suggested by geometric forms, proximity in the alphabet, and functional form similarity could be either consistent or inconsistent with the order of precedence of arithmetical operators (e.g. multiplications are calculated before additions). The above equalities are all true, but participants make far more errors when the perceptual and form-based groupings are inconsistent rather than consistent.

Grouping consistent with order

of precedence rulesGrouping inconsistent with

order of precedence rules

R * E + L * W = L * W + R * E?

R * E + L * W = L * W + R * E?

Spacing

R * E + L * W = L * W + R * E? ?

Grouping by Connectedness

R * E + L * W = L * W + R * E

R * E + L * W = L * W + R * E? ?

Grouping by Common Region

R * E + L * W = L * W + R * E

(9*o)*(c*c)+(f*f)*(8*k)=(f*f)*(8*k)+(9*o)*(c*c)?

(f*f)*(c*c)+(9*o)*(8*k)=(9*o)*(8*k)+(f*f)*(c*c)?

Functional Form Similarity

A * B + X * Y = X * Y + A * B? ?

Alphabetic Proximity

A * X + Y * B = Y * B + A * X


a factor in reasoning, the factor should normatively not be used, and people agree, when debriefed, that they

were wrong to use the factor (Tversky & Kahneman, 1974). The second impressive aspect of the results is that

participants continued to show large influences of grouping on equation verification even though they received

trial-by-trial feedback. Constant feedback did not eliminate the influence of the perceptual cues. This suggests

that sensitivity to grouping is automatic or at least resistant to strategic, feedback-dependent control processes.

The third impressive aspect of the results is that an influence of grouping is found in mathematical reasoning.

Mathematical reasoning is often taken as a paradigmatic case of purely symbolic reasoning, moreso even than

language which, in its spoken form, is produced and comprehended before children have formal operations

(Inhelder & Piaget, 1958). Algebra is, according to many people’s intuition, the clearest case of widespread

symbolic reasoning in all human cognition. Showing that perceptual factors influence even algebraic reasoning

provides prime facie support for the premise that grounding cannot be ignored for any cognitive task.


Our further experiments have extended this observed influence of grouping factors on algebraic reasoning.

The second and third rows of Figure 7 show manipulations of other Gestalt law of perceptual organization.

According to the principle of connectedness (Palmer & Rock, 1994), objects that are physically connected to one

another have a tendency to be grouped together. Connecting the circles in Row 2 of Figure 7 causes them to be

grouped together, and experimental results show that when they are grouped together, they tend to group the

mathematical symbols with which they are vertically aligned. The third row shows that this grouping pressure

does not require physical connection, but rather can also be formed through implied common region. In both of

Figure 8. Results from a manipulation of physical spacing in Experiment 1 of Landy and Goldstone (under review). When spacing was inconsistent (right side of Figure 7) with the order of precedence, the error rate increased more than six-fold compared to when spacing was consistent. This effect was only found when equations were sensitive to physical spacing – trials where different consistent versus inconsistent answers would be given to the question, “Are the two sides of this equation necessarily equal?” if a participant were influenced by spacing.


these examples, even though participants know that the geometric forms are irrelevant to their mathematical task,

when the geometric forms are consistent with order of operations, far better algebraic performance results than if

the forms are inconsistent. The fourth row of Figure 7 shows that alphabetic proximity has an analogous effect

to physical proximity. Participants apparently have a tendency to group letters that are close in the

alphabet. If close letters, like “A” and “B,” are related by an addition operator, this hinders algebraic

performance; if the close letters are related by a multiplication operator, this facilitates performance. This

influence of alphabetic proximity on mathematical reasoning is surprising because by standard formal accounts

all variable names are equivalent and participants, in their explicit description of the task, endorse the statement

that the letters used to designate variables are irrelevant. Finally, the last row shows an influence of the

functional form of the terms being multiplied and added. The term “(c * c)” has a form-based similarity to “(f *

f)” – they both involve a variable being multiplied by itself. This similarity is sufficient to bias participants to

group these terms together. Again, this form-based group helps performance if the terms are related by

multiplication, but hinders performance if they are related by addition.

The cumulative weight of these results indicates that even the highly symbolic activity of algebraic

calculation is strongly affected by perceptual grouping. Most accounts of the importance of notation in

mathematics, and indeed of symbols generally, hinge on their abstract character. Since symbols are largely

considered amodal, symbolic reasoning is also easily assumed to be amodal; that is, symbolic reasoning is

supposed to depend on internal structural rules which do not relate to explicit external forms. In contrast, the

mathematical groupings that our participants create are heavily influenced by groupings based on perception and

superficial similarities. The theoretical upshot of this work is to question the assumption that cognition operates

generally like systems of algebra or mathematical logic. By traditional symbolic manipulation accounts, to

cognize is to apply laws to structured strings, where those laws generalize to the shape of the symbols, and that

shape is arbitrarily related to the symbols’ content (Fodor, 1992). The laws of algebra work just like this.

Multiplication is commutative no matter what terms are multiplied. Mathematical cognition could have worked

like this too. If it had, it would have provided a convenient explanation of how people perform algebra


(Anderson, 2005). However, the conclusion from our experiments is that seeing “X + Y * A” cannot trivially

be translated into the amodal mentalese code * ( + (G001 , G002), G003). The physical spacing, superficial

similarities between the letters, and background pictorial elements are all part of how people treat “X * A + Z.”

Although it would be awfully convenient if a computational model of algebraic reasoning could aptly assume

the transduction from visual symbols to mental symbols, taking this for granted would leave all of our current

experimental results a mystery. We must resist the temptation to posit mental representations with forms that

match our intellectualized understanding of mathematics. A more apt input representation to give our future

computational models would be a bitmap graphic of the screen that includes the visually presented symbols as

well as their absolute positions, spacings, sizes, and accompanying non-mathematical pictorial elements, as well

as the associations carried by particular shapes.

Conclusions

We have tried to develop an account of embodied cognition that is consistent with the goal of teaching

scientific and mathematical knowledge that is transportable. Toward this end, our chain of argumentation

includes several links. 1) We argued that understanding complex systems principles is important scientifically

and pedagogically. Seemingly unrelated systems are often deeply isomorphic, and the mind that is prepared to

use this isomorphism can borrow from understanding a concrete scenario. 2) If one is committed to fostering the

productive understanding of complex systems, then one must be interested in promoting knowledge that can be

transported across disciplinary boundaries. 3) Our observation of students interacting with complex systems

simulations indicates that one of the most powerful educational strategies is to have students actively interpret

perceptually present situations. A situation’s events inform and correct interpretations, while the interpretations

give meaning to the events. 4) Perspective-dependent interpretations can promote transfer where formalism-

centered strategies fail, by educating people’s flexible perception of similarity. Transfer, by this approach,

occurs not by applying a rule from one domain to a new domain, but rather by allowing two scenarios to be seen


as embodying the same principle. 5) Even algebraic reasoning is sensitive to perceptual grouping factors,

suggesting that perhaps all cognition may intrinsically involve perceptually grounded processes.

Considerable psychological evidence indicates that far transfer of learned principles is often difficult and

may only reliably occur when people are explicitly reminded of the relevance of their early experience when

confronted with a subsequent related situation (Gick & Holyoak, 1980; 1983). This body of evidence stands in

stark contrast to other evidence suggesting that people automatically and unconsciously interpret their world in a

manner that is consistent with their earlier experiences (Kolers & Roediger, 1984; Roediger & Geraci, 2005).

These statements are reconcilable. A lasting influence of early experiences on later experiences occurs when

perceptual systems have been transformed. A later experience will be inevitably processed by the systems that

have been transformed by an initial experience, and priming naturally occurs. When a principle is simply grafted

onto an early experience but does not change how the experience is processed, there is little chance that the

principle will come to mind when it arises again in a new guise. The moral is clear: to reliably make an

interpretation come to mind, one needs to affect how a situation is interpreted as it is “fed forward” through the

perceptual system rather than tacking on the interpretation at the end of processing.

An embodied cognition perspective offers promise of scientifically grounded educational reform in both

of our domains of empirical consideration: complex systems simulations and algebraic calculation. In particular,

reforms are advocated that involve changes to perception, or the co-option of natural perceptual processes for

tasks requiring symbolic reasoning. From this perspective, it is striking that developing expertise in most

scientific domains involves perceptual learning. Biology students learn to identify cell structures, geology

students learn to identify rock samples, and chemistry students learn to recognize chemical compounds by their

molecular structures. In mathematics, perceptually familiar patterns are a major source of solutions for solving

novel problems. Progress in the teaching of these fields depends on understanding the mechanisms by which

perceptual and conceptual representations mutually inform and interpenetrate one another. Specific hypotheses

for future exploration stem from an embodied perspective. How can we modify the perceptual aspects of

mathematical notation conventions to cue students as to their formal properties? The minus sign “-“ is just as


symmetrical as the plus sign “+” but subtraction is order-dependent in a way that addition is not. Should the

signs reflect this formal difference? Can physical spacing be used to scaffold learning formal mathematical

properties? Can we help students learn the order of precedence by presenting them with “A + X*Y” before

giving them the more neutral notation of “A + B * C?”

More generally, our analyses of science and mathematics have led us to reconsider what “abstract

cognition” entails. “Abstract cognition” should be interpreted akin to “abstract art.” Abstract art is still based in

perception – it would not be art if it couldn’t be seen. Similarly, for cognition, abstract is not the opposite of

perceptual, nor are “perceptual” and “superficial” synonyms. A fertile and novel perspective on abstraction may

be that it is the process of a perceptual system interpreting a situation in a useful manner. Interpretation

presupposes that there is an external situation to interpret. Often times, the perceptual system will need to be

trained in order to avoid superficial or misleading attributes. We believe that viewing abstraction as the

deployment of trained and strategic perceptual/motor processes can go a long way toward demystifying symbolic

thought, hopefully leading to the development of robust and working models of cognition.


Discussion

Art Graesser: Part of the problem with transfer from one science domain to another is that the problems may

be rather different. If you use one solution you may have an isomorphic problem, but you are really imposing

it, trying to make sense of it. If it is good enough, you may say that there is an analogy, and it is that process

that helps you perceive things. Give that trajectory of how people make analogies, how would you interpret

your data?

Rob Goldstone: You do not want to have rampant analogy making if there are real, important differences

between domains. In part, what I am arguing for is not just the cognitive science of transfer, but I am also

making claims about what science pedagogy should look like. What are the principles that should be taught in

our schools? There I would make the claim for teaching the kinds of complex systems principles that I have

described. So, I do agree with you that it is problematic when you have principles that apply in one domain but

not another and you try to shoe-horn them into alignment. That clearly happens. But at the same time, exactly

the same formalisms often can be successfully applied in different domains. For example, I think some of the

most important lessons that we should be teaching in our science classes concern positive and negative feedback

systems. Naturally, the nature of these systems will need to be tailored to a scenario, but these skeletal

structures do arise time after time. Teaching these principles can be a lot more valuable than teaching a student

the name for some amino acid because these are principles that at least have the possibility of being relevant for

a new situation that a student is likely to come across at a later point. I believe there are a number of such

principles: oscillations, reaction diffusion systems where the presence of one element gives rise to more of a

second element that may destroy the first element at the same time that the first element is spreading out, or

lateral inhibition between neighboring elements at the same layer in a network. These principles are general

and do arise in different guises. It would be wrong to leave things at that level of description and say that we

have completely explained the system. We then need to adapt the characterization to the specifics of the

scenario, just as case-based reasoning researchers would have us do.


Max Louwerse: I want to challenge your conclusions that just because something is formal reasoning doesn’t

make it amodal. I think that’s wrong. I think that if it is formal reasoning then it is amodal. What you’ve

pointed to is that there could be performance problems. So, I would want to change the claim to – just because

something is formal reasoning does not mean that perception can’t affect performance. If I’m teaching

predicate calculus in a logic class I’ll tell students, “Here’s a rule, modus ponens, that is sensitive only to the

shapes of symbols. But if I make the material implication symbol a mile long so that students have to walk

several blocks to see what the consequent of the material implication is, they are not going to do as well because

they have already forgotten the antecedent. But they engage in formal reasoning when they do reason.

Rob Goldstone: I’m really skeptical of the performance/competence distinction. When you are doing formal

reasoning, it is just physical manipulation of chicken scrawls. When you learn logic you’re learning that you

cross this bit out and replace it with another kind of chicken scrawl. I think very literally it is still just concrete

and perceptual. You’re just learning different types of transformation and translation procedures. In our

experiments, we don’t just get response time differences; we get accuracy differences. If you’re thinking about

it from a formal symbol systems perspective, you would need to say that people are coming up with different

algebraic tree structures based upon perceptual properties.

Max Louwerse: Why is it wrong to think that what I’m teaching my students to do is to behave like a Turing

Machine? Do you think that a Turing Machine would show the same kinds of performance problems that a

human being would?

Rob Goldstone: I think it’s revealing that Alan Turing described the Turing Machine in a particular physical

context. He wasn’t describing it only in terms of a pure mathematical formalism. He was talking about a

specific kind of machine that had specific properties such as tape heads that are moving back and forth. So, I

will take the more radical position that what we are talking about as formal symbol manipulation is what Newell

and Simon referred to as physical symbol systems. They are physical. They are based upon the form of the

objects. So it matters what those forms are and how they are processed. This relates to something that Dr.

Pulvermüller discussed yesterday. Word representations activate Broca’s area because part of their


representation is their production. It’s not as though words are formal systems. They are being produced and

understood as scrawls or sound waves.

Friedemann Pulvermüller: Thanks very much for this important contribution. I want to add a little thing.

I’m fully with you in everything you say, especially with regard to education. I’m coming from a

neuropsychological domain called aphasia therapy research, and teaching patients language after lesions. It is

disastrous to see that the majority of therapists would present words and pictures in a far-removed situation in a

“naming game.” A neurophysiologist had a patient with global aphasia, and in testing her he asked, “Who is

this person sitting to your left?” (it was the patient’s daughter). She responded “…I can’t come up with it. I’m

sorry, I have tried and tried ….. My poor Jacqueline, I can’t even remember your name.” This is exactly the

point you are making. The situation and community of intentions count. The language game that is being

played is very important. So it is not just the relation between the picture and word that counts. Non-

communicative language wouldn’t even produce the same speech output. With regard to the

competence/performance distinction, if we think of the brain in terms of real neuronal networks, we may have

difficulty preserving the distinction. It is clear that the neural assemblies and their connections are the

implementation of both the rules and the words – the competence of the system. The activity of these same

networks is the correlate of performance. So, this distinction does not make too much sense.

Rob Goldstone: The only thing that I would add to that is that although I am skeptical of a

performance/competence distinction, I’m all in favor of having responses based on different types of

representations, giving rise to different performances on different occasions. Some of these representations will

give rise to output that looks more in accord with what we expect by our intellectualized understanding of logic.

But I think it would be wrong to say that those that do correspond to our mathematical understanding are not

done by cognitive, physical manipulations.

Deb Roy: I’d like to stay on the topic of performance versus competence and the question of whether we can

keep this distinction or not. Of course, probably everyone here is thinking of Chomsky when we think about

this distinction. David Marr used slightly different terminology, but his general idea was that when one is


trying to understand a system, usually a complex system, it is useful to distinguish what it is doing from how it

is doing it. If you don’t have a clear idea of what it is doing then it is problematic to ask the question of “how.”

So, if we get rid of the labels “performance” and “competence” and rephrase the question to ask, “Is it still

worthwhile to keep Marr’s distinction?” Or, would you eliminate that distinction as well?

Rob Goldstone: I’m happier preserving a “what” versus “how” distinction, and I also still want to be able to

talk about errors in processing. We want to be able to say things like, “What we want to get out is a three-

dimensional representation of a scene, but if we use this particular stereopsis algorithm that combines

information from the two eyes, then we won’t get the desired output.” That is, we do need to be able to say that

the operation of an algorithm falls short of its intended computation.

Deb Roy: Then, I’d say that is completely in line with the original performance/competence distinction. You’re

still on board, but just using different terminology.

Luc Steels: There was more to the original distinction. In a usage based approach to language, the actual

variation and performance is influenced all the time.

Deb Roy: So you’re saying that that performance versus competence assumes dynamic versus static systems?

Luc Steels: You can tease out different meanings, but “competence” ended up being a description of an abstract

system without worrying about how using the system constantly impacts the original system.

Deb Roy: Maybe there is a family of interpretations. When I think of Marr’s “what” level, I never imagined

that it needs to be fixed. When one thinks of Chomsky, one may think that competence is innate and does not

change perhaps.

Rob Goldstone: For me, what is added with the performance/competence distinction is that it suggests that it is

useful to ask, “What would this system be able to do if you took away all of the contextualizing variables that

Dr. Pulvermüller just described, or all of the system’s performance-based limitations. However, from my

perspective, it’s exactly because of those contextual and performance-based factors that the system is able to do

anything at all. It doesn’t make sense to talk about, extracting out contextual and physical considerations, what


the competence of the system would be. However, it still might make sense to ask, from a teleological

perspective, what the aim of a system is - Marr’s “what is it doing?” question.

Deb Roy: I can live with that.


References

Anderson, J. (2005). Human symbol manipulation within an integrated cognitive architecture.

Cognitive Science, 29, 313-341.

Arnheim, R. (1970). Visual thinking. Faber and Faber: London.

Ball, P. (1999). The self-made tapestry. Oxford, England: Oxford University Press.

Barnett, S. M., & Ceci, S. J. (2002). When and where do we apply what we learn? A taxonomy for

far transfer. Journal of Experimental Psychology: General, 128, 612-637.

Barsalou, L.W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22, 577-660.

Barsalou, L.W., Simmons, W.K., Barbey, A.K., & Wilson, C.D. (2003). Grounding conceptual

knowledge in modality-specific systems. Trends in Cognitive Sciences, 7, 84-91.

Bar-Yam, Y. (1997). Dynamics of complex systems. Reading, MA: Addison-Wesley.

Bassok, M. (1996). Using content to interpret structure: Effects on analogical transfer. Current

Directions in Psychological Science, 5, 54-58.

Bassok, M., Chase, V. M., & Martin, S. A. (1998). Adding apples and oranges: Alignment of

semantic and formal knowledge. Cognitive Psychology, 35, 99-134.

Bassok, M. (2001). Semantic alignments in mathematical word problems. In D. Gentner, K. J.

Holyoak, & B. N. Kokinov (eds.) The analogical mind. MIT Press: Cambridge, MA. (pp. 401-

433).

Batty, M. (2005). Cities and complexity: Understanding cities with cellular automata, agent-based

models, and fractals. MIT Press: Cambridge, MA.

Bentley, W. A., & Humphreys, W. J. (1962). Snow crystals. New York: Dover.

Bialek, W., & Botstein, D. (2004). Introductory science and mathematics education for 21st-century

biologists. Science, 303, 788-790.


Carraher, D., & Schliemann, A. D. (2002). The transfer dilemma. Journal of the Learning Sciences,

11, 1-24.

Casti, J. L. (1994). Complexification. New York: HarperCollins.

Chi, M. T. H. (2005). Commonsense conceptions of emergent processes: Why some misconceptions

are robust. Journal of Learning Sciences, 14, 161-199.

Chi, M. T. H., Feltovich, P., & Glaser, R. (1981). Categorization and representation of physics

problems by experts and novices. Cognitive Science, 5, 121-152.

Csermely, P. (1999). Limits of scientific growth. Science, 5420, 1621.

Darwin, C. (1859). On the origin of species by means of natural selection. London: Murray.

DeLoache, J. S. (1991). Symbolic functioning in very young children: Understanding of pictures and

models. Child Development, 62, 736-752.

DeLoache, J. S. (1995). Early understanding and use of symbols: The model model. Current

Directions in Psychological Science, 4, 109-113.

Deloache, J. S. (2000). Dual representation and young children's use of scale models. Child

Development, 71, 329-338.

DeLoache, J. S., & Burns, N. M. (1994). Symbolic functioning in preschool children. Journal of

Applied Developmental Psychology, 15, 513-527.

DeLoache, J. S., & Marzolf, D. P. (1992). When a picture is not worth a thousand words: Young

children's understanding of pictures and models. Cognitive Development, 7, 317-329.

Detterman, D. R. (1993). The case for prosecution: Transfer as an epiphenomenon. In: Detterman D K,

Sternberg R J (eds.) Transfer on trial: Intelligence, cognition, and instruction. Westport, CT:

Ablex.

Einstein, A. (1989). The collected papers of Albert Einstein. Vol. 2: The Swiss years: writings, 1900-


1909. Princeton, NJ: Princeton University Press.

Flake, G. W. (1998). The computational beauty of nature. Cambridge, MA: MIT Press.

Fodor, J. A.(1992). A theory of content and other essays. Cambridge, MA: MIT Press.

Garcia-Ruiz, J. M., Louis, E., Meakin, P.m & Sander, L. M. (1993). Growth patterns in physical

sciences and biology. New York: Plenum Press.

Gentner D., & Stevens A. L. (eds.) (1983) Mental models. Hillsdale, NJ: Erlbaum.

Gick, M. L., & Holyoak, K. J. (1980). Analogical problem solving. Cognitive Psychology, 12, 306-

355.

Gick, M. L., & Holyoak, K. J. (1983). Schema induction and analogical transfer. Cognitive

Psychology, 15, 1-39.

Glenberg, A., De Vega, M., & Graesser. A. (in press). Framing the debate.

Glenberg, A. M., Jaworski, B., Rischal, M, & Levin, J.R. (in press). What brains are for: Action,

meaning, and reading comprehension. To appear in D. McNamara (Ed.), Reading comprehension

strategies: Theories, interventions, and technologies. Mahwah, NJ: Lawrence Erlbaum Publishers.

Glenberg, A. M., Gutierrez, T., Levin, J. R., Japuntich, S., & Kaschak, M. P. (2004). Activity and

imagined activity can enhance young children's reading comprehension. Journal of Educational


Goldstone, R. L. (1994). Influences of categorization on perceptual discrimination. Journal of

Experimental Psychology: General, 123, 178-200.

Goldstone, R. L. (1998). Perceptual Learning. Annual Review of Psychology, 49, 585-612.

Goldstone, R. L. (2003). Learning to perceive while perceiving to learn. In R. Kimchi, M. Behrmann,

& C. Olson (Eds.) Perceptual organization in vision: Behavioral and neural perspectives. (pp.

233-278). New Jersey: Lawrence Erlbaum Associates.


Goldstone, R. L., & Barsalou, L. (1998). Reuniting perception and conception. Cognition, 65, 231-

262.

Goldstone, R. L., Lippa, Y., & Shiffrin, R. M. (2001). Altering object representations through

category learning. Cognition, 78, 27-43.

Goldstone, R. L., & Sakamoto, Y. (2003). The transfer of abstract principles governing complex

adaptive systems. Cognitive Psychology, 46, 414-466.

Goldstone, R. L., & Son, J. Y. (2005). The transfer of scientific principles using concrete and

idealized simulations. The Journal of the Learning Sciences, 14, 69-110.

Graesser, A. C., & Jackson, G. T. (this volume). Body and Symbol in AutoTutor: Conversations that

are Responsive to the Learners’ Cognitive and Emotional States.

Graesser, A. C., & Olde, B. A. (2003). How does one know whether a person understands a device?

The quality of the questions the person asks when the device breaks down. Journal of Educational


Hadamard, J. (1949). The psychology of invention in the mathematical field. Princeton University

Press: Princeton, NJ.

Harnad, S., Hanson, S. J., & Lubin, J. (1995). Learned categorical perception in neural nets:

Implications for symbol grounding. in V Honavar & L. Uhr (Eds.), Symbolic processors and

connectionist network models in artificial intelligence and cognitive modelling: Steps toward

principled integration. Boston: Academic Press (pp. 191-206).

Hatano, G. & Greeno, J. G. (1999). Alternative perspectives on transfer and transfer studies.

International Journal of Educational Research, 31, 645-654.

Hegarty, M. (2004a). Mechanical reasoning by mental simulation. Trends in Cognitive Science, 8,

280-285.


Hegarty, M. (2004b). Dynamic visualizations and learning: getting to the difficult questions.

Learning and Instruction, 14, 343-351.

Hmelo-Silver, C. E., & Pfeffer, M. G. (2004). Comparing expert and novice understanding of a

complex system from the perspective of structures, behaviors, and functions. Cognitive Science,

28, 127-138.

Horgan, J. (1996). The end of science: Facing the limits of knowledge in twilight of the scientific age.

Reading, MA: Addison-Wesley.

Inhelder, B., & Piaget, J. (1958). The growth of logical thinking from childhood to adolescence. New

York: Basic Books.

Jacobson, M. J. (2001). Problem solving, cognition, and complex systems: Differences between

experts and novices. Complexity, 6, 41-49.

Jacobson, M. J., & Wilensky, U. (2006). Complex systems in education: scientific and educational

importance and research challenges for the learning sciences. Journal of Learning Sciences, 15, 11-

34.

Koffka, K.(1935). Principles of gestalt psychology. New York: Harcourt Brace Jovanovich.

Kohonen, T. (1995). Self-organizing maps. Berlin: Springer-Verlag.

Kolers, P. A., & Roediger, H. L. (1984). Procedures of mind. Journal of Verbal Learning & Verbal

Behavior, 23, 425-449.

Lakoff, G., & Nuñez, R. E. (2000). Where mathematics comes from. Basic Books: New York.

Landy, D., & Goldstone, R. L. (under review). How abstract is symbolic thought?

Lave, J. (1988). Cognition in practice: Mind, mathematics, and culture in everyday life. New York:

Cambridge University Press.


Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. New York:

Cambridge University Press.

Leeper, R. (1935): A study of a neglected portion of the field of learning: The development of sensory

organization, Journal of Genetic Psychology, 46, 41-75

Livingston, K. R., Andrews, J. K., & Harnad, S. (1998). Categorical perception effects induced by

category learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24,

732-753.

Nathan, M. J., Kintsch, W., & Young, E. (1992). A theory of algebra word problem comprehension

and its implications for the design of computer learning environments. Cognition and Instruction,

9, 329-389.

Nathan, M. J., Long, S. D., & Alibali, M. W. (2002). The symbol precedence view of mathematical

development: A corpus analysis of the rhetorical structure of textbooks. Discourse Processes, 33,

1-21.

Nathan, M. J., & Petrosino, A. (2003). Expert blind spot among preservice teachers. American

Educational Research Journal, 40, 905-928.

Nunes, T. (1999). Mathematics learning as the socialization of the mind. Mind, Culture, and Activity,

6, 33–52.

Özgen, E. (2004). Language, learning, and color perception. Current Directions in Psychological

Science, 13, 95-98.

O'Reilly, R.C. (2001). Generalization in interactive networks: The benefits of inhibitory competition

and Hebbian learning. Neural Computation, 13, 1199-1242.

Palmer, S., & Rock, I. (1994). Rethinking perceptual organization: The role of uniform

connectedness. Psychonomic Bulletin & Review, 1, 29-55.


Penner, D.E. (2000). Explaining processes: Investigating middle school students’ understanding of

emergent phenomena. Journal of Research in Science Teaching, 37, 784-806.

Reed, S. K., Ernst, G. W., & Banerji, R. (1974). The role of analogy in transfer between similar

problem states. Cognitive Psychology, 6, 436-450.

Resnick, M. (1994). Turtles, termites, and traffic jams. Cambridge, MA: MIT Press.

Resnick, M., & Wilensky, U. (1993). Beyond the deterministic, centralized mindsets: New thinking

for New Sciences. American Educational Research Association. Atlanta. GA.

Roberson, D., Davies, I., & Davidoff, J. (2000). Color categories are not universal: Replications and

new evidence in favor of linguistic relativity. Journal of Experimental Psychology: General, 129,

369–398.

Roediger, H. L., & Geraci, L. (2005). Implicit memory tasks in cognitive research. In A. Wenzel &

D. C. Rubin (Eds.), Cognitive methods and their application to clinical research. (pp. 129-151).

Washington, DC, US: American Psychological Association.

Ross, B. H. (1987). This is like that: the use of earlier problems and the separation of similarity

effects. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13, 629-639.

Ross, B. H. (1989). Distinguishing types of superficial similarities: Different effects on the access and

use of earlier problems. Journal of Experimental Psychology: Learning, Memory, and Cognition,

15, 456-468.

Roy, D. (this volume). Simple languages.

Roy, D., & Pentland, A. (2002). Learning words from sights and sounds: A computational model.

Cognitive Science, 26, 13-146.

Rumelhart, D. E., & Zipser, D. (1985). Feature discovery by competitive learning. Cognitive Science,

9, 75-112.


Schwartz, D. L. & Black, T. (1999). Inferences through imagined actions: Knowing by simulated

doing. Journal of Experimental Psychology:Learning Memory and Cognition, 25 (1): 116-136.

Shepard, R. N. (1984). Ecological constraints on internal representations: Resonant kinematics of

perceiving, imagining, thinking, and dreaming. Psychological Review, 91, 417-447.

Shepard, R. N. (1987). Toward a universal law of generalization for psychological science. Science,

237, 1317-1323.

Simmons, K., & Barsalou, L.W. (2003). The similarity-in-topography principle: Reconciling theories

of conceptual deficits. Cognitive Neuropsychology, 20, 451-486.

Sloutsky, V. M., Kaminski, J. A., & Heckler, A. F. (2005). The advantage of simple symbols for

learning and transfer. Psychonomic Bulletin & Review, 12, 508-513.

Smith, L. B. (2003). Learning to recognize objects. Psychological Science, 14, 244-250.

Thurston, W. (1994). On proof and progress in mathematics. Bulletin of the American Mathematical

Society, 30, 161-177.

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: heuristics and biases. Science,

185, 1124-1131.

Uttal, D. H., Liu, L. L., & DeLoache, J. S. (1999). Taking a hard look at concreteness: Do concrete

objects help young children learn symbolic relations? in C. S. Tamis-LeMonda (Ed) Child

psychology: A handbook of contemporary issues (pp. 177-192). Philadelphia, PA: Psychology

Press.

Uttal, D. H., Scudder, K. V., & DeLoache, J. S. (1997). Manipulatives as symbols: A new perspective

on the use of concrete objects to teach mathematics. Journal of Applied Developmental



von der Malsburg, C. (1973). Self-organization or orientation-sensitive cells in the striate cortex.

Kybernetik, 15, 85-100.

Wason, P.C., & Shapiro, D. (1971). Natural and contrived experience in a reasoning problem. Quarterly

Journal of Experimental Psychology, 23, 63-71.

Wilensky, U. (1991). Abstract meditations on the concrete and concrete implications for mathematics

education. In I. Harel, & S. Papert (Eds.) Constructionism. (pp. 193-203). Westport, CT: Ablex.

Wilensky, U. & Reisman, K. (1999). ConnectedScience: Learning biology through constructing and

testing computational theories -- An embodied modeling approach. InterJournal of Complex

Systems, 234, 1-12.

Wilensky, U. & Reisman, K. (2006). Thinking like a wolf, a sheep or a firefly: Learning biology

through constructing and testing computational theories -- An embodied modeling approach.

Cognition and Instruction. (in press).

Wiley, J. (2003) Cognitive and educational implications of visually-rich media: Images and

imagination. In M. Hocks and M. Kendrick (Eds.) Eloquent images: Word and image in

the age of new media. MIT Press.

Wittgenstein, L. (1953). Philosophical investigations. G. E. M. Anscombe, trans., New York:

Macmillan.


Author Notes

Many useful comments and suggestions were provided by Arthur Glenberg, Arthur Graesser, Linda

Smith, Manuel De Vega, Uri Wilensky, and in particular Mitchell Nathan. This research was funded by

Department of Education, Institute of Education Sciences grant R305H050116, and NSF REC grant 0527920.

Further information about the authors’ laboratory can be found at http://cognitrn.psych.indiana.edu/.

a well grounded education: the role of perception in...

Documents