chapter 10 - nederlands tweelingen register270 chapter 10 dynamics of development a complex systems...

270

CHAPTER 10

Dynamics of DevelopmentA Complex Systems Approach

Han l. J. van der maas, kees Jan kan, Abe Hofman, and maartje e. J. Raijmakers

Complex systems are everywhere. They are studied in fields such as mathematics, physics, chemistry, biology and economy. Figure 10.1 gives an impression of the diverse topics and applications in complex systems research. The psychological system is probably the most complex system that we can study because it involves the brain’s neural network but also the various social and societal networks in which it is imbed-ded. In addition, it is a developmental system. Its development in a newborn with only basic reflexes to an adult with the ability to lecture, reason, organize, and many more things, is even astonishing from a complex systems view.

Clearly, the study of human development is very challenging because of (1) its complexity, (2) the many (ethical) restrictions on research methods, and (3) its para-doxical status. With regard to the latter, it would be problematic to conclude that humans are incapable of scientific study. One consequence of these severe challenges is that psychology is still largely characterized by informal verbal descriptions, and most research is based on unconnected verbal mini- theories. In our view, one route to prog-ress in psychological science is in studying and copying ways to model and investigate complex systems in the natural sciences.

In this chapter we follow this route by providing three examples of modeling and investigating complex systems. First, we present a new model for general intelligence based on a mathematical model for ecological networks. Second, we discuss ways to study phase transitions in psychological systems. Third, we introduce a completely new approach to collect high- frequency data on children’s development, which is a neces-sity for studying complex systems. With these three examples we hope to demonstrate the viability of the complex systems approach to the study of human development.

CHAPTER 10

Molenaar_HbkDvlpmntlSysThryMthdlgy.indb 270 9/18/2013 5:24:28 PM

A Complex Systems Approach 271

Modeling Complex Systems: The Case of General Intelligence

One of the most replicated findings in differential psychology is the positive manifold of correlations between scores on all kinds of cognitive tests in unrestricted samples of human subjects (Jensen, 1998). Simply put, if you score well on one type of cognitive test, you will probably also score well on any other cognitive test. If these correla-tions are subjected to factor analysis we usually arrive at some sort of factor model (e.g., hierarchical) with one main factor at the apex, Spearman’s g factor. The impact

Modeling Complex Systems: The Case of General Intelligence

Community Identification

Scale-free networks

Social network analysis Small-worldnetworks

Centrality

Graphtheory

Motifs

Scaling

Systemsbiology

Robustness/vulnerability

Dynamical networks

Adaptive networks

Social dynamics

Collective intelligence

Self-organized criticalityHerd

mentality

Agent-based

modeling

Phasetransition

Synchronization Ant colony optimization

Particle swarm optimization

Swarm behavior

Prisoner’s dilemma (PD)

Rational decisionmaking

Iterative PD

Boundedrationality

n-person PD

Irrationalbehavior

Cooperation versuscompetition

Spatial/networkgame theory

Evolutionarygame theoryTime series analysis

Ordinary differential equations

Iterative maps Phase space

StabilityanalysisAttractors

ChaosPopulation dynamics

Multistability Bifurcation

Coupled maplattices

Homeostasis

Feedbacks Self-reference

Goal-oriented/guided behavior

System dynamics

Sensemaking

Entropy

Autopoiesis

Computationtheory

Cybernetics

Information theory

Complexitymeasurement

Spatial fractals

Reaction-diffusion systems

Partial differential equations

Dissipativestructures

Percolation

Cellularautomata

Spatial ecology Self-replication

Spatial evolutionary biology

Geomorphology

Artificial neural networks

Evolutionary computation

Genetic algorithms/programming

Artificiallife

Machinelearning

Evo-Devo Artificial intelligence

Evolutionary robotics

Evolvability

Evolution andAdaptation

Networks

CollectiveBehavior

GameTheory

NonlinearDynamics

SystemsTheory

PatternFormation

Emergence

Self-Organization

over scale

over time

Complex Systems

fIGURe 10.1. The field of complex system research. Created by Hiroki Sayama, DSc, Collec-tive Dynamics of Complex Systems (CoCo) Research Group at Binghamton University, State University of New York.


272 DYNAMICS OF DEVELOPMENT

of Spearman’s (1904, 1927) factor- analytic work on practical and theoretical work in psychology can hardly be overstated. Psychologists routinely propose new latent psychological constructs, develop ability tests or questionnaires, and analyze the data according to this g-factor protocol (cf. Cramer, Waldorp, van der Maas, & Borsboom, 2010).

Although we do not have objections to the statistical work, we challenge the theo-retical account of the g factor. First, the g-factor model of intelligence lacks any devel-opmental component (Ackerman & Lohman, 2003). Are we born with g? Does g itself develop? Standard g theory does not provide answers, but a variant of this theory, investment theory, might. Cattell’s (1971) investment theory distinguishes fluid (gf) and crystallized intelligence (gc). During development gf is invested to result in gc. Apart from difficulties with the definitions of gf and gc (Kan, Kievit, Dolan, & van der Maas, 2011), this theory is still rather vague on development. Apparently, we should not take the metaphor of investment too literally, since gf does not decrease in this process (a good example of the problems of verbally stated theories in psychology). But more importantly, it is now unclear what the status of gf is.

Second, apart from its developmental role, we need to know how g relates to brain variables and how g is connected to cognitive processing. Many have attempted to identify g with measurable variables (e.g., speed of nerve conductance, reaction time, glucose metabolism in the brain). These studies have produced interesting results, but have not revealed the single underlying cause of the g factor (Ackerman, Beier, & Boyle, 2005; Luciano et al., 2005).

Sampling Theory

Interestingly, there are alternative mechanisms for general intelligence that can pro-duce exactly the same correlational data as the g-factor model. The first is called sampling theory, proposed by Thompson in 1914 (Bartholomew, Deary, & Lawn, 2009; Thorndike, 1927). It says that solving cognitive tasks requires many lower-order uncorrelated (neural) processes or modules. It is assumed that tasks will call upon overlapping samples of modules, causing positive correlations between the test scores. Therefore, complex tasks, involving many modules, will load strongest on the g factor. Theoretically, if we were able to devise unidimensional cognitive tests for each module, the positive manifold would disappear. In that sense, general intelligence is a measurement artifact.

Sampling theory is largely ignored in the intelligence literature but seems to be at least partly correct. Psychologists are unable to create strictly unidimensional tests (Lumsden, 1976), and sampling from the same underlying processes seems inevitable. In addition, this mechanism of sampling may also play a role in the relation between genes and intelligence. The so- called watershed model of Cannon and Keller (2005) describes how specific genes play a role in “upstream,” narrowly defined endopheno-types, which in turn play a role in a number of more upstream cognitive abilities. In our opinion, besides the genetic and measurement parts of the explanation of general intelligence, a second, developmental mechanism is relevant.



Mutualism in Networks of Cognitive Abilities

van der Maas and colleagues (2006) proposed another mechanism to explain the posi-tive manifold of correlations. As in sampling theory, it is assumed that the cognitive system consists of many basic, lower-order cognitive processes that are initially uncor-related. The development of each process has two parts, an autonomous part, best described by a simple logistic growth equation, and a mutualistic part. The mutualistic part is based on the assumption that the growth of each ability is influenced by other abilities, and that all abilities are organized in a network with mainly positive interac-tions. An example would be the growth of working memory and arithmetic ability. An increase in working memory probably allows for more complicated mental calcu-lations, and vice versa; daily training in arithmetic in schools might increase general working memory capacity (Siegler & Alibali, 2005). Other examples are syntactic and semantic bootstrapping (Fisher, Hall, Rakowitz, & Gleitman, 1994; Pinker, 1994).

van der Maas and colleagues (2006) applied a well- studied mathematical model for mutualistic networks that has been developed for ecological systems (the Lotka– Volterra mutualism model; May, 1973; Murray, 2002):

1

d(1 / ) /

d

Wi

i i i i i ij j i ijj i

xa x x K a M x x K

t =≠

= − + ∑ for i = 1 . . . W (10.1)

The xi represents W cognitive processes. Parameters ai are growth parameters, influ-encing the steepness of the logistic growth function associated with each xi. Ki rep-resents the limited resources of the logistic growth processes. The matrix M encom-passes the strength, Mij, of the interactions between the basic processes, x. Parameters x0, a, and K, are random parameters, whereas M is a population parameter, meaning that we assume that the interaction strengths are equal over subjects. In simulations, we sample uncorrelated values for x0, a, and K, choose M, and compute the develop-ment of x over time using an LSODA (a solver for first-order ordinary differential equations). This equation gives a developmental pattern as shown in Figure 10.2.

fIGURe 10.2. The development of a set of cognitive abilities in the mutualism model.



We repeat this procedure for N subjects and take the values of x at some point in time (e.g., t = 2) and subject these values to correlation and factor analysis. van der Maas and colleagues (2006) have shown that this results in typical positive manifold correlation matrices, which yield one dominant factor as in the g model. If all connec-tions have the same positive value, we obtain the standard one- factor model. If the connections are sampled from a normal distribution with positive mean, we obtain data that fit with a hierarchical factor model.

If we assume the simplest additive structure for the effects of genes and environ-ment on the K parameter (resources parameter), the heritability of the dominant factor obtained by factor analysis shows an increase with age, as found in empirical data (Bartels, Rietveld, van Baal, & Boomsma, 2002). Also, the low predictability of intel-ligence from early childhood performance and the differentiation effects in intelligence are easily explained in the mutualism model.

The Cattell–Jensen Effect

One key effect we could not easily explain is the so- called Jensen effect for heritabil-ity. This effect pertains to the correlations between the vector of heritabilities and the vector of g loadings of cognitive tests (i.e., the method of correlated vectors). These correlations are typically larger than .5, which is taken to suggest a high influence of the genetic component of g (Jensen, 1998). It appears that the mechanism of mutual-ism leads to such a correlation only if we introduce weak positive correlations between the genetic parts of resources K. However, introducing correlations between model parameters weakens the idea of the mutualism model.

This issue was reason to further study this Jensen effect. In Kan, Wicherts, Dolan, and van der Maas (in press), we analyzed the results of 23 twin studies with regard to the heritability coefficients and g loadings (see Figure 10.3). In most cases we indeed found a positive correlation between these parameters. However, detailed analysis of these correlations revealed an unexpected pattern. It appears that the most herita-ble and most g-loaded tests are the typical gc subtests. These crystallized or culture- loaded tests are tests such as vocabulary, spelling, and arithmetic. When culture load is reduced, fluid tests such as picture completion and digit span show lower heritability coefficients and g loadings. In these analyses we control for differences in reliability.

From a standard g theoretical point of view this effect, which we call the Cattell– Jensen effect, is hard to explain. Jensen himself, for instance, did not expect much of these cultural tests:

Information tests consisting of questions like “Who was the first President of the United States?,” “Whose picture is on a penny?,” and so on, make poor test items mainly for two reasons: (a) they do not get at complex mental process, and (b) they cannot be steeply graded in difficulty level without introducing items of information to which there is a relatively low probability of exposure, in which case social status and educational differences become practically impossible to avoid. The same holds true for vocabulary tests. . . . The difficulty levels differ only because of frequency of



exposure. Such items based on information and vocabulary are rightly regarded as more culturally loaded than items which vary in difficulty because of the complexity of the mental processes involved. (1973, p. 184)

Yet precisely these information tests are responsible for the Jensen effect.

Explaining the Cattell–Jensen Effect

Since the Cattell– Jensen effect concerns a differentiation in cultural (crystallized) and noncultural (fluid) ability tests, we have to think of an extension of the mutualism model that allows for these abilities. In Kan (2011, Chap. 7) such a mutualism model is proposed. In this simulation model there are eight fluid and eight crystallized abilities and an external environment. The mutualistic weights are specified such that:

• Fluid → fluid interactions are mildly positive.

• Fluid → crystallized interactions are strong.

fIGURe 10.3. The relations among g loading, heritability coefficients (h2), and cultural load in the Minnesota Twin Study. Culture- loaded tests are the most g loaded and show the highest heritability coefficients. Adapted from Kan, Haring, Wicherts, Dolan, and van der Maas (in press).



• Crystallized → fluid interactions are mildly positive.

• Crystallized → crystallized interactions are mildly positive.

• Effect of environment on crystallized and fluid abilities is (equally) high.

• Environment is only influenced (strongly) by crystallized abilities.

As in van der Maas and colleagues (2006), we used a standard additive genetic model for the resources K, with small intercorrelations between the genetic parts of K. The latter effect represents genetic sampling. Kan shows that this scenario, as well as some related alternative scenarios, lead to the Cattell– Jensen effect. The main mechanism in this explanation is a gene– environment correlation.

Kan (2011) summarizes our view of general intelligence in Figure 10.4. Correla-tions between fluid abilities result from mutual beneficial interactions between basic cognitive processes during development. Crystallized abilities are the result of these basic cognitive processes. The development of crystallized abilities has a beneficial effect on the development of fluid abilities. Some individuals turn out to be more intel-ligent because they possess higher levels of cognitive abilities. These individuals will be more likely to end up in cognitive environments conducive to the further development

fIGURe 10.4. The explanation of general intelligence based on the combined effects of devel-opmental mutualism, environmental multipliers, genetic sampling, and test sampling. The fac-tor g is, like IQ, nothing more than an index of general intelligence, not the common cause.

g

E K

G

xf

xf

xf

f1

f2

f3

f4

f5

f6EK

G

xcr

xcr

xcr

c1

c2

c3

c4

c5

c6

Ec

EKG

K

E

IQ

geneticsampling

test sampling

factoranalysis/

SEM

fluid

multiplier

crystallized

mutualism



of crystallized abilities (which is, in turn, beneficial to the further development of fluid abilities). The growth of the cognitive abilities will be constrained by genetically and environmentally influenced limited capacities. These capacities are possibly weakly intercorrelated due to genetic sampling. Sampling on the test level may also be present. According to this model general intelligence, as the outcome of factor analysis, is an index of cognitive functioning.

Analyzing Complex Systems: Phase Transitions

Although the mutualism model explains a wide variety of phenomena in intelligence research, its dynamics are rather dull. Normally, we will see curves like those in Fig-ure 10.2, where all abilities will raise to some stable value. The dynamics depend on the matrix of mutualistic weights, which are all positive in the basic mutualism model. If some are also negative, implying competitive relations, the behavior becomes more irregular. These irregularities could be chaotic but could also result from alterna-tive stable states in the system. Such irregularities have been demonstrated in similar dynamic models by van Geert (1991, 1994).

The occurrence of alternative stable states in complex systems is especially rel-evant when in developmental systems (Waddington, 1966a). We find it notable that the equilibration theory of Piaget (1964/1997) does fit very well in the modern complex systems view of development. Piagetian concepts such as disequilibrium, stages, tran-sitions, and reorganization, are all terms that have well- defined meanings in current dynamical theory (Molenaar & Raijmakers, 2000).

Interestingly, this correspondence goes beyond theory. It has been extremely dif-ficult to test Piaget’s theory empirically (Flavell & Wohlwill, 1969). Take, for instance, the idea of transitions between developmental stages in the development of the con-servation ability. The developmental researchers in the 20th century could never reach consensus on what a transition exactly was and how it could be detected (Brainerd, 1978; Wohlwill, 1973). But in complex systems theory, especially catastrophe theory or bifurcation theory, phase transitions are well defined mathematically. Moreover, there are many new methods with which to investigate phase transitions empirically.

The Cusp Catastrophe

In fact, there are many different types of phase transitions, among other things, depending on the types of equilibria or attractors involved. The attractors can be simple point attractors but also strange attractors associated with complex chaotic behavior. In most cases, the crucial point is that the current attractor loses its stability and the system moves (often abruptly) to a new state.

A prototypical example of a phase transition is the cusp catastrophe, which mod-els the transition between two point attractors as a function of two control variables. An illustrative example is shown in Figure 10.5. The cusp model is the most applied phased transition model in catastrophe theory.

Analyzing Complex Systems: Phase Transitions



To explain the usefulness of this model in developmental psychology, we discuss the example of Jansen and van der Maas (2001). They studied the transition from rule I to rule II on the balance scale test of proportional reasoning. In this test chil-dren have to predict the movement of a balance scale on which weights are placed at varying distances from the fulcrum. Rule I users, typically between 4 and 8 years of age, completely ignore the distance information. Rule II users, somewhat older, also ignore distance when the number of weights left and right differs, but they do use distance information when the number of weights left and right is equal. Such items, with equal weights and unequal distances, are called distance items. Higher, more advanced rules, also incorporate distance in other items. To distinguish these rules other item types are required, but to distinguish between rules I and II, a test with only distance items suffices.

An important first indication of a catastrophic switch from rule I to rule II is the bimodal distribution of sum scores on sets of distance items (van der Maas & Mole-naar, 1993). Jansen and van der Maas (2001) show bimodality for several sample data-sets. Further evidence for the phase transition hypothesis can be gathered by testing for the presence of other indicators or flags of catastrophic change (Gilmore, 1981). The most appealing flag is hysteresis, the phenomenon that jumps up and down are delayed (see Figure 10.5). Hysteresis can be proven by testing where jumps take place

Fh Fh jump

Fv

Fv

a.

Fv

Fv

b.

Businesscard

equilibria of thecusp

bends right

bends left sudden jump/hysteresis

Inaccessiblearea horizontal

force (Fh)a.

vertical force (Fv)

b.

fIGURe 10.5. The cusp catastrophe can be illustrated with a simple business card. Vertical force (Fv) can be exerted by holding the card between the thumb and index finger of one hand. By pressing with the other index finger (Fh), the card bends from left to right. For high Fv this change will be sudden or catastrophic. The dynamics of this system are described by the cusp catastrophe on the right, in which Fv and Fh are the control variables (the splitting and normal variable, respectively). The surface represents the bending of the card. Note that for high Fv, the middle position (straight) card is instable, something that can be verified easily.



when slowly increasing and decreasing the normal control variable. It can be shown, for instance, that in disturbance- free conditions, water freezes at –4°C, whereas ice melts at 0°C.

Jansen and van der Maas (2001) applied such a design to balance scale learning. They used a series of items for which the distance difference on distance items was sys-tematically varied. The distance difference refers to the difference in distance between the equal number of weights left and right on the balance scale. Figure 10.6 displays the item set and an answer pattern associated with hysteresis.

Jansen and van der Maas (2001) found a statistically significant number of hyster-esis patterns in the responses of 314 children. They tested for hysteresis in sets of items administered in alternative orders and controlled for alternative explanations by using items sets in which other stimulus characteristics were varied. The combined evidence for bimodality and hysteresis makes a strong case for a real developmental transition. Another recent example of this empirical approach is described in Dutilh, Wagenmak-ers, Visser, and van der Maas (2010).

The Link to Categorical Latent Structure Modeling

A logical next step in testing the cusp catastrophe model would be to fit the cusp model to data consisting of measurements of the behavioral variable and both the control variables. Based on the work of Cobb (1980) we developed an R package that does exactly this (Grasman, van der Maas, & Wagenmakers, 2009). We refer to this paper for an extended explanation of this technique. An application of Cobb’s method can be found in Ploeger, van der Maas, and Hartelman (2002).

Here we discuss an alternative line of work in which we make a connection between complex systems theory and a set of statistical techniques known as cat-egorical latent structure modeling. This type of latent structure modeling is used when

fIGURe 10.6. Balance scale items used to detect hysteresis in the transition from rule I to rule II in proportional reasoning. The administration order of items is depicted with the arrows. Answers left and right display a typical hysteresis pattern.

in balance

tipss to the right

tips to the right

tips to the right

in balance

in balance

in balance

in balance

tipss to the right



individual differences are categorical or discrete. We think complex systems theory sheds a new light on the continuing debate in psychology concerning the dichotomy of categorical/discrete versus dimensional/continuous (e.g., De Boeck, Wilson, & Acton, 2005).

A good example to explain our point concerns attitudes, such as people’s ideas on abortion. The question is whether the individual differences (pro-life vs. pro- choice) are categorical or continuous. Is pro-life or pro- choice a typology or a dimension? A simple cusp model of this attitude has two control variables. The normal variable (“Fh” in Figure 10.5) relates to political and especially religious beliefs of the person, since religious beliefs are more associated with the pro-life point of view. The splitting variable (vertical pressure; “Fv” in Figure 10.5) can best be interpreted as involvement in the issue. Within the group of highly involved subjects we expect a strong discrete typology of pro-life and pro- choice positions. In this group change is hard and always sudden. When involvement is low, however, we expect a continuum of positions than can change more easily. It is reasonable to expect a typology in the United States where involvement in the abortion issue is high, and a continuum in the Netherlands, where abortion is not a big issue. Thus, whether a variable is discrete or continuous might depend on a third variable (“Fv,” or involvement) in a continuous way.

Another point to make is that, in general, both categorical/discrete and dimen-sional/continuous patterns occur in complex systems. However, there is a hierarchy here. Phase transitions or bifurcations demarcate qualitatively different dynamical regimens in systems. These regimens are the types or categories. Quantitative continu-ous variation may exist within regimens. Thus, we have to look for typologies first and then, within types, we may apply standard quantitative analyses. A clear example is the butterfly’s life cycle. We have to distinguish between the different stages (egg, caterpillar, pupa, adult) before doing quantitative analysis on, say, length and weight.

This point has implications for the way we analyze developmental data. We should first look for clusters or classes with techniques such as latent class analysis, finite mixture modeling, or cluster analysis. Within these classes or clusters we can do item response modeling, factor analysis, etc. Ideally, we do both analyses in one step in latent structure models in which dimensional analyses are nested within classes. One such technique is mixture item response theory (IRT) modeling (Rost, 1990) for discrete data. Here we present an example for continuous data using mixture factor models.

Dolan and van der Maas (1998) take a usual conservation task as example. In this so- called conservation anticipation task, water is poured from a filled glass into an empty glass with different dimensions (smaller or wider glass; see Figure 10.7). Children have to indicate how high the water level will rise in the empty glass. This level is measured in millimeters. Nonconservers are expected to predict a level equal to the level in the filled glass, since they ignore the differences in width of the two glasses. Conservers do note this difference and estimate a different level. In Dolan and van der Maas these predicted levels are modeled with a finite mixture model with two normal distributions as components, representing the conservers and nonconservers. Over sets of items nonconservers make small unsystematic errors in aligning the level



to the level of the filled glass. The correlations between items within this group are roughly zero. Conservers that overestimate (underestimate) on one item tend to over-estimate (underestimate) the level on other items too. This bias can be described with a one- factor model. The complete model (mixture with factor models) can be estimated simultaneously (e.g., Lubke & Muthén, 2005). Note that this statistical analysis fol-lows the complex system idea of continuous traits within qualitative types or regimens.

Measuring Complex Systems: Math Garden

The last topic we discuss concerns data collection for the study of complex develop-mental systems. Again we seek inspiration in other disciplines. A quick assessment of complex system research in other fields shows us that researchers in these fields put much effort into collecting high- quality, high- frequency data. An extreme case is the study of stock markets. The data in this field are collected with amazing sampling rates (e.g., nanosecond trading). But also in ecosystem research, climate research, biol-ogy, physiology, etc., time- intensive data collection is essential. In longitudinal studies in developmental or educational research data are quite different, usually with sample rates of one or twice per year. We asked ourselves if it were possible to collect daily data on many children over long periods.

In an effort to acquire such data, we first tried to analyze the exercise books and tests used in standard educational methods. However, this task turned out to be impractical, tedious, and ultimately unsatisfactory. In 2007 we developed a web-based adaptive training and monitoring system for primary education to collect time- intensive data. This system is called Rekentuin in the Netherlands, which means Math Garden (see www.mathsgarden.com for an English version).

In this system children play different cognitive games based on either scholas-tic abilities or more abstract reasoning skills. Games consist of items that should be

Measuring Complex Systems: Math Garden

fIGURe 10.7. An example of a mixture factor modeling. Children have to indicate the level of water in the empty glass after pouring from the filled glass. The distribution of indicated levels is bimodal (a mixture of two normal distributions). Within each mode we can do standard factor analysis. In the nonconserver mode correlations are zero, but in the conserver mode a one- factor model describes the choice of levels over items.

item 2item 1 distribution

r > 0

r = 0

f f

?



answered within a limited amount of time (Klinkenberg, Straatmeier, & van der Maas, 2011). Currently, most children use Math Garden as an additional learning tool, but as development of the system continues, future versions may replace, and expand upon, traditional exercise books. By framing exercises in arithmetic and other topics as games and including direct feedback, Math Garden provides an engaging and reward-ing platform that greatly enhances children’s motivation to train their abilities. One indication of the popularity of Math Garden is the large proportion of item responses (about 25%) that are generated after school. The immediate feedback provided to the children frees the teachers from correcting the children’s exercise books. Teachers are also provided with detailed information about the progress and ability of children, including the errors they make. They can use this information, which is accumulated over the time that the child spends on the system, to optimize (individual) instruction.

Furthermore, Math Garden provides researchers with an invaluable dataset. The high- frequency data of a large number of children makes it possible to investigate fundamental questions about the dynamics of cognitive development. These data stem from a subgroup of children who visit Math Garden almost daily and play for extended periods. The data provided by these children are rich in quantity and in dynamics, as the following graph of a child solving the arithmetic problem “21 divided by 3” illustrates (Figure 10.8). The child starts by clicking the question mark (don’t know answer) and occasional incorrect answers, followed by a sequence of correct responses that first increases and later decreases in response time. This sequence clearly indicates different phases of learning.

Psychometrics of the Math Garden

The basis of Math Garden is an extension of classic computerized adaptive testing (CAT) methods. CAT is a testing method based on IRT, which contains a variety of item response models. Generally, these models assume one- dimensionality and con-ditional independence. The method used by Math Garden relies on the simplest item response model, the 1-PL or Rasch model.

In CAT, the order of presentation of items depends on responses to previous items provided by the testee (Wainer, 2000): If the immediately preceding response is correct

fIGURe 10.8. The x-axis displays the week of playing, the y-axis, the response time. The answer “q” means that the child selected the “I don’t know” answer option. Initially, the child quickly selects this answer. Halfway (week 42), the child learns the correct answer and starts to respond with highly fluctuating response times. This is followed by a period with quick and generally correct responses.



(incorrect), a more (less) difficult item is presented next. The advantage of using CAT is that abilities, such as arithmetic ability, can be estimated using fewer items than in standard tests. Currently, CAT is primarily used for testing, but in Math Garden it is used for testing and training at the same time. Therefore Math Garden uses an extended CAT technique based on two crucial innovations described here.

First, Math Garden uses a new self- organizing system that incorporates an “on the fly” Elo estimation algorithm (Klinkenberg et al., 2011), which originated in chess competitions (Elo, 1978). Elo estimation provides a self- organizing testing system in which both the ability estimates of children and the difficulty estimates of items are continually updated in real time, based on the responses of the children. The reliability of the Elo estimation system is well analyzed analytically and in simulations (Batchel-der & Bershad, 1979; Glickman, 2001). The most prominent advantage of this system is that it does not require pretested items, as in normal CAT. This requirement of pretested items makes standard CAT very expensive and applicable only in large-scale educational testing applications.

The second crucial innovation is the use of both accuracy and response time when updating ability and difficulty estimates by means of a new scoring rule (Klinkenberg et al., 2011; Maris & van der Maas, 2012). This new rule elicits important additional information about the ability of the child and renders the whole computerized proce-dure more game-like in practice. In Math Garden, items usually have a time limit of 20 seconds. In the scoring rule that is applied, the score equals the remaining time (RT; 20 seconds minus RT) in case of a correct response, but equals –1 times the response time if the response is incorrect. As a consequence, guessing is risky, and if a child has no clue about the answer, he or she can best refrain from responding, which provides a score of zero. This scoring rule is presented visually, such that even very young chil-dren can understand it. This new scoring rule has two important advantages. First, it solves the notorious speed– accuracy tradeoff problem (Wickelgren, 1977) since sub-jects now know how speed and accuracy are weighted in the scoring. Second, Maris and van der Maas (2012) have shown that under certain mild statistical assumptions, this scoring rule implies a standard two- parameter IRT model, in which discrimina-tion is a linear function of the time limit of an item. Therefore, already a lot is known about the model properties; for example, about the marginal and conditional distribu-tions of the model estimates.

Math Garden is very successful. Currently (July, 2013), it is used by more than 800 schools and many private families. In total over 100,000 Dutch children use Math Garden. We collected more than 200 million responses in 4 years.

Conclusions

In this chapter we intended to reveal some new research fields in developmental psy-chology based on the idea that the developing psychological system is a typical com-plex system. In fact, it may be the most complex system studied in science. Recognition of this fact helps us find new ways to model, analyze, and measure developmental

Conclusions



systems. We did not attempt to give an overview of all new options for research. Instead we gave examples from our own work in each of these areas (modeling, analy-sis, measurement). For each of these examples we expect a bright future. The network modeling that we applied to intelligence has great potential in other psychological fields— for instance, in the modeling of psychological disorders (Cramer et al., 2010). Phase transition research has a great promise, especially since much better tools have become available (Grasman et al., 2009). Finally, we expect many results from our Math Garden system. Analyzing the invaluable dataset provided by this system is chal-lenging, but the richness of data is amazing.

RefeRences

Ackerman, P. L., Beier, M. E., & Boyle, M. O. (2005). Working memory and intelligence: The same or different constructs? Psychological Bulletin, 131, 30–60.

Ackerman, P. L., & Lohman, D. F. (2003). Education and g. In H. Nyborg (Ed.), The scientific study of general intelligence (pp. 275–292). Amsterdam: Pergamon Press.

Bartels, M., Rietveld, J. H., van Baal, G. C. M., & Boomsma, D. I. (2002). Genetic and envi-ronmental influences on the development of intelligence. Behavior Genetics, 32, 236–249.

Bartholomew, D. J., Deary, I. J., & Lawn, M. (2009). A new lease of life for Thomson’s bonds model of intelligence. Psychological Review, 116, 567–579.

Batchelder, W. H., & Bershad, N. J. (1979). The statistical analysis of a Thurstonian model for rating chess players. Journal of Mathematical Psychology, 19, 39–60.

Brainerd, C. J. (1978). The stage question in cognitive- developmental theory. Behavioral and Brain Sciences, 2, 173–213.

Cannon, T. D., & Keller, M. C. (2005). Endophenotypes in genetic analyses of mental disor-ders. Annual Review of Clinical Psychology, 2, 267–290.

Cattell, R. B. (1971). Abilities: Their structure, growth, and action. Oxford, UK: Houghton Mifflin.

Cobb, L. (1980). Estimation theory for the cusp catastrophe model. Proceedings of the Section on Survey Research Methods, pp. 772–776.

Cramer, A. O. J., Waldorp, L. J., van der Maas, H. L. J., & Borsboom, D. (2010). Comorbidity: A network perspective. Behavioral and Brain Sciences, 33, 137–150.

De Boeck, P., Wilson, M., & Acton, G. (2005). A conceptual and psychometric framework for distinguishing categories and dimensions. Psychological Review, 112(1), 129–158.

Dolan, C. V., & van der Maas, H. L. J. (1998). Fitting multivariate normal mixtures subject to structural equation modeling. Psychometrika, 63, 227–253.

Dutilh, G., Wagenmakers, E.-J., Visser, I., & van der Maas, H. L. J. (2010). A phase transition model for the speed– accuracy trade-off in response time experiments. Cognitive Science, 34, 211–250.

Elo, A. (1978). The rating of chessplayers, past and present. Georgetown, CT: Arco.Fisher, C., Hall, D. J., Rakowitz, S., & Gleitman, L. (1994). When it is better to receive than

to give: Syntactic and conceptual constraints on vocabulary growth. In L. Gleitman & B. Landua (Eds.), The acquisition of the lexicon (pp. 333–375). Cambridge, MA: MIT Press.

Flavell, J. H., & Wohlwill, J. E. (1969). Formal and functional aspects of cognitive develop-ment. In D. Elkind & J. H. Flavell (Eds.), Studies in cognitive development: Essay in honor of Jean Piaget (pp. 67–120). New York: Oxford University Press.



Gilmore, R. (1981). Catastrophe theory for scientists and engineers. New York: Wiley.Glickman, M. E. (2001). Dynamic paired comparison models with stochastic variances. Jour-

nal of Applied Statistics, 28, 673–689.Grasman, R. P. P. P., van der Maas, H. L. J., & Wagenmakers, E.-J. (2009). Fitting the cusp

catastrophe in R: A cusp- package primer. Journal of Statistical Software, 32, 1–28.Jansen, B. R. J., & van der Maas, H. L. J. (2001). Evidence for the phase transition from rule I

to rule II on the balance scale task. Developmental Review, 21, 450–494.Jensen, A. R. (1973). Educability and group differences. New York: Harper & Row.Jensen, A. R. (1998). The g factor: The science of mental ability. Westport, CT: Praeger/Green-

woodKan, K.-J. (2011). The nature of nurture: The role of gene– environment interplay in develop-

ment. Unpublished doctoral dissertation, University of Amsterdam.Kan, K.-J., Kievit, R. A., Dolan, C. V., & van der Maas, H. L. J. (2011). On the interpretation

of the CHC factor Gc. Intelligence, 39(5), 292–302.Kan, K.-J., Wicherts, J. M., Dolan, C. V., & van der Maas, H. L. J. (in press). On the nature

and nurture of intelligence and specific cognitive abilities: The more heritable, the more culture- dependent? Psychological Science.

Klinkenberg, S., Straatemeier, M., & van der Maas, H. L. J. (2011). Computer adaptive prac-tice of maths ability using a new item response model for on the fly ability and difficulty estimation. Computers and Education, 57, 1813–1824.

Lubke, G. H., & Muthén, B. O. (2005). Investigating population heterogeneity with factor mixture models. Psychological Methods, 10, 21–39.

Luciano, M., Posthuma, D., Wright, M. J., de Geus, E. J. C., Smith, G. A., & Geffen, G. M. (2005). Perceptual speed does not cause intelligence, and intelligence does not cause per-ceptual speed. Biological Psychology, 70, 1–8.

Lumsden, J. (1976). Test theory. Annual Review of Psychology, 27, 251–280.Maris, G., & van der Maas, H. L. J. (2012). Speed- accuracy response models: Scoring rules

based on response time and accuracy. Psychometrika, 77(4), 615–633.Molenaar, P. C. M., & Raijmakers, M. E. J. (2000). A causal interpretation of Piaget’s theory of

cognitive development: Reflections on the relationship between epigenesis and nonlinear dynamics. New Ideas in Psychology, 18, 41–55.

Murray, J. D. (2002). Mathematical biology: I. An introduction (3rd ed.). Berlin: Springer Verlag.

Piaget, J. (1997) Development and learning. In M. Gauvain & M. Cole (Eds.), Readings on the development of children (2nd ed., pp. 19–28). New York: Freeman. (Original work published 1964)

Pinker, S. (1994). How could a child use verb syntax to learn verb semantics? In L. Gleitman & B. Landua (Eds.), The acquisition of the lexicon (pp. 377–410). Cambridge, MA: MIT Press.

Ploeger, A., van der Maas, H. L. J., & Hartelman, P. A. I. (2002). Stochastic catastrophe analy-sis of switches in the perception of apparent motion. Psychonomic Bulletin and Review, 9(1), 26–42.

Rost, J. (1990). Rasch models in latent classes: An integration of two approaches to item analy-sis. Applied Psychological Measurement, 14, 271–282.

Siegler, R. S., & Alibali, M. W. (2005). Children’s thinking (4th ed.). Upper Saddle River, NJ: Prentice Hall.

Spearman, C. (1904). General intelligence, objectively determined and measured. American Journal of Psychology, 15, 201–293.



Spearman, C. (1927). The abilities of man: Their nature and measurement. New York: Mac-millan.

Thorndike, R. L. (1927). The measurement of intelligence. New York: Teachers College.van der Maas, H. L. J., Dolan, C. V., Grasman, R. P. P. P., Wicherts, J. M., Huizenga, H. M.,

& Raijmakers, M. E. J. (2006). A dynamical model of general intelligence: The positive manifold of intelligence by mutualism. Psychological Review, 113, 842–861.

van Geert, P. (1991). A dynamic systems model of cognitive and language growth. Psychologi-cal Review, 98, 3–53.

van Geert, P. (1994). Dynamic systems of development: Change between complexity and chaos. New York: Harvester.

Waddington, C. H. (1966). New patterns in genetics and development. New York: Columbia University Press.

Wainer, H. (Ed.). (2000). Computerized adaptive testing: A primer (2nd ed.). Mahwah, NJ: Erlbaum.

Wickelgren, W. A. (1977). Speed– accuracy trade-off and information processing dynamics. Acta Psychologica, 41, 67–85.

Wohlwill, J. F. (1973). The study of behavioral development. New York: Academic Press.


chapter 10 - nederlands tweelingen register270 chapter 10 dynamics of development a complex systems...

Documents