the role of neuroimaging in the discovery of processing stages a review

& -z __ EB

ELSEVIER

acts psychologica

Acts Psychologica 90 (1995) 63-79

The role of neuroimaging in the discovery of processing stages

A review

G. Mulder a,*, A.A. Wijers ‘, J.J. Lange a, B.M. Buijink a, L.J.M. Mulder a, A.T.M. Willemsen b, A.M.J. Paans b

a Institute for Experimental and Work Psychology, Uniuersity of Groningen, Grote Kruisstraat 2 /I, 9712 TS Groningen, The Netherlands

b PET Centre, Uniuersity Hospital, Uniuersity of Groningen, 9712 TS Groningen, The Netherlands

Abstract

In this contribution we show how neuroimaging methods can augment behavioural methods to discover processing stages. Event Related Brain Potentials (ERPs), Brain Electrical Source Analysis (BESA) and regional changes in cerebral blood flow (rCBF) do not necessarily require behavioural responses. With the aid of rCBF we are able to discover several cortical and subcortical brain systems (processors) active in selective attention and memory search tasks. BESA describes cortical activity with high temporal resolution in terms of a limited number of neural generators within these brain systems. The combination of behavioural methods and neuroimaging provides a picture of the functional architecture of the brain. The review is organized around three processors: the Visual, Cognitive and Manual Motor Processors.

1. Introduction

In 1969, Saul Sternberg (Sternberg, 1969) introduced the additive factor method for interpreting reaction-time data from factorial experiment% Additivity of experimental effects suggests that the underlying mechanisms are independent and serially arranged. Sternberg has shown strong additivity of variables in a binary decision memory scanning task. Equally Sanders has presented impressive evi-

?? Corresponding author. E-mail: [email protected].

OOOl-6918/95/$09.50 0 1995 Elsevier Science B.V. Al1 rights reserved SSDI OOOl-6918(95)00035-6

64 G. Mulder et al. /Acts Psychologica 90 (1995) 63-79

dence for additivity of the effects of task variables in choice reaction time tasks (Sanders, 1980,199O). Sanders (1990) identified at least seven stages of processing: Preprocessing (typical task variable: retina1 lotus); feature extraction (typical task variables: signal quality and discriminability), identificution (typical task variable: stimulus orientation), response selection (typical task variable: response compatibil- ity), motor programming (typical task variables: movement direction and velocity), program loading (typical task variable: relative signal frequency) and motor adjustment (typical task variables: response specificity and fore period duration). Meyer et al. (1995) discuss a new theoretical framework, the EPIC (Executive/Interactive Control) architecture with specific modules for perceptual (e.g. visual), cognitive and motor processing. The first two stages (preprocessing, feature extraction) are parts of the visual processor of EPIC’s architecture; the stage of identification is a part of the cognitive processor and the last three stages make up the motor processor.

A century before Sternberg’s introduction of the additive factor method, Don- ders (1868/1969) had introduced the subtraction method. This method assumes strong additivity. RT in simple versions of a task is subtracted from RT in a more complex version and the differente in RT is believed to reflect the duration of the additional stages engaged in the complex version.

Strong additivity, however, is unlikely. As Roberts and Sternberg (1994) argue, nothing in the anatomy or physiology of the brain leads one to expect such simplicity. But something implausible is apparently true: the vertebrate brain has a simple functional structure in a wide range of situations as is shown by the results of both Sanders (1990) and Sternberg (1969). Stage models of human information processing are typical examples of functional architectures (Pelyshyn, 1984). An- other example of a functional architecture is the EPIC architecture (Meyer et al., 1995) and we shall use its different “components” throughout this contribution. Neuroimaging methods may aid at identifying physiological and anatomical “component? of the nervous system that have “something” to do with information processing. In addition they may provide converging evidente for the plausibility for postulating stages of information processing (Churchland and Sejnowski, 1992). Of course it would be convenient if we could understand the nature of human information processing without understanding the nature of the brain itself. It is however impossible to theorize effectively on these matters in absente of neurobio- logica1 constraints. The primary reason is that the computational space is enor- mous, and there are too many conceivable solutions. Neuroimaging methods provide alternative or supplementary ways for the discovery of “components” and of the nature of processing operations in the brain. A well-known neuroimaging method suitable for the discovery of processing stages is based on the recording of the ongoing electroencephalogram (EEG) during the performance of information processing tasks (e.g. Mulder et al., 1984). The events (external, exogeneous or internal, endogeneous) produce ERPs (Event Related Brain Potentials), voltage fluctuations that reflect postsynaptic potentials generated in large populations of cortical neurons that are activated in synchrony during information processing. The averaged ERP waveform consists of a sequence of positive and negative

G. Mulder et al. /Acts Psychologica 90 (1995) 63-79 65

“components”. The latency of each indicates the time course of neuronal activity and the scalp distribution suggests the neuroanatomical source. The functional significante of ERP components has been investigated by systematically studying the effects of experimental factors on their latency and amplitude. ERP components can be defined in two broad classes: exogeneous and endogeneous. Exoge- neous components are obligatory brain responses to any stimulus in a certain modality. Their latencies and topographies are dependent on physical characteristics such as location, colour, spatial frequency, pitch, and contrast. Endogeneous components are dependent on the task and the instructions given to the subject. From experimental studies it appears that a very early component in the visual ERP, NP80 (onset at 50 to 60 ms and a negative polarity for upper field stimuli and a positive polarity for lower field stimuli (i.e. being sensitive to retina1 lotus) shows characteristics consistent with a striate cortex generator and the processing stage “preprocessing ” (visual processor) (Mangun et al., 1993). Pl (peaking between 90 and 140 ms), anterior Nl (160-190 ms), posterior Nl (ca 175 ms) can be related to the processing stage “feature extraction” (visual processor). Pl is unaffected by retina1 location and source localization (sec below) reveals a source in the contralateral occipital scalp with a maximum situated about 6 cm lateral to the midsagittal plane (Mangun et al., 1993), suggesting that Pl is not generated in the striate cortex. It is suggested that Pl reflects modulation of the information flow along the ventral (“what”) pathway encoding form, colour and pattern. Nl, on the other hand, is believed to reflect the modulation of processing in the dorsal (“where”) pathway encoding the spatial properties of the stimulus (Desimone and Ungeleider, 1989; Mangun et al., 1993). Slow motor related potentials (Readiness Potential, RP, Deecke et al., 1976; Deecke and Kornhuber, 1977,1978) and Motor Potential (MP) and their lateralization (Lateralized Readiness Potential (LRP) in combination with the recording of the Electromyogram (EMG) provide on line information about the stages “motor programming”, “program loading” and “motor adjustment” (motor processor) (see Coles, 1989; and Coles et al., 1995; Mulder et al., 1993; and Smid and Mulder, 1995). Some variable time before movement onset the RP starts to lateralize, with larger amplitudes above the hemisphere contralateral to the side of the activated response (Kutas and Donchin, 1980). This lateralization, however, is confounded by processing and structural asymmetries, which are assumed to be equal for left- and right-hand responses. Thus lateralization of the RP can be corrected for nonmotor asymmetries by subtracting right-hand asymmetries from left-hand asymmetries, resulting in a differente potential consisting of “pure” motor-related activity. This differente potential is termed the LRP, the Lateralized Readiness Potential. Osman et al. (1992) presented evidente that lateralization can occur to a multi-attribute NO-GO stimulus, of which one attribute is associated with a response and the other with the inhibition of that response. The LRP can also occur in the absente of peripheral activation. It has been shown that partial information about the stimulus (letter name, letter features, colour, shape, or stimulus location) can already start the LRP before stimulus processing has been completed. In other words, there is partial information transfer between the Kwal and the Manual Motor Processor (sec Mulder et


al., 1994; and Smid, 1993, for a review). The LRP (Coles and Gratton, 1986; DeJong et al., 1988) and EMG can be used as indices of centra1 response activation and peripheral response activation respectively. ERPs with later latencies appear to reflect operations between stimulus and response processing. ERPs elicited by words contain a negative component that peaks about 400 ms after stimulus onset (N400), and that is assumed to reflect a process of integration of identified words with semantic or other contextual information (Brown and Ha- goort, 1993). NO-GO response decisions show a frontocentrally distributed “P300” component and GO response decisions show a later, more parietal central, “P300”. These components could originate from the hypothetical stage “binary decision”. Finally slow negative shifts have been identified associated with memory search operations (Okita et al., 1985), mental rotation (Wijers et al., 1989b) and long term memory retrieval (Rösler et al., 1994). It is tempting to relate these negativities to the stages of identification (Sanders, 1990) or serial comparison (Sternberg, 1969, i.e. the cognitive processor).

2. The role of attention

Donders’ subtraction method is used to determine the effects of selective attention on these different components. By subtracting ERPs elicited by unattended stimuli from ERPs elicited by attended stimuli, it is possible to observe for example the early effects of spatial attention on the Pl, Nl and N2 components (Hillyard et al., 1994; Mulder et al., 1994). The amplitude of these components is increased if the subjects attention was directed to the location of the stimulus. Attending to colour, on the other hand, results in a more anterior enhanced positivity in the 150-170 ms range and in a more posterior prolonged “selection- negativity”, starting at the rising flank of the Nl with an onset latency of about 170 ms (Hillyard and Münte, 1984; Wijers et al., 1989a). This differente in time and place between spatial and nonspatial attention could correspond to the differente between the dorsal (“where”) and the ventral (“what”) route (Ungeleider and Mishkin, 1982) dealing respectively with spatiul and object properties of a stimulus. If subjects are required to selectively attend either to the spatial frequency of a stimulus (low 0.8 deg/c or high 3.2 deg/c) or to the orientation of the bars (vertical VS horizontal), it appears that the feature “spatial frequency” is earlier available (about 120 ms after stimulus onset) than the feature “orientation” (about 150 ms). The effect of attention to a conjunction of frequency and orientation appears later, about 250 ms (N2b). The selective processing of orientation appears to be hierarchically dependent on spatial frequency, i.e. the relevante of orientation has no consequente for the processing of irrelevant spatial frequenties (Wijers et al., in press). A similar observation can be made if stimuli differ in colour (red or blue, i.e. “what”) and location (left or right visual hemifield, i.e. “where”). The processing of the relevante of colour is hierarchically dependent on the relevante of the location, suggesting that spatial attention is a prerequisite for nonspatial


selections. However, the early ERP-effect related to colour also suggested that selection by position may not be the only selection mechanism. This early effect may reflect an increased sensitivity of neurons for task relevant colours (Motter, 1994a,b). The time course of the attention-specific response of such neurons in the fusiform/lingual gyrus, V4 in humans, closely corresponds to the onset-latency of the colour specific ERP-effect (i.e. with an onset latency of about 150 ms). The effects of attending to location and colour interact in the 250-300 ms range (N2b). The more centrally located negativity in this range primarily occurs when both feature levels are relevant. The N2b is considered to reflect a general, feature independent attention process since it is always present in the ERPs to attended stimuli (Wijers, 1989; Mulder and Wijers, 1991; Mulder et al., 1994). In addition, stimuli to be ignored (unattended targets and unattended nontargets), for instance those appearing on an irrelevant location or having an irrelevant colour, do not elicit the late negativities associated with memory load or mental rotation or elicit only its initial phase (Wijers et al., 1989a; see also Näätänen, 1992). Al1 these observations are consistent with early selection theories. With the aid of neuroimaging methods the underlying sources of these components can be discovered in time and space. Brain electrical source analysis (BESA) aims at the discovery of a limited set of neural generators underlying a set scalp recorded wave forms (sec Näätänen, 1992; Scherg and Picton, 1991). With regional cerebral bloodflow (rCBF) we visualize and compute the extra demand on glucose and oxygen required if a task contains an extra cognitive component (stage). In contrast to BESA, rCBF does not provide information over the time course of activation of brain systems (Roland et al., 1995).

3. Imaging the visual processor

In a recent experiment (Lange et al., in preparation) grating stimuli in the colour red or blue were presented randomly to the left or the right of a fiiation point. Subjects were instructed to attend to a specific location, colour, or conjunction of location and colour, and responded to the presentation of target gratings in the attended category. Selectively attending the location or the colour did not affect the first prominent deflection in the ERP. We believe that this component is comparable to the NP80 reported by Mangun et al., 1993) (sec Introduction). BESA of this component resulted in an equivalent dipole that was located anterior to the 01/2 electrode, which according to Steinmetz et al. (19891, roughly corresponds to the leve1 of the calcarine sulcus. (See Fig. 1.) Note that the position of the source is dependent on retinal location (LVF VS RVF).

Selective attention affected the ERPs after this initial, presumably striate, component. As a result of attending the location at which the stimulus was presented, both the contralateral and ipsilateral Pl amplitudes were enhanced. Furthermore, in the conjunction task, the Nl amplitude was enhanced. And finally, a centra1 N2b negativity was elicited. Attending the colour of the stimulus


Fig. 1. Topographical and source analysis of the NP80 component in the average ERP (averaged over al1 tasks and stimulus categories, separately for LVF and RVF stimuli). The isopotential maps in the left column show the scalp distribution of the component, as viewed from above, with the frontal scalp at the top of the map. Grey areas indicate scalp negativity, white is scalp positivity. Isopotential line spacing is 0.4 pV. The results of source analysis are shown in three projections, from above, from the back, and from the right side of the head. The top row shows the results for the NP80 in the ERPs elicited by LVF stimuli, and the bottom row shows the results for the NlOO elicited by RVF stimuli. Source analysis was performed at 100 ms. RVs were 10.3% for the LVF stimuli, and 9.22% for the RVF stimuli.

elicited a smal1 parietal positivity overlapping in time with the negative going flank of the Pl, a contralateral occipitotemporal negativity in the Nl interval, and a centra1 N2b. In the attend conjunction task, the ERP effects of attending stimulus colour were only present for stimuli presented at the attended location. This is a replication of the findings of Hillyard and Miinte (19841, and suggests that in a conjunction paradigm colour selection is hierarchically dependent on location selection. Furthermore, in the attend location task the N2b was only elicited for stimuli presented at the attended location in the attended colour, i.e. the attended conjunction. Interestingly, BESA of the colour selection positivity at 150 ms ( see also Introduction) yielded a dipole in the basal occipital cortex. This concurs with PET and single cel1 studies that have demonstrated the importante of this human V4 area for colour perception (Zeki, 1990) and selective attention to colour (Motter, 1994a,b; Corbetta et al., 1990).

The N2b negativities elicited by the attended location, attended colour, and attended conjunction stimuli are quite comparable in their timing and scalp distribution. BESA indicated that a dipole in deep frontal brain areas, oriented in the direction of the centra1 scalp, explains the N2b reasonably well. (Sec Fig. 2.)

We hypothesize that the N2b component might be generated in the anterior cingulate cortex. The anterior cingulate has been shown to be involved in genera1 attentional processes, target detection, and motor processes (Posner and Petersen, 1990; Paus et al., 1993). This suggestion, however, has to be considered as

G. Mulder et al. /Acts Psychologica 90 (1995) 63-79

Fig. 2. Topographical and source analysis of the N2b component in the average selection wave (averaged over location selection, colour selection, and conjunction selection, separately for LVF and RVF stimuli). The top row shows the results for LVF stimuli, and the bottom row shows the results for RVF stimuli. Isopotential line spacing is 0.2 FV. Source analysis was performed at 260 ms. Residual variances were 8.68% for the LVF stimuli. and 13.7% for the RVF stimuli.

extremely tentative, and consequently needs further confirmation with other brain imaging techniques, such as rCBF.

The next study was an attempt to further explore this suggestion. In this study we used rCBF and the subjects were instructed to attend to one type of stimulus and detect target gratings within the attended category, ignoring al1 other targets and nontargets. Target detection had to be shown with a button-press with the left hand. In four different conditions subjects attended to: (1) gratings to the left of fiiation, (2) gratings to the right, (3) red gratings, (4) blue gratings. In a fifth condition subjects passively fixated the screen. In a first comparison increases in rCBF in al1 attention conditions were tested against the fixation-only condition. This comparison showed significantly enhanced blood flow in the cerebellum cl), in the anterior cingulate gyrus (21, supplementary motor area (SMA) (3), the prefrontal areas in both hemispheres (41, the thalamus (5), the basal ganglia (6), the motor cortex in the right hemisphere (71, the premotor cortex in both hemispheres (8), and the lateral inferior parietal cortex (9), area 40 bilaterally (sec Fig. 3).

Further comparisons looked for changes in blood flow between the different attention conditions. The attend-right condition was subtracted from the attend-left condition to reveal those brain areas that show lateralized activity as a function of the direction of attention. Two regions in the right hemisphere showed increased bloodflow, first a posterior, inferior extrastriate occipital area in the fusiform/lingual gyrus, and second a prefrontal area. Almost mirror-symmetrie activation’s were found in the left hemisphere resulting from the attend-right minus attend left subtraction, but the occipital activation failed statistical significante.

G. Mulder et al. /Acts Psychologica 90 (1995) 63-79


To determine the brain areas involved in attending to colour, both attend location conditions (attend-left and attend-right) were subtracted from the two attend colour conditions (attend-red and attend-blue). This subtraction showed increased bloodflow in the right parahippocampal gyrus, in the posterior cingulate gyrus and media1 cuneus/precuneus, and in the left inferior frontal cortex. Thus, the rCBF data refine our earlier conclusions: different parts of the occipital brain are involved in attending to location (fusiform gyrus) and attending to colour (parahippocampal gyrus). In a recent study Heinze et al. (1994) also report activation in the fusiform gyrus with spatial attention and they presented further evidente that Pl most probably is generated from this brain area. This is again evidente for the involvement of the extrastriate ventral projection route. In al1 the attention tasks, i.e. independent of the particular selection cue, we observed activation of the anterior cingulate cortex in combination with activity in extensive areas in lateral prefrontal regions in both hemispheres. As mentioned above, in selective attention tasks the N2b is also always present, also independent of the particular selection cue. This ERP component could be a scalp signature of the activity in the anterior cingulate cortex. Corbetta et al. (1991) suggested that these areas are active in divided attention but not in focused attention, and participate in further processing and response selection when response decisions cannot be made from output from posterior processing areas alone. The present data show that the anterior system also can be active in focused attention conditions. We suggest that this system depends on perceptual as wel1 as on motor-related factors. In Posner and Petersen’s theory of attention (Posner and Petersen, 19901, the anterior cingulate is the major component of the “anterior attentional” system, with “attention for action” as major function. The conditions that we used put a heavy load on response selection because of the fast rate at which (GO/NO-GO) response decisions had to be made. A striking finding concerned the increased blood flow of posterior cingulate cortex and cuneus/precuneus when attention was directed to colour instead of position. The precuneus has been found involved in spatial attention shifting (Corbetta et al., 1993). In the colour-conditions the position of relevant coloured stimuli is unknown (and task-irrelevant). We therefore tentatively suggest that the activity of this media1 posterior region reflects that spatial attention is in the divided mode before the stimulus is presented and has to “zoom in” on the stimulus after its presentation; this in contrast with the position-conditions in which spatial attention could constantly remain focused on the same position. This suggests that selection of an object based on a nonspatial attribute necessarily depends on attention being directed to its position, according to “position-special” theories of attention (Van der Heijden, 1992). The fact that this region was also more active in the fixation-only condition, in which the subjects were free to divide and move their spatial attention, might support this idea.

To summarize: This study learned that with rCBF it is possible to discover separate subsystems involved in the processing of colour and location, but the important subtraction of al1 attention conditions minus fixation-only revealed that at least nine brain systems are involved in these tasks and they should correspond to stages or systems of stages (e.g. processors).


4. Imaging the motor processor

One of the above-mentioned nine systems is the anterior cingulute: this system is activated in visual selective attention tasks (sec above, and Corbetta et al., 1991) but also in single word processing tasks (Petersen et al., 1988; Petersen and Fiez, 19931, in motor planning tasks (Deiber et al., 1991), and in tasks in which a prepotent habitual response has to be inhibited (Pardo et al., 19901, while its activation is modulated by practice-related effects (Raichle et al., 1994). It seems quite reasonable to hypothesize that this structure corresponds to the hypothesized stage of response selection, a stage assumed in many stage models of choice reaction time @anders, 1990). Several brain systems are related to hypothesized stages of Motor Programming, Program loading and Motor adjustment @anders, 1990). The activation of these different brain areas suggests that the Manual Motor Processor also consists of many different subsystems. Lesions in the Supple- mentary Motor Area (SMA) produce difficulties in self-initiated movements (Pas- singham et al., 1989). Animals with such lesions have difficulties in the programming of movements based on internal cues. rCBF studies of Roland et al. (1980) showed that the SMA bilaterally and the primary motor cortex unilaterally receive increased blood flow if the subjects have to perform a complex “motor sequence” task. If subjects are only required to mentally perform the task the bilaterally increased blood flow in the SMA remains. Simple motor tasks, tasks consisting of only a repetition of movements did not result in an increased blood flow in the SMA. A selective visual attention task is not a simple, overlearned task, and as a consequente we should expect SMA to be active in this type of tasks. Kosslyn and Koenig (1992) consider the SMA as the major component of the action progrum- ming subsystem (i.e. motor programming). Besides the SMA we found in the subtraction “al1 attention conditions minus fixation-only” an increased blood flow in the premotor area of both hemispheres. The premotor area is situated anterior to the primary motor area. It gets massive inputs from area 7b (parietal cortex). It seems that the neurons in this area compute the extrinsic spatial relation between target objects and the body, and these relations are translated in a pattern of proximal movements. Kosslyn and Koenig (1992) consider the premotor area as the neural implementation of the instruction generation subsystem (i.e. program loading). This brain area gets input from the anterior cingulate and has outputs to the prefrontal cortex (an area that also gets an increased blood flow) and the primary motor cortex. The premotor area seems to be especially involved in the production of new movement patterns. Roland (1982) did not find an increased blood flow in this area if movements were automatized or had only to be mentally performed. Finally, it is not surprising that we could observe an increased blood flow to the right motor cortex (the subjects had to respond with their left index finger on target stimuli). The right motor cortex has direct connections to the alpha motor neurons in the spinal cord and we observe here the activity of the movement execution subsystem (Kosslyn and Koenig, 1992) or what was mentioned motor adjustment by Sanders (1990). Deecke and Kornhuber (1977) and Deecke and Kornhuber, 1978) observed that the readiness potential (RP) (sec also introduc-


tion) has a timing that is closely related to the pattern of firing of SMA neurons. The potential is largest above the SMA. Deecke and Kornhuber (1977,1978) consider the RP as “real-time index of SMA activity”. Epicortical recordings (Ikeda et al., 1992) in man and magnetoencephalography (MEG) in a patient with a unilateral SMA lesion (Lang et al., 1991) show that the SMA is active at least during the early part of the RP. However, until now the literature is not definite on source localisation of the RP. Bötzel et al. (1993) in a recent study argued for the existente of at least three sources: bilateral sources for the early part of the RP, the activity of a source contralateral to the moving finger (late part of the RP), and a source wave peaking about 100 ms after EMG, reflecting reafferent processes in the postcentral gyrus. Böcker (19941, however, was unable to find any source for reafferent processes. In a recent study (Buijink, at al, in preparation) we attempted to model with BESA the brain activity preceding (- 1000 ms) and shortly after a button press response (+95 ms) in several movement tasks. We restrict the discussion now to uoluntury mouemenrs (subjects were asked to make voluntary movements with the left or the right finger at free choice). BESA showed that across subjects brain activity preceding and following voluntary movements could be most satisfactorily modelled with three dipoles, one dipole (1) in the contralateral motor cortex became significantly active before movement onset (interval - 1000 until - 775 and from - 351 ms on; the second dipole (10, also contralateral but now more posterior, shows significant activity after movement onset( 0 until +95 ms), while the last centrally located dipole (dipole 111) was active before movement onset (from -375 ms on>.

These three dipoles could be interpreted as follows: dipole 111, which is centrally located, is believed to signal the activity of SMA; dipole 1, unilateral and contralateral most probably signals the activity of the primary motor cortex, finally dipole 11 could represent reafferent feedback information. Dipole 11 reflects a part of the mouement rnonitoring subsystem (Kosslyn and Koenig, 1992). The movement monitoring subsystem is believed to receive input from perceptual systems about the position of a moving limb and output from the instruction generation subsystem about the expected trajectory. It is believed that a copy of the command to the muscles is sent back to the movement execution subsystem itself. The dipoles 1 and 11 are in close proximity. It seems that neurons in the primary motor cortex (area Ml) receive feedback from the muscles they activate. This dipole solution fits reasonably wel1 the brain activity observed in other movement tasks, except for GO/NO-GO tasks (Buijink et al., in preparation). In these tasks multidimensional stimuli are used. Some dimensions (e.g.letter name identity) are earlier available and could already start response activation, a later dimension (e.g. letter size) indicates that response activation should be interrupted (NO-GO) (sec Smid and Mulder, 1995). This type of NO-GO stimuli evokes a frontal negativity starting at 210 ms after the NO-GO stimulus, which could be modelled with one dipole in the frontal cortex. Until now the cerebral subsystem generating this component is not known. Note that about the same time the NO-GO-LRP is also aborted (Smid and Mulder, 1995). As soon as we move from voluntary movements to tasks with an

74 G. Mulder et al. /Acts Psychologica 90 í1995) 63-79

external stimulus to respond to, an’additional dipole appears at occipital locations. This dipole clearly represents the total activity of the Visual Processor.

5. Imaging the cognitive processor

In the last study to be reviewed we attempted to load and image the systems involved in the cognitive processor. Subjects were presented visual and memory search tasks designed after Shiffrin and Schneider (1977). Memory set size (MI was either 1 or 4 as was display size CD). Reaction time increased both with display load and memory load. The interaction between display load and memory load tended towards significante. The subtraction of the most difficult task (M4D4) from the easiest task (MlDl) shows that a statistically significant increase in blood flow occurs in four areas of the brain: the cerebellum (11, the superior tempora1 gyri of both hemispheres (area 21/22/42) (2), extrastriate occipital cortex in the left hemisphere (area 18) (3), and a prefrontal area in the left hemisphere (insular cortex/dorsolateral prefrontal cortex, area 45/47) (4). These results suggest that search operations in the working memory component of the cognitive processor involve a network of brain areas, with code-specific regions engaged in operations in the different working memory slave systems. The activity in the left occipital lobe (3) most probably reflects search operations in the hypothesized “visuospatial sketch pad” (Baddeley, 1992). Interestingly, the exact area being involved seems to depend on the nature of the memory task. Jonides et al. (1993) investigating memory for the spatial layout of a visual scene found increased activity in a more superior occipital area in the right hemisphere. Therefore, the organization of the visual memory system appears to be consistent with the traditional view of hemispheric specialization: spatial visual memory involving mainly the right hemisphere and visual memory of language-related material (letters) involving the left hemisphere. The operations involving the hypothesized phonological subsystem involve the superior tempora1 lobes of both hemispheres (2) (sec for converging evidente Jonides et al., 1993; and Paulesu et al., 1993). Area 22/42 in the left hemisphere corresponds to Wernicke’s area (Mayeux and Kandel, 1991). The participation of the prefrontal area (4) confirms observations made by Paulesu et al. (1993). These authors found bilateral activation of insular cortex at coordinates very closely to the area reported here. In addition they found also increased activity in Broca’s area. Together with the cerebellum these areas could constitute the articulatory loop, a subvocal rehearsal process. However, Jonides et al. (1993) used a visual memory task, not likely to involve subvocal rehearsal, and they too found increased activation in a prefrontal area of the right hemisphere, close to the area reported as insular cortex by Paulesu et al. and about mirror-symmetrie to the left prefrontal area activated in the present experiment. Jonides et al. (1993) regarded this area as equivalent to a brain area in monkeys in the vicinity of the principal sulcus. Animal research of Goldman-Rakic (1987) has shown that this area is important for visuo-spatial working memory. However, the prefrontal/insular cortex in genera1 is active in working memory tasks (possibly a reflection of the


activity of the hypothesized Genera1 Executive, Baddeley, 1992), while the hemispheric lateralization apparently depends on the specific working memory task involved. Remember that in the search tasks at least two processes are manipulated: the duration (1) of search operations in working memory should be affected by both display load and memory load. The process of subvocally rehearsing the memory set (2), on the other hand, should be mainly manipulated by the factor memory load. However, al1 the areas mentioned before show a systematic increase in rCBF from the easiest task to the most difficult one (i.e., MlDl < MlD4 < M4Dl < M4D4). This result strongly suggests that there is a close communication between the visual and phonological stores in al1 variants of the tasks and that these brain areas are more closely associated with search processes than with subvocal rehearsal. Another prefrontal region (area 44, Broca’s area), however, appeared to be more closely associated with subvocal rehearsal, since it showed increased rCBF as a function of memory load but not display load. Finally, these results emphasize that the classica1 view of the cerebellum serving a purely motor function probably seems to be unduly narrow.

6. Discussion and conclusions

While rCBF has a relative high spatial resolution of the activation or deactiva- tion of cortical and subcortical brain systems, ERPs and BESA, on the other hand, have a relative low spatial resolution but a high tempora1 resolution. In the near future high spatial sampling (128 recording sides), the simulataneous recording of the evoked magnetic fields (EMF’s, providing only the tangential component of the source), more realistic head models and more sophisticated source localization methods wil1 also improve the spatial resolution of these ERP-based source localization methods. Neuroimaging methods should mutually complement and constrain the solutions obtained by each of them. Our approach starts from functional architectures of human information processing (e.g. EPIC, Meyer at al., 1995) and their associated cognitive tasks. In this review we studied a selective attention task, a simple motor task (voluntary movements) and a combined visual and memory search task and we found strong evidente for modularity. In selective attention tasks we found the involvement of different modular brain systems within the visual processor. The availability and order of processing of spatial frequency, orientation, colour and location under the control of attention could be studied without the requirement of the subject to respond. The results clearly indicated that the stages preprocessing and feature extraction could be distinguished but that feature extraction consists of several submodules, differing in space and time. Attentional systems have a very early access to these modules. The primary role of spatial attention became in particular clear. During selective attention tasks requiring also overt responses, we found with the aid of rCBF nine active brain systems. In combination with behavioural models it was possible to generate hypotheses about their functional role. Brain imaging (rCBF and BESA) the motor processor shows the involvement of the anterior cingulate (response selection), the


supplementary motor area (motor p~ogrumming ), the primary motor cortex (motor execution) and the somatomotor cortex (response monitoting). The timing of the last three became available with BESA on a ms-to-ms base. LRP studies (sec Coles et al., 1995; Smid and Mulder, 1995) have shown that partial information from the visual processor can already activate the motor processor, emphasizing asyn- chronous information transfer between both processors. The scalp signature of the anterior cingulate is most probably visible at about 220 ms visible in the N2b component of the ERP. During memory tasks (imaging the cognitive processor) we found at least four different brain systems active providing converging evidente for the different slave systems of Baddeley’s working memory model. Visual search made use of the visual spatial sketch pad (extrastriate occipital cortex), the phonological store (superior tempora1 gyri), al1 under the control of the genera1 executive (insular/dorsolateral prefrontal cortex). Memory load in addition activated Broca’s area, suggesting a role of this brain area in subvocal rehearsal. Finally, the data emphasize the important role of the cerebellum in cognitive tasks.

Acknowledgements

We are grateful to Michael Coles, Jean Requin and Andries Sanders for valuable comments on an earlier version of this article.

References

Baddeley, A., 1992. Working memory: The interface between memory and cognition. Journal of Cognitive Neuroscience 4, 281-288.

Böcker, K.B.E., 1994. Spatiotemporal dipole models of slow cortical potentials. Ph.D. thesis, Tilburg University.

Bötzel, K., H. Plendl, W. Paulus and M. Scherg, 1993. Bereitschaftspotential: Is there a contribution of the supplementary motor area? Electroenceph. and Clin. Neurophysiology 89, 187-196.

Brown, C. and P. Hagoort, 1993. The processing nature of the N400: Evidente from masked priming. Journal of Cognitive Science 10.5, 34-44.

Buijink, B.M., A.A. Wijers and G. Mulder, in preparation. Modelling motor processes. A brain electrical source analysis approach.

Churchland, P.S. and T.J. Sejnowski, 1992. The computational brain. Cambridge, MA: MIT Press. Coles, M.G.H., 1989. Modern mind-reading: Psychophysiology, physiology and cognition. Psychophysi-

ology 26, 251-269. Coles, M.G.H., M.K. Scheffers and L. Fournier, 1995. Where did you go wrong? Errors, partial errors,

and the nature of human information processing. Acts Psychologica 90, 129-144 (this volume). Coles, M.G.H. and G. Gratton, 1986. ‘Cognitive psychophysiology and the study of states and

processes’. In: G.R. Hockey, A.W.K Gaillard and and M.G.H. Coles (eds.), Energetics and human information processing (pp. 409-424). Dordrecht: Martinus Nijhoff.

Corbetta, M., F.M. Miezin, S. Dobmeyer, G.L. Schulman and S.E. Petersen, 1990. Attentional modulation of neural processing of shape, color, and velocity in humans. Science 248, 1556-1559.

Corbetta, M., F.M. Miezin, S. Dobmeyer, G.L. Shulman and S.E. Petersen, 1991. Selective and divided attention during visual discriminations of shape, color, and speed: Functional anatomy by positron emission tomography. The Journal of Neuroscience 11, 2383-2402.

Corbetta, M., F.M. Miezin, G.L. Shulman and S.E. Petersen, 1993. A PET study of visuospatial attention. Journal of Neuroscience 13, 1202-1226.


Deecke, L., B. Goezinger and H.H. Kornhuber, 1976. Voluntary finger movements in man: Cerebral potentials and theory. Biological Cybernetics 23, 99-119.

Deecke, L. and H.H. Kornhuber, 1977. ‘Cerebral potentials and the initiation of voluntary movements’. In: J.E. Desmedt (ed.), Attention, voluntaty contraction and event-related cerebral potentials. Progress in clinical neurophysiology, Vol 1. (pp. 132-150). Basel: Karger.

Deecke, L. and H.H. Kornhuber, 1978. An electrical sign of participation of the mesial “supplementary” motor cortex in human voluntary finger movement. Brain Research 159, 473-476.

Deiber, M.P., R.E. Passingham, J.G. Colebatch, K.J. Friston, P.D. Nixon and R.S.J. Frackowiak, 1991. Cortical areas and the selection of movement: a study with positron emission tomography. Experi- mental Bram Research 159, 473-476.

DeJong, R., M. Wierda, G. Mulder and L.J.M. Mulder, 1988. The use of partial information in response preparation. Journal of Experimental Psychology: Human Perception and Performance 14, 682-692.

Desimone, R. and L.G. Ungeleider, 1989. ‘Neural mechanisms of visual processing in monkeys’. In: F. Boller and J. Grafman (eds.), Handbook of neuropsychology, Vol. 2 (pp. 267-299).

Donders, F.C., 1868/1969. On the speed of mental processes (W.G. Koster translation). Acts Psycho- logica 30, 412-431.

Goldman-Rakic, P.S., 1987. ‘Circuitry of primate prefrontal cortex and regulation of behavior by representational memory’. In: V.B. Mountcastle, F. Blum and S.R. Geiger (eds.), Handbook of physiology, Section 1: The nervous system, Vol V: Higher functions of the brain (pp. 373-417) Betshesda, MD: American Physiological Society.

Heinze, H.J., G.R. Mangun, W. Burchert, H. Hinrichs, M. Scholz, T.F. Münte, A. Gös, M. Scherg, S. Johannes, H. Hundeshagen, M.S. Gazzaniga and S.A. Hillyard, 1994. Combined spatial and tempora1 imaging of brain activity during visual selective attention in humans. Nature 372, 543-546.

Hillyard, J.C. and T.F. Münte, 1984. Selective attention to color and location: An anlysis with event related potentials. Perception and Psychophysics 36, 185-198.

Hillyard, S.A., S.J. Luck and G.R. Mangun, 1994. ‘The cuing of attention to visual field locations: Analysis with ERP recordings’. In: H.J. Heinze, T.F. Münte and G.R. Mangun teds.), Cognitive Electrophysiology (pp. 1-25). Boston, MA: Birkhäuser.

Jonides, J., E.E. Smith, R.A. Koeppe, E. Awh, S. Minoshima and M.A. Mintun, 1993. Spatial working memory in humans as revealed by PET. Nature 263, 623-625.

Ikeda, A., H.O. Liiders, A.C. Burgess and H. Shibasaki, 1992. Movement-related potentials recorded from supplementary motor area and primary motor area. Role of supplementary motor area in voluntary movements. Brain 115, 1017-1043.

Kosslyn, S.M. and 0. Koenig, 1992. Wet mind. New York: Free Press. Kutas, M. and E. Donchin, 1980. Preparation to respond as manifested by movement-related brain

potentials. Brain Research 202, 95-115. Lang, W., D. Cheyne, R. Kristeva, G. Lindinger and L. Deecke, 1991. ‘Functional localization of motor

processes in the human cortex’. In: C.H.M. Brunia, G. Mulder and M.N. Verbaten (eds.1, Event-related brain research. Electroencephalography and Clinical Neurophysiology (Suppl. 42) (pp.97-115). Amsterdam: Elsevier Science.

Lange, J.J., A.A. Wijers, G. Mulder and L.J.M. Mulder, in preparation. Multidimensional stimulus arrays: Effects of selectively attending location, color, or conjunction.

Mangun, G.R., S.A. Hillyard and S.J. Luck, 1993. ‘Electrocortical substrates of visual selective attention’. In: D.E. Meijer and S. Kornblum (eds.1, Attention and performance (pp. 219-2451. Hilldale, NJ: Erlbaum.

Mayeux, R. and E.R. Kandel, 1991. ‘Disorders of language: The aphasias’. In: E.R. Kandel, J.H. Schwartz and T.M. Jessell (eds.1, Principles of neural science. London: Prentice Hall.

Meyer, D.E., D.E. Kieras, E. Lauber, E. Schumacher, J. Glass, E. Zurbriggen, L. Gmeindl and D. Apfelblat, 1995. Adaptive executive control: Flexible multiple-task performance without pervasive immutable response-selection bottlenecks. Acts Psychologica 90, 163-190 (this volume).

Motter, B.C., 1994a. Neural correlates of feature selection for color or luminance in extrastriate area V4. The Journal of Neuroscience 14, 2190-2199.

Motter, B.C., 1994b. Neural correlates of feature selective memory and pop-out in extrastriate area V4. The Journal of Neuroscience 14, 2190-2199.


Motter, B.C., 1994b. Neural correlates of feature selective memory and pop-out in extrastriate area V4. The Journal of Neuroscience 14, 2190-2199.

Mulder, G., A.B.M. Gloerich, K.A. Brookhuis, H.J. van Dellen and L.J.M. Mulder, 1984. Stage analysis of the reaction process using brain-evoked potentials and reaction time. Psychological Research 46, 15-32.

Mulder, G., H.G.O.M. Smid and L.J.M. Mulder, 1993. ‘On the transfer of partial information between perception and action’. In: D.E. Meijer and S. Kornblum (eds.), Attention and performance XIV (pp. 567-588). Hilldale, NJ: Erlbaum.

Mulder, G. and A.A. Wijers, 1991. ‘Selective attention and mental chronometry’. In: C.H.M. Brunia, G.Mulder and M.N. Verbaten (eds.), Event-related brain research (pp. 228-243). Amsterdam: Elsevier Science.

Mulder, G., A.A. Wijers, K.A. Brookhuis, H.G.O.M. Smid and L.J.M. Mulder, 1994. ‘Selective visual attention: Selective cuing, cognitive processing, and response processing’. In: H.J. Heinze, T.F. Miinte and G.R. Mangun (eds.), Cognitive electrophysiology (pp. 26-80). Boston, MA: Birkhäuser.

Näätänen, R., 1992. Attention and brain function. Hillsdale, NJ: Erlbaum. Okita, T., A.A. Wijers, G. Mulder and L.J.M. Mulder, 1985. Memory search and visual spatial

attention: An event-related brain potential analysis. Acts Psychologica 60, 263-292. Osman, A., T.R. Bashore, M.G.H. Coles, E. Donchin and D.E. Meyer, 1992. On the transmission of

partial information: Inferences from movement-related brain potentials. Journal of Experimental Psychology: Human Perception and Performance 18, 217-232.

Pardo, J.V., P.J. Pardo, K.W. Janer and M.E. Raichle, 1990. The anterior cingulate cortex mediates processing selection in the Stroop attentional conflict paradigm. Proceedings of the National Academy of Sciences of the USA 87, 256-259.

Passingham, R.E., Y.C. Chen and D. Thaler, 1989. ‘Supplementary motor cortex and self-initiated movement’. In: M. Ito (ed.), Neural programming fpp.13-24). Basel: Karger.

Paulesu, E., C.D. Frith and R.S.J. Frackowiak, 1993. The neural correlates of the verbal component of working memory. Nature 362, 342-345.

Paus, T., M. Petrides, A.C. Evans and E. Meyer, 1993. Role of the human anterior cingulate in the control of ocular, manual, and speech responses: A positron emission tomography study. Joumal of Neurophysiology 70, 453-469.

Pelyshyn, Z.W., 1984. Computation and cognition: Toward a foundation for cognitive science. Cam- bridge, MA: MIT Press.

Petersen, S.E. and J.A. Fiez, 1993. The processing of single words studied with positron emission tomography. Annual Review of Neurosciences 16, 509-530.

Petersen, S.E., P.T. Fox, M.I. Posner, M.A. Mintun and M.F. Raichle, 1988. Positron emission tomographic studies of the cortical anatomy of single-word processing. Nature 331, 585-589.

Posner, M.I. and S.E. Petersen, 1990. The attention system of the human brain. Annual Review of Neurosciences 13, 25-42.

Raichle, M.E., J.A. Fiez, T.O. Videen, A.M.K. MacLeod, J.V. Pardo, P.T. Fox and S.E. Petersen, 1994. Practice-related changes in human brain functional anatomy during nonmotor learning. Cerebral Cortex 4, 8-26.

Roberts, S. and S. Sternberg, 1994. ‘The meaning of additive reaction-time effects: The tests of three alternatives’. In: D.E. Meyer and S. Kornblum (eds.), Attention and performance XIV. Cambridge. MA: MIT Press.

Roland, P.E., 1982. Cortical regulation of selective attention in man. A regional cerebral blood flow study. Journal of Neurophysiology 48, 1059-1077.

Roland, P.E., R. Kawashima, B. Gulvás and B. G’Sullivan, 1995. ‘Positron emission tomography in cognitive science: Methodological constraints, strategies, and examples from learning and memo@. In: M.G. Gazzaniga (ed.), The cognitive neurosciences. Cambridge, MA: MIT Press

Roland, P.E., B. Larsen, N.A. Lassen and E. Skinhoj, 1980. Supplementary motor area and other cortical areas in the organization of voluntary movements in man. Journal of Neurophysiology 43, 118-136.

Rösler, F., M. Heil and E. Hennighausen, 1994. ‘Slow potentials during long-term memoty retrieval’. In: H.J. Heinze, T.F. Miinte and G.R. Mangun (eds.), Cognitive electrophysiology. Boston, MA: Birkhäuser.


Sanders, A.F., 1980. ‘Stage analysis of reaction processes’. In: GE. Stelmach and J. Requin (eds.), Tutorials in motor behavior (pp. 331-354) Amsterdam: North-Holland.

Sanders, A.F., 1990. Issues and trends in the debate on discrete vs. continuous processing of information. Acts Psychologica 74, 123-167.

Scherg, M. and T.W. Picton, 1991. ‘Separation and identification of event-related potential components by brain electrical source analysis’. In: C.H.M. Brunia, G. Mulder and M.N. Verbaten (eds.), Event related brain research (pp. 24-37) (Supplement 42 to Electroencephalography and Clinical Neuro- physiology). Amsterdam: Elsevier Science.

Shiffrin, R.M. and W. Schneider, 1977. Controlled and automatie human information processing. 11. Perceptual learning, automatie attending, and a general theory. Psychological Review 84, 127-190.

Smid, H.G.O.M., 1993. When action starts before perception is ready: A chronopsychophysiological approach. Thesis, University of Groningen.

Smid, H.G.O.M. and G. Mulder, 1995. The availability versus the use of partial information: Multiple levels of selective processing between perception and action. Acts Psychologica 90, 145-162 (this volume).

Steinmetz, H., G. Fiirst and B.U. Meyer, 1989. Craniocerebral topography within the international 10-20 system. Electroencephalography and Clinical Neurophysiology 72, 499-506.

Sternberg, S., 1969. The discovery of processing stages: Extensions of Donders’ method. Acts Psycho- logica 30, 276-315.

Ungeleider L.G. and M. Mishkin, 1982. ‘Two cortical visual systems’. In: D.J. Ingle, M.A. Goodale and R.W.J. Mansfield (eds.), Analysis of visual behavior (pp. 549-586). Cambridge, MA: MIT Press.

Van der Heijden, A.H.C., 1992. Selective attention in vision. Londen: Routledge. Wijers, A.A., 1989. Visual selective attention. An electrophysiological approach. Ph.D. Thesis, Univer-

sity of Groningen. Wijers, A.A., J.J. Lange and G. Mulder, in press. ‘Hersenmechanismen van selectieve aandacht’. In: A.

Kok and J. Boelhouwer (eds.), Psychofysiologie van de aandacht. Wijers, A.A., G. Mulder, T. Okita, L.J.M. Mulder and M.K. Scheffers, 1989a. Attention to color: An

analysis of selection, controlled search, and motor activation, using event-related potentials. Psy- chophysiology 26, 89-109.

Wijers, A.A., L.J. Otten, S. Feenstra, G. Mulder and L.J.M. Mulder, 1989b. Brain potentials during selective attention, memory search, and mental rotation. Psychophysiology 26, 452-467.

Zeki, S., 1990. La construction des images par le cerveau. La Recherche 21, 712-721.

the role of neuroimaging in the discovery of processing stages a review

Documents