development of perception of the unity of musical events

Cognitive Development, 9,355-375 (1994)

Development of Perception of the Unity of Musical Events

Anne D. Pick

Dana Gross

Marian Heinrichs

Margaret love

Carolyn Palmer

University of Minnesota

Infants’ and young children’s perception of the unity of musical events was investigated in three studies. In the first two, children watched video displays of two musicians playing different musical instruments side by side in synchrony, and heard a soundtrack in synchrony with both instruments but specific to one. The children judged which instrument was producing the music they heard. Three- to 4-year-olds differentiated instruments from different families but not instruments from the same family. Five- to 7-year-olds additionally differentiated instrument pairs differing in size and pitch range (e.g., violin, cello). In the third study, infants were presented some of the same musical events in order to assess whether specific experience with the instruments is necessary for perceiving the unity of musical events. Looking times revealed that 7- to 9-month-olds detected the correspondence of the sight and sound of some musical instruments. Specific experience with a variety of instruments is evidently not necessary for detecting correspondences of audible and visible properties and for differentiating instruments from different families.

Perceiving events usually is a multisensory experience, and an important question about the development of perception of events is how we come to discern their unity. When an infant sees and hears his or her father singing a lullaby, how

We thank the infants, children, parents, and teachers whose participation and assistance made

this research possible. We gratefully acknowledge the assistance at various phases of the research of

Anne Abrams, Lisa Baron, Maureen Bigbee, Gail Fury, Becky Jacobson, Patricia Melendez, Leo

Saavedra, Betsy Sitkoff, Nelson Soken, and Cheng Yuanshan. Finally, we thank Donald MacEachern for help in planning statistical analyses. This research was supported in part by Program Project

Grant HD05027 from the National Institute of Child Health and Human Development to the

University of Minnesota.

Dana Gross is now at St. Olaf College, and Carolyn Palmer is now at Vassar College. Margaret Love is now in Lexington, KY.

Correspondence and requests for reprints should be sent to Anne D. Pick, Institute of Child

Development, University of Minnesota, 51 E. River Rd., Minneapolis, MN 55455.

Manuscript received November 23, 1993; revision accepted March 22, 1994. 355

356 A.D. Pick, D. Gross, M. Heinrichs, M. Love, and C. Palmer

does he or she know that the face he or she sees is the source of the sound he or she hears? Our understanding of early perceptual development has been advanced significantly by research on infants’ perception of the unity of a variety of visible and audible events (Bahrick, 1983, 1987; Kuhl & Meltzoff, 1982, 1984, 1988; Spelke, 1979; Spelke, Born, & Chu, 1983; Walker, 1982; Walker- Andrews, 1986). In the present studies, infants’ and young children’s perception of musical events that are both seen and heard is investigated.

When we listen to music, we know something about its sources-the instruments or voices that are producing it. When a melody is sung, we can perceive properties of the singer as well as what is being sung. We can usually tell whether the singer is male or female, adult or child; we can identify some musical instruments by listening to music being played on them.

Musical instruments are artifacts that are made to be played and listened to. They are made for humans to interact with in particular ways to produce events. Like most other events, musical events are multimodally specified. The properties of these events are specified optically and acoustically as we watch and listen to instruments being played. Among those properties are the substance and surface layout of the instruments and the quality of their sound, their manner of playing (e.g., plucking or blowing), the synchrony of a player’s movements and the sounds produced, and the size of instruments and their pitch ranges. Thus, musical events provide the possibility for investigating what properties might underlie our perception of the unity of such events. They also provide the possibility for investigating the course of learning to perceive properties through the early years and as we become skilled listeners. in the first two studies reported here, the sensitivity of young children to properties specific to musical instrument families and to size and pitch range was investigated. In the third study, it was asked whether infants show evidence of such sensitivity.

Instruments in the same musical family have similar substances and surface layouts, are played in similar ways, and share sound qualities (e.g., similarity of timbre). The hypothesis that musical instrument families may display properties that are relevant for the early perception of musical events has emerged from an earlier investigation (Palmer, Jones, Hennessy, Unze, & Pick, 1989). Predicta- bly, young children and adults differed in the accuracy with which they could identify by sight a photograph of a specific instrument they heard. However children, like adults, were highly accurate at knowing what kind of instrument they were hearing. They mistook violins for cellos, but not for saxophones or flutes. In addition, children and adults listening to Chinese instruments that they likely never or rarely heard before also were very accurate at knowing the type of instrument they were hearing. Again, they mistook a photograph of an instrument they were hearing for others in the same family. In the present research the relevance of properties specific to families for perceiving the unity of musical events was investigated.

In the studies reported here, use was made of the technique Spelke (1976)

Perception of Musical Events 357

developed to investigate the basis of infants’ perception of the unity of bimodally specified events. For the first two studies, the technique was adapted to ask questions about young children’s perception of musical events. They watched a video display of two instruments being played in synchrony with both instruments but specific to one of them. The children indicated which instrument was producing the sound they were hearing. This technique permits varying systematically the availability of properties that ordinarily provide a possible basis for perceiving unified events in order to investigate the course of development of such perception.

STUDY 1

Method

Subjects. Twenty-seven children (13 boys and 14 girls), ranging in age form 4;0 (years; months) to 4;ll (A4 = 4;1), participated in the study. It had been established in pilot testing that children in this age range were the youngest who could easily perform the task. Data for 6 additional children were not included in the analyses for the following reasons: Five did not perform correctly on at least two of three practice trials and 1 responded in a systematic left-then-right sequence throughout the procedure.

Materials. Twenty-four musical event displays were videotaped in color. Each display showed two musicians playing different musical instruments from the brass, woodwind, and string families. The instruments of each pair were from different families and were selected to have overlapping pitch ranges and sizes. In 12 of the displays the instrument pairs included trumpets, violas, and flutes- four instances of an instrument paired with each of the other two. In the other 12 displays the instrument pairs included trombones, clarinets, and cellos-again four instances of an instrument paired with each of the other two. Thus the brass-woodwind pairs were trumpet-flute and trombone-clarinet; the brass- string pairs were trumpet-viola and trombone-cello; and the string-wood pairs were viola-flute and cello-clarinet.

The two instruments of a display were seen being played in synchrony, and a synchronous soundtrack generated by one of the instruments was heard. The soundtracks were melody excerpts of approximately 15 set duration with some variation due to the phrase structure of the excerpts. The excerpts were from a variety of songs and each was judged to be appropriate for both instruments of the display for which it was the soundtrack. In all, excerpts of 12 songs were used. Each song was played twice, once by each member of a pair of musical instruments. Each pair of instruments played two songs altogether, one that was judged by the experimenters to be familiar to many young children (e.g.,

358 A.D. Pick, D. Gross, M. Heinrichs, M. love, and C. Palmer

“Sesame Street” theme, Puff the Magic Dragon) and one that was judged to be unfamiliar (e.g., “Tradition” from Fiddler on the Roof, Simple Gifts). This variable of familiarity was not defined in any additional formal manner.

For each pair of instruments, there were two displays in which one instrument was the soundtrack source and two displays in which the other instrument was the soundtrack source. Each instrument producing the sound was seen on the left in one display and on the right in a second display. The 24 displays were presented in an unsystematic sequence, except that two displays of a particular pair of instruments occurred in each half of the 24 displays. The right-left position of the instruments being heard was uncontrolled in the sequence of 24 displays. There were two different sequences of the displays and each was presented to half of the children.

The flute, viola, and clarinet were played by three of the experimenters. Three band and orchestra members were recruited and paid a modest fee to play the trombone, trumpet, and cello. All of the musicians were seen in costumes that disguised their identity The costumes were intended to serve two purposes: to enhance the children’s interest in the “music game” and to prevent the children from recognizing any of the musicians as also being experimenters for the study. In pilot work, it was found that when children (younger than those who participated in the study) recognized that an experimenter was playing an instrument, they tended to attribute the instrument played by that person as the source of a soundtrack regardless of whether or not it was the actual sound source.

The recording was done with the help of technicians in a professional recording studio. Separate cameras were used to record the two instrumentalists of each pair. This was done so that each instrument could be recorded from a perspective and distance that highlighted the musician’s manner of playing it. For this reason, size relations among the instruments were not all completely preserved in the video displays although the relation of the size of the different instruments to the size of the adult human body was accurately portrayed. The sound of one instrument was recorded simultaneously with the video recording of both instruments being played. This was done by placing the two musicians in different rooms, each with a view of a “conductor” who directed their playing.

The musical displays were presented via a color monitor (48 cm) and were preceded by a video recorded “space commander” who explained the procedure to the children and guided them through three practice trials termed “fun runs.” These were musical displays consisting of an electronic piano and a drum and it was assumed that the children would be able to distinguish the two instruments fairly easily The piano was the sound source for two of these displays and the drum was the source for the third. These displays were constructed exactly like the 24 for the study itself. The space commander reappeared after the first 12 displays to remind the children of the task and to encourage their attention. The


sound level was not precisely controlled but was at a comfortable level for listening (for the adult experimenters).

The children sat facing and within easy reaching distance of the monitor. They judged which instrument of a display was producing the sound by pressing a button located under the left or right instrument that caused a bulb above the instrument to light up. The purpose of this procedure was to engage the children in the task by having them interact with the displays instead of merely pointing or making verbal judgments.

Procedure. The children participated in the procedure individually with an experimenter. The experimenter first explained that the child would play a music game with the TY The space commander appeared at the beginning of the videotape to describe the game, show the child the buttons and light, and guide the child through the three practice trials. The space commander’s description and encouragement were amplified by the experimenter.

The children watched the 24 displays in succession, indicating for each one which instrument was being heard by making the light go on above one of the two instruments. The experimenter urged the children to look and listen carefully while making their choices. Occasionally, a child did not make a choice until the instruments of a display had stopped playing. In that event the experimenter stopped the tape while the child chose an instrument and then resumed playing the tape with the next display. In general, the children enjoyed the task and the entire procedure took about 15 min.

Results The number of instruments correctly chosen as the sound sources, (i.e., correct identifications of instruments) was computed for each child. The mean number of instruments correctly identified by the children was 17 (of 24); their performance ranged from 12 to 24 correct. The children’s errors were analyzed initially for any effects of order, left-right correct position, familiar versus unfamiliar melody, gender, and age (younger vs. older half of the subjects). Comparisons showed that errors were evenly distributed across the two display orders (M = 7.36, SD = 3.18; M = 7.00, SD = 3.16), respectively left (M = 8.92, SD = 4.35) and right (A4 = 7.25, SD = 3.28) correct position, familiar (M = 8.50, SD = 3.50) and unfamiliar (M = 7.67, SD = 4.31) melodies, and younger (M = 6.54, SD = 3.02) and older children (M = 7.42, SD = 3.68). Girls (M = 8.71, SD = 1.77) made significantly more errors than did boys (A4 = 5.54, SD = 3.86), t(25) = 2.78, p < .05. (All probabilities reported herein are for two-tailed tests.) This difference may be due largely to the perfect performance of one boy whose family was said to be very musical.

Three sets of analyses will be presented. The first will assess the children’s accuracy of identifying individual instruments when they were seen and heard.


The second analyses, of instrument pairs, will identify any biases in the children’s choices. The third will assess the children’s accuracy of identifying instruments in the same family.

Individual Instruments. Each of the six instruments was seen and heard playing in 4 of the 24 displays. Figure 1 shows the mean number of correct identifications of the individual instruments. Using t tests, it was found that the children correctly identified each instrument, except the flute, significantly more often than chance (all p < .Ol; flute p > .05).

An analysis of variance (ANOVA) revealed significant differences among the instruments in the children’s correct identifications, F(5, 130) = 4.64, p < .OOl. Dunn’s planned comparisons revealed that the trombone was correctly identified significantly more frequently than were the clarinet (JJ < .05), cello (p < .05), and flute 0) < .Ol).

Instrument Pairs. Each pair of instruments was seen four times in the 24 displays, twice when one instrument was heard playing and twice when the other instrument was heard. There were no significant differences in the children’s performance for the six pairs of instruments. However, there were differences in the children’s performance for some pairs of instruments depending on which

4.00

3.00

2.00

1.00

c

3.04 - 2.89

2.59

- I - Trombone Viola Trumpet C&

INSTRUMENTS

m Brass

0 String

fza Woodwind

2.59

Chance Score

Figure 1. Mean number of correct indentifications (out of 4) of individual instruments in Study 1.


instrument was heard playing, F( 11, 286) = 4.65, p < .OOl. As revealed by Dunn’s planned comparisons, in instrument pairs that include the trombone, the children’s performance was significantly better when the trombone was heard than when either the cello 0, < .OS) or the clarinet @ < .Ol) was heard. Thus, the trombone “captured” the sound when it was seen next to another instrument. This bias was not complete, however, because on average the children correctly identified the cello (M = 1 .OO) and the clarinet (M = 1.04) as the soundtrack in about half the pairs when they were seen with the trombone. In other words, the “false alarm rate” for the trombone was about 50%. The children’s performance for the other instrument pairs did not differ depending on which instrument produced the sound.

Instrument Families. Instruments of each family were seen and heard playing in 8 of the 24 displays. The children correctly identified the instruments of all three families more often than chance (brass: M = 6.26, p < .Ol; string: M = 5.60, p < .Ol; woodwind: M = 4.93, p < .05). An ANOVA revealed differences among the families in the children’s performance, F(2, 52) = 7.09, p < .Ol. Dunn’s planned comparisons showed that instruments in the brass family were correctly identified with significantly greater frequency than were instruments in the woodwind family @ < .Ol). This difference occurred because, as noted, the trombone was compelling when it was seen and the frequency of the children’s correct identifications of the flute was not significantly greater than chance.

Discussion The children’s accurate identification of all instruments except the flute reflects their use of the optical and acoustical information available in the displays to perceive the unity of the musical events. Pitch range and instrument size as well as synchrony were comparable for the two instruments of a pair. Consequently other properties, visible and audible, were the basis for the children’s accuracy of instrument identification. Consistency of performance for instruments within families might reflect the importance of properties common to members of the same instrument family and in particular the visible substance and surface layout and manner of playing of the instruments and as the qualities of sounds produced by them.

The children’s performance was idiosyncratic for two instruments: the trombone and the flute. Their difficulty with the flute may have been a consequence of the angle and distance of the video camera from the musician. The mouth and finger movements of the flute player were barely discernible in the videotape.

Why the trombone compelled the sounds of instruments it was paired with is not altogether obvious. The sweeping arm movements made by a trombone player may contribute to the effect. However, cellists also make sweeping arm


movements. Again, the effect may be due to the particular way the trombonist was recorded.

In the next study we explored systematically the relevance of family membership as well as pitch range and instrument size for young children’s perception of the unity of musical events.

STUDY 2

Children of two ages participated in Study 2 in which the properties of family membership and size and pitch range were varied systematically in order to directly assess their relevance for the development of children’s perception of the unity of bimodally specified musical events.

Method

Subjects. Forty 3- to 4-year-old children (20 boys and 20 girls, age range = 2;9-4;7; A4 = 3;8) and 40 5 to 7-year-old children (20 boys and 20 girls, age range = 5;5-7;9, M = 6;3) participated in the study. The children in the younger group were selected to be comparable in age range to those in the first study and also because they could understand and perform the task. The older children were selected to be somewhat older than the younger group, but still interested in the task. Because there are no previously reported age-related findings for a comparable musical event task, there was no particular reason to select children of a narrow chronological age range within each group. Data for 11 additional 3-year-olds were not included in the analyses for the following reasons: Three children did not understand the task after several repetitions of the introductory displays, 5 children either consistently responded on only one side or alternated left and right throughout the 24 trials, 2 children declined to complete the task, and 1 child consistently pointed to the wrong instrument declaring his amusement at “tricking” the experimenter.

Materials and Procedure. There were 48 videotaped musical event displays. As in Study 1, each display showed two musicians playing different musical instruments side by side in synchrony, and a soundtrack was heard in synchrony with both instruments but specific to only one of them. Four types of instrument pairs were represented with equal frequency in the displays: (a) two instruments from the same family with overlapping pitch ranges and similar sizes (e.g., violin and viola), (b) two instruments from the same family with different pitch ranges and sizes (e.g., violin and cello), (c) two instruments from different families with overlapping pitch ranges and similar sizes (e.g., violin and clarinet), and (d) two instruments from different families with different pitch ranges and sizes (e.g., violin and baritone). The pairs of instruments of each of the four types are listed in Table 1. The table shows that there were two instances of each specific family/size combination.


Table 1. Pairs of Instruments Used in Study 2

Family

Same Different

Cello-string bass (A)

Violin-viola (B)

Trumpet-French horn (A)

Baritone-trombone (B)

Flute-clarinet (A)

Saxophone-bassoon (B)

Violin-clarinet (A)

String bass-saxophone (B)

Cello-trombone (A)

Viola-trumpet (B)

Bassoon-baritone (A)

Flute-French horn (B)

Size

Same Different

Viola-string bass (A) Violin-baritone (A)

Violin-cello (B) French horn-string bass (B)

French horn-trombone (A) Flute-cello (A)

Trumpet-baritone (B) Viola-bassoon (B)

Clarinet-bassoon (A) Saxophone-trumpet (A)

Flute-saxophone (B) Clarinet-trombone (B)

Note. The (A) and (B) notations refer to the instrument pairs making up each of the two sets.

As in Study 1, the instruments were from the brass, woodwind, and string families. The brass instruments were a trumpet, a French horn, a baritone, and a trombone. The woodwinds were a flute, a clarinet, a saxophone, and a bassoon. The strings were a violin, a viola, a cello, and a string bass.

Each display was approximately 15 set in length (again, with some variation due to phrase structure of the excerpts). The soundtracks were excerpts of 12 songs selected to be generally appealing to young children and appropriate for the instruments they were played on. Both familiar and unfamiliar melodies were included in the set; familiarity was not systematically varied in this study. Excerpts were selected in which there was movement in the melody and not much repetition of the same note which would diminish the finger and arm movements of players, especially of blown instruments.

The same song was played by both instrument pairs of a particular family/size combination. For example, the “Sesame Street” theme was played by the flute-clarinet pair and by the saxophone-bassoon pair. These are the two woodwind pairs with overlapping pitch ranges and similar sizes. For instrument pairs of different size, the key (and consequent pitch range) of the melody was determined by the musicians themselves so as to be appropriate for each instrument. In all, each song was played four times throughout the 48 displays, once by each member of each of two pairs of instruments.

There were two sets of 24 displays each. Half of the children in each age group saw and heard one set. Each set of 24 displays contained one instrument


pair of every family/size combination. (The A and B notations in Table 1 indicate the instrument pairs making up each set.) Each pair was seen and heard twice in the 24 displays, once when one instrument was the soundtrack source and once when the other instrument was the soundtrack source. The 24 displays were presented in an unsystematic sequence except that each instrument pair occurred once in the first half of the displays and once in the second half. The instrument seen on the right was the sound source in six of the first half of the displays and in six of the second half of the displays. The right-left position of the individual pairs of instruments was not controlled. Two different orders were constructed for each set of 24 displays.

The recording procedure differed somewhat from that used for Study 1; it was not carried out in a professional recording studio. For each display, a soundtrack was first recorded onto an audio tape recorder. Then the two musicians of a pair were video recorded playing in synchrony with the soundtrack, which was heard via a speaker from the audio tape recorder and was simultaneously fed into the video recorder via an audio input cord. As before, separate cameras were used to record the musicians of each pair. (Attempts were made, based on intuitive judgments, to better highlight the flute player displays, and to somewhat diminish the salience of the trombonist’s large arm movements.)

Six practice trials whose purpose was to introduce children to the procedure preceded the 24 displays: two pairs were bells and sticks with each producing the soundtrack once, two were cymbals and a drum with each producing the soundtrack once, one was a guitar and an electronic piano with the soundtrack produced by the guitar, and one was a recorder and a xylophone with the xylophone producing the soundtrack.

Musicians were recruited as in Study 1. The manner of presenting the displays, with the guidance of a space commander, was as in Study 1 except that the musicians were not disguised, as that proved unnecessary. The children participated in the procedure and made their judgments in the same way as in Study 1.

At the conclusion of the procedure, the children participated in two picture tasks designed to assess their knowledge about the instruments they had seen in the displays. The children were shown colored photographs (10 cm x 18 cm) of the 12 instruments. The photographs were mounted on posters with three instruments from different families on each poster. For the first task, the experimenter showed the children a poster and, naming an instrument, asked the children to point to it. Then she asked the children to point to a second named instrument on the poster followed by the third. After the children had been asked about all the instruments on a poster, another poster was selected and the procedure continued in the same way until the children had been asked about the instruments on all four posters. The order in which the posters was presented varied unsystematically across children.

The second picture task was the same as the first except that the children were


asked if they knew the names of the instruments instead of being asked to point to instruments named by the experimenter. The order of the two tasks was the same for all children.

Results

Instrument Displays. As before, the number of instruments correctly chosen as the sound sources was computed for each child. The mean number of instruments correctly identified by the older children was 18.0 (range = 13-22). The mean number correct for the younger children was 14.0 (range = 9-21). Although there was considerable overlap in the performance of the two groups, the older children were significantly more accurate than the younger children, t(78) = 7.06, p < .Ol. The children’s errors were evenly distributed across gender (boys: M = 8.0, SD = 2.96; girls: M = 7.7, SD = 3.05), across the two tapes (Tape A: M = 8.1, SD = 3.21; Tape B: M = 7.6, SD = 2.77), and across the two display orders (M = 7.8, SD = 2.88; M = 7.9, SD = 3.12, respectively).

Two sets of analyses will be presented. The first will examine the children’s accuracy of identification of the six kinds of instruments (large and small brasses, large and small woodwinds, large and small strings). In addition, the children’s accuracy of identification of the three families will be examined. The effects on the children’s judgments of the family/size relations of the instrument pairs will be assessed in a second set of analyses.

Instrument and Family Kinds. Each kind of instrument was seen and heard playing in 8 of the 48 displays. T tests revealed that, with one exception, the children correctly identified each type of instrument significantly more frequently than chance (all p < .Ol except p < .05 for younger children’s identification of small brass instruments). The exception was the younger children’s identification of large woodwind instruments (p > .05). The difficulty with this group appeared to be specific to the bassoon, and as with the flute in Study 1, it probably was due to the bassoonist’s barely discernible finger movements.

ANOVAs revealed some differences among the different types of instruments in the children’s correct identifications: younger children, F(5, 38) = 3.19, p < .Ol; older children, F(5, 38) = 6.52, p < .Ol. For both age groups, large brass instruments were most frequently correctly identified and large woodwinds were least frequently correctly identified (again reflecting the problem with the bassoon displays).

In order to find out if there were differences among instrument families in the children’s accuracy of identification, ANOVAs with family (as a within-subject variable) and tape (as a between-subject variable) were computed for each age group. For both age groups, only the effect of family was significant, F( 1, 38) =


4.31 for younger children and 3.02 for older children, both p =L .05. Children in both age groups correctly identified brass instruments significantly (younger: p

< 01; older: p < .05) more often than they did woodwinds. In addition, the younger children correctly identified string instruments more frequently than the woodwinds @ < .05).

Family l Size Relations. The major independent variables of interest were the effects on the children’s judgments of the family relation and of the size relation of the members of a display pair. Statistical analyses were carried out for each

age group separately because there were no specific developmental hypotheses being tested about the effects of the variables on the children’s judgments.

A three-way ANOVA (with family relation and size relation as within-subject variables and tape as a between-subject variable) of the younger children’s

correct judgments revealed a significant effect of the family relation of the instruments in a display, F(1, 38) = 4.99, p < .05. These children made more correct judgments of instrument pairs that were from different families (M = 7.50) than of instrument pairs from the same family (M = 4.78). In fact, these children’s performance with same-family pairs was not significantly more

accurate than chance. The analysis of the older children’s correct judgments revealed significant

effects of the family relation, F(1, 38) = 62.15, p -=c .OOl, and of the size relation, F(1, 38) = 38.47, p ==L -001, of the instrument pairs. The interaction of family and size was also significant, F( 1, 38) = 12.06, p < .Ol. These children were more accurate with instrument pairs differing in family or size (different family different size: M = 5.18; different family same size: M = 4.80; same family different size: M = 4.70) than with pairs in which the family and size were the same (M = 3.33), all p < .Ol. For the latter pairs, the older children’s performance was not significantly more accurate than chance.

The three-way interaction was also significant for the older children, F( 1, 38) = 12.06, p < .Ol. This modified the Family X Size interaction in that for one tape, the children were more accurate for pairs differing in family and size than with pairs differing either in family or size. In general, the analyses indicate that the younger children differentiated among instruments based on properties that distinguish among musical instrument families. The older children demonstrated additional sensitivity to the correspondence of physical size and pitch

range.

Picture Task. When the children were asked to point to the named instrument, most of the children in both age groups correctly identified more instruments than would be expected by chance. Thirty-two (of 40) of the younger children and 37 of the older children correctly identified more than four instruments. The difference in the number of children in the two age groups who performed above chance was not significant, (X(1) = 2.64, p = .lO. Overall, the oIder children


correctly identified more instruments (M = 8.40) than did the younger children

(M = 6.45), p < .Ol . The younger children’s performance was related to their performance with the musical events: younger r = .39, p < .05; older r =

-.ll.

All of the older children and all but two of the younger children participated in the final posttest task, naming the instruments in the photographs. The children either responded correctly or incorrectly or said they did not know. In addition to noting their correct responses, the children’s incorrect responses were collated in terms of whether or not they were names of instruments in the same family as the correct instrument.

There was considerable variation across families in the frequency with which the children’s labels were correct. The percent of the younger children’s labels that were correct were: woodwinds 19%, strings 13%, and brass 5%. The percentage of children’s labels that were in the right family (i.e., either correct or another instrument of the same type, e.g., labeling a clarinet by the name “flute”) were: woodwinds 32%, strings 31%, and brass 34%. Many children used one or two labels for all instruments in a family, for example, calling all the strings “violin.”

There was considerable systematically also in the labels children used that did not refer to instruments in the same family as the target instrument. Altogether, the children applied 30 names other than woodwinds to woodwind instruments, of which 22 were names of brass instruments; they applied 20 nonbrass names to brass instruments of which 14 were names of woodwind instruments. The children applied only five nonstring names to strings.

The percent of the older children’s labels that were correct were: woodwinds 38%, strings 28%, and brass 27%. The percent of their labels that referred to instruments in the right family were: woodwinds 53%, strings 68%, and brass 61%. They applied 12 nonwoodwind names to woodwind instruments, 12 nonbrass names to brass instruments, and 8 nonstring names to strings. The pattern characterizing these labels for the younger children was not apparent for the older children who have, of course, acquired more specific knowledge of different types of instruments.

Discussion The results of this study replicate and extend those of Study 1. The 3- to 7-year- old children in this study differentiated between visible and audible musical events based on properties that distinguish instruments in different families. The older children also demonstrated sensitivity to the relation of instrument size and pitch range in musical events; they differentiated between instruments varying in size and producing higher and lower pitch range of sounds.

There was some evidence that instruments from the three families were not equally well differentiated, especially by the younger children. In particular, the children’s greater success with brass than with woodwind instruments might


suggest that they differentiate blown from bowed sounds and instruments before different kinds of blown sounds and instruments. Evidence for this suggestion is weakened by the specific problematic bassoon displays. However, the sys- tematicity of the younger children’s labels in the second posttest are consistent with this idea. Specifically, the younger children used string names to label string instruments more frequently than they used brass or woodwind names to label brass or woodwind instruments, respectively Also, they frequently applied woodwind names to brass instruments and vice versa. Their explicit knowledge might reflect that they differentiate blown from plucked or bowed instruments before they differentiate brasses from woodwinds.

The hypothesis that young children are better able to differentiate instruments from different families than members of the same family has been supported in the studies presented thus far. In addition, children develop sensitivity to the size-pitch range relation of musical events.

How have these young children learned about the audible and visible properties of the musical events? Is it a matter of associating the co-occurring visual and aural experiences of seeing and hearing musical instruments being played? Such an account would depend on specific opportunities and experiences with musical instruments. The children who participated in these studies have probably heard most of the instruments on previous occasions and they have specific knowledge of some of them. It is possible, though somewhat unlikely, that they all have previously seen and heard all of the instruments with sufficient frequency to have learned to associate the sight with the sound of each. However, it is quite implausible that infants have had such experiences during their first several months of life. Thus, infants participated in the next study in order to investigate the role of specific experience with instruments in the development of perception of bimodally specified properties of musical events. Obviously we could not present infants with the same task the children performed, that is, to judge which instrument of a pair is producing the sound. Further, we would expect both video displays of a pair to attract infants’ attention because they portray people in motion manipulating objects. However, if during the course of the trials the infants look longer at the sound-specified instruments of the pairs, that would be evidence that they detect some visible and audible correspondences in the musical events.

STUDY 3

Method

Subjects. Thirty 7- to 9-month-old infants (range = 30-38 weeks; = 33) participated in the study. Data for four additional infants were not included in the analyses because two did not complete the procedure, one looked only at the


right monitor throughout the procedure, and one was excluded because of

experimenter error.

Materials. The musical event displays were those of Study 1. The videotape had the instruments’ sounds recorded on one audio track. The second audio channel contained a recorded tone whose beginning and ending coincided with the beginning and ending of each musical event display. The presence of the tone was detected by electronic circuitry that was used by a computer to determine the duration of the trial. When the computer detected the end of a trial, it sent a signal to the VCR remote to pause the VCR. The intertrial interval was manually determined and successive trials began when the experimenter began the VCR.

Procedure. Each infant was seated on his or her parent’s lap facing two 48 cm color monitors. The monitors were approximately 60 cm from the infant and were separated by 25 cm with a speaker located centrally between them. A gray partition occluded the equipment (except for the monitors), experimenter, and observers from the infant’s view. There was a 1 cm X 1 cm hole located 31 cm below the monitors and 76 cm from the infant through which the infant’s eye movements were observed.

The infant viewed two instruments being played in synchrony, one on each monitor, and heard the synchronous soundtrack of one instrument from the centrally located speaker. Immediately preceding each trial, a light was shone through a small hole in the partition between the two monitors to direct the infant’s gaze. When the infant’s gaze was centered, the trial was initiated. The observer had two push buttons that were depressed during the trial to signal to the computer the observer’s judgment of the infant’s looking to the right and to the left monitor. The observer was unaware of which display of a pair was sound- specified. The computer recorded the duration and direction of each look. At the end of the session, the computer printed a summary of the total cumulated looking time to the right and left monitors for each trial.

Three adults acted as observers with one observer for each infant. Interrater reliabilities were established by each pair of observers carrying out the procedure with individual babies. A second peephole and pair of push buttons were used in these cases and the computer recorded and compared the two observers’ simultaneous inputs. To achieve an appropriate level of reliability, each pair of observers carried out the procedure with three to five (pilot) babies. The mean reliability achieved by the three observer pairs was kappa r = .89 and Pearson r = .99.

The procedure lasted 12 to 15 min. All infants saw the practice displays first (although looking times for these displays were not included in subsequent data analyses). The displays were presented to all infants in the same order.

370 A.D. Pick, D. Gross, M. Heinrichs, M. Love, and C. Palmer

Results and Discussion’

The infants looked at the displays throughout each trial; the mean total looking time was 15.03 set reflecting that the musical events were of obvious interest to them. The results of three analyses converge to show that infants differentiated sound-specified from silent instruments in the displays. The data for the first analysis were the proportions of the infants’ total looking times directed toward the two instruments of a pair. A I test revealed that the mean proportion of the infants’ total looking time directed toward the sound-specified instruments (52.4%) was significantly greater than chance (50%), t(29) = 3.43, p < .Ol. In a second analysis, the infants’ mean looking times to sound-specified (M = 7.87, SD = 1.07) and silent (M = 7.16, SD = 1.13) members of the instrument pairs were compared using a t test for related measures. The infants explored and watched both displays of the pairs, but overall the infants looked significantly longer at the sound-specified instruments of the pairs, t(29) = 2.96, p < .Ol. Finally, though small in magnitude, this effect was highly reliable: 23 of the 30 infants looked longer at sound-specified than at silent instruments through the 24 display pairs (binomial p < .Ol). Thus, both events of the pairs engaged the infants and elicited much looking from them but they looked slightly, significantly, and reliably longer at the sound-specified instruments.

Figure 2 presents, for each type of instrument pair, the number of trials (out of the 60 trials when each instrument was both seen and heard) when infants looked longer at the sound-specified instrument of the pair. As can be seen, the three types of instrument family pairs were quite similar in the number of trials (out of 120) in which the infants looked longer at the sound-specified instruments of the pairs (i.e., string-wood pairs = 71 and 67 trials, brass-string pairs = 65 and 70 trials, brass-wood = 63 and 59 trials). The infants, like the children in Study 1, showed a comparable level of sensitivity of differentiating instruments across the three types of family pairs.

Also apparent in Figure 2 is that within some family pair types, the frequency with which infants looked longer at the sound-specified instruments depended on the particular instruments participating in the event. For example, in string- wood pairs, the infants looked longer at the seen and heard instrument more than twice as frequently when it was the viola (51 out of 60 trials) or cello (47 out of 60 trials) than when it was the flute (20 out of 60 trials) or clarinet (20 out of 60 trials). A similar asymmetry can be seen for the trombone-clarinet pairs: The infants looked longer at the sound-specified instrument on more than twice as many occasions when it was the trombone (44 out of 60 trials) as when it was the

I Lisa Baron, for an honors thesis project, used this same procedure with a group of 6- to

&month-old infants and one set (Set A) of the musical event displays from Study 2. The infants’ looking times were highly variable, both across babies and within babies across displays. Although

there was a marginally statistically significant difference favoring sound-specified looking for one

set of displays, we are not confident the result is reliable and it is not reported or discussed here.


Vioh-Flute Cello-Cltinet Trombone-Cello Trumpet-Viola Trombone-Clarinet Trumpet-Flute

INSTRUMENT PAIRS

Figure 2. Number of trials (out of 60 trials per instrument) when infants looked longer at the sound-specified instrument of a pair in Study 3.

clarinet (19 out of 60 trials). These biases are like those shown by the children in the first studies. The sweeping arm movements of the viola, cello, and trombone players may attract the infants’ visual interest more easily than the finer, subtler motor movements of the flute and clarinet players and also may facilitate detecting the audible and visible correspondences of these musical events. The fact that the infants’ looking patterns are not directed by the visible displays alone, however, is evident in their significantly longer looking times to the sound-specified members of the instrument pairs.

This study provides evidence that infants, like young children, can detect the correspondence of the sight and sound of some musical instruments when they see and hear displays of them being played. Synchrony as well as pitch range and instrument size were comparable for the instruments in each pair. Consequently, other properties specific to instruments or instrument families are the basis for the infants’ perception of the unity of these events.

GENERAL DISCUSSION

The results of these studies support the hypothesis that properties specific to musical instrument families are relevant for young children’s perception of musical events. When 3- and 4-year-old children see and hear pairs of musical instruments being played synchronously they differentiate instruments from different families but not instruments in the same family. Five- to 7-year-old children also differentiate instruments varying in size and pitch range. Specific experience with a variety of instruments is evidently not necessary for detecting


correspondences of audible and visible properties and for differentiating instruments from different families. Infants in the second half year of life can detect the correspondence of the sights and sounds of some musical instruments when they see and hear them being played.

The findings of these studies raise at least two general questions about the development of perception of bimodally specified events. One concerns the information that is the basis for perceiving the unity of such events. What properties of events are detectable by more than one modality enabling even young infants to perceive their unity? The second general question concerns the course of development of event perception. What kind of learning is involved in perceiving musical events? These questions will be considered in order.

What information is available for detecting the unity of bimodally specified events? Generally, synchrony of the sights and sounds of an event as well as common tempo and rhythm of sights and sounds are properties to which young infants are sensitive (Spelke, 1979, 1981; Spelke et al., 1983). For example, when watching the face of a parent who is singing a lullaby, an infant sees and hears vocalizations and facial motions that have the same rhythm and tempo and that occur in synchrony. In normal circumstances, these are likely relevant properties for perceiving the unity of musical events involving instruments as well. When a musical instrument is played, there is synchrony and common rhythm of the player’s visible movements and the sounds produced. However, the musical events of the present studies could not be differentiated on the basis of synchrony because the instrument sounds were synchronous with both visible displays of a pair.

There is considerable evidence that young infants are sensitive to audiovisual correspondences in addition to synchrony that specify properties of events. For example, infants at least as young as those of the present study detect, bimodally, information for the gender of gesturing faces and voices (Walker-Andrews, Bahrick, Raglioni, & Diaz, 1991), objects approaching or receding (Walker- Andrews & Lennon, 1985), and affective expressive behaviors (Soken & Pick, 1992; Walker-Andrews, 1988). Most relevant to the present case is evidence that infants can detect the substance of some moving, sounding objects based on the temporal microstructure of an impacting event (Bahrick, 1983, 1987).

It is clear from the present studies that the important correspondences of musical events, for infants’ and young children’s perception of musical events, will be ones specific to musical instrument families. Although some effects of individual instruments were found, they were idiosyncratic to one or the other set of displays. (Studies 1 and 3 used the same displays.) Thus, these effects are likely due to the nature of the video recording (e.g., camera angle) rather than reflecting how infants and children perceive events involving those instruments.

A description of audiovisual correspondences specifying and distinguishing among the sounds and sights of brass, string, and woodwind instruments being played might be sought by varying properties of the sounds based on spectral


analyses of them. For example, it has been shown that adults can perceive breaking and bouncing events based on acoustical information alone (Warren &

Verbrugge, 1984). The temporal-spectral patterns of the two events correspond to optical information that can be picked up by watching the events as well. Kuhl and Meltzoff (1988) reported that adults can match visual information for speech sounds with pure tones derived from the auditory information for those speech sounds. Infants aged 18 to 20 weeks, however, cannot detect the intermodal match under these circumstances; they seem to require the entire speech sound. It is likely the same would hold true for infants’ intermodal perception of complex nonspeech sounds such as those of musical instruments. A procedure like that of Bahrick (1987) might appropriately be adapted to answer the question.

The results of the present study with infants are consistent with earlier ones demonstrating that learning to perceive bimodally specified events begins early in life. The results of the first two studies further demonstrate that this learning continues through several years. Consideration of experts (e.g., musicians who can distinguish visible and audible characteristics (like the relation of the wood and finish and sound qualities) of Italian from German violins or 17th- to 19th- century Italian violins while watching and listening to them being played, suggests that the development of sensitivity to optical and acoustical correspondences of musical events can continue indefinitely. The course and conditions of this learning make up the second general question of interest. How do the infants and children know which instruments in the visible displays are the source of the sounds they hear? The results of the studies with infants render an unlikely association-based account of what is learned. An alternative account is that it involves detecting invariant relations in the optical and acoustical information produced by an event.

A recent study by Bahrick (1988) provides some evidence about the nature of this learning. Three-month-old infants saw and heard two events. For one event, a transparent Plexiglas cylinder containing one large marble was rotated. The sound of this event was of one discrete impact as the marble hit the bottom of the container. For the other event, an identical cylinder containing many small marbles was rotated. The sound of this event was of longer, multiple impacts as the marbles hit each other and the bottom of the container. Different groups of infants were familiarized with these events with either the appropriate or inappropriate sound paired with the visible event. Thus, some infants were presented with (a) the cylinder containing one large marble paired with the sound of the cylinder containing many marbles and (b) the cylinder containing many marbles paired with the sound of the cylinder containing one marble. Other infants saw each cylinder paired with its appropriate sound. In order to find out what the infants had learned about the events, they were presented both visible events simultaneously, side by side in synchrony, and they heard the soundtrack of one event. Infants who had been familiarized to the appropriate


soundtracks synchronous with the visible events looked longer at the sound- specified events during the test phase. The fact that their learning involved detecting invariant relations rather than association was inferred from the fact that infants familiarized with the inappropriate soundtrack synchronous with the visible event did not show evidence of learning.

Gibson (1992) suggested that “perceptual learning of multimodally specified properties of objects is correlated with development of the exploratory system that characteristically reveals information for them” (p. 231). The infants of the present studies have had much experience manipulating and exploring objects to discern their properties. Although they obviously do not play musical instruments, their experience with a variety of visible, sounding objects has given them opportunity to learn something about correspondences of sound and sight.

The present studies were not directed at understanding how infants and young children acquire sensitivity to properties of musical instruments. Studies modeled on Bahrick’s (1988) would be one way to investigate the question. For example, infants could be familiarized with the sounds and sights of one kind of musical instrument and then asked whether such experience is sufficient to detect visible and audible correspondences of other kinds of musical instruments. An additional, more general question is whether learning to detect optical and acoustical correspondences of one kind of object generalizes to other visible sounding objects. Investigation of these questions would be important contribu- tions to our further understanding of the course of how we learn to perceive the events of our world.

REFERENCES

Bahrick, L.E. (1983). Infants’ perception of substance and temporal synchrony in multimodal events.

Infant Behavior and Development, 6, 429-451.

Bahrick, L.E. (1987). Infants’ intermodal perception of two levels of temporal structure in natural

events. Infant Behavior and Development, IO, 387-416.

Bahrick, L.E. (1988). Intermodal learning in infancy: Learning on the basis of two kinds of invariant

relations in audible and visible events. Child Development, 59, 197-209.

Gibson, E.J. (1992). How to think about perceptual learning: Twenty-five years later. In H. Pick, P

van den Broek, & D. Knill (Eds.), Cognition: Conceptual and methodological issues (pp.

215-239). Washington: American Psychological Association.

Kuhl, PK., & Meltzoff, A.N. (1982). The bimodal perception of speech in infancy. Science, 218, 1138-1141.

Kuhl, PK., & Meltzoff, A.N. (1984). The intermodal representation of speech in infants. Infant

Behavior and Development, 7, 361-381.

Kuhl, PK., & Meltzoff, A.N. (1988). Speech as an intermodal object of perception. In A. Yonas (Ed.), The Minnesota Symposia on Child Psychology: Perceptual development in infancy (pp.

235-266). Hillsdale, NJ: Erlbaum.

Palmer, C.E, Jones, R.K., Hennessy, B.L., Unze, M.G., & Pick, A.D. (1989). How is a trumpet known? The “basic object level” concept and perception of musical instruments. American

Journal of Psychology, 102, 17-37.


Soken, N.H., & Pick, A.D. (1992). Intermodal perception of happy and angry expressive behaviors

by seven-month-old infants. Child Development, 63, 787-795.

Spelke, E.S. (1976). Infants’ intermodal perception of events. Cognitive Psychology, 8, 553-560.

Spelke, E.S. (1979). Perceiving bimodally specified events in infancy. Developmental Psychology,

15, 626-636.

Spelke, E.S. (1981). The infant’s acquisition of knowledge of bimodally specified events. Journal of

Experimental Child Psychology, 31, 279-299.

Spelke, E.S., Born, W.S., & Chu, E (1983). Perception of moving, sounding objects by four-month-

old infants. Perception, 12, 719-732.

Walker, A.S. (1982). Intermodal perception of expressive behaviors by human infants. Journal of

Experimental Child Psychology, 33, 514-535.

Walker-Andrews, A.S. (1986). Intermodal perception of expressive behaviors: Relation of eye and

voice? Developmental Psychology, 22, 373-377.

Walker-Andrews, A.S. (1988). Infants’ perception of the affordances of expressive behaviors. In C.

Rovee-Collier & L. Lipsitt (Eds.), Advances in infancy research (Vol. 5, pp. 173-221).

Norwood, NJ: Ablex.

Walker-Andrews, A.S., Bahrick, L.E., Raglioni, S.S., & Diaz, I. (1991). Infants’ bimodal perception of gender. Ecological Psychology, 3, 55-75.

Walker-Andrews, A.S., & Lennon, E.M. (1985). Auditory-visual perception of changing distance by

human infants. Child Development, 56, 544-548.

Warren, W.H., & Verbrugge, R.R. (1984). Auditory perception of breaking and bouncing events: A

case study in ecological acoustics. Journal of Experimental Psychology: Human Perception

and Performance, 10, 704-712.

development of perception of the unity of musical events

Documents