auditory perception of breaking and bouncing events: a ... · the auditory perception of these...

9
Journal of Experimental Psychology: Human Perception and Performance 1984, Vol. 10, No. 5, 704-712 Copyright 1984 by the American Psychological Association, Inc. Auditory Perception of Breaking and Bouncing Events: A Case Study in Ecological Acoustics William H. Warren, Jr. University of Connecticut Robert R. Verbrugge University of Connecticut and Haskins Laboratories, New Haven, Connecticut The mechanical events of bouncing and breaking are acoustically specified by single versus multiple damped quasi-periodic pulse patterns, with an initial noise burst in the case of breaking. Subjects show high accuracy in categorizing natural tokens of bouncing and breaking glass as well as tokens constructed by adjusting only the temporal patterns of components, leaving their spectral properties constant. Differences in average spectral frequency are, therefore, not necessary for perceiving this contrast, though differences in spectral consistency over successive pulses may be important. Initial noise corresponding to glass rupture appears unnecessary to categorize breaking and bouncing. The data indicate that higher order temporal properties of the acoustic signal provide information for the auditory perception of these events. Research in auditory perception has tended to emphasize the detection and processing of sound elements with quasi-stable spectral structure, such as tones, formants, and bursts of noise. In the spectral domain, these ele- ments are distinguished by frequency peak or range, bandwidth, and amplitude. In the temporal domain, acoustic analysis has often focused on the durations of sound elements, the intervals and phase relations between them, and the influence of these on pitch and loudness perception, temporal acuity, mask- ing, and localization. The auditory system has often been approached as an analyzer of essentially time-constant functions of fre- quency, amplitude, and duration, on the as- sumption that complex auditory percepts are compositions over sound elements having those properties, with certain temporal inter- This research was supported by Grant HD-01994 from the National Institute for Child Health and Human Development, by Biomedical Research Support Grant RR-05596 to Haskins Laboratories, and by a National Science Foundation graduate fellowship to the first author. The authors wish to thank Michael Studdert-Kennedy, Michael Turvey, Robert Shaw, and Ignatius Mattingly for their comments on an earlier draft. Requests for reprints should be addressed to William H. Warren, Jr., who is now at the Walter S. Hunter Laboratory of Psychology, Brown University, Providence, Rhode Island 02912. Robert R. Verbrugge is now at Bell Laboratories, Holmdel, New Jersey. actions (Fletcher, 1934; Helmholtz, 1863/ 1954; Plomp, 1964; see Green, 1976). The perceptual role of time-varying prop- erties of sound has received less attention. Exceptions to this are found in research on amplitude and frequency modulation, in- cluding auditory phenomena such as beats, periodicity pitch, and frequency glides. In general, research on time-varying properties has been most common in the study of classes of natural events, such as human speech, music, and animal communication, where an analysis of sound into quasi-stable elements is often problematic. In the case of speech, for example, many phonemic con- trasts can be denned by differences in the direction and rate of change of major speech resonances (see Liberman, Cooper, Shank- weiler, & Studdert-Kennedy, 1967; Liberman, Delattre, Gerstman, & Cooper, 1956). Some research on the perception of music has also demonstrated the perceptual significance of time-varying properties. Identification of mu- sical instruments, for example, is strongly influenced by the temporal structure of tran- sients that accompany tone onsets (Luce & Clark, 1967; Saldanha & Corso, 1964). In particular, the relative onset timing and the rates of amplitude change of upper harmonics have been found to be critical properties of attack transients that permit distinctions among instrument families (Grey, 1977; Grey 704

Upload: others

Post on 24-May-2020

9 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Auditory Perception of Breaking and Bouncing Events: A ... · the auditory perception of these events. Research in auditory perception has tended to emphasize the detection and processing

Journal of Experimental Psychology:Human Perception and Performance1984, Vol. 10, No. 5, 704-712

Copyright 1984 by theAmerican Psychological Association, Inc.

Auditory Perception of Breaking and Bouncing Events:A Case Study in Ecological Acoustics

William H. Warren, Jr.University of Connecticut

Robert R. VerbruggeUniversity of Connecticut and HaskinsLaboratories, New Haven, Connecticut

The mechanical events of bouncing and breaking are acoustically specified bysingle versus multiple damped quasi-periodic pulse patterns, with an initial noiseburst in the case of breaking. Subjects show high accuracy in categorizing naturaltokens of bouncing and breaking glass as well as tokens constructed by adjustingonly the temporal patterns of components, leaving their spectral propertiesconstant. Differences in average spectral frequency are, therefore, not necessaryfor perceiving this contrast, though differences in spectral consistency oversuccessive pulses may be important. Initial noise corresponding to glass ruptureappears unnecessary to categorize breaking and bouncing. The data indicate thathigher order temporal properties of the acoustic signal provide information forthe auditory perception of these events.

Research in auditory perception has tendedto emphasize the detection and processing ofsound elements with quasi-stable spectralstructure, such as tones, formants, and burstsof noise. In the spectral domain, these ele-ments are distinguished by frequency peakor range, bandwidth, and amplitude. In thetemporal domain, acoustic analysis has oftenfocused on the durations of sound elements,the intervals and phase relations betweenthem, and the influence of these on pitch andloudness perception, temporal acuity, mask-ing, and localization. The auditory systemhas often been approached as an analyzer ofessentially time-constant functions of fre-quency, amplitude, and duration, on the as-sumption that complex auditory percepts arecompositions over sound elements havingthose properties, with certain temporal inter-

This research was supported by Grant HD-01994 fromthe National Institute for Child Health and HumanDevelopment, by Biomedical Research Support GrantRR-05596 to Haskins Laboratories, and by a NationalScience Foundation graduate fellowship to the first author.

The authors wish to thank Michael Studdert-Kennedy,Michael Turvey, Robert Shaw, and Ignatius Mattingly fortheir comments on an earlier draft.

Requests for reprints should be addressed to WilliamH. Warren, Jr., who is now at the Walter S. HunterLaboratory of Psychology, Brown University, Providence,Rhode Island 02912. Robert R. Verbrugge is now at BellLaboratories, Holmdel, New Jersey.

actions (Fletcher, 1934; Helmholtz, 1863/1954; Plomp, 1964; see Green, 1976).

The perceptual role of time-varying prop-erties of sound has received less attention.Exceptions to this are found in research onamplitude and frequency modulation, in-cluding auditory phenomena such as beats,periodicity pitch, and frequency glides. Ingeneral, research on time-varying propertieshas been most common in the study ofclasses of natural events, such as humanspeech, music, and animal communication,where an analysis of sound into quasi-stableelements is often problematic. In the case ofspeech, for example, many phonemic con-trasts can be denned by differences in thedirection and rate of change of major speechresonances (see Liberman, Cooper, Shank-weiler, & Studdert-Kennedy, 1967; Liberman,Delattre, Gerstman, & Cooper, 1956). Someresearch on the perception of music has alsodemonstrated the perceptual significance oftime-varying properties. Identification of mu-sical instruments, for example, is stronglyinfluenced by the temporal structure of tran-sients that accompany tone onsets (Luce &Clark, 1967; Saldanha & Corso, 1964). Inparticular, the relative onset timing and therates of amplitude change of upper harmonicshave been found to be critical properties ofattack transients that permit distinctionsamong instrument families (Grey, 1977; Grey

704

Page 2: Auditory Perception of Breaking and Bouncing Events: A ... · the auditory perception of these events. Research in auditory perception has tended to emphasize the detection and processing

ECOLOGICAL ACOUSTICS 705

& Gordon, 1978). Animal vocalizations aresimilarly rich in time-varying properties (suchas rhythmic pulsing, frequency modulation,and amplitude modulation), and many ofthese properties have been shown to be criticalfor distinguishing the species, sex, location,and motivational state of the producer (e.g.,Brown, Beecher, Moody, & Stebbins, 1978;Konishi, 1978; Peterson, Beecher, Zoloth,Moody, & Stebbins, 1978).

It is noteworthy that in each of these areasof research on natural events, the discoveryor explanation of perceptually significant,time-varying acoustic properties has beenmotivated by an analysis of the time-varyingbehavior of the sound source. In the case ofspeech, for example, an analysis of speechproduction has been an integral part of thesearch for the acoustic basis for speech per-ception (e.g., Fant, 1960; Fowler, 1977; Fowler,Rubin, Remez, & Turvey, 1980; Liberman etal., 1967; Verbrugge, Rakerd, Fitch, Tuller,& Fowler, in press). It is also worth notingthat researchers in these areas have oftenfound it useful to characterize acoustic infor-mation in terms of higher order structure insound—that is, in terms of functions overthe variables of frequency, amplitude, andduration. Given the time-varying behavior ofthe sound sources involved, it is not surprisingthat many of these functions are time-depen-dent, denning rates of change and styles ofchange in lower order acoustic variables (forexample, the direction, rate, and change inrate of formant frequency transitions). Finally,the example of systems capable of measuringhigher order physical properties, such as area,without prior measurement of more elemen-tary properties, such as length (Runesbn,1977), has encouraged some researchers tobelieve that, analogously, higher order acousticstructure may be directly detectable.

The role of time-varying properties in theperception of other familiar events in thehuman environment is largely unknown. Ourgoal in this article is to demonstrate thathigher order temporal structure can be im-portant for perceiving such events.

It is apparent from everyday experiencethat listeners can detect significant aspects ofthe environment by ear, from a knock at thedoor to the condition of an automobile engineand the gait of an approaching friend (see

Jenkins, 1984). Such casual observations werecorroborated in experiments by VanDerveer(1979a, 1979b). She presented 30 recordeditems of natural sound in a free identificationtask and found that many events such asclapping, footsteps, jingling keys, and tearingpaper were identified with greater than 95%accuracy. Subjects tended to respond bynaming a mechanical event that producedthe sound and reported their experiem^ interms of sensory qualities only when sourcerecognition was not possible. VanDerveer(1979a) also found that confusion errors inidentification tasks and clustering in sortingtasks tended to show the grouping of eventsby common temporal patterns. For example,hammering was confused with walking, andthe scratching of fingernails was confusedwith filing, but hammering and walking werenot confused with the latter two events.

These results support the general claimthat sound in isolation permits accurate iden-tification of classes of sound-producing eventswhen the temporal structure of the sound isspecific to the mechanical activity of thesource (Gibson, 1966; Schubert, 1974). Ifhigher order information is found to be spe-cific to events, whereas values of lower ordervariables per se are not, then it may be morefruitful to view the auditory system as beingdesigned for the perception of source eventsvia higher order acoustic functions, ratherthan for the detection of quasi-stable soundelements. Schubert (1974) put this succinctlyin his source identification principle for au-ditory perception: "Identification of soundsources, and the behavior of those sources, isthe primary task of the [auditory] system"(p. 126).

This general perspective on auditory per-ception may be called ecological acoustics,by analogy to the ecological optics advocatedby Gibson (1961,1966, 1979) as an approachto vision. The ecological approach combinesa physical analysis of the source event, theidentification of higher order acoustic prop-erties specific to that event, and empiricaltests of the listener's ability to detect suchinformation, in an attempt to avoid the in-troduction of ad hoc processing principles toaccount for perception (Shaw, Turvey, &Mace, 1981). In any natural event, identifiableobjects participate in identifiable transfor-

Page 3: Auditory Perception of Breaking and Bouncing Events: A ... · the auditory perception of these events. Research in auditory perception has tended to emphasize the detection and processing

706 WILLIAM H. WARREN, JR., AND ROBERT R. VERBRUGGE

(a) (b)

Figure 1. Caricatures of mechanical events: Panel a, bouncing; Panel b, breaking.

mations or styles of change. The informationthat specifies the kind of object and its prop-erties under change is known as the structuralinvariant of an event; reciprocally, the infor-mation that specifies the style of change itselfis known as the transformational invariant.These invariants may be described mathe-matically in terms of properties that remainconstant and those that vary systematicallyunder change (Mark, Todd, & Shaw, 1981;Pittenger & Shaw, 1975; Shaw, Mclntyre, &Mace, 1974; Warren & Shaw, 1984).

The present research explores the acoustic,consequences of dropping a glass object andits subsequent bouncing or breaking. Bounc-ing and breaking are two distinct styles ofchange that may be applied to a variety ofobjects. By acoustic and perceptual studiesof these events, we hope to identify the trans-formational invariants specific to the twostyles of change and sufficient to convey themto a listener. (Structural invariants specifyingindividual properties of the objects such assize, shape, and material will not be discussedhere.)

Consider first the mechanical action of abottle bouncing on a hard surface (see Figure1, Panel a). Each collision consists of aninitial impact that briefly sets the bottle intovibration at a set of frequencies determinedby its size, shape, and material composition.This is reflected in the acoustic signal as aninitial burst of noise followed by spectralenergy concentrated at a particular set of

overtone frequencies. Over a series of bounces,the collisions between object and groundoccur with declining impact force and de-creasing ("damped") period, although someirregularities in the pattern may occur becauseof the asymmetry of the bottle. The spectralcomponents are similar across bounces, withrelative overtones varying slightly because ofthe varying orientations of the bottle at im-pact. (The spectrum within each pulse isquasi-stable and is conventionally describedin terms of spectral peaks in a cross sectionof the signal.) These acoustic consequencesmay be described as a single damped quasi-periodic pulse train in which the pulses sharea similar cross-sectional spectrum (Figure 2,Panel a). We suggest that this single pulsetrain constitutes a transformational invariantof temporal patterning for the bouncing styleof change.

Turning to the mechanical action of break-ing (Figure 1, Panel b), we see that a cata-strophic rupture occurs upon impact. As-suming an idealized case, the resulting piecesthen continue to bounce without furtherbreakage, each with its own independent col-lision pattern. The acoustic consequences ap-pear as an initial rupture burst dissolving intooverlapping multiple damped quasi-periodicpulse trains, each train having a differentcross-sectional spectrum and damping char-acteristic (Figure 2, Panel b). We proposethat a compound signal, consisting of a noiseburst followed by such multiple pulse trains,

Page 4: Auditory Perception of Breaking and Bouncing Events: A ... · the auditory perception of these events. Research in auditory perception has tended to emphasize the detection and processing

ECOLOGICAL ACOUSTICS 707

>>o

»'1

p»» pwi jr o r f> • r >» * *

f I'

r » (a)

(b)

Figure 2. Spectrograms of natural tokens: Panel a,bouncing (BNCI); Panel b, breaking (BRK.1).

constitutes a transformational invariant forthe breaking style of change.

Aside from these aspects of temporal pat-terning and initial noise, certain crude spectraldifferences between breaking and bouncingcan be observed by comparing spectrogramsof natural cases (Figure 2). First, the overtonesof breaking events are distributed across awider range of frequencies than are those ofbouncing events. Second, the overtones ofbreaking are denser in the frequency domain.Both of these properties can be traced to thecontrast between a single object in vibrationand a number of disparate objects simulta-neously in vibration.

The following experiments test the hypoth-esis that temporal patterning alone, withoutdifferences in quasi-stable spectral properties,provides effective information for listeners tocategorize breaking and bouncing styles ofchange. (Spectral differences may provide im-portant information about the nature andmaterial of the objects involved.) By super-imposing recordings of pulses from individualpieces of broken glass, artificial cases ofbreaking and bouncing can be constructedfrom a common set of pulses by varying the

temporal correspondences among their colli-sion patterns. In Experiment 1 we establishthat listeners can categorize natural cases ofbreaking and bouncing with high accuracy.In Experiment 2, we examine performancewith constructed cases that include an initialnoise burst and compare it with the resultsfor natural sound. In Experiment 3 we assessthe contribution of the burst by removing itfrom both natural and constructed cases ofbreaking.

Experiment 1: Natural Tokens

The first experiment investigated whethernatural sound provides sufficient acoustic in-formation for listeners to categorize the eventsof breaking and bouncing.

Methodt

Subjects. Fifteen graduate and undergraduate studentsparticipated in the experiment for payment or coursecredit.

Materials. Natural recordings were made of threeglass objects dropping onto a concrete floor covered bylinoleum tile in a sound-attenuated room. The sound ofeach object was recorded with a Crown 800 tape deckwhen the object was dropped from a 1-ft (0.305 m)height (bouncing) and when it was dropped from a 2- to5-ft (0.61-1.525 m) height (breaking). This procedureyielded three tokens of bouncing and three tokens ofbreaking. The objects used and the durations of thebouncing (BNC) and breaking (BRK) events are asfollows: (a) 32-oz (0.946 1) jar: BNCI = 1,600 ms, 22collisions; BRK1 = 1,200 ms. (b) 64-oz (1.892 1) bottle:BNC2 = 1,600 ms, 15 collisions; BRK2 = 550 ms. (c)1-liter bottle: BNC3 = 1,300 ms, 17 collisions; BRK.3 =700 ms. The recordings were digitized at a 20-kHzsampling rate using the Pulse Code Modulation (PCM)system at Haskins Laboratories. A test tape was thenrecorded, containing 20 trials of each natural token inrandomized order for a total of 120 test trials. A pauseof 3 s occurred between trials, and a pause of 10soccurred after every six trials.

Procedure. Subjects, run in groups of 2 to 5, listenedto the tape binaurally through headphones. They weretold that they would be hearing recordings of objectsthat had either bounced or broken after being droppedbut were told nothing about the nature of the objectsinvolved. Their three-choice task was to categorize eachevent as a case of breaking or bouncing, with a don'tknow option, by placing a check in the appropriatecolumn on an answer sheet. To minimize the possibilitythat they would choose one of the two event categoriesif they found the sound unconvincing, subjects wereinstructed to use the don't know category if they couldnot make up their minds, if they could not tell what theevent was, or if it sounded like some other type of event.They were specifically instructed to ignore the nature of

Page 5: Auditory Perception of Breaking and Bouncing Events: A ... · the auditory perception of these events. Research in auditory perception has tended to emphasize the detection and processing

708 WILLIAM H. WARREN, JR., AND ROBERT R. VERBRUGGE

the object involved and attend to "what's happening toit." Subjects received no practice trials or feedback. Therewas a short break after 60 trials, and a test session lastedabout 20 min.

Results and Discussion

Overall performance on natural bouncingtokens was 99.3% correct ("bouncing" judg-ments), and on breaking tokens was 98.5%correct ("breaking" judgments). Don't knowresponses accounted for 0.2% of all answerson bouncing tokens and 0.7% on breakingtokens. Experiment 1 demonstrates that suf-ficient information exists in the acoustic signalto permit unpracticed listeners to categorizethe natural events of bouncing and breaking,at least within the limits of the three-choicetask.

This successful categorization could bebased on various temporal and spectral prop-erties of the signal, as discussed in the intro-duction. Although the bouncing tokens wereall of longer duration than were the breakingtokens, duration is a function of object elas-ticity and height of drop and does not providereliable information specific to a style ofchange. Both spectral properties and tokenduration were controlled in Experiment 2 toisolate the effect of temporal patterning.

Experiment 2: Constructed Tokens

In Experiment 2 we attempted to modelthe time-varying information contained innatural recordings by using constructed casesof bouncing and breaking, eliminating averagespectral differences between the two.

MethodSubjects. Fifteen graduate and undergraduate students

participated in the experiment for payment or coursecredit. None of them had participated in Experiment 1.

Materials. Tokens intended to model bouncing andbreaking were constructed by the following method.Initially, individual recordings were made of four majorpieces of glass from a broken bottle as each piece wasdropped and bounced separately from a low height. Theserecordings were combined in two ways using the PCMsystem.

To construct an artificial bouncing token, the temporalpattern of the recording of each piece was adjusted tomatch a single master periodicity arbitrarily borrowedfrom a recording of a natural bouncing bottle (Figure 3,Panel a). This was accomplished by inserting silencebetween the bounce pulses in recordings of the individualpieces. After the recordings of all four pieces had been

adjusted so that their onsets matched the same pulsepattern, they were superimposed by summing the instan-taneous amplitudes of the digitized recordings. The resultwas a combined pulse pattern with synchronized onsetsfor all bounces, preserving the invariant of a singledamped quasi-periodic pulse train to model bouncing(Figure 4, Panel a).

An artificial breaking token was constructed by read-justing the recordings of the same four pieces to matchfour different temporal patterns (Figure 3, Panel b). Asa first approximation, these master patterns were borrowedfrom measurements of four different bouncing bottles,because the likely patterns of individual pieces of glassin the course of natural breaking were unknown. Thesefour patterns were initiated simultaneously, preceded by50 ms of noise burst taken from the original rupturethat produced the four pieces of glass. The result aftersuperimposing these four independent temporal serieswas a combined pattern with asynchronous pulse onsets,preserving the temporal invariant of multiple dampedquasi-periodic pulse trains to model breaking (Figure 4,Panel b). Note that the variables of temporal patterningand initial noise were confounded in this experiment. To

3 I

I I

I III

I III

I III

(a)

1 I I I I I I

2

3

4

II

II

I ItiI

i i MIi i i iiin

me — »•

(b)

Figure 3. Schematic diagrams of constructed tokens,combining four component pulse trains: Panel a, bounc-ing, with synchronous pulse onsets; Panel b, breaking,with initial noise burst and asynchronous pulse onsets.(Each pulse train has unique spectral properties.)

Page 6: Auditory Perception of Breaking and Bouncing Events: A ... · the auditory perception of these events. Research in auditory perception has tended to emphasize the detection and processing

ECOLOGICAL ACOUSTICS 709

the experimenters' ears the burst improved the quality ofapparent breakage, but this assumption was later testedin Experiment 3.

Three constructed cases of bouncing and three corre-sponding cases of breaking were produced by this method,each pair constructed from a unique set of recordings oforiginal pieces and matched to a unique set of masterperiodicities. To help assure the generality of the method,the first three pairs constructed in this manner were usedin the experiment. The original objects, and the durationsof the bouncing or synchronous (SYN) and breaking orasynchronous (ASYN) tokens constructed from theirpieces, were as follows: (a) 32-oz (0.946 1) jar: SYN 1 =1,000 ms, 8 collisions; ASYN1 =950 ms. (b) 32-oz(0.946 1) jar: SYN2 = 1,400 ms, 13 collisions; ASYN2 =650 ms. (c) 64-oz (1.892 1) bottle: SYN3 = 920 ms, 9collisions; ASYN3 = 600 ms.

Hence, the only differences between constructedbouncing and breaking tokens were in the temporalregistration of pulse onsets and the presence (or absence)of initial noise. The range and distribution of spectralfrequencies, averaged over time, were comparable in thetwo cases. Tokens SYN1, SYN3, and ASYN1 had similardurations of 920 to 1.000 ms, and thus if subjects basedcategorization on this factor, large errors should occurwith these tokens.

There were certain problems with the constructedcases. The process of superimposing pulse patterns alsosummed tape hiss and hum so that background noisewas increased. Moreover, using recordings of only fourlarge pieces of glass concentrated more spectral energyin the lower frequency ranges as compared to naturaltokens (cf. Figures 2 and 4), and constructing the soundof a single bouncing object by combining the spectralcomponents of these four pieces produced two tokens

(a)

Table 1Percentage of Trials in Which ConstructedTokens of Bouncing and Breaking WereCategorized as Such (Experiment 2)

Token Bouncing Breaking

MSD

MSD

MSD

Overall

93.718.7

1.7

94.318.92.0

84.316.93.7

90.7

89.017.82.9

71.314.34.3

99.719.90.3

86.7

Figure 4. Spectrograms of constructed tokens: Panel a,bouncing (SYN1); Panel b, breaking (ASYN1).

Note. Scores are based on 20 trials per subject per cell,N= 15.

that sounded more like metal than like glass material(SYN1 and SYN2). Nevertheless, the temporal invariantwas preserved. Finally, the use of only four pieces ofglass to model breaking, the assumption that their peri-odicities were akin to those of whole bouncing bottles,and the assumption of no further breakage after theinitial catastrophe, were all rather arbitrary idealizations.Nevertheless, if temporal patterning constitutes sufficientinformation for breaking and bouncing, subjects shouldbe able to make reliable judgments of these tokens.

Procedure. The procedure was the same as that inExperiment 1. Instructions to the subjects were the same,including the instruction to ignore object properties andconcentrate on the style of change.

Results and Discussion

The results for each constructed tokenappear in Table 1 and are consistent with thepredictions of the temporal patterning hy-pothesis. Bouncing judgments on synchronoustokens averaged 90.7%, and breaking judg-ments on asynchronous tokens averaged86.7% (these judgments being treated as cor-rect). Don't know answers accounted for 0.1%of all responses on bouncing tokens and 1.3%on breaking tokens. Considering the artificialnature of the constructed cases and the ideal-izations involved, this performance may beconsidered quite high.

The absence of any large systematic dec-rement for tokens SYN 1, SYN3, and ASYN 1indicates that subjects were not relying onsimple duration in making their judgments.

Page 7: Auditory Perception of Breaking and Bouncing Events: A ... · the auditory perception of these events. Research in auditory perception has tended to emphasize the detection and processing

710 WILLIAM H. WARREN, JR., AND ROBERT R. VERBRUGGE

Some departures from the general patternwere found for token ASYN2, which showedmarkedly lower performance (71.3%), a higherintersubject standard deviation, and a rela-tively high rate of don't know responses (4.0%).These differences were primarily due to thelow performance of 5 subjects who averaged44% on this token, whereas the performanceof the other 10 averaged 85%. It may benoted that the summed background noisewas greater in ASYN2 than in the other twobreaking cases. The fact that overall perfor-mance in this case was well above chanceindicates that even the poorest token con-tained usable information. It is not surprisingthat some tokens of constructed breaking aremore convincing than others, because thereare certainly some natural instances that aremore compelling than others. The differencesamong tokens may involve both the spectraldistinctiveness of the component pieces ofglass and their degree of asynchrony.

In general, performance with constructedtokens was well above the chance level, similarto that found for natural tokens in Experiment1. Performance with constructed cases wasonly about 10% lower than with naturalcases, despite the summation of noise, theidealizations of construction, and the metallicsound of tokens SYN1 and SYN2. The datapermit us to conclude that temporal pattern-ing with initial noise provides sufficient in-formation for listeners to categorize breakingand bouncing events.

Experiment 3: Initial NoiseSpecific to Rupturing

To isolate the variable of single versusmultiple pulses and to assess the importanceof the initial noise burst, the first two exper-iments were repeated with initial noise re-moved from both natural and constructedcases of breaking. Experiment 3 was con-ducted to determine whether initial noise, inaddition to the pulse patterns, was necessaryfor subjects to categorize breaking andbouncing.

MethodSubjects. Thirty graduate and undergraduate students

participated in the experiment for payment. None hadparticipated in the previous experiments.

Materials. Both natural and constructed tokens wereprepared. Bouncing tokens were the same as those used

in the two previous experiments. For breaking cases, theconstructed tokens from Experiment 2 were modified byremoving the 50 ms of initial noise that had been addedfor that experiment. The natural breaking tokens fromExperiment 1 were modified by removing the naturallyoccurring burst. Because there was no distinct boundaryin the natural waveform between the rupture burst andthe subsequent collision pulses, the natural tokens wereedited by removing noise identifiable on an oscillogramand by listening for the absence of a burst. This techniqueresulted in the removal of the initial 80 ms from BRK.1,SO ms from BRK.2, and 60 ms from BRK3. In sum,there were three tokens of bouncing and three tokens ofbreaking (without initial noise) in both the natural andconstructed conditions.

Procedure. The natural and constructed conditionswere run using a between-group design, with IS subjectsin each group. The procedure and instructions were thesame as before, with each group receiving 120 randomlyordered trials in blocks of six.

Results and DiscussionThe results for each token appear in Table

2. With natural cases, the overall performancewas 99.8% judgments of bouncing withbouncing tokens and 96.0% judgments ofbreaking with breaking tokens; with the con-structed cases it was 93.0% for bouncing and86.7% for breaking. These results were nearlyidentical to those of Experiment 1 with nat-ural tokens and to Experiment 2 with con-structed tokens. Don't know answers ac-counted for 0.0% of all responses on naturalbouncing tokens, 1.0% on natural breakingtokens, 2.0% on constructed bouncing tokens,and 4.0% on constructed breaking tokens.

Hence, removal of initial noise frombreaking tokens does not reduce perceptualperformance. This finding for the naturalcases indicates that the burst is not necessaryfor categorization of the two events. Thesame finding with constructed cases demon-strates that variation in the temporal pattern-ing of pulse onsets is alone sufficient topermit the categorization of breaking andbouncing.

These results are consistent with the hy-pothesis that multiple quasi-periodic pulsetrains provide sufficient information for lis-teners to categorize breaking. Further workremains to be done to determine whether theinitial noise is necessary for identifyingbreakage under conditions less constrainedthan those in the present experiment.

General DiscussionIn the preceding experiments we have at-

tempted to determine whether higher order,

Page 8: Auditory Perception of Breaking and Bouncing Events: A ... · the auditory perception of these events. Research in auditory perception has tended to emphasize the detection and processing

ECOLOGICAL ACOUSTICS 711

Table 2Percentage of Trials in Which Natural andConstructed Tokens of Breaking and BouncingWere Categorized as Such, Without InitialNoise (Experiment 3)

Natural Constructed

Token Bouncing Breaking Bouncing Breaking

1%MSD

2%MSD

3%MSD

Overall %

99.719.90.3

99.719.9,0.3

100.020.00.0

99.8

93.718.72.4

97.719.51.6

96.719.32.3

96.0

94.018.82.2

97.319.50.9

87.717.52.7

93.0

83.716.72.9

76.715.34.1

99.719.90.3

86.7

Note. Scores are based on 20 trials per subject per cell,N= 15 in the natural and constructed conditions.

time-varying properties constitute effectiveacoustic information for the events of bounc-ing and breaking. The results show that cer-tain damped periodic patterns provide suffi-cient information for listeners to categorizethe two events, without significant differencesin average spectral properties and without aninitial noise burst. The results provide evi-dence that listeners can utilize these transfor-mational invariants for the auditory percep-tion of breaking and bouncing events. Ofcourse, the number of possible acoustic prop-erties, although bounded, is in principle in-definitely large. Only reasonable candidatesfrom'our analysis were experimentally ma-nipulated, and other hypotheses remain opento test.

The strength of the results is mitigated bythe limitations of the three-choice categori-zation task. By definition, the task requiresforeknowledge of the event classes and doesnot exclude the possibility that a token maybe categorized as breaking because it soundssomething like that event, even though itwould not uniquely convey breaking outsidethe restricted conditions of the experiment.Although a free identification task might testthis possibility, the tokens were not con-structed to fully simulate real events of break-

ing and bouncing glass, as evidenced by themetallic sound of bouncing tokens SYN1 andSYN2 and the use of only four componentpieces to construct breaking tokens. Successfulsimulation would require still more complexmanipulations of acoustic structure, includingspectral information for the material com-position of the object, predicated on findingssuch as those reported here.

The amplitude and periodicity require-ments of the temporal patterns in bouncingevents were considered in two simple dem-onstrations worth mentioning here. Iteratinga recording of one bounce pulse to matchthe timing of a natural bouncing sequenceproduced a clear bouncing event, althoughthe usual declining amplitude gradient wasabsent. However, adjusting the pulse patternto create equal 100-ms intervals betweenpulse onsets, thereby eliminating the dampingof the periodic pattern, destroyed the effectof perceived bouncing, even with a decliningamplitude gradient. The rapid staccato soundwas like that produced by a machine, suchas a jackhammer. A damped series of colli-sions, constrained by gravity and the imperfectelasticity of the system, appears necessary tothe information for bouncing, as originallyhypothesized. Experiments are in progress toassess the efficacy of period damping as in-formation for elasticity or "bounciness" itself.

Further research is needed to determinethe acoustic patterns necessary to specifysingle and multiple pulse trains. As suggestedin the original hypotheses, distinct spectralproperties for each asynchronous pulse trainin a breaking token may be required tosegregate the pulse trains and convey theexistence of separate pieces of glass (althoughthe superimposition of synchronous pulsetrains with different frequency spectra didnot yield the perception of separate pieces inthe bouncing tokens). Reciprocally, it may benecessary for the successive pulses of abouncing token to be spectrally similar inorder to convey the unity of the pulse trainand to cohere as the bouncing of a singleobject.

The present experiments exemplify an eco-logical approach to auditory perception,seeking to identify higher order acoustic in-formation for environmental events. Theacoustic consequences of two distinct typesof mechanical events were analyzed for their

Page 9: Auditory Perception of Breaking and Bouncing Events: A ... · the auditory perception of these events. Research in auditory perception has tended to emphasize the detection and processing

712 WILLIAM H. WARREN, JR., AND ROBERT R. VERBRUGGE

temporal and spectral structure, and sometime-varying properties sufficient to conveyaspects of the events to a listener were em-pirically determined. Such work is prelimi-nary to modeling auditory mechanisms ca-pable of detecting these structural and trans-formational invariants (see Mace, 1977).

ReferencesBrown, C. H., Beecher, M. D., Moody, D. B., & Stebbins,

W. C. (1978). Localization of primate calls by OldWorld monkeys. Science, 201, 753-754.

Fant, G. (1960). Acoustic theory of speech production.The Hague: Mouton.

Fletcher, H. (1934). Loudness, pitch, and the timbre ofmusical tones and their relation to the intensity, thefrequency, and the overtone structure. Journal of theAcoustical Society of America, 6, 59-69.

Fowler, C. A. (1978). Timing control in speech production.Dissertation Abstracts International, 38, 3927B. (Uni-versity Microfilms No. 7731074)

Fowler, C. A., Rubin, P., Remez, R. E., & Turvey, M. T.(1980). Implications for speech production of a generaltheory of action. In B. Butterworth (Ed.), Languageproduction. Speech and talk (Vol. 1, 373-420). NewYork: Academic Press.

Gibson, J. J. (1961). Ecological optics. Vision Research,1, 253-262.

Gibson, J. J. (1966). The senses considered as perceptualsystems. Boston: Houghton Mifflin.

Gibson, J. J. (1979). The ecological approach to visualperception. Boston: Houghton Mifflin.

Green, D. M. (1976). An introduction to hearing. Hillsdale,NJ: Erlbaum.

Grey, J. M. (1977). Multidimensional perceptual scalingof musical timbres. Journal of the Acoustical Societyof America, 61, 1270-1277.

Grey, J. M., & Gordon, J. W. (1978). Perceptual effectsof spectral modifications on musical timbres. Journalof the Acoustical Society of America, 63, 1493-1500.

Jenkins, J. J. (1984). Acoustic information for places,objects, and events. In W. H. Warren & R. E. Shaw(Eds.), Persistence and change: Proceedings of the firstinternational conference on event perception (pp. 115-138). Hillsdale, NJ: Erlbaum.

Helmholtz, H. L. F. von (1954). On the sensations oftone as a physiological basis for the theory of music.New York: Dover. (Original work published 1863)

Konishi, M. (1978). Ethological aspects of auditory patternrecognition. In R. Held, H. Leibowitz, & H. L. Teuber(Eds.), Handbook of sensory physiology, Vol. 8: Percep-tion (pp. 289-309). New York: Springer-Verlag.

Liberman, A. M., Cooper, F. S., Shankweiler, D. P., &Studdert-Kennedy, M. (1967). Perception of the speechcode. Psychological Review, 74, 431-461.

Liberman, A. M., Delattre, P. C., Gerstman, L. T., &Cooper, F. S. (1956). Tempo of frequency change as acue for distinguishing classes of speech sounds. Journalof Experimental Psychology, 52, 127-137.

Luce, D., & Clark, M. (1967). Physical correlates ofbrass-instrument tones. Journal of the Acoustical Societyof America, 42, 1232-1243.

Mace, W. M. (1977). James Gibson's strategy for per-ceiving: Ask not what's inside your head, but whatyour head's inside of. In R. E. Shaw & J. Bransford(Eds.), Perceiving, acting, and knowing: Toward anecological psychology (pp. 43-65). Hillsdale, NJ: Erl-baum.

Mark, L. S., Todd, J. T., & Shaw, R. E. (1981). Theperception of growth: A geometrical analysis of howdifferent styles of change are distinguished. Journal ofExperimental Psychology: Human Perception and Per-formance, 7, 355-368.

Petersen, M. R., Beecher, M. D., Zoloth, S. R., Moody,D. B., & Stebbins, W. C. (1978). Neural lateralizationof species-specific vocalizations by Japanese macaques(Macaca fuscata). Science, 202, 324-327.

Pittenger, J. B., & Shaw, R. E. (1975). Aging faces asviscal-elastic events: Implications for a theory of non-rigid shape perception. Journal of Experimental Psy-chology: Human Perception and Performance, 1, 374-382.

Plomp, R. (1964). The ear as a frequency analyzer.Journal oj'the Acoustical Society of 'America, 36, 1628-1636.

Runeson, S. (1977). On the possibility of "smart" per-ceptual mechanisms. Scandinavian Journal of Psy-chology, 18, 172-179.

Saldanha, E. L., & Corso, J. F. (1964). Timbre cues andthe identification of musical instruments. Journal ofthe Acoustical Society of America, 36, 2021-2026.

Schubert, E. D. (1974). The role of auditory perceptionin language processing. In D. D. Duane & M. B.Rawson (Eds.), Reading, perception, and language (pp.97-130). Baltimore: York Press.

Shaw, R. E., Mclntyre, M., & Mace, W. M. (1974). Therole of symmetry in event perception. In R. B. MacLeod& H. L. Pick (Eds.), Perception: Essays in honor ofJames J. Gibson (pp. 276-310). Ithaca: Cornell Uni-versity Press.

Shaw, R. E., Turvey, M. T, & Mace, W. M. (1981).Ecological psychology: The consequence of a commit-ment to realism. In W. Weimer & D. Palermo (Eds.),Cognition and the symbolic processes (Vol. 2, pp. 159-226). Hillsdale, NJ: Erlbaum.

VanDerveer, N. J. (1979a, June). Acoustic informationfor event perception. Paper presented at the Celebrationin Honor of Eleanor J. Gibson, Cornell University,Ithaca, NY.

VanDerveer, N. J. (1979b, June). Confusion errors inidentification of environmental sounds. Paper presentedat the Meeting of the Acoustical Society of America,Cambridge, MA.

Verbrugge, R. R., Rakerd, B., Fitch, H., Tuller, B., &Fowler, C. A. (in press). Perception of speech events:An ecological approach. In R. E. Shaw & W. M. Mace(Eds.), Event perception: An ecological perspective.Hillsdale, NJ: Erlbaum.

Warren, W. H., & Shaw, R. E. (1984). Events andencounters as units of analysis for ecological psychology.In W. H. Warren & R. E. Shaw (Eds.), Persistence andchange: Proceedings of the first international conferenceon event perception (pp. 1-27). Hillsdale, NJ: Erlbaum.

Received January 25, 1984Revision received June 4, 1984 •