Temporal guidance of musicians’ performance movement is an acquired skillRodger, M. W. M.1., 2., O’Modhrain, S.2., 3. and Craig, C. M.1.

1. School of Psychology, Queen’s University Belfast, Belfast, UK2. Sonic Arts Research Centre, School of Creative Arts, Queen’s University Belfast, Belfast, UK3. School of Music, Theatre and Dance, University of Michigan, MI, USA.

Correspondence concerning this article should be addressed to: Matthew W. M. Rodger, School of Psychology, Queen’s University Belfast, David Keir Building, 18-30 Malone Road, Belfast, UK. BT9 5BNE-mail: m.rodger@qub.ac.uk

Telephone: +44 (0) 28 9097 5653

Fax: +44 (0) 28 9097 5486

Abstract

The ancillary (non-sounding) body movements made by expert musicians during performance

have been shown to indicate expressive, emotional and structural features of the music to

observers, even if sound of the performance is absent. If such ancillary body movements are a

component of skilled musical performance then it should follow that acquiring the temporal

control of such movements is a feature of musical skill acquisition. This proposition is tested using

measures derived from a theory of temporal guidance of movement, ‘General Tau Theory’ (Lee

1998; Lee et al. 2001), to compare movements made during performances of intermediate-level

clarinetists before and after learning a new piece of music. Results indicate that the temporal

control of ancillary body movements made by participants was stronger in performances after the

music had been learned, and was closer to the measures of temporal control found for an expert

musician’s movements. These findings provide evidence that the temporal control of musicians’

ancillary body movements develops with musical learning. These results have implications for

other skilful behaviors and non-verbal communication.

Keywords: music performance, skill acquisition, tau theory, temporal control of

movement

1

Introduction

Nearly all actions require the skilful control of movement prospectively

through time to realize their goals, whether these are interceptive (e.g. catching a

ball), locomotive (e.g. timing steps in a triple-jump), or expressive (e.g. ballet

dance). Expressive musical performance is amongst the most skilful activities that

humans are able to achieve. Hundreds of hours of practice are required to master

the fine-grain motor control necessary to produce an aesthetically satisfying

sequence of sounds (Sloboda 1996). However, it is not enough to sound the

correct notes in the right order; in addition, the timing and temporal unfolding of

sequences of notes must be carefully controlled to accomplish an expressive,

engaging musical performance (Palmer 1989; Clarke 1999). Making music,

therefore, is only possible by skillfully coupling the temporal control of

movements to unfolding perceptual goals.

In addition to the effective movements required to excite, sustain or

modulate sound from an instrument, such as bowing across violin strings or

striking the surface of a drum, there are also non-sounding, ancillary movements

made by experts during performance that are musically relevant. Ancillary

movements --- such as upper body sway, nodding, toe-tapping, head shaking, and

other non-sounding gestures --- are perceptually communicative to audiences of

expressive, emotional and structural characteristics of the music, even in the

absence of sound (Dahl and Friberg 2007; Davidson 1993; Vines et al. 2006;

Vines et al. 2011). Indeed, non-musicians have been found to be more accurate at

judging the intended expressive characteristics of performance from visual

observance of ancillary movements in the absence of sound (Davidson 1993).

Furthermore, it is not only the expressive intentions of the performer that can be

detected in their ancillary body movements; the skill level of the player can also

be perceived from vision of body motion alone (Rodger et al. 2012), which

suggests that ancillary performance movements change with and reflect musical

skill acquisition.

For movement to be perceived as skillful there must be some quality in the

biological motion signal that relates to the skilled control of the movement

(Schmidt 2012; Brault et al. 2012). Therefore, if musicians’ ancillary movements

can perceptually specify the skill of the player, this must entail some motor

2

control-relevant property of these movements that is available to the perceiver.

Following this line of argument, changes in musicians' body movements in

performance that occur with learning were studied within the context of a general

theory of temporal guidance of movement: General Tau Theory.

General Tau Theory

General Tau Theory (Lee 1998; Lee et al. 2001) puts forward a unified

account of how organisms can skillfully navigate their environments by coupling

the timing of their movements to the dynamics of external sensory information

and/or oscillations of internal neural power. Drawing on both Gibson’s ecological

psychology (Gibson 1979) and Bernstein's work on the coordination of movement

(Bernstein 1967), the theory posits that the control of any purposeful movement

can be considered as the prospective closure of action-gaps, that is, the gap

between the current state of the organism’s effector and a goal state. Examples of

such action gaps include the distance between a fielder's hand and a cricket ball

(displacement gap), the angle between one’s gaze and the direction of a flash of

light in the periphery of vision (angle gap), the pitch gap between middle-C and

F4 when singing (pitch gap), etc. In all cases, gap closure needs to be controlled

prospectively in time in order to execute a successful action. Thus, to understand

the motor control of a given action, there is a need to identify the temporal

information which is used to guide the closure of the motion gap prospectively in

time (Lee 2004).

It is proposed within this theory that a primary informational invariant for

the prospective control of movement is ‘tau’, which is the changing time-to-

closure of a motion-gap, defined as the ratio between the magnitude of the gap

and its current rate of closure. The control of movement involves keeping the tau

of movement (τX(t)) coupled to a guide tau (τY(t)) at a constant ratio (k),

Eq. 1: τX(t) = kX,Y × τY(t).

As the tau of external events is geometrically related to the tau of the energy array

on different sensory systems (Lee 1998), this invariant property can be directly

perceived, and has been implicated as a control variable in a number of different

behaviors across different species. It has been shown that bats and humming-

birds use auditory and visual tau respectively to control their approach to land

3

(Lee et al. 1995; Delafield-Butt et al. 2010), while gannets time the tucking of

their wings based on optical tau during dives (Lee and Reddish 1981). In humans,

drivers use tau information in optic flow to control braking (Lee 1976; Bootsma

and Craig, 2003), and long-jumpers time their footfalls during a run up based on

the tau between them and the board (Lee et al. 1982). The ability to couple the tau

of movement to sensory tau for control of action has also been found to develop in

infants, with increased tau coupling developing to improve performance in

catching a moving toy (Kayed and van der Meer 2009).

In many actions there is no temporal information directly specified by an

unfolding event in the environment onto which one can couple the tau of

movement. For example, the unaccompanied solo singer before she begins her

first note does not have an extrinsic guide for the temporal closure of a pitch

action gap. General Tau Theory posits that in such instances the temporal control

of movement is intrinsically coupled to an internal tau guide ‘tauG’ generated in

the neural system. This intrinsic guide may be considered as “a changing

canonical energy gap” (Schogler et al. 2008, p362). Mathematically, the dynamics

of this guide can be described as the tau of an object moving under constant

gravity (Lee 1998),

Eq. 2: τG(t) = 0.5 × (t – TG2/t),

where TG is the period of the closure of this guide gap, and t is time across

the duration of TG. Intrinsic guidance of movement through tauG-coupling can thus

be described by the equation,

Eq. 3: τX(t) = kX,G × τG(t),

Evidence for intrinsic tauG-coupling has been found in golfers guiding

their swings in putting (Craig et al. 2000a), musicians controlling note-producing

actions (Schogler et al. 2008), timekeeping movements to intercept beats (Craig et

al. 2005), and infants controlling sucking pressures (Craig and Lee 1999). As with

extrinsic tau-guidance, evidence suggests that strength of tauG-coupling

corresponds to motor skill development. Measures of tauG-coupling control for

both increasing and decreasing suction in preterm infants' were initially markedly

weaker when compared with term infants (Craig et al. 2000b). As preterm infants

matured and became more adept at feeding, the tau-coupling control measures

were found to approach those values obtained for term infants.

4

In both extrinsically and intrinsically tau guided movements, the k-value

or constant ratio between the two taus determines the velocity and acceleration

characteristics of the movement. As k increases from 0 to 1, the temporal position

of peak velocity moves from near the start of the movement (e.g. for k=0.3)

towards the end of the movement. (e.g. for k=0.8). A k value of 0.5 represents a

bell-shaped velocity profile. Qualitatively, tau-coupled movements with greater k-

values have the quality of greater force at the endpoint, or more “oomph”

(Schogler et al. 2008, p363), than movements with lower k-values. In controlling

pitch and intensity glides, double bass players and singers were shown to vary the

k-values of tauG-coupled notes between performances with different emotional

intentions (Schogler et al. 2008). This indicates that tau-coupling as a control of

movement also shapes the expressive qualities of actions.

In this study, tauG-coupling in musicians’ ancillary body movements was

measured as players learned to play a novel piece of music. The learning of a new

piece constitutes a dimension of musical skill acquisition, as entirely novel

sequences and patterns of actions must be acquired even if individual component

actions have been mastered (Sloboda 1996; Drake & Palmer 2000). As the

ancillary movements of expert musicians have been related to the expressive,

emotional and structural flow of music in performance, these movements

constitute an interesting component of skilful communicative performance. In this

study, we investigated whether temporal control of performers’ ancillary

movement changed with the acquisition of musical skill.

Method

Participants

Three clarinetists, with a playing ability ranging from grade levels 5-8 as

defined by the Associated Board of the Royal Schools of Music1, were recruited

through advertisement, and took part voluntarily. One professional clarinetist took

part in a single session to give benchmark values for tauG-coupling of ancillary

movements. This expert was recruited through an artist-in-residence program

within the School of Creative Arts at Queen’s University Belfast.

1 Details of ABRSM grade definitions can be found at the following online address:

http://www.abrsm.org/regions/fileadmin/user_upload/syllabuses/clarinet0510.pdf5

Apparatus and Setup

3D motion-capture data were recorded using 4 Qualisys ProReflex infrared

cameras, running on Qualisys Track Manager software, sampling at 120Hz.

Participants wore 6 reflective passive markers: one on each leg above the knee

joint; one on each upper arm, midway between the elbow and shoulder joints; one

on the bell of the clarinet; and one on a cap worn on the head so that the position

of this marker was above the forehead. These marker placements were chosen

because existing research has identified these anatomical locations as key sites of

musically-relevant movement in clarinet performance (Wanderley et al. 2005).

Audio recordings were taken of each performance, using one Rode stereo

condenser microphone positioned 1 m in front of the participant connected to a

TASCAM HD-P2 digital audio recorder unit.

The piece of music used for all trials in this study was `Strange But True'

from the 2008-2013 ABRSM Clarinet Grade 3 performance syllabus [Associated

Boards of the Royal Schools of Music 2007]. This piece was chosen to be

sufficiently easy to be learned by all participants over the course of the trials.

Procedure

The 3 participants each took part in 5 sessions, with sessions consisting of

4 trial performances. Recording sessions were separated by 1-2 days and

participants were told not to practice the piece between sessions. For each trial,

the participants began playing the music from the score positioned in front of

them, after an auditory signal to indicate the beginning of the trial. Recordings

ended when either the participant had finished playing the piece or a predefined

maximum recording time (60 seconds) had elapsed. The expert took part in a

single session, consisting of 3 trial performances of the same piece, with the same

instruction and recording process. Only movements made during the

performances of the music (between the onset of the first note and offset of the

final note) were used for analysis to ensure that all trials contained movements

coinciding with musical sound. Ethical approval for this study was granted by a

local ethics committee.

6

Analysis

All data were filtered using a 10th-order reverse Butterworth filter, with a

cut-off frequency of 20 Hz. Principal Component Analysis (PCA) was carried out

on the 18-vector (6 markers × 3 dimensions) x, y, z position data from motion

capture of each trial performance. PCA is a data reduction technique for

compression of large data sets (Jolliffe 2002), and has been shown to be

appropriate for feature extraction in human movement analysis (Daffertshofer et

al. 2004). This technique allows the extraction of common motional variance

across different movement points and dimensions, by reducing the dimensional

space of the data set on the basis of covariance between signals. Projecting the

original data onto this new dimensional space results in a signal that maximally

represents this covariance. Hence, for example, a swooping motion of the upper

body which would contain motion of a number of markers in different dimensions

would be captured by PCA as the common covariance of that movement across

each of these different position signals. This covariance is then represented by a

single signal which captures the underlying movement. For the motion analysis in

this study, the first principal component was used, which represents the changing

quantity of movement that best described the overall body motion of the

performer in each trial, that is, the greatest common variance from all markers that

contribute to this movement dimension. From the principal component of each

trial, the individual movements – semi-periods – were calculated as the movement

displacement values on the principal component vector between velocity zero-

crossings. Movements with durations less than 100 ms were discarded as these

were taken to be too short in duration to be relevant to performance.

Following the analysis method used by Craig et al. (2005) and Schogler et

al. (2008), the changing tau profile within each movement semi-period was

calculated as the displacement divided by the velocity with respect to time

throughout the movement. Then, for each movement semi-period, a corresponding

tauG guide of equal duration was calculated using Equation 2, with TG being equal

to the movement duration. The tau of the movement was linearly regressed

against the hypothetical tauG guide recursively, by successively excluding the first

n samples (n1 = 0), until the r-squared of the regression model exceeded 0.95. The

measure of coupling-strength from the analysis is the percentage of the original

movement that remains in the model after n samples have been excluded to give

7

an r-squared greater than 0.95, with higher values indicating stronger coupling of

the tau of the movement to a tauG guide. A second more qualitative measure is the

slope of the regression line between the tauG guide and the tau of the movement,

which is an estimate of the average k-value in the movement.

In addition to the tau analysis of movement, the jerk of movements were

also analysed for comparison, as the inverse of mean jerk of movement is often

used as a measure of smoothness. A dimensionless measure of jerk was calculated

for each individual movement, to account for variance in movement duration and

amplitude, as suggested by Hogan and Sternad (2009), using the following

equation,

Eq 4: (∫t 1

t 2

x⃛ (t )2 dt ¿ D5/ A2

,where D is the duration of an individual movement between t1 and t2, and A is the

movement amplitude.

Musicians’ ratings of audio recordings of performances

To check that the participants had improved in their performance of the piece over

the course of the study, 15 trained musicians, with an average of 17.4 years

playing experience (s.d. = 6.6 years), rated audio recordings of performances

which were selected randomly from each participant in the first, third and final

sessions. Musicians rated performances on two 7-point Likert-scales, technicality

(‘accuracy of pitch, timing, dynamics, timbre’) and expressivity (‘emotion, colour,

flow, personality’). Participants in this rating study were given a copy of the

musical score to assist in making their judgments. The order of presentation of the

audio clips was randomized. Participation was voluntary.

Results

The ratings of audio recordings of performances by trained musicians

(Table 1) revealed that all participants improved in both technical accuracy and

degree of expressivity over the 5 sessions, showing that learning of this piece of

music had taken place. Analysis of improvement in musical performance on each

measure was carried out for each participant by subtracting the ratings given in

the final session from those given in the first and using single sample t-tests to

8

compare these differences to 0 (using a Bonferroni-corrected of alpha value

of .0167 for multiple t-tests). In terms of technicality, ratings in the 5th session

were significantly higher than in the first for all three participants (Participant1:

t(14) = 13.23, p < .001, Cohen’s d = 3.41; Participant2: t(14) = 6.10, p < .001,

Cohen’s d = 1.57; Participant3: t(14) = 2.75, p < .001, Cohen’s d = 0.71). On

expressivity, significant increases in ratings between session 1 and 5 were found

for Participant 1 (t(14) = 3.67, p = .003, Cohen’s d = 0.95) and Participant 2 (t(14) =

5.00, p < .001, Cohen’s d = 1.29). However, although ratings of expressivity for

Participant 3 did increase, this change was not significant after correcting for

multiple t-tests (t(14) = 2.57, p = .022, Cohen’s d = 0.66). It can also be noted from

Table 1 that the mean ratings for the expert performer were higher than those of

the participants, as would be expected.

Table 1. Mean ratings of technicality and expressivity (on 7-point scales) for performances by

each participant in sessions 1, 3 and 5, and for the expert (standard deviations in brackets).

Significant differences in ratings between the first and final sessions are shown,

Technicality Expressivity

Session 1 Session 3 Session 5 Session 1 Session 3 Session 5

Participant 1 1.3 (0.5) 1.9 (0.5) 2.9 (0.6)* 1.5 (0.6) 2.0 (0.7) 2.3 (0.9)

Participant 2 3.5 (1.0) 4.2 (1.3) 5.3 (0.6)* 3.3 (1.0) 4.7 (1.4) 5.0 (1.0)

Participant 3 4.7 (0.9) 4.7 (1.0) 5.4 (1.1)* 3.5 (1.1) 4.1 (1.1) 4.3 (1.2)

Expert 5.8 (0.9) 5.9 (0.7)

* - significant difference between first and final session after Bonferroni correction of alpha value.

Analysis of the ancillary movements of each of the three participants was

carried out as individual case studies. The outcomes of the principal component

analysis for each of the participants, as well as that of the expert clarinetist, are

shown in Figure 1. The figure shows the variance explained by the first n

principal components in each performance that cumulatively sum up to explain >

90 % of the overall variance in the original movement signals. For all participants,

a large amount of the full body movement is captured by the first principal

component. Analysis of correlation between the first principal component and

each of the original motion capture signals revealed that for participants 2 and 3,

the first component correlated highly with movement of the all 6 markers in their

horizontal axes (mean r’s across performances of markers in horizontal axis: P2

9

=.89 – .97; P3 = .80 – .90). This indicates that the main source of ancillary

movement for these participants was in lateral full body sway. This is similar to

results found for the expert, although the principal component of the expert’s

ancillary movement showed more lateral sway in the upper body (mean r’s for

head and arms in horizontal axis: .83 – .95; mean’s for clarinet bell and legs in

horizontal axis: .70 – .78). For participant 1, the movements that correlated

highest with the principal component varied across the study, coming mostly from

the sagittal movement of the upper body in the first session (mean r’s for head and

arms in sagittal axis: .83 – .98), and from the the horizontal movement of the

upper body in the final session (mean r’s for head and arms in sagittal axis: .78

– .97). Thus, this participant’s style of ancillary movement changed over the

course of learning to play the piece of music.

Figure 1. Results from principal component analysis averaged over the 4 performances in each

session for each of the three participants and the expert. Bars show percentage variance from

original data captured by the first principal components that cumulatively sum up to > 90 %

variance explained.

Figure 2 illustrates the steps to calculate measures of tauG-coupling for an

example movement taken randomly from a captured performance. Mean values

for these measures were calculated for each trial, and then a mean average was

taken across the trials for each session. Figure 3 shows the means for percentage-

coupling in movements for each participant, averaged over the four performances

for each of the five sessions, with the mean for the expert shown as a horizontal

line in each case. Single-subject model statistics (Bates et al. 2004) were used to

compare means of percentage tau-coupling between the first and final session for

each participant. Mean percentages of movement-coupled were found to increase

over the five sessions for all three participants (P1: 85.4 % → 93.0 %, p < .01; P2:

90.9 % → 97.6 %, p < .01; P3: 84.7 % → 92.6 %, p < .05). This indicates a

significant increase in the degree of tauG coupling of body movements in

performance as the skill to play the piece of music was acquired. Figure 3 shows

that, for all participants, this increase in mean percentage tau-coupling values tend

towards the benchmark value of the expert performer (mean = 97.3 %).

10

Figure 2. Stages of tau analysis with an example movement. (a) Movement velocities were

calculated and individual movements were extracted from points between velocity zero-crossings.

(b) The tau of each movement was calculated from X (displacement) / Ẋ (velocity). Corresponding

intrinsic tauG guides for each movement were also calculated. (c) The tau of each movement was

recursively linearly regressed against the tauG guide, until the r-squared of the regression exceeded

0.95. The gradient of the regression line constituted the estimated k-value. In this example: r-

squared = 0.987; percentage of movement coupled = 100%; estimated k-value = 0.292.

Figure 3. Mean values for percentage of movement tauG-coupled (percentage of movement

remaining when r > .95 in recursive regression model) for each participant from tau analysis of the

participants’ ancillary body movements over the five recording sessions. The mean percentage tau-

coupling value for the expert clarinetist from one session is shown as a dashed horizontal line. * -

difference between means significant, p < .05. ** - difference between means significant, p < .01.

Error lines represent standard error of means across the four performances within each session.

Estimated k-values from the tau-coupling analysis are shown for each

participant as means across the four performances for each of the five sessions in

Figure 4. These were also compared between the first and final session using

single-subject model statistics. For participant 1 and 2, the decreases in the k-

value over the course of the five sessions were statistically significant (P1: 0.851

→ 0.550, p < .05; P2: 0.676 → 0.484, p < .01). Although participant 3 also

showed a decrease in the k-value of movements over the sessions, this did not

reach significance (P3: 0.760 → 0.605, p > .05). This result indicates that, on the

whole, the average abruptness of body movements decreased as learning of the

musical piece progressed. It can be noted (see Figure 4) that this change in value

was in the direction towards the mean k-value of movements in the expert

performances (mean = 0.529).

Figure 4. Mean estimated k-values from tau analysis of ancillary body movements averaged over

the four performances in each of the five sessions, for each of the three participants. The estimated

k-value for the expert clarinetist is shown as a horizontal dashed line in each case. * - difference

between means significant, p < .05. ** - difference between means significant, p < .01. Error lines

represent standard errors of means across the four performances within each session.

Analysis of the mean dimensionless jerk of movements using single-

subject model statistics (Bates et al. 2004) revealed no significant difference

between the first and the final session for any participant (Figure 5). However, it 11

can be noted that participant 3 did appear to show a decrease in jerk over the five

sessions, even though this was not significant. Also, the mean jerk of movements

made by the expert was lower than the means of the participants. This would

suggest that there may be some slight relationship between musical skill and the

inverse of jerk as a measure of smoothness in ancillary movements. Correlation

analysis between jerk and the tau analysis measures revealed no significant

correlations between jerk and either percentage coupling (P1: r(23) = -.04; P2: r(23)

= -.14; P3: r(23) = -.05, all p > .05) or k-value (P1: r(23) = -.04; P2: r(23) = -.14; P3:

r(23) = -.05, all p > .05). This would indicate that the changes observed in the tauG-

coupling of musicians’ ancillary movements over the course of learning to play

the piece of music are not related to reduction in movement jerk, which is a more

standard measure of movement smoothness, but rather reveal something different

about the manner with which the movements are controlled in time.

Figure 5. Mean dimensionless jerk of ancillary body movements averaged over the four

performances in each of the five sessions, for each of the three participants. The mean jerk of

movement for the expert clarinetist is shown as a horizontal dashed line in each case. Error lines

represent standard errors of means across the four performances within each session.

Discussion

As participants learned to play a new piece of music, the strength of tauG-

coupling of the ancillary movements made while playing the piece of music were

found to significantly increase over the 5 training sessions. This would suggest

that the control of such ancillary movements, which have elsewhere been shown

to be musically relevant in perception and performance (Davidson 1993;

Wanderley et al. 2005), improves over the course of musical skill acquisition.

This trend for greater strength of tauG-coupling of movement after learning is

further supported by comparisons with a professional expert clarinetist, whose

movements in performance revealed the highest degree of tauG guidance.

Coupling strength between the tau of movement and tauG guide have previously

been linked to levels of skill (Craig et al. 2000a) and motor control ability (Craig

et al. 2000b). The results from the present experiment provide evidence that the

performance gestures of musicians (which are generally unintentional, yet related

12

to the music itself) become more controlled as the music being played is learned

and the movements required to play the notes also become more controlled. This

suggests that ancilliary body movements may be an acquired skill in embodied

musical performance.

The results also showed that the average coupling constant, k, of tauG-

guided body movements decreased between the first (unlearned) and final

(learned) performances. As with the results for the strength of temporal control of

movement, the k-values of movements made by the participants in their final

session performances were closer to the results found in performance movements

made by the expert professional musician. The change in the k-value measure

over the course of the trials may relate to the changing intentional qualities of

movements as the piece of music is acquired. Musicians have been shown to

modulate this property for tauG-guidance of their effective, sound-producing

actions for control of expressive effects (Schogler et al 2008), and so the results

here could reflect a convergence towards a particular form of expressive

movement for performance of this piece of music. On the other hand, in a study of

acquisition of balance control in gait initiation from infant to adult, k-values in

tauG-coupled centre-of-pressure movements were found to decrease with age

(Austad and van der Meer, 2007), which was taken to show that as balance control

developed, movements were controlled to be less collision-like and more touch-

like. The present results, therefore, might be taken to show that musicians’

performance body movements became less abrupt while learning the piece of

music, reflecting a greater degree of skilled control.

If the control of ancillary movements in learned music performance is

tauG-coupled, as these results would suggest, then it may be the tau of movement

as it is picked up by observers that is allowing them to accurately judge the skill

level of performers from vision of their movements alone (Rodger et al. 2012). In

rugby, it has been found that expert players tune into the tau of key body

movement segments in detecting deceptive movements made by an attacking

player (Brault et al. 2012). Sensitivity to the tau of movement may also allow

audiences to perceive information about musical performance from musicians’

body movements.

In this study, the sound recordings of performances were judged as both

more technically accurate, and more expressive following musical learning. As

13

the measures of tau-coupling also increased over the sessions, it appears that

improvements in musical technique are accompanied by improvement in the

temporal control of ancillary movements. It is interesting to note that changes in

ratings of expression over the sessions were not significant for Participant 3, and

nor were the changes in k-value, suggesting that this measure of ancillary

movement may be related to musical expressivity.

It would be informative to find a way to relate the tau-coupling of

musicians’ ancillary movements to the relevant higher-order properties of the

sound of musical performance. Schogler et al. (2008) showed that the bowing

movements of cellists and vocal fold movements of singers matched the tau

patterns found in pitch and intensity glides in the resulting musical sound. From

the results in the present study, it could be hypothesized that the temporal patterns

of musicians’ ancillary body movements would correlate with the tau of a higher-

order property of the musical sound, such as the temporal envelope of changing

intensity or tempo across multiple notes, which have been shown to be vehicles

for musical expression (Clarke 1999; Palmer 1989). Todd (1992, 1995) has

argued that the temporal shapes of expressive musical timing follow contours

similar to those created by the motion of objects under gravity, and so given that

the formulation of tauG guides is based on such motion, there may be a natural

link between the patterns of timing in musicians’ ancillary body movements and

the patterns of timing found in musical expression. Presenting mismatched visual

recordings of body movements and sound has been shown to disrupt perceivers’

judgment of skill level in music performance (Rodger et al. 2012), which would

suggest some common neural processes are involved in the perception of musical

sound and ancillary body movement. It is possible that the tau of movements and

that of changing properties in the sound are linked neurally in a multi-modal,

embodied perception of music.

The results found here, namely that musicians’ ancillary performance

movements become more tauG-coupled over the course of learning a new piece of

music, have implications for other skilful and communicative behaviors. The tauG-

coupling of movements in sport or dance may provide a useful metric of a

performer’s developing motor control as they acquire new movement-based skill,

which could assist in training evaluation and instruction. In the domain of speech

behavior, the communicative power of non-verbal gesture (Goldin-Meadow 2003;

14

McNeill 2005), which has parallels with non-sounding gestures in music

(Wanderley and Vines 2006), entails that speech accompanying movements

require skilful motor control. Measuring the degree of temporal guidance of

movements and changes in intentional quality using the techniques employed in

this experiment may provide insight into the control of such movements, and

could help to identify disorders or atypical development in communication.

Conclusion

The ancillary body movements made by clarinetists during performances

showed increasing temporal control during the learning of a new piece of music as

measured by tauG-coupling. Moreover, coupling constant k-values, which describe

the unfolding quality of a movement, decreased over the sessions, indicating a

transition toward more smooth movements as a piece is learned. These results

follow other research into changes in tau guidance for motor control that occur

with improved movement ability. It follows that expressive body movements

made by expert musicians during performance require skilful temporal control,

and that this develops during musical learning. The implications of these findings

are not just important for understanding music performance and the observation of

skill on the part of audiences, but are also relevant to other skilful or

communicative behaviors, such as sports performance, or non-verbal gestural

communication in speech.

Acknowledgements

This research was supported by the SKILLS integrated project funded by the European

Commission (FP6-IST, Proposal/Contract no.: 035005) and by the TEMPUS-G project funded by

the European Research Council (210007 StIG). Thanks go to Rob Plane for giving of his time to

record the expert performances used in this study.

References

Associated Board of the Royal Schools of Music (2007) Selected clarinet exam pieces 2008-2013

Grade 3. ABRSM Publishing, London

Austad H, Van der Meer ALH (2007) Prospective dynamic balance control in healthy children and

adults. Exp Brain Res 181:289-295. doi:10.1007/s00221-007-0932-1

Bates BT, James CR, Dufek JS (2004) Single-subject analysis. In: Sterigou N (Ed.) Innovative

Analysis of Human Movement. Human Kinetics, Leeds, pp 3-28.

15

Bernstein NA (1967) The co-ordination and regulation of movements. Pergamon Press, Oxford

Bootsma RJ, Craig CM (2003) Information used in detecting upcoming collision. Percept 32:525-

544. doi:10.1068/p3433

Brault S, Bideau B, Kulpa R, Craig CM (2012) Detecting deception in movement: The case of the

side-step in rugby. PLoS One 7:e37494. doi: 10.1371/jounral.pone.0037494

Clarke EF (1999) Rhythm and timing in music. In: Deutsch D (Ed.) The psychology of music, 2nd

edition. Academic Press, New York, pp 473-500.

Craig CM, Delay D, Grealy MA, Lee DN (2000a) Guiding the swing in golf putting. Nat 405:295-

296. doi:10.1038/35012690

Craig CM, Grealy MA, Lee DN (2000b) Detecting motor abnormalities in preterm infants. Exp

Brain Res 131:359-365. doi:10.1007/s002219900227

Craig CM, Lee DN (1999) Neonatal control of sucking pressures. Evidence for an intrinsic (tau)-

guide. Exp Brain Res 124:371-382. doi:10.1007/s002210050634

Craig CM, Pepping G, Grealy M (2005) Intercepting beats in predesignated target zones. Exp

Brain Res 165:490-504. doi:10.1007/s00221-005-2322-x

Daffertshofer A, Lamoth CJC, Meijer OG, Beek PJ (2004) PCA in studying coordination and

variability: a tutorial. Clin Biomech 19:415-428. doi:10.1016/j.clinbiomech.2004.01.005

Dahl S, Friberg A (2007) Visual perception of expressiveness in musicians' body movements.

Music Percept 24:433-454. doi:10.1525/mp.2007.24.5.433.

Davidson J (1993) Visual perception of performance manner in the movements of solo musicians.

Psychol Music 21:103-114. doi:10.1177/030573569302100201.

Delafield-Butt J, Galler A, Schogler B, Lee DN (2010) A perception - action strategy for

hummingbirds. Percept 39:1172-1174. doi:10.1068/p6708

Drake C, Palmer C (2000) Skill acquisition in music performance: relations between planning and

temporal control. Cogn 74:1-32. doi:10.1016/S0010-0277(99)00061-X

Gibson JJ (1979) The ecological approach to visual perception. Lawrence Erlbaum Associates,

Hillsdale, NJ

Goldin-Meadow S (2003) Hearing gesture: how our hands help us think. Harvard University Press,

Harvard, Belknap

Hogan N, Sternad, D (2009) Sensitivity of smoothness measures to movement duration, amplitude,

and arrests. J Motor Behavior 41:529-534. doi:10.3200/35-09-004-RC

Jolliffe IT (2002) Principal component analysis, 2nd ed. Springer, New York

Kayed NS, Van Der Meer ALH (2009) A longitudinal study of prospective control in catching by

full-term and preterm infants. Exp Brain Res 194:245-258. doi:10.1007/s00221-008-1692-2

Lee DN (1976) A theory of visual control of braking based on information about time to collision'.

Percept 5:437-459

Lee DN (1998) Guiding movement by coupling taus. Ecol Psychol 10:221-250. doi:

10.1207/s15326969eco103&4_4.

Lee DN (2004) Tau in action in development. In: Reisser JJ, Lockman JJ, Nelson, CA (eds) Action

as an organizer of learning. Lawrence Erlbaum Associates, Hillsdale, NJ, pp 3-49

16

Lee DN, Georgopoulos AP, Clark MJO, Craig CM, Port NL (2001) Guiding contact by coupling

the taus of gaps. Exp Brain Res 139:151-159. doi:10.1007/s002210100725

Lee DN, Lishman JR, Thomson JA (1982) Regulation of gait in long jumping. J Exp Psychol:

Hum Percept Perform 8:448-459. doi:10.1037/0096-1523.8.3.448

Lee DN, Reddish PE (1981) Plummeting gannets: a paradigm of ecological optics. Nat 293:293-

294. doi:10.1038/293293a0

Lee DN, Simmons JA, Saillant PA, Bouffard F (1995) Steering by echolocation: a paradigm of

ecological acoustics. J Comp Physiol 176:347-354. doi:10.1007/bf00219060

McNeill D (2005) Gesture and thought. University of Chicago Press, Chigago

Palmer C (1989) Mapping musical thought to musical performance. J Exp Psychol: Hum Percept

Perform 15:311-346. doi:10.1037/0096-1523.15.2.331

Rodger MWM, Craig CM, O’Modhrain (2012) Expertise is perceived from both sound and body

movement in musical performance. Hum Mov Sci 31:1137-50. doi:10.1016/j.humov.2012.02.012

Schmidt A (2012) Movement pattern recognition in basketball free-throw shooting. Hum Mov Sci.

doi:10.1016/j.humov.2011.01.003

Schogler B, Pepping G-J, Lee DN (2008) TauG-guidance of transients in expressive musical

performance. Exp Brain Res 189:361-372. doi:10.1007/s00221-008-1431-8

Sloboda J (1996) The acquisition of musical performance expertise: deconstructing the "talent"

account of individual differences in musical expressivity. In: Ericsson KA (ed) The road to

excellence: the acquisition of expert performance in the arts and sciences, sports, and games

Lawrence Erlbaum Associates, Hillsdale, NJ, pp 107-126

Todd NPM (1992) The dynamics of dynamics: A model of musical expression. J Acoust Soc Am

91:3540-3550. doi: 10.1121/1.402843

Todd NPM (1995) The kinematics of musical expression. J Acoust Soc Am 97:1940-9

Vines BW, Krumhansl CL, Wanderley MM, Levitin DJ (2006) Cross-modal interactions in the

perception of musical performance. Cogn 101:80-113. doi:10.1016/j.cognition.2005.09.003

Vines BW, Krumhansl CL, Wanderley MM, Dalca IM, Levitin DJ (2011) Music to my eyes:

Cross-modal interactions in the perception of emotions in musical performance. Cogn 118:157-

170 doi:10.1016/j.cognition.2010.11.010

Wanderley MM, Vines BW (2006) Origins and Functions of Clarinettists' Ancillary Gestures. In:

King E, Gritten A (eds) Music and gesture: new perspectives on theory and contemporary practice.

Ashgate, Hampshire, UK, pp 165-191

Wanderley MM, Vines BW, Middleton N, McKay C, Hatch W (2005) The musical significance of

clarinetists' ancillary gestures. J New Music Res 34:97-113. doi:10.1080/09298210500124208

17

Figure 1. Results from principal component analysis averaged over the 4 performances in each

session for each of the three participants and the expert. Bars show percentage variance from

original data captured by the first principal components that cumulatively sum up to > 90 %

variance explained.

18

Figure 2. Stages of tau analysis with an example movement. (a) Movement velocities were

calculated and individual movements were extracted from points between velocity zero-crossings.

(b) The tau of each movement was calculated from X (displacement) / Ẋ (velocity). Corresponding

intrinsic tauG guides for each movement were also calculated. (c) The tau of each movement was

recursively linearly regressed against the tauG guide, until the r-squared of the regression exceeded

0.95. The gradient of the regression line constituted the estimated k-value. In this example: r-

squared = 0.987; percentage of movement coupled = 100%; estimated k-value = 0.292.

19

Figure 3. Mean values for percentage of movement tauG-coupled (percentage of movement

remaining when r > .95 in recursive regression model) for each participant from tau analysis of the

participants’ ancillary body movements over the five recording sessions. The mean percentage tau-

coupling value for the expert clarinetist from one session is shown as a dashed horizontal line. * -

difference between means significant, p < .05. ** - difference between means significant, p < .01.

Error lines represent standard error of means across the four performances within each session.

20

Figure 4. Mean estimated k-values from tau analysis of ancillary body movements averaged over

the four performances in each of the five sessions, for each of the three participants. The estimated

k-value for the expert clarinetist is shown as a horizontal dashed line in each case. * - difference

between means significant, p < .05. ** - difference between means significant, p < .01. Error lines

represent standard errors of means across the four performances within each session.

21

Figure 5. Mean dimensionless jerk of ancillary body movements averaged over the four

performances in each of the five sessions, for each of the three participants. The mean jerk of

movement for the expert clarinetist is shown as a horizontal dashed line in each case. Error lines

represent standard errors of means across the four performances within each session.

22