Author template for journal articles

Temporal guidance of musicians’ performance movement is an acquired skill

Rodger, M. W. M.1., 2., O’Modhrain, S.2., 3. and Craig, C. M.1.

1. School of Psychology, Queen’s University Belfast, Belfast, UK

2. Sonic Arts Research Centre, School of Creative Arts, Queen’s University Belfast, Belfast, UK

3. School of Music, Theatre and Dance, University of Michigan, MI, USA.

Correspondence concerning this article should be addressed to: Matthew W. M. Rodger, School of Psychology, Queen’s University Belfast, David Keir Building, 18-30 Malone Road, Belfast, UK. BT9 5BN

E-mail: m.rodger@qub.ac.uk

Telephone: +44 (0) 28 9097 5653

Fax: +44 (0) 28 9097 5486

Abstract

The ancillary (non-sounding) body movements made by expert musicians during performance have been shown to indicate expressive, emotional and structural features of the music to observers, even if sound of the performance is absent. If such ancillary body movements are a component of skilled musical performance then it should follow that acquiring the temporal control of such movements is a feature of musical skill acquisition. This proposition is tested using measures derived from a theory of temporal guidance of movement, ‘General Tau Theory’ (Lee 1998; Lee et al. 2001), to compare movements made during performances of intermediate-level clarinetists before and after learning a new piece of music. Results indicate that the temporal control of ancillary body movements made by participants was stronger in performances after the music had been learned, and was closer to the measures of temporal control found for an expert musician’s movements. These findings provide evidence that the temporal control of musicians’ ancillary body movements develops with musical learning. These results have implications for other skilful behaviors and non-verbal communication.

Keywords: music performance, skill acquisition, tau theory, temporal control of movement

Introduction

Nearly all actions require the skilful control of movement prospectively through time to realize their goals, whether these are interceptive (e.g. catching a ball), locomotive (e.g. timing steps in a triple-jump), or expressive (e.g. ballet dance). Expressive musical performance is amongst the most skilful activities that humans are able to achieve. Hundreds of hours of practice are required to master the fine-grain motor control necessary to produce an aesthetically satisfying sequence of sounds (Sloboda 1996). However, it is not enough to sound the correct notes in the right order; in addition, the timing and temporal unfolding of sequences of notes must be carefully controlled to accomplish an expressive, engaging musical performance (Palmer 1989; Clarke 1999). Making music, therefore, is only possible by skillfully coupling the temporal control of movements to unfolding perceptual goals.

In addition to the effective movements required to excite, sustain or modulate sound from an instrument, such as bowing across violin strings or striking the surface of a drum, there are also non-sounding, ancillary movements made by experts during performance that are musically relevant. Ancillary movements --- such as upper body sway, nodding, toe-tapping, head shaking, and other non-sounding gestures --- are perceptually communicative to audiences of expressive, emotional and structural characteristics of the music, even in the absence of sound (Dahl and Friberg 2007; Davidson 1993; Vines et al. 2006; Vines et al. 2011). Indeed, non-musicians have been found to be more accurate at judging the intended expressive characteristics of performance from visual observance of ancillary movements in the absence of sound (Davidson 1993). Furthermore, it is not only the expressive intentions of the performer that can be detected in their ancillary body movements; the skill level of the player can also be perceived from vision of body motion alone (Rodger et al. 2012), which suggests that ancillary performance movements change with and reflect musical skill acquisition.

For movement to be perceived as skillful there must be some quality in the biological motion signal that relates to the skilled control of the movement (Schmidt 2012; Brault et al. 2012). Therefore, if musicians’ ancillary movements can perceptually specify the skill of the player, this must entail some motor control-relevant property of these movements that is available to the perceiver. Following this line of argument, changes in musicians' body movements in performance that occur with learning were studied within the context of a general theory of temporal guidance of movement: General Tau Theory.

General Tau Theory

General Tau Theory (Lee 1998; Lee et al. 2001) puts forward a unified account of how organisms can skillfully navigate their environments by coupling the timing of their movements to the dynamics of external sensory information and/or oscillations of internal neural power. Drawing on both Gibson’s ecological psychology (Gibson 1979) and Bernstein's work on the coordination of movement (Bernstein 1967), the theory posits that the control of any purposeful movement can be considered as the prospective closure of action-gaps, that is, the gap between the current state of the organism’s effector and a goal state. Examples of such action gaps include the distance between a fielder's hand and a cricket ball (displacement gap), the angle between one’s gaze and the direction of a flash of light in the periphery of vision (angle gap), the pitch gap between middle-C and F4 when singing (pitch gap), etc. In all cases, gap closure needs to be controlled prospectively in time in order to execute a successful action. Thus, to understand the motor control of a given action, there is a need to identify the temporal information which is used to guide the closure of the motion gap prospectively in time (Lee 2004).

It is proposed within this theory that a primary informational invariant for the prospective control of movement is ‘tau’, which is the changing time-to-closure of a motion-gap, defined as the ratio between the magnitude of the gap and its current rate of closure. The control of movement involves keeping the tau of movement (τX(t)) coupled to a guide tau (τY(t)) at a constant ratio (k),

Eq. 1: τX(t) = kX,Y × τY(t).

As the tau of external events is geometrically related to the tau of the energy array on different sensory systems (Lee 1998), this invariant property can be directly perceived, and has been implicated as a control variable in a number of different behaviors across different species. It has been shown that bats and humming-birds use auditory and visual tau respectively to control their approach to land (Lee et al. 1995; Delafield-Butt et al. 2010), while gannets time the tucking of their wings based on optical tau during dives (Lee and Reddish 1981). In humans, drivers use tau information in optic flow to control braking (Lee 1976; Bootsma and Craig, 2003), and long-jumpers time their footfalls during a run up based on the tau between them and the board (Lee et al. 1982). The ability to couple the tau of movement to sensory tau for control of action has also been found to develop in infants, with increased tau coupling developing to improve performance in catching a moving toy (Kayed and van der Meer 2009).

In many actions there is no temporal information directly specified by an unfolding event in the environment onto which one can couple the tau of movement. For example, the unaccompanied solo singer before she begins her first note does not have an extrinsic guide for the temporal closure of a pitch action gap. General Tau Theory posits that in such instances the temporal control of movement is intrinsically coupled to an internal tau guide ‘tauG’ generated in the neural system. This intrinsic guide may be considered as “a changing canonical energy gap” (Schogler et al. 2008, p362). Mathematically, the dynamics of this guide can be described as the tau of an object moving under constant gravity (Lee 1998),

Eq. 2: τG(t) = 0.5 × (t – TG2/t),

where TG is the period of the closure of this guide gap, and t is time across the duration of TG. Intrinsic guidance of movement through tauG-coupling can thus be described by the equation,

Eq. 3: τX(t) = kX,G × τG(t),

Evidence for intrinsic tauG-coupling has been found in golfers guiding their swings in putting (Craig et al. 2000a), musicians controlling note-producing actions (Schogler et al. 2008), timekeeping movements to intercept beats (Craig et al. 2005), and infants controlling sucking pressures (Craig and Lee 1999). As with extrinsic tau-guidance, evidence suggests that strength of tauG-coupling corresponds to motor skill development. Measures of tauG-coupling control for both increasing and decreasing suction in preterm infants' were initially markedly weaker when compared with term infants (Craig et al. 2000b). As preterm infants matured and became more adept at feeding, the tau-coupling control measures were found to approach those values obtained for term infants.

In both extrinsically and intrinsically tau guided movements, the k-value or constant ratio between the two taus determines the velocity and acceleration characteristics of the movement. As k increases from 0 to 1, the temporal position of peak velocity moves from near the start of the movement (e.g. for k=0.3) towards the end of the movement. (e.g. for k=0.8). A k value of 0.5 represents a bell-shaped velocity profile. Qualitatively, tau-coupled movements with greater k-values have the quality of greater force at the endpoint, or more “oomph” (Schogler et al. 2008, p363), than movements with lower k-values. In controlling pitch and intensity glides, double bass players and singers were shown to vary the k-values of tauG-coupled notes between performances with different emotional intentions (Schogler et al. 2008). This indicates that tau-coupling as a control of movement also shapes the expressive qualities of actions.

In this study, tauG-coupling in musicians’ ancillary body movements was measured as players learned to play a novel piece of music. The learning of a new piece constitutes a dimension of musical skill acquisition, as entirely novel sequences and patterns of actions must be acquired even if individual component actions have been mastered (Sloboda 1996; Drake & Palmer 2000). As the ancillary movements of expert musicians have been related to the expressive, emotional and structural flow of music in performance, these movements constitute an interesting component of skilful communicative performance. In this study, we investigated whether temporal control of performers’ ancillary movement changed with the acquisition of musical skill.

MethodParticipants

Three clarinetists, with a playing ability ranging from grade levels 5-8 as defined by the Associated Board of the Royal Schools of Music[footnoteRef:1], were recruited through advertisement, and took part voluntarily. One professional clarinetist took part in a single session to give benchmark values for tauG-coupling of ancillary movements. This expert was recruited through an artist-in-residence program within the School of Creative Arts at Queen’s University Belfast. [1: Details of ABRSM grade definitions can be found at the following online address: http://www.abrsm.org/regions/fileadmin/user_upload/syllabuses/clarinet0510.pdf]

Apparatus and Setup

3D motion-capture data were recorded using 4 Qualisys ProReflex infrared cameras, running on Qualisys Track Manager software, sampling at 120Hz. Participants wore 6 reflective passive markers: one on each leg above the knee joint; one on each upper arm, midway between the elbow and shoulder joints; one on the bell of the clarinet; and one on a cap worn on the head so that the position of this marker was above the forehead. These marker placements were chosen because existing research has identified these anatomical locations as key sites of musically-relevant movement in clarinet performance (Wanderley et al. 2005). Audio recordings were taken of each performance, using one Rode stereo condenser microphone positioned 1 m in front of the participant connected to a TASCAM HD-P2 digital audio recorder unit.

The piece of music used for all trials in this study was `Strange But True' from the 2008-2013 ABRSM Clarinet Grade 3 performance syllabus [Associated Boards of the Royal Schools of Music 2007]. This piece was chosen to be sufficiently easy to be learned by all participants over the course of the trials.

Procedure

The 3 participants each took part in 5 sessions, with sessions consisting of 4 trial performances. Recording sessions were separated by 1-2 days and participants were told not to practice the piece between sessions. For each trial, the participants began playing the music from the score positioned in front of them, after an auditory signal to indicate the beginning of the trial. Recordings ended when either the participant had finished playing the piece or a predefined maximum recording time (60 seconds) had elapsed. The expert took part in a single session, consisting of 3 trial performances of the same piece, with the same instruction and recording process. Only movements made during the performances of the music (between the onset of the first note and offset of the final note) were used for analysis to ensure that all trials contained movements coinciding with musical sound. Ethical approval for this study was granted by a local ethics committee.

Analysis

All data were filtered using a 10th-order reverse Butterworth filter, with a cut-off frequency of 20 Hz. Principal Component Analysis (PCA) was carried out on the 18-vector (6 markers × 3 dimensions) x, y, z position data from motion capture of each trial performance. PCA is a data reduction technique for compression of large data sets (Jolliffe 2002), and has been shown to be appropriate for feature extraction in human movement analysis (Daffertshofer et al. 2004). This technique allows the extraction of common motional variance across different movement points and dimensions, by reducing the dimensional space of the data set on the basis of covariance between signals. Projecting the original data onto this new dimensional space results in a signal that maximally represents this covariance. Hence, for example, a swooping motion of the upper body which would contain motion of a number of markers in different dimensions would be captured by PCA as the common covariance of that movement across each of these different position signals. This covariance is then represented by a single signal which captures the underlying movement. For the motion analysis in this study, the first principal component was used, which represents the changing quantity of movement that best described the overall body motion of the performer in each trial, that is, the greatest common variance from all markers that contribute to this movement dimension. From the principal component of each trial, the individual movements – semi-periods – were calculated as the movement displacement values on the principal component vector between velocity zero-crossings. Movements with durations less than 100 ms were discarded as these were taken to be too short in duration to be relevant to performance.

Following the analysis method used by Craig et al. (2005) and Schogler et al. (2008), the changing tau profile within each movement semi-period was calculated as the displacement divided by the velocity with respect to time throughout the movement. Then, for each movement semi-period, a corresponding tauG guide of equal duration was calculated using Equation 2, with TG being equal to the movement duration. The tau of the movement was linearly regressed against the hypothetical tauG guide recursively, by successively excluding the first n samples (n1 = 0), until the r-squared of the regression model exceeded 0.95. The measure of coupling-strength from the analysis is the percentage of the original movement that remains in the model after n samples have been excluded to give an r-squared greater than 0.95, with higher values indicating stronger coupling of the tau of the movement to a tauG guide. A second more qualitative measure is the slope of the regression line between the tauG guide and the tau of the movement, which is an estimate of the average k-value in the movement.

In addition to the tau analysis of movement, the jerk of movements were also analysed for comparison, as the inverse of mean jerk of movement is often used as a measure of smoothness. A dimensionless measure of jerk was calculated for each individual movement, to account for variance in movement duration and amplitude, as suggested by Hogan and Sternad (2009), using the following equation,

Eq 4: (

,where D is the duration of an individual movement between t1 and t2, and A is the movement amplitude.

Musicians’ ratings of audio recordings of performances

To check that the participants had improved in their performance of the piece over the course of the study, 15 trained musicians, with an average of 17.4 years playing experience (s.d. = 6.6 years), rated audio recordings of performances which were selected randomly from each participant in the first, third and final sessions. Musicians rated performances on two 7-point Likert-scales, technicality (‘accuracy of pitch, timing, dynamics, timbre’) and expressivity (‘emotion, colour, flow, personality’). Participants in this rating study were given a copy of the musical score to assist in making their judgments. The order of presentation of the audio clips was randomized. Participation was voluntary.

Results

The ratings of audio recordings of performances by trained musicians (Table 1) revealed that all participants improved in both technical accuracy and degree of expressivity over the 5 sessions, showing that learning of this piece of music had taken place. Analysis of improvement in musical performance on each measure was carried out for each participant by subtracting the ratings given in the final session from those given in the first and using single sample t-tests to compare these differences to 0 (using a Bonferroni-corrected of alpha value of .0167 for multiple t-tests). In terms of technicality, ratings in the 5th session were significantly higher than in the first for all three participants (Participant1: t(14) = 13.23, p < .001, Cohen’s d = 3.41; Participant2: t(14) = 6.10, p < .001, Cohen’s d = 1.57; Participant3: t(14) = 2.75, p < .001, Cohen’s d = 0.71). On expressivity, significant increases in ratings between session 1 and 5 were found for Participant 1 (t(14) = 3.67, p = .003, Cohen’s d = 0.95) and Participant 2 (t(14) = 5.00, p < .001, Cohen’s d = 1.29). However, although ratings of expressivity for Participant 3 did increase, this change was not significant after correcting for multiple t-tests (t(14) = 2.57, p = .022, Cohen’s d = 0.66). It can also be noted from Table 1 that the mean ratings for the expert performer were higher than those of the participants, as would be expected.

Table 1. Mean ratings of technicality and expressivity (on 7-point scales) for performances by each participant in sessions 1, 3 and 5, and for the expert (standard deviations in brackets). Significant differences in ratings between the first and final sessions are shown,

Technicality

Expressivity

Session 1

Session 3

Session 5

Session 1

Session 3

Session 5

Participant 1

1.3 (0.5)

1.9 (0.5)

2.9 (0.6)*

1.5 (0.6)

2.0 (0.7)

2.3 (0.9)

Participant 2

3.5 (1.0)

4.2 (1.3)

5.3 (0.6)*

3.3 (1.0)

4.7 (1.4)

5.0 (1.0)

Participant 3

4.7 (0.9)

4.7 (1.0)

5.4 (1.1)*

3.5 (1.1)

4.1 (1.1)

4.3 (1.2)

Expert

5.8 (0.9)

5.9 (0.7)

* - significant difference between first and final session after Bonferroni correction of alpha value.

Analysis of the ancillary movements of each of the three participants was carried out as individual case studies. The outcomes of the principal component analysis for each of the participants, as well as that of the expert clarinetist, are shown in Figure 1. The figure shows the variance explained by the first n principal components in each performance that cumulatively sum up to explain > 90 % of the overall variance in the original movement signals. For all participants, a large amount of the full body movement is captured by the first principal component. Analysis of correlation between the first principal component and each of the original motion capture signals revealed that for participants 2 and 3, the first component correlated highly with movement of the all 6 markers in their horizontal axes (mean r’s across performances of markers in horizontal axis: P2 =.89 – .97; P3 = .80 – .90). This indicates that the main source of ancillary movement for these participants was in lateral full body sway. This is similar to results found for the expert, although the principal component of the expert’s ancillary movement showed more lateral sway in the upper body (mean r’s for head and arms in horizontal axis: .83 – .95; mean’s for clarinet bell and legs in horizontal axis: .70 – .78). For participant 1, the movements that correlated highest with the principal component varied across the study, coming mostly from the sagittal movement of the upper body in the first session (mean r’s for head and arms in sagittal axis: .83 – .98), and from the the horizontal movement of the upper body in the final session (mean r’s for head and arms in sagittal axis: .78 – .97). Thus, this participant’s style of ancillary movement changed over the course of learning to play the piece of music.

Figure 1. Results from principal component analysis averaged over the 4 performances in each session for each of the three participants and the expert. Bars show percentage variance from original data captured by the first principal components that cumulatively sum up to > 90 % variance explained.

Figure 2 illustrates the steps to calculate measures of tauG-coupling for an example movement taken randomly from a captured performance. Mean values for these measures were calculated for each trial, and then a mean average was taken across the trials for each session. Figure 3 shows the means for percentage-coupling in movements for each participant, averaged over the four performances for each of the five sessions, with the mean for the expert shown as a horizontal line in each case. Single-subject model statistics (Bates et al. 2004) were used to compare means of percentage tau-coupling between the first and final session for each participant. Mean percentages of movement-coupled were found to increase over the five sessions for all three participants (P1: 85.4 % → 93.0 %, p < .01; P2: 90.9 % → 97.6 %, p < .01; P3: 84.7 % → 92.6 %, p < .05). This indicates a significant increase in the degree of tauG coupling of body movements in performance as the skill to play the piece of music was acquired. Figure 3 shows that, for all participants, this increase in mean percentage tau-coupling values tend towards the benchmark value of the expert performer (mean = 97.3 %).

Figure 2. Stages of tau analysis with an example movement. (a) Movement velocities were calculated and individual movements were extracted from points between velocity zero-crossings. (b) The tau of each movement was calculated from X (displacement) / Ẋ (velocity). Corresponding intrinsic tauG guides for each movement were also calculated. (c) The tau of each movement was recursively linearly regressed against the tauG guide, until the r-squared of the regression exceeded 0.95. The gradient of the regression line constituted the estimated k-value. In this example: r-squared = 0.987; percentage of movement coupled = 100%; estimated k-value = 0.292.

Figure 3. Mean values for percentage of movement tauG-coupled (percentage of movement remaining when r > .95 in recursive regression model) for each participant from tau analysis of the participants’ ancillary body movements over the five recording sessions. The mean percentage tau-coupling value for the expert clarinetist from one session is shown as a dashed horizontal line. * - difference between means significant, p < .05. ** - difference between means significant, p < .01. Error lines represent standard error of means across the four performances within each session.

Estimated k-values from the tau-coupling analysis are shown for each participant as means across the four performances for each of the five sessions in Figure 4. These were also compared between the first and final session using single-subject model statistics. For participant 1 and 2, the decreases in the k-value over the course of the five sessions were statistically significant (P1: 0.851 → 0.550, p < .05; P2: 0.676 → 0.484, p < .01). Although participant 3 also showed a decrease in the k-value of movements over the sessions, this did not reach significance (P3: 0.760 → 0.605, p > .05). This result indicates that, on the whole, the average abruptness of body movements decreased as learning of the musical piece progressed. It can be noted (see Figure 4) that this change in value was in the direction towards the mean k-value of movements in the expert performances (mean = 0.529).

Figure 4. Mean estimated k-values from tau analysis of ancillary body movements averaged over the four performances in each of the five sessions, for each of the three participants. The estimated k-value for the expert clarinetist is shown as a horizontal dashed line in each case. * - difference between means significant, p < .05. ** - difference between means significant, p < .01. Error lines represent standard errors of means across the four performances within each session.

Analysis of the mean dimensionless jerk of movements using single-subject model statistics (Bates et al. 2004) revealed no significant difference between the first and the final session for any participant (Figure 5). However, it can be noted that participant 3 did appear to show a decrease in jerk over the five sessions, even though this was not significant. Also, the mean jerk of movements made by the expert was lower than the means of the participants. This would suggest that there may be some slight relationship between musical skill and the inverse of jerk as a measure of smoothness in ancillary movements. Correlation analysis between jerk and the tau analysis measures revealed no significant correlations between jerk and either percentage coupling (P1: r(23) = -.04; P2: r(23) = -.14; P3: r(23) = -.05, all p > .05) or k-value (P1: r(23) = -.04; P2: r(23) = -.14; P3: r(23) = -.05, all p > .05). This would indicate that the changes observed in the tauG-coupling of musicians’ ancillary movements over the course of learning to play the piece of music are not related to reduction in movement jerk, which is a more standard measure of movement smoothness, but rather reveal something different about the manner with which the movements are controlled in time.

Figure 5. Mean dimensionless jerk of ancillary body movements averaged over the four performances in each of the five sessions, for each of the three participants. The mean jerk of movement for the expert clarinetist is shown as a horizontal dashed line in each case. Error lines represent standard errors of means across the four performances within each session.

Discussion

As participants learned to play a new piece of music, the strength of tauG-coupling of the ancillary movements made while playing the piece of music were found to significantly increase over the 5 training sessions. This would suggest that the control of such ancillary movements, which have elsewhere been shown to be musically relevant in perception and performance (Davidson 1993; Wanderley et al. 2005), improves over the course of musical skill acquisition. This trend for greater strength of tauG-coupling of movement after learning is further supported by comparisons with a professional expert clarinetist, whose movements in performance revealed the highest degree of tauG guidance. Coupling strength between the tau of movement and tauG guide have previously been linked to levels of skill (Craig et al. 2000a) and motor control ability (Craig et al. 2000b). The results from the present experiment provide evidence that the performance gestures of musicians (which are generally unintentional, yet related to the music itself) become more controlled as the music being played is learned and the movements required to play the notes also become more controlled. This suggests that ancilliary body movements may be an acquired skill in embodied musical performance.

The results also showed that the average coupling constant, k, of tauG-guided body movements decreased between the first (unlearned) and final (learned) performances. As with the results for the strength of temporal control of movement, the k-values of movements made by the participants in their final session performances were closer to the results found in performance movements made by the expert professional musician. The change in the k-value measure over the course of the trials may relate to the changing intentional qualities of movements as the piece of music is acquired. Musicians have been shown to modulate this property for tauG-guidance of their effective, sound-producing actions for control of expressive effects (Schogler et al 2008), and so the results here could reflect a convergence towards a particular form of expressive movement for performance of this piece of music. On the other hand, in a study of acquisition of balance control in gait initiation from infant to adult, k-values in tauG-coupled centre-of-pressure movements were found to decrease with age (Austad and van der Meer, 2007), which was taken to show that as balance control developed, movements were controlled to be less collision-like and more touch-like. The present results, therefore, might be taken to show that musicians’ performance body movements became less abrupt while learning the piece of music, reflecting a greater degree of skilled control.

If the control of ancillary movements in learned music performance is tauG-coupled, as these results would suggest, then it may be the tau of movement as it is picked up by observers that is allowing them to accurately judge the skill level of performers from vision of their movements alone (Rodger et al. 2012). In rugby, it has been found that expert players tune into the tau of key body movement segments in detecting deceptive movements made by an attacking player (Brault et al. 2012). Sensitivity to the tau of movement may also allow audiences to perceive information about musical performance from musicians’ body movements.

In this study, the sound recordings of performances were judged as both more technically accurate, and more expressive following musical learning. As the measures of tau-coupling also increased over the sessions, it appears that improvements in musical technique are accompanied by improvement in the temporal control of ancillary movements. It is interesting to note that changes in ratings of expression over the sessions were not significant for Participant 3, and nor were the changes in k-value, suggesting that this measure of ancillary movement may be related to musical expressivity.

It would be informative to find a way to relate the tau-coupling of musicians’ ancillary movements to the relevant higher-order properties of the sound of musical performance. Schogler et al. (2008) showed that the bowing movements of cellists and vocal fold movements of singers matched the tau patterns found in pitch and intensity glides in the resulting musical sound. From the results in the present study, it could be hypothesized that the temporal patterns of musicians’ ancillary body movements would correlate with the tau of a higher-order property of the musical sound, such as the temporal envelope of changing intensity or tempo across multiple notes, which have been shown to be vehicles for musical expression (Clarke 1999; Palmer 1989). Todd (1992, 1995) has argued that the temporal shapes of expressive musical timing follow contours similar to those created by the motion of objects under gravity, and so given that the formulation of tauG guides is based on such motion, there may be a natural link between the patterns of timing in musicians’ ancillary body movements and the patterns of timing found in musical expression. Presenting mismatched visual recordings of body movements and sound has been shown to disrupt perceivers’ judgment of skill level in music performance (Rodger et al. 2012), which would suggest some common neural processes are involved in the perception of musical sound and ancillary body movement. It is possible that the tau of movements and that of changing properties in the sound are linked neurally in a multi-modal, embodied perception of music.

The results found here, namely that musicians’ ancillary performance movements become more tauG-coupled over the course of learning a new piece of music, have implications for other skilful and communicative behaviors. The tauG-coupling of movements in sport or dance may provide a useful metric of a performer’s developing motor control as they acquire new movement-based skill, which could assist in training evaluation and instruction. In the domain of speech behavior, the communicative power of non-verbal gesture (Goldin-Meadow 2003; McNeill 2005), which has parallels with non-sounding gestures in music (Wanderley and Vines 2006), entails that speech accompanying movements require skilful motor control. Measuring the degree of temporal guidance of movements and changes in intentional quality using the techniques employed in this experiment may provide insight into the control of such movements, and could help to identify disorders or atypical development in communication.

Conclusion

The ancillary body movements made by clarinetists during performances showed increasing temporal control during the learning of a new piece of music as measured by tauG-coupling. Moreover, coupling constant k-values, which describe the unfolding quality of a movement, decreased over the sessions, indicating a transition toward more smooth movements as a piece is learned. These results follow other research into changes in tau guidance for motor control that occur with improved movement ability. It follows that expressive body movements made by expert musicians during performance require skilful temporal control, and that this develops during musical learning. The implications of these findings are not just important for understanding music performance and the observation of skill on the part of audiences, but are also relevant to other skilful or communicative behaviors, such as sports performance, or non-verbal gestural communication in speech.

Acknowledgements

This research was supported by the SKILLS integrated project funded by the European Commission (FP6-IST, Proposal/Contract no.: 035005) and by the TEMPUS-G project funded by the European Research Council (210007 StIG). Thanks go to Rob Plane for giving of his time to record the expert performances used in this study.

References

Associated Board of the Royal Schools of Music (2007) Selected clarinet exam pieces 2008-2013 Grade 3. ABRSM Publishing, London

Austad H, Van der Meer ALH (2007) Prospective dynamic balance control in healthy children and adults. Exp Brain Res 181:289-295. doi:10.1007/s00221-007-0932-1

Bates BT, James CR, Dufek JS (2004) Single-subject analysis. In: Sterigou N (Ed.) Innovative Analysis of Human Movement. Human Kinetics, Leeds, pp 3-28.

Bernstein NA (1967) The co-ordination and regulation of movements. Pergamon Press, Oxford

Bootsma RJ, Craig CM (2003) Information used in detecting upcoming collision. Percept 32:525-544. doi:10.1068/p3433

Brault S, Bideau B, Kulpa R, Craig CM (2012) Detecting deception in movement: The case of the side-step in rugby. PLoS One 7:e37494. doi: 10.1371/jounral.pone.0037494

Clarke EF (1999) Rhythm and timing in music. In: Deutsch D (Ed.) The psychology of music, 2nd edition. Academic Press, New York, pp 473-500.

Craig CM, Delay D, Grealy MA, Lee DN (2000a) Guiding the swing in golf putting. Nat 405:295-296. doi:10.1038/35012690

Craig CM, Grealy MA, Lee DN (2000b) Detecting motor abnormalities in preterm infants. Exp Brain Res 131:359-365. doi:10.1007/s002219900227

Craig CM, Lee DN (1999) Neonatal control of sucking pressures. Evidence for an intrinsic (tau)-guide. Exp Brain Res 124:371-382. doi:10.1007/s002210050634

Craig CM, Pepping G, Grealy M (2005) Intercepting beats in predesignated target zones. Exp Brain Res 165:490-504. doi:10.1007/s00221-005-2322-x

Daffertshofer A, Lamoth CJC, Meijer OG, Beek PJ (2004) PCA in studying coordination and variability: a tutorial. Clin Biomech 19:415-428. doi:10.1016/j.clinbiomech.2004.01.005

Dahl S, Friberg A (2007) Visual perception of expressiveness in musicians' body movements. Music Percept 24:433-454. doi:10.1525/mp.2007.24.5.433.

Davidson J (1993) Visual perception of performance manner in the movements of solo musicians. Psychol Music 21:103-114. doi:10.1177/030573569302100201.

Delafield-Butt J, Galler A, Schogler B, Lee DN (2010) A perception - action strategy for hummingbirds. Percept 39:1172-1174. doi:10.1068/p6708

Drake C, Palmer C (2000) Skill acquisition in music performance: relations between planning and temporal control. Cogn 74:1-32. doi:10.1016/S0010-0277(99)00061-X

Gibson JJ (1979) The ecological approach to visual perception. Lawrence Erlbaum Associates, Hillsdale, NJ

Goldin-Meadow S (2003) Hearing gesture: how our hands help us think. Harvard University Press, Harvard, Belknap

Hogan N, Sternad, D (2009) Sensitivity of smoothness measures to movement duration, amplitude, and arrests. J Motor Behavior 41:529-534. doi:10.3200/35-09-004-RC

Jolliffe IT (2002) Principal component analysis, 2nd ed. Springer, New York

Kayed NS, Van Der Meer ALH (2009) A longitudinal study of prospective control in catching by full-term and preterm infants. Exp Brain Res 194:245-258. doi:10.1007/s00221-008-1692-2

Lee DN (1976) A theory of visual control of braking based on information about time to collision'.

Percept 5:437-459

Lee DN (1998) Guiding movement by coupling taus. Ecol Psychol 10:221-250. doi: 10.1207/s15326969eco103&4_4.

Lee DN (2004) Tau in action in development. In: Reisser JJ, Lockman JJ, Nelson, CA (eds) Action as an organizer of learning. Lawrence Erlbaum Associates, Hillsdale, NJ, pp 3-49

Lee DN, Georgopoulos AP, Clark MJO, Craig CM, Port NL (2001) Guiding contact by coupling the taus of gaps. Exp Brain Res 139:151-159. doi:10.1007/s002210100725

Lee DN, Lishman JR, Thomson JA (1982) Regulation of gait in long jumping. J Exp Psychol: Hum Percept Perform 8:448-459. doi:10.1037/0096-1523.8.3.448

Lee DN, Reddish PE (1981) Plummeting gannets: a paradigm of ecological optics. Nat 293:293-294. doi:10.1038/293293a0

Lee DN, Simmons JA, Saillant PA, Bouffard F (1995) Steering by echolocation: a paradigm of ecological acoustics. J Comp Physiol 176:347-354. doi:10.1007/bf00219060

McNeill D (2005) Gesture and thought. University of Chicago Press, Chigago

Palmer C (1989) Mapping musical thought to musical performance. J Exp Psychol: Hum Percept Perform 15:311-346. doi:10.1037/0096-1523.15.2.331

Rodger MWM, Craig CM, O’Modhrain (2012) Expertise is perceived from both sound and body movement in musical performance. Hum Mov Sci 31:1137-50. doi:10.1016/j.humov.2012.02.012

Schmidt A (2012) Movement pattern recognition in basketball free-throw shooting. Hum Mov Sci. doi:10.1016/j.humov.2011.01.003

Schogler B, Pepping G-J, Lee DN (2008) TauG-guidance of transients in expressive musical performance. Exp Brain Res 189:361-372. doi:10.1007/s00221-008-1431-8

Sloboda J (1996) The acquisition of musical performance expertise: deconstructing the "talent" account of individual differences in musical expressivity. In: Ericsson KA (ed) The road to excellence: the acquisition of expert performance in the arts and sciences, sports, and games Lawrence Erlbaum Associates, Hillsdale, NJ, pp 107-126

Todd NPM (1992) The dynamics of dynamics: A model of musical expression. J Acoust Soc Am 91:3540-3550. doi: 10.1121/1.402843

Todd NPM (1995) The kinematics of musical expression. J Acoust Soc Am 97:1940-9

Vines BW, Krumhansl CL, Wanderley MM, Levitin DJ (2006) Cross-modal interactions in the perception of musical performance. Cogn 101:80-113. doi:10.1016/j.cognition.2005.09.003

Vines BW, Krumhansl CL, Wanderley MM, Dalca IM, Levitin DJ (2011) Music to my eyes: Cross-modal interactions in the perception of emotions in musical performance. Cogn 118:157-170 doi:10.1016/j.cognition.2010.11.010

Wanderley MM, Vines BW (2006) Origins and Functions of Clarinettists' Ancillary Gestures. In: King E, Gritten A (eds) Music and gesture: new perspectives on theory and contemporary practice. Ashgate, Hampshire, UK, pp 165-191

Wanderley MM, Vines BW, Middleton N, McKay C, Hatch W (2005) The musical significance of clarinetists' ancillary gestures. J New Music Res 34:97-113. doi:10.1080/09298210500124208

Figure 1. Results from principal component analysis averaged over the 4 performances in each session for each of the three participants and the expert. Bars show percentage variance from original data captured by the first principal components that cumulatively sum up to > 90 % variance explained.

Figure 2. Stages of tau analysis with an example movement. (a) Movement velocities were calculated and individual movements were extracted from points between velocity zero-crossings. (b) The tau of each movement was calculated from X (displacement) / Ẋ (velocity). Corresponding intrinsic tauG guides for each movement were also calculated. (c) The tau of each movement was recursively linearly regressed against the tauG guide, until the r-squared of the regression exceeded 0.95. The gradient of the regression line constituted the estimated k-value. In this example: r-squared = 0.987; percentage of movement coupled = 100%; estimated k-value = 0.292.

Figure 3. Mean values for percentage of movement tauG-coupled (percentage of movement remaining when r > .95 in recursive regression model) for each participant from tau analysis of the participants’ ancillary body movements over the five recording sessions. The mean percentage tau-coupling value for the expert clarinetist from one session is shown as a dashed horizontal line. * - difference between means significant, p < .05. ** - difference between means significant, p < .01. Error lines represent standard error of means across the four performances within each session.

Figure 4. Mean estimated k-values from tau analysis of ancillary body movements averaged over the four performances in each of the five sessions, for each of the three participants. The estimated k-value for the expert clarinetist is shown as a horizontal dashed line in each case. * - difference between means significant, p < .05. ** - difference between means significant, p < .01. Error lines represent standard errors of means across the four performances within each session.

Figure 5. Mean dimensionless jerk of ancillary body movements averaged over the four performances in each of the five sessions, for each of the three participants. The mean jerk of movement for the expert clarinetist is shown as a horizontal dashed line in each case. Error lines represent standard errors of means across the four performances within each session.

22