1

2

A Five-Stage Process for the Development and Validation of a Systematic

Observation Instrument: The System for Observing the Teaching of Games in

Physical Education (SOTG-PE)

Submitted: August 2010

Word Count = 7,734

1 Faculty of Education, Community, and Leisure, Liverpool John Moores University, IM Marsh Campus, Barkhill Road, Liverpool, L17 6BD. Email: s.roberts2@ljmu.ac.uk; Phone: 0151 231 5426

2 REACH Group, and Faculty of Education, Community, and Leisure, Liverpool John Moores University, IM Marsh Campus, Barkhill Road, Liverpool, L17 6BD. Email: s.j.fairclough@ljmu.ac.uk; Phone: 0151 231 5384

3

ABSTRACT

The primary purpose of this study was to develop and validate a systematic

observation instrument for the teaching of games in physical education (SOTG-PE).

Primary pilot testing of SOTG-PE was conducted in a large selective, all boys

secondary school in the north-west of England. Two hundred and eighty three pupils

aged 11-16 volunteered as participants for the pilot testing phase of this study. The

SOTG-PE coded 1379 minutes of physical education games lessons across three

games categories, including; invasion, net/wall and striking and fielding. In addition,

the teaching interactions of four full-time teachers of physical education and one pre-

service teacher were also coded. The inter-observer and intraobserver mean re-test

coefficients suggest that SOTG-PE is a valid instrument for evaluating and recording

child activity, lesson context and teacher interactions during various physical

education games lessons in the United Kingdom.

KEY WORDS

Physical education

Teaching games

Child activity

Lesson context

Teacher interactions

4

BIOGRAPHICAL NOTE

Simon Roberts is the programme leader for Sports Coaching and a Senior Lecturer in

Physical Education and Sport Pedagogy at Liverpool John Moores University.

Dr Stuart Fairclough is a Reader in Physical Activity Education at Liverpool John

Moores University with research interests in young people’s health and physical

activity in school contexts.

5

Introduction

The teaching of games in physical education has generally dominated school

curricular both internationally and in the United Kingdom (UK) (Fairclough et al.,

2002; Oslin & Mitchell, 2006; Sage, 2003). Indeed, evidence from the recent UK

School Sport Survey indicated that the number of activities offered to pupils in

physical education lessons during 2007-08 increased from a baseline of 14.5 to 17.5

activities. Of the 45 activities listed 23 (51%) were categorised as games, with

association football the most commonly offered activity (TNS, 2007).

According to Almond (1986b), games can be classified into four different

categories (e.g., invasion, striking/fielding, net/wall and target games) and there has

been general agreement amongst researchers regarding the degree of sophistication

and complexity within these various games categories (Oslin, Mitchell & Griffin,

1998; Turner, 2005). For example, Butler and McCahan, (2005) and Hopper (2007)

suggested net/wall games are more complex than striking/fielding games due to their

responsive and continuous nature. For example, in a net/wall game such as tennis

performers are required to return the ball and recover to the centre of the court, place

the ball in the opponent’s court and control the ball flight to make the ball difficult to

return (Hopper, 2007). Moreover, invasion games (i.e. soccer, rugby, netball etc) are

reported to be more complex than net/wall games as the opposition can interact with

different spaces and individuals on the pitch, field or court, and require the use of

sending, receiving and retaining skills, with lots of off-the-ball cognitive actions

based on the movement of the ball and other players in relation to the goal. Gréhaigne

& Godbout, (1995) added a new dimension to invasion game complexity by defining

the four potential roles performed by participants in invasion games as: scoring,

conserving the ball (or projectile), recovering the ball (or projectile), and defending

6

one’s goal. These are similar to the invasion game roles described by Wilson (2002),

namely: on-the-ball attacker, off-the-ball attacker, on-the-ball defender and off-the-

ball defender.

In recent years there has been considerable debate surrounding the most

appropriate method of teaching these games in physical education settings (Wright et

al., 2005). This debate has evolved due to the evolution of alternative instructional

models associated with teaching games, such as Teaching Games for Understanding

(TGfU) (Bunker & Thorpe, 1982) and Sport Education (Siedentop, 1994; Siedentop,

Hastie & van der Mars (2007). The TGfU model is designed to promote pupil

decision making and places an emphasis on pupils developing and solving tactically

based problems before acquiring technical skills (Holt, Strean & Bengoechea;

Hopper, 2002; Metzler, 2005). A significant element of the TGfU model is the

employment by the teacher of specific pedagogic principles, including, amongst

others, the modification of the game and the promotion of a questioning strategy

(Griffin, Brooker & Patton, 2005). For example, the teacher can, modify the playing

area (e.g., make it larger/smaller), modify the equipment (e.g., use batting T’s), adapt

the rules (e.g., no fielders can move until all the balls have been hit) and restrict the

number of participants (e.g., 3v3, 4v4) (Hopper, 2002).

It has been almost three decades since TGfU was first conceived and during

this period there have been a number of alternative tactical games models which have

emerged including; play practice (Launder, 2001), games sense (den Duyn, 1997),

tactical approaches to teaching games (Griffin, Mitchell, & Oslin, 1997), the games

concept approach (Rossi et al., 2007) and the tactical-decision learning model

(Gréhaigne, Wallian & Godbout, 2005). Thus, there are now a number of

instructional approaches associated with tactical games pedagogy, therefore, to avoid

7

confusion and to provide consistency within the remainder of this paper, all tactical

approaches to teaching games will be referred to as Tactical Games Models (TGM)

(Metzler, 2005). The evolution of TGM originated with a dissatisfaction with the

traditional, skill-drill approach to teaching games, which has been reported to follow a

sequence of warm-up, skills/ technique followed by a short game. (Gréhaigne,

Richard & Griffin, 2005). The skill-drill approach has received criticism for a

number of reasons, including; perpetuating the learner’s lack of skilful ability and

leading to a decrease in motivation (Wright et al., 2005) and a concern that a

dominant skill base approach could lead to a ‘cycle of perceived incompetence’

(Fairclough et al., 2002, p.3). On the other hand, advocates of teaching skill

proficiency in physical education, report associations between skill level and

perceived competency and maintain that a key function of physical education is the

development of motor skills (Lounsbery & Coker, 2008).

Despite criticisms of the skill-drill approach and the immense interest

generated by TGM (Griffin et al., 1995; Kirk & MacPhail., 2002; Metzler, 2005;

Mitchell et al., 1995; Wright et al., 2005), there is insufficient empirical evidence to

support the widespread adoption of TGM pedagogy by teachers of physical education.

A possible reason for this is forwarded by Butler (2005) who suggested, critics of

TGM generally reject the notion that pupils have sufficient knowledge to play the

game before mastering techniques and skills. The lack of evidence supporting the

wide-spread use of TGM pedagogy is somewhat surprising, considering the attention

TGM has received in Initial Teacher Education (ITE) and pre-service teacher

education research in preparing teachers to teach games (Gurvitch et al., 2008;

Roberts, 2007; Rovegno, 1992, 1993; Wright et al., 2005; Wright et al., 2006).

8

Although advocates and enthusiasts of TGM are well documented in the

physical education literature (Alexander & Penney, 2005; Butler, 2005; Griffin et al.,

1995; Holt et al., 2002; Hopper, 2003; Kirk & McPhail, 2002; Light, 2004; Mitchell

et al., 1995; Wright et al., 2005;), there still remains some doubt as to the actual

impact TGM have had on teachers and coaches (Evans & Clarke, 1998; Laws; 1994).

This is somewhat surprising considering the volume of column inches dedicated to

TGM. A comprehensive review of the major TGM studies and their respective

methodologies conducted by Oslin & Griffin (2006) revealed how researchers have

progressed from studying TGM from an information processing perspective (i.e.

comparing technical and tactical performance in the form of skill tests) to

understanding the pedagogic processes required to teach from a TGM perspective. A

number of these studies reported research designs which measured the impact or

effectiveness of TGM on either pupils or teachers and are referred to as the ‘teaching

experiment’ approach (Brooker et al., 2000; 2006; Holt et al., 2006; Lee & Ward,

2009). Historically, the ‘teaching experiment’ approach has centred on attempts by

researchers to provide superior evidence for one approach to teaching games (i.e.

technique/skills) against another (i.e. tactics) (Hopper, 2002). However, these studies

were mainly inconclusive due to insufficient results generated by concerns regarding

the study designs, length of time to see a difference and interaction of prior

experiences that tended to be skill based instruction (Allison & Thorpe, 1997; French

& Thomas, 1987; Rink, 1996; Turner & Martinek, 1999).

A recent game performance study which adopted a ‘teaching experiment’ approach

reported some interesting findings regarding the participants’ improvements in

learning tactical concepts in tag rugby (Lee & Ward, 2009). However, the researchers

provided no objective evidence to demonstrate the effective delivery of the intended

9

tactical and technical instructional approaches. To avoid this situation future studies

of TGM, or indeed studies examining the technical elements of games play should

consider following the guidelines of Hastie et al., (2009) and demonstrate instruction

and treatment validity. In other words, researchers should demonstrate that the

intended instruction or instructional approach under scrutiny meets accepted levels of

conformity (Hastie et al., 2009). Therefore, it is proposed that the development and

validation of a systematic observation instrument, designed specifically to record the

interactions which exist between teachers adopting skill learning approaches and

tactic learning progressions will enable researchers involved in future studies of

TGM to more robustly demonstrate instruction and treatment validity (Hopper, 2003).

The purpose of this proposed observation system is therefore different to existing

game performance systems such as the Teams Sport Assessment Procedure (TSAP)

(Gréhaigne, Godbout & Bouthier, 1997) and the Game Performance Assessment

Instrument (GPAI) (Oslin, Mitchell, & Griffin, 1998). The TSAP and GPAI are

multi-dimensional systems designed to measure games playing ability. The TSAP

was initially designed to measure invasion game performance and enabled teachers or

researchers to record various measures of games playing performance such as

receiving the ball, passing the ball and shooting the ball (Memmert & Harvey, 2008).

The GPAI was also designed to record game play components and included decision-

making, skill execution, support, game involvement and overall game performance

measures (Oslin & Mitchell, 2006). The use of the GPAI has been reported in a

number of studies (Griffin et al., 1995; Mitchell & Oslin, 1999; Roberts, 2007) and

recently has received modifications to the scoring and coding system (see Memmert

& Harvey, 2008). Although both of these games performance systems are multi-

dimensional and enable game performance data to be collected across all four game

10

categories (i.e., invasion, net/wall, striking and fielding and target) they were not

designed to collect additional contextual variables which exist in the games teaching

environment. The proposed observation system is therefore unique and in our view,

will provide researchers interested in the pedagogic approaches adopted by teachers

of physical education the flexibility to record skill learning instruction as well as

tactic-to-skill instruction. The present study therefore had two aims; a) to develop a

systematic observation instrument for the teaching of games in physical education,

termed the System for Observing the Teaching of Games in Physical Education

(SOTG-PE); b) to determine the reliability and validity of SOTG-PE.

Development and validation of SOTG-PE

The SOTG-PE permits the simultaneous recording of an individual target

child’s physical activity type, lesson activity context and teacher interactions. Data

for different individual children and teachers can be summed to provide information

on the overall games lesson environment. Thus, the system provides for the

simultaneous recording of students activities and behaviours during games lessons

and allows comparisons among children within the same games lesson and over time.

SOTG-PE uses momentary time sampling techniques (10-seconds of observation

followed by 10-seconds for recording what was observed), where the observation of a

child’s activity, lesson context, are recorded during each interval. Partial interval

coding techniques are also employed in the observation interval to record the

pedagogic interactions of the teacher. The remainder of this paper will provide a

comprehensive outline of the methodology employed in the development of SOTG-

PE. The methodology will follow the five-stage process outlined by Brewer & Jones

(2002) in their validation of a systematic observation instrument specific for rugby

union.

11

Stage 1: Observer Training

According to Bakeman & Gottman (1997, p.56) when implementing coding

schemes and recording measurements of observable behaviour,

it becomes especially important to convince others that what was observed

does not unduly reflect either the investigator’s desires or some

idiosyncratic worldview of the observer.

The development of SOTG-PE therefore followed the guidelines of Bakeman &

Gottman (1997), Brewer & Jones (2002), McKenzie (2002) and Sharpe & Koperwas

(1997) by first addressing the issue of researcher reliability in direct observation

techniques. Therefore the first author undertook a rigorous process of observer

training using a pre-validated observation instrument.

This first stage included viewing and coding physical education lessons using

the System for Observing Fitness Instruction Time (SOFIT) training video and

manual (McKenzie, 2002; McKenzie et al., 1991). The decision to use the SOFIT

training video and manual was based essentially on its previous reliability and validity

in physical education activity studies and in particular games lessons (Arnett & Lutz,

2003; McKenzie et al., 2002) and its employment of momentary time sampling

recording procedures. Within SOFIT, every 20 seconds the observer is prompted by a

digitised audio file to observe a target child and then record the child’s behaviour;

then the lesson context, and finally the teachers’ behaviour.

The SOFIT training videos also provided important information regarding the

development of training and assessment video tapes for later use in the study. For

example, the training video demonstrated the importance of zooming in on the target

child, while at the same time, preserving sufficient background information to enable

accurate coding of additional categories. The advantage of subjects wearing remote

12

microphones is that their verbal comments can be recorded and the observer can focus

on the smooth operation of the camera to maintain the observer’s true focus

throughout the lesson (McKenzie, 2002). The first author followed the guidelines

outlined in the SOFIT training video and practised recording children’s lesson activity

(i.e., student physical activity engagement), lesson context (i.e., how is the time

allocated for the class as a whole), and teacher behaviour (i.e., what is the teacher

doing). The SOFIT training video permits immediate feedback regarding the

accuracy of recorded codes, plus it enables the observer to become familiar with the

reporting of the total number of observed intervals and the calculation of the

percentage in time for each of the SOFIT categories.

In order to demonstrate the intraobserver reliability of the first author using

momentary time sampling protocols, a pre-recorded physical education games lesson

unconnected to the current study was coded. Each randomly selected child was

observed on a rotational basis for four minutes. The SOFIT categories; lesson activity

(i.e., lying, sitting, standing, walking and very active) lesson context (i.e., general

content, PE knowledge content and PE motor content) and teacher behaviour

(promotes fitness, demonstrates fitness, instructs generally, manages, observes and

other tasks) were coded until the end of the lesson. After a period of ten days, a

sufficiently lengthy period to ensure complete memory lapse, the same lesson was

coded again (Brewer & Jones, 2002). Mean retest values of intraobserver agreements

for lesson activity were (91%), lesson context (90%) and teacher behaviour (87%).

Agreement levels were initially set at (80%) which according to Kazdin (1982) is

appropriate. However, previous validation studies of SOFIT have set reliability

values of 90% (Pope et al., 2002). Therefore, the teacher behaviour scale was

reviewed until reliability was established at 90%. The first author’s proficiency in

13

momentary time sampling procedures, memorising SOFIT codes and accurately

distinguishing between SOFIT category codes culminated in approximately eight

hours of training.

Stage 2: Development of SOTG-PE

The SOTG-PE was initially developed through independent consultation with

four experts (three males and one female) from the field of physical education,

physical activity research, sports coaching and Initial Teacher Education (ITE).

Specifically, they included two level three hockey coaches with international

coaching experience, a researcher in physical activity and an experienced teacher of

physical education employed in a large secondary school in Merseyside, United

Kingdom (UK). All of the experts had at least 10 years teaching/coaching experience

and were recruited specifically for their expertise in the teaching of games and in the

development of observation systems. Two of the experts have published material on

the application of decision making and instructional approaches to teaching games.

One of the experts was qualified in the implementation and development of

systematic observation instruments such as SOFIT and System for Observing

Children’s Activity and Relationships during Play (SOCARP) (Ridgers, Stratton, &

McKenzie, 2010). The final expert was an experienced teacher of physical education

who was also a highly qualified UK National Governing Body of sport coach

education tutor.

At the development phase of any systematic observation instrument it has

been suggested that the instrument includes the primary range of behaviours exhibited

by both performers and practitioners, and no behaviours are omitted or given

unnecessary prominence (Brewer & Jones, 2002). Therefore the panel of experts

were requested to comment on the range of suggested behaviours and report on

14

whether they were congruent with those expected to be exhibited in the teaching of

games in physical education. The panel of experts were in agreement that the primary

purpose of the proposed systematic instrument should contrast to that of the Games

Performance Assessment Instrument (GPAI) (Oslin, Mitchell & Griffin, 1998) and the

Teams Sport Assessment Procedure (TSAP) (Gréhaigne, Godbout & Bouthier, 1997).

In other words the instrument was not designed to measure games performance and

therefore SOTG-PE does not discriminate between variations in actual games playing

ability. The final version of SOTG-PE which achieved complete agreement amongst

the panel of experts is presented in Table 1.

[Insert Table 1 about here]

Stage 3: Primary Pilot Testing of SOTG-PE

Primary pilot testing of SOTG-PE was conducted in a large selective, all boys

secondary school in the north-west of England. The school has a strong reputation

both locally and nationally for its excellence in physical education and school sport

and enjoys particular success in the major team games of rugby union, hockey, cricket

and European handball. Two hundred and eighty three pupils aged 11-16 years,

volunteered as participants for the pilot testing phase of this study. All participants

provided written parental informed consent for the study which had ethical approval

from a local university’s ethics committee. In addition, four full-time members of the

physical education department plus a pre-service student teacher [on practicum]

volunteered to take part. The four full-time teachers were selected because of their

recognised expertise and knowledge in the teaching of games. Although the teachers

were informed about the nature of the study they were asked not to modify or change

15

their teaching behaviours in any way. They were not shown a copy of SOTG-PE

protocols before data collection commenced.

Procedures

The pilot-phase observations were conducted by the first author, who visited

the school twice a week for four consecutive weeks during March 2009. A total of

twelve 50 minute lessons (European handball=9; volleyball=1; field hockey=1;

basketball=1) were observed and recorded. After accounting for the time taken for

pupils to change into their physical education uniforms, the longest lesson was

recorded at 49 minutes and the shortest 35 minutes, resulting in a total of 545 minutes

of video footage to be coded using the SOTG-PE. Four children were randomly

selected from the register, before each lesson. Each randomly selected child wore a

highly visible coloured vest, which contrasted with the school’s regulation physical

education uniforms worn by the other participants in the class.

Eleven lessons were video recorded in the school sports hall from a viewing

gallery which overlooked the lesson environment and one lesson (field-hockey) was

recorded outdoors on the school’s astro-turf pitch. A Sony HDV 1080i camera with

wide-angle lens was mounted onto a tripod and the Sennheiser EW 100-ENG G2

Wireless Microphone System was configured to identical frequency banks in order to

capture the verbal comments of the teacher. The transmission outputs were highly

reliable during all of the lessons, including the field-hockey session which was

outdoors and required an extended range of reception. The observer used a stop-watch

and the counter on the video-camera viewer to ensure each of the target children were

observed for four minutes before re-locating to the next child. The radio transmitter

was connected and mounted to the video camera enabling the capture of both the

lesson context and audible teacher interactions. In total, the behaviours of 48 children

16

and four teachers were observed and recorded. The video footage was digitised into

electronic format (DVD) by specialist University technical support staff and loaded

onto a password protected computer for coding.

All 12 lessons were coded using SOTG-PE by the first author. A pre-recorded

audio file with the 10-second “observe” and “record” prompts was used. At the

beginning of the record prompt the target child’s activity type and lesson context were

recorded. Following the “to observe” prompt, partial interval recording was used to

record the teacher interactions. After a 10 day period two lessons were re-coded for

intraobserver reliability of the observer and the instrument. The total number of

observations, agreements and disagreements can be viewed in table 2.

[Insert table 2 about here]

Stage 4: Observer Training

The fourth stage in the validation of the SOTG-PE was the training of an

observer unfamiliar with the observation instrument. A research student enrolled in a

sport related Master’s programme agreed to undertake observer training. This

student was experienced in the use and application of notational analysis and team

play analysis using digital video technology.

Intraobserver reliability

The first author re-visited the school and recorded six physical education

lessons following identical procedures to those already described (European

handball=2; basketball= 1; field hockey =1; volleyball=1; cricket=1) the total duration

of these lessons were 280 minutes. The longest lesson was timed at 48 minutes with

the shortest timed at 44 minutes. In addition 24 children were observed as well as

four members of the physical education department and one pre-service student

teacher.

17

Following a discussion of the SOTG-PE categories the observer was provided

with recorded examples of the behaviours and event occurrences specific to the child

activity type category, lesson context category and teacher interaction category

(Sharpe & Koperwas, 1997). Using the videotaped examples, the observer practiced

coding the lessons. However, each of the SOTG-PE categories were coded

independently at first. For example, at the 10-second prompt the observer only coded

the child activity type category. During this period the observer received feedback

from the first author. The procedure for coding the remaining categories was similar

and again the first author provided feedback in relation to lesson context and teacher

interactions. During the training phase the observer was requested to memorise the

observational protocols and codes, become familiar with the instrument notation,

discriminate between different child activity, lesson context and teacher interactions

and practice coding independently of the first author (Ridgers, Stratton & McKenzie,

2010).

For intraobserver purposes the observer coded two lessons, using the full

SOTG-PE instrument, a week apart. The first lesson was timed at 46 minutes the

second lesson was timed at 44 minutes. Intraobserver levels were set at (85%)

(Brewer & Jones, 2002). The intraobserver agreement scores for the observer using

SOTG-PE were: child activity (88%), lesson context (91%) and teacher interactions

(87%). The total training time required to establish intraobserver agreement was 18

hours.

Inter-observer reliability

Following accepted levels of intraobserver agreements levels the next stage in

the development and validation of the SOTG-PE was to conduct video recorded

coding and establish inter-observer values for both the first author and the observer

18

(Sharpe & Koperwas, 1997). The first author and the observer visited the school twice

a week over a period of two consecutive weeks and recorded 12 physical education

games lessons (rounders=6; cricket=6). The overall duration of these lessons were

554 minutes. Away from the school environment the first author and the trained

observer simultaneously and independently coded the recorded video lessons using

the SOTG-PE and inter-observer agreement levels were established.

Stage 5: Content Validity of SOTG-PE

The fifth and final stage in the development and validation of SOTG-PE was

to establish content validity of the SOTG-PE instrument. According to Morrow

(2002, p.42) ‘content validity is based on the logical interpretation made about the

truthfulness of an instrument’. In order to establish content validity and follow the

validation procedures outlined by Brewer & Jones (2002) and Oslin, Mitchell and

Griffin (1998) a decision was made to recruit physical education specialists in order to

provide comments and feedback on each of the SOTG-PE categories. A sample of 12

recently qualified pre-service teachers who specialised in model based games

instruction (Metzler, 2005) and four expert teachers of physical education with an

average of 11.0 (± 3.4) years teaching experience and no previous knowledge of the

SOTG-PE agreed to take part. The four expert teachers were recruited because of

their high levels of knowledge and experience in the various games categories. The

four expert teachers all held UK coaching level 3 qualifications in rugby union,

cricket, badminton and hockey. The number of reviewers recruited for this stage in

the validation process is consistent with previous validation studies (Brewer & Jones,

2002; Oslin, Mitchell & Griffin, 1998, Ridgers, Stratton & McKenzie, 2010). Each

of the 16 reviewers was provided with a copy of the SOTG-PE instruction manual, a

19

copy of the SOTG-PE recording sheet and crib sheet with a number of questions

similar to those asked by Brewer & Jones (2002):

(1) Are important areas in the teaching of games omitted from the behaviour

categories or lesson context categories?

(2) Are unimportant elements of the behaviours or lesson context areas

erroneously included?

(3) Are all aspects of the content indicative of the child activity type, lesson

context and teacher behaviour in a typical physical education games lesson

environment?

All of the reviewers were encouraged to observe physical education games

lessons in their respective schools whilst simultaneously annotating comments on

either the recording sheet or the instruction manual. After two weeks all the

reviewers had returned their comments regarding the terms and definitions of SOTG-

PE. No additional categories or behaviours were deemed necessary by the reviewers

However, a number of the panel did comment on the sophisticated nature of the

instrument and had concerns it would not be possible to observe and code all the

items accurately. Two of the reviewers in particular were concerned that they would

have to use the instrument live as a form of games assessment. However, after a

direct correspondence via e-mail to both reviewers it was pointed out that SOTG-PE

was not designed to measure pupils’ performance in games per se and thus is very

different to other games performance instruments such as the Games Performance

Assessment Instrument (Memmert & Harvey, 2008; Oslin, Mitchell & Griffin, 1998)

and the Team Sport Assessment Procedure (Grehaigne et al., 1997). One question

posed by a member of the panel surrounded the transferability of the SOTG-PE into a

20

sports coaching context where the content and teaching/coaching interactions were, in

his view, similar.

[INSERT FIGURE 1 Sample SOTG-PE coding form here]

Data Analysis

Reliability

In total 12 of the 30 (40%) video recorded physical education levels were

coded both simultaneously and independently to establish inter-observer agreement.

The level of inter-observer reliability was established for each SOTG-PE category

(i.e. child activity type, lesson context and teacher interactions) by using the point-by-

point formula presented below. In accordance with the guidelines outlined by Brewer

& Jones (2002) accepted inter-observer agreement levels were set at (85%).

Number of Agreements_________________________________=% Agreement

Number of Agreements + Disagreements x 100

Validity

A total of 30 physical education games lessons were observed and recorded,

resulting in 1379 minutes of video footage to be coded using the SOTG-PE

instrument. The relationships between student activity, lesson context and teacher

interaction variables were examined using Spearman’s rank order correlations (non

parametric data). All analyses were conducted using the Statistical Package for the

Social Sciences v.15 (SPSS Inc, Chicago, IL, USA), and the alpha level was set at p <

0.05.

21

Results

Inter-Observer Agreement

The average inter-observer values for the 12 video-recorded lessons are

presented in table 2. The largest observer error was recorded in third lesson and the

teacher interaction category. This was due to the trained observer coding promoting

tactical behaviour when the teacher was actually acting as the official. It was

concluded that both intra- and inter-observer met the required level for reliability

purposes as suggested by Brewer & Jones (2002).

[Insert Table 3 Here]

Student Activity, Lesson Context and Teacher Interaction Relationships

Spearman’s rank order correlations revealed a significant positive relationship

between student inactivity and general management (r = 0.62, p < 0.01). There were

also significant negative relationships between student inactivity and locomotion (r =

0.-78, p < 0.05), motor response (r = 0.-60, p < 0.05), and full-game (r = 0.-49, p <

0.05). A significant positive relationship was found between motor response and

applied skill practice (r = 0.52, p < 0.02). There was also a significant positive

relationship between technical practice and motor/locomotion (r = 0.41, p < 0.02) and

verbally promoting technical behaviour (r = 0.49p < 0.05). A significant inverse

association was observed between verbally promoting tactical behaviour and technical

practice (r = 0.-48, p < 0.05), however a significant positive relationship was found

between verbally promoting tactical behaviour and modified game (r = 0.46, p <

0.01).

Discussion

The aims of this study were; a) to develop a systematic observation instrument

for the teaching of games in physical education, termed the System for Observing the

22

Teaching of Games in Physical Education (SOTG-PE); and b) to conduct a five-stage

process to determine the reliability and validity of using SOTG-PE in a contextualised

environment. The SOTG-PE was developed primarily by games experts working

independently within the field of physical education and sports coaching and permits

trained observers to simultaneously record an individual target child’s physical

activity type, the lesson activity context and teacher interactions. Consequently, data

for different individual children and teachers can be summed to provide contextual

information on the overall games lesson environment. This instrument is therefore

unique and can be utilised in a number of different environments. For instance,

following accepted levels of inter-observer and intraobserver values the SOTG-PE

could be used by trained observers in future studies of TGM or in the measurement

and analysis of generic physical education games lessons. There is however, a note of

caution. First, the SOTG-PE was not validated in a co-educational environment and

the number of observers was limited. Therefore a follow up study is planned that will

include both male and female participants as well as increasing the number of

observers. Second, the number of observations was limited to 30 lessons due to

unforeseen circumstances within the school. Therefore, it was not possible to code

games from all four of the recognised games categories; the efficacy of the instrument

would be stricter with additional observations including those from the target games

category.

As mentioned previously the development and validation of the SOTG-PE will

respond to the request of Hastie et al.’s (2009) urging that future studies employing

the use of instructional models, such as TGM as well as other instructional

approaches, will be in a position to demonstrate instruction and treatment validity

with greater levels of confidence. The SOTG-PE could be employed by ITE or pre-

23

service teacher education establishments to provide quantitative feedback of lessons

whilst on practicum or in the modular delivery of instructional pedagogy, which

includes elements of technical and tactical games pedagogy. In addition, SOTG-PE

could be configured to a user-friendly computerised system which facilitates the live

capture of multiple behaviours, such as The Behavioural Evaluation Strategies and

Taxonomies (BEST; Sharpe & Koperwas, 1997). The BEST software has been used

previously to record behaviours of pre-service teachers (Sproule et al., 2002) and has

been used more recently to provide an insight into the behaviours and motivational

climate in a physical education environment (Morgan et al., 2005). This combination

would provide a meaningful and unique addition to current methods of observing pre-

service teachers as it would be specific to the delivery of games in physical education.

Moreover, the combination of SOTG-PE to a video-based software system such as the

Dart Trainer (Tagging module) would enable pre-service teachers to view real-time

footage of their pedagogy and to observe how they performed in any of the SOTG-PE

categories.

Finally, the SOTG-PE will provide researchers with a valid instrument to

evaluate and objectively quantify traditional and alternative instructional approaches

to teaching games. Indeed, with the development of SOTG-PE researchers could be

asking the question ‘what is going on here?’ And ‘how do teachers teach games in

physical education’? Followers and advocates of TGM have, over the years, made a

number of enthusiastic claims, and its philosophy has been shared, and adopted by a

number of researchers and practitioners. It is our belief that future large scale studies

using SOTG-PE will finally be in a position to substantiate the role and purpose of

TGM in the delivery of physical education games lessons.

24

Conclusions

This study has documented that the SOTG-PE is a valid observation system

for recording child activity, lesson context and teacher interactions during various

physical education games lessons in the UK. These included games from the

invasion, net/wall and the striking and fielding category. The instrument permits

simultaneous recording of variables that can influence the child’s activity type and the

lesson context, whilst providing additional information regarding the instructional

approach of the teacher. Whilst the feasibility, reliability and validity of the SOTG-

PE in additional contexts, such as girls and co-educational physical education requires

further assessment, this study has indicated that SOTG-PE shows the potential to be a

useful instrument in evaluating and recording the teaching of games in a physical

education environment.

Acknowledgements

The authors would like to thank the pupils and the teachers of physical

education who kindly participated in this study. The authors would also like to

acknowledge the helpful comments of an anonymous reviewer.

25

References

Alexander, K. & Penney, D. (2005) ‘Teaching under the influence: feeding Games for Understanding into the Sport Education development refinement cycle’, Physical Education and Sport Pedagogy 10 (3): 287-301.

Allison, S. & Thorpe, R. (1997) ‘A comparison of the effectiveness of two approaches to teaching games within physical education. A skill approach versus a game for understanding approach’, British Journal of Physical Education, Autumn 1997.

Almond, L. (1986b) ‘Reflecting on themes: A games classification’, in Thorpe, R., Bunker, D., and Almond (Eds), Rethinking games teaching, Loughborough, England.

Arnett, M.G. & Lutz R.B. (2003) ‘Measurement of moderate to vigorous physical activity of middle school girls, using TriTrac activity monitors during small-sided, game-based lessons’, Measurement in Physical Education and Exercise Science, 7(3): 149-159.

Bakeman, R. & Gottman, M. J. (1997) ‘Observing Interaction: An introduction to sequential analysis, second edition’, Cambridge University Press.

Brewer, J., C., & Jones, L. R. (2002) ‘A Five-Stage Process for Establishing Contextually Valid Systematic Observation Instruments: The Case of Rugby Union’, The Sport Psychologist, 16: 138-159.

Brooker, R., Kirk, D., Braiuka, S., & Bransgrove, A. (2000) ‘Implementing a game sense approach to teaching junior high school basketball in a naturalistic setting’, European Physical Education Review (6): 7-26.

Bunker, D. & Thorpe, R. (1982) ‘A model for the Teaching of Games in Secondary Schools’, Bulletin of Physical Education 18(1): 5-8.

Butler, I.J. (2005) ‘TGfU pet-agogy: old dogs, new tricks and puppy school’, Physical Education and Sport Pedagogy 10 (3): 225-240.

Butler, I. J., & McCahan, J.B. (2005) ‘Teaching Games for Understanding as a Curriculum Model’, In L. Griffin & J. Butler (Eds), Teaching Games for Understanding, Theory, Research and Practice:IL Campaign, Human Kinetics.

Den Duyn, N. (1997) ‘Game Sense: Developing thinking players’, Canberra: Australian Sports Commission.

Doyle, W. (1990) ‘Themes in teacher education research’. In W.R. Houston, M. Haberman, & J. Sikula (Eds) Handbook of Research on Teacher Education (pp.3-24): New York, Macmillan.

Evans, J. & Clarke, C. (1988) ‘Changing the face of physical education’, in J. Evans (Eds) Teachers, teaching and control: London, Falmer.

Fairclough, S., Stratton, G. & Baldwin, G. (2002) ‘The contribution of secondary school physical education to lifetime physical activity’, European Physical Education Review 8(1): 69-84.

French, K.E. & Thomas J.R. (1987) ‘The relation of knowledge development to children’s basketball performance’, Journal of Sport Psychology, 9: 15-32.

French, K.E., Werner, P.H., Taylor, K. & Jones, J. (1996) ‘The effects of a 6-week unit of tactical, skill, or combined tactical and skill instruction on badminton performance of ninth grade students’, Journal of Teaching in Physical Education, 15: 439-463.

Gréhaigne, J.F. & Godbout P. (1995) ‘Tactical knowledge in team sports from a constructivist and cognitivist perspective’, Quest, 47: 490-505.

26

Gréhaigne, J.F., Godbout, P. & Bouthier, D. (1997) ‘Performance assessment in team sports’, Journal of Teaching in Physical Education, 16: 500-516.

Gréhaigne, J.F., Richard, J.F. & Griffin, L.L. (2005) ‘Teaching and Learning Team Sports and Games’, RoutledgeFalmer: New York.

Gréhaigne, J.F., Wallian, N., & Godbout, P. (2005) ‘Tactical-decision learning model and student practices’, Physical Education and Sport Pedagogy, 10: 255-269.

Griffin, L., Oslin, J. L. & Mitchell, S. A. (1995) ‘An analysis of two instructional approaches to teaching net games’, Research Quarterly for Exercise and Sport, 66: A-64.

Griffin, L., Mitchell, S. & Oslin, J.L. (1997) ‘Teaching sport concepts and skills. Champaign’, IL: Human Kinetics.

Griffin, L.L., Brooker, R. & Patton, K. (2005) ‘Working towards legitimacy: two decades of Teaching Games for Understanding’, Physical Education and Sport Pedagogy, 10, 3: 213-223.

Gurvitch, R., Blankenship, B.T., Metzler, M.W. & Lund, J.L. (2008) ‘Chapter 3: Student Teachers’ Implementation of Model Based Instruction: Facilitators and Inhibitors, Journal of Teaching in Physical Education, 27: 466-496.

Hastie, P.A., Sinelnikov, O.A., & Guarino, A.J. (2009) ‘The development of skill and tactical competencies during a season of badminton’, European Journal of Sport Science, 9(3): 133-140.

Holt N. L., Strean, W.B. & Bengoechea E.G. (2002) ‘Expanding the teaching games for understanding model: new avenues for future research and practice’, Journal of Teaching in Physical Education 21: 162-176.

Holt, E.J., Ward, P. & Wallhead, T.L. (2006) ‘The transfer of learning from play practices to game play in young adult soccer players’, Physical Education and Sport Pedagogy, 11, 2: 101-118.

Hopper, T. (2002) ‘Teaching Games for Understanding. The importance of student emphasis over content emphasis’, Journal of Physical Education, Recreation and Dance, 73, 7: 44-48.

Hopper, T. (2007) ‘Teaching tennis with assessment ‘for’ and ‘as’ learning: A TGfU net/wall example’, Journal of Physical Health Education, 73, 3: 1-11.

Kazdin, A. E. (1982) Single case research designs. New York: Oxford University Press.

Kirk, D. & MacPhail, A. (2002) ‘Teaching games for understanding and situated learning: rethinking the Bunker-Thorpe model’, Journal of Teaching in Physical Education 21: 177-192.

Launder, A. (2001) ‘Play Practice: the games approach to teaching and coaching sports’, Adelaide, SA: Human Kinetics.

Laws, C. (1994) ‘Rhetorical justification for new approaches to teaching games: are teachers deluding themselves?’ Proceedings of the 10th Commonwealth and Scientific Congress, University of Victoria, Canada.

Lee, M.A. & Ward, P. (2009) ‘Generalization of tactics in tag rugby from practice to games in middle school physical education’, Physical Education and Sport Pedagogy, 14, 2: 189-207.

Light, R. (2004) ‘Coaches’ experiences of Game Sense: opportunities and challenges’, Physical Education and Sport Pedagogy, (9) 2, 115-131.

Lounsbery, M. & Coker, C. (2008) ‘Developing skill-analysis competency in physical education teachers’, Quest, 60: 255-267.

27

McKenzie, T.L., Sallis, J.F., & Nadar, P.R. (1991) ‘SOFIT: System for Observing Fitness Instruction Time’, Journal of Teaching in Physical Education, 11: 195-205.

McKenzie, T.L. (2002) ‘Use of direct observation to assess physical activity’, in Welk, G. (Eds), Physical Activity Assessments for Health Related Research: Champaign, IL: Human Kinetics.

Memmert, D. & Harvey, S. (2008) ‘The Game Performance Assessment Instrument (GPAI): Some concerns and Solutions for Further Development’, Journal of Teaching in Physical Education, 27: 220-240.

Metzler, M. (1990) ‘Instructional supervision for physical education’, Champaign, IL: Human Kinetics.

Metzler, M.W. (2005) ‘Instructional Models for Physical Education’, Holcomb Hathaway, Publishers, Inc.

Miles, M.B. & Huberman, A. M. (1994) ‘Qualitative data analysis: A source book of new methods’, Beverley Hills, CA: Sage.

Mitchell, S.A., & Oslin, J.L. (1999) ‘An investigation of tactical transfer in net games’, European Journal for Cognitive Psychology, 4: 162-172.

Mitchell, S.A., Griffin, L.L. & Oslin, J.L. (1995) ‘The effects of two instructional approaches on game performance’, Pedagogy in Practice: Teaching and Coaching in Physical Education and Sports 1(1): 36-48.

Morgan, K., Sproule, J., Weigand, D. & Carpenter, P. (2005) ‘A computer-based observational assessment of the teaching behaviours that influence motivational climate in Physical Education’, Physical Education and Sport Pedagogy, 10: 183-105.

Morrow, J.R. (2002) ‘Measurement Issues for the Assessment of Physical Activity’, in Welk, G. (Eds), Physical Activity Assessments for Health Related Research: Champaign, IL: Human Kinetics.

Oslin, J. L., Mitchell, S. A. & Griffin, L. L. (1998) ‘The games performance assessment instrument: development and preliminary validation’, Journal of Teaching in Physical Education 17: 231-243.

Oslin, J. L., Mitchell, S. A. (2006) ‘Game-centred approaches to teaching physical education’, in: Kirk, D., Macdonald, D., and O’Sullivan, M. (Eds) The Handbook of Physical Education, Sage Publications, London.

Pope, R.P., Coleman, K.J., Gonzalez, E.C., Barron, F. & Heath, E.M. (2002) ‘Validity of a Revised System for Observing Fitness Instruction Time (SOFIT)’, Paediatric Exercise Science, 14: 135-146.

Ridgers, N.D., Stratton, G. & McKenzie, T.L. (2010) ‘Reliability and validity of the System for Observing Children’s Activity and Relationships during Play (SOCARP)’, Journal of Physical Activity and Health, 7: 17-25.

Rink, J.E. (1996) ‘Tactical and skill approaches to teaching sport and games [monograph], Journal of Teaching in Physical Education, 15(4),3.

Roberts, S.J. (2007) ‘Performance in invasion games: An assessment of ITT/PGCE physical education teachers’, Physical Education Matters, 2, 3: 42-48.

Rossi, T., Fry, J.M., McNeill, M. & Tan, C.W.K. (2007) The Games Concept Approach (GCA) as a mandated practice: views of Singaporean teachers, Sport, Education and Society, 12, 1: 93-111.

Rovegno, I. (1992) ‘Learning to Teach in a Field-Based Methods Course: The Development of Pedagogical Content Knowledge’, Teaching and Teacher Education, 8: 69-82.

Rovegno, I. (1993) ‘Content Knowledge Acquisition during Undergraduate

28

Teacher Education: Overcoming Cultural Templates and Learning through Practice’, American Educational Research Journal, 30: 611-42.

Sage, G. (2003) ‘Forward, in The future of physical education: Building a new pedagogy’, London: Routledge.

School Sport Survey 2007/2008: Report Number DCSF-RW063, TNS, London.Sharpe, T. & Koperwas, J. (1997) ‘Software assist for education and social science

settings’, Behaviour Evaluation Strategies and Taxonomies and Accompanying Qualitative Applications Handbook.

Sharpe, T. & Hawkins, A. (1993) ‘Behavioural Field Systems Evaluation in Movement Education Classrooms: Practice and Implications’, Studies in Educational Evaluation, 19: 327-346.

Siedentop, D. (1994) ‘Sport Education’. Champaign, IL: Human Kinetics.Siedentop, D., Hastie, P.A., & van der Mars, H. (2007) ‘The complete guide to sport

education’. Champaign, IL: Human Kinetics.Sproule, J., Kinchin, G., Yelling, M., McMorris, T. & McNeill, M. (2002) ‘Using

Computer Technology to Compare Student Teaching Behaviours’, European Journal of Physical Education, 7: 123-135.

Turner, A.P. & Martinek, T. J. (1999) ‘An investigation into teaching games for understanding: effects on skill, knowledge and game play’, Research Quarterly for Exercise and Sport, 70 (3): 286-296.

Turner, A.P. (2005) ‘Teaching and learning games at the secondary level’, in: L.L. Griffin and J.L. Butler (Eds) Teaching Games for Understanding: theory, research and practice (Champaign, IL, Human Kinetics).

Ward, P. & Barrett, T. (2002) ‘A review of behaviour analysis research in physical education’, Journal of Teaching in Physical Education, 21: 242-266.

Wilson, G.E. (2002) ‘A framework for teaching tactical game knowledge’, Journal of Physical Education, Recreation and Dance, 73 (1): 20-26, 56.

Wright, S., McNeill, M., Fry. J. & Wang, J. (2005) ‘Teaching teachers to play and teach games’, Physical Education and Sports Pedagogy 10(1): 61-82.

Wright, S., McNeil, Fry, J., Tan, S., Tan, C. & Schempp, P. (2006) ‘Implications of Student Teachers’ Implementation of a Curricular Innovation’, Journal of Teaching in Physical Education, 25(3), 310-328.

29

Table 1 Behaviours and coding classifications for SOTG-PE

Behavioural classification Behaviour definition Child Activity TypeInactive Target child was Inactive. Examples include: sitting,

queuing, standing, lying.Motor response Target child was performing or in the act of executing a

game related motor response. Examples include: Throwing, catching, stopping, bowling, striking, blocking, kicking, heading, passing.

Locomotion Target child was engaged on-task in a locomotor activity that was part of a practice, drill or game. Examples include: walking, running, shuffling, skipping, jumping and diving

Motor/locomotion Target child was engaged on-task performing a locomotive task whilst performing a game related motor response (For example, jumping whilst shooting, running whilst dribbling).

Motor/Locomotion Off-Task Target child was engaged in an off-task, non-learning motor or locomotor activity (i.e., not part of the planned lesson activity, such as practice, game play etc). (For example, running to retrieve a ball or projectile, chasing another child, running around in space, throwing a ball to a partner).

Lesson ContextWarm-up Class time when the pupils are involved in warm-up

related activities (e.g. aerobic activity, stretches, mobility, skill related activities and cool down).

General management The class was not intended to be involved in PE content. Examples include teacher instruction, change of activities, register being taken, and setting up equipment.

Technical Practice The class were involved in an activity solely to enhance technique. This involves the class practicing techniques in a de-contextualised environment (e.g. static passing drills, isolated shooting drills).

Applied Skill Practice The class were involved in a practice where the technique was exposed to pressure but elements of decision making were also required. The numbers in the practice should be uneven (e.g. 2v1, 3v1, 3v2, 4v2, 5v2).

Modified Game The class are engaged in a modified related game. Modification of the game includes: rules (the ball or projectile is not allowed over a certain distance/height), conditions and equipment. (e.g. throw-catch badminton, using batting T’s, alternative scoring zones, rolling the ball instead of using hockey-sticks, throwing the ball instead of using a bat). The game reduces the dominance of skills and techniques. The numbers in the teams must be equal for it to be considered a game and not an overload practice (1v1, 2v2, 3v3, 4v4).

Small Sided Game The class were engaged in small sided games with no conditions. For example, a 3v3 cross-court game of basketball which uses regulation size basketball hoops and there is no restriction on the skills and techniques i.e. dribbling, lay-ups. A 6v6 small-sided soccer game with no conditions other than the numbers and the playing area.

Full Game The class were involved in a full version of the game including numbers and pitch/court size.

30

Free Play The class were involved in an activity with no conditions attached.

Other The class were engaged on task in roles other than one of performing. Examples include official, coach, scorer or helping to organise equipment.

Teacher Interactions

Verbal technical behaviour The teacher employs a direct instructional approach where the focus is on technical/skill learning. Providing concurrent technical feedback, for example, good catch, nice pass.

Non verbal technical behaviour Includes demonstrating a technique or asking a child to demonstrate a technique. Assisting a child physically with a technique.

Verbal tactical behaviour The teacher is engaged in asking the children problem solving questions, for example, where is the space in half-court singles? Where could you move to isolate a 2v1. What did the demonstration show you about the importance of width in attack? Includes promoting tactical instructional approaches through the use of modified games. In addition verbal tactical feedback should be included, examples include support the ball.

Non verbal tactical behaviour Includes the use of demonstrations to promote a tactical concept or using pedagogy such as freeze frames and walk through’s where the teacher assists children to specific game positions.

None The teacher is not engaged in any of the above interactions. The teacher may be off-task, acting as an official, observing a group, setting up equipment.

31

Table 1 Intraobserver agreements of pilot-phase for SOTG-PE

Activity Total Obs Total Disagree Total Agree %

__________________________________________________________________________________________

Child Activity Handball 147 14 133 90.5

Lesson Context Handball 147 12 135 91.8

Teacher Interactions Handball 147 15 132 89.8

Child Activity Volleyball 105 8 97 92.4

Lesson Context Volleyball 105 6 99 94.3

Teacher Interactions Volleyball 105 11 94 89.5

__________________________________________________________________________________________

The overall mean retest coefficients for both the lessons were child activity ≥ 92%; lesson context ≥ 93% and teacher interactions ≥ 89%.

32

Table 2 Intraobserver agreements of pilot-phase for SOTG-PE

Activity Total Obs Total Disagree Total Agree %

__________________________________________________________________________________________

Child Activity Handball 147 14 133 90.5

Lesson Context Handball 147 12 135 91.8

Teacher Interactions Handball 147 15 132 89.8

Child Activity Volleyball 105 8 97 92.4

Lesson Context Volleyball 105 6 99 94.3

Teacher Interactions Volleyball 105 11 94 89.5

__________________________________________________________________________________________

The overall mean retest coefficients for both the lessons were child activity ≥ 92%; lesson context ≥ 93% and teacher interactions ≥ 89%.

33

Table 3. Average Inter-observer % Agreement for 12 Physical Education Lessons. Lesson Number 1 2 3 4 5 6 7 8 9 10 11 12 Overall ____________________________________________________________________________________________________________________ Child Activity Type 0.94 0.87 0.91 0.88 0.90 0.86 0.92 0.84 0.87 0.91 0.91 0.92 0.89 Lesson Context 0.95 0.92 0.89 0.90 0.94 0.89 0.90 0.89 0.94 0.92 0.91 0.93 0.92 Teacher Interactions 0.89 0.85 0.72 0.88 0.90 0.89 0.85 0.92 0.88 0.90 0.91 0.89 0.87 ____________________________________________________________________________________________________________________

34

35

36

37

38

39