Strong literacy and numeracy

in co

Strong literacy and numeracy

in communities pilot

Evaluation Report

2011

Strong literacy and numeracy

Contents

List of Tables and Figures 1

About Strong Literacy and Numeracy in Communities Pilot and Evaluation

2

Executive summary 3

1. Introduction 5

2. Evaluation methodology 10

3. Features of program delivery 11

4. Findings

4.1 Outcomes 15

4.2 Acceptability 23

4.3 Sustainability 27

4.4 Unexpected impact on practice and personnel 31

5. Conclusion and recommendations 33

References 35

1

List of Tables and Figures

Table 1 Percentage of Indigenous students attaining reading and numeracy benchmarks in very remote schools as compared to outcomes for the NT student population (2007

6

Figure 1 Percentage of Indigenous and non-Indigenous children by level of disadvantage (SEIFA): NT, 2009 from NT Report on AEDI Results

6

Table 2 Round 1Scope of pilot project elements conducted at each school site. 13-14

Table 3 Round 2 Scope of pilot project elements conducted at each school site 14

Figure 2 Strong Literacy and Numeracy in Communities Program Logic 15

Table 4 Course completions by qualification and Aboriginal and Torres Strait Islander status

16

Table 5 Round 1 ESSA Pre and Post Training Mean Scores for Aboriginal and Torres Strait Islander trainees only (Paired Data Only included)

17

Table 6 Round 2 ESSA Pre and Post Training Mean Scores by Aboriginal and Torres Strait Islander status (Paired Data Only included)

17

Table 7 Summary of Round 1 school sites and achievement against performance indicators in Element 1 Training and professional learning for paraprofessional staff working with Aboriginal and Torres Strait Islander students

18

Figure 3 Proportion of trainees’ self ratings on skill improvement after course completion 18

Table 8 Summary of Round 1 school sites and achievement against performance indicators in Element 2 and 3 School based literacy and numeracy initiatives (4-8 year olds)

20-21

Table 9 Trends in active enrolment of Indigenous students in pilot schools using paired data only across 2009-2010

22

Table 10 Summary of Round 1 school sites and achievement against performance indicators in Element 4 Community based early learning (0-3 year olds) 23

Figure 4 Proportion of acceptability of training support features as reported by trainees 24

Figure 5 Number of trainees’ and agreement to participate in training based on fully or partially funded and if delivered out of work hours

29

Figure 6 Proportion of trainees suggesting improvements to course 30

Table 11 Enabling components of successful qualification completions 35

2

About Strong Literacy and Numeracy in Communities Pilot and Evaluation Partnership

The aim of the Literacy and Numeracy Pilots Initiative, funded under the ‘Education Revolution – Improving our schools – National Action Plan for Literacy and Numeracy’ 2008 (Department of Employment, Education and Workplace Relations) was to improve outcomes for children in low socio-economic status school communities. In the Northern Territory, the government school sector proposed taking a two pronged approach, focused on i) early childhood learning and care programs for 0-3 year olds and ii) professional learning and formal qualifications for assistant teachers in the early childhood schooling years. The initial design was determined by central, and bureau level data interpretation and higher level consultations.

At the time of developing the project proposal the major and pressing issues identified at the system level and also in national partnership agreement discussions was engaging and supporting families in low SES communities to promote positive child development.

The Literacy and Numeracy Pilots are expected to focused on literacy and/or numeracy around one or more of the following key reform areas:

(a) effective and evidence based teaching of literacy and numeracy; (b) strong school leadership and whole school engagement with literacy and

numeracy; and (c) effective use of student data to drive literacy and numeracy improvement.

The major efforts of the pilot since August 2009, as identified by the project management and staff have been directed to: i) training and professional learning for paraprofessional staff working with Aboriginal and Torres Strait Islander students; ii) school based literacy initiatives (4-8 year olds); iii) school based numeracy initiatives (4-8 year olds), and iv) community based early learning (0-3 year olds). These elements were implemented with respect to the local or school level needs and assets and hence the evaluability of these elements is varied. This pilot project in Northern Territory Government schools makes some contribution to a) and c) in particular.

The evaluation partnership between Menzies School of Health Research (Menzies) and the Northern Territory Department of Education and Training was first proposed in August 2008. MSHR undertook a formative research relationship with the succession of project managers and the variations and iterations of the pilot. The relationship with Menzies ceased in October, 2010 due to Northern Territory Government’s decline to make payment. Menzies was approached in May 2011 to resume the role as the evaluation partner. This significant gap from October, 2010 to May 2011 in the research partnership between Menzies and DET on this project has lead to some discontinuity in the evaluation of processes and monitoring of project outcomes by Menzies. The quality research partnership that should have underpinned the iterative and formative work towards this evaluation report has been impacted.

3

Executive Summary The major focus and investment of the Strong Literacy and Numeracy in Communities pilot project was on capacity building for school based paraprofessionals. The exceptional success, in terms of an 81% completion rate is attributed to the design and delivery features. These include: the use of registered training organisations to provide accredited, formal qualifications in certificate III and IV; fully funded; in work hours; in a school based study group; with in school mentors, and project funded facilitators. In addition to this, 13 trainees were of a standard satisfactory for enrolment in Diploma of Education Support. Four trainees have gone onto enroll in teacher degree courses. This is of particular importance in Northern Territory schools, particularly those in low socio-economic and disadvantaged communities because of the high turn-over of staff and importance of high quality teaching and support staff to meet the needs of students. The qualitative data identified that this specific work based training delivery approach, with its layers of support to participants, was acceptable and manageable at all levels. Participants identified many ways that the training had enhanced their professional practice as education workers and expressed high levels of satisfaction with both their participation and overall academic achievements. School principals and line supervisors identified enhanced levels of professional competency and worker confidence in their staff as the main outcomes from an individual’s participation. Participants were unanimous in their recommendation of the model to others, and emphasised that it should be provided as an ongoing opportunity by education authorities. They stressed that other Education Support Workers should ‘have a go’ as the investment is worth it. The content was perceived to be relevant to all school contexts; urban and remote. When asked if they would encourage more general community members to do the training responses were enthusiastic, however participants recognised that tenuous employment conditions in schools strongly influences interest in formal training. The content was perceived to be relevant to all support workers in school contexts; urban and remote and more broadly to all people interested in supporting children to get the most from an education environment both in the home and formal school setting. A number of responses stressed that the program should be refunded, expanded and be more widely advertised. The second major focus in the Strong Literacy and Numeracy in Communities pilot project was the 0-3 Early learning in community element. Quantitative data for individual student achievement in this element of the pilot project evidence framework is not appropriate or relevant at this stage of implementation. The number of children participating in each school and the transience between schools will be a challenge to demonstrating effect or impact. This element of the pilot project developed a substantial and quality teacher resource for identifying the needs of parents and children, and supporting their successful transition to school.

The school based initiatives for improving early childhood students’ literacy and numeracy were largely delivered as part of the literacy and numeracy plans of schools rather than the

4

Strong Literacy and Numeracy pilot. The pilot did provide resourcing in terms of additional education support staff and professional learning activities. Two legacy professional learning support packages were developed for paraprofessionals. Quantitative data for individual student achievement in the pilot project evidence framework is not appropriate or relevant at this stage of the implementation for the particular focus of the Strong Literacy and Numeracy in Communities pilot activities and the exposure of students to strategies or approaches that are not directly attributable to the project.

With time the overall trends in NAPLAN data and Student Administration Management System (enrolment and attendance) data may provide valid measures of the impact on outcomes of initiatives toward improvement literacy and numeracy outcomes depending on the consistency, continuity and fidelity of program implementation.

5

1 Introduction – Project context and rationale

Of particular relevance to the Strong Literacy and Numeracy in Communities pilot project is the evidence base for the effectiveness of early childhood investment in reducing inter-generational disadvantage and improving social inclusion and participation1,4.

Health research has long provided consistent evidence of the importance of Early Childhood Education and Care as a determinant of health and well-being across the life course1,2,3. This research shows that the gradient of this relationship is strongest where inequalities in early childhood experiences, education, income, and housing quality are greatest. International policy and research advocate the importance of health and education systems being integrated7,16 for optimal outcomes. Investment in early childhood is recognised as an effective strategy for breaking the intergenerational cycle of disadvantage and closing the gap on inequality4.

The importance of early childhood education and care, and particularly high-quality early education and care, to a child’s learning, development and health has been recognised in recent times at a policy level with the Australian Government’s range of education and early childhood national partnerships. The prevailing gap in school achievement and life outcomes between many socio-economically disadvantaged and Indigenous children and families, and mainstream Australia is complex and in many situations not a simple issue of just being able to access a quality service but more an issue of social inclusion and responsive services5.

International social policy research emphasises the importance of building positive and supportive relationships between home and school for children’s learning outcomes6, 7, 8 supported by reviews and reports into Indigenous learning outcomes in Australia9,10. Further, the international evidence shows a strong relationship between intergenerational, socio-economic impacts such as social exclusion11,12,13, crime and teenage pregnancy and the interest of parents (especially fathers) in school learning14. The need to better engage socially disadvantaged families, and in particular, Indigenous and minority groups, in educational partnerships has been a long standing matter for Ministerial Council for Education, Employment, Training and Youth Affairs (MCEETYA)15 and DETYA (2000) / DEST (2002).

This project is designed to support the need for substantial improvement in NT Indigenous students’ literacy and numeracy results. There are stark differences in the literacy and numeracy outcomes for Indigenous students in very remote communities and for the Northern Territory population as a whole. Table 1 highlights the differences in outcomes measured on NAPLAN reading16 indicating that the gap between the two groups increases as students progress through their schooling years.

6

Table 1 Percentage of Indigenous students attaining reading and numeracy benchmarks in very remote schools as compared to outcomes for the NT student population (2007)16.

Percentage Indigenous students attaining reading benchmark

Percentage NT students attaining reading benchmark

Percentage Indigenous students attaining numeracy benchmark

Percentage NT students attaining numeracy benchmark

Yr 3 40.5 72.8 50 82.3

Yr 5 16.5 72.3 13.4 70.2

Yr 7 16.4 68 10 66.7

There are two distinguishing demographic characteristics that stand out in the Northern Territory Australian Early Development Index Results (2009)17 that help to define socio-economic disadvantage in the Northern Territory. Firstly, there is a much higher proportion of NT Indigenous children living in areas of geographic remoteness as defined by the ABS standard classification of remoteness (Figure 1). Secondly, a high proportion of NT Indigenous children (71.6%) live in communities within the highest quintile of socio-economic disadvantage according to the ABS socio-economic indicators for areas (SEIFA). Figure 1 Percentage of Indigenous and non-Indigenous children by level of disadvantage (SEIFA): NT, 2009 from NT Report on AEDI Results 17

The Strong Literacy and Numeracy in Communities project’s focus on increasing activities and family engagement in the early childhood period (0-3years of age) acknowledges the gap in readiness for formal schooling and the diverse early life experiences that contribute to the characteristics or baseline of skills evident for Indigenous children as highlighted by the following findings in the NT Report on AEDI Results (2009)17.

7

• The proportion of NT children who live in very disadvantaged areas (i.e. lowest SEIFA quintile) is almost double the proportion living in similar areas of disadvantage nationally (41.4% vs. 21.7%). Nine of the ten most disadvantaged communities in Australia are located in the NT.

• More than one third (38.4%) of NT children who commenced school in 2009 were developmentally vulnerable (in the bottom 10% nationally) on one or more AEDI domain. These children are considered likely to experience some difficulty in making the transition into formal school learning. This compares with 23.4% of all Australian children who were similarly classified.

• Nearly one quarter (23.4%) of NT children were developmentally vulnerable on two or more AEDI domains. These children are considered to be highly likely to need some form of special support to keep up with their year one classmates. This compares with a figure of 11.8% for all Australian children.

• A comparison of the Australian and NT population of five year olds living in areas of high socio-economic disadvantage shows that a much higher proportion of NT children are developmentally vulnerable on one or more domains (59.3% in the NT vs. 31.8% nationally) and on two or more domains (40.9% in the NT vs. 17.5% nationally).

• Indigenous children are particularly vulnerable in the Language and Cognitive Functioning domain (46.9% in the NT compared to 28.6% of Indigenous children nationally and 7.9% for non-Indigenous children nationally).

The evidence on the impact of teacher qualities on outcomes is well understood18,19. This pilot’s major focus on increasing the number and capacity of Indigenous staff is supported by the evidence20,21,22 available, and sought to further develop evidence over time of the impact on student outcomes beyond the life of the pilot. In Australia, there is some evidence that Indigenous cultural presence in schools encourages a ‘sense of belonging’ amongst Indigenous students. Further, Australian studies 21,22 suggest that schools should have an Indigenous staff presence, in the form of teachers, tutors and Aboriginal and Torres Strait Islander Education Workers (AIEWs). The presence of Indigenous preschool staff has b een found to have a statistically significant effect on the likelihood of Indigenous children attending preschool23.

The relationship between teachers and parents is understood to matter. In particular children’s educational opportunities are enhanced when parents have confidence in the capabilities of the principal and teachers at their child’s school24. In addition effective transition programs provide parents with opportunities to build positive associations with staff and lower problem behaviour, which can be associated with adjusting to school culture25. Much has been written about strategies that will improve Indigenous children’s school readiness although the empirical evidence base is still limited. Culturally-appropriate teaching is frequently promoted in Australian literature 21,26,27,28,29. Other strategies include engaging Indigenous community members in schools, culturally-appropriate professional development for non-Indigenous teachers and the training and support of additional Indigenous staff 22. The initial concept of this pilot sought to examine these propositions using rigorous methodologies, however there are limitations in establishing effect due to the pilot design and inconsistency that comes with an observational study of multiple centres with a wide range of approaches.

8

There is significant evidence indicating that having high quality teaching and learning experiences is a key factor in determining students’ outcomes at school. The pilot aims to address the need for Indigenous students to have ongoing access to high quality literacy and numeracy teaching. The rate of teacher turnover in the Territory’s very remote communities is very high, and can be as much as 50% in one year in some schools. These turnover rates impact on schools’ capacity to provide high quality teaching and learning. A key strategy for addressing this issue is to address the professional learning needs of paraprofessional staff in the pilot schools so they can take a more prominent role in literacy and numeracy instruction and can provide the continuity of teaching and learning that is a feature of many high quality literacy and numeracy programs.

In addition, many of these staff members have been working in schools for twenty years or more. Some are now upgrading their qualifications to become teachers. As a result, DET has recognised it is vital to identify, prepare and mentor new staff into these Assistant Teacher positions to ensure the ongoing renewal of the Indigenous workforce in NT schools. Improving the level and quality of explicit literacy and numeracy instruction and classroom or group management for all teaching staff, particularly assistant teachers in classrooms where children experience more cumulative disadvantage, is essential to addressing the needs of students.

The Strong Literacy and Numeracy in Community Pilot Project

This project aligns with the following Australian and NT Government strategic directions

• Achievement of the Council of Australian Governments (COAG) outcomes to halve the gap for Indigenous students in reading, writing and numeracy within a decade.

• NT Government’s Transforming Indigenous Education strategy, in particular the Sturdy Foundations and Growing our Own initiatives, as well as the NT Government’s early childhood initiative, Families as First Teachers.

• DET’s Literacy and Numeracy Strategy and other initiatives including Accelerating Literacy and the Quality Teaching Package.

The following project description and parameters are taken from the Department of Education and Training project brief provided to the Australian Government as a variation to the original project brief (December, 2009).

Pilot Aims

The pilot project aims to

• support the achievement of COAG agreed targets for literacy and numeracy, specifically, halving the gap for Indigenous students in reading, writing and numeracy within a decade and the NT government’s commitment to Transforming Indigenous Education, in particular, the initiative: Growing Our Own: A Strategy to Build our Local Indigenous Workforce.

• trial evidence based approaches to provide sustained improvement in the literacy and numeracy skills of Indigenous students in low SES schools and communities

• lift the capacity of Indigenous staff to support student learning in literacy and numeracy using socially inclusive approaches

9

• build the capacity of Indigenous parents, community members and service providers to support early literacy and numeracy learning in young children, nurturing English literacy and numeracy competency across age groups

• test approaches and strategies that, if successful, can be applied more broadly to support transformational change in other Indigenous communities.

Pilot Objectives

The pilot project will

• expand early childhood literacy and numeracy programs for Indigenous children in the Northern Territory

• increase the number of Indigenous staff, parents, community members and service providers working with children in early childhood literacy and numeracy programs

• increase the capacity of Indigenous staff to participate in the delivery of early learning and school based literacy and numeracy programs

• enhance the capacity of Indigenous parent and community providers to participate in formal and informal early learning literacy and numeracy activities

• increase the number of Indigenous Assistant Teachers working in school based literacy and numeracy programs

• increase the capacity of Indigenous staff to participate in the delivery of school based literacy and numeracy programs

• test the validity of strategies used in the pilot to support sustained improvement in literacy and numeracy

• provide examples of evidence based literacy and numeracy projects that can be used as models for implementation in other low SES Indigenous communities.

Project Scope

The pilot project will create opportunities for early literacy and numeracy development for Indigenous students from infancy to entry to pre-school and through their years of primary education in selected sites.

This will be achieved through the implementation of two distinct but related components. The first component focuses on the establishment and delivery of early learning opportunities for young Indigenous children from infancy until their entry into preschool (0–3 years). The other component involves providing professional learning in literacy and numeracy to existing and new Indigenous Assistant Teachers, other paraprofessionals and teachers to better support Indigenous students in literacy and numeracy classrooms.

Piloting this place-based approach will allow for further roll-out across the Northern Territory in low SES communities.

Documentation and packages will be developed to support the trialled methodologies so they can be adapted for local content and use in other settings.

The mixed-mode delivery, which includes utilising distance learning infrastructure, face-to-face delivery and local program content, allows flexibility to cater for site-specific contexts.

10

Key Performance Indicators (KPIs)

• The number of Indigenous staff working in the early learning networks (a play based activity supported by Indigenous staff and program support staff) that complete the professional learning programs and/or Certificate 3 or 4 literacy and numeracy competencies as a result of their participation in the pilot.

• The effectiveness of Indigenous staff working in the early learning networks. • The number and level of engagement of parents and community providers

participating in the early learning networks. • The number of new Indigenous ATs working in school based literacy and numeracy

programs as a result of the pilot. • The number of Indigenous ATs who complete literacy and numeracy professional

learning programs and/or Certificate 3 or 4 literacy and numeracy competencies as a result of their participation in the pilot.

• The effectiveness of Indigenous ATs working in school-based literacy and numeracy programs.

• The number of students who demonstrate age-appropriate developmental milestones on entry to pre-school and are ready for pre-school learning.

• Improved student attendance and parent involvement in pre school programs. • Over time, improved student literacy and numeracy skills in classrooms where

Indigenous ATs have undertaken further professional learning or competency-based training.

Project Outputs

• A number of early learning networks. • A variety of early learning programs and resources suitable for use in early learning

networks. • A range of early childhood professional learning programs for Indigenous staff in

early learning networks. • A range of literacy and numeracy professional learning programs for Indigenous

Assistant Teacherss. • A suite of adult teaching and learning methodologies in literacy and numeracy and

early childhood learning (0 -3 years). • A Project Evaluation Report.

In all components of the pilot project, the focus will be on the development and delivery of place-based programs designed in conjunction with key clients and stakeholders in the community. A priority in all project work will be comprehensive and ongoing community engagement and consultation to design site-specific programs.

11

2 Evaluation Methodology

Research question How effective is the development of non-teaching staff through certificate training and provision of literacy and numeracy resources in improving learning outcomes for students? (This research question relates to element 1 Training and professional learning for paraprofessional staff working with Aboriginal and Torres Strait Islander students)

Research design Quantitative data was sourced, where feasible against the Evaluation Framework (Attachment A) and analysed at school and aggregated levels. The early learning component of this project did not yield sufficient data to run statistical analyses. Some baseline administrative data may support future tracking of participation rates but cohort comparison of growth in literacy and numeracy competencies requires students to be identified for longitudinal study. In addition the numbers are very small, with high attrition rates due to movement between schools.

Qualitative data was collected through semi structured interviews and a thematic analysis.

Sampling Round one (2010) included all of the elements of the project brief (December 2009) between the five schools participating. Round Two (2010 – 2011) comprised certificate training including intercultural workshops and literacy resource trialling for 10 participating schools. Qualitative interviews (n=34) were conducted by Menzies staff in 2010 and were supplemented with purposive sampling of trainees, teachers and principals to cover outcomes of the project elements listed below (Table 2 and 3).

Data collection and analysis All data collected by Department of Education and Training was provided to Menzies including surveys of trainees, pre and post training ESSA (Education Support Self Assessment Tool), all National Assessment Program for Literacy And Numeracy, Assessment of Student Competencies, Northern Territory Curriculum Framework, and Enrolment and Attendance data.

Interviews with trainees, teachers and principals on the training and professional development in school specific strategies were collected by Menzies.

Interviews with key staff including principals and teachers on 0-3 and additional professional learning activities and materials were collected by Menzies.

Reporting of results This evaluation report provided by Menzies provides quantitative and qualitative summary of findings against the research question. Caution will be taken around reporting at the school level to ensure staff and students cannot be identified. This report does not seek to provide analysis of data against the four elements of this pilot project as design and data reliability and validity does not support an impact evaluation at this stage.

12

3 Features of program delivery

Project design – program logic and expected outcomes

An overview of the pilot project elements undertaken in each school site for Round 1 in 2009 (Table 2) reflects the localised priorities. Each school’s specific literacy and numeracy program initiatives and assessment or monitoring was identified with school staff and mapped onto a program logic for each school. This enabled the project management and research teams to better communicate with schools about common elements for all pilot participants (Figure 2) and distinguish what specific site based initiatives were being implemented in parallel or discreet to the pilot activities. During Round 2 in 2010 the project focused on completion of two professional learning resources and the training of 63 staff from across 10 schools (Table 3). Delivery

As outlined above and earlier in section 1, the expected project scope provided by the Department of Education and Training included four elements delivered to participating schools which varied between sites during round 1 in 2009. Only the training element and completion of literacy and numeracy professional learning resources were delivered in round 2 during 2010. The actual deliverables for each pilot element are described below with the evaluable components noted. Element 1 Training and professional learning for paraprofessional staff working with Aboriginal and Torres Strait Islander students

The focus in this component was the implementation of a place-based professional learning program to improve the Indigenous Team Teachers and Assistant Teachers’ capacity to support teachers and students in literacy and numeracy teaching and learning. The project officer and program manager worked with Indigenous Team Teachers and Assistant Teachers, Registered Training Organisation and other relevant stakeholders to design a professional learning package to suit individual needs and relevant to the attainment of formal qualifications.

In addition, the pilot funded the employment of additional Indigenous Team Teachers and Assistant Teachers in the school to work with current staff in 0-3year old programs and early childhood classrooms.

Element 2 School based literacy initiatives (4-8 year olds) Element 3 School based numeracy initiatives (4-8 year olds)

The additional staff employed in Round 1 schools for 2009 were largely deployed in the early childhood classrooms to improve group sizes for literacy and numeracy instruction. In addition, staff were involved in training for use of a numeracy program and a sensory motor

13

based literacy and oral language program. All schools in Round 1 and 2 had a wide variety of programs running in parallel as part of their school based literacy and numeracy plansa.

The pilot project funded the development, trialling, workshops and production of two high quality professional learning resources, Assisting Paraprofessional Learning in Literacy Support and Talking Namba.

Element 4 Community based early learning (0-3 year olds)

This component focused on providing two additional Indigenous staff to engage parents, carers, other service providers and key members of the community in developing and providing early childhood learning opportunities. The primary role of the two community workers in each pilot site, was to coordinate an early childhood network while receiving ongoing professional development, advice and support.

The project manager assisted the Indigenous staff, and where available local coordinator, to design and implement a play-based learning program of experiences that supported children’s readiness for pre-school participation. This element of the pilot project included supporting parents and carers in their roles and connection with pre-schools, mobile pre-schools and by other government and non-government agencies in the community.

Table 2 Round 1 Scope of pilot project elements conducted at each school site.

Round 1 Sites Project Elements Delivery / Activity Yirrkala 1. Training and professional

learning 2. Literacy initiatives 3. Numeracy initiatives 4. 0-3 Community based early

learning

Extra Indigenous staff employed 4 – 8 Extra Indigenous staff employed, 0 – 3 0-3 initiative SLNC rolling over to *FaFT in 2011

Gapuwiyak 1. Training and professional learning

2. Literacy initiatives 3. Numeracy initiatives 4. 0-3 Community based early

learning

PD, school-based initiatives around staff capacity building Extra Indigenous staff employed 4 - 8 numeracy Extra Indigenous staff employed, 0-3, 0-3 initiative SLNC rolling over to FaFT in 2011

Manunda 1. Training and professional learning

2. Literacy initiatives 3. Numeracy initiatives 4. 0-3 Community based early

learning

Certificate Training, Home 2 School professional learning Extra Indigenous staff employed 4 – 8, Gateways Numeracy professional learning, parent workshop

a Northern Territory Department of Education and Training require school based literacy and numeracy plans

as a component of the Accountability and Performance Improvement Framework, for which all principals are

accountable.

14

Round 1 Sites Project Elements Deliverables / Activities Gillen 1. Training and professional

learning 2. Literacy initiatives 3. Numeracy initiatives 4. 0-3 Community based early

learning

Certificate Training Extra Indigenous staff employed 4 - 8, Gateways Numeracy professional learning, parent workshop Home 2 School+ Case Study (Little Strong Yerrampes),

Moulden Park 1. Training and professional learning

2. Literacy initiatives 3. Numeracy initiatives 4. 0-3 Community based early

learning

Certificate Training, Home 2 School professional learning Extra Indigenous staff employed 4 - 8, Gateways Numeracy professional learning Offsite Playgroup 0-3

Table 3 Round 2 Scope of pilot project elements conducted at each school site.

Round 2 Sites Deliverables / Activities

MacFarlane

Malak

Wagaman

Millner

Wulagi,

Gray

Certificate training for 39 paraprofessionals

Bradshaw

Ross Park

Larapinta

Sadadeen

Certificate and diploma training for 24 paraprofessionals

NB Shepherdson College participated in the development of the numeracy professional development for preparing the “Talking Namba” resource.

15

Figure 2. Strong Literacy and Numeracy in Communities Program Logic

Intermediate Outcomes

Concrete Clear and concise Measurable Relevant Achievable

Project Objectives

Capacity Funding Facilitation Reporting

Limiting Factors

Time

Availability of personnel

Enabling strategies

Deliverables – Outputs

Extra staff Laptops for urban Training Professional learning in literacy and numeracy education support – knowledge about child development , how children learn Numeracy skills through resource production 0-3 staff cert training

For each participating school identify action plan against the following common elements:

1. Indigenous Capacity building

2. Lit Plan 3. Numeracy Plan 4. 0-3 program

Measures:

ELSSA pre and post (Staff self evaluation

survey)

Teacher Survey

Staff profile / database tracking PD A&E

(incl Indigenous staff participation in formal

prog)

Performance Review process

AEDI

Number of parents involved in activities school

FAFT survey / PPSR

Y3 NAPLAN

Moderated folios on C-eTOOL

ILSS annotated transcripts

SAMS A&E (incl 0-3 fields)

ASC

Parent Survey/ interview

Pre and post testing as related to site specific

initiatives eg ELR, SPAT-R and SPAN-R

Frequency of l/ship engagement with staff,

parents and Community

Events

Aspirational Outcomes

Lit & Num performance of students

Numeracy proficiency

English language proficiency of ESL students

Student attendance, retention, participation and engagement

Indigenous Staff Capacity

School leadership qualities

Parent and community engagement and support of the school

Improved community capacity to support school readiness

1.Indigenous

teaching capacity

3. Family

Engagement

2. Student

Outcomes

16

4 Findings

4.1 Outcomes

Element 1 Training and professional learning for paraprofessional staff working with Aboriginal and Torres Strait Islander students

The intended outcomes for paraprofessional training focussed on certificate training in Education Support certificate III and IV (CHC303808 and CHC41708). Project management collected baseline data through from trainees through a pre-enrolment survey including demographics, past training and future aspirations, and Education Support Self-Assessment (ESSA) tool or English Language Self-Assessment (ELSA). Other quantitative data provided by the Department of Education and Training included the administrative data for: • Trainee completion numbers and proportion of enrolments;

• Proportion of Aboriginal and Torres Strait Islander trainees; • reasons for withdrawal where known.

The pilot did not achieve delivery of training in the remote school (two schools in Round 1). This was at the discretion of the school leaders and variation to include training in urban centres was approved by Department of Education, Employment and Workplace Relations. Total training enrolments for the pilot project was 111 with 90 completions of either certificate III, certificate IV or Diploma of Education Support (Table 4). The overall completion rate of 81% far exceeds the national average completion rate for Vocational Education and Training courses in the education field of 45%30. For Aboriginal and Torres Strait Islander trainees the completion rate (n=37) was 77% which is also well above the national average. There were 20 withdrawals and one incomplete. Only six withdrawals were for “unknown” reasons. Nine withdrawals were due to change of employment circumstances and four withdrawals were due to moving interstate or very remote. One withdrawal was to enrol in teacher training at University.

Table 4 Course completions by qualification and Aboriginal and Torres Strait Islander status

Certificate III Certificate IV Diploma

Aboriginal and Torres Strait Islander

9 22 6

Other 12 34 7

The Education Support Self Assessment tool was adapted for the urban context by the Strong Literacy and Numeracy in Communities project management team from the English Literacy Self Assessment tool trialled at Yirrkala in 2009. It asks the self-assessor/ trainee to rate their

performance in particular school based tasks. The ESSA includes an Administrator’s Guide and a Participant’s Booklet. The ELSA and ESSA provide certificate trainee pre and post data and were developed in alignment with the Australian Core Skills Framework (ACSF). The interpretation of the Round 1 trainees self assessment collected pre and post training is limited due to internal design which lead to overestimation of skills and confidence. This resulted in very little growth being evident in these assessments over the duration of the

17

course with the exception of writing skills which increased significantly, p <.05 (Table 5). Significant work was undertaken by the project management team for Round 2 pre and post assessments to improve the tools, and in addition, administrators were used to collect the responses. Table 5 Round 1 ESSA Pre and Post Training Mean Scores for Aboriginal and Torres Strait Islander trainees only (Paired Data Only included)

ATSI (n = 23)

Mean Pre Score

Mean Post Score

ES 95% CI P(T<=t) one-tail

Numeracy Reading Learning Oral Writing

4.86

4.48

4.57

4.76

4.52

5.00

4.76

4.67

4.90

4.90

0.42

0.47

0.14

0.38

0.80

0.32

0.29

0.33

0.28

0.33

0.093

0.055

0.340

0.093

0.008

Not Significant

Not Significant

Not Significant

Not Significant

Significant

Round 2 Self assessments demonstrated statistically significant increases in trainees skills and confidence in all measures (Table 6). Further, a comparison of the Aboriginal and Torres Strait Islander and other trainees suggests parallel growth in skills with the exception of numeracy which suggests Aboriginal and Torres Strait Islander trainees reduced the gap in achievement with other trainees, although not significantly.

Table 6 Round 2 ESSA Pre and Post Training Mean Scores by Aboriginal and Torres Strait Islander status (Paired Data Only included)

ATSI n = 23

Mean Pre Score

Mean Post Score

ES 95% CI P(T<=t) one-tail

Numeracy Reading Learning Oral Writing

3.83

3.57

3.30

4.00

3.52

4.78

4.61

4.57

4.74

4.65

0.87

1.24

1.20

1.21

1.31

0.25

0.27

0.30

0.28

0.28

0.000303

2.07E-06

5.48E-05

4.65E-06

5.45E-07

Significant

Significant

Significant

Significant

Significant

Other n = 27

Pre Post ES 95% CI P(T<=t) one-tail

Numeracy Reading Learning Oral

Writing

4.30

3.85

3.41

3.78

3.63

4.96

4.93

4.70

4.85

4.85

1.26

1.61

1.25

1.29

1.57

0.24

0.34

0.28

0.27

0.33

3.61E-06

9.82E-07

7.17E-06

1.93E-06

1.09E-06

Significant

Significant

Significant

Significant

Significant

18

School based professional learning that initiatives that the two remote schools in Round 1 identified as operating in parallel or complementary to the pilot project activities are included in the table below summarizing achievement of objectives (Table 7).

Table 7 Summary of Round 1 school sites and achievement against performance indicators in Element 1 Training and professional learning for paraprofessional staff working with Aboriginal and Torres Strait Islander students

Deliverables / Activities Performance Indicators

Achieved/Not achieved – which schools

• Enrol new and existing staff in certificate courses - all schools.

• Provide Professional Learning

for Support staff in child development, language acquisition & meaning making, early literacy and numeracy development- all schools.

Assistant Teachers/support staff successfully complete competencies and participate in professional learning

Achieved Moulden, Manunda, Gillen

In progress Gapuwiyak, Yirrkala

The additional staff (provided by the pilot) support small group work independently in early years classes

Achieved Moulden, Manunda, Gillen, Gapuwiyak, Yirrkala

Perceptions in the ELSA/ESSA tools are improved

Achieved Moulden, Manunda, Gillen

In progress Gapuwiyak, Yirrkala

• Improve Assistant Teachers’ English competence through participation in Rosetta Stone program – Gapuwiyak.

Quiz at end of unit, progress to higher levels. (Rosetta Stone)

Achieved Gapuwiyak

• Implement FAST Train the Trainer - Yirrkala

Families are involved Achieved Yirrkala, Gillen

Qualitative Theme analysis

Skills (pre & post training, specifics, literacy and numeracy, behaviour management, working with Low SES, working with Indigenous , broader employability)

Participants were asked to identify changes in their skills levels post training. The majority of participants perceived positive changes in skills levels post participation. Figure 3 Proportion of trainees’ self ratings on skill improvement after course completion

The most significant changes were described in the areas of:

Greatly Improved

Somewhat Improved

Unchanged

Not Sure

19

• Interpersonal skills including; patience, confidence and competence in everyday interactions with children

• Enhanced ability to observe and interpret children behaviours and respond to their needs

• Enhanced ability to be a reflective practitioner within their role, and in their working relationships with teachers

• Improved ability to instruct in small groups • Enhanced understanding of the their role and work environment, including duty of

care • Enhanced understanding of the importance of children backgrounds

Also many examples to a wider repertoire of strategies, games and activities ( the tools of a teacher) to apply to their classroom with students were described for being more responsive to learner needs:

I’m more aware of different kids needs.. when I was first on the job I wasn’t aware of that, so many different cultures and backgrounds

More organised, more confident more patient, provide more collaboration with others, ideas to suggest to teacher, learn to play with the kids, understand their needs

Participants were asked to reflect on their learning about working with children from culturally diverse or low SES backgrounds. Some participant have had previous training in cultural diversity, working with Aboriginal students or long experience in this area and felt it was difficult to identify changes. Others noted the course had increased their awareness and acknowledgement of culture as an influence on children’s development, and their ability to get the most from the classroom environment. Some participants noted the English as a Second language components of professional learning gave them a better understanding of how to better support students of diverse backgrounds.

Element 2 School based literacy initiatives (4-8 year olds) and Element 3 School based numeracy initiatives (4-8 year olds)

Quantitative data for individual student achievement in the pilot project evidence framework is not appropriate or relevant at this stage of the implementation for the particular focus of the Strong Literacy and Numeracy in Communities pilot activities and the exposure of students to strategies or approaches that are not directly attributable to the project.

With time the overall trends in NAPLAN data and Student Administration Management System (enrolment and attendance) data may provide valid measures of the impact on outcomes of initiatives toward improvement literacy and numeracy outcomes depending on the consistency, continuity and fidelity of program implementation.

School based literacy and numeracy initiatives Round 1 schools identified as operating in parallel or complementary to the pilot project activities are included in the table below

20

summarizing achievement of objectives (Table 8). Although not a directly attributable activity of the pilot project, school based initiatives such as intensive student assessment and data analysis for reading at Manunda Terrace enabled the school to identify student learning needs for expressive and receptive oral language skills (in Standard Australian English) as precursors to reading.

Table 8 Summary of Round 1 school sites and achievement against performance indicators in Element 2 and 3 School based literacy and numeracy initiatives (4-8 year olds)

Deliverables / Activities Performance Indicators

Achieved/Not achieved

Implement Developmental Curriculum for seamless learning in Early Childhood • Provide professional learning to Assistant

Teachers in the ADC Program including learning through play – Gapuwiyak.

Quality Early Learning Environments for children

Achieved

Gapuwiyak

Build networks with other ADC schools, hub school (Early Years Learning Framework) and 0 – 3 service providers – Gapuwiyak • Implement literacy groups to support 1st

and 2nd wave literacy learners, division of 5 classes between 7 teachers to provide intensive, explicit literacy teaching with the support of an additional adult where appropriate.

Achieved

Gapuwiyak

Implement ‘Gateways to Literacy’ Perceptual Motor Program • 2009 all participating students tested -PM

Benchmarking, Southerland Phonological awareness, concepts of print & Educheck and Group students based on these results – Manunda.

Collection of PM Benchmarking, Southerland, Print concepts, & Educheck.

Achieved Manunda

Provide support to enable sustained improvement in literacy and numeracy skills of Indigenous students. • Implement small group intensive, explicit

literacy teaching for six months Collect data at 6 month and 12 months – Manunda.

Improvement is evident through Collection of assessment data

Achieved Manunda

Provide professional learning to support staff in testing procedures, best literacy practice – Manunda.

Achieved Manunda

Provide professional learning to support staff and teachers in Gateways to Literacy program - Manunda, Gillen, Moulden

Support staff confidently and effectively support the Gateways program Perceptual motor skills are assessed and improvement demonstrated.

Achieved Manunda, Gillen, Moulden

21

Deliverables / Activities Performance Indicators Achieved/Not achieved Provide professional learning to enable staff to successfully fulfill the role of literacy and numeracy classroom assistant - all schools.

Qualitative feedback from staff on professional development

Achieved Moulden, Manunda, Gillen, Gapuwiyak, Yirrkala

Provide literacy and numeracy support to students and small groups - all schools.

Evidence of progress through moderation of work samples

Evidence of progress through testing of individual literacy and numeracy levels

Achieved (Evidence in CeTool) Moulden, Manunda, Gillen, Gapuwiyak, Yirrkala Not achieved/shared Gapuwiyak, Moulden, Gillen

Construct literacy & numeracy resources – Gillen.

Resources are used in classroom activities

Achieved Gillen

Conduct base-line testing and in conjunction with classroom teacher implement appropriate support activities – Gillen.

Achieved Gillen

Maintain identified student portfolios – Gillen.

Portfolios for identified students are maintained

Achieved Gillen

Implement FAST Train the Trainer – Gillen. Achieved Gillen

Implement multilingual education based on the study Literacy for Multilingual Learners from UNESCO

Provide professional learning to support staff in multilingual education – Yirrkala.

Early Years students are working towards achieving Level 2 Listening and Speaking (NTCF ESL Levels) Records of student achievement and targets are set for further learning.

Achieved Yirrkala Achieved Yirrkala, (ESL A-E reporting requirement)

Develop a multi media resource for paraprofessional staff to support the delivery of numeracy programs in the early formative years.

• Provide professional learning to Assistant teachers/ Support staff on a foundational mathematical concept - all schools.

The school has access to the completed multimedia resource

Achieved

Gapuwiyak, Yirrkala, Moulden, Manunda, Gillen

22

Deliverables / Activities Performance Indicators Achieved/Not achieved

Participate in the development and trial of the multimedia resource - all schools.

Assistant teachers/ Support staff independently run numeracy activities for small groups

Achieved

Gapuwiyak, Moulden, Manunda, Gillen

An early indication of changes in active enrolment for Indigenous students in the pilot project schools is provided below (Table 9) but in the absence of analysis against baselines requires caution in attributing these trends to the pilot project.

Table 9 Trends in active enrolment of Indigenous students in pilot schools using paired data only across 2009-2010

Schools where percentage of actively enrolled Indigenous students attending school

increased

Schools where percentage of actively enrolled Indigenous students attending school declined

Gapuwiyak, Gillen, Gray, Larapinta, McFarlane, Manunda, Sadadeen, Wagaman, Wulagi,

Bradshaw, Malak, Millner, Ross Park, Yirrkala

Note: No paired data available for Moulden

Qualitative Theme analysis

Skills for literacy and numeracy instruction

In addition to the qualitative data about general skill acquisition provided above, specific literacy and numeracy instruction in the remote and English as an Additional Language context the following observation was made by a teacher undergoing a teacher upgrade course (funded by this project):

“(I’m more) confident in teaching literacy and numeracy, knowing what to teach, where to start and what to do. When I'm teaching I read Yolngu to the children, then explain it to them, and the use English for the English children. I know all the children, where they are up to, because of my years of experience. Through study I have learned where to start, observing, small details, focussing on children's behaviour and learning. I watch and listen to the children, language, ESL, scaffolding the children’s language rather than saying "don’t say it like that!". I try to make learning in a fun way, using a doll to teach maths, science learning about caterpillars, responding to the children's interests.”

Element 4 Community based early learning (0-3 year olds)

Quantitative data on participation in and impact on individual outcomes was not available for statistical analysis. In the remote communities activities to develop 0-3 early learning programs was superseded by the establishment of a large Australian Government funded and Northern Territory Government administered program, Families as First Teachers – Indigenous Parent Support Services. The early contribution of staff to initiate the activity is

23

acknowledged and the programs are considered successful with 2011 enrolments reaching 81 in Gapuwiyak and 91 in Yirrkala (as reported by FAFT-IPSS managers). Manunda Terrace and Gillen schools provided early learning activities on site, whilst Moulden Park utilised the Strong Literacy and Numeracy in Communities funded paraprofessional staff to run an outreach program in the local parks. Consistent with the evidence around implementing such programs, this establishment phase yielded very limited and varied results. The case study prepared by Gillen School staff, Strong Little Yerrampes, demonstrates very strong commitment and planning for sustained growth of this program. At the school level, a good evidence about how and to what effect this program impacts children’s outcomes over time.

School based 0 – 3 year old community based early learning initiatives Round 1 schools identified as operating in parallel or complementary to the pilot project activities are included in the table below summarizing achievement of objectives (Table 10).

Table 10 Summary of Round 1 school sites and achievement against performance indicators in Element 4 Community based early learning (0-3 year olds)

Deliverables / Activities Performance Indicators Achieved/Not achieved

Support existing early learning community networks for children (0-3 years) and their families.

Support and add value to current programs for families of children age 0-3 years, including parent education - Manunda, Moulden, Gillen.

Qualitative feedback from staff and participants

Children and families ready for Preschool

Achieved

Manunda, Moulden, Gillen

Achieved

Manunda, Moulden, Gillen (anecdotal reports)

Establish an early learning community network for children (0-3 age group) and their families. Test parent education tools.

Implement program for families of children age 0-3 years, including parent education, parent and children play based activities - Gapuwiyak, Yirrkala.

Number of parents and children attend sessions regularly

Achieved

Gapuwiyak, Yirrkala

Provide professional learning for community based staff and education for families based on Early Years Learning Framework and child development - all schools.

Achieved

Gapuwiyak, Yirrkala, Manunda, Moulden, Gillen (this activity was not part of pilot)

Use Effective Lifelong Learning Inventory (ELLI) to develop parent understandings about learning dispositions - Yirrkala

Learner profiles reflect changes in learning dispositions

Not Achieved

Yirrkala (program discontinued)

24

4.2 Project Acceptability

Element 1 Training and professional learning for paraprofessional staff working with Aboriginal and Torres Strait Islander students

Participant Satisfaction Overall there was a high level of satisfaction with participation in the training (40/42) Figure 4. Figure 4 Proportion of acceptability of training support features as reported by trainees Participant Overall Training Satisfaction

Relationship with Mentor

Satisfaction with Mentor Support

Qualitative theme analysis Satisfaction with training model and content; instructor; group; in school; funded; mentor; project support officers

The provision of free accessible training provided during work hours was appreciated and enhanced overall work satisfaction for the majority of participants. The layers of support provided by the Strong Literacy and Numeracy in Communities training model were viewed as acceptable and a strength for completing the courses study requirements and applying newly learned skills and knowledge.

Participants identified a range of reasons why the training was satisfying. The two key themes were:

1) the outcome of gaining a qualification was very useful and the achievement satisfying

Very Satisfied

Satisfied

Not Satisfied

Very Good

Good

Not Good

Very Satisfied

Satisfied

25

2) the training was well focused on skills and knowledge to better implement their everyday work role

I’m more knowledgeable, more confident An amazing opportunity, very grateful, plenty of job satisfaction Content well presented, good assignments

The content of units, in that it focused on the underlying knowledge required for on the job requirements, was seen as very relevant and directly beneficial to meeting children’s needs. Participants described how post training they could better understand and respond to children needs, understand the teachers role and the classroom program. The course content assisted them in understanding stages of development and provided resources to further support individual and group learning. Having an increased range of education support strategies was useful and satisfying. The new skills could be readily used after training. This was particularly evident in feedback linked to the provision of the content for numeracy units.

The practical ideas were very good I use the games in class Lots of resources to pass on and organise things Was helpful to see how to program and plan Maths ideas and how to observe and correct things that aren’t done right

Additionally for a number of the participants the challenge presented by the study requirements was personally satisfying

more satisfied than I thought I would as a group get to discuss things without being interrupted…I thought I'd hate it because of going back to study

Some participants noted that the course appeared to be focused more towards the roles of ISA’s, rather than literacy and numeracy support workers, or general Teacher Assistant’s the preschool work environment.

Previous training experiences

Some participants had attended previous training, primarily short courses or workshops. Examples included:

literacy and numeracy reading, writing, numeracy hearing support asthma awareness kids matter ( mental health) and school systems and administration (QUICKSMART, SAMS)

A small number of participants had also previously enrolled in formal education support worker courses (n=3).

26

Satisfaction with the model and layers of supports

Participants expressed overall support for the model, and that they were satisfied with the layers of support provided within the provision of the content and materials by the RTOs/instructor, the school for generally providing support for training including the provision of a school mentor and also with the support Strong Literacy and Numeracy in Communities personnel. RTO /instructors were described as competent in delivering the content and in their general approach to participant support (n=13).

Information was good, informative, well structured, helpful, knowledgeable, supportive.

Trainers were easy to talk to, very supportive, willing to help out, could call or email. The course was satisfactory, we were able to understand everything.

Relationships with school mentors were described to be good or very good (n=19).

For some mentor support was highly valued and strongly appreciated:

Not just a mentor I go to L for everything, she oversees our work and gives feedback for course.

( My mentor)…..tells me everything I need to know.

Mentors were described as; generally accessible as they were on site in the work environment, and commended as being able to provide a timely and responsive resource. Few challenges to the mentor role, apart from the pressure of time, were described.

Having the SLNiC personnel as a link to the overall project was described as important and useful to participants.

Peer group supports were assessed to have worked well where they were established, and many positive benefits were described :

Excellent because of the sharing of ideas/ opinions and information.

Added to the overall experience as we support each other and then it’s easy.

Supporting the participants to undertake their study was described positively by principals and senior staff. The program in some instances was viewed as beneficial for the whole staff

wonderful to have other people, look at what they were doing

brought the school together great support

you could see the activity in the staff room and asking about professional issues

A small number of challenges around time management skills for balancing work and study requirements and the pressure on participants negotiating the volume of assessment tasks were described. Recommendations to others about the course

27

Participants were unanimous in their recommendation of the model to others. They emphasised that the course should be provided as an ongoing opportunity through schools, and stressed that other Education Support Workers should ‘have a go’ as the investment is worth it. When asked if they would encourage more community members to do the training responses were enthusiastic, however participants recognised that tenuous employment influenced workers interest and ability to undertake formal training. It was perceived that the course outcomes are very relevant and have broad applications. The content was perceived to be relevant to all support workers in school contexts; urban and remote and more broadly to all people interested in supporting children to get the most from an education environment both in the home and formal school setting. When asked if they would encourage more general community members to do the training participant responses were enthusiastic, however they recognised that tenuous employment conditions in schools strongly influences interest in formal training. A number of responses stressed that the program should be refunded, expanded and be more widely advertised.

28

4.3 Project sustainability

Element 1 Training and professional learning for paraprofessional staff working with Aboriginal and Torres Strait Islander students

Of the 111 enrolments in the training courses, 82 remained employed in schools (not necessarily the school in which training commenced) at the close of data collection. The expectations of the training were exceeded with four trainees progressing to undertake teacher training, three of whom are Aboriginal and Torres Strait Islander. Although a cost – benefit analysis is beyond the scope of this evaluation, the completion rate as a product of the specific training model design warrants close consideration by comparison with standard investment per capita for half the completion rate. The success of this professional learning and training for paraprofessionals measured by the completion outcomes and acceptability for participants and school leaders is dependent on the particular design features. The sustainability and application to other schools with low SES and high Aboriginal and Torres Strait Islander student cohorts is dependent on the support structures, funding and timing.

Qualitative theme analysis Capacity building, cost and retention of staff

Job Satisfaction for participants was generally high and their education support work has a level of challenge which means it is neither easy or hard.

All participants strongly agreed that they enjoyed working with children and supporting their growth and development. The core of the role, in that it requires interpersonal skills and a capacity to have warm relationships with children was satisfying and for many very enjoyable.

I’m a people person, I love the children and the teamwork here………..working with kids, makes me happy.

Similar to research in the area with qualified teacher staff, participants identified guiding and managing student behaviour as the biggest job challenge for them as teacher assistants. Two participants responded that cultural diversity and supporting ATSI learning needs is challenging.

During interviews participants identified the ways their work could be changed to be more satisfying. Suggestions included: support for permanency ( n=7), improved position status (n= 3), increased support from their classroom teacher, and NT DET implementing a career pathway to teaching through having an apprenticeship. Importantly a number of participants expressed that there is no need to make any changes to their role (n= 5).

Motivation

Participants expressed mixed motivations to undertaking the training. Sources for motivation could be broadly described as internal and external. High among the internal motivators identified were;

• the importance of mental s• value of investing in your own professional development and professional future

• valuing of the completion of a qualification• wanting to do a better job and learn professionally

• seizing the moment as access free training is a rare opportunity

After all these years of experience now was a chance for recognition, this was good for my confidence

External motivators identified were;• Opportunities for more money

• Other teaching staff motivating them to have a go even when they were not confident about their potential for success or level of academic skills on entry

Funding and workplace delivery approach as motivators

Participants were asked to reflect on funding support and time scheduling of course delivery.Responses establish that provision of fullmotivator for participation. Respondents express mixed motivation to attend had the course had been delivered out of school hours Figure 5 Number of trainees’ partially funded and if delivered out of work hours.

Participants expressed mixed motivations to undertaking the training. Sources for motivation could be broadly described as internal and external. High among the internal motivators

the importance of mental stimulation value of investing in your own professional development and professional future

valuing of the completion of a qualification wanting to do a better job and learn professionally

seizing the moment as access free training is a rare opportunity

ter all these years of experience now was a chance for recognition, this was good for

External motivators identified were; Opportunities for more money

Other teaching staff motivating them to have a go even when they were not confident their potential for success or level of academic skills on entry

Funding and workplace delivery approach as motivators

Participants were asked to reflect on funding support and time scheduling of course delivery.Responses establish that provision of full funding, making enrolment free, has been a major motivator for participation. Respondents express mixed motivation to attend had the course had been delivered out of school hours (Figure 5).

of trainees’ and agreement to participate in training partially funded and if delivered out of work hours.

29

Participants expressed mixed motivations to undertaking the training. Sources for motivation could be broadly described as internal and external. High among the internal motivators

value of investing in your own professional development and professional future

ter all these years of experience now was a chance for recognition, this was good for

Other teaching staff motivating them to have a go even when they were not confident their potential for success or level of academic skills on entry

Participants were asked to reflect on funding support and time scheduling of course delivery. funding, making enrolment free, has been a major

motivator for participation. Respondents express mixed motivation to attend had the course

aining based on fully or

30

Changes to improve the course

Interviews also focused on the areas of ideas for changes to improve the course in the future. A large number of respondents noted that no changes were required (n= 19). Others suggested changes related to timing of delivery, course documentation and a number of specific issues were identified (Figure 6). Figure 6 Proportion of trainees suggesting improvements to course

More time for completion of course requirements, and especially more time to access the instructor, were identified as areas for change in the future.

more time for group instruction for large groups or time for individual support learning time with the instructor to support individual work

Other changes suggested were:

• Review timing and structure of program delivery: possibly over longer time, delivery through block sessions, maybe even a block during the holidays or a half day session

• Clarify course documentation, workbook clarity and assessment guidelines were indentified as requiring change.

• Other suggestions included: Occasional quest speakers and extend recognition of varied roles and duties of the Education Support Workers roles.

Element 2 School based literacy initiatives (4-8 year olds) Element 3 School based numeracy initiatives (4-8 year olds)

Two high quality digital and print based resources were developed for supporting paraprofessionals learning, Talking Namba and Assisting Paraprofessionals Literacy Learning through Strong Literacy and Numeracy in Communities pilot project. The resource development was accompanied by delivery of professional learning workshops and was available to all schools. These resources continue to be freely available through the Department of Education website.

No Change

Timing

Course Documentation

Other Ideas

31

Element 4 Community based early learning (0-3 year olds)

The sustainability of activities that provide opportunities for capacity building of Aboriginal and Torres Strait Islander parents, community members and service providers to support early literacy and numeracy learning in very young children is largely dependent on the provision of the personnel through FaFT-IPSS in remote schools and school based resources in urban sites. Although not aimed at the families of 0-3 year olds, the Strong Literacy and Numeracy in Communities pilot project developed a transition to school resource, Home 2 School workshop outline. This resource is a guide for teachers on conducting a needs analysis and actions to support families and children for the transition into full time formal schooling.

32

4.4 Unexpected impact on practice and personnel

Element 1 Training and professional learning for paraprofessional staff working with Aboriginal and Torres Strait Islander students

Qualitative theme analysis

Participants attitudes, beliefs and values

Participants were asked to identify their own personal beliefs about ‘what makes a good school teacher’. A range of interpersonal abilities, personal attributes and relevant teacher based skills and knowledge were consistently identified as important. Participants had high expectations of humanistic behaviour by teachers and expect them to have and use high levels of professional knowledge and judgement.

They must be able to communicate with the class, capture children’s attention, make it fun and enjoyable Strict and consistent so the kids know their boundaries Have high expectations of children, including children with special needs Good behaviour management skills Organised in their work and with their time Overall they need to be a positive role model

A good teacher was also identified to be culturally aware and inclusive, especially with Aboriginal students and students with disabilities. Has there been any perceivable impact from the training? For some it was difficult to judge if there has been any impact from the training (n=4). Others described impacts on competency around literacy and numeracy activities, knowledge of the teachers and support workers roles, confidence in interactions and using your own initiative within the work role. For example:

everyone has a lot more literacy interested at the moment AT's to know what teachers do its been really beneficial...learnt how to do different strategies which are helpful instead of just guessing...can apply what we're taught and of course that’s going better.

Longer term plans There was strong agreement that the training could enhance future employment (n=17). Specifically identified in responses were the provision of skills and knowledge around:

• supporting cultural diversity

• high level communication

33

• understanding the importance of workplace cultures, policies and procedures including workplace policies

A perceived outcome was that the training could potentially link them to wider employment opportunities in child and community service areas. Impact of tenuous employment contracts of longer term plans Some participants were sure of future employment in education as they held or had been offered a permanent school based position ( 14), but the majority were in tenuous employment situations. They expressed that they were hopeful, but uncertain, of future employment at their current schools (19) . Some were uncertain around employment and were considering further study or employment in other fields. Lack of status and low financial remuneration for their work were noted in responses:

Its financial pressure I’m applying for a different job…would rather stay in current job but pays just not high enough.

When asked to share long term goals the most significant aspiration was for pursuing higher education. A number of the ATs had commenced or are planning for a career in teaching (n=6). Others are satisfied with their current work with children in the education environment (n=8).

34

5 Conclusions and recommendations

The feasibility of drawing conclusions about the impact of this pilot project are limited for those elements other than the training or paraprofessionals. This reflects the evidence for good implementation science and the need to “bed down” program activity successfully in a staged manner which also enables fidelity and considered adaptations31,32.

“Poorly designed services delivered by staff who are inadequately trained, underpaid and/or overburdened with heavy caseloads generally cost less but are unlikely to produce significant benefits. Knowledge-based interventions that are funded sufficiently and delivered effectively by well-compensated staff with appropriate skills can produce important outcomes that generate a substantial return on the investment.” (Shonkoff, 2004)33

The circumstances of the NT with high proportions of disadvantaged, remote and Indigenous population, and under qualified and transient workforce limits the feasibility of implementing services and interventions that research has shown to be most effective in other settings. The evidence from implementation science about effective program implementation points to the importance of systematic program delivery methods to ensure that the implementation is capable of replicating the key program elements and ensuring that participants have sufficient exposure to and experience of the essential program features. In this project the flexibility for local, school based initiatives for literacy and numeracy programs and 0-3 early learning programs did not provide for coherent and consistent measure or study of impact or effectiveness.

Good implementation science requires investment in establishing strong systems to support the fidelity, consistency and sustainability of program delivery which was evident in the design of the paraprofessional training. Further, the requirements for the training element of this project, processes and structures for support which were monitored and tracked contributed to optimizing outcomes. This element of the Strong Literacy and Numeracy in Communities pilot project offers some transferable findings for expanding the capacity building required for educational support staff across the Territory who often are the more consistent staff working closely with more disadvantaged students.

35

Table 11 Enabling components of successful qualification completions

Enablers Details Funding All training fees fully funded. Have all training in school time and not after school hours.

Where using productivity places this could be as little as $500 per person. All trainees said the cost of training was prohibitive. Most paraprofessionals are paid for face-to-face contact on casual rates only and have family commitments that prevent them staying after school hours.

Group training Dependent on the school’s capacity to absorb the impact, enrol four or more from each school to create a training cohort. Bring trainees from different schools together for the training sessions and extra professional development.

Schools varied. Some were easily able to support up to six and one had 18. This needs the full support of the Principal and teachers. All students identified the fact that their peers were also engaged in training as a positive factor. Trainees indicated anecdotally and in formal interviews that doing the training with paraprofessionals from other schools was a positive factor in the training.

Support Support the training with a training support teacher across a number of schools; number determined by the size of each cohort. Provide VET awareness training to trainee identified teachers.

The project officers funded by the pilot supported the trainees through their training requirements, liaised between the school, trainees, RTO and pilot team; keeping all focused throughout. Teacher mentors trained in round one were not always the ones who ultimately supported trainees. A better approach would be to provide training around VET processes to those who the trainee advises is their school support person. (often the teacher they work with the most).

Content Provide professional learning in line with the certificate training in the areas of literacy, numeracy and child development.

The pilot provided professional learning in supporting student learning in oral language, reading, writing and numeracy. The legacy products from this professional learning are available for use by DET and other sectors.

Timing Enrol and begin training in term one and complete in term four.

For external reasons round two of the pilot was conducted over terms 4 of 2010 and terms 1 and 2 of 2011. This was not a preference of the pilot and placed too much pressure on trainees and RTOs alike. However, having a specific end date is likely to ensure a higher completion rate and many trainees felt that they were glad it was all done in a shorter time (than is often the case).

36

References 1 McCain, MN, Mustard, FJ and Shanker, S (2007). Early Years Study 2: Putting Science into Action. Toronto: Council for Early Child Development. 2 Shonkoff, JP and Phillips, DA (2000). From Neurons to Neighbourhoods: The Science of Early Childhood Development., National Research Council and Institute of Medicine Washington, DC: National Academy Press. 3 Nettleton, C, Napolitano, DA and Stephens, C (Editors) (2007). An overview of current knowledge of the social determinants of Indigenous health. London: London School of Hygiene and Tropical Medicine. 4 Marmot M, Allen J, Goldblatt P, Boyce T, McNeish D, Grady M, et al. (2010). Fair Society, Healthy Lives: The Marmot Review. London: University College of London, Research Department of Epidemiology and Public Health. 5 McTurk, N, Nutton, G, Lea, T, Robinson, G & Carapetis, J (2008). The school readiness of

Australian Indigenous children: A review of the literature. Darwin: Charles Darwin University. 6 Desforges, C and Abouchaar, A (2003). The impact of parental involvement, parental support and family education on student achievement and adjustment: a literature review. Department of Education and Skills: Queens Printer. Retrieved from http://www.dcsf.gov.uk/research/data/uploadfiles/RR433.pdf 7 Henderson, A T and Mapp, K L (2002). A New Wave of Evidence The Impact of School, Family, and Community Connections on Student Achievement: Annual Synthesis 2002. Texas: National Center for Family and Community Connections with Schools. 8 Eccles, J S, and Harold, R D (1996). Family involvement in children's and adolescents' schooling. In A. Booth and J. F. Dunn (Eds.), Family school links (pp. 3–34). Mahwah, NJ: Lawrence Erlbaum Associates. 9 Department of Education, Training and Youth Affairs (DETYA) (2000). 10 Department of Education Science and Training [DEST] (2005). 11 Vinson, T (2009a). Social inclusion: Indigenous social exclusion. DEEWR: Canberra. 12 Vinson, T (2009b). Social inclusion: Early Childhood Development. DEEWR: Canberra. 13 Vinson, T (2009c). Social Inclusion. Markedly socially disadvantaged localities in Australia: Their Nature and Possible Remediation. DEEWR: Canberra. 14 Hobcraft (1998). Intergenerational and Life-Course Transmission of Social Exclusion: Influences and Childhood Poverty, Family Disruption and Contact with the Police, CASE Paper 15. Centre for Analysis of Social Exclusion, London School of Economics: London. 15 MCEETYA (2001) Task Force Paper. 16 ACARA (2009). 2009 National Assessment Program: Literacy and numeracy. 17 Silburn, S.R., McKenzie, J.W., Moss, B. "Northern Territory results for the Australian Early Development Index 2009". 2010 Menzies School of Health Research & Northern Territory, Department of Education and Training. 18 Louden, W., Wildy, H., Rohl, M., Barratt-Pugh, C., Brown, C., Cairney, T., Elderfield, J., House, H., Meiers, M., Rivalland, J. and Rowe, K. 2005, In Teachers’ Hands: Effective Literacy Teaching Practices in the Early Years of Schooling. Edith Cowan University: Perth.

37

19 Department of Education, Science and Training, 2005 National Inquiry into the Teaching of Reading. Commonwealth of Australia: Canberra. 20 SNAICC 2004 Research Priorities for Indigenous Children and Youth. Fitzroy: Secretariat of National Aboriginal and Islander Child Care) 21 Frigo, T. and Adams, I. 2002, Diversity and Learning in the Early Years of School. Paper presented at the Annual Conference of the Australian Association for Research in Education 22 Dockett, S., Perry, B., Mason, T., Simpson, T., Howard, P., Whitton, D., Gilbert, S., Pearce, S., Sanagavarapu, P., Skattebol, J. & Woodrow, C. (2008). Successful transition programs from prior-to-school to school for Aboriginal and Torres Strait Islander children: Project Reference Group 23 Biddle, N. (2007). Indigenous Australians and Pre-school Education. Australian Journal of Early Childhood, 32(3), 9-16. 24 DEST. (2005). Report of the National Enquiry into Teaching and Literacy. 25 Margetts, K. (2007). Preparing Children for School: Benefits and Privaleges. Australian Journal of Early Childhood, 32(2), 43-50. 26 Waller, M. A., Okamoto, S.K., Henkerson, A.A., Hibbeler, T., Hibbeler, P., McIntyre, P. and McAllen-Walker, R. (2002). The Hoop of Learning: A Holisitc, Multisystemic Model for Facilitating Educaitonal Resilience among Indigenous Students. Journal of Sociology and Social Welfare, 29(1), 97-116. 27 Purdie, N., Tripcony, P., Boulton-Lewis, G., Fanshawe, J. & Gunstone, A. (2000). Positive Self–Identity for Indigenous Students and its Relationship to School Outcomes. 28 Thorpe, K., Tayler, C., Bridgstock, R., Grieshaber, S., Skoien, P., Danby, S., 2005, Preparing for School: Report of the Queensland Preparing for School Trials 2003/4 , Brisbane: School of Early Childhood, QUT 29 Hanlen, W. (2007). Indigenous literacies: moving from social construction. In L. Makin, C. Lones Diaz & C. McLachlan (Eds.), Literacies in Childhood: Changing views, challenging practice (2nd ed., pp. 230-242). Marrickville, NSW: Elsevier. 30 National Centre for Vocational Education Research 2011, Australian vocational education and training statistics: Likelihood of completing a VET qualification 2005 -07, Melbourne: NCVER. 31 McTurk, N., Carapetis, J., Lea, T., Robinson, G. and Nutton, G. 2008, The School Readiness of Australian Indigenous Children: A Review of the Literature, Menzies School of Health Research and School for Social Policy Research, Charles Darwin University. 32 Fixsen D, Naoom S, Blase D, Friedman R, Wallace F.2005, Implementation Research: A Synthesis of the Literature. Florida: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network. 33 Shonkoff JP. Science, Policy and the Young Developing Child: Closing the gap between what we know and what we do. Chicago: Ounce of Prevention Fund, Brandeis University, 2004.