Aboriginal and Aboriginal and Torres Strait Islander delegates are advised that this presentation will contain images and names of people who may have died.

A school-based, service-learning model: what are the outcomes for

Aboriginal pupils? Louise Brown, Pascale Dettwiller, Trish Maroney

Acknowledgments

• We acknowledge the Australian Aboriginal and Torres Strait Islander peoples as the first inhabitants of this country and pay our respects to the Traditional Owners and Elders, past and present, of the land on which we stand and learn today.

• We would like to express our gratitude to the Elders past, present and future without whom we would not be here.

3

The third team member

Trish Maroney, Wadarman woman from the Katherine region and Speech Pathologist, whose inspiration and passion for Indigenous Health and Wellbeing have guided this work, and who unexpectedly passed away in July 2016. She was working on this project until her last days and is sadly missed by us all. He family are very proud of her work and have provided this picture.

The program’s development • Speaking Easy for Living and Learning (SELL) demonstrates collaboration

between health and education to address the health services gaps in speech pathology.

• Great support from Binjari Community, Bev Paterson of Good Beginnings and the staff and students of Clyde Fenton School

• The program has serviced primary school pupils since late 2013 with students from Sydney University and James Cook University (JCU)

• In 2016, two groups of six JCU students worked at two schools in Katherine

• In 2017 students from Flinders University, James Cook University, Edith Cowan University and Curtin University are providing services to two schools for the full school year

• Funding for supervision from various sources including Smith Foundation, Flinders Uni, JCU, UDRH

• Funding for students varies across Unis.

Student led, Service Learning Framework

• Service learning: • benefits the community which is in need of a service, • engages students in challenging community settings students • Appears to shape students’ willingness to work with underserved communities

• Service learning is increasingly being recognised as an important part of health curricula and contributes to the community engagement of universities.

• International literature has recognised its contribution to community engagement and acknowledged its offering of a sustainable model to address gaps in health care services in rural and remote areas.

• (Jacoby, 2003; Van Doort et al, 2013.)

7

Special Ed teacher Gaby Stephens SP students from JCU Jessica and Megan.

Students from University of Sydney in the Clinic

Pascale and Trish in action Clyde Fenton School Why are we doing this?

Ball & Lewis, 2011; Snow & Pownell, 2012.

https://www.slideshare.net/PippaTotraku1/presentation-brighton-uni-september-16-3/10

Objectives of this paper

• Firstly, to investigate whether criterion-referenced scaling, applied to paediatric speech can be used as one systematic method of documenting some movements or changes toward the therapeutically desirable objectives assisting for planning and evaluating overall progress over time;

• Secondly, to establish whether the criterion-referenced scaling may be a culturally acceptable measure for language difference and language disorder for Australian Indigenous pupils as it has been recommended for other First Nation communities.

Outcome measurement of S.E.L.L. Criterion referenced Goal Attainment Scaling (GAS)

• RW Schlosser (2004) described use in paediatric speech pathology

• Krasny-Pacini (2016) – describe GAS as ‘individualized, criterion referenced measure of a client’s goal achievement’

• They developed a Critical Appraisal approach for GAS which we will apply to our processes later in this paper

10

The specific use of C-GAS for S.E.L.L.

• To define goals suited to Australian Aboriginal pupils underpinned by an accurate identification of disorders and needs.

• To design a program that will use this technique for each pupil: not a group comparative method but an internal comparative method: each pupil is their own standard.

• To include Speech Pathology students in the program providing both the therapy and acting to provide interrater agreement (multiple raters measuring goal attainment for the each pupil).

Individual Goals

• Individual children may have very strong first and/or second language, but less fluency or experience in English. (Lowell, 1997)

• Other children may have extreme language difficulty, delay or disorder. (Gould, 2008a)

• Tests and outcome measures cannot be the same for them, even though they may be in the same grade. (Gould, 2008b)

Method - outline

• Location: Clyde Fenton Primary School • Demographic: 49 pupils, 52% male.

• Length of clinic: 7 weeks therapy with 1:1 and/or group therapy • Number of SP students: 26 students (Master and Bachelor) and

universities (JCU and USydney) over 3 years

• Goals for each cycle of therapy were developed for each child – SP with student clinicians

13

Examples of goals for different 9 year old children Child 1 – grammatical form

• able to use 'he' and 'she' with >90% accuracy for identifying male and female at phrase level

• able to produce past tense 'ed' during spontaneous conversation to 60% accuracy

• Child 2 – increasing sentence comprehension and length • able to follow 8 ICW instructions to >90% accuracy • able to use 5 ICW instructions to > 90% accuracy

• Child 3 – phonological awareness • able to identify the initial phoneme in CVC words to >90% accuracy

Results: • A set of 112 individuals’ results was gathered over the seven cycles of

student–led clinics at one primary school.

• Summary T-scores were calculated using a statistical package for the social sciences. T-scores are incrementally increasing with the pupils' progress. The interesting aspect of this mathematical translation to this study is drawn from the aggregation of attainment across individuals, and may be used for the entire cohort.

Individual result for one pupil

16

1

2

4

2

1

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

-2 -1 0 1 2

Fre

quen

cy %

Criteria referenced results for one pupil

Individual results (after 5 cycles of occasion of care)

T-Scores for the cohort (n=49)

17

26

60

86

41

65

-2 -1 0 1 2

FREQ

UEN

CY

Criteria referenced results for each value of the five value of scaling (-2,-1,0,1,2)

Aggregated data for all criteria referenced results (278 occasions of care)

Mean: 54.56; SD: 12.06

How well did the group perform?

18

y = 4.4804x + 51.245 R² = 0.8953

0

10

20

30

40

50

60

70

80

90

100

-6 -4 -2 0 2 4 6 8 10

MEA

N (C

riter

ia re

fern

eced

resu

lts)

278 T-Scores

Seven Cycles of occasion of care for 49 pupils

Findings: • The pupils presented with measurable and significant improvement

during one cycle over six weeks of therapy and the series of cycles. • The positive and raised slope of the trend line demonstrates the

overall progress the pupil cohort has achieved from seven cycles of six weeks block intervention over a three year period.

• Overall, over 50% of Aboriginal students achieved their expected goals level with 70 % achieving above expected level.

• Can we be confident that these findings are reliable and valid?

Critical Appraisal of GAS (Krasny-Pacini et al, 2016; Steenbeek et al 2007)

Content Validity • Collaborative goal setting • Relevance/importance • ICF classification of goal types • Specificity

Reliability – scale construction • Equidistant levels • Preintervention performance • Attainability / difficulty • Time specificity

Reliability of scale rating (IRR) • IRR reported • Precise description of levels • Measurability • Unidimensionality • Context defined

Other criteria • Training • Examiner bias • Statistical analysis • Examples provided

SOCIAL VALIDITY

Content Validity: Collaborative goal setting

It is usually preferred to have client set own goals In this case, SP and students and sometimes teachers collaborated:

“When spelling sight words (from Spelling List Unit provided by teacher), X… will independently self-correct errors as he is writing in 8/10 opportunities” Note: Need to collaborate with the most relevant people in this context

Content Validity: Relevance/importance Here again relevance for child’s success in school was considered:

“able to accurately rephrase at least 80% the key points in news articles, independently and in his own words” Note: Students needed guidance from SP to ensure the relevance and importance of goals for this setting.

Content Validity: ICF classification of goal type Most goals were at activity level, with some moving closer to participation.

‘able to use 3 appropriate social communication features in a role-play situation’

Note: In future include participation (social and school based) goals. Need to have overall plan (not just for this cycle) to address child’s needs. (especially with student clinicians)

Content Validity: Specificity Some were specific: “able to appropriately respond to ‘why?’ questions with a ‘because’ clause to 80% accuracy” Some less specific: able to use 1 ICW to 60% accuracy with low levels of clinician prompting. NOTE: define levels of clinician input

Reliability – Equidistant levels

Criterion 1 -2 Able to follow 2 ICW with 60% accuracy -1 Able to follow 2 ICW with 75% accuracy 0 Able to follow 2 ICW with >90% accuracy +1 Able to follow 3 ICW with 40% accuracy +2 Able to follow 3 ICW with 60% accuracy

Note: Are these equidistant steps?

Reliability – Preintervention performance

All children were evaluated in Week 1 of program. This formed the basis for goal setting. NOTE: Cycle to cycle measures need to be considered. If children had previously been included, evaluations of previous attainments were also reviewed. Individual scores showed: • improvement in 34% of children; • maintenance of achievements in 45% and • drop in performance of 20%

Reliability – Attainability / difficulty

The SP helped the SP students to accurately plan attainable but achievable goals (which also were relevant and important) NOTE: Clinician expertise in school based intervention and cultural needs; Clinical and educational and linguistic knowledge required. Improved student clinical reasoning and treatment planning.

Reliability – Time specificity

• 6 week cycles imposed by duration of student placements

NOTE: Opportunity to compare effects of placement / intervention duration.

Reliability of scale rating : IRR reported • This is an area which we need to address more thoroughly and / or

explicitly

NOTE: • Protocols and CAT recommended that ‘blinded’ person complete final

rating.

Reliability of scale rating: Precise description of levels and Measurability

• Able to produce 3 identical CV structures 5 times in 4 seconds • Able to produce 2 alternating CV combinations 5 times within 10

seconds • Able to produce 2 alternating CV combinations 5 times within 5

seconds • Able to produce 3 alternating CV combinations 5 times within 10

seconds • Able to produce 3 alternating CV combinations 5 times within 5

seconds NOTE: Very specific details, but also showing planned progression

Reliability of scale rating: Unidimensionality

• No goals addressed more than one rateable behaviour.

• NOTE: Examples from other literature: reduction in perception of pain with improving QOL ratings

Reliability of scale rating: Context defined

• The context for these goals was within the therapeutic setting (1:1 or small group)

• NOTE: include broader contexts or situations which are not directly being addressed but which could result from the therapeutic intervention.

Other criteria: Training

The students received outstanding cultural and contextual training. We would like to improve the specific training in GAS for student clinicians and new staff NOTE: introduce into third year clinic at Shalom Christian College and Uni clinic

Other criteria: Examiner bias

• As interrater reliability was not specifically covered, the possibility of examiner bias cannot be excluded.

• NOTE: videotape initial and final sessions, have blinded examiners sample these

Other criteria: Statistical analysis

• We followed the protocols originally described, however there has been discussion about the suitability of this approach (eg Schlosser, 2004)

• NOTE: investigate further

Other criteria: Examples provided Yes!

-2 -1 0 +1 +2 Criterion 1 Able to follow 2 ICW

with 60% accuracy Able to follow 2 ICW with 75% accuracy

Able to follow 2 ICW with >90% accuracy

Able to follow 3 ICW with 40% accuracy

Able to follow 3 ICW with 60% accuracy

Criterion 2 Able to use 1 ICW with 70% accuracy

Able to use 1 ICW with >90% accuracy

Able to use 2 ICW with 60% accuracy

Able to use 2 ICW with 75% accuracy

Able to use 2 ICW with >90% accuracy

Criterion 3 Able to identify and name 0 colours

Able to identify and name 2 colours

Able to identify and name 4 colours

Able to identify and name 6 colours

Able to identify and name 8 colours

Criterion 4 Able to produce 3 identical CV structures 5 times in 4 seconds

Able to produce 2 alternating CV combinations 5 times within 10 seconds

Able to produce 2 alternating CV combinations 5 times within 5 seconds

Able to produce 3 alternating CV combinations 5 times within 10 seconds

Able to produce 3 alternating CV combinations 5 times within 5 seconds

Social Validity

‘Social significance and acceptability of goals, methods and outcomes’ (Schlesser et al 2004 p228)

• Trish: ‘my approach is to take a look at world view through categorisation and organisation of lexicon and how that impacts success at school in second language/dialect. Obviously we are focussing on Katherine Region Kriol, but can easily be adapted to local context.’

• Student feedback: the process was inspiring and empowering and the detail about language, phonology, gesture / sign in particular had been of great value clinically.

Testimonies: qualitative data • Students: • “This placement gave me the opportunity to see children with a range of different diagnoses. Working

independently has increased my confidence in my skills and abilities.”(S1)

• “… the cultural support has a paramount effect on the placement together with the supervisor support. My confidence across all areas of competencies has improved as a result of this placement.”(S2)

• Parents: • “I am so glad that the program was offered again. My (child name) is using more words, gets ready in the

morning to go to school and is more chatty when he comes back.”(P2)

• Teacher: • “Having the students at CFS is a huge benefit for our students with speech difficulties who struggle to access

these professional services on a regular basis. Any access to this service is great, but the girls were professional, caring and dedicated to their work. Was great to have help with (student name)’s processing difficulty as well as the usual speech difficulties.”(T2) 38

Conclusion: • Critical Appraisal was used to evaluate the use of criterion-referenced GAS

to measure therapeutically desirable outcomes • This process has assisted in confirming the children’s overall progress over

time during the S.E.L.L. program Individual scores showed: • improvement in 34% of children; • maintenance of achievements in 45% and • drop in performance of 20%

• We suggest that such criterion-referenced scaling may be a culturally

acceptable outcome measure for language difference and language disorder for Australian Indigenous pupils as it has been recommended for other First Nation communities.

References • Ball J and Lewis M (2011) ‘An altogether different approach’: Roles of Speech-Language Pathologists in

supporting Indigenous children’s language development. Canadian Journal of Speech-Language Pathology and Audiology. Volume 35, No 2, Summer 2011, pp 144-158

• Gill, C., Moorer-Cook, L., Armstrong, E., & Gill, K. (2012). The Ability to Follow Verbal Directions: Identifying Skill Levels and Measuring Progress. Canadian Journal of Speech-Language Pathology and Audiology, 36 (3), 234-247.

• Gould, J. (2008a). Non-standard assessment practices in the evaluation of communication in Australian Aboriginal children. Clinical Linguistics & Phonetics, 22(8), 643-657.

• Gould, J. (2008b). Language difference or language disorder: Discourse sampling in speech pathology assessments for Indigenous children. In J. Simpsons, & G. Wigglesworth (Eds.), Children’s language and multilingualism: Indigenous language use at home and school (pp. 194-215). London: Continuum International Publishing Group.

• Jacoby B. (2003) Building Partnerships for Service-Learning. San Francisco, CA: John Wiley and Sons Inc.,. • Kerisuk & Sherman • Lowell A, Gurimangu, Nyomba & Yingi (1997) Yolngu ways of helping their children: Communication at home

in an Australian Aboriginal Community. Australian Communication Quarterly, Summer 1996/97 pp, 20-22

References • McDougall J & King G (2007) Goal Attainment Scaling: Description, Utility and Applications in

Paediatric Therapy Services. Australian Occupational Therapy Journal ; 50: (4) • Schlosser R. Goal attainment scaling as a clinical measurement technique in a communication

disorders: a critical review. Journal of Communication Disorders; 37: 217-239. • Snow P and Powell M. Youth (in)justice: Oral language competence in early life and risk for

engagement in antisocial behaviour in adolescence. (2012) Australian Institute of Criminology: Trends and Issues in crime and criminal justice. No.435. Australian Institute of Criminology. Australian Government,.

• Steenbeek, D., Ketelaar, M., Galama, K. and Gorter, J. W. (2008), Goal Attainment Scaling in paediatric rehabilitation: a report on the clinical training of an interdisciplinary team. Child: Care, Health and Development, 34: 521–529. doi:10.1111/j.1365-2214.2008.00841.x

• Tennant A. Goal attainment scaling: Current methodological challenges. Disability and Rehabilitation; 29: 1583–1588

• Van Dort S, Coyle J, Wilson L, Ibrahim H. (2013)Implementing the World report on Disability in Malaysia: A student-led service to promote knowledge and innovation. International Journal of Speech-Language; 15(1): 90-95.