development of communication, designing effective aac ... · development of communication, language...

8/8/16

RERC-AAC.ORG 1

DesigningeffectiveAACsystemsforyoungchildrenwithcomplex

communicationneedsJaniceLight,Ph.D.

PennStateUniversityForhandoutsvisithttp://aac.psu.edu

Developmentofcommunication,language&literacyskills

• Duringthefirst5yearsoflife,typicallydevelopingchildrendemonstrateremarkablegrowth– frombirthwhentheyare

• preintentional &presymbolic

– tothestartofschoolyearswhentheyareableto• expressawiderangeofintents• understand&expressthousandsofvocabularyconcepts• generatecomplexsentencestocommunicateideas• tellstories,provideexplanations,&developpersuasivearguments• acquireconventionalliteracyskillstoread&write

2

Whatdoweknow?• GrowingbodyofresearchthatdemonstratesthatAACinterventioncanhaveapositiveeffectoncommunicationofindividualswithCCN– Turntaking/participation– Requesting/commenting– Receptive&expressivevocabulary– Length&complexityofmessages– Phonologicalawareness/reading&writingskills– Challengingbehavior,etc.

• Thesebenefitsarerealizedatnorisktospeechdevelopment

3

LanguageandcommunicationoutcomesforchildrenwithCCN

• Despitethispotential,mostchildrenwithCCN– Experiencesignificantlimitationsintheircommunicativecompetence

– Demonstratelimitationsinpragmatic,semantic,syntactic,&morphologicalknowledge&skills

– Enteradulthoodwithoutfunctionalliteracyskills

• ButitdoesNOThavetobethisway

4

8/8/16

RERC-AAC.ORG 2

ChallengesofAACintervention• Thereisnothingeasyabout

currentAACintervention– AACsystemsaredifficultto

learn&timeconsumingtoprogram

– AACinterventionrequirespartnerstochangebehaviorsthatarelongestablished

• Needtoreducethedemands– AACsystems/technologies

thatareeasytoprogram/use– Interventionstrategiesthat

areeasytolearn&implementindailylife

5

HowcanweimproveAACinterventiontomaximizeoutcomesforyoungchildrenwithcomplex

communicationneeds?

Requireevidence-basedAACintervention

• Interventionwiththechildwithcomplexcommunicationneeds– EffectivedevelopmentallyappropriateAACsystems– Instructioninlinguistic,operational,social,andstrategicskillstodevelopcommunicativecompetence

• Interventionwithprimarypartners– Ensureopportunitiesforcommunication– Ensureappropriatesupportsforeffectivecommunication

7

LackofattentiontothedesignofAACsystems/technologies

• Todate,wehavegivenonlylimitedattentiontothedesignofAACsystemsforchildren– Evenwiththeadventofmobiletechnologies/apps

• LackofattentiontothedesignofAACsystems/appsforchildrenwithCCNisironic– ThiscomponentofinterventionsubstantiallyaffectsperformanceAND

– Itistheinterventioncomponentthatismosteasilyamenabletochange

8

8/8/16

RERC-AAC.ORG 3

Goalsofthepresentation

• ToreflectontraditionalapproachestoAACsystems

• TosuggestanewapproachtoAACsystemdesigndrivenbychildren’sneeds&skills

• Toshareshapshots ofresearchstudiesthatmayinformusaboutthedesignofmoreeffectiveAACsystemsforyoungchildren

• Toconsiderdirectionsforfutureresearch&developmenttoimprovethedesignofAACsystemsforyoungchildren

9

DesigningeffectiveAACinterventionsforyoungchildren

• Inordertobeoptimallyeffective,AACinterventionsshouldbedrivenby– Child’sneedsandskills

• Butisthiswhatweactuallydo?– Let’sconsidercurrentAACsystems/apps

10

WhydowedesignAACsystemsforchildreninthewaywetypicallydo?

• TypicallydesignedbynondisabledEuropeanAmericanadults– Reflectperspectivesofthedevelopers

• ChildrenwithCCNhavedifferentperspectives– Age/developmentalstatus

– Culture/ethnicity– Disabilitystatus

11

Thedesignchallenge

• ChildrenwithCCNmaynot findAACsystems– Appealing– Easytolearnoruse

• ResearchsuggeststhatyoungchildrenhavedifficultyusingAACsymbolsinthisway– Comprehension– Expression(Trudeau,Sutton&Morford,2014)

12

8/8/16

RERC-AAC.ORG 4

Thechallenge

• HowdowedesignAACsystemsthatare– Developmentallyappropriate?

– Dynamic?– Easytolearn&use?– Appealingtoyoungchildren?

13

Puttingchildrenfirst

• ProposeanewapproachtodesigningAACsystems– Putschildrenfirst– Startswithyoungchildren’sneedsandskillstodrivethedesign

– ResultsinAACsystemsthatare• Developmentallyappropriate• Dynamic• Easytolearnanduse• Appealing

14

Whatdoweknowaboutchilddevelopmentgenerally?

• Visiondevelopment– Nearvisiondevelopsfirst– Attentiontopeople

• Motordevelopment– Childrendevelopcontrolfrom“insideout”

• Bestcontrolatmidline– Contactgesturesdevelopfirst;laterdistal

• Cognitive/linguisticdevelopment– Initialattentiontopersonorobject/event

• Latercoordinatedjointattentiontoperson&object/event– Initiallychildrenarecontext-bound

• Learnnewconceptsincontext– Relyoneventschematosupportcommunication

• Developdisplaced“talk”later• Developmetalinguisticskillsmuchlater

15

Processofchildlanguagedevelopment

PreintentionalIntentionalbutnotsymbolic

EarlysymbolicSemantic

development

Dev’t ofsyntax

/morphology

Dev’t ofmetalinguistic&literacyskills

16

8/8/16

RERC-AAC.ORG 5

Whatdoweknowaboutlanguagedevelopment?

• Children’slanguagesystems– Differsignificantlyfromthoseofadults– Changesignificantlyovertimeastheydevelop

• Ateachstageoflanguagedevelopment,childrendemonstrate– Quantitativedifferences– Qualitativedifferences

• WeneedtodesignAACsystems– Tofitchildren’sneeds&skillsateachstage– Toaccommodatedevelopmentalchangesseamlessly

17

RequirementsforAACsystemstosupportyoungchildren

Providestrongcontextualsupportforearlysymbolicdevelopment

Supportvocabularyexplosion/earlysemanticrelations

Supportacquisitionofsyntax&morphology/

abstractlanguage

18

AACsystemsforbeginningcommunicators

Providestrongcontextualsupport

forsymbolicdevelopment



abstractlanguage

19

ComponentsofAACsystems

• AACsystems/appsforyoungchildreninvolvefivekeycomponents– Representationoflanguageconcepts

– Organizationandlayoutoftheseconcepts

– Navigation– Selection– Output

20

8/8/16

RERC-AAC.ORG 6

Howshouldwerepresentlanguageconceptsforyoung

childrenwithCCN?

HowdowemaptheinternaldevelopinglanguagesystemofchildrenwithCCNtotheexternalAACsystem/app?

TraditionalAACsymbols

• LanguageconceptsrepresentedbyseparateAACsymbols

• Representationstakelanguageconceptsoutofcontext

• Understandingsymbolsreliesonsemanticmemory– “dictionary”definition

22

Howdoyoungchildrenrepresentlanguageconcepts?

• Researchquestions(Light,Worah,Bowker,Drager,Burki,D’Silva,Kristiansen,Jones,&Hammer,2012)

– Howdoyoungchildrenfromdifferentculturalbackgroundsrepresentearlyemergingabstractlanguageconceptsgraphically?

– Howdothechildren’sgraphicrepresentationscomparetothoseofexistingAACsymbolsets?

– DoyoungchildrenunderstandtraditionalAACsymbols?

23

Howdoyoungchildrenrepresentlanguageconcepts?

• Participants– 50typicallydevelopingchildren;ages3-6years– 5differentculturalgroups

• AfricanAmerican;Hispanic;EuropeanAmerican;ImmigrantsfromIndia;Norwegian

• 10earlyemerginglanguageconcepts– allgone,big,come,eat,more,open,up,want,what,who

• Twotasks– Askedtodrawconcepts&describetheirdrawings– ShownPCSforconcepts&askedtonamePCS

24

8/8/16

RERC-AAC.ORG 7

Come

• Only10%identifiedthePCSforcome– Othersthoughtitwas“pointerfinger”,“aboo-boo”,“twodrivewaysandahand”,etc

25“Somebody’s coming, my friend is visiting.”

Children’s representation of “come”

26

Want

• Only4%identifiedthePCSforwant– Theothersthoughtitwas“aTV”,“cutoffhands”,“handsandsoap”,etc

27

Children’srepresentationof“want”

“That’sme.That’smybrother.AndIwantit.”28

8/8/16

RERC-AAC.ORG 8

Who

• 0%identifiedthePCSforwho– Thechildrenthoughtitwas“abackofahead”,“aboyeating”,“ahaircut”,“a7withears”,etc

29

Children’srepresentationof“who”

“Girl says, ‘Mom, who is that?’‘This is your new daddy.’

30

Howdochildren’srepresentationsdifferfromtraditionalAACsymbols?

• Reflectdifferentunderlyingconceptualizations• Includedepictionsofentirescenesorevents

– Conceptsareembeddedincontext– Scenesincludefamiliarpeopleandevents

• Seldomincludepartsofobjectsorpeople– Partsrequireinferenceofthecompleteobject/person&intent

• Donotincludeemblemsorarbitrarysymbols– E.g.,arrows,movementlines,punctuation

31

Implicationsforrepresentinglanguageconcepts

• Youngchildrendonot initiallyunderstandmosttraditionalAACsymbols

• LackoftransparencyofAACsymbols– Addsadditionalcognitivedemands

• Learnlanguageconcept• ANDlearnrepresentation/symbol

– Thereisnoinherentvalueinlearningthissymbol

• Additionallearningdemandsslowdownrateoflanguagelearning

32

8/8/16

RERC-AAC.ORG 9

Howshouldwerepresent,organize,&layoutdisplaysforyoungchildrenwithCCN?

HowdowemaximizeunderstandinganduseoflanguageviaAAC?

TraditionalAACdisplays

Traditionalgridlayout• Eachlanguageconceptis

representedbyseparateAACsymbolsin“boxes”organizedinrows&columns

• Languageistakenoutofcontext;understandingreliesonsemanticmemory

• Eachrepresentationmustbeprocessedseparately,understood,&thenintegrated

Gridfor“playingtelephone”

34

AlternativeapproachtoAACdisplaysVisualscenedisplays

Visualscenedisplay(VSD)• Vocabularyembeddedunder“hotspots”inintegratedvisualscene

• Languageispresentedinmeaningfulcontext

• Sceneisprocessedasanintegratedunit

• Meaningisderivedfromtheentirescene

VSDfor“playingtelephone”

35

ResearchonlayoutofAACdisplays

• Seriesofstudiestoinvestigatetheeffectsofdifferentlayouts– Griddisplays– Visualscenedisplays

• Investigateperformanceofchildrenacrossvariousdevelopmentalstages– Infants(9-12monthsold)– Toddlers(2&3yearsold)– Preschoolers(4&5yearolds)

36

8/8/16

RERC-AAC.ORG 10

Researchontheeffectoftypeofdisplay

• Infantstudy(Wilkinson&Light,inprogress)– 4differentcontextsfamiliartoinfants

• Feeding,bathing,playingball,etc– InfantsviewedpairsofAACdisplaysforeachcontext

• E.g.,PCSgridvs.photoVSD• Position&ordercounterbalanced

– Eyetrackingtechnology• Measurevisualattention/interest

37 38

Typeofdisplayaffectsvisualattentionofinfants

• Infantslookedfirst&longestatphotoVSDcomparedtoPCSgrid

• Infantsat“firstwords”stagedemonstratedstrongpreferenceforphotoVSDs

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

PCSgrid VSD

Meanlengthofviewing,VSDvsgrid

39

Researchontheeffectoftypeofdisplay

• Studieswithtoddlersandpreschoolers– 2½yearolds&3yearolds

• Drager,Light,Curran-Speltz,Fallon,&Jeffries,2003• Drager,Light,Carlson,etal.,2004

– 4&5yearolds• Light,Drager,McCarthy,etal.,2004

• Methods– Childrenaskedtolocatevocabularyusingdifferenttypesofdisplays&tousedisplayscommunicatively

• VSDs(organizedbyevent)• Traditionalgridorganizedschematically(byevent)• Traditionalgridorganizedtaxonomically(bycategory)

40

8/8/16

RERC-AAC.ORG 11

Whatdoesresearchtellus?

• ToddlersweremoreaccuratelocatingvocabularyusingVSDsthangriddisplays(Drager,Light,etal.,2003)

• Itwasnotuntilchildrenwere4&5yearsoldthattheywereabletolocatevocabularywithsimilaraccuracyusingVSDsorgriddisplays(Light,etal.,2004)

41

Whatdoesresearchtellus?

• 4&5yearoldsperformedmuchmoreaccuratelywithVSDsorgridsthanwithiconicencoding/Minspeak(Light,etal.,2004)

• Withappropriateintervention,4-5yearoldswithCCNcanacquirebasicliteracyskills– Requiresystemswithaccess

totraditionalorthography(Light&McNaughton,2009)

42

ImplicationsfordesigningAACdisplaysforchildrenwithCCN

• VSDsmaybebettersuitedthangriddisplaysfor• Infants• Toddlers• Youngerpreschoolers• Olderbeginningcommunicatorswithseveredisabilities

• Comparedtotraditionalgriddisplays,VSDsmay• Attractmorevisualattention• Resultinmoreaccurateperformance• Supportmorerapidlexicaldevelopment/learning

• VSDsarenotappropriateforeveryone• Olderpreschoolers/schoolagechildrenrequireaccessto

traditionalorthography– tosupportmoreadvancedlanguage&literacydevelopment

43

WhydoVSDsenhancetheperformanceofyoungchildren?

• Capturecontextsinwhichyoungchildrenlearnlanguage&communicationskills– Providevisualsupportsforlanguagelearning&use

• Replicateeventsexperiencedbythechildren– Supportaccesstolanguageconceptsviaepisodicmemory;donotrequiresemanticmemory

• Areprocessedasawholeratherthanindividualunits– Reduceworkingmemorydemandscomparedtogrids

44

8/8/16

RERC-AAC.ORG 12

WhydoVSDsenhanceperformanceofyoungchildren?

• Preservetheconceptual&spatialrelationshipsbetweenpeople&objectsasintherealworld– Preserveproportionalsize,location,&function

– Providegreatersupportforcomprehension&use

45

VisualcognitiveprocessingofVSDs(Wilkinson,Light&Drager,2012)

• VSDsalsoexploitthehumancapacityforrapidvisualprocessingofnaturalisticscenes

• Humansprocesssceneswithinfirstglance(Olivia&Torralba,2007)

– Humansconstantlyprocessscenesindailylife– Focusattentiononkeypeople/events– Ignoreirrelevantbackgrounddetails– Presenceofcontextinvisualscenedoesnotappeartoaddvisual“complexity”

46

WhatelementsshouldbeincludedinVSDsfor

youngchildrenwithCCN?HowcanwedesignVSDstomaximize

communication&languagedevelopment?

SignificantvariationinthedesignofVSDs

48

8/8/16

RERC-AAC.ORG 13

DifferencesinthedesignofVSDsforyoungchildren

• TheseVSDsdiffersignificantly– Presence/absenceofhumansengagedinsocialinteraction

– Peopleareafundamentalcomponentofchildren’searlycommunicationexperiences

– Typeofimage• Photosorlinedrawings

– Personalizationofthescene• Personalizedornonpersonalized

49

EffectsofpeopleinVSDs(Wilkinson&Light,2011;Wilkinson&Light,2014)

– Question• Whatistheeffectofpeopleinvisualscenesonvisualprocessing?

• ChildrenwithTD,ASD,DS,IDD

– Procedures• Presentedphotosrepresentingvisualscenes

• Usedeyetrackingresearchtechnologytomeasurewhereparticipantslooked,when,andforhowlong

50

51

Results&ImplicationsfordesignofeffectiveVSDs

• PeopleinVSDsattractedvisualattentionwithinthefirst1-1.5sofviewing– ResultswererobustacrosschildrenwithTD,DS,ASD,IDD

• PeoplewithinVSDsattractmuchmorevisualattentionthanotherelements– Despitethepresenceofmultiplecompetingelementsinthescene

• Elementsthatarelarge,bright,and/orcolorful– Evenwhenthepeopleareverysmall

• Occupyonly2-5%oftheVSD

52

8/8/16

RERC-AAC.ORG 14

Results&implicationsfordesignofeffectiveVSDs

• Participantslookfirst&longestatpeopleinVSDs,buttheyalsofixateonotherelementsinVSDs– Especiallyotheranimatefigures

• E.g.,animals– Alsomoreprominent,centrallylocated,meaningfulitems

• E.g.,Christmastree,objectswithwhichhumansareengaged– Viewerstypicallyignorestaticnon-meaningfulbackgroundelements

• E.g.,floor,curtains,furniture– Keyelementsofscenesareusedtosupportprocessing&understandingofscene

53

ImplicationsforthedesignofeffectiveVSDsforyoungchildren

• VSDsthatincludepeople– Capturethehuman/socialelementsthatarecentraltocommunicationdevelopment

• Thesearethecontextsinwhichlanguagelearningoccurs– Exploitchildren’sinnatevisualattentiontopeople

• VSDsofemptyroomswithnopeopleorstickfiguresmay– Failtocaptureorfocusvisualattention– Failtocapturethesocialelementsthatareintegraltocommunicationdevelopment

54

Constraintsofstaticphotovisualscenedisplays

• Unmetneed– YoungchildrenwithCCNbenefitfromVSDstosupportcommunication

– CurrentAACappsarelimitedtostaticphotoVSDs;failtocapturedynamicevents

– VideoofferspotentialtocapturedynamiceventsbutcurrentAACtechnologiesonlysupportpassivevideoviewing

55

Developingtechnologieswithvideovisualscenedisplays

• Engineering solution– Capturevideoofdailyroutines

• Viabuiltincameras&wirelessimport

– Allowpauseofvideo• CreateVSDsatthesejunctures

• Createhotspotswithspeechoutput

– InitialprototypedevelopedbyInvoTek

– Evaluationstudiesinprogress(staytuned)

56

8/8/16

RERC-AAC.ORG 15

HowshouldwedesignAACsystemstosupportnavigation byyoungchildrenwithCCN?

Navigation

• TraditionalAACsystemshaveutilizedseparatenavigationalmenus– Difficultforyoungchildrentounderstand

– Requirescognitivemapof“hidden”displays

– RequiresunderstandingthatsymbolrepresentsanAACdisplaynotaconcept

58

Alternativedesignstofacilitatenavigation

• UsethumbnailsofVSDsfornavigationmenu– Explicatelinktodisplay

• Includenavigationmenuondisplay– Limitmemorydemands

• Animatetransitionfrommenutomaincommunicationdisplayuponselection– Supportconceptualmodelof

navigation• Model/scaffoldnavigationfor

youngchildreninitially– Donotholdbacklanguage

development

59

Locationofnavigationmenus

• Whereshouldnavigationmenusbelocatedwithindisplays– tominimizedistractionduringcommunication– tomaximizeaccuracy&efficiencyduringnavigation?

• Goal– Evaluateeffectofmenulayoutsonvisualattention&navigationaccuracy&latency

• Tophorizontal,bottomhorizontal,rightvertical,leftvertical– ChildrenwithASD,IDD,DS

60

8/8/16

RERC-AAC.ORG 16

Investigatingvisualcognitiveprocessingdemandsofnavigationlayouts

(Wilkinson,Light,O’Neill,inprogress)

3studieswithchildrenwithCCN

• Procedures– FreeviewingofVSDs

• Determineelementsofdisplaysthatcapturevisualattention

– Cuedviewing• LocationoftargetVSDfornavigation

• Staytunedforresults

EyetrackingdataduringcuedviewingforparticipantwithDownsyndrome

61

How should we design displays to support selection by

young children with CCN?

Howcanwereduceprocessingdemandsandmaximizeaccuracy&

efficiency?

Traditionalapproachestodisplays:Implicationsforselection

• TraditionallyAACdisplaysforyoungchildrenhaveutilizedhorizontalrowsofsymbolsorsimplegrids

• Whatistheeffectoftheselayoutsonselectionaccuracyandefficiency?

63

Effectsoftargetlayoutonaccuracy&efficiencyofselection(Costigan,Light&Newell,2012)

• Participants– 3yearolds&4yearoldswithtypicaldevelopment

• Procedures– Requiredtoselecttargetsofvarioussizeswithmouse

• Horizontaltrajectoryfromstart• Verticaltrajectory• Diagonaltrajectory

– Measuredaccuracy&efficiencyofselection

64

8/8/16

RERC-AAC.ORG 17

Results&implicationsfordesign

Verticaltrajectory Horizontaltrajectory

65

Results&implicationsforselection

Diagonaltrajectory Implications

• 3yearoldswithtypicaldevelopment– Weremostaccurate&efficientwithverticallayoutsoftargets

– Werelessaccurate&efficientwithhorizontal&diagonallayoutsoftargets

66

ImplicationsfordesigningAACsystems

• Considerverticallayoutsoftargetsatmidlineforyoungchildrenandbeginningcommunicators– Offeringchoices– DesigningearlyAACdisplays

67

Whatabouttheoutput ofAACsystems?

Whatoutputwillmaximizelearninganduse?

8/8/16

RERC-AAC.ORG 18

OutputofAACsystems

• Adultstypicallyutilize“motherese”intheirinteractionswithyoungchildren

• Child-directedtalk– Exaggeratedintonation– Shorter,simplerwords&syntax– Soundeffects

• Benefitsofchild-directedtalkintypicaldevelopment– Capturesattentionofinfants&toddlers

• Preferchild-directedtalk– Conveyspositiveaffect– Facilitateslanguagelearning

69

Utilizechild-directedspeechoutput• Reasonabletoproposechild-directedspeechoutputinAAC

systemsmighthavesimilarbenefitsforyoungchildrenwithCCN– IncreaseappealofAAC– CaptureattentiontoAAC– Supportunderstanding– Facilitatelanguagelearning

• Vocabulary• Syntax

• Limitedresearchontheeffectsofchild-directedspeechasAACoutput– PartofpackageAACinterventionthatdemonstratespositiveeffects/difficulttoteaseoutspecificeffects(Light,etal.,2016)

– PreliminaryevidencesuggestingsoundeffectsmayfacilitatelearningofAACsymbols(Harmon,Schlosser,etal.,2014)

70

Supportingsemanticdevelopment





abstractlanguage

71

SemanticdevelopmentTypicaldevelopment

• Oncechildrenhaveacquired“firstwords”,theyexperienceavocabularyexplosion– Rapidlyacquirehundredsofnewconcepts– Learnnewwordsincontext– Adultsscaffoldthisvocabularydevelopment

• Respondtochildren’sneedsandinterests• Introducenewwords/concepts• Providemodelsofuse

– Richvocabularyiscriticaltocognitive&languagedevelopment

72

8/8/16

RERC-AAC.ORG 19

ChallengesforchildrenwhouseAAC

• YoungchildrenwhorelyonAACfacesubstantialchallengesintheirvocabularydevelopment

• 2approaches– Customizeddynamicdisplaysystems– Systemswithpreset“corevocabulary”

73

TraditionalAACinterventionmaylimitchildren’svocabularydevelopment

• AACtechnologiesareoftendifficultforyoungchildrentolearn&use– Children’srateoflanguagelearningisslowedbytheoperationaldemands

– Childrenarenotactivelyinvolvedinvocabularyselection/programmingbecauseofthecomplexity

• Programmingnewvocabularyaddsdemandstofamilies&professionalswhoarealreadyjugglingmanyresponsibilities– Children’slanguagelearningislimitedifnewvocabularyisnot

addedregularly– Families&professionalscannotcapitalizeonteachablemoments

duringdailyinteractions

74

TraditionalAACinterventionmaylimitchildren’svocabularydevelopment

• “Corevocabulary”isnotdevelopmentallyappropriateforyoungchildrenattheearlystagesoflanguagedevelopment– Doesnotincludehighinterest,salient,personalizedconcepts

– Doesnotsupportmeaningfulcommunicationforarangeoffunctions

– Includesanabundanceofstructuralwords• Theseemergelaterindevelopment

– Isnotresponsivetochild’sneeds&interests• Isteacher/SLP-directedratherthanchildresponsive

– Instructionisdisplacedfrommeaningfulcontext• Reliesonsemanticmemorynotepisodicexperiences

75

Potentialbenefitsof“justintime”programming

• JITprogrammingofnewvocabularyoccursontheflywithindailyinteractionsinresponsetochild’sneeds&interests

• AnyAACtechnologycouldbeused“just-in-time”– Provideditissimple&easyto

program• JITprogramming

– Significantlyreducestimedemandsforpartners

– Increasespartnerresponsivity– Allowschildrentobeinvolved

invocabularyselection

76

8/8/16

RERC-AAC.ORG 20

JITAACtechnology

• JITAACsystemsreducenumber&complexityofprogrammingsteps– Allowquick&easyimportofphotosasVSDs

• UsingonboardcameraorcellphonewithBluetoothconnection

– Allowquick&easyadditionofhotspotsandprogrammingofvocabulary

• Drawingofhotspotswithfingerorstylus• Recordingofdigitizedspeech

– Provideprogrammingcontrolseasilyunderstood&usedbyyoungchildren

77

Vocabularydevelopmentispartof,notseparatefrom,dailyinteractions

Programmingofconceptsoccursontheflyduringinteractionsasthe

need&interestoccurs.

JITprogramming- PlayingbabydollsOlivia Selects VSDofbathingdoll

Requests“DOLL”“BATH”PretendsdolliseatingsoapShakesherhead&vocalizes;laughs

Nurse Snaps photoofsceneasnewVSD

Olivia Pointstosoap thendoll’smouthonVSD

Nurse Adds hotspot“SOAP”Addshotspot“EAT”

Olivia Shakesherheadandlaughs

Nurse AddshotspotofOliviashakingherhead“NO”

Olivia Selects“EATSOAP”Selects“NO”“NO”“NO”andlaughs 79

JITprogramming- Playingbabydolls

Olivia Playingdolls;dropsdoll

Nurse Makescryingsound;picksupdoll&kissesthedollOlivialaughs

Olivia PointstoimageofnursekissingonVSD

Janice Adds hotspot“KISSING”+soundeffect

Olivia Selects“KISSING”andlaughsPointstonurse

Nurse Kisses dollOlivialaughs

Janice Writes“k”ondisplay tofosterearlyliteracyskillsinmeaningfulcontextAddshotspot“k”

80

8/8/16

RERC-AAC.ORG 21

Increasedmotordemands• Withincreased

vocabulary,comeincreasedmotordemands– Childrenmaynothaveisolatedpointwithindexfinger

• Utilizingadaptivestrategies– Usethumbtoselect– Allowsprecision– Avoidsunwantedselections

81

Supportingtheacquisitionofmoreadvancedlanguage





abstractlanguage

82

Developmentofsyntax&morphologyTypicaldevelopment

• Childrendevelopearlysemanticrelations– Continuetoexpandvocabulary

• Buildonthisfoundationanddevelopmoresophisticatedsyntax/morphology– Abletocombineconceptstocommunicatemorecomplexmessages

– Includefunctors,morphologicalendings,structuralwords

83

ChallengesforchildrenwhouseAAC• Developmentofsyntax&morphologyverychallengingforchildrenwhouseAAC– Oftenhavelimitedopportunitiestocommunicatecomplexmeanings

• Messagesareco-constructed• Partnersfillinsyntaxandmorphology

– Oftenhavelimitedvocabularies– TypicallyusegraphicAACsymbolsets

• Thesearesemanticsystems;nottruelanguagesystems– Oftendonothaveaccesstomorphologicalmarkers

• Iftheydohaveaccess,thesehaveabstractrepresentationsandrequiremetalinguisticskillstoutilize

84

8/8/16

RERC-AAC.ORG 22

Isthereabetterwaytosupportthedevelopmentofadvanced

languageskills?

Supportingtheacquisitionofadvancedlanguageskills

• Childrenrequireaccesstotruelanguagesystem– Mostlogicalsystemisnativelanguage– Introduceliteracyskills

• Literacyskillsprovidemultiplebenefits– Improvedaccesstoeducation,employment,socialinteraction,mainstreamtechnology,socialmedia

– Supportforlanguagelearning• Accesstorichvocabulary• Visualsupportforlearningsyntaxandmorphology

– Accesstotruegenerativecapacityoflanguage

86

CurrentAACtechnologies/apps• ChildrenwhorequireAAC

typicallyuseAACsystems/appswithgraphicsymbols– E.g.,GriddisplayswithPCS,

Symbolstix,etc.– Minspeak icons– Visualscenedisplays,etc

• Thesesystems/appsdonotsupportthetransitionfromgraphicsymbolstoliteracy– Staticpresentationoftext– Displacedpresentationoftext

inmessagebar

87

AACtechnologiestosupportthetransitiontoliteracy(T2L)

• AACappsthatsupportthetransitiontoliteracy(T2L)– IndividualselectsapicturesymbolfromAACdisplay

– Writtenwordappearsdynamically

– Writtenwordisspokenbytheapp

• 2apps– Grid-basedT2LappdevelopedbySaltillo(Hershberger)

– VSDT2LappdevelopedbyInvoTek (Jakobs)

• IncorporatedintoSnapScenebyTobiiDynavox 88

8/8/16

RERC-AAC.ORG 23

InvestigatingAACtechnologiestosupporttransitiontoliteracy

• 6studiesinprogress– Withchildren&adultswithASD,IDD,CP

– Usinggrid-based&VSDT2Lapps

• PreliminaryresultsdemonstratepositiveeffectofapponliteracyskillsforindividualswithCCN– Withminimalexposuretowrittenwords

89

Providingliteracyinstructiontomaximizelanguagedevelopment

• YoungchildrenwithCCNrequireliteracyinstruction– Startearly– Usemotivating&meaningfulmaterials– Targetappropriateskills

• Lettersoundcorrespondences,phonologicalawareness,decoding/encoding,sightwordrecognition,sharedreading/writing,etc.

– Utilizeeffectiveinstruction• Model,guidedpractice,independentpractice• Providenumerousopportunitiestoapplyskillsinmeaningfulcontexts

– Provideadaptationstosupportparticipation• Alternativeresponsemodalities• Oralscaffoldingsupport• Visualsupports

90

Useliteracyskillstobolsterspeech,language,&communication

• Usewrittenlanguageto– Supportspeechdevelopment

• Lettersasvisualsupports• Lettersoundknowledge

– Supportlanguagedevelopment• Buildmorecomplexcommunication

– Expandsemantic,syntactic,morphological,discoursedev’t– Providevisualsupportoftext;highlightrelevantlanguage

– Fostersocialinteraction• Sharedreading,texting,socialmedia,etc

91

Outcomesofliteracyintervention• ChildrenwithCCNcanlearnliteracyskills

– 15participantsinlargerliteracystudy(Light&McNaughton)– 3yearsoldtoadolescents– Widerangeofspecialneeds

• autism,cerebralpalsy,developmentalapraxia,Downsyndrome,multipledisabilities

– Usingvariousmeansofcommunication• speechapproximations,signs,PECSorotherlowtechsystems,speechgeneratingdevices(SGDs),mobiletechnologyandAACapps

• 100%ofparticipantslearnedtoread– Timetoacquisitionvariedacrossparticipants– Schools&familiesreportedhighlevelsofsatisfactionwithinstruction&outcomes

92

8/8/16

RERC-AAC.ORG 24

AACsystemstosupportyoungchildren’slanguage&communication


Usephoto&videoVSDsofmeaningfuleventswithchild-directedoutput

Supportvocabularyexplosion

UseJITAACtechnologytoprovidearichvocabulary

Involvechildinvocabselection&programming

Supportacquisitionofadvancedlanguage

IntroduceT2LappsTeachliteracyskills

93

FrequencyofturnsexpressedbyJacksonatbaseline&duringintervention(15-37months)

0

50

100

150

200

250

B1 B3 3 5 7 9 13 16 20 22 25 33 37 39 41 43 48 52

94

%ofturnsexpressedviadifferentmodesbyJacksonatbaseline&duringintervention(15-37months)

0102030405060708090

100

B1 B3 3 5 7 9 13 16 20 22 25 33 37 39 41 43 48 52

vocalizations/ speech approximations gestures/signs Aided AAC

95

%ofturnscommunicatedviaspeechbyJackson

0102030405060708090

100

B1 B3 3 5 7 9 13 16 20 22 25 33 37 39 41 43 48 52

vocalizations/ speech approximations

96

8/8/16

RERC-AAC.ORG 25

Conclusions

• WehaveagrowingbodyofevidencethatshowsthatAACinterventionscanhavepositiveeffectsoncommunication,language&literacyskills– Demonstratesthepossible

• WehavenotyetdeterminedhowtooptimizeAACinterventions– CurrentAACsystems/interventionsimposesignificantlearningdemands

– Theymayconstraintherate&scopeoflanguage,literacy,&communicationasaresult

97

Researchpriorities

• Futureresearchisurgentlyrequiredto– Determinevisual,cognitive,language&motorprocessing/developmentofchildrenwithCCN

– Design&evaluateAACsystemsthatrespondtochildren’sneedsacrossdevelopment

• Aredevelopmentallyappropriate• Areeasytolearnanduse• Aredynamic• Areappealingandfun• Provideaccesstothemagicandthepoweroflanguageandcommunication

98

Thefuturechallenge

– Ensurethatwhatispossiblebecomeswhatisoptimal• sothatchildrenwithCCN

canmaximize theircommunication,language,&literacyoutcomes

– Ensurethatwhatispossiblebecomeswhatisprobable• sothatall childrenwith

CCNhavetheopportunitytoachievetheirfullpotential

99

Forhandouts,visithttp://aac.psu.edu

100

8/8/16

RERC-AAC.ORG 26

Acknowledgements• Weareespeciallygratefultotheindividualswhoparticipated

inthisproject.Thankyouforallowingustobeapartofyourlives.

• ThecontentsofthispresentationweredevelopedunderagrantfromtheNationalInstituteonDisability,IndependentLiving,andRehabilitationResearch(NIDILRRgrantnumber#90RE5017)totheRehabilitationEngineeringResearchCenteronAugmentativeandAlternativeCommunication(RERConAAC).http://rerc-aac.org

• NIDILRRisaCenterwithintheAdministrationforCommunityLiving(ACL),DepartmentofHealthandHumanServices(HHS).ThecontentsofthispresentationdonotnecessarilyrepresentthepolicyofNIDILRR,ACL,HHS,andyoushouldnotassumeendorsementbytheFederalGovernment.

101

development of communication, designing effective aac ... · development of communication, language...

Documents