Designing educational tools, based on body interaction, for children with special needs
who present different motor skills
Benoît Bossavit, Alfredo Pina
Public University of Navarre, Spain
Benoît Bossavit 17 /10/2014
General Motivation
Benoît Bossavit 17 /10/2014
Existing work
• Exergames – Hernandez H.A., Ye Z., Graham T.C.N., Fehlings D., Switzer L. (2013). Designing Action-based
Exergames for Children with Cerebral Palsy. In Proc. of international conference on Human Factors in Computing Systems, pp. 1261-1270
Benoît Bossavit 17 /10/2014
Existing work
• Rehabilitation – Luna-Oliva L., Ortiz-Gutiérrez R.M., Cano-De la Cuerda R., Piédrola R.M., Alquacil-Diego I.M.,
Sánchez-Camarero C., Martínez-Culebras M.C. (2013). Kinect Xbox 360 as a therapeutic modality for children with cerebral palsy in a school environment: a preliminary study. Journal of NeuroRehabilitation, 33(4), 513-521
Benoît Bossavit 17 /10/2014
Existing work
• Learning postures and gestures – Casas X., Herrera G., Coma I., Fernández M. (2012). A Kinect-based Augmented Reality System for
Individuals with Autism Spectrum Disorders. GRAPP/IVAPP, pp 440-446. ScieTePress
Benoît Bossavit 17 /10/2014
Issues
• Accessibility – motor disabilities may hinder the use of
applications beyond the physical interaction
• Impact – gap between research development and its
applicability in schools
• Adaptation – each child has specific, individual needs
Benoît Bossavit 17 /10/2014
Organization
Social skills
Family Education
Evaluation
Interactions +
Technology Adaptation
Benoît Bossavit 17 /10/2014
Organization
Social skills
Family Education
Evaluation
Interactions +
Technology Adaptation
Which technology? Which skill?
Benoît Bossavit 17 /10/2014
Specialized State school
• Educational program
Learning Areas
• Autonomy • Sensory integration • Participation within the environment • Motor skills • Language / Audition • Communication / Expressive language • Music • Health
Benoît Bossavit 17 /10/2014
Specialized State school
• Educational program
Learning Areas
• Autonomy • Sensory integration • Participation within the environment • Motor skills • Language / Audition • Communication / Expressive language • Music • Health
Body skills
Benoît Bossavit 17 /10/2014
Body skills
• Positions / Balance
• Sensation / Perception
• Proprioception
• Visual / Auditory stimulation
Benoît Bossavit 17 /10/2014
• Game device for Xbox
• Interaction with the body • Multi-player • Compact • Cheap
• Small resolution • Little precision
What is the Microsoft Kinect?
Benoît Bossavit 17 /10/2014
Specialized State school
• Educational program
Learning Areas
• Autonomy • Sensory integration • Participation within the environment • Motor skills • Language / Audition • Communication / Expressive language • Music • Health
Creativity
Benoît Bossavit 17 /10/2014
Creativity
• Music
• Painting
• Imagination
• Corporal expression
Benoît Bossavit 17 /10/2014
Objectives
• all children would be able to interact with activities regardless of their motor skills
• provoke interest and motivation for children working on educational aspects
Benoît Bossavit 17 /10/2014
To take into account
• Must work individually or in mini-group • To promote active participation
• Each child has his own challenges, needs • Need different kind of applications
• Each child has his own motor skills
• Need different ways of interaction
Benoît Bossavit 17 /10/2014
Objective 1: Make the application accessible to all
• User profile
• Adapt the Graphics User Interfaces (GUIs) – Use of WIMP to interact
• Windows • Icons • Menus • Pointers
Benoît Bossavit 17 /10/2014
How to adapt the GUIs?
• How to access all the part of the screen? – Windows & Pointers
Benoît Bossavit 17 /10/2014
How to adapt the GUIs?
• How to activate a button? – Icons
Benoît Bossavit 17 /10/2014
How to adapt the GUIs?
• How to activate a menu? – Menus
Benoît Bossavit 17 /10/2014
How to adapt the GUIs?
• How to navigate through a menu? – Menus
Benoît Bossavit 17 /10/2014
How to adapt the GUIs?
• How to activate a state?
Benoît Bossavit 17 /10/2014
Objective 2: How to motivate children
• “Painting” activity
Benoît Bossavit 17 /10/2014
Objective 2: How to motivate children
• “Discover an image” activity
Benoît Bossavit 17 /10/2014
Objective 2: How to motivate children
• “Creating music” activity
Benoît Bossavit 17 /10/2014
Case studies
• 6 Children (4 boys and 2 girls) • 3 sessions (1 per week) • Setup:
Benoît Bossavit 17 /10/2014
Case studies Name Age Cognitive
age Cognitive disability Motor disability
Mr. Brown 15 8-9 Intellectual disabilities Oppositional defiant disorder
-
Benoît Bossavit 17 /10/2014
Mr. Brown
• 20 min / session
• Drawing a house with the same colour • Drawing a house with different colours
• Choosing a musical instrument • Playing specific sequences of notes
Benoît Bossavit 17 /10/2014
Mr. Brown
• Personal objectives
– Regulation of his conduct
– Expression of emotions
– Interact with the body
– Understanding and execution of an order – Encouraging the use of technology
Session 1 Session 2 Session 3
Benoît Bossavit 17 /10/2014
Case studies Name Age Cognitive
age Cognitive disability Motor disability
Mr. Brown 15 8-9 Intellectual disabilities Oppositional defiant disorder
-
Mr. White 15 7-8 Intellectual disabilities Cerebral palsy
Tetra-paresis (wheelchair)
Benoît Bossavit 17 /10/2014
Mr. White
• 15 min / session
• Clearing partially the screen • Discovering the content • Answering the questions
Benoît Bossavit 17 /10/2014
Mr. White
• Personal objectives
– Knowledge of the topic of the class
– Expression of emotions
– Interact with the body
– Encouraging the use of technology
Session 1 Session 2 Session 3
Benoît Bossavit 17 /10/2014
Case studies Name Age Cognitive
age Cognitive disability Motor disability
Mr. Brown 15 8-9 Intellectual disabilities Oppositional defiant disorder
-
Mr. White 15 7-8 Intellectual disabilities Cerebral palsy
Tetra-paresis (wheelchair)
Mr. Orange 14 7-8 Intellectual disabilities
Degenerative muscular dystrophy
Benoît Bossavit 17 /10/2014
Mr. Orange
• 15 min / session
• Choosing a musical instrument • Playing specific sequences of notes
Benoît Bossavit 17 /10/2014
Mr. Orange
• Personal objectives
– Expression of emotions
– Interact with the body
– Understanding and execution of an order
– Encouraging the use of technology
Session 1 Session 2 Session 3
Benoît Bossavit 17 /10/2014
Case studies Name Age Cognitive
age Cognitive disability Motor disability
Mr. Brown 15 8-9 Intellectual disabilities Oppositional defiant disorder
-
Mr. White 15 7-8 Intellectual disabilities Cerebral palsy
Tetra-paresis (wheelchair)
Mr. Orange 14 7-8 Intellectual disabilities
Degenerative muscular dystrophy
Ms. Blonde 18 4-5 Cognitive and intellectual disabilities
-
Benoît Bossavit 17 /10/2014
Ms. Blonde
• 20 min / session
• Drawing circles with one hand • Drawing circles with the other hand • Drawing circles with both hands
• Clearing partially the screen • Discovering the content
o classmate o event she had participated in
Benoît Bossavit 17 /10/2014
Ms. Blonde
• Personal objectives
– Knowledge in plastic art
– Show interest
– Show self esteem
– Encouraging the use of technology
Session 1 Session 2 Session 3
Benoît Bossavit 17 /10/2014
Case studies Name Age Cognitive
age Cognitive disability Motor disability
Mr. Brown 15 8-9 Intellectual disabilities Oppositional defiant disorder
-
Mr. White 15 7-8 Intellectual disabilities Cerebral palsy
Tetra-paresis (wheelchair)
Mr. Orange 14 7-8 Intellectual disabilities
Degenerative muscular dystrophy
Ms. Blonde 18 4-5 Cognitive and intellectual disabilities
-
Ms. Pink 20 6-7 Cognitive and intellectual disabilities Cerebral palsy
Difficulties to execute voluntary movement
(wheelchair)
Benoît Bossavit 17 /10/2014
Ms. Pink
• 10 min / session
• Clearing partially the screen • Discovering the content
o classmate o event she had participated in
Benoît Bossavit 17 /10/2014
Ms. Pink
• Personal objectives
– Show interest
– Show self esteem
– Encouraging the use of technology
Session 1 Session 2 Session 3
Benoît Bossavit 17 /10/2014
Case studies Name Age Cognitive
age Cognitive disability Motor disability
Mr. Brown 15 8-9 Intellectual disabilities Oppositional defiant disorder
-
Mr. White 15 7-8 Intellectual disabilities Cerebral palsy
Tetra-paresis (wheelchair)
Mr. Orange 14 7-8 Intellectual disabilities
Degenerative muscular dystrophy
Ms. Blonde 18 4-5 Cognitive and intellectual disabilities
-
Ms. Pink 20 6-7 Cognitive and intellectual disabilities Cerebral palsy
Difficulties to execute voluntary movement
(wheelchair)
Mr. Blue 8 6 months Intellectual disabilities Cerebral palsy
Partial control of the arms (wheelchair)
Benoît Bossavit 17 /10/2014
Mr. Blue
• 30 min / session
• Clearing partially the screen • Discovering a classmate
Benoît Bossavit 17 /10/2014
Mr. Blue
• Personal objectives
– Motivation in using arms
– Recognition of the person
– Encouraging the use of technology
Session 1 Session 2 Session 3
Benoît Bossavit 17 /10/2014
Discussion
• Objective 1 • all children would be able to interact with
activities regardless their motor skills
– 5 out of 6 children interacted without difficulties – Interaction techniques fit the different needs – Mr. White was willing to improve the control of his
movements
– Kinect can be used as adaptive assistive technology
Benoît Bossavit 17 /10/2014
Discussion
• Objective 2 • provoke interest and motivation to children in
working educational aspects
– Simple interfaces to motivate – Almost all children were willing to participate – Teachers preferred the “discovering an image” activity
– The activity should be customizable by the teachers in order to adapt it to the children needs
Benoît Bossavit 17 /10/2014
Thank you!
Any questions?
Benoît Bossavit, Alfredo Pina
Public University of Navarre, Spain