hci challenges in human movement analysis

HCI Challenges in Human Movement Analysis:Touchscreen interaction for older adults

Lilian Genaro Motti Ader, PhD

University College DublinSchool of Public Health,Physiotherapy and Sports Science

EU H2020 Marie Curie Career-FIT - MF2017-0204 Co-fund No: 713654

Human Movement Analysis• Detect, identify, track human body

• Technologies: motion capture, sensors (orientation, acceleration)...• Postures: body position, body segments, joint angles...• Movements: amplitudes, ranges of motion, duration, speed...

30/11/2018 lilian.mottiader[at] ucd.ie - HCI@UCD 2

http://resources.mpi-inf.mpg.de/biomechanics/

Qualisys

Manus VR

Fitbit

TekScan

Applications for HCI• Evaluate ergonomics• Improve performances• Design new interaction techniques

...

3

Ader et al. 2016 Bachynskyi et al. 2015 Bossavit et al. 2014

http://resources.mpi-inf.mpg.de/biomechanics/

30/11/2018 lilian.mottiader[at] ucd.ie - HCI@UCD

Current research: Assessment of gait and balance

4

Sources.Kinesis,Otago exercises

Codamotion, UCD Gait LabPressure matShimmer IMUs


HCI Challenges in Human Movement Analysis

5

HCI Biomechanics


Motti Ader 2018

PhD Thesis “Study of the interaction of older adults with touchscreen”

IRIT Lab, University of Toulouse, France

HandiBio, University of Toulon, France

Touchscreen interaction for Older Adults

7

Usability and Acceptance of technologies

Mallenius et al. 2009, Barnard et al. 2013,Caprani et al. 2015

Fortes et al. 2015


8

Fisk et al. 2009, Lepicard & Vigouroux 2012, Motti et al. 2013, Findlater et al. 2015


Objective

Understand and explain the difficulties older adults find to execute the gestures of interaction on touchscreen


New approach:Human movement analysis

10

• State of the art on movements of tactile interaction Bachynskyi et al. 2015, Irwin et al. 2012, Irwin & Sesto 2011, Kim et al. 2013, Jacquier-Bret et al. 2014, Sesto et al. 2012, Pereira et al. 2013, Shin & Zhu 2011, Werth & Babski-Reeves 2014, Young et al. 2012, Young et al. 2013


Design of research study

=> Understand differences in performances between older and younger users of touchscreen by evaluation of their movements


HCI

- represent diversity- users- settings

- performances- time- errors

Biomechanics

- homogeneity- morphologies- gestures

- controlled environments

- iterations

12

Design of research study


Interactive system “Puzzle Touch”

13

• Facilitate the recruitment and create social interaction• Game, Images

• Existing guidelines + iterative design• Drag-and-drop

HTML CSS + Javascript + PhPParameters (pieces, levels)


Motti Ader 2016

Interactive system “Puzzle Touch”

14

• Facilitate the recruitment and create social interaction• Game, Images

• Existing guidelines + iterative design• Drag-and-drop

HTML CSS + Javascript + PhPParameters (pieces, levels)


Motti Ader 2016

Familiarisation task with “Puzzle Touch”


Movement and performance

• 15 older adults (65-84) vs. 15 adults (18-45)• pre-test: vision, cognition, motor, practice trials

• Smartphone vs. tablet

• Finger vs. Pen

• Targets: 9 large vs. 16 small

30/11/2018 lilian.mottiader[at] ucd.ie - HCI@UCD 16

HandiBio Lab, Toulon, 2014Motti Ader 2016

Equipment

Reflective markers for optical motion capture(22 + 12 + 3)

17

(ISB) Wu et al. 2005


Motti Ader 2016

Motion capture


Motti Ader 2016

Joint: The wrist

• Mobilisation during tactile interaction• Index of movements of upper limbs• Ageing effects on motor control and

proprioception

19

Jacquier-Bret et al. 2014, Wright et al. 2011, ISO 9241


Postures of the wrist

20

Flexion

Extension

Ulnar deviation

Radial deviation

α < 0

α > 0


Detection and tracking of the wrist

21

MCP2: metacarpi 2MCP5: metacarpi 5RSP: radial styloid processusUSP: ulnar styloid processus


Motti Ader 2016


Technical markers on the device

Anatomical markers on the user’s hand

Ulnar deviation(α < 0)

Dorsal view of the hand

Radial deviation(α > 0)

Touchscreen(frontal view)


Touchscreen(side view)

Technical markers on the device

Anatomical markers on the user’s hand

Flexion(α < 0)

Medio-lateral view of the handExtension

(α > 0)

Data analysis

2 devices x 2 modalities x 25 targets= 100 gestures per participant, per iteration

=> 4,500 gestures from older participants (3 iterations)

=> 7,500 gestures from younger participants (5 iterations)

Measures• Performances: time and number of errors (per target)

• Movements of the wrist: angular deviations


Results: Finger interaction on tablet

• Posture: radial deviation and extended

25

Radial deviation(93% of the time)

Extension(68% of the time)

Adults

Olderadults

0° to 36° 33°

-8° to 50° 56°

-5° to 19° 23°

-6° to 33° 39°


26

Ulnar => Radial deviation (Younger adults)

Ulnar => Radial deviation (Older adults)



27

Flexion => Extension angles (Younger adults)

Flexion => Extension angles (Older adults)



Results: performances

28

2.7 (1.3)

Time (s)

0.2 (0.3)

4.4 (1.5) 0.4 (0.9)

Number of errors

Adults

Olderadults


Results: performances

29

2.7 (1.3)

Time (s)

0.2 (0.3)

4.4 (1.5) 0.4 (0.9)

Number of errors

Adults

Olderadults

***

** **

** **

*** strong positive correlation ** moderate positive correlation (Spearman)


Discussion: older users

• Study of the relationship between movements and performances

• Wrist: amplitude and radial deviation• Risk of discomfort could explain longer times and

increased number of errors• Differents strategies of movements

• Priority to mobilization of distal joints (Hsiao & Cho 2012)

• Difficulty of postural stabilization (Wright et al. 2012)


Discussion: younger adults

• Joint angles close to neutral• Increased mobilisation and dexterity of fingers• Better performances may be related to comfort of use

• Future work• Stabilisation of the wrist: compensatory movements

(elbow, shoulder, trunk)• Ergonomics guidelines

• Other joints and postures• Different settings


Discussion: limitations

• Older adults: heterogeneity ⇔ variability• Diversity among users

• Discomfort• Perceived individually, further evaluations:

• Assessment of individual joint range of motion• Physiological measures (e.g. muscle activity)

• Acceptance of experiment protocol• Duration and iterations• Accessibility• Equipment


Conclusion

• Study of differences in performances through biomechanical analysis

• Contribution: demonstrate differences in movements• Increased amplitude of movements associated to risk

of discomfort and related to lower performances for older adults

• Wrist (younger vs. older)• Further studies: compensatory movements, individual

discomfort, ergonomics guidelines• Perspectives: new approach to evaluate and design

devices and interactive systems


• Accessibility and acceptance of the protocol• Variability among participants• Data collection (synchronisation)• Defining measures and parameters• Individual skills vs. collective measures• Laboratory settings vs. real context

34

HCI Challenges in Human Movement Analysis:Touchscreen interaction for older adults


Impact

Human movement analysis:

improving ergonomics and usability of technologies for today's and future generation of users


References

M. Bachynskyi, G. Palmas, A. Oulasvirta, J. Steimle, and T. Weinkauf, “Performance and Ergonomics of Touch Surfaces: A Comparative Study Using Biomechanical Simulation,” in Proceedings of ACM CHI’15, 2015, pp. 1817–1826.

B. Bossavit and A. Pina, “Designing educational tools, based on body interaction, for children with special needs who present different motor skills,” Proc. - 2014 Int. Conf. Interact. Technol. Games, iTAG 2014, pp. 63–70, 2014.

B. Bossavit, A. Marzo, O. Ardaiz, L. Diaz De Cerio, and A. Pina, “Design choices and their implications for 3D Mid-Air Manipulation Techniques,” Presence, vol. 23, no. 4, pp. 377–392, 2014.

M. McCarthy et al., “The REACHES Study: Lessons Learned From Conducting Remote Assessments within a Nursing Home Environment,” no. March, 2018.

B. R. Greene, S. J. Redmond, and B. Caulfield, “Fall risk assessment through automatic combination of clinical fall risk factors and body-worn sensor data,” IEEE J. Biomed. Heal. Informatics, vol. PP, no. 99, p. 1, 2016.

E. Smith, L. Walsh, J. Doyle, B. Greene, and C. Blake, “The reliability of the quantitative timed up and go test (QTUG) measured over five consecutive days under single and dual-task conditions in community dwelling older adults,” Gait Posture, vol. 43, pp. 239–244, 2016.


References

R. Grant and N. Birch, “Otago strength and balance training exercise programme. An information guide for patients,” pp. 1–27, 2015.

S. Mallenius, M. Rossi, and V. Tuunainen, “Factors affecting the adoption and use of mobile devices and services by elderly people–results from a pilot study,” in 6th Annual Global Mobility Roundtable, 2007.

Y. Barnard, M. D. Bradley, F. Hodgson, and A. D. Lloyd, “Learning to use new technologies by older adults: Perceived difficulties, experimentation behaviour and usability,” Comput. Human Behav., vol. 29, no. 4, pp. 1715–1724, Jul. 2013.

N. Caprani, N. O’Connor, and C. Gurrin, “Touch screens for the older user,” in Assistive Technologies, InTech, 2012, no. 1, pp. 95–118.

R. P. M. Fortes, G. A. Martins, and P. C. Castro, “A Review of Senescent’s Motivation in the Use of Tactile Devices,” Procedia Comput. Sci., vol. 67, no. Dsai, pp. 376–387, 2015.

A. D. Fisk, W. a Rogers, N. Charness, S. J. Czaja, and J. Sharit, Designing for older adults: Principles and creative human factors approaches. 2009.


References

G. Lepicard and N. Vigouroux, “Comparison between single-touch and multi-touch interaction for older people,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2012, vol. 7382 LNCS, no. PART 1, pp. 658–665.

L. G. Motti, N. Vigouroux, and P. Gorce, “Interaction Techniques for Older Adults Using Touchscreen Devices: A Literature Review from 2000 to 2013,” J. Interact. Pers., vol. (in review, p. 125:125–125:134, 2013.

L. Findlater, J. Froehlich, K. Fattal, J. O. Wobbrock, and T. Dastyar, “Age - Related Differences in Performance with Touchscreens Compared to Traditional Mouse Input,” in Proceedings of the SIGCHI Conference CHI’13, 2013, pp. 343–346.

M. E. Sesto, C. B. Irwin, K. B. Chen, A. O. Chourasia, and D. A. Wiegmann, “Effect of touch screen button size and spacing on touch characteristics of users with and without disabilities,” Hum. Factors, vol. 54, no. 3, pp. 425–436, 2012.

C. B. Irwin, T. Y. Yen, R. H. Meyer, G. C. Vanderheiden, D. P. Kelso, and M. E. Sesto, “Use of force plate instrumentation to assess kinetic variables during touch screen use,” Univers. Access Inf. Soc., vol. 10, no. 4, pp. 453–460, Jan. 2011.

C. B. Irwin and M. E. Sesto, “Performance and touch characteristics of disabled and non-disabled participants during a reciprocal tapping task using touch screen technology,” Appl. Ergon., vol. 43, no. 6, pp. 1038–43, Nov. 2012.


References

J. H. Kim, L. Aulck, M. C. Bartha, C. a. Harper, and P. W. Johnson, “Differences in typing forces, muscle activity, comfort, and typing performance among virtual, notebook, and desktop keyboards,” Appl. Ergon., vol. 45, no. 6, pp. 1406–1413, 2014.

J. Jacquier-Bret, P. Gorce, L. G. Motti, and N. Vigouroux, “Biomechanical analysis of interaction strategies using touchscreen: preliminary study,” in Computer Methods in Biomechanics and Biomedical Engineering, 2014, vol. Vol. Suppl, pp. 86–87.

J. Jacquier-bret, P. Gorce, G. M. Lilian, and N. Vigouroux, “Biomechanical analysis of upper limb during the use of touch screen : motion strategies identification,” Ergonomics, vol. 0139, no. April, pp. 1–8, 2017.

A. Pereira, T. Miller, Y.-M. Huang, D. Odell, and D. Rempel, “Holding A Tablet Computer With One Hand,” Proc. Hum. Factors Ergon. Soc. 57th Annu. Meet., vol. 57, no. Straker 2008, pp. 1634–1638, 2013.

L. G. Motti Ader, “Study of the interaction of older adults with touchscreen,” Universtié Toulouse 3 Paul Sabatier, 2016.

L. G. Motti Ader, N. Vigouroux, and P. Gorce, “Biomechanical analysis of the user’s movements during tactile interaction: Postures of older aged users’ wrists,” in ACM International Conference Proceeding Series, 2017, vol. Part F1350.


References

G. Shin and X. Zhu, “User discomfort , work posture and muscle activity while using a touchscreen in a desktop PC setting,” Ergonomics, vol. 54, no. 8, pp. 733–744, 2011.

A. Werth and K. Babski-Reeves, “Effects of portable computing devices on posture, muscle activation levels and efficiency,” Appl. Ergon., vol. 45, no. 6, pp. 1603–1609, 2014.

J. G. Young, J.-H. Lin, C.-C. Chang, and R. W. McGorry, “The natural angle between the hand and handle and the effect of handle orientation on wrist radial/ulnar deviation during maximal push exertions.,” Ergonomics, vol. 56, no. March 2015, pp. 682–691, 2013.

J. G. Young, M. B. Trudeau, D. Odell, K. Marinelli, and J. T. Dennerlein, “Wrist and shoulder posture and muscle activity during touch-screen tablet use: Effects of usage configuration, tablet type, and interacting hand,” Work, vol. 45, no. 1, pp. 59–71, 2013.

L.-P. Hsiao and C.-Y. Cho, “The effect of aging on muscle activation and postural control pattern for young and older computer users,” Appl. Ergon., vol. 43, no. 5, pp. 926–932, 2012.

M. L. Wright, D. E. Adamo, and S. H. Brown, “Age-related declines in the detection of passive wrist movement,” Neurosci. Lett., vol. 500, no. 2, pp. 108–112, 2011.


hci challenges in human movement analysis

Documents