Math-Puzzle:Math-Puzzle:

Equation Tutor for Sighted and Equation Tutor for Sighted and

Visually-Impaired ChildrenVisually-Impaired ChildrenJarno Jokinen

Department of Computer Sciences

University of Tampere Finland

jarno.jokinen@uta.fi

April, 2005

AAFG 2005

Math-Puzzle

J. A. Jokinen 2 / 15 18. 04. 2005

• communication of mathematics is usually visual

• formulas, diagrams, graphs etc.

• it is very difficult for blind and partially sighted people /

students to do mathematics and is one of the biggest

obstacles for them in school and at the university

introduction

Math-Puzzle

J. A. Jokinen 3 / 15 18. 04. 2005

most of the work reported in developing techniques that deal with

mathematics can be presented through next categories:

• tactile as in Braille and other raised representations

• audio aids that read equations with tools to help in reading process

• tonal representing equations and graphs (sonification / audification)

• haptic or forced feedback devices represent shapes and curves

• integrated /multimodal approaches

Introduction into the games

Math-Puzzle

J. A. Jokinen 4 / 15 18. 04. 2005

http://www.boowakwala.com/kids/math-game-kids.html

http://www.learn4good.com/games/kids/double_digits.htm

Math-Puzzle

J. A. Jokinen 5 / 15 18. 04. 2005

http://www.gamealbum.com/keyword/math/

Math-Puzzle

J. A. Jokinen 6 / 15 18. 04. 2005

project impetus

• created a game instead of making tests with an existing game

• mathematical game for blind children

• after different ideas, Math-Puzzle was chosen

• training in logics, memory and equations’ manipulation strategy

the problems in question

• is it possible to solve equations using only short speech cues?

• what are the limitations?

• what is the easy way for blind interaction to edit the equations in

static or dynamical puzzle? (memory capacity or external memory aid

see next slide)

• how long does it take to solve a puzzle?

• what are the parameters of the gameplay progress and player

performance?

• how much does it make difference when player has visual feedback

or all the tasks (including navigation within the game-field) are

presented through sounds?

• how and in which order the equations could be solved? (strategies)

Math-Puzzle

J. A. Jokinen 7 / 15 18. 04. 2005

Math-Puzzle

J. A. Jokinen 8 / 15 18. 04. 2005

different matrix approaches

• static puzzle on the left

• dynamic puzzle on the right

game concept

• 5x5 matrix with one equation in each row

• equations are predefined and randomly picked for the matrix

• the goal of the game is to change the places of the equation members

so that all of the equations are true

• the figures (only in the same column) can be swapped by clicking one

number and then the other one

• operators cannot be swapped as well

other controls are:

• alt+R -> new game (reset)

• alt+M -> minimize / maximize browser window

• alt+S -> show / hide figures and operators

• the puzzle completion time and the number of moves are calculated

• the game is currently implemented only in www with limitations

Math-Puzzle

J. A. Jokinen 9 / 15 18. 04. 2005

testing procedure

• 5 technically aware adults (age ranged from 26 to 34)

• 10 games per player with all three playing modes:

1st visual, 2nd blind (hidden labels with sound cues), 3rd blindfolded

• laptop pc with external mouse and headphones

Math-Puzzle

J. A. Jokinen 10 / 15 18. 04. 2005

problems

• sound feedback is not supported with

mozilla browsers

• completing the fourth equation

usually completes also the fifth

• if not, the answers are crossing each

other in a way that makes it very

difficult to solve the puzzle

• the program sometimes gives 1 or

more correct equations at the start

which may also result in an error on

the browser

Math-Puzzle

J. A. Jokinen 11 / 15 18. 04. 2005

results and discussions

• the game (blind mode) is hard for adults that can see

• what about kids that can’t? They have better spatial understanding

• some matrixes can be a lot faster solved than others – repetitions

needed for a good estimate of skills

• if you wan’t to hear the sound cues you need to be patient

• sound is heard when the mouse is moved over the square, so if you

want to hear it again you must move the mouse away and over again

• completing the fourth equation is crucial as explained on next slide

Math-Puzzle

J. A. Jokinen 12 / 15 18. 04. 2005

about the fourth equation

Math-Puzzle

J. A. Jokinen 13 / 15 18. 04. 2005

OK

OK

OK

OK

conclusions

• game requires a lot of memorizing, but there are strategies that help

• completing equations in some order

• using mathematical rules (division and multiplication)

• when playing the game blindfolded, the increase in speed was bigger

than when playing the visual version

Math-Puzzle

J. A. Jokinen 14 / 15 18. 04. 2005

Math-Puzzle

J. A. Jokinen 15 / 15 18. 04. 2005

References

http://www.boowakwala.com/kids/math-game-kids.html http://www.learn4good.com/games/kids/double_digits.htm http://www.gamealbum.com/keyword/math/Children’s math project http://www.udel.edu/educ/cmp2/http://www.educational-software-directory.net/math/ Lambda-project: Linear Access to Mathematic for Braille Device and Audio-synthesis http://www.lambdaproject.org/Karshmer, A.I., Gupta, G., Gillan, D. Architecting an Auditory Browser for Navigating Mathematical Expressions, ICCHP 2002, LNCS 2398, p. 477

http://link.springer.de/link/service/series/0558/papers/2398/23980477.pdfGaura, P. REMathEx - Reader and Editor of the Mathematical Expressions for Blind Students, 2002, LNCS 2398, p. 486, http://link.springer-ny.com/link/service/series/0558/papers/2398/23980486.pdfFitzpatrick D. Speaking Technical Documents: Using Prosody to Convey Textual and Mathematical Material, ICCHP 2002, LNCS 2398, p. 494, http://link.springer.de/link/service/series/0558/papers/2398/23980494.pdf

http://www.computing.dcu.ie/~dfitzpat/publications.htmlMath project, http://www.cs.york.ac.uk/maths/index.htmlProsody in Mathtalk http://www.cs.york.ac.uk/maths/robert/prosody.htmlMathematical Access for Technology and Science, http://www.papenmeier.de/reha/research/mathe.htmEdwards, A. D. N., Stevens, R. D. and Pitt, I. J. Représentation non visuelle des mathématiques, (translated by A. Assimacopoulos) in A. B. Safran and A. Assimacopoulos (editors) Le Déficit Visuel, Éditions Masson, pp. 169–178 (1995), http://www.cs.york.ac.uk/ftpdir/pub/alistair/publications/ps/geneva.psKarshmer, A.I., Gupta, G., Geiger, S., and Weaver, C.: Reading and Writing Mathematics: The MAVIS Project, BIT (Behaviour & Information Technology), January 1999