Slide 1

Making a Computer Speak Algebra However You WantCEC 2013 Convention & ExpoSAN ANTONIO, TEXASApril 5, 20131

12Presented bySusan A. Osterhaus Texas School for the Blind and Visually ImpairedOutreach Programs1100 West 45th StreetAustin, TX 78756susanosterhaus@tsbvi.edu www.tsbvi.edu/math/ Presentation OverviewProject Motivation, Goals, and BackgroundDevelopment to date: discussion and demonstrationResearch StudiesNext Steps33Project MotivationText-to-speech (TTS) vs. Math-to-speechLimited choice of syntax/semanticsLimited screen-reader supportLimited or no within-expression navigationDifficult to authorWorks in limited environmentsETS Experience4ETS Voiced GRE4Project GoalsHigh school algebraClassroom-like synthesized speech style: ClearSpeakSpeech is flexible so computer will speak the way the teacher wantsCan be used in any math document including both instruction and assessmentInteractive navigationAuthor in Word with easy playback

55Not Part of ProjectBrailleSupport for other formatsAuthoring by people with visual impairmentSolving math problemsTutoring/CurriculumIntegration with standardized testing

6other formats = large print, pdf, ebookssolving math problems = students showing their work, working through solutions, etc. tutoring/curriculum: not supporting online tutoring or curriculum enhancements, etc. 6What is MathML?Mathematical Markup Language (MathML) is an application of XML for describing mathematical notations and capturing both its structure and content. It aims at integrating mathematical formulae into World Wide Web pages and other documents. It is a recommendation of the W3C math working group. From Wikipedia, the free encyclopedia

MathML = Math Accessibility

7Behind the Scene: MathML (What is MathML?)8 x 3

8Project Software for MathMathML Tools:Authoring: MathType+WordAudio rendering: MathPlayerScreen readerWindow-Eyes (Word & IE)9MathType is common math editor in Word and elsewhere; MathPlayer is free plug-in for IE and, coming in the future with other browsers and WordFor the project, GW Micro is part of the grant but other AT vendors will support9Many ways to Speak MathHow do you say10

talk about the options for speaking these to get audience to think about how they might say themRe x^(i+1) how to differentiate from x^i + 110Many ways to Speak MathHow do you say11

talk about the options for speaking these to get audience to think about how they might say themRe x^(i+1) how to differentiate from x^i + 111Many ways to Speak MathHow do you say12

talk about the options for speaking these to get audience to think about how they might say themRe x^(i+1) how to differentiate from x^i + 112Many ways to Speak MathHow do you say13

talk about the options for speaking these to get audience to think about how they might say themRe x^(i+1) how to differentiate from x^i + 113Many ways to Speak MathHow do you say14

talk about the options for speaking these to get audience to think about how they might say themRe x^(i+1) how to differentiate from x^i + 114Many ways to Speak MathHow do you say15

talk about the options for speaking these to get audience to think about how they might say themRe x^(i+1) how to differentiate from x^i + 115How Math Speech WorksRulesPreferencesExact Speech16Rules try to get it right,preferences allow authors to choose: eg, x to the i vs x to the i powerExact speech for very special cases16ClearSpeakMany ways to speak mathClearSpeak design philosophyOther design philosophiesMathSpeakSimpleSpeech

17ClearSpeak: use common ways of speaking math with pauses or words to disambiguateEx: the fraction with numerator 1 and denominator x+1Mention MathSpeak (follows braille), SimpleSpeech (make simple things simple)

17DemonstrationAuthoringUsing MathType in Word to enter math and use preferencesEntering exact speech in WordPlaybackWord with Window-Eyes18continue the n+1/n+2 exampleShorten slide to make more basic/genericDo demonstration that Lois will create for me.18Project Goals RecapClearSpeakAuthor in WordAllow setting preference and exact speechPlayback in Word and IEInteractive navigation1919Creating Expressions and Questions for StudiesLevelDegree of complexityRelevance of speech variationsProvide useful dataTest comprehension, not computationGuide decisions about speech rules, preferences

18Typical of expressions found in high school algebra curricula, targeting second half of courseOf sufficient complexity to offer areas of potential ambiguity that a successful speech style should resolve for the listener; that is, not be so simple that students would instantly understand them, no matter how they were spokenOf sufficient simplicity that excessive computation is not required in order to answer the math questions posed about the statement.Speaks significantly differently in at least two of the three speech styles being tested (ClearSpeak (the style we are developing) vs. the existing MathSpeak and Simple Speech styles).Lends itself to math questions that will be diagnostic of aspects in which the speech style succeeds or fails to make a math expression clear, as opposed to those that assess whether the student can do the mathematics required to work with the expression. For example, if a speech style makes the start or end of a fraction clear, a student is more likely to correctly identify the denominator of the fraction than if the style fails to make those boundaries clear.

20Feedback Study #1Compared ClearSpeak, MathSpeak, SimpleSpeechFractions, exponents, parenthesesFocus: was expression understood?Research/Statistical consultation16 HS students: blind or low vision

21completed16 high school students with blindness or low vision, currently enrolled in Algebra 1 or aboveSome students had additional disabilities, including TBI, auditory processing, autism spectrum, other physical disabilitiesLois

15 completed the study; 1 partially

Thanks to Nan Kong and Eric Hansen for experiment design and data analysis

21Results Summary ClearSpeak more understandable than Simple Speech or MathSpeakStudents overall preferred ClearSpeak Students got more answers correct with ClearSpeak than with Simple Speech or MathSpeak22ClearSpeak student comments:

[Liked]: Fraction, numerator, denominator, and raised to the power of.The phrase raised to the power could just be to the power, raised to the power 2 could just be squared, and the phrase the fraction with numerator and denominator could be made with less words. [power 2 was on purpose because of the nature of the expression, for pedagogical purposes; i.e. idea is to divide and subtract exponents, so using 2 rather than squared desirable]

I would take the times out.Simple Speech: I liked that it used squared and to the to refer to exponents; The way it reads negative exponents should be change (for example, to the negative 3rd instead of super negative 3 end super)I liked that it was organized so I could hear all the parts clearly.I couldnt understand the statement.; It said super for all the exponents and it sounded like it skipped part of it.I wouldnt say superscript. I would say exponent.I would take out the superscript for the exponents. I would put a pause between the end of the exponent and the beginning of the next variable. Use exponent.MathSpeak:I liked that it used cubed to refer to the third power; the superscript and baseline for exponents should be changedI liked that it was clear about what fraction was supposed to be which, and that it was organized in a clear order.I think taking out some of the things like the level deep and if not need the baseline, these made it a really long and confusing read.I would insert multiply or times after baseline and before next variable or I would pause between them; otherwise, it was too fast.Dont like dot. Prefer multiply by because it gives me time.I still dont understand what nested one deep and superscript mean. I think I may understand that superscript means to the power of something and subscript is to the base of something. Like x^7, the 7 is the superscript and x7, the 7 is the subscript. Teachers note: The student then erased that, and rewrote: 5x7

22One Expressions Results23P+Very FamiliarSomewhat FamiliarSomewhat UnfamiliarVery UnfamiliarSimple Speech30817MathSpeak31546ClearSpeak*66810

Essential Differences:Simple Speech: fraction ... x to the ninth...z super negative 7 end super... over...y squared...end fractionMathSpeak: start fraction nested one deep x superscript 9 baseline...z superscript minus 7 over ... end fraction nested one deepClearSpeak: the fraction with numerator x raised to the power 9 times... and denominator y raised to the power 2...

Results (from Nan Kong & Eric Hansen)Students got the ClearSpeak versions right significantly more often (95% confidence level) than the other speech styles.

ClearSpeak student comments:

[Liked]: Fraction, numerator, denominator, and raised to the power of.The phrase raised to the power could just be to the power, raised to the power 2 could just be squared, and the phrase the fraction with numerator and denominator could be made with less words. [power 2 was on purpose because of the nature of the expression, for pedagogical purposes; i.e. idea is to divide and subtract exponents, so using 2 rather than squared desirable]But note nested bug in MathSpeak, and that turned most students off, so MathSpeak would likely have fared better without the bug

I would take the times out.Simple Speech: I liked that it used squared and to the to refer to exponents; The way it reads negative exponents should be change (for example, to the negative 3rd instead of super negative 3 end super)I liked that it was organized so I could hear all the parts clearly.I couldnt understand the statement.; It said super for all the exponents and it sounded like it skipped part of it.I wouldnt say superscript. I would say exponent.I would take out the superscript for the exponents. I would put a pause between the end of the exponent and the beginning of the next variable. Use exponent.MathSpeak:I liked that it used cubed to refer to the third power; the superscript and baseline for exponents should be changedI liked that it was clear about what fraction was supposed to be which, and that it was organized in a clear order.I think taking out some of the things like the level deep and if not need the baseline, these made it a really long and confusing read.I would insert multiply or times after baseline and before next variable or I would pause between them; otherwise, it was too fast.Dont like dot. Prefer multiply by because it gives me time.I still dont understand what nested one deep and superscript mean. I think I may understand that superscript means to the power of something and subscript is to the base of something. Like x^7, the 7 is the superscript and x7, the 7 is the subscript. Teachers note: The student then erased that, and rewrote: 5x7

2324ClearSpeakSimpleSpeakMathSpeakVery familiar48%24%24%Somewhat familiar50%50%31%Not very familiar2%9%23%Very unfamiliar0%16%22%Very easy to understand52%17%16%Somewhat easy to understand33%37%38%Somewhat hard to understand13%24%23%Very hard to understand3%22%22%Very sure understood71%42%39%Somewhat sure understood24%37%33%Not sure understood5%7%11%Definitely did not understand0%14%17%Familiarity and Understanding24Feedback Study #2Can Prosody help understanding?PausesPitchRate ChangesVolume ChangesRejected pitch and volume changesnon-speech sounds not an optionFocus on pauses, rate-change vs. start/end

2525Feedback #2 Example vs.

With Pauses / rate changeExpression 1Expression 2With end rootExpression 1Expression 2

26

26Feedback #2 Another ExampleNested Parentheses:

Uniform PausesNon-uniform pausesNon-uniform pauses and First Paren 27

Talk about problems with parens and a couple of solutions27NavigationSimple char by char mode for beginnersPowerful navigation features for expertsReadDescribeMultiple ways of movingTreeCharacter, Placemarker, SemanticAfter tree move: read or describe10 Placemarkers, 2 Cursors, Where am I?

28

28Participation OpportunitiesBlind or Visually Impaired StudentsTeachers2929Contact InformationLois Frankel: lfrankel@ets.orgNeil Soiffer: neils@dessci.comBeth Brownstein: bbrownstein@ets.orgSusan Osterhaus: susanosterhaus@tsbvi.edu MathPlayer: http://www.dessci.com/en/products/mathplayer/

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R324A110355 to the Educational Testing Service. The opinions expressed are those of the authors and do not represent views of the Institute or the U.S. Department of Education.

30

Thank you for attending this session.31LF: Will we use handouts? Can adapt ones used for AER and/or project meeting if so; also we need to get a fairness review before uploading the ppt. 31