30034795

8/7/2019 30034795

1/32

A Meta-Analysis of the Effects of Calculators on Students' Achievement and Attitude Levelsin Precollege Mathematics ClassesAuthor(s): Aimee J. EllingtonSource: Journal for Research in Mathematics Education, Vol. 34, No. 5 (Nov., 2003), pp. 433-463Published by: National Council of Teachers of MathematicsStable URL: http://www.jstor.org/stable/30034795 .Accessed: 09/04/2011 13:14

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless

you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you

may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at .http://www.jstor.org/action/showPublisher?publisherCode=nctm. .

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed

page of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms

of scholarship. For more information about JSTOR, please contact [email protected].

National Council of Teachers of Mathematics is collaborating with JSTOR to digitize, preserve and extend

access toJournal for Research in Mathematics Education.

http://www.jstor.org
http://www.jstor.org/action/showPublisher?publisherCode=nctmhttp://www.jstor.org/stable/30034795?origin=JSTOR-pdfhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/action/showPublisher?publisherCode=nctmhttp://www.jstor.org/action/showPublisher?publisherCode=nctmhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/stable/30034795?origin=JSTOR-pdfhttp://www.jstor.org/action/showPublisher?publisherCode=nctm

8/7/2019 30034795

2/32

Journal orResearch n MathematicsEducation2003, Vol 34, No. 5, 433-463

A Meta-Analysis of the Effects ofCalculators on Students' Achievementand Attitude Levels in PrecollegeMathematics Classes

Aimee J. Ellington,Virginia Commonwealth University

Thefindings f 54 researchtudieswere ntegratedhroughmeta-analysiso deter-mine heeffectsof calculatorsn student chievementndattitudeevels.Effect izesweregeneratedhrough lassianechniquesfmeta-analysis,ndHedges ndOlkin's(1985) nferentialtatisticalmethodswereused o test hesignificancef effectsizedata.Results evealed hatstudents' perationalkillsandproblem-solvingkillsimproved hen alculatorserean ntegralart ftesting nd nstruction.heresultsforboth kill ypesweremixedwhencalculators erenotpart f assessment,ut nall cases,calculator se did not hinder hedevelopment f mathematicalkills.Studentssing alculatorsadbetter ttitudesowardmathematicshan heirnoncal-culatorounterparts.urtheresearchs neededntheretentionf mathematicskillsafter nstructionnd ransferf skills oothermathematics-relatedubjects.Keywords:Achievement;Attitudes;Calculators;Meta-analysis;Statisticalpower/effectize

Over the last century, pedagogical methods in mathematics have been in agradual etconstant tateof change.Oneinstigator f change n mathematics lass-roomshas beentechnology.Kaput 1992) described he role of technology n math-ematics educationas "a newly active volcano-the mathematicalmountain ..changingbeforeoureyes, withmyriad orces operatingon it and within it simul-taneously" p.515). Technologyandthepedagogical hangesresulting rom thavea decisive impacton what is included nthemathematics urriculum. nparticular,whatstudentsaretaughtandhowtheylearnaresignificantlynfluencedbythe tech-nological forces at work on and within "the mathematicalmountain."The situa-tion is compounded by the fact that technology is evolving at a rapid pace.Mathematics ducatorshavethearduous ask of keepingupwith the advancesandincorporatinghem in lessons and activities.Althoughthis is noteasy to do, mosteducators odaycannotimaginea classroomwithouttechnology.The calculator s a technologicalforce that has been a catalystfor lively debatewithin the mathematics educationcommunity duringthe last 30 years. In the1970s, the educationalrelevanceof the calculatorwas a controversial opic.More

Thisarticle s basedon the author'sdoctoraldissertationompleted t theUniversityfTennessee nderhedirection f Donald .Dessart.

8/7/2019 30034795

3/32

434 Effects fCalculators: Meta-Analysis

recently, calculators have become commonplace, and discussion has focusedaroundways to helpstudentsachieve maximumbenefits fromthe use of thistech-nology. The highlightsof the debate have been outlined in a series of reviews ofcalculatoruse research,which I summarizenext.The Calculator nformationCenter(CIC)at Ohio StateUniversityandseveralindependentreviewers(Neubauer,1982; Parkhurst,1979; Rabe, 1981; Roberts,1980;Sigg, 1982)reported n concernsraisedbythe calculator's ntroduction ntothe classroom. The reviews reportedon the successes and pitfalls in the imple-mentation of calculator use in American schools (Suydam, 1978, 1979, 1980,1981, 1982), responded o criticism that the calculatornegativelyaffected resultsof standardized mathematics achievement tests (Suydam, 1979, 1980), andaddressed he possibilitythatnegativecalculatoreffects outweighedthe benefitsof calculatoruse(Sigg, 1982).Thetwomostsignificant indingswere that he calcu-latordid notnegativelyaffectstudentachievementnmathematics ndthatstudents'attitudes owardmathematicswere not influenced n apositiveornegative way bycalculatoruse. The lack of availabilityof calculatorresearch or educators n thefield (Sigg, 1982)and the inadequateuse of calculators n theassessmentprocess(Roberts,1980;Sigg, 1982) were two issues of concern raisedby theresearchers.The tone of the debate shifted in the late 1980s with the introductionof thegraphingcalculator.When the discussion focused on how to incorporate alcula-tors nthe most effectivemanner,he NationalCouncilof Teachersof Mathematics(NCTM, 1989) gave the graphingcalculatorcredit for "theemergenceof a newclassroomdynamic nwhichteachersandstudentsbecome natural artnersndevel-oping mathematical deas andsolving mathematicalproblems" p. 128).Reviewspublishedduring his timereportedmixed results forcalculatoruse, withpositiveresultsbecomingmoreprevalentas timepassed, particularlyor thedevelopmentof problem-solving kills (Gilchrist, 1993).Graphing echnologywas determinedto be thecentralreason for student mprovement n three areas:understanding fgraphicalconcepts,theabilityto makemeaningfulconnectionsbetween functionsand theirgraphs,and enhancedspatialvisualization skills (Penglase & Arnold,1996).Thefindingsrelating o students'achievement nmathematicswere incon-clusive due to theprevalentuse of skill-basedtesting procedures Gilchrist,1993;Penglase& Arnold, 1996).Inroughly he same timeframeas coveredbythe calculator eviewssummarizedabove, Hembreeand Dessart(1986, 1992) statistically ntegrateda set of quanti-tative calculator studies in a comprehensivereview throughmeta-analysis.Theresultswere mostsignificantfor calculatoruse in Grades3 through9. Eachstudyincluded nthemeta-analysisnvolved statistical omparisons f studentswho usedcalculatorswith studentswho studied he samemathematicalmaterialbut withoutthe use of calculators.Two important indings were (a) the calculatorhad nosignificanteffect on students'conceptualknowledgeof mathematicsand(b) thecalculatorhada positive influence on students'attitudes owardmathematics.For computationaland problem-solvingskills, Hembree and Dessart (1986)separatedhestudiesbasedon mode of testingandanalyzedeachgroupseparately.

8/7/2019 30034795

4/32

Aimee .Ellington 435

When calculators were partof the assessment process, the computationalandproblem-solving skills of students of low or average ability improved. Whenstudentsin experimentalgroupswere not allowed access to calculatorsduringtesting, averagestudentswho used calculatorsduring nstructionmproved nboththeircomputational ndproblem-solving kills. Theonly exceptionwas thefourthgradewherecalculatorshad a negativeeffect oncomputational kills.Overall, heresultswereencouragingor the role of calculators n mathematics lassrooms.Thenegativeresult in Grade4 was a reminder o educators hat "calculators, houghgenerallybeneficial,maynotbe appropriateor use at all times, in all places, andfor all subjectmatters" Hembree& Dessart, 1992,p. 25).The numberof classroomsnotincorporatingalculatorswithinthe mathematicscurriculum as diminishedsignificantly n the last few years,andyet the concernsover calculators re stillprevalent.Onsucceedingpagesof theMay/June1999 ssueof MathematicsEducationDialogues, Ralston(1999) encouragedthe completeabolishment of paper-and-pencil omputations(p. 2), whereas Mackey (1999)recommended he use of calculatorsbe extremelylimited (p. 3). Otherevidencesuggested that educatorswere most comfortable with the middle ground.Forexample,in the sameeditionof MathematicsEducationDialogues results from asurveyrevealed hatmost educatorsbelieved "calculators houldbe usedonlyafterstudentshad learnedhow to do the relevantmathematicswithout hem" Ballheim,1999,p. 6).To investigate urther he effects of calculators, designedandconducteda meta-analysisfor three reasons.First,the literature urrently ontains over 120studiesfeaturinga single aspectof this technologicalforce-the effects of calculatoruseon students n mathematics lassrooms.Recent calculatorreviewsfeaturing omeof these studies(Gilchrist,1993;Penglase& Arnold,1996)did notemployinfer-ential methods of evaluation.Thus, a statisticalanalysis of studies conductedduring he last 15yearswas warranted. econd,the calculatorcontroversyhas notbeen resolved in the yearssince the Hembreeand Dessartmeta-analysisappearedinprint,suggesting hatresearchnthis areamustcontinue.Third, he mathematicsclassroom has experienceda variety of changes since the mid-1980s includingsignificantadvances ntechnology,such as the introduction f thegraphing alcu-lator,an increase in the level of technological sophisticationof the mathematicseducationpopulation,and documentedencouragementby organizations ike theNCTM forexploringthe pedagogicaluses of calculators n classrooms.Statistical ntegrationof resultsfrom the bodyof studies conductedduring histime periodis an appropriateway to assess the calculator's mpacton students nthe modernclassroom.This articleaddresses he concernsexpressedby educatorsduring he last3 decadesthroughan examinationandsynthesisof resultsprovidedby a set of calculator-basedresearch studies featuringprecollege mathematicsstudents.Inparticular,he analysiscovers the calculator's nfluenceon students'performance n the following areas:(a) operational,computational,andconcep-tual skills; and (b) general problem-solving skills including two aspects: thenumberof problemsattemptedas the resultof havingaccess to a calculatorduring

8/7/2019 30034795

5/32


instructionand theabilityto select theappropriate roblem-solving trategy.Thismeta-analysis lso considers hecalculator's oleinthedevelopment f studentatti-tudes towardmathematics.

METHODThis study followed the proceduresoutlined by Lipsey and Wilson (2001).Othermeta-analytical echniquesestablishedby expertsin the field (e.g., Cooper& Hedges, 1994; Hedges & Olkin, 1985; Hedges, Shymansky,& Woodworth,1989) were also incorporated s necessary. In the next sections of this article,I

present nformationon variousaspectsof themeta-analysis.ConstructsandDesigns in Calculator-UseResearch Studies

Reviews of calculator-based tudies over the last 30 years revealed that mostresearch nvolving use of calculators ocused on changes to studentachievementlevels and attitudes toward mathematics.These two constructswere featured nHembree and Dessart's(1986) meta-analysis.Because there has been no changein the focus of recentresearchreports, he currentstudyfeatured he same cate-gories andsubcategoriesof achievementandattitudeas those outlined n the firstmeta-analysis.Thedefinitionsbeloworiginallyappearednthewritingsof Hembree& Dessart(1986, 1992).For the achievement onstruct sorted hedata nto threecategories:acquisition,retention,and transferof mathematicalskills. Skills acquisitionwas measuredimmediatelyaftertreatment; kills retentionwas measuredafter a predeterminedtimelapse followingtreatment; ndskills transferwas measuredby evaluating heways that studentsused the skills in othermathematics-related reas.These skillswere furthersorted into one of two subcategories,which I call CategoryI andCategoryIIandexplainbelow.Category I included skills that I identified as operational, computational,orconceptual.Operational kills were those I identified as the specific skills neces-saryto solve themathematical roblemsontests of studentachievement. f the skillwas clearly computational,then I included data from that study in a separateanalysis of computationalskills, and likewise for studies that involved under-standingof mathematical oncepts. If an authordid not provideinformation hatallowed a skill to be identified as strictly computationalor conceptual, then Iincluded the data n the operational kills category.

Category II involved a subcategoryof problem-solvingskills that were notexplicitlystatedwith the mathematical roblemsused for assessment. nstead, hesewere skills that studentsselected from their mathematical epertoire o solve theverbalproblems istedon theachievement ests.Although he numberof problemscorrect (and the number of problemspartiallycorrect when partialcredit wascounted in assessment)was coveredby the generalcategoryof problem-solvingskills,severalstudies ookedat two otheraspectsof problem olving:productivity-

8/7/2019 30034795

6/32


the numberof problems attemptedby students and selectivity-the number ofappropriate trategiesthey used. The selectivity categorywas somewhatsubjec-tive because it was based on the researcher'sopinionandexpertiseas to whetherthe strategywas appropriateo a particular ituation.The attitudeconstruct included the six attitudinal actors of the MathematicsAttitude Inventory developed throughthe MinnesotaResearch and EvaluationProject(Sandman, 1980).The factors are these:(a) attitude owardmathematics,(b) anxietytowardmathematics, c) self-conceptin mathematics, d) motivationto increasemathematicalknowledge,(e) perceptionof mathematics eachers,and(f) value of mathematics nsociety. Most attitude-relatedesults nvolvedonly thefirst factor.Studiesthateitherexplicitlycited the MathematicsAttitudeInventoryor used other available attitude measures like the scales developed by Aiken(1974) and Fennemaand Sherman 1976) providedresultsrelated o the otherfivefactors.One otherfactorthatwas included n thismeta-analysisbut thatHembreeand Dessert(1986, 1992)did not include was students'attitudes oward heuseofcalculators n mathematics.

In most studies,two groupsof studentswere taughtby equivalentmethods ofmathematicalnstructionwith the treatment roupusingcalculators ndthecontrolgroup havingno access to calculators.Several studiescompounded he situationby including pecialcurriculummaterialsdesigned or calculatoruse. In bothcases,the effects of calculatoruse were measuredby comparing he groups' responsesto posttreatmentvaluations.Theroleof the calculatornposttreatmentssessmentwas a significant actor nthemeta-analysis eportedhere When treatment roupswere not allowed access to calculatorsduring testing, the studies were used toanalyzestudentdevelopmentof mathematical killsduring he calculator reatment.Whentreatment roupshad access to calculatorsduringposttreatmentvaluations,the studies were used to evaluate the calculator'srole in the extension of studentmathematical kill abilities after treatmentwas concluded.

Identificationof Studies or theMeta-AnalysisThe initial search for studies involved a perusalof the EducationalResourcesInformationCenter ERIC)andthe DissertationAbstracts nternationalDAI)data-bases. A manual search of the Journal or Research in MathematicsEducation(JRME),School ScienceandMathematics, ndEducationalStudies n Mathematicsfrom the beginning of 1983 to March, 2002 was used to locate citations andabstracts.I paid particularattention to the annualbibliographies compiled by

Suydamandpublished nJRME rom 1983 to 1993. Whenevaluatinga studyforinclusioninthemeta-analysis, scanned heaccompanyingbibliography or otherinclusionpossibilities.Thefinal criteria or inclusion n themeta-analysiswere thatthestudywas publishedbetweenJanuary1983 and March2002; it featured heuseof a basic, scientific,or graphingcalculator; t involved students n a mainstreamK-12 classroom;and thereportof findings provideddatanecessaryforthe calcu-lationof effect sizes. In the case of missingdata,I attempted o gatherthe infor-

8/7/2019 30034795

7/32

438 Effects f Calculators: Meta-Analysis

mationfrom the authorsof the originalstudies. Forexample,one reportomittedclass samplesizes and fourreports ailed to specify whetheror not the treatmentgrouphad access to calculatorsduringposttreatment valuations.In all cases, themissing informationwas successfullyobtainedbeforedataanalysiscontinued.The relationshipbetween thecharacteristics f a studyand its resultsis crucialto meta-analysis.Therefore,quantifying he findingsandstudycharacteristics sa significantpartof dataorganization.Once all of the studies arecoded, the tech-nique of meta-analysis attempts to determine statistical similarities betweenresearch results for the variousstudy characteristics Glass, McGaw, & Smith,1981). For the meta-analysisreportedhere,characteristics f studies featured nthemeta-analysisappear n Table 1and wereconsideredas independent ariables.Althoughsome characteristics n the table areself-explanatory,othersneed elab-oration.Forthetreatmentength, estonlyrefers o caseswhere hecalculatorswerea factoronly on thetest(i.e., availableor notavailableforstudents o use) withoutan instructional omponentbeforehand.Withregard o curriculum, pecial mate-rials are those designedfor instructionwithcalculatorsas opposedto traditionalmaterials used by both treatmentand controlgroups. Pedagogical use referstousing thecalculatoras an essential element in the teachingandlearningof math-ematics;functionaluse meansthat t was usedonly in activities such as computa-tion, drillandpractice,andchecking paper-and-pencilwork.

Table 1Characteristicsof Studies Featured n theMeta-AnalysisCharacteristic

Publicationstatus Journal,Dissertation,Otherunpublished ourceTest instrument Standardized,Nonstandardizedteachermade)Educationaldivision Elementary,MiddleSchool, HighSchoolAbilityof students Mixed, Low, HighTreatmentength Testonly; 0-3 weeks, 4-8 weeks, 9 ormore weeksCurriculum Traditional,SpecialCalculatoruse Functional,PedagogicalCalculator ype All typesallowed, Basic, Scientific,GraphingStudydesign Random,NonrandomSamplesize 1-100, 101-200, 210-1000, over 1000

Effect sizes calculated from the numericaloutcomes from achievement andattitudeassessmentsweredependentvariables.Because theauthorsof the studiesprovidedmeansandstandard eviationsorinformation romstatistical ests basedon means and standarddeviations, the effect size measure for the meta-analysis wasthe standardized mean difference-the difference in the experimental and controlgroup means divided by a pooled standard deviation (Lipsey & Wilson, 2001). A

8/7/2019 30034795

8/32


positive effect size indicated hat the experimentalgrouphada highermean thanthe controlgroupfora particular tudy,whereasanegativeeffect size impliedthatthe controlgroupperformedbetter hanthetreatment roup;an effect size of zeroindicatedthat there was no difference betweenthe treatmentand controlgroups.Given thatHedges andOlkin(1985) provedthat the raweffect size has distribu-tionbias,each raw value was corrected or thisproblemandtheresultingvalue wasused in furtheranalysis.Themagnitudeof theeffect sizes varyaccording o manydifferent actors, ncluding heresearcher'smethodsandthesubjectof the research.Mindfulof theseconsiderations,Cohen(1988) has establishedbasicguidelinesonevaluating hemagnitudeof effect sizes, withvaluesnear0.2, 0.5, and 0.8 consid-ered to be small, medium,andlarge, respectively.With two exceptions, the skill achievementor attitudedatagathered rom onearticle was used to generateone effect size for the meta-analysis.Liu (1993) andPennington 1998) studiedgroupsof students nvolvedin twodifferent reatmentswith each treatment roupbeing comparedo acontrolgroup.Onegroupwastaughtwith calculatorsby traditional nstructionmethods andtheother treatment roupwas taughtwith special calculator-related nstruction materials. Because eacharticlepresenteddata on two treatmentgroupsthat differedsignificantlyin themethod of treatment, he resultingeffect sizes were not averaged nto one value.For these two articles, the two independent groups were considered separateprimary tudies for the purposeof analysis.Data AnalysisProcedures

Hedge's Q tatisticwas used to test thehomogeneityof a groupof effect sizes.This has a chi-squaredistributionwith k - 1 degrees of freedom,where k is thenumberof effect sizes. A set of effect sizes is calledhomogeneous f each elementin the set is an estimateof thepopulation'seffect size (Hedges&Olkin, 1985)andthe variance n effect sizes is the resultof samplingerror.Because the potentialexisted for othersourcesof variability,a randomeffects model outlinedby Lipsey& Wilson (2001) was used to address he variationamongeffect sizes.For ahomogeneousset of effect sizes, thepopulation ffect size is best estimatedby aweightedmeanof unbiasedeffect sizes;therefore,a weightedmean andcorre-sponding95% confidence intervalwere generatedfor each homogeneous set ofeffect sizes. The mean effect size was determined o be statistically significant(representing significantdifferencebetween the treatment roupand the controlgroup's achievement or attitude scores) when the corresponding confidenceintervaldid not containzero.If the test forhomogeneityrevealed ignificanthetero-geneity (i.e., p < .01), outliers were removed one at a time until a nonsignificantQwas obtained.Although the traditionaldefinition of outlier is a value that issignificantly larger or smaller than the others in a data set, outliers in meta-analysiscan also resultfromsamplesize. Forexample,if an effect size was gener-ated for a study with a sample size significantly smaller thanthat in the otherstudies,then that effect size could be an outlier. In this study,I used a methodfor

8/7/2019 30034795

9/32


identifyingoutliersdevelopedby Huffcuttand Arthur 1995) thatconsiderseffectsize value andsamplesize.Finally, an analysis of independentvariableswas conductedto determinethe

effect of moderator ariableson heterogeneityof the effect size data sets andthemagnitudeof theweightedmean effect sizes. Theseanalyseswere conductedwiththeentire set of effect sizes includingthose deemed outliers at an earlierstageofanalysis.Description of the Studies

The initial searchof thebroadlydefinedcategoryof calculator-basedresearchin the K- 12 classroom uncovered86 studies for the meta-analysis.After evalu-atingthe studiesaccording o the criteria ssentialformeta-analysis e.g., presenceof a treatmentand controlgroup;datanecessaryforcalculatinganeffect size), 32studies were eliminated.The final set of 54 studies(see Appendix or thecitations)publishedbetweenJanuary1983throughMarch2002 provideddatafor 127 effectsizes. Eachstudywas classifiedaccording o the characteristics hown inTable 1,andthe distributionof the studiesaccordingto themappears n Table2. Lookingat the breakdownof studies within aparticular haracteristic,he numbersdo notsum to 54 since several studiesprovidedseparatedatafor morethan one classifi-cation(e.g.,onestudyprovideddataon bothmiddleandhighschoolstudents).Also,the curriculum ndcalculatoruse variableswere notcodedforstudies hatfeaturedonly a test and no instructionwith calculators.

Table2Distribution fStudiesFeatured n theMeta-AnalysisAccording o the Setof CharacteristicsNumber NumberCharacteristic of studies Characteristic of studiesPublication tatus CurriculumJournal 9 Traditional 41Dissertation 37 Special 6Other npublishedource 8 Calculator seTest instrument Functional 11Standardized 24 Pedagogical 36Nonstandardized 33 Calculator typeEducational ivision Alltypesallowed 4Elementarychool 9 Basic 25Middle chool 20 Scientific 3Highschool 26 Graphing 22Ability of students Study designMixed 46 Random 44

Low 2 Nonrandom 10High 7 Sample izeTreatment length 1-100 28Testonly 7 101-200 180-3 weeks 17 201-1000 44-8 weeks 9 Over1000 49 ormoreweeks 21

8/7/2019 30034795

10/32

Aimee . Ellington 441

Overall,85% of the studiesappeared ither asjournalarticles,dissertations,ormasterstheses. Of the remainingstudies,one was an unpublishedreportand theothereight were ERIC documents.A varietyof standardizedests were used toassess achievement e.g., ScholasticAptitudeTest,Iowa Test of Basic Skills), andnonstandardmethods of assessment used by some researcherswere teacher orresearcherdesignedtests.Withregard o grade evel, roughlytwo thirdsof thereports eaturedmore thanone gradelevel, andas a consequenceit was not possible to analyzethe databyindividualgrades.Instead, sorted he studies nto educationaldivisions-elemen-tary,middle,andhighschool.Theelementary radeswererepresented ythe fewestnumberof studies. Nearly 70% of the studies involved at least one of grades 8through12. Based on this distribution,nferences drawnfrom this meta-analysisare best appliedto mathematics tudents n highergrades.Thelengthof calculatorreatmentanged romtestonly(i.e., the studies nvolveda test with no instructionaluse of calculatorsbeforetesting)to 650 days(i.e., 3 1/2schoolyears).Theduration f the treatment haseexceeded 30 daysfornearly60%of the studies.Only threestudies evaluatedstudentsaftera predetermined eten-tionperiod rangingfrom 2 to 12 weeks.Randomassignmentof classes of students o the calculator reatmentwas usedin 81%of the studies. In the remainingstudiesit was either clear thatassignmentto treatmentwas not randomor thestudy designwasnot obvious fromreading hearticle.Althougha randomized, trictlycontrolledstudyis the ideal, this type ofstudy is not always possible in educationalresearch.Studies using randomandnonrandom ssignmentwere included nthemeta-analysis.Thestudydesignvari-able was codedtodeterminewhetherornotincludingnonrandomizedtudies nflu-enced the meta-analytical indings(Lipsey& Wilson, 2001).Combined sample sizes of treatmentand control groups rangedfrom 14 to48,081. Eighty-five percentof the studies were conductedwith samples of 200participantsor less. Four calculator studies were conducted with over 4,000studentsparticipating.

RESULTSThe sections thatfollow presentresults from the meta-analysisandinterpreta-tions of the findings.Forcomparisonpurposes,analysisof heterogeneoussets ofeffect sizes was conducted twice: (1) with all of the effects and(2) with outliers

removedyieldingahomogeneousset of effects. Thetablesprovide he 95%confi-denceintervals hatwereused to determinehe statistical ignificanceofg, the corre-spondingweighted mean effect size. Hedges Qstatistics,used to determinethehomogeneityof each set of effect sizes, arealso includedin these tables. For theskill datathatwas heterogeneous n the firstroundof the analysis, I included anindependentvariableanalysis to gain informationaboutthe heterogeneityof thedata.The resultsfor studentattitudesand thecorrespondingndependentvariableanalysisconclude the resultssection.

8/7/2019 30034795

11/32


EffectSizesTable 3 contains the number of effect sizes gatheredfor each achievementconstruct(acquisition,retention, ransfer)andcategoryof skills analyzedin this

study.Eachnumber n the tablerepresents omparisonof achievementdata fromtreatmentand controlgroups.The results are organized accordingto method oftesting-with or withoutcalculators.

Table3Numberof EffectSizesfor the Achievementand Skill Constructs,TheirCategories, andCalculatorUse in TestingTesting ithoutalculators Testing ith alculators

Acquisitionetention ransfercquisitionetention ransfer TotalCategoryskillsOperational 15 0 0 25 2 1 43Computation 15 1 0 12 2 0 30Concepts 8 0 0 11 0 0 19CategoryI skillsProblemolving 7 0 0 14 2 1 24Productivity 0 0 0 1 0 0 1Selectivity 3 0 0 6 1 0 10Total 48 1 0 69 8 2 127

With regardto the numberswithin Table 3, there were 15 effect sizes used toanalyzestudentacquisitionof operational kills. Thesevalues weregathered rom15 studies thatprovideddataon studentacquisitionof operational kills in whichthe skills were notstrictlycomputational rconceptual,but nsteadwas acompositeof the two. Eachreportcontainedquantitativedata on the comparisonof a treat-ment and controlgroup in which the treatmentgrouphad access to calculatorsduring nstructionbut not duringtesting. Similarly,the results outlined below onstudentacquisitionof problem-solving kills whencalculatorswerepartof testingand instructionare based on the analysisof 14 effect sizes.The general category of operationalskills contains the most information oranalysiswith 43 effect sizes acrossbothtestingconditions.Resultsonproductivityare notprovidedsinceonly one study provideddataon thisproblem-solving kill.Sixty-nineeffect sizes were available to analyzeskills acquisitionwhen calcula-torswerepartof testingas well as instruction.The resultsof theacquisitionof oper-ational and problem-solvingskills when calculatorswere not partof the testingprocess are based on 48 effect sizes.

Although echnicallyananalysiscan be conductedwith as few as twoeffectsizes,the resultsfrom such a small numberof studies are not a strongreflection of thepopulationunderconsideration.Therefore, choseto analyzeonly thosecategorieswith three or more effect sizes. The 54 studies includedin this meta-analysisdidnotprovidemuch nformation n skills retentionandtransfer o thismeta-analysisdoes notprovideinformationon these two aspectsof achievement.

8/7/2019 30034795

12/32


EffectSize Findings or Acquisitionof Skills-Testing WithoutCalculatorsTable 4 includes the results from an analysisof the achievementconstructof

acquisitionof operationalandproblem-solving kills in testingsituations hatdidnot permitthe use of calculators.The results showed that there weretwo signifi-cantfindings n whichthe95%confidence ntervaldid notcontainzero:operationalskills (.03, .31) andselectivityskills (.15, .44). Because the Q tatisticwas signif-icant for operationalskills (indicatingthat the data set was not homogeneous),Iconducted urtheranalysis,which resulted nthe removalof anoutlierand in Qnolonger being statisticallysignificant.Theoperational kills weightedmeaneffectsize (g = .17) was slightlysmaller(g = .14) afterthis was done, resulting n a newconfidence interval of (.01, .38) that also did not contain zero. These findingssuggestthatfor assessmentsof operational kills andproblem-solvingselectivityskills in which calculatorswere not allowedduringtesting,studentsusingcalcu-latorsduringinstructionperformedbetter than the control group.For problem-solving selectivityskills, themean effect (g = .30) was basedon threestudies thatassessed theabilityof studentsusingcalculators o select theappropriate roblem-solving strategies,andconsequently his resultshould be interpretedwith caution.

Table4Results rom theAnalysis ofAcquisition ofSkills-Testing WithoutCalculatorsSkill Type k g CI U3 Q Ne N

Operational killsAll studies 15 .17 (.03, .31) 57 29.5" 1069 1065Outliersremoved 14 .14 (.01, .38) 25.5 1044 1047ComputationalkillsAll studies 15 .03 (-.14, .20) 32.1* 843 886Outliersremoved 14 -.02 (-.16, .11) 17.7 763 786ConceptualskillsAll studies 8 .05 (-.20, .29) 28.1* 650 715Outliersremoved 7 -.05 (-.19, .09) 7.3 590 655Problem-solvingskillsAll studies 7 .16 (-.01, .32) 5.2 287 273Selectivity skillsAll studies 3 .30 (.15, .44) 62 1.0 346 420Note.k= number f studies; = weightedmeaneffectsize;CI= 95%confidencenterval org;U3=percentagef area elow onthestandardormalurvereportednly orCIs hat onotcontainzero);Q= homogeneitytatistic;Ne= combinedxperimentalroup ample ize;Nc= combinedcontrol roup ample ize.* p < .01

Both of these values (g = .17 andg = .30) are considered o be smallaccordingto theguidelinesforevaluating hemagnitudesof effect sizes. The U3statisticwascalculated for each weighted mean effect size in order to presenta clearer nter-pretationof each value. The U3statistic convertsan effect size to the percentage

8/7/2019 30034795

13/32


of area falling below the given effect size value on the standardnormal curve(Cohen, 1988). U3 s thepercentageof students n the treatment roupwho scoredhigherthan the median score of the controlgroupwhile, based on the definitionof median,50%of studentsnthe controlgroupscoredhigher han he medianscoreof the controlgroup.Forthe weightedmeaneffect sizes for these constructs, heU3statisticsare 57 foroperational kills and 62 forselectivityskills.The value 57meansthatwhile50%of students n the controlgroupscoredhigher han hemedianon achievement ests of operational kills,57%of studentsusingcalculatorsduringinstruction coredhigherthan the median score of the controlgroupon achieve-ment tests of operationalskills. Based on the writingsof Cohen (1988), anotherinterpretations theaveragestudentwho had access to a calculatorduring nstruc-tion had a mathematics chievement corethatwasgreaterhan57%of the studentswho did not have access to calculatorsduringinstruction.The U3 statistic forproblem-solving electivityskills wasslightlyhigherat62, but wasbasedon asmallnumberof studies.

The computational,conceptual,and problem-solvingskills categoriesdid notyield statistically significantresults because theirconfidence intervalscontainedzero.Because the datasets forthecomputational ndconceptualskills constructswere not homogeneous in the initial stage of analysis, the outlier analysis wasconducted.However,evenafterremovingoutliers, he confidence ntervals ortheweighed mean effect sizes still contained zero. Therefore, students who usedcalculators during instruction did not perform significantly higher on tests ofmathematical chievementwithoutcalculators hantheirnoncalculator-use oun-terparts.Whereasstudentsdid not benefit from the use of calculatorswhen devel-oping computationalandconceptualskills, their abilitieswere also not hinderedby calculatoruse.Theproblem-solvingdata set was homogeneousafter he firststageof analysis,so it was notnecessaryto runthe outlieranalysis. Althoughthe lowervalue of theconfidence interval s negative, the value is small enoughto be considered zero.Therefore, he students n the treatment nd controlgroupswere not significantlydifferenton assessmentmeasuresof problem-solvingskills.

EffectSize Findings or Acquisitionof Skills-Testing WithCalculatorsAs shown in Table5, statisticallysignificantweightedmean effect sizes weregenerated for four of the five constructcategories in which calculators wereallowedduring esting.Selectivitywas theonly one thatdid nothavea significanteffect size. Because theQ tatisticwas significant orthesefourcategories,outlieranalysis was conducted.Threeconstructs,operational kills (g = .38), computa-tional skills (g = .43), andproblem-solving kills (g = .33), wereslightlyaffectedby the removalof outliersresulting n g values of .32, .41, and .22, respectively.However,these changesin effect size magnitudewereminimal,and the resultingeffect size values for all fourconstructscan be consideredas small to medium.

8/7/2019 30034795

14/32

AimeeJ. Ellington 445

Table5Results rom theAnalysis of Acquisitionof Skills-Testing WithCalculatorsSkillType k g CI U3 Q Ne N

OperationalkillsAllstudies 25 .38 (.28, .48) 65 243.1" 32892 31397Outliers emoved 19 .32 (.21,.42) 30.6 3589 3534ComputationalkillsAll studies 13 .43 (.18,.67) 67 63.2* 3277 2123Outliers emoved 11 .41 (.23, .59) 24.7 3213 2069ConceptualkillsAll studies 11 .44 (.20, .68) 67 60.4* 3100 2444Outliers emoved 8 .44 (.19,.69) 17.1 2653 2090Problem-solvingkillsAll studies 14 .33 (.12,.54) 63 41.6" 3226 2089Outliersemoved 12 .22 (.01, .43) 19.9 400 404SelectivitykillsAll studies 6 .20 (-.01, .42) 2.4 153 189Note: = numberfstudies; = weighted ean ffect ize;CI=95% onfidencentervalorg;

U3= percentage f areabelowg on the standard ormal urve(reported nlyfor CIs thatdo not containzero); Q = homogeneitystatistic;Ne = combinedexperimentalgroup sample size; Nc = combinedcontrolgroupsamplesize.*p

8/7/2019 30034795

15/32


results than the studies in which calculators were not part of the assessmentprocess.Forthe resultsoutlinedabove that were based on homogeneousdata sets in thefirststageof analysis, t canbe assumedthat heweightedmeaneffect size was thebest estimateof thepopulation epresented ythedata.Forstudies hatdidnot allowcalculatorsduringtesting,the problem-solvingskills categoryand the selectivityskills categorywerehomogeneous n the firststageof analysis.This was also trueforselectivityskills whencalculatorswerepartof instruction nd esting.Therefore,the weightedmean effect sizes andcorresponding onfidence intervalsfor theseconstructsadequately epresentedhepopulation rom which thedatacame.The weighted mean effect size was not the best estimate of the population(Lipsey&Wilson,2001) for the sets of effect sizes that wereheterogeneous nthefirststageof analysis(i.e., operational kills, computational kills, andconceptualskills when calculatorswere not includedin testing; operational kills, computa-tionalskills, conceptualskills, andproblem-solving skills whentestingincludedcalculators)and the differencewas likely based on a study'scharacteristics i.e.,independentvariables).In order to determine he influence of independentvari-ables on theheterogeneity f effectsizes ineach achievement onstruct, conductedan analysisof moderator ariables.This was done to gain insightinto the reasonsa set of effects was heterogeneousandto help explainthe influence of the codedindependentvariableson the studentachievementconstructsunderconsideration.

Analysis UsingModeratorVariablesIn the moderator ariableanalysis,allcharacteristicsxceptforsamplesize listedin Table 1 were included with some merged in order to produce meaningfulcomparisons.The sections that ollow presentresults ortheindependent ariablesthatyielded significantresults;for the variablesthatdid not, suchresults are notreported.Thesignificance evel,p < .01, was used to determinewhether herewas

a significantdifference in effect size magnitudes or each independentvariable.Ninety-five percentconfidence intervals were used to determine the statisticalsignificanceof g, thecorrespondingweightedmean effect size.

Testingwithout alculators.Becausethe initial est forhomogeneity orproblem-solving skillsandselectivityrevealedahomogeneousset of effect sizes (see Table4), anindependent ariableanalysiswas not conducted orthis setof data.Instead,the focus for this partof the studywas on the skill type variableson operational,computational,andconceptual.The resultsfromthis analysis appear n Table 6.The data in the top portionof Table6 show thatforoperational kills, only oneindependentvariable-treatment length-produced significant differences foreffect size magnitudesacrossthreetreatment ategories:0-3 weeks, 4-8 weeks,and9 or more weeks. Treatmentength(QB = 14.5,p < .01) resulted n a negativeweightedmean effect size (g = -.17) for studies conductedover a 4-8 week treat-mentperiod.However,the value was not significantlydifferentfromzero based

8/7/2019 30034795

16/32

Aimee J. Ellington 447

Table6ModeratorVariableAnalysis of SkillEffects-Testing WithoutCalculatorsOperationalkills

Variable k g CI Qw QBTreatmentength0-3 weeks 4 .31 (.14, .48) 1.8 14.5"4-8 weeks 3 -.17 (-.36, .02) 1.59 or moreweeks 8 .24 (.05, .42) 11.7Computationalkills

Variable k g CI Qw QBTreatmentength0-3 weeks 3 .14 (-.51, .78) 7.2 13.4"4-8 weeks 3 -.25 (-.49, -.01) 1.99 or moreweeks 9 .06 (-.08, .20) 9.6Conceptualkills

Variable k g CI Qw QBEducationalivisionElementarychool 4 -.06 (-.29, .18) 5.5 16.5"Middle chool 2 .52 (-.26, 1.29) 5.8High chool 2 -.15 (-.38,.09) 0.2Treatmentength0-3 weeks 3 .26 (-.48, 1.00) 17.4" 10.3"4-8 weeks 2 -.29 (-.55, -.04) 0.39 or moreweeks 3 .08 (-.06, .22) 0.2CalculatorseFunctional 4 -.21 (-.42, .01) 2.3 7.1*Pedagogical 4 .21 (-.15, .57) 18.7*Note.k= number f studies; = weightedmeaneffectsize;CI= 95%confidencentervalorg;Qw=homogeneity statistic; QB= ifference between contrasted categories* p< .01

on the confidence interval values. Positive values were generatedfor calculatortreatmentsof operational kills lasting0-3 weeks (g = .31) and9 or moreweeks(g = .24); in bothcases, the weightedmean effect sizes were significantly largerthan zero.Therefore, he operational kills of studentsusingcalculators ess thanorequalto 3 weeks or 9 or moreweeks improved.Because the resultfor 9 or moreweeks was based on eight studies,this result was the most credibleof the resultspresented or the analysis by treatmentength.As shownin the middleportionof Table6, treatmentength(QB= 13.4,p < .01)alsoproduced significant esult or effect sizesresulting romcomputationalkillsassessments n which calculatorswerenotpartof testing.The 0-3 weeks and 9 ormore weeks categoriesyielded positive weightedmean effect sizes, butthe valueswere not significantlydifferentfrom zero.Therefore, or these treatmentengths,studentsusing calculatorsduringinstructionbut not duringtesting were neitherhelpednor hinderedby calculatoruse. Thenegativeweightedmean effect size for

8/7/2019 30034795

17/32

448 Effects of Calculators:A Meta-Analysis

studiesconducted vera 4-8 week treatment eriod ndicated hatstudentsnotusingcalculatorsduring nstructionoutperformedheircalculatorcounterparts n testsof computational kills.The conceptualskills construct(see the lower portionof Table 6) resulted insignificantdifferences or three ndependent ariables: ducationaldivision, treat-

mentlength,and calculatoruse. Withrespectto educationaldivision (QB = 16.5,p < .01), theweightedmeaneffect sizes generatedorelementary chool (g = -.06),middle school (g = .52), andhigh school studies(g = -.15) did not correspond oa significantdifference n theconceptualskills assessmentoutcomesfor calculatorand noncalculatorstudents.With respect to treatment ength, the results weresimilar o thosereportedor thecomputational kills construct.The 0-3 weeks and9 or more weeks time framesyielded positive weightedmeaneffect sizes thatwerenot significantlydifferentfrom zero. Therefore,for these two treatment engths,studentsusing calculatorsduring instructionon conceptual skills were neitherhelpednor hinderedbycalculatoruse. The 4-8 week time frameresulted n a nega-tive weightedmeaneffect size, suggestingthat studentswho did not have accessto calculators outperformedstudents who used calculators during lessons onconceptualskills. Themagnitudeof effect sizes for theconceptualskillsconstructalso differed significantly with respect to the calculator use variable(QB = 7.1,p < .01). The weightedmean effect sizes for functionaluse andpedagogicalusewere small, and neithervalue was significantlydifferentfrom zero.

Testingwithcalculators.This section presentsresults of an analysisof opera-tional skills, computational kills, andconceptualskills by independentvariableforstudiesin whichthe calculatorswerepartof testing.Therewas no single inde-pendent variable for which a significant difference existed across all threeconstructs.Theoperationalkillsandconceptual killsconstructswere the two areasmost affectedby thevariables hatwere featured n thisanalysis.Thetop portionof Table7 contains he results or theoperationalkillsanalysisofthe independent ariables.The analysisrevealed hatthe magnitudeof effect sizesdifferedsignificantlywith respectto publication tatus QB = 140.6,p < .01). Theweightedmeaneffectsize was smallest orstudiespresenteds otherunpublishedocu-ments such as those from ERIC(g = .27). The valuegenerated romdissertations(g = .31) wasslightly arger.Theweightedmeaneffect size forstudies hatappearedasjournalarticles g = .50) was moderate n size. All threevalueswerestatisticallysignificant n favorof studentswho had access to calculators uring nstruction.Based on the significantdifference in the test instrumentvariable(QB = 18.3,p < .01), nonstandardizedests yielded a slightly larger(g = .44) weightedmeaneffect size whencomparedwith standardizedests(g = .32). However,both valueswere moderate n size butstatisticallysignificant.Therefore, tudents akingstan-dardized and nonstandardized eacher-made ests of operationalskills benefitedfromcalculatoruse during nstruction.The educational division variable also had a significant influence on themagnitudeof effect sizes (QB = 17.1,p < .01) butonly in the middle school and

8/7/2019 30034795

18/32

8/7/2019 30034795

19/32


The significantdifferencein effect size magnitudewithrespectto abilitylevel(QB = 14.1, p < .01) resulted rom 3 studies of high ability students hatwere sepa-rated rom theremaining22 studies conducted n mixedabilityclassrooms.Therewere no studies of operational kills thatwere conductedsolely with low abilitystudents.The weightedmeaneffect size for thehigh abilitystudies(g = .69) wasin the high range,while the correspondingstatisticfor the mixed ability studies(g = .35) was moderate n size. Both values were statistically significant.The analysisof independentvariablesrevealed thatthe magnitudeof the effectsizes differedsignificantlywith respectto treatmentength (QB = 28.3, p < .01).Unlike the similaranalysisdescribedabove for studies in which calculatorswerenot allowedduringtesting,there wereno negative weightedmean effect sizes forany of the four treatment ength categories. The effect size value for studiesfeaturingonly a test (g = .29) was small, whereasthe weightedmean effect sizesfor the 0-3 weeks and 9 or moreweeks categoriesfell in the moderaterange.Theresultsfor the testonly, 0-3 weeks, and9 or more weeks categorieswere statisti-cally significant n favor of studentsusingcalculators.For studies conductedovera 4-8 week timeframe,studentswereneitherhelpednorhinderedby the inclusionof calculators n testingandinstruction.Calculatorype(QB= 28.5,p < .01) alsorevealedsignificantdifferences neffectsize magnitudes,and all threeresults were significantlydifferent rom zero. Fourstudiesallowed students o useany typeof calculator basic,scientific,orgraphing)anddid not featureanyformof mathematics nstruction.Eachstudywas acompar-ison of students akinga test with access to calculatorsand students aking he sametest withoutaccess to calculators.The studies with basic or scientific calculatorsyielded a higher weightedmean effect size (g = .55) whencompared o the effectsize forstudiesfeaturing hegraphing alculator g = .40).The results or this inde-pendentvariablerevealthatfor all typesof calculators,studentsusingcalculatorsduringtestingand instructionperformedbetterthan their noncalculator ounter-partson tests of operational kills.

Lastly,themagnitudeof effect sizes for theoperational killsconstructdifferedsignificantlywithrespectto study design (QB = 19.9,p < .01). The four studies inwhich the treatment roupwas not selectedbyrandomassignmentgeneratedarela-tively large weighted mean effect size (g = .68). The studies in which randomassignmentwas used resulted n a moderate ffect size (g = .33).Both valuesrepre-sent a statisticallysignificantdifference between assessmentresults of studentsusingcalculators uring nstruction ndstudentswith no access tocalculators uringinstruction.Due to selection bias in the nonrandomized esign(Lipsey& Wilson,2001), it appears hat the nonrandomized tudies overestimated he magnitudeoftheeffect of calculatorson operational kills.The lower portionof Table 7 shows thatsignificantdifferences in effect sizemagnitudes or two independent ariables esulted rom theanalysisof thecompu-tationalskills construct.Forpublication tatus QB = 19.6,p < .01), the result wassimilarto theone reportedaboveforoperational kills.Theweightedmean effectsize for studiesappearingnjournals(g = .82) was fairly large.The effect size for

8/7/2019 30034795

20/32


dissertations(g = .18) was not significantly different from zero, whereas thejournalresult was statistically significantin favorof the calculatorgroup. Studydesign also revealedsignificantdifferences(QB = 27.6, p < .01) in the magnitudeof effect sizes. The threestudiesconductedwithoutrandomassignment othe treat-mentgroupyieldeda large weightedmeaneffect size (g = 1.18).Based on the sizeof this value, the nonrandomized tudiesappear o overestimate he overall effectof calculatorson computational kills. The effect size for studies conductedwithrandomassignment (g = .24) was smaller, but both values were significantlydifferentfrom zero.

The upperportionof Table 8 contains the results of the independentvariableanalysis for conceptual skills and shows significant differences in effect sizemagnitudes or five variables.For test instrumentQB = 7.5, p < .01), the studiesusingnonstandardizedests(g = .60) yieldeda weightedmean effect size thatwasstatistically ignificant.Therefore, tudentswho took teacher-madeests of concep-tualskillsbenefitedfromcalculatoruse. The resultfor standardizedests (g = .16)was not statisticallysignificant.Withrespectto educationaldivision (QB = 13.0,p < .01), the middle school divisiongenerated helargestweightedmeaneffect size(g = .70) followed by the high school division (g = .43). The effect size for theelementarydivision (g = -.14) was based on only two studies andwas not statis-tically significant.The middle andhigh school values weresignificantlydifferentfrom zero.

A significant difference in effect size magnitudewas found for ability level(QB= 11.5, p < .01). The studies that featured high ability students were sepa-rated from the studies conducted in mixed ability classrooms. The result was ahigher weighted mean effect size for the high ability classes (g = .84), but thevalue was not significantly different from zero. For calculatoruse (QB = 17.5,p < .01), the studiesin whichthe calculatorhad a functionalroleyieldeda smallereffect size value (g = .12) when comparedwith the studies in which the calcu-latorhada pedagogical role (g = .69). The effect size for the functional studieswas not statistically significant in favor of the studentsusing calculatorsbut thevalueforpedagogicalstudies revealed that studentswho used calculatorsoutper-formed their noncalculatorcounterpartson assessments of conceptual skills. Asignificant difference in effect size magnitudewas generatedby calculatortype(QB= 9.7, p < .01). The weightedmeaneffect size (g = .69) for the graphing alcu-lator studies was in the high range.The effect size value for the studiesfeaturingbasic and scientific calculators(g = .13) was not statistically significant.The lower portionof Table 8 contains the independentvariableanalysis forproblem-solving ffect sizes from studies n which calculatorswere allowedduringtesting.Significantdifferencesineffect size magnitudesresulted romanalysisofabilitylevel (QB = 12.0,p < .01) andcalculator ype(QB = 12.9,p < .01). Thehighabilitycategorycontainedonly one effect size. The low abilitycategory,consistingof two effect sizes, yieldeda negativeweightedmean effect size (g = -.18) buttheresultwas notstatisticallysignificantfor eitherthe calculatoror the noncalculatorgroup.The mixed abilitystudiesgenerateda moderateeffect size value (g = .43)

8/7/2019 30034795

21/32

452 Effectsof Calculators:A Meta-Analysis

Table8ModeratorVariableAnalysisof Conceptual ndProblem-Solving killEffects-Testing WithCalculatorsConceptualkills

Variable k g CI Qw QBTest nstrumenStandardized 3 .16 (-.12, .44) 18.2" 7.5*Nonstandardized 8 .60 (.16,1.05) 34.7*EducationalivisionElementarychool 2 -.14 (-.42, .15) 0.1 13.0"Middle chool 5 .70 (.13,1.27) 34.9*Highschool 4 .43 (.03,.82) 12.6"AbilityevelMixed 8 .29 (.06, 52) 32.6* 11.5*High 3 .84 (-.04,1.71) 16.2*CalculatorseFunctional 2 .12 (-.05, .29) 0.5 17.5*Pedagogical 7 .69 (.23, 1.16) 31.0*CalculatorypeBasic/scientific 4 .13 (-.14, .40) 19.7" 9.7*Graphing 7 .69 (.23, 1.15) 31.0"

Problem-SolvingkillsVariable k g CI Qw QBAbilityevelMixed 11 .43 (.20,.65) 29.6* 12.0*Low 2 -.18 (-.59, .23) 0.0High 1 .15 (-.31, .62) 0.0CalculatorypeBasic/scientific 11 .23 (-.01, .47) 19.9 12.9*Graphing 3 .61 (.12, 1.10) 8.9Note.k= number f studies; = weightedmeaneffectsize;CI= 95%confidencenterval org;Qw= homogeneitytatistic;QB= difference etween ontrastedategories*p < 01

thatwas significantlydifferent romzero.Therefore, heproblem-solving kills ofstudents n mixedabilityclassroomsimproved rom calculatoruse during estingand nstruction.Withregard o the calculator ypevariable, hestudies hat eaturedthe graphingcalculator yielded a weighted mean effect size (g = .61) in themoderate ohigh range hatwas statistically ignificantnfavorof thestudentsusingcalculators.The studies that involved a basic or scientific calculatorgeneratedarelativelysmall effect size value(g = .23) thatdid notsignificantly avorthe calcu-latorgroup.Findings Regarding Student Attitudes

Table9 summarizes hemeta-analyticalindingsregardinghe attitude onstructs.The data set for the attitude owardmathematicsconstructwas heterogeneousat

8/7/2019 30034795

22/32


the initial stage of analysis.Therefore, ust as with the skills constructs,analysiswas conductedwith all studies andthen after he removalof outliers.Due to insuf-ficientdata,inferential tatisticscould not be generated or four of thecategories:anxiety owardmathematics,motivation o learnmathematics, ttitude owardmath-ematicsteachers,and students'perceptionsof the value of mathematicsn society.

Table9Results rom theAnalysis of AttitudeConstructsConstruct k g CI U3 Q Ne NcAttitudeowardmathematicsAll studies 18 .32 (.07, .58) 63 134.5" 1366 1286Outliers emoved 12 .20 (.01, .40) 22.8 491 457

Self-conceptn mathematicsAllstudies 4 .05 (-.06, .16) 2.6 706 631Attitudeoward se ofcalculatorsn mathematicsAll studies 3 .09 (-.19, .36) 3.7 645 556Note:k= number f studies; = weightedmeaneffectsize;CI= 95%confidence nterval org;U3=percentagef area elow onthe tandardormalurvereportednly orCIs hat onotcontainzero);Q= homogeneitytatistic;Ne= combined xperimentalroup ample ize;Nc= combinedcontrol roup ample ize.*p

8/7/2019 30034795

23/32


weightedmean effect size (g = .35) was small to moderate.The value was signif-icantlydifferent rom zero. The effect size value for the ERICdocumentsor otherunpublisheddocumentswas negative(g = -.01) but close to zero and not statisti-cally significant n favor of students n the controlgroup.The test instrument ari-able(QB = 22.5,p < .01)yieldeda weightedmeaneffect size forstudiesusingstan-dardized tests (g = .32) slightly larger than the value for studies usingnonstandardized ests (g = .28). The result for standardized ests is statisticallysignificantin favor of studentswho had access to calculatorsduring nstruction.

Table10ModeratorVariableAnalysis-Attitude ConstructVariable k g CI Qw QBPublicationtatusJournal/dissertation 16 .35 (.09, .62) 106.5" 11.7"Other 2 -.01 (-1.48, 1.46) 16.3"Test nstrumentStandardized 10 .32 (.05, .59) 49.8* 22.5*Nonstandardized 8 .28 (-.23, .78) 62.2*EducationalivisionElementarychool 4 .15 (-.19, .49) 4.2 10.7"Middle chool 7 .28 (-.12, .69) 64.1"Highschool 7 .38 (-.12, .89) 55.4*AbilityevelMixed 16 .23 (-.01, .46) 87.2* 28.7*High 2 1.06 (-.24, 2.37) 18.5*Treatmentength0-3 weeks 5 .21 (-.26, .67) 18.7* 14.2*4-8 weeks 4 .40 (-.43, 1.22) 79.9*9 or moreweeks 9 .32 (.06, .58) 21.8"CalculatorseFunctional 5 .36 (-.18, .90) 64.6" 27.9*Pedagogical 13 .32 (.07, .58) 42.0*CalculatorypeBasic/scientific 10 .17 (-.10, .44) 39.9* 47.8*Graphing 8 .49 (.11,.87) 46.7*Note.k = number f studies; = weightedmeaneffectsize;CI= 95%confidence nterval org;Qw= homogeneitytatistic; B = difference etween ontrastedategories*p

8/7/2019 30034795

24/32


effect size value for mixedabilityclasses (g = .23) was smallandnot statisticallysignificant.The 9 or moreweekscategoryproduced smallto moderateweightedmeaneffectsize (g = .32) duringthe analysisof treatment ength (QB = 14.2,p < .01). Thisresult significantly favored the studentswho had access to calculatorsduringinstruction.The effect size values for the0-3 weeks categoryand4-8 weeks cate-gory(g = .21 andg = .40, respectively)wererelativelysimilar n size, but neithervalue was statisticallysignificant.The weighted mean effect sizes for the functional(g = .36) and pedagogical(g = .32)categorieswere close insize after heanalysisof calculatoruse(QB= 27.9,p < .01). The pedagogicalresult was statisticallysignificantin favor of studentswho hadaccess to calculatorsduring nstruction.Lastly, significantdifferences nthemagnitudes f effect sizes were foundwithrespect o calculatorype(QB= 47.8,p < .01). The studies that featured he graphingcalculatorgenerateda moderateweightedmean effect size (g = .49) that was statisticallysignificantfor studentswho had access to calculators.The effect size value for studiesusingbasic or scien-tific calculatorswas small (g = .17) and not statisticallysignificant.SummaryofMajor Findings

When calculatorswere includedin instructionbut not testing, the operationalskills and theabilityto select theappropriate roblem-solving trategies mprovedfor the participating tudents.Underthese conditions,there were no changes instudents' omputationalkills andskills used to understandmathematicaloncepts.When calculatorswere partof bothtestingandinstruction, he operational kills,computational kills, skills necessaryto understandmathematical oncepts, andproblem-solving killsimproved orparticipatingtudents.Under heseconditions,therewerenochanges n students'abilityto select theappropriate roblem-solvingstrategies.Studentswho usedcalculatorswhilelearningmathematicseportedmorepositive attitudestowardmathematicsthan their noncalculatorcounterpartsonsurveystakenat theend of the calculator reatment.

DISCUSSIONThepurposeof thismeta-analysiswas to determine he effects of calculatorsonstudents'acquisitionof operationalandproblem-solvingskills as well as studentattitudes owardmathematics.The studies on which these resultswere basedwere

conductedprimarilyn classrooms n which studentswereengagedin a traditionalmathematicscurriculum.The readershouldkeep in mind that in most cases, theparticipatinglassroomswere notusingcurriculummaterials pecificallydesignedfor calculatoruse, but at the sametime, it should be noted thatin two thirdsof thestudies the calculatorhadanactive role in the teachingandlearningprocess.

8/7/2019 30034795

25/32


Overviewof FindingsWhen calculators were available during instructionbut not during testing,students in grades K-12 maintainedthe paper-and-pencil kills and the skills

necessary to understandmathematicalconcepts. The operationalskills of thesestudents mprovedas a resultof calculatoruseduring nstruction. tudents eceivedthe most benefit when calculatorshadapedagogicalrole in the classroomand werenotjustavailable or drillandpracticeorcheckingwork.The results oroperationalskills favored mixed abilityclasses, with high abilitystudentsneitherhelpednorhinderedbycalculatoruseduring nstruction.Themeta-analysis eportedheredoesnot include results for low ability students. In orderto have a positive influenceon students' operationalskills, the findings suggest that calculatoruse duringinstructionshould be long term(i.e., 9 or moreweeks). Withrespectto problemsolving,the skills of precollegestudentswere not hinderedbythe inclusionof calcu-lators n mathematicsnstruction.Basedon a limitednumberof studies,the skillsnecessaryto select the appropriate roblem-solvingstrategiesmay improveas aresult of calculatoruse.

When calculatorswere included in testing and instruction,students in gradesK-12 experienced improvement n operationalskills as well as in paper-and-pencilskills andtheskillsnecessary orunderstandingmathematicaloncepts.Withregard ooperational killsandconceptualskills, the resultsof calculatoruse weremost significantfor classes in which the calculator's role was pedagogical.Thecalculatorbenefitedstudents nmixedabilityclasses andclasses consistingof highability students.The meta-analysisdoes not reportresults sufficient for general-izations to be made for classes of low ability students.When the calculatorwasincluded ntestingandinstruction f conceptualskills, studentsbenefitfrom shortterm(0-3 weeks) use of calculators.Benefits to operational kills can be seenwithshort termor long term(9 or moreweeks) calculatoruse.Under he sametestingandinstruction ircumstances,mprovementnproblem-solvingskills for students nmixedabilityclassesappearednthe results.This meta-analysisdoes notprovidesufficient dataforgeneralizations or classes consistingof low orhigh abilitystudents.Students'abilities o select theappropriateroblem-solving strategieswere not hinderedby the calculator'srolein testingand instruc-tion. The increase in problem-solvingskills may be most pronouncedundertwoconditions:(1) whenspecialcurriculummaterialshave beendesignedto integratethe calculator n the mathematicsclassroom and(2) when the technologicaltoolin use was thegraphing alculator.These resultsshouldbe interpretedwith cautionbecause the data are based on a smallnumberof studies.

Allowing students o usecalculators nmathematicsmayresult nbetterattitudestoward mathematics. n this study,attitudesshowed the most improvementafter9 ormoreweeks of calculator se. Students' elf-conceptn mathematicsndattitudetoward he use of calculatorsn mathematicswere not hinderedby calculatoruse.In thismeta-analysis crossallconstructs,he results or studies asting4-8 weekseither favored studentswho did not have access to calculatorsduring nstruction

8/7/2019 30034795

26/32


or did not show significant differences between the two groups. For manyconstructs, the results based on studies lasting less than4 weeks or more than8 weeks were favorable for calculator use. This discrepancy may be relatedtostudents'abilitiesto retainwhattheylearn.Inshort-termtudies,retentionwas notassessed,but in long-termstudies,retentionwas somewhatsignificant especiallywithconcepts learnedearlyin the treatmentphase.Basedon thenatureof thedatagathered rom the 54 studies,the effect of calcu-lator use in individual grades could not be determined. Hembree and Dessart(1986, 1992)reportedn theirmeta-analysishatwhen calculatorswere not allowedduring testing, the use of calculators n instructionhad a negative effect on thecomputationalskills of students in fourthgrade. Unfortunately, his particularresultcould not be supportedordisprovedby the currentmeta-analysis.Based onstudiesconductedwithin heelementary ivision, hedevelopment f computationalskillswasnothindered ycalculator seduringnstructionor bothwith andwithoutcalculatoruse in testing.When calculatorswere not allowedduring esting,resultswere notsignificantlydifferent or onetypeof calculator scomparedo the others.Whencalculatorswerean integralpartof the testing process,the resultsbased on graphingcalculatoruseweresignificantly etter han he resultsof basicor scientificcalculatorsn two areas:conceptualskills andproblem-solving kills.Operational kills benefitedfrom allthree ypesof calculators.Lastly,graphing alculatorshad a moresignificant nflu-ence on students'attitudeswhencomparedwith othertypesof calculators.Recommendationsor ClassroomUsage

The results from this meta-analysissupport he use of calculators n all precol-lege mathematics classrooms. When considering the grade distributionof thestudies based on the educationaldivisions (elementary,middle, high school),length of calculator availability during instruction should increase with eachincreasinggrade evel. Becauselimited researchhas been conducted eaturing heearly grades,calculatoruse should be restricted o experimentationandconceptdevelopmentactivities.Calculators hould be carefully ntegratedntoK-2 class-roomsto strengthenheoperational oalsof thesegrades,as well as fosterstudents'problem-solvingabilities.Calculators houldespeciallybe emphasizedduring he instruction f problem-solving skills in middle andhigh school (i.e., Grade6 throughGrade12) mathe-matics courses. Thisemphasismayresult n increasedsuccess in problemsolvingas well as more positive attitudestoward mathematics.Teachers should designlessons that ntegrate alculator-based xplorationsof mathematical roblemsandmathematical concepts with regular instruction, especially in these grades.Calculatorsshould be available duringevaluations of middle and high schoolstudents'problem-solving kills andtheirunderstandingf mathematicaloncepts.This recommendations based onthe resultsreportedn thismeta-analysis,andtheinconsistenciesnotedbyotherreviewers Gilchrist,1993;Penglase&Arnold,1996;

8/7/2019 30034795

27/32

458 Effects of Calculators: A Meta-Analysis

Roberts, 1980) that occurwhen tests aregiven without calculatorsafter instruc-tion has takenplace withcalculators.

Recommendationsor FutureResearchConsidering he searchconducted o gatherrelevantstudiesandrecognizingthefact thatthispaperdoes notfully addressmany questionsthathavebeenraisedbymathematicseducators, proposeseveral areas in which further alculator-basedresearch s needed.Onlya few studies involved the calculator'srole in the reten-tion andtransfer f operational kills andstudents'abilities o select theappropriateproblem-solvingskills. Researchersneed to consider students'abilities to select

appropriateproblem-solvingstrategies in light of available technology and toretaintheiroperationalandproblem-solvingskills afterinstructionwith calcula-tors.Also, further esearch s neededregarding hetransfer f skills to othermath-ematicalsubjectsandto areasoutside of mathematics.Based on the definition used to identify a mathematicalskill as a problem-solving skill in preparing or this meta-analysis, ittle informationwas availableon the relationshipbetween the graphingcalculatorand studentachievement inproblem-solvingskills. The studies featuringthe graphingcalculatorprimarilyfocused on theacquisitionof operational kills;consequently, heproblem-solvingresults wereprimarilybased on basic and scientific calculators.Therefore, utureresearchshould includestudies of graphingcalculatoruse in the developmentofproblem-solvingskills.Inspiteof the fact thattheNCTM(1989, 2000) has beenadvocatingchangestothe mathematicscurriculumwith computerand calculator echnologyas an inte-gral component, the search for studies for this meta-analysis yielded only sixstudies in which special curriculummaterialswere designed for calculatoruse.Because this numberreflectsonly 11%of the studiesanalyzed,this is an area inwhich moreresearchneeds to be conducted.

REFERENCESAiken, L. (1974). Two scales of attitude oward mathematics.Journal or Research in MathematicsEducation,5, 67-71.Ballheim,C. (1999). Howour readers eel aboutcalculators. nZ. Usiskin(Ed.),Mathematics duca-tiondialogues (p.4). Reston,VA: National Council of Teachersof Mathematics.Cohen,J. (1988). Statisticalpoweranalysis or the behavioralsciences. Hillsdale,NJ:Erlbaum.Cooper,H.,& Hedges,L. (Eds.).(1994). Thehandbookof researchsynthesis.New York,NY: RussellSage Foundation.Fennema,E.,& Sherman, . (1976).Mathematics ttitude cales: Instruments esignedto measureatti-tudes toward helearning f mathematics y males andfemales.JournalforResearch n MathematicsEducation, 7, 324-326.Gilchrist,M. E. (1993). Calculatoruse in mathematics nstructionandstandardized esting:An adulteducation inquiry. Dayton, VA: VirginiaAdult EducatorsResearch Network.(ERICDocumentReproductionServiceNo. ED 372919)Glass,G.V., McGaw,B., & Smith,M. L. (1981).Meta-analysis n social research.BeverlyHills,CA:Sage.Hedges,L.V., &Olkin,I.(1985).Statisticalmethodsformeta-analysis. anDiego,CA: AcademicPress.

8/7/2019 30034795

28/32


Hedges,L. V., Shymansky,J. A., & Woodworth,G. (1989). Apractical guide to modernmethodsofmeta-analysis. Washington, DC: National Science Teachers Association. (ERIC DocumentReproductionService No. ED309952)Hembree,R. & Dessart, D. J. (1986). Effects of hand-held calculators in precollege mathematics

education:A meta-analysis.Journal or Research in MathematicsEducation, 17, 83-99.Hembree,R., & Dessart,D. J. (1992). Researchon calculators n mathematics ducation.In J. T. Fey(Ed.),Calculators n mathematics ducation pp.23-32). Reston,VA: NationalCouncilof Teachersof Mathematics.Huffcutt,A., &Arthur,W.(1995).Developmentof a new outlier tatistic ormeta-analytic ata.JournalofApplied Psychology,80, 327-334.Kaput,J. J. (1992). Technology and mathematicseducation. In D. A. Grouws (Ed.), Handbookofresearch on mathematics eachingand learning(pp.515-556). New York: Macmillan.Lipsey,M., & Wilson,D. (2001). Practicalmeta-analysis.ThousandOaks,CA:Sage.Liu, S. T. (1993). Effects of teachingcalculatoruse andproblemsolving strategieson mathematicsperformance nd attitudeof fifthgradeTaiwanese male and female students Doctoraldissertation,MemphisStateUniversity, 1993).DissertationAbstractsInternational,54, 4354A.Mackey,K. (1999). Do we need calculators?In Z. Usiskin (Ed.), Mathematicseducationdialogues(p. 3). Reston,VA: National Councilof Teachersof Mathematics.NationalCouncilof Teachersof Mathematics.1989). Curriculum ndevaluation tandardsforschoolmathematics.Reston,VA: Author.NationalCouncilof Teachersof Mathematics.2000).Principlesand standardsor school mathematics.Reston,VA: Author.Neubauer,S. G. (1982). Theuseofhand-heldcalculators nschools:A review.SouthBend,IN: IndianaUniversity.(ERICDocumentReproductionService No. ED220272)Parkhurst,S. (1979). Hand-held calculators in the classroom: A review of the research. (ERIC

DocumentReproductionServiceNo. ED200416)Penglase, M., & Arnold, S. (1996). The graphicscalculator in mathematicseducation:A criticalreview of recentresearch.MathematicsEducationResearchJournal, 8, 58-90.Pennington,R. (1998). A studyto determine he effect of instructionneffective use of a calculatorontest scores of middle school students(Master'sthesis, Salem-Teikyo University, 1998). (ERICDocumentReproductionServiceNo. ED434030)Rabe,R. M. (1981). Calculators n the mathematics urriculum:Effectsandchanges.SouthBend,IN:IndianaUniversity.(ERICDocumentReproductionService No. ED204178)Ralston,A. (1999). Let's abolishpencil-and-paperrithmetic. n Z. Usiskin(Ed.),Mathematics duca-tiondialogues (p. 2). Reston,VA: NationalCouncil of Teachersof Mathematics.Roberts,D. M. (1980). The impactof electronic calculatorson educationalperformance.ReviewofEducationalResearch,50, 71-88.Sandman,R. S. (1980). The mathematics ttitude nventory: nstrument nd user's manual.JournalorResearchin MathematicsEducation, 11, 148-149.Sigg, P. O. (1982). Thehand-heldcalculator:Effectson mathematicalabilities and implicationsorcurriculumchange. South Bend, IN: IndianaUniversitySchool of Education.(ERICDocumentReproductionService No. ED218147)Suydam,M. N. (1978). State-of-the-art eviewon calculators:Theiruse in education.Columbus,OH:Calculator nformationCenter.(ERICDocumentReproductionService No. ED167426)Suydam,M. N. (1979). The use of calculators in pre-college education:A state-of-the-artreview.Columbus,OH: CalculatorInformationCenter.(ERICDocumentReproductionService No. ED

171573)Suydam, M. N. (1980). Using calculators in pre-college education: Third annual state-of-the-artreview.Columbus,OH:Calculator nformationCenter. ERICDocumentReproduction erviceNo.ED206454)Suydam,M. N. (1981). The use of calculatorsinpre-college education: Fourthannualstate-of-the-art review.Columbus,OH: Calculator nformationCenter. ERICDocumentReproductionServiceNo. ED206454)

8/7/2019 30034795

29/32


Suydam,M. N. (1982). The use of calculatorsinpre-college education:Fifthannual state-of-the-artreview.Columbus,OH:Calculator nformationCenter. ERICDocumentReproduction ervice No.ED220273)

AuthorAimee J. Ellington, Department f MathematicsandAppliedMathematics,VirginiaCommonwealthUniversity, 1001 West Main Street, P.O. Box 842014, Richmond, VA 23284-2014;[email protected]

APPENDIXBibliographyof StudiesIncluded n theMeta-Analysis

Abuloum, K. (1996). Graphingcalculators:Teachersperceptions,training,and attitude(Doctoraldissertation,Universityof Nebraska,1996).DissertationAbstractsInternational,57, 1063A.Aldridge,W. B. (1991). The effects of utilizingcalculatorsand a mathematicscurriculum tressingproblemsolving techniques.DissertationAbstractsInternational,54, 0404A. (UMI No. AAC91-18846)Ansley, T., Spratt,K., & Forsyth,R. (1989). The effects of usingcalculators o reduce thecomputa-tionalburdenon a standardizedestof mathematics roblem olving.EducationalandPsychologicalMeasurement,49, 277-286.Autin,N. P. (2001). The effects of graphingcalculatorson secondarystudents'understanding f theinverse rigonometricunctions.DissertationAbstracts nternational, 2, 0890A. (UMINo. AAT30-09261)Bartos,J.J. (1986). Mathematicsachievementandthe use of calculators ormiddleelementarygradechildren.DissertationAbstractsInternational,47, 1227A. (UMI No. AAC86-14912)Boudreau,J. (1999). A treatment n middleschool students o increasestudentachievementongeom-etryskills (Master's hesis,CaliforniaStateUniversity, 1999).MastersAbstracts nternational,37,1297A.Bridgeman,B., Harvey,A., & Braswell,J. (1995). Effects of calculatoruse on scoreson a test of math-ematicalreasoning.Journalof EducationalMeasurement, 2, 323-340.Burnett,C. M. (1985). Theeffectiveness of usinga hand-heldcalculatoras an instructional id in theteaching of fraction to decimalconversion to sixth-gradestudents(Doctoraldissertation,BostonUniversity, 1985).DissertationAbstractsInternational,46, 2174A.Chandler,P. A. (1992). The effect of the graphingcalculatoron high school students'mathematicalachievement (Doctoral dissertation, University of Houston, 1992). Dissertation AbstractsInternational,53, 3832A.Colefield, R. P. (1985). The effect of the use of electroniccalculatorsversushandcomputationonachievement in computationalskills and achievement in problem-solvingabilities of remedialmiddle school students n selected businessmathematics opics (Doctoraldissertation,New YorkUniversity, 1985).DissertationAbstractsInternational,46, 2168A.Devantier,A. (1992). The impactof graphingcalculatorson theunderstanding f functions andtheirgraphs(Master's thesis,CentralMichiganUniversity, 1992).MastersAbstractsInternational,31,535A.Drottar, .(1999). Ananalysisof the effect of thegraphing alculatoronstudentperformancenalgebraII (Doctoraldissertation,BostonCollege, 1999).DissertationAbstractsInternational,60, 56A.Edens, H. S. (1983). Effects of the use of calculators on mathematicsachievement of first gradestudents Doctoraldissertation,Universityof Virginia, 1983).DissertationAbstracts nternational,43, 3248A.

8/7/2019 30034795

30/32


Ellerman,T. (1998). A studyof calculatorusageon the mathematics chievementof seventhandeighthgrade studentsand on attitudes of students and teachers(Doctoraldissertation,Louisiana TechUniversity,1998).DissertationAbstractsInternational,59, 1101A.Fleener,M. (1988). Usingthecomputeras aninstructional ool to aid inmathematical roblemsolving

(Doctoraldissertation,Universityof NorthCarolina,1988).Dissertation AbstractsInternational,50, 0611A.Frick,F. A. (1988). A studyof the effect of theutilizationof calculatorsand a mathematics urriculumstressing problem-solving techniques on studentlearning(Doctoral dissertation,University ofConnecticut,1988).Dissertation AbstractsInternational,49, 2104A.Giamati,C. M. (1991). The effect of graphingcalculatoruse on students'understandingf variationsonafamilyof equationsandthe transformationsf theirgraphs.DissertationAbstracts nternational,52, 0103A. (UMINo. AAC91-16100)Glover,M. (1992). The effect of the hand-heldcalculatoron the computationand problem solvingachievementof studentswith learningdisabilities(Doctoraldissertation,StateUniversityof NewYork atBuffalo, 1991).DissertationAbstractsInternational,52, 3888A.Graham,A., & Thomas,M. (2000). Buildinga versatileunderstanding f algebraicvariableswith agraphiccalculator.EducationalStudiesinMathematics,41, 265-282.Graham,A., & Thomas,M. (2000). A graphiccalculatorapproachto understanding lgebraic vari-ables. Paperpresentedat the TIME 2000 InternationalConference,Auckland,New Zealand.Hall,M. K. (1992). Impactof thegraphingcalculatoron the instructionof trigonometricunctions nprecalculus classes (Doctoral dissertation, Baylor University, 1992). Dissertation AbstractsInternational,54, 0451A.Harskamp,E., Suhre,C., & VanStreun,A. (1998). Thegraphicscalculator n mathematics ducation:An experiment n the Netherlands.HiroshimaJournalof MathematicsEducation,6, 13-31.Heath,R. D. (1987). The effects of calculatorsandcomputerson problemsolving ability, computa-tionalability,and attitudeowardsmathematicsDoctoraldissertation,NorthernArizonaUniversity,1987).DissertationAbstractsInternational,48, 1102A.Hersberger, . (1983). The effects of a problem-solvingorientedmathematicsprogramon giftedfifth-gradestudents.DissertationAbstractsInternational,44, 1715A.(UMI No. AAC83-23998)Humphries, . H. (1986). The effectiveness of the Little Professorcalculatoras asupplemento kinder-gartenmathematics rogramDoctoraldissertation,EastTexas StateUniversity,1986).DissertationAbstractsInternational,47, 2014A.Kelly, M. G. (1984). The effect of the use of the hand-held alculatoron thedevelopmentof problem-solving strategies (Doctoral dissertation,Utah State University, 1984). Dissertation AbstractsInternational,46, 2169A.Langbort,C. R. (1983). An investigationof theabilityof fourthgradechildren o solve wordproblems

using hand-heldcalculators(Doctoraldissertation,Universityof California, 1983). DissertationAbstractsInternational,43, 2914A.Lawrence,I., & Dorans,N. (1994). Optionaluse of calculatorson a mathematical est:Effecton itemdifficultyand score equating. (ReportNo. ETS-RR-94-40).Princeton,NJ: EducationalTestingService. (ERICDocumentReproductionServiceNo. ED382656)Lesmeister,L. (1997). The effect of graphingcalculatorson secondarymathematicsachievement(Master'sthesis, Universityof Houston ClearLake, 1996). Masters AbstractsInternational,35,0039A.Lewis, J., & Hoover,H. D. (1983). Theeffectonpupil performanceof usinghand-heldcalculators onstandardizedmathematicsachievement ests.Paperpresentedatthe annualmeetingof theNationalCouncil on Measurementn Education,Los Angeles, CA. (ERICDocumentReproductionService

No. ED204152)Lim, C. (1992). Effects of usingcalculatorson computationalabilityon non-collegeboundstudents,their attitudestoward mathematicsand achievementson unit tests in probabilityand statistics(Doctoraldissertation,Temple University,1991).DissertationAbstractsInternational, 2, 2450A.Liu, S. T. (1993). Effects of teachingcalculatoruse andproblem solving strategieson mathematicsperformance nd attitudeof fifthgradeTaiwanese male and female students Doctoraldissertation,MemphisStateUniversity,1993).DissertationAbstractsInternational,54, 4354A.

8/7/2019 30034795

31/32


Lloyd,B. (1991). Mathematics estperformance:Theeffects of itemtypeandcalculatoruse.AppliedMeasurementn Education,4, 11-22.Long,V. M., Reys, B., & Osterlind,S. J. (1989). Usingcalculatorson achievement ests.MathematicsTeacher,82, 318-325.Magee,E. F. (1985). The use of theminicalculator s aninstructional ool inapplyingconsumermath-ematics concepts (Doctoral dissertation,University of Virginia, 1985). Dissertation AbstractsInternational,47, 2104A.Merckling,W. (1999).Relationship(s) etweentheperceptual references f secondary chool studentsand their achievement n functionsusing a graphingcalculator(Doctoraldissertation,St. John'sUniversity, 1999).DissertationAbstractsInternational,60, 371A.Morgan,R., & Stevens,J. (1991). Experimental tudy of theeffects of calculator use on the advancedplacementcalculus examinations. ReportNo. ETS-RR-91-5).Princeton,NJ: EducationalTestingService. (ERICDocumentReproductionServiceNo. ED392816)Ottinger,T. P. (1994). Conceptualandprocedural earning n first-yearalgebrausinggraphingcalcu-latorsandcomputers Doctoraldissertation,GeorgiaStateUniversity,1993).DissertationAbstracts

International,54, 2934A.Pennington,R. (1998). A studyto determine he effect of instructionneffective use of a calculatorontest scores of middle school students(Master's thesis, Salem-Teikyo University, 1998). (ERICDocumentReproductionService No. ED 434030)Rich, S. B. (1991). The effect of the use of graphingcalculatorson the learningof functionconceptsinprecalculusmathematicsDoctoraldissertation,Universityof Iowa,1990).DissertationAbstractsInternational,52, 0835A.Riley, A. G. (1992). Effects of integratingTI MathExplorercalculators nto the grades3 through6mathematics urriculum nachievementandattitudeDoctoraldissertation,Texas A&MUniversity,1992).DissertationAbstractsInternational,54, 1661A.Rodgers,K. (1996). The effects on achievement,retentionof mathematicalknowledge,and attitudes

towardmathematics s aresultof supplementinghe traditional lgebra Icurriculumwithgraphingcalculator ctivities Doctoraldissertation, outhern llinoisUniversity,1995).DissertationAbstractsInternational,57, 0091A.Ruthven,K. (1990). The influence of graphiccalculatoruse on translation romgraphicto symbolicforms.EducationalStudies in Mathematics,21, 431-450.Ryan,W.(1999). Theeffectsof usingtheTI-92calculator o enhance uniorhighstudents'performancein and attitudetowardgeometry. (ERICDocumentReproductionService No. ED436414)Scott, B. A. (1994). The effect of graphingcalculators n algebraII classrooms:A studycomparingachievement, attitude,and confidence (Doctoraldissertation,Universityof NorthTexas, 1994).DissertationAbstractsInternational,55, 2755A.Siskind, T. (1994). The effect of calculator use on mathematicsachievementfor ruralhigh school

students.RuralEducator,16 (2), 1-4.Stacey, K., & Groves, S. (1994). Calculators in primarymathematics.Paper presentedat the 72ndmeeting of the NationalCouncilof Teachers of Mathematics, ndianapolis, N. (ERICDocumentReproductionService No. ED373963)Starr,S. (1989). The effects of includingcalculators on the problem-solvinginstructionfor low-income sixth-grade students (Master's thesis, Mercer University, 1989). (ERIC DocumentReproductionService No. ED309070)Szetela, W., & Super,D. (1987). Calculatorsand instruction n problemsolving in grade7. Journalfor Research in MathematicsEducation, 18, 215-229.Upshaw,J. T. (1994). The effect of thecalculator-based, raph-explorationmethodof instructiononadvancedplacement alculusachievementDoctoraldissertation,University f SouthCarolina,1993).

DissertationAbstractsInternational,54, 4023A.Vazquez,J.L. (1991).The effect of the calculatoron studentachievement ngraphing inear unctions.DissertationAbstractsInternational,51, 3660A. (UMI No. AAC91-06488)Weber,T. (1999). Graphing echnologyand its effect on solving inequalities.DissertationAbstractsInternational,60, 88A. (UMI No. AAC99-15749)

8/7/2019 30034795

32/32


Whisenant,M. A. (1989). The effects of the use of calculators n algebraI classes on basic skills main-tenance and algebra achievement (Doctoral dissertation, University of North Texas, 1989).DissertationAbstractsInternational,54, 0450A.Wilkins,C. (1995). The effect of thegraphing alculatoron studentachievement nfactoringquadratic

equations (Doctoral dissertation, Mississippi State University, 1995). Dissertation AbstractsInternational,56, 2159A.

30034795

Documents