the effect of promoting heuristic literacy on the mathematical aptitude of middle-school students
TRANSCRIPT
This article was downloaded by: [Western Kentucky University]On: 17 October 2014, At: 07:08Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
International Journal of MathematicalEducation in Science and TechnologyPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/tmes20
The effect of promoting heuristicliteracy on the mathematical aptitudeof middle-school studentsB. Koichu a , A. Berman a & M. Moore aa Technion – Israel Institute of Technology , IsraelPublished online: 03 Jul 2007.
To cite this article: B. Koichu , A. Berman & M. Moore (2007) The effect of promoting heuristicliteracy on the mathematical aptitude of middle-school students, International Journal ofMathematical Education in Science and Technology, 38:1, 1-17
To link to this article: http://dx.doi.org/10.1080/00207390600861161
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
International Journal of Mathematical Education inScience and Technology, Vol. 38, No. 1, 15 January 2007, 1–17
The effect of promoting heuristic literacy on the
mathematical aptitude of middle-school students
B. KOICHU*, A. BERMAN and M. MOORE
Technion – Israel Institute of Technology, Israel
(Received 6 July 2005)
Heuristic literacy – an individual’s capacity to use heuristic vocabulary in discourseand to apply the selected heuristics to solution of routine and non-routinemathematical tasks – was indirectly promoted in a controlled five-monthclassroom experiment with Israeli 8th grade students (N¼ 92). The experimentachieved a moderate mean effect size, which is in line with some previous researchon heuristics. The novel result of the study is that those students of the experimentalgroup who were below sample average at the beginning of the experiment benefitedfrom the heuristically-oriented intervention significantly more than the rest of thestudents. It is argued here that this is, in part, due to communicational aspectsof the intervention.
1. Introduction
Doing mathematics includes adaptation and applying a variety of heuristics –systematic approaches to representation, analysis and transformation of problems,used by a solver of those problems in planning and monitoring a solution [1–3].Universal heuristics, such as ‘decompose a problem’, ‘think from the end to thebeginning’ or ‘find first what is easy to find’, are often treated as cognitive tools that(good) problem solvers should possess [1–6].
The idea of teaching universal heuristics as a means of enhancing themathematicalachievements of students has attracted educators for at least thirty years [5–10].During the 1970s and 1980s, many researchers experimented with particularheuristics, but, even though some promising results appeared, the high expectationswere not sufficiently well backed up [9, 10]. For instance, Schoenfeld [3, 9] developedand tested a pedagogical approach based on decomposing universal heuristics intodomain-specific, prescriptive procedures, which can be used by students along withadequate problem-solving recourses. On the one hand, the approach appearedsuccessful in terms of the developed (non-standardized) measures. On the other hand,Schoenfeld pointed out that teaching a great number of domain-specific heuristicsin a classroom is a daunting task [9]. He also enumeratedmany limitations, which wereembedded in the implemented research methodology [3].
*Corresponding author. Email: [email protected]
International Journal of Mathematical Education in Science and TechnologyISSN 0020–739X print/ISSN 1464–5211 online � 2007 Taylor & Francis
http://www.tandf.co.uk/journalsDOI: 10.1080/00207390600861161
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
Based on meta-analysis of many pedagogical experiments, Hembree [10] foundthat heuristic training typically leads to modest and moderate mean effect sizes. In acritical response to Hembree’s paper, Goldin [11] suggested that the effectiveness ofteaching heuristics crucially depends on many classroom-related conditions as well ason the quality of the research instruments implemented. Recently, the ongoingchallenge of making heuristics work has been reconsidered in light of the reformmovement in mathematics education [e.g. 4, 12]. The questions ‘How should thesestrategies be taught? Should they receive explicit attention, and how should they beintegrated with the mathematical curriculum?’ reappear in Principles and Standardspublished by the NCTM [4, p. 54].
In this paper we report on a classroom experiment conducted during the2001–2002 school year as a part of the PhD study of the first author [13]. We showthat heuristic literacy – an individual’s capacity to use heuristic vocabulary indiscourse and to apply the selected heuristics to the solution of routine andnon-routine tasks – can be effectively promoted in a regular classroom through aseries of deliberately organized practices and communications. The measure of theeffectiveness of the classroom experiment was the mathematical part of theScholastic Aptitude Test (SAT). The main purpose of this paper is to justifythe cause–effect relationship between promoting heuristic literacy and SAT scoresand report on an interesting phenomenon, which, to our knowledge, is not describedin the literature: those students of the experimental group, whose mathematicalaptitude was below sample average at the beginning of the heuristically-orientedintervention, improved their SAT scores significantly more than other students.
2. Experimental design
A conservative pre-post quasi-experimental design with a control group isimplemented in order to estimate the mean effect size of the intervention [14, 15].Wilkinson et al. [16] offer the term contrast group instead of control group in studieswhen neither randomization nor total control of variables that might modify effectscan be realized, as in our research. They point out that in such studies: (i) thetreatment(s) should be operationally defined; (ii) the effect(s) of covariant variablesshould be reported and adjusted by relevant statistical analysis. We closely followthese recommendations, as follows.
(i) Twenty-five lessons in each experimental class and six lessons in the contrastclasses are observed and discussed with the teachers who conducted the lessons.Field notes made by the first author and a portfolio of the teaching materialshelp us to operationally define the intervention and the parallel treatment.
(ii) As will be shown, the contrast and experimental groups were remarkably similarat the beginning of the experiment. Furthermore, the between-classes differencesin pretest scores are adjusted using multiple regression analysis [17, 18].
3. The instruments
Mathematical aptitude is a dependent variable of the experiment, and themathematical part of the Scholastic Aptitude Test (SAT) is used to measure it.The SAT consists of 35 items related to algebra, geometry, series and word problems;
2 B. Koichu et al.
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
the test is given for 30 minute work. Many studies of SAT indicate its high internal
consistency, predictive, construct and content validity [19]. We argue that SAT is a
relevant instrument in a study concerning heuristics. Indeed, Stanley and Benbow
[20] and Zohar [21] point out that even though SAT consists of multiple-choice tasks,
it is likely to activate middle-school students’ mathematical reasoning. To succeed in
SAT, students can try to reduce the amount of technical work by constructing
heuristically-loaded shortcut solutions. We illustrate this point with an example
(figure 1). Before reading the following explanations, we invite the reader to spend no
more than 1–2 minutes, as in the real test, attempting the task without a calculator.When we piloted this task with a group of 7th graders, most of the students
started by multiplying 4 (the last digit in AB4) by 8 and found that C¼ 2. They then
failed when trying to multiply straightforwardly AB4 by 8. Some of the students
acted differently. They also started from a find-what-is-easy-to-find strategy, that is,
found C¼ 2, but then they wisely used this result in order to simplify the rest of
the solution. They divided 5392 by 8, and found that A¼ 6 and B¼ 7. This think-
from-the-end-to-the-beginning strategy appeared to be more effective than the first,
straightforward, one. In summary, the more sophistication is involved, the less
technical performance is needed in order to handle this task, and this idea is in line
with the intended intervention.In addition to SAT, a background questionnaire and a Progressive Matrix Test
(PMT) are used at the pretest. The background questionnaire provides us with
information needed to control factors for interpretation of the scores in the other
tests. The PMT is made up of 30 odd items from the well-known 60-item Raven’s
Progressive Matrix Test and is given for 20 minute work. Raven et al. [22] reported a
significant correlation between PMT and the full Wechsler test; therefore it provides
a good indication of general intelligence. PMT also enables us to describe the
research sample in comparison with hypothetically true norms, that is, with the PMT
results of representative samples [22]. Following Carpenter et al. [23], we claim that
PRT is a relevant instrument in research concerning heuristics. Indeed, Carpenter
et al. argue that this test measures ‘the common ability to decompose problems into
manageable segments and iterate through them, the differential ability to manage the
hierarchy of goals and subgoals generated by this problem decomposition . . . ’ ([23],
p. 429).The posttest (POST) consists of a questionnaire, SAT-2, developed for this
research as an instrument parallel to SAT. SAT-2 has the same structure as SAT,
consists of 35 items and includes the same sorts of problems. In order to justify the
use of SAT-2, we combined it with SAT and gave the 70-item test to 381 students.
A, B and C are the digits that fit the following product. Find their sum A+B+C.
a) 5 b) 7 c) 9 d) 13 e) 15
4BA
8
C935
×
Figure 1. An example of a SAT item.
Effect of heuristic literacy on mathematical aptitude of school students 3
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
These students did not participate in the experiment, but were of similar background
and studied the same mathematics curriculum as the students who did. The test was
administered under similar conditions (the same week, equipment, instructions and
explanations) by trained supervisors, who were not regular teachers of the
examinees. The students were randomly divided into four groups, and each group
was given the 70-item test in one of four different orders of the items. The reshuffling
of the tasks was done in order to give all items a ‘fair’ chance of being approached
by many students. Then the following statistical parameters were calculated:
– The correlation between SAT and SAT-2 scores was 0.74 (p� 0.001).– The internal consistency of SAT-2, calculated by the Kuder–Richardson
formula 20 [17], was 0.78. For the sake of comparison, the internal consistency
of SAT was calculated and found to be 0.77.– The hypothesis ‘on average, a paired difference of SAT and SAT-2 scores equals 1’
was not rejected (t¼ 0.80; df¼ 380; p¼ 0.21).
Based on these findings, we define POST scores equal SAT-2 scores plus one, and
conclude that POST scores are equivalent to SAT scores.
4. Experimental and contrast groups at the beginning of the experiment
Four Israeli 8th grade classes (N¼ 92) from an underprivileged school district took
part in the experiment. They learned mathematics in accordance with the MOFET
curriculum. MOFET is an acronym of ‘Mathematics’, ‘Physics’ and ‘Culture’
in Hebrew. MOFET is an Israeli educational project for middle school students who
are interested in mathematics and science. In this project, the students have
five lessons of algebra and three lessons of geometry a week starting at 7th grade
(see [24] for details of the project).The decision of how to divide the four classes into experimental and contrast
group was based on the pretest results and on exploration of the participants during
the school year preceding the experiment. We now describe the observed similarities
and differences between the two groups. There were 37 students (17 boys and 20
girls) of the experimental group and 46 students (30 boys and 16 girls) of the contrast
group, who took part in both pretest and posttest. The quantitative information
reported here is based on these 83 students. The SAT and PMT means of the contrast
and experimental groups were remarkably close (see table 1). We also learn from the
PMT means that an average participant in the experiment was not exceptionally
gifted. Indeed, by comparing the PMT means with hypothetically true norms [22],
Table 1. The sample at the pretest: experimental versus contrast group.
PMT SATMeana (SD) Meanb (SD)
Experimental group 23.54 (2.58) 13.05 (4.8)Contrast group 22.97 (4.21) 13.09 (5.03)
aOut of 30.bOut of 35.
4 B. Koichu et al.
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
one can see that between 50% and 25% of a corresponding representative samplewould obtain higher scores than an average student who took part in the experiment.
In the rest of the paper, we refer to the teachers of the experimental group as E1and E2 and use the same notation for their classes, whenever this does not causeconfusion. Similarly, C1 and C2 denote either the teachers of the contrast group ortheir classes. All the teachers – E1, E2, C1 and C2 – had BSc degrees in mathematicsgained in the former USSR, and Israeli teaching certificates; they had all taughtmathematics in middle schools for more than 15 years. The four teachers taught thesame curriculum using the same mathematics textbooks. They also systematicallyattended the same teacher workshops organized by MOFET.
Based on observations carried out in the lessons of the teachers prior to theexperiment, we found that their teaching styles were similar with respect to a numberof characteristics. All the teachers started the observed lessons with a shortdiscussion about a previous homework, and then explained new material to thewhole class. In explanations the teachers often used a blackboard in order to showthe students solutions to a sample problem or an exercise. The demonstration wasusually followed by students’ individual work on selected tasks from the textbooks.Sometimes, the teachers or their stronger students presented the tasks on the board.Mostly, the students were not encouraged to talk. However, the teachers were alwayswilling to help by providing additional explanations and demonstrations. Thatteaching style is often referred to as a ‘traditional’ one. There is additionalinformation about the participants that influenced the experiment:
– Teachers E1 and C1 had broad experience in mentoring other teachers.– There were many distracting students in E1 class.– Teacher C1 said in one of the conversations: ‘There is nothing new for me in the
idea to teach heuristics in problem solving. I always do it; I just don’t use the word‘‘heuristics’’.’
– The mathematical achievements of the classes E1 and C1, measured eight monthsprior to the experiment, were very similar; the achievements of the classes E2 andC2 were also very similar. The first pair of classes demonstrated, on average, betterresults than the second pair [25].
In conclusion, splitting the sample into two groups (E1 and E2 versus C1 and C2)makes our quasi-experimental design very conservative. Needless to remark, all theteachers were aware of the comparative character of the experiment.
5. The intervention (E1 and E2)
During the five-month experiment many curricular-determined topics had to betaught, so the classroom intervention might not lead to reducing the amount ofmaterial covered. The mathematical content of the intervention was designatedby the teachers, and our role as researchers was to help the teachers to revealthe heuristic potential embedded in the given content. Our collaboration with theteachers can be described as a sequence of one-week cycles. Usually, the teacherstold us what subject matter they were going to teach during the next week,often indicating the pages and exercise numbers from the textbooks. Then wedesigned a 90 minute heuristically-oriented lesson, either in algebra or in geometry,along with each of the teachers. As a rule, the same activities were conducted in
Effect of heuristic literacy on mathematical aptitude of school students 5
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
both classes. After the lessons, which were conducted by E1 or E2 and observed bythe first author, we had follow up conversations with the teachers, either individuallyor together. During the conversations we discussed the heuristic aspects of thelessons, awkward situations, if any, and plans for the next heuristically-orientedlessons.
As we have mentioned above, the goal of the intervention was to promoteheuristic literacy. This general goal included three subgoals. The first subgoalwas to enable students and teachers to acquire a common heuristic vocabulary.The main effort toward this goal was made during the first month of theintervention. The vocabulary included ‘thinking forward,’ ‘thinking from the endto the beginning,’ ‘recalling related problems,’ ‘find what is easy to find,’ ‘create amodel,’ ‘eliminate,’ etc. Frequently, the names of heuristics arose in classroomdiscussions. For example, the expression ‘think from the end to the beginning’appeared in a geometry lesson (see the activity ‘Mouse in the maze’ in [26]), and thenbecame a commonly shared intimation, which was readily used by the students bothin algebra and geometry contexts. The students were encouraged to use the names ofthe heuristics when asked to reflect on problem solving (e.g. ‘I decomposed theproblem into three parts, attempted the last one and solved it. It was ‘‘thinking fromthe end to the beginning’’’). In addition, the teachers were encouraged to think outloud in front of the classes while solving (seemingly) unfamiliar problems. In time,heuristic vocabulary was internalized by many participants.
The second subgoal was to start using the heuristic vocabulary in actual problemsolving. This subgoal governed designing many classroom activities in the first partof the intervention. Many difficult problems were proposed for class work andhomework. Some of the tasks included the option of a stepwise help provided interms of the heuristic vocabulary. During the lessons, every student or small group ofstudents could challenge the problems on their own, but they also could take as manyheuristic hints as they needed in order to handle a problem. A weaker as well as astronger student could solve the most difficult problems, using a different number ofhints. For example, a 90 minute lesson was built on the following geometry problem.
Diagonals of a parallelogram ABCD intersect in a point O. The points M, N, Kand L are the intersection points of the bisectors in triangles ABO, BCO, CDO andDAO, respectively. What can you say about the quadrilateral MNKL? Formulatea conjecture and prove it.
The problem, which was very challenging to the 8th grade students, was given forsolution in pairs. A set of written hints – from more general to more specific – wasavailable for each pair; each pair could take as many hints as they wanted andwhenever they wanted. The first three hints were:
Hint 1: A (good) picture can help! Draw a large picture using a ruler. Consider:Do you really need to draw all 12 bisectors mentioned in the problem?
Hint 2: Look at the picture [a computer-made picture was provided] and assignthe points.
At this stage, the teacher conducted a short classroom discussion on the question‘What can you say about the inner quadrilateral?’ Eventually, the students decidedthat the conjecture ‘‘MNKL is a rhombus’’ is worth checking.
Hint 3: Try to plan a solution. Read the problem again and think forward!What do you need to know in order to prove that MNKL is a rhombus?
6 B. Koichu et al.
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
Consider what way seems to be more promising: to prove that the four sides areequal or to prove something about the diagonals?
The hints became more and more prescriptive (interested readers can find all thehints in [26]), and were ended with the following:
Hint 10: If you still cannot see a solution, remain calm and think backward!Try to put in good order all the stages you have done. Reread the problem.What given haven’t you used yet?
Hint 11: Raise your hand and call: ‘‘Help me!’’
In the two classes, nobody used all the hints. We observed that the activity triggereda special competing behaviour. Nobody could solve a problem with no help, butmany students tried to solve the problem using a smaller number of hints than their’rivals’. The lesson was concluded by a whole-class reflective discussion of theimplemented heuristics.
The third subgoal was to promote heuristic transfer – a capacity to use thelearned heuristics in relatively new situations. Many lessons were designed aroundpairs or triples of problems that we deemed mathematically different but heuristicallysimilar. By heuristically similar we mean problems that (potentially) evoke deep-level connections that could be expressed in terms of the same heuristics. Consider,for example, the following textbook problem:
ABCD is a trapezoid, AB kDC, AB5DC, AD¼BC, AE?DC, EF¼DB andEF kDB. Prove that AFCE is a rectangle.
This problem can be seen as heuristically similar to the above problem abouta parallelogram. The similarity can be found at the level of heuristics involved in thesolutions to the two problems. We leave the tasks of indicating these commonheuristics to the readers. The lessons deliberately aimed at teaching for heuristictransfer were conducted mainly in the second part of the intervention. Articulationof the strategies’ names was not encouraged deliberately at that stage. It is importantthat in the experimental group either weaker or stronger students were engaged indiscussions of the most difficult problems. The stronger (or quicker) students weretrained not to deprive the rest of the class of the pleasure of contributing. Instead,they were encouraged to share their ideas indirectly, as (heuristic) intimations.Interestingly, at the advanced stage of the intervention, the privilege to formulate‘the smallest possible help’ for classmates became an influential award for thestudents.
An additional important characteristic of the intervention is that many routineproblems and exercises were used in order to implicitly promote heuristic literacy.One illustration of this point can be found in [26]. Another illustration deals withteaching the distributive law in E1 and E2. The students learned how to transform aproduct of binomials into a canonic polynomial. They solved several tasks like thefollowing one: (x� 1) � (xþ 2)¼ x2þ 2x� x�2¼ x2þx� 2. Afterwards, the teacherasked the students to solve a similar task without writing the intermediate steps(e.g. (x� 2) � (xþ 5)¼ x2þ 3x� 10). This appeared to be a difficult task to manystudents. The challenge was to keep in memory an intermediate result, and then tomanipulate it mentally. Some students suggested how to bypass this difficulty.Eventually, the following strategy emerged from the classroom discussion.Before opening the brackets, one should consider the structure of the resultingpolynomial. In the above example, it should contain three components, so it is worth
Effect of heuristic literacy on mathematical aptitude of school students 7
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
decomposing the problem into three parts, thinking first of x-squared, then of x andthen of the x-free part. How can one obtain x-square? There is only onecombination, x � x¼x2, and one can write x2 as part of the answer. How can oneobtain x? There are only two combinations, 5x� 2x, and one can write 3x as a partof the answer. Similarly, only one combination contributes to a number in theresulting polynomial, �2 � 5¼�10, and thus the answer is found. We believe thatheuristics like ‘thinking from the end to the beginning’ or ‘thinking forward’ wereimplicitly trained in this lesson. In particular, the students were encouraged to thinkof the structure of a final answer prior to diving into technical details. It is interestingto note that after a little training, students were able to obtain without writing,a canonical polynomial given some products of three or more first-degree binomials.We invite the readers to ask their students to transform the following product toa canonical polynomial form: ‘(xþ 1) (x� 2) (xþ 4)¼ 222’.
We close this section with two remarks. First, both E1 and E2 teachers observedthat their students felt more comfortable when taking part in heuristic-orientedlessons than in more traditional lessons. The teachers related this to extensive useof work in pairs (mostly, in E1 class) or in small groups of 3–5 students (mostly,in E2 class), where the ‘slower’ students had more chances to be heard and tocontribute. This was particularly evident in class E2. In teacher E1’s words, ‘. . . thestudents began to make some stops and to think before doing. They don’t trysomething without evaluating if it is worth trying. It was not that way before [theexperiment]’. Second, the SAT sorts of tasks were excluded from teaching in E1 andE2 classes during the experiment, in order to avoid tutoring towards the post-test.It should be mentioned that in four classes the students and the teachers did not evenknow their pretest scores until the end of the experiment.
6. The parallel treatment
6.1. Class C1
From personal communication with C1 before the experiment, it was evident thatshe had a solid mathematical background and a very rich pedagogical experience.As we have mentioned above, the teacher told us before the experiment thatheuristics were not new to her (she had read ‘How to solve it?’ by Polya) and that shealways used them in teaching. From the observations, we learned that even thoughthe teacher did not pay (explicit) attention to problem-solving strategies and did notuse in her teaching the ideas described in the previous subsection, promotingheuristic literacy might implicitly occur in her lessons. For example, she system-atically encouraged her students to perform computations without writing or using acalculator. That might progress ‘thinking forward’ in the meaning discussed above.Now we consider in some detail one 90 minute geometry lesson, which, we believe,is quite representative.
C1 built the lesson around individual solving of relatively difficult geometryproblems from the textbook. Seven problems were given and later presented on theboard. The aforementioned problem ‘ABCD is a trapezoid, AB kDC, AB5DC,AD¼BC, AE?DC, EF¼DB and EF kDB. Prove that AFCE is a rectangle’ wasamong them. The teacher selected these problems as they all were about a trapezoidand about proofs. No attempts to build deep-level connections among the problemswere indicated.
8 B. Koichu et al.
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
The teacher asked her students to solve the above problem in more than one way,and several students were able to do so. However, the teacher did not use theirsolutions as a learning opportunity for the whole class, and just moved on to the nextproblem. It seems that the teacher asked for another solution in order to occupy her‘quicker’ students, while the ‘slower’ ones complete the first solution. In general,the voices of the majority of the students were not heard in the lesson. We observedthat only five students (out of 27) systematically took part in creating solutions tothe problems and presenting them on the board. Noticeably, they presented theproblems addressing the teacher rather than their peers. If there was a mistake on theboard, the teacher fixed it straightforwardly. There was no direct communicationbetween the presenting students and the rest of the class; the teacher was always amediator involved in one or more dialogues. Genuine whole-class or small-groupdiscussions did not occur. Some of the students copied solutions from the board, andothers did not. They did not ask questions. In two difficult cases, the teacherrecapitalized a solution on the board and asked: ‘Who does not understand?’ or‘Who wants me to explain that again?’ Silence was the answer, and the experiencedand caring teacher explained again. The teacher occasionally mentioned heuristicslike ‘draw a picture’ or ‘recall the related problem’ in her explanations, but thestudents did not have a chance to reflect on their solutions. However, thinking of thislesson through ‘heuristic lenses’, we suggest that a few students were challenged bythe given problems, and, perhaps, implicitly learned the associated domain-specificstrategies.
6.2. Class C2
Unlike C1, C2 did not consider the idea of teaching problem-solving strategiesbefore our conversations. However, from personal communication with the teacher,we learned that he liked challenging problems and problem solving. We now describea fragment of the algebra lesson, which, we believe, is quite representative. The topicof the lesson was ‘Algebraic expressions’. It began with the teacher’s question to thewhole class, and two students (St.1 and St.2) responded:
C2: What is an algebraic expression?
St.1: This is an exercise with a variable.
C2: What is a variable?
St.2: An algebraic expression with letters, which can be substituted withany number.
C2: OK, an algebraic expression is an expression with letters.
Discussion of the mathematical content of this short but interesting conversation isbeyond the scope and goals of this paper. We only want to note that St.2 ignoredthe actual question and addressed the first one. For this reason, the flow ofthe conversation was broken, at least, from the observer’s perspective. Indeed,C2 responded by a question to St.1, apparently, expressing his disagreement with heranswer. Then the teacher accepted and summarized the answer of St.2, in fact,ignoring the dialogue with St.1. We observe in this small piece of data two patternsof communication in C2 class: first, the students are expected to answer the teacher’squestions, but not to listen to each other, and second, the teacher is an authority
Effect of heuristic literacy on mathematical aptitude of school students 9
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
establishing what the ‘right answer’ is. Such patterns of communication certainlydo not promote heuristic literacy. Similar communicational patterns were alsoobserved in the experimental classes, especially at the beginning of the experiment,and one of the effects of the intervention was that they were gradually changed.
7. Results and analysis
7.1. Experimental versus contrast group
Adjusted SAT-2 scores named POST (see section 3) represent the post-test results.On average, at the post-test, the experimental group improved SAT scores by 4.27(SD¼ 4.07) points, and the contrast group by 2.72 (SD¼ 4.75). This means thatthe experiment achieved a moderate mean effect size of 0.35, calculated usingCohen’s [14] procedure.
In order to explain this finding, stepwise multiple regression analysis with thedependent variable POST was used. The following variables were considered aspotential predictors of the variance in POST:
– E – taking on value ‘1’ for the students of the experimental group and ‘0’otherwise,
– PMT scores,– SAT scores.
A significant (F¼ 29.45, df¼ 79, p� 0.001) regression equation (1) with theentered (p� 0.05) variables PRE, MAT and E provides a model that accounts for53% of the variance in the sample (R-square adjusted is 0.51):
POST ¼ 0:84þ 0:56ð0:073Þ
� SATþ 0:33ð0:10Þ
�PMTþ 1:35ð0:69Þ
�E ð1Þ
The first variable entered was SAT (43% of variance). Variable E was third inimportance. Still significant (p� 0.05), it accounted for an additional 2.3% ofvariance in comparison with the model that included only two predictors, SAT andPMT. This result is consistent with the reported mean effect size of the experiment.We also learn from equation (1) that, on average, membership in the experimentalgroup (E1 or E2) ‘promises’ higher post-test scores than membership in the contrastgroup (C1 or C2).
The next inquiry was this: which parts of the experimental and contrast groupssignificantly changed their scores at the post-test? The above analysis is not sensitiveto this inquiry, since Cohen’s [14] mean effect size and regression models do notdistinguish between a situation in which many relatively small changes aredistributed among many subjects, on the one hand, and a situation in which fewrelatively large changes are distributed among only a few subjects, on the other hand.To establish significant changes, we used the following criterion: a personal changeby more than one standard deviation of the SAT scores in the entire sample(4.8 points) was called significant by the sample-oriented criterion. By this criterion,17 students from the experimental group (46%) and 12 students from the contrastgroup (26%) increased their results significantly at the post-test; in both groups nostudents significantly decreased their results. In summary, the experimental groupimproved more that the contrast group, and this improvement was distributedamong a larger percentage of the students.
10 B. Koichu et al.
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
7.2. Between-classes differences
Descriptive statistics related to SAT scores at pre- and post-tests in four classes,
taken separately, are presented in table 2. We learn from table 2 that, on average,
the four classes improved their results, but not equally.The following question was naturally posed: which factors modify between-class
differences? In order to answer this question, six stepwise multiple regressions
with POST as dependent variable were computed – for the six pairs of classes: E1
and E2, E1 and C1, E1 and C2, E2 and C1, E2 and C2, C1 and C2. Again, potential
predictors of the variance in POST were SAT and PMT (see section 7.1).
In addition, three new variables were considered instead of the variable E:
– E1 – taking on value ‘1’ for E1 students and ‘0’ otherwise,– E2 – taking on value ‘1’ for E2 students and ‘0’ otherwise,– C1 – taking on value ‘1’ for C1 students and ‘0’ otherwise.
The use of E1, E2 and C1 as variables is consistent with their previous use as names
for the classes. Table 3 contains only the variables entered (p� 0.05) in the resulting
significant (p� 0.001) regression models. In terms of a regression analysis, these
variables are ‘responsible’ for the variances in POST scores.We learn from table 3 that explanations of between-class differences are partially
provided by variables related to the pre-test (SAT or PMT)—for all the pairs.
For example, on average, there was a difference between classes E1 and E2 with
respect to SAT. In addition, since the variable E1 did not enter the model by the pair
of classes E1 and E2, one can suggest that the learning trajectories in these classes
were similar. One can also see that there is an additional factor that isolates the class
C2 from classes E1, E2 and C1.The above between-class differences are summarized in a significant (p� 0.001)
regression model (2), which distinguishes the four classes instead of the experimental
and contrast groups. The variables E1, E2 and C1 entered (p� 0.05) into this model,
which accounts for 59% of variance in POST (F¼21.85, df¼77, R squared adjusted
is 0.56):
POST ¼ 1:55þ 0:47ð0:08Þ
� SATþ 0:28ð0:10Þ
�PMTþ 3:44ð0:95Þ
�E1þ 3:18ð0:98Þ
�C1þ 2:69ð1:07Þ
�E2 ð2Þ
Table 2. SAT and POST scores in the four classes.
SAT POSTMean (SD) Mean (SD)
Class (N) Rangea Rangea
E1 (24) 13.5 (5.34) 17.87 (4.29)5–27 8–28
E2 (13) 12.23 (2.05) 16.31 (3.22)9–15 10–20
C1 (26) 15.35 (5.13) 18.50 (4.20)5–23 10–26
C2 (20) 10.15 (3.27) 12.30 (2.70)5–17 8–17
aOut of 35.
Effect of heuristic literacy on mathematical aptitude of school students 11
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
The variable E1 entered third into the model and accounts for an additional 1.9%of the variance in comparison with the model that included the two predictors SATand PMT. The variables C1 and E2 entered fourth and fifth and accounted for anadditional 2.9% and 3.4% of the variance, respectively. We learn from equation (2)that, on average, membership in E1, C1 and E2 classes ‘promises’ higher post-testscores than membership in C2.
To recall, regression models are not sensitive to the inquiry ‘Which part of eachclass significantly changed their scores at the post-test?’. To address that question,we use the following criterion of significance: a personal change by more thanone standard deviation of the SAT scores calculated in each class, taken separately,is called significant by a class-oriented criterion (see table 2 for the exact numbersof the standard deviations at the SAT). By this criterion, nine E1 students (38%),ten E2 students (77%), five C1 students (19%) and six C2 students (30%)significantly increased their results; one E2 student significantly decreased hisresult. These findings are consistent with those reported at the end of section 7.2: theimprovements in classes E1 and E2 (especially in E2) are for a larger percentage ofthe students than in classes C1 and C2.
7.3. ‘Weaker’ versus ‘stronger’ students
The last inquiry was: did ‘weaker’ students improve their SAT scores more than‘stronger’ ones? This was of interest both in the experimental and in the contrastgroup. We use the following definition: a student is called Stronger if she or hereceived 14 or more points in the pretest, and Weaker otherwise (14 is the firstwhole number greater than the average 13.07 of SAT scores in the entire sample).This definition enabled us to split the sample into four subgroups:
– WE – Weaker students of the Experimental group,– SE – Stronger students of the Experimental group,– WC – Weaker students of the Contrast group,– SC – Stronger students of the Contrast group.
The variable DIF¼POST–SAT was calculated in WE, SE, WC and SCsubgroups, taken separately. Figure 2 depicts DIF in the four subgroups.
We learn from figure 2 that WE progressed the most. However, the questionremains whether or not the between-subgroup mean differences are significant at the0.05 level? Since ANOVA or equivalent regression models may only indicate whetheror not there are significant differences among the means and do not provide pairwisecomparison, we use Fisher’s least significant difference pairwise multiple comparisontest [18]. By Fisher’s test, the following pairs of means are significantly (p� 0.05)
Table 3. Variables that entered the significant regression modelsby pairs of classes.
Classes E2 C1 C2
E1 SAT SAT, PMT SAT, PMT, E1E2 SAT SAT, PMT, E2C1 SAT, PMT, C1
12 B. Koichu et al.
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
different: WE and SE, WE and SC, WC and SC, and WE and WC. The first threeresults are trivial (Weaker are indeed weaker than Stronger), but the last one is veryimportant: the students of the experimental group, who obtained SAT scores belowsample average at the beginning of the experiment, improved their mathematicalaptitude significantly more than other students.
8. Discussion
In this paper we reported on a pre–post quasi-experiment with an experimantal anduntreated contrast group concerning promoting heuristic literacy. Promotingheuristic literacy was based, in part, on ideas adapted from past research onmathematical problem solving [1–3, 5–9, 11]. We learned from many sources, yet wepoint out that the intervention described deviates essentially from some commonlyaccepted schemes of teaching and learning problem-solving strategies. Specifically,instead of decomposing universal strategies into more prescriptive substrategies[e.g. 3, 9], we concentrated on direct and indirect teaching of universal heuristics like‘thinking forward’ and ‘thinking from the end to the beginning’ by means ofrevealing the heuristic potential of routine as well as non-routine tasks in manycurriculum-determined contexts. We have shown that the described interventionimproved the mathematical aptitude of 8th graders as measured by the mathematicalpart of the Scholastic Aptitude Test (SAT). Furthermore, we have shown that thetreatment was more effective for those students in the experimental group whoobtained SAT scores below sample average at the beginning of the experiment.
The mean effect size of the five-month experiment calculated by Cohen’s [14]method is 0.35, which is a moderate effect. In his meta-analysis, Hembree [10]
Mea
n D
IF
7
6
5
4
3
2
1
0WE SE WC SC
5.92
1.23
3.92
1.15
N=24 N=13 N=26 N=20
Figure 2. Mean differences in adjusted SAT scores by subgroups WE, SE, WC and SC.
Effect of heuristic literacy on mathematical aptitude of school students 13
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
referred to comparable findings from studies focused on deliberate promotion ofguess-and-test strategy, verbalizing mathematical concepts and mathematicalvocabulary. Our experiment was focused on related issues, namely, planning andself-evaluation strategies, reflecting on problem solving and heuristic vocabulary,thus, there are no surprises at this point. We only want to note that the age ofthe participants was lower than in most studies about learning/teachingof problem-solving strategies. It is also important that the effect was reached in arelatively short time in real school conditions, not in a laboratory.
Is the positive effect of the experiment due to promoting heuristic literacy?Paraphrasing Schoenfeld [9], we can say that it should be, with respect to the appliedmethodology. Indeed, splitting of the sample into experimental and contrast groupsmade our quasi-experiment very conservative. In addition, let us recall that theSAT-like tasks were carefully avoided in the experimental group, in order not totutor the students for the post-test. However, since not more than 51% of thevariance in post-test scores was explained by the regression model (1), deeperanalyses were carried out.
A regression model (2), which distinguishes the four classes in the sample,explained 59% of the variance, but an overall picture became more intriguing at thislevel. In particular, we observed that the explanatory variables for the contrast classC2 were different from those for the other classes. In light of our knowledge aboutthe C2 teacher, we reject the speculation that he might have been less educated,experienced or caring than teachers E1, E2 and C1 as not supported by the facts athand. Wearing the ‘heuristic lenses,’ we suggest that the difference between teachersE1, E2 and C1, on the one hand, and teacher C2, on the other hand, is in theirapproaches to seizing learning opportunities in problem solving. Indeed, E1 and E2teachers put tremendous effort into promoting heuristic literacy during theexperiment, and, to some extent, changed their ‘natural’ teaching styles. In E1teacher’s words,
I often started some [heuristic-oriented] activity in your presence, andcontinued it for one or two additional lessons. At the beginning, my studentshad distinguished between ‘your’ and ‘my’ activities, but after a monthor two it wasn’t important for them, it did not work like ‘[with] heuristics’[in your presence] and ‘without heuristics’ [in your absence], not at all.
Teacher C1 seems to us to be a ‘natural’ provider of heuristic literacy. The fact thatthe experienced and knowledgeable teacher could promote heuristic literacy withoutour assistance does not weaken the claim that heuristic literacy is worth promoting.As we have shown, C2 is the least heuristically-oriented teacher among the four.
When these results were first presented in a research seminar, we asked theparticipants to suggest competing explanations. The first suggestion was thatthe C2 students may have been weaker than others with respect to SAT (see table 2),and thus it was more difficult for them to progress. This suggestion is rejected for thefollowing reasons. First, an additional ‘isolating’ factor appears in statistical models,in which the differences in SAT scores had been accounted for by the regressionequations, and, subsequently, neutralized. Second, the pre–post results of the‘weaker’ students of the experimental group, in particular, E2 students, provide agood counterexample for the above suggestion. Another suggestion was thatthe C2 students may have been weaker than others, and there may have beenfewer problem solvers with relatively powerful heuristic arsenals in the class to learn
14 B. Koichu et al.
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
from; thus it was more difficult for relatively weaker students to progress.Comparison of the pretest’s standard deviations and ranges of scores in E2 andC2 classes (see table 2) disprove this suggestion: the parameters related to the classes’heterogeneity are very similar, yet the amounts of post-test progress are verydifferent.
The next finding concerns distribution of changes in mathematics aptitude withinthe four classes. Even though class C1 was hardly distinguishable from E1 and E2 interms of regression analysis, they were very different with respect to percentages ofstudents who significantly improved their results at the post-test. The percentage ofC1 students who significantly improved their SAT scores by two different criteriais smaller than in both experimental classes. In comparison with class C1, thepercentage of such students in class C2 is more sizable, but still smaller than in bothexperimental classes. Further, the highest percentage of students who significantlyimproved their mathematical aptitude is in class E2. We observed that this ascendingorder of the percentages, C1, C2, E1 and E2, corresponds to the extent by whichproblem-solving discourse among students was encouraged and practised in theclasses.
The most novel and promising finding of the study is that students of theexperimental group who obtained SAT scores below sample average at the beginningof the experiment (the so called ‘weaker’ students), improved their mathematicalaptitude more significantly than other students, including ‘weaker’ students of thecontrast group. We offer the following two-part explanation. First, the interventiondealt with those heuristics and habits of problem solving, which were more typicalfor successful problem solvers. Hence, we suggest that promoting heuristic literacywas more novel and useful for ‘weaker’ problem solvers. They had sharpened theirproblem-solving skills by practising new strategies, whereas the strongest studentsmight have possessed these strategies prior to the experiment. Second, manyheuristic-oriented activities were designed so that the ‘weaker’ students had a betterchance to be heard and to contribute to the whole-class discussions than during theirregular classes. This might help less confident or ‘slower’ students to increase theirconfidence and, in turn, to advance them from attempts to understand the solutionsby others to attempts to produce their own ideas. This explanation is consistent withthe Vygotskian theory of the role of social context in the process of learning, whichSchoenfeld [27] applied to exploration of mathematical problem solving.
9. Conclusions, limitations and suggestions for future research
The implemented quasi-experimental design prescribes modesty in the results’interpretations and generalizations [17]. Though we have explored and rejected somenon-heuristic explanations of the results, we cannot be sure that other explanationsor some combination of the listed ones do not disprove the conclusion that we weretrying to support: namely, that promoting heuristic literacy improved themathematical aptitude of 8th graders. The treatment had a greater effect on thosestudents of the experimental group who obtained below sample average SAT scoresat the beginning of the experiment.
From the long history of pedagogical experimentation dealing with mathematicalproblem-solving, one can infer that any success in teaching/learning heuristics is bothpromising and challenging. There is a promise: the moderate success of the reported
Effect of heuristic literacy on mathematical aptitude of school students 15
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
experiment suggests that, under appropriate circumstances, promoting heuristicliteracy can be a vehicle for decreasing differences between ‘stronger’ and ‘weaker’students. It is also important that promoting heuristic literacy can be done in a realclassroom, using both routine and non-routine tasks that can be found in manymathematical curricula. There is also a challenge: to replicate and extend the researchwith other teachers and students, perhaps, of different backgrounds and ages, and tosee whether the reported phenomena is observed again. It is particularly interestingto investigate if additional measures of mathematical aptitude would be as sensitiveto the described promoting heuristic literacy as SAT was.
Acknowledgements
The research of the first author was supported by the SAKTA RASHI Foundation(budget No 765-076-00) and by the Technion Graduate School. The research of thesecond author was supported by the Fund for Promotion of Research at theTechnion.
We are grateful to the teachers of MOFET, the association that has enabled usto gather data for this study, for their enthusiasm, professionalism and cooperation.
References
[1] Goldin, G., 1998, Representational systems, learning, and problem solving inmathematics. Journal of Mathematical Behavior, 17(2), 137–165.
[2] Verschaffel, L., 1999, Realistic mathematical modeling and problem solving.In: J. H. M. Hamers, J. E. H. Van Luit and B. Csapo (Eds) Teaching and LearningThinking Skills (Lisse: Swets & Zeitlinger), pp. 215–239.
[3] Schoenfeld, A., 1985, Mathematical Problem Solving (New York: Academic Press).[4] NCTM (National Council of Teachers of Mathematics), 2000, Principles and Standards
for Teaching Mathematics (Reston, VA: NCTM).[5] Polya, G., 1945/1973, How to Solve it (Princeton, NJ: Princeton University Press).[6] Leinhardt, G. and Schwarz, B., 1997, Seeing the problem: an explanation from Polya.
Cognition and Instruction, 15, 395–434.[7] Mayer, R.E., 1992, Thinking, Problem Solving, Cognition, 2nd edn (New York: Freeman).[8] Schoenfeld, A., 1979, Explicit heuristic training as a variable in problem-solving
performance. Journal of Research in Mathematical Education, 10, 173–187.[9] Schoenfeld, A., 1992, Learning to think mathematically: problem solving, metacognition,
and sense-making in mathematics. In: D. Grouws (Ed.), Handbook for Research onMathematics Teaching and Learning (New York: MacMillan), pp. 334–370.
[10] Hembree, R., 1992, Experiments and relational studies in problem solving:a meta-analysis. Journal for Research in Mathematical Education, 23, 242–273.
[11] Goldin, G., 1992, Meta-analysis of problem-solving studies: a critical response. Journalfor Research in Mathematical Education, 23, 274–283.
[12] Mauch, E. and Shi, Y., 2003, One approach to explore a range of problem solvingstrategies in a classroom. International Journal of Mathematical Education in Science andTechnology, 35, 912–917.
[13] Koichu, B., 2003, Junior high school students’ heuristic behaviors in mathematicalproblem solving. PhD Thesis, Technion – Israel Institute of Technology.
[14] Cohen, J., 1988, Statistical Power Analysis for the Behavioral Sciences, 2nd edn(New Jersey: Lawrence Erlbaum).
[15] Cook, T. and Campbell, D., 1979, Quasi-experimentation. Design and Analysis Issues forField Settings (Chicago: Rand McNally).
[16] Wilkinson, L. and Task Force on Statistical Inference, 1999, Statistical methods inpsychology journals: Guidelines and explanations. American Psychologist, 54, 594–604.
16 B. Koichu et al.
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014
[17] Guilford, J. and Fruchter, B., 1973, Fundamental Statistics in Psychology and Education(New York: McGraw-Hill).
[18] Winer, B., 1971, Statistical Principles in Experimental Design (New York: McGraw-Hill).[19] Young, J. and Kobrin, J., 2001, Differential validity, differential prediction, and college
admission testing: A comprehensive review and analysis. Research report No. 2001-6.Available online at: http://www.collegeboard.com/repository/differential_validity_10539.pdf (Accessed 24 June, 2005).
[20] Stanley, J. and Benbow, C., 1981–82, Using the SAT to find intellectually talented seventhgraders. College Board Review, 122, 2–7, 26.
[21] Zohar, A., 1990, Mathematical reasoning ability: Its structure, and some aspects of itsgenetic transmission. PhD thesis, Hebrew University, Jerusalem.
[22] Raven, J., Court, J. and Raven, J., 1983, Manual for Raven’s Progressive Matrices andVocabulary Scales (London: Lewis).
[23] Carpenter, P., Just, M. and Shell, P., 1990, What one intelligence test measures:A theoretical account of the processing in the Raven Matrices Test. Psychological Review,97, 404–431.
[24] Schneiderman O., Levit E., Marcu L., & Zakharova A., 2003. From Mediocrity toExcellence: The MOFET Group for the Advancement of Teaching. In: E. Velikova (Ed.)Proceedings of the 3rd International Conference ‘Creativity in Mathematics Education andthe Education of Gifted Students’ (pp. 144–149). (Rousse, Bulgaria, University of Rousse).
[25] Koichu, B., 2001. Evaluation of mathematics education in secondary school classesof the project MOFET. Technical report to Ort Israel and Association MOFET(in Hebrew).
[26] Koichu, B., Berman, A. and Moore, M., 2004, Promotion of heuristic literacy in a regularmathematics classroom. For the Learning of Mathematics, 24(1), 33–39.
[27] Schoenfeld, A., 1987, What’s all the fuss about metacognition? In: A. Schoenfeld (Ed.)Cognitive Science and Mathematics Education (Hillsdale, NJ: Lawrence Erlbaum),pp. 189–215.
Effect of heuristic literacy on mathematical aptitude of school students 17
Dow
nloa
ded
by [
Wes
tern
Ken
tuck
y U
nive
rsity
] at
07:
08 1
7 O
ctob
er 2
014