mathematical literacy of school leaving pupils in south africa

International Journal of Educational Development 22 (2002) 603–615www.elsevier.com/locate/ijedudev

Mathematical literacy of school leaving pupils in SouthAfrica�

S. Howiea,1,, T. Plompb,*

a Faculty of Education, University of Pretoria, 0002 Pretoria, South Africab Faculty of Educational Science and Technology, University of Twente, Enschede, Netherlands

Abstract

This paper discusses some results of South African (SA) grade 12 pupils on an international test of mathematicalliteracy, administered in the framework of the Third International Mathematics and Science Study (TIMSS) under theauspices of the International Association for the Evaluation of Educational Achievement (IEA). Three questions areaddressed: (1) What are the strengths and weaknesses of SA school-leavers in mathematical reasoning and social utilityfrom an international comparative perspective?: (2) What is the growth of SA pupils’ mathematical literacy from grade8 to 12? (3) What are the background variables that influence the level of mathematical literacy of SA school-leavers?Finally some implications of the results for SA education will be discussed. 2002 Published by Elsevier Science Ltd.

Keywords:Comparative education; Literacy; Educational policy; Curriculum; Mathematics education

1. Introduction

In this paper some of the results of SouthAfrican (SA) grade 12 pupils on an internationaltest of Mathematical Literacy are discussed. Thetest was administered in the framework of theThird International Mathematics and Science Study(TIMSS), which was conducted under the auspices

� An earlier version of this article has been presented at theannual conference of the European Conference on EducationalResearch (ECER’99), Lahti, Finland, 22–25 September 1999.

* Corresponding author. Tel.:+31-53-489-3595; fax:+31-53-489-3759.

E-mail addresses:[email protected] (S. Howie),[email protected] (T. Plomp).

1 At the time of conducting the research the first author wasworking for the Human Sciences Research Council of SouthAfrica.

0738-0593/02/$ - see front matter 2002 Published by Elsevier Science Ltd.PII: S0738 -0593(01 )00030-X

of the International Association for the Evaluationof Educational Achievement (IEA). The SA datawere collected in 1995 by the Human SciencesResearch Council (HSRC) and the first results werepublished nationally by Howie and Hughes (1998)and internationally by Mullis et al. (1998).

International comparative studies might have avariety of functions for educational policy makers,practitioners and researchers (Plomp, 1998). Onesuch function is themirror function that is thedescription of the national results in an inter-national context. Other functions relate tobench-marking and monitoring. Both refer to examiningand analysing the data, but from different angles.Benchmarking refers to the possibility of analysinginformation about the status of the achievement ofpupils in certain subject areas compared to theresults of one or more ‘relevant’ other countries or

604 S. Howie, T. Plomp / International Journal of Educational Development 22 (2002) 603–615

the results in the country of interest in an earlierstudy. In contrast, the monitoring angle impliesthat the data are also analysed to formulate re-commendations for changes when and whereneeded (‘ informed decision making’ ). Anotherfunction is to analyse data with the intention tounderstand reasons for observed performanceseither in a national context, or in an internationalcomparative perspective.

In this paper the SA results on the TIMSS Math-ematical Literacy test will be analysed and dis-cussed from the perspective of some of the func-tions mentioned. Specifically, the followingresearch questions will be addressed:

1. What are the strengths and weaknesses of SAschool leavers in mathematical reasoning andsocial utility? A good description of this pro-vides one with a mirror of how well SA pupilsare performing in important aspects of their uti-lisation of the mathematics they learn in school.

2. What is the growth of SA pupils’ literacy inmathematics from grade 8 to grade 12?By ta-king the grade 8 results as benchmark, insightcan be obtained into the effectiveness of math-ematics teaching and learning during the lastfour years of comprehensive education.

3. Which if the pupil background variables(including language) are related to or influencethe level of mathematical literacy of SA school-leavers?Understanding of some of these factorsmay be a starting point for development projectsaimed at the improvement of mathematics andscience education in South Africa.

In Section 2 some key aspects of SA educationwill be highlighted, thereby providing a context forinterpreting the SA results in this study. Section 3presents information about the IEA and TIMSS.Section 4 summarises the design information rel-evant for the questions addressed in this paper,after which the results will be presented in Section5. Finally, some of the implications of the resultsfor SA education will be discussed in Section 6.This section will focus on the role that educationalresearch plays in gleaning a better understanding ofSouth Africa pupils’ achievement in mathematics.

2. Context of South Africa

South Africa is an economically developingcountry where the first and third worlds meet. Avast majority of the population resides in typicalThird World impoverishment in contrast to thestate-of-the-art technology in manufacturing andmining being designed and developed by the coun-try’s scientists and engineers. In 1997 South Africawas placed 44 out of the 53 countries in the annualWorld Competitiveness report of the EconomicForum and the Harvard Institute for InternationalDevelopment (van Eldik, 1998), below countrieslike Brazil and Colombia and was ranked 48/48 forthe development of human resources for a competi-tive economy. Further, South Africa was placedseventh, following Mauritius, Tunisia, Botswana,Namibia, Morocco and Egypt in the 1998 AfricaCompetitiveness report, clearly illustrating thecountry’s struggle to compete equally with someof its African peers.

The South African Institute of Race Relations’survey of 1995/1996 calculated an unemploymentrate of 32.6% of the economically active popu-lation of 14.3 million South Africans, with the ratebeing 41.1% for the African population. It isfurther estimated that in 2010, there may be eightmillion unemployed people and a shortage of200 000 skilled workers. Only seven out of every100 people who seek employment will find jobs(Gouws, 1997) and South Africa needs a realgrowth of its GDP of 6.1% by the year 2000 torectify this situation, in contrast to the current fig-ures of 2.0 and 2.5% (van Eldik, 1998).

2.1. Education in South Africa

Within this context, one of the biggest chal-lenges facing South Africa at present is the pro-vision of quality education for its people as inSouth Africa 3.5 million adults (just less than 10%of the SA population) above the age of 16 havenever attended school (Asmal, 1999). The result ofthis is the 55% illiteracy rate amongst SouthAfrica’s disadvantaged communities (Gouws,1997).

However, between 1984 and 1994 there havebeen significant increases in the access to school

605S. Howie, T. Plomp / International Journal of Educational Development 22 (2002) 603–615

for African pupils; for instance an additional 3.6million pupils entered the schooling system ofwhich 95% of these were African. As a result ofthis increase, the number of African pupils writinggrade 12 examinations increased by 35.5% in thisperiod (Education Foundation, 1994).

South Africa’s political history is well knownand its impact on the education system and theyouth that have passed through this system isespecially devastating. The present education sys-tem is an amalgamation of 19 different educationdepartments that were merged in the late 1994 intonine provincial departments as a result of thechange in government policies. Under-resourcedand often mismanaged, many of these departmentsprovided insufficient and ineffective education tomillions of young people, especially from theAfrican, coloured and Indian communities. Con-ditions in the majority of SA schools previouslyserving these communities are poor, for instance,in 24% of the country’s schools, there is no run-ning water within walking distance; 67% of theschools have no electricity; and 50–80% of theschools have no telecommunications. Manyschools have a serious shortage of classrooms andof these, many are in an uninhabitable condition.The pupil to teacher ratio is also very high, and inthree provinces it was found to be more than 40:1.It is not uncommon to find classes of more than100 pupils being taught by one teacher inclassrooms designed for 30 pupils. (HSRC, 1997).

In the past five years, a number of governmentpapers have emerged, including the White Paperon Education, highlighting the government’sawareness of their inheritance as well as intendedpolicies to bring about some change. These papersmarked the beginning of a new era in education inSouth Africa. The SA White Paper on Educationsingles out mathematics and science as importantschool subjects and recognises the importance ofmore pupils leaving school proficient in these sub-jects.

2.2. Matriculation examinations

The matriculation examination (the external finalexaminations written at the end of grade 12) figureshas been used as an indicator of SA pupils’ per-

formance at school level. In 1993, 470 948 pupilswrote the matriculation exams at the end of grade12 and 58% of these passed (Education Foun-dation, 1994, p. 9). However, by 1998 the numberof candidates had increased to 553 151 with only51% of these passing the examinations again sig-nalling a problem with both the efficiency andeffectiveness of the system. (Education Foun-dation, 1999, p. 7). The results at the end of 1997were the lowest recorded since 1979 with only47% of the pupils passing their matriculationexaminations (Education Foundation, 1999, p. 4).The inefficiency of the government school sector,in particular, results in resources being applied topupils who do not pass through the system success-fully. This is problematic as most of these schoolshave very scarce resources. For instance, De Villi-ers (1997) found that only eight out of every 100African pupils entering Grade 1 complete theirmatriculation examination within 12 years

Specifically in mathematics the picture is verysimilar. In 1993, of the 157 701 pupils who wrotethe mathematics exam, only 80 050 (51%) passedrepresenting 17% of the total number of candidatesentering the matriculation exams in that year(Education Foundation, 1994). By 1998, when theenrolment figure had increased by more than120 000 pupils, the pass rate had dropped to 42%(Education Foundation, 1999). The situationregarding the enrolments in mathematics (andscience) and the matriculation exemption withthese subjects also varies dramatically betweenracial groups. For instance, only one among 312African pupils entering the school system leaveswith physical science and mathematics as final yearsubjects as compared to one in five White pupils(Blankley, 1994). In 1999, only 2900 of the 27 187Grade 12 pupils who passed higher grade (highergrade refers to a higher level of difficulty of thesubject taught from grade 10 onwards and thiscourse is usually elected by the higher abilitypupils) were black.2

2 In South Africa, the term ‘black’ is used to refer to African,Indian as well as those of mixed race ‘Coloured’ people(Sunday Times 19 November 2000).


3. IEA and TIMSS

Against this backdrop TIMSS was conducted(under the auspices of IEA) in South Africa and40 other countries. The IEA is an independentinternational non-governmental organisation(based in the Netherlands) whose members arenational research institutions and governmentalresearch agencies. The IEA’s primary purpose isto conduct large-scale comparative studies of edu-cational achievement with the aim of gaining adeep understanding of the effects of policies andpractices within and across systems of education(IEA, 1998).

TIMSS was developed to assess pupils’ achieve-ment in mathematics and sciences in the contextof the national curricula, instructional practices inthe schools and the social environment of thepupils. TIMSS focused on 9-year-old pupils(population 1), 13-year-old pupils (population 2)and final year secondary school pupils (population3). The achievement testing in populations 1 and2 and part of population 3 was based on an analysisof the curricula in mathematics and science in theparticipating countries. The other component of thetesting in population 3 pertained to mathematicaland science literacy of pupils at the end of second-ary education. Background data on pupils, teachers,and schools, as well as data on classroom processeswere collected through questionnaires to pupils,mathematics and science teachers and school prin-cipals.

TIMSS was undertaken in 41 countries fromboth the northern and the southern hemispheres,with data collection occurring in 1995. SouthAfrica was the only country in Africa to do so.South Africa participated in population 2 (seeHowie, 1997) and in the general mathematics andscience literacy test in population 3 (see Howie andHughes, 1998), rather than the specialised math-ematics and physics tests. The reason for this wasthat the mathematics and physical science curricu-lum followed by SA schools at the final year levelis not as specialised as that of other countries.Therefore, it was felt that it would be inappropriatefor SA pupils to write these specialised tests.Further, it was also relevant for researchers, pol-icymakers and educational practitioners, to try to

ascertain the mathematical and scientific literacy ofthe general population of pupils in their final yearof schooling.

4. Research design

As previously stated one focus of TIMSS forpopulation 3 was mathematics and science literacy(MSL). The framework for the MSL-test com-prised four elements (Orpwood and Garden, 1998):mathematics content; science content; reasoning inmathematics, science and technology; and socialutility of mathematics, science and technology.

In this paper some results for the mathematicalliteracy of the final year secondary school pupils(TIMSS population 3; Grade 12 in South Africa)in South Africa are analysed and discussed.

The first results (univariates) for population 3 inSouth Africa were published by Howie and Hughes(1998) showing that SA pupils were by far the leastliterate in mathematics (and science) amongst the22 countries participating in this part of TIMSS.Given this situation, it is important to analyse theSA results further to obtain a better understandingof the factors that underlie these poor results. Inthis paper a few specific research questions havebeen compiled to try to do this, namely:

1. What are the strengths and weaknesses of SAschool leavers in mathematical reasoning andsocial utility? This question will be answered byanalysing the scores of SA school leavers on thereasoning and social utility items.

2. As 11 mathematics items were included in thetests to both populations 2 and 3, the growth ofSA pupils in mathematics from grade 8 to grade12 can be investigated. This analysis will bebased on a comparison of percentage correct onthe so-called ‘ linking items’ .

3. As the majority of SA pupils receive instructionin a language other than the language they speakat home, and given the disparity in social–econ-omical opportunities in the country, exploringwhat relates the pupil background variables, orinfluences the level of mathematical literacy ofSA school leavers was thought important. As theSA data on teachers, school and classroom pro-


cesses were not included in the international dat-abase, only the relationship between pupilsbackground variables and achievement could beexplored. This has been carried out by applyingCONFIRM, reported in this paper as well as bystepwise regression analyses (for details seeHowie and Pietersen, 1998). CONFIRM is aformal inference-based recursive modellingtechnique for continuous dependent variables,which can be used to identify predictors fromthe list of all possible predictors (Hawkins,1995).

4.1. Instruments

Instruments administered to the TIMSS popu-lation 3 pupils consisted of a test for MSL and apupil questionnaire. A rotated test design was usedand four clusters of items (A–D) were rotatedbetween the two test booklets. Clusters A and Bwere included in both booklets.

As only the mathematical literacy part of the testis dealt with in this paper, the description of thetest is restricted to this component. The mathemat-ics literacy component of the test consisted of thefollowing two parts (see Orpwood and Garden,1998):

� Mathematics content – 38 items, total testingtime 45 min.

� Mathematics based reasoning and social utility –6 items, total testing time 15 min.

The characteristics of the test are summarised inTable 1. Research question 1 will be discussed onthe basis of the six reasoning and social utilityitems. A characterisation of these items will bepresented in Section 5.1, together with a discussionof the results.

Ten of the mathematics content items and oneof the reasoning and social utility items were linkitems with population 2. The answers to researchquestion 2 (see Section 5.2) will be based onthese items.

Twenty-seven background variables (see Table4) were identified from the pupils’ questionnairefor analysis in relation to the pupils’ mathematics

Table 1Population 3 mathematics literacy test (MC, multiple-choiceitems; SA, short-answer items; ER, extended-response items)

Topics Number of Items

MC SA ER Total Timeallocated(min)

Math contentNumber 15 1 0 16 17senseAlgebraic 5 2 0 7 9sense/dataMeasurement 11 4 0 15 19andEstimationMathematics 3 1 2 6 15basedreasoningand socialutility

literacy score (research question 3). These includedpersonal details of the pupils (age, gender, homelanguage versus language of test), questions con-cerning the home background (number of booksand number of possessions in the home (out of alist of 16), education levels of parents), attitudesof the pupil, family and friends to mathematics,attitudes of pupils, family and friends to careeropportunities after school, and finally whether thepupil is repeating the final year of school.

4.2. Sample

A national sample of 140 schools (including allnine provinces, all ethnic groups, and both urbanand rural communities) was randomly selected. Intotal a sample of 90 schools was realised consistingof 3695 pupils in Grade 12. As data collection tookplace during the period of the matriculation exams,many schools decided not to participate in this partof the study. Participation by schools using themedium of the Afrikaans language was particularlypoor. Further, there were problems with testingschools in the Western Cape Province and parti-cularly the Northern Cape Province that resulted infewer schools being tested in these provinces than


intended. Further analysis of the school profileswas not possible due to the absence of school leveldata. Therefore the results in this paper should beinterpreted with some caution.

4.3. Procedure

The Human Sciences Research Council (HSRC)undertook TIMSS in South Africa. The researcherswere trained by IEA’s International Study Centrefor the TIMSS instrument preparation and testadministration procedures. The instruments weretranslated into English and Afrikaans, the two lan-guages of instruction still used in the schools. Asthe approval for the project was delayed, theTIMSS instruments could be piloted only in eightschools. Another consequence of the time pressurewas that the international study centre of the studycould process only the achievement tests andpupils questionnaires. The fieldwork (datacollection) in itself was a big challenge arisingfrom the following:

� the lack of information about schools, becauseof incomplete lists of schools or departmentalstaff not co-operating;

� the difficulty in communicating with theschools, owing to no telecommunications orinefficiency within schools;

� the isolated situation of schools (and evenschools that could only be reached on foot);

� the fact that at this time the provincial bound-aries were being redefined;

� the presence of schools in the sample that didnot exist, a phenomena in South Africa called‘ghost’ schools;

� the inefficient postal service that resulted in alarge number of lost letters and returned ques-tionnaires.

Most of these challenges were addressed by thepersistence of the field-workers and the nationalcentre project staff, for instance some schools werevisited up to four times in an effort to secure test-ing times and to test the pupils. Replacementschools were used in cases where even this persist-ence was inadequate. All these efforts resulted in a

response rate of 60% for Grade 12 classes sampled(including replacement schools).

5. Mathematical literacy in South Africa

The results of SA school leavers on the test formathematical literacy will be discussed accordingto each of the three research questions.

5.1. Strengths and weaknesses of the SouthAfrican pupils in mathematical reasoning andsocial utility

Orpwood and Garden (1998, p. 62) elaboratedthe concept of mathematical literacy as applied tothe TIMSS study where mathematical reasoningand social utility is an important aspect. Theywrote that “central to the notion of mathematicsliteracy is the ability to apply the knowledge andskills learned in school mathematics in everydaysituations. The intent is not only that pupils areable to carry our simple computations and manipu-lations involved in household tasks and leisureactivities. Mathematics educators also want pupilsto recognise when to apply mathematical reasoningto understand information in newspapers and otherprint media; to aid in decision making in regard topersonal investments or major financial options;and to clarify implications of the social and econ-omic proposals made by local and national govern-ments. Pupils should also be able to apply the prob-lem-solving skills of mathematics in wider socialsetting than are encountered in the classroom” .Table 2 (adapted from Orpwood and Garden, 1998)summarises the mathematical reasoning and socialutility items in terms of the content areas, the per-formance expectations under which they are classi-fied, an indication of their nature and intent, thepercentage correct of the SA pupils, and the inter-national average.

In ‘mirror’ terms, the general picture reflectedby the data is that the SA pupils performed poorlyin reasoning and social utility in mathematics (seeTable 2). It would appear from the pattern of thescores for the multiple choice items (each havingfive alternatives) that South African pupils simplyguessed their answers. Their performance was also


Table 2Mathematics-based reasoning and social utility items in TIMSS and the South African results (international item codes: MC, multiple-choice items; SA, short-answer items; ER, extended-response items)

Item code Type Content category Performance Notes SA % correct Intl. % correctexpectation

A3 MC Percentages Using more complex Bicycle accidents — 20.4 64procedures percent of percent

A4 MC Estimation and number Using more complex Extrapolation of 27.4 72sense; linear procedures population line graphextrapolation

A5 MC Operation with Solving problems; Minimum number to 20.4 50numbers; equations justifying cover costs of trip

A8a SA Data representation Solving problems Interpret and use inter- 12 (23.6) 44and analysis dependent information(interpreting graphs) from circle and bar

graphsA10a ER Data representation Conjecturing; Graph of age/height 3.3 (17.6) 19

and analysis describing relationship(representing data)

A12a ER Data representation Relating Select and use 9.5 (38.4) 50and analysis representations appropriate(interpreting tables); information — linkestimation item with population 2

a % in brackets represent the partially correct responses.

low for those items requiring written answers andappears to be substantially lower than their per-formance in the multiple-choice items.

In comparison with the international meanscores, the SA pupils performed very poorly withregard to the performance expectations. Theyappeared to have problems with those questionsrequiring complex procedures (A3, A4), solvingproblems (A5 and A8) andconjecturing/describing (A10).

Item A10, where pupils were asked to sketchtheir own line graph, was answered extremelybadly by almost all pupils in the study pupil andonly 3.3% of SA pupils answered this question cor-rectly. In general, it appeared to have been a diffi-cult question for all pupils. On the whole, SApupils found questions on graphs difficult to inter-pret and answer.

Further, SA pupils performed very poorly onitem A12 that required them to relate represen-tations, meaning that they had to select and useappropriate information from the graphs provided.However, the data also show that quite a numberof pupils (about 44%) did not respond to this item,

which suggests that many pupils apparently did noteven know how to start answering the question.Only 5.4% of the pupils taking the test answeredthis item correctly (instead of the 9.5% of thepupils who tried to answer this item), and only21.7% of the pupils in the sample (instead of the38.4% mentioned in Table 2) responded partiallycorrectly to this item. As item A12 is an extendedresponse item, these data suggest that one of themajor problems of SA pupils is phrasing a solutionto a problem in their own words. Whether this isbecause of the lack of proficiency in the languageof instruction (as so many pupils wrote the test ina second of third language) or because of the lackof mathematical knowledge and skills is a questionthat needs further investigation.

5.2. Growth from grade 8 to grade 12

There were 11 items in the Mathematics Liter-acy test that were also included in the test for popu-lation 2. Characteristics of these items and the SAresults for both populations are summarised inTable 3, with the exception of one item on an


Table 3Comparison of percentages correct on linking items (international item codes of population3: MC, multiple-choice items; SA, short-answer items; ER, extended-response items. % Correct of (n) is taken from those who responded to the items)

Item code Type Content All schools All schools Overlapping schools Overlapping schoolspopulation 3 category population 2 population 3 population 2 population 3

% Correct of % Correct of % Correct % of Non- % Correct % of Non-(n) (n) of (n) respondents of (n) respondents

B16 MC Number 29.5 (1112) 54.2 (3620) 28.9 (287) 1 52.3 (2088) 7sense

B23 MC Number 41.8 (1215) 39.6 (3478) 44 (301) 2 40.3 (2128) 5sense

D7 MC Number 36.2 (4669) 42.5 (1765) 35.8 (1191) 3 44.6 (1074) 3sense

D14 MC Number 27.9 (3541) 31.4 (1742) 27.9 (886) 3 34.4 (1059) 4sense

C4 MC Algebraic 25.4 (3573) 37.8 (1772) 25.8 (897) 3 38.1 (1080) 4sense

C6 MC Measurement 18.4 (1254) 21.1 (1779) 20.5 (303) 1 25.4 (1108) 2andestimation

C11 MC Measurement 12.2 (1254) 19.8 (1779) 11.4 (316) 2 22.3 (1089) 4andestimation

D16aa ER Measurement 4.3 (1976) 10.2 (1547) 4.9 (485) 22 12.5 (933) 15andestimation

D16ba ER Measurement 6.4 (1350) 11.3 (1399) 6.6 (349) 44 13.2 (868) 21andestimation

A12a ER Reasoning 35.3 (1254) 38.4 (2072) 43.6 (282) 54 43.9 (1259) 44and socialutility

a % correct also includes partially correct responses.

algebraic sense that was not included because ofthe missing data in population 2. The results forpopulation 2 only reflect the scores of grade 8pupils from 114 schools and not those of thegrade7 pupils.

As the test given to both population groups wasof a rotated design, not all the pupils were testedon the same items that explain the differences inthe number of respondents on the respective items.For instance, in population 3, items B16, B23 andA12 were included in both the booklets for allpupils while items in clusters C and D wereincluded in either of the two booklets only

The overall picture with regard to the growthfrom population 2 to population 3, is not good. Theresults for those schools where both the grade 8

and grade 12 classes were tested in the same schoolare also included in the table and reflect theachievement of pupils from 62 schools. As can beseen from the table, the picture is virtually thesame as for the whole sample of pupils across thetwo population groups. In general, the growth isnot substantial (below 8%) for most of the items,the exceptions being B16 and C4 of all the items.There is also no real decline (except for one item,B23). Nonetheless it does raise a number of ques-tions, particularly as 82% of the population 3pupils had received an additional four years of tui-tion in mathematics. One might reasonably expectthat grade 12 pupils would perform significantlybetter than the grade 8 pupils. In both populationsthe results on ‘measurement and estimation’ are


noticeably the lowest, although this topic is a partof the SA curriculum.

In D16 (an extended response item) in whichpupils were asked to give an estimation (part a)and an explanation (part b), it is also worth notingthat pupils were less likely to attempt to presentan explanation. In population 2, 32% of the pupils,who responded to part a, did not give an expla-nation (part b) and 10% of those in population 3who responded to part a did not respond to part band explain how they had reached their estimation.

Table 3 also illustrates the importance of the lan-guage issue. For example, if one looks at the per-formance of the pupils from the overlappingschools for items D7 and D14, only 3–4% of thepupils, respectively, did not respond to these items.This is in contrast to items D16a and b wherebetween 15 (population 3 for D16a) and 44%(population 2 for D16b) of the pupils did notrespond to these items. Both D7 and D14 are mul-tiple-choice questions and therefore one wouldexpect the majority of pupils to at least attempt theitem, hence the low non-response rates. D16a andD16b (but also A12) on the other hand areextended response items and not only were thescores on all these items throughout the test low,but the rate of non-respondents in general was alsomuch higher (the highest being 54% for A12 forpopulation 2) for extended response items than formultiple-choice items (the highest being 7%).

5.3. Pupil background related to mathematicsliteracy

The results of the CONFIRM analysis arepresented in the dendrogram in Fig. 1. Owing tothe missing data on some of the background vari-ables, they are based on 2649 pupils.

The analysis identified the most significant pre-dictors (see Table 4 for a complete list of variablesanalysed) for achievement, which are presentedfrom top to bottom in levels of their relative impor-tance.

For example, the first split in the dendogramrevealed those possessions at home (how many outof a list of 16 items do the pupils have at home,as an indication of socio-economic wealth orbackground) is the most significant predictor of

pupils’ achievement. Here, the pupils were cat-egorised in four groups according to the numberof items they report were in their house: subgroup1 contains all pupils with 10 or less items at homebeing the least privileged in terms of socio-econ-omic status; in subgroup 2, all pupils with 11 or12 items; in subgroup, 3 all pupils with 13 or 14items; and in subgroup 4, all pupils with 15 or 16items. The split indicates that groups of pupils withmore items at home perform better than the sub-groups with fewer items. The analysis indicates persubgroup what the next most significant predictorfor that subgroup is. As a result, in the subgroupof highest performing pupils ‘age’ is the mostimportant predictor; younger pupils perform betterthan the older pupils, while in subgroups 2 and 3‘ language’ (‘how often is the language of the testspoken at home’ ) was found to be the most sig-nificant predictor. In subgroup 1 (consisting of themajority of pupils) the construct ‘ the pupils percep-tion of what others think about the importance ofdoing well in mathematics in school’ is the mostsignificant predictor. This implies that the greaterthe number of people whom the pupils believe feelthat “ it is important to do well in mathematics” ,the better the pupil performed.

In conclusion, the CONFIRM analysis showsthat socio-economic status and language spoken athome are strong determinants of the mathematicsliteracy score. However, it is important to note thatsocio-economic status and language spoken athome are confounding variables in themselves, andin the SA context, also with race. Unfortunately,in 1995 data on race were not collected owing tothe political tension at that time.3

Howie and Pietersen (1998) present a fullexploratory analysis of the relationship betweenpupils’ background variables and achievement.Their results confirm the CONFIRM analysis.They also found, as in further steps in the CON-FIRM analysis, that other important predictors ofachievement are: whether it is important to do well

3 In the repeat of TIMSS in 1998 data on race, home langu-age, exposure to English and an English proficiency test havebeen included as a national option for South Africa, so that therelationship between these variables and achievement can bescrutinised more closely.


Fig. 1. CONFIRM analysis of pupil achievement (first two levels).

in mathematics; the attributes that the pupils con-sider important to do well in mathematics (suchas hard work or ability); and the expectations oraspirations concerning their future career afterschool.

6. Discussion

While the results of the SA pupils in the math-ematics literacy test are poor (354 out of amaximum of 800 scale points, while the inter-national average is 513 points), a closer look at theresults presented in the previous section haveraised a number of issues, which are discussed inthe this section in the context of the research ques-tions posed at the beginning of the paper.

6.1. Strengths and weaknesses of SA schoolleavers in mathematical reasoning and socialutility

SA school leavers leave the education systemwith a very low level of mathematical literacy and

the findings conclude that the pupils are unable toapply the knowledge and skills learnt in schoolmathematics in everyday situations. The impli-cation of this is that they are incapable of carryingout the simple calculations involved in tasks athome and in their leisure activities. This also sug-gests that they are neither unable to cope with, orapply, problem-solving skills to personal issues,such as finance, nor able to comprehend and inter-pret information printed in the media.

6.2. Growth of SA pupils’ mathematical literacyfrom grade 8 to grade 12

The results of 10 of the common items that wereincluded across all three population groups inTIMSS were compared between the SA pupils ingrade 8 and with grade 12. The conclusion fromthe findings is that growth between these grades islimited and not as great as one would haveexpected, given the four additional years of school-ing. There is even an example where grade 8 pupilsoutperformed the grade 12 pupils. In an attempt toscrutinise the results more closely, the schools


Table 4List of students’ background variables identified for analysis

Variable name Description of variable

AGE Age of studentGENDER Gender of studentLANGUAGE How often is the language of the test

spoken at homeBOOKS The number of books at homeITEMS5 How many of the next 5 items are at

home: calculator, study desk, dictionary,electricity, TV

ITEMS16 How many of all 16 items do the studenthave at home

IMP For how many people (Father, Mother,Friend or self) is it important that thestudent do well in Maths at school

MFEDUC Education level of Mother of studentFFAFTER What the Father of the student thinks

he/she should do after schoolMFAFTER What the Mother of the student thinks

he/she should do after schoolTFAFTER What the Teacher of the student thinks

he/she should do after schoolFRFAFTER What the student’s friend thinks he/she

should do after schoolFFIMP Is it important for the student’s father that

he/she do well in Maths at schoolMFIMP Is it important for the student’s Mother that

he/she do well in Maths at schoolFRFIMP Is it important for the student’s friend that

he/she do well in Maths at schoolSFIMP Is it for the student important that he/she

do well in Maths at schoolENJOY The student enjoy learning MathsBORE Mathematics is boringEASY Mathematics is an easy subjectLIFE Mathematics is important to everyone’s lifeWORK The student would like a job that involved

MathsLIKE How much the student likes MathematicsTALENT If the student thinks one needs lots of

natural talent to do well in MathsLUCK If the student thinks one needs good luck

to do well in MathsHFWORK If the student thinks one needs hard work

to do well in MathsMEMORIZE If the student thinks one has to memorize

the textbook or notes to do well in MathsFINYEARP If the student has completed any other Std

10-/programs of study

where both grades 8 and 12 participated in TIMSS(which occurred in 62 schools) were selected se-parately. The results were the same; a similar pat-tern of limited growth could be found within theseschools and therefore served to confirm the find-ings of the overall pupil population.

6.3. Pupil background variables linked tomathematical literacy

The secondary analysis conducted focusing onthe predictors of mathematics achievement (seealso Howie and Pietersen, 1998) revealed therelationship between the socio-economic back-ground, pupils’ home language, age, pupils’ per-ception of the importance of mathematics and theirachievement. However, the analysis by Howie andPietersen (1998) also revealed the strong relation-ship between the socio-economic background andthe language of the pupil. With the context inSouth Africa providing a very strong link betweenthese two variables, further exploration of thebackground variables and their interdependency isrequired. Their causal relationship (particularlywith regard to the pupils’ language) in relation topupils’ achievement in mathematics is of particularinterest to researchers, practitioners and pol-icymakers in South Africa. The exploration of thelanguage issue is an integral component of theTIMSS-R project in South Africa and together withthe Pupils Understanding of Mathematics projectcarried out in the Eastern Cape will further explorethe causal factors of mathematics achievement inSouth Africa.

7. Conclusion

Most recently, the new Minister of Educationreported in Parliament that he regarded literacy(including numeracy) as a ‘priority’ and that thecountry needed a national strategy to deal with this(Asmal, 1999). Given the ambitious goals forteaching and learning that are being introducedthrough Curriculum 2005, a thorough understand-ing of the issues surrounding pupils’ understandingof and achievement in mathematics is needed, toprovide support for practitioners to implement the


new curricula. From the TIMSS results as dis-cussed here (and in the national reports on the SAresults), it would seem that much of the basicmathematics knowledge expected of pupils leavingschool has not be attained by them. This raises aninteresting question — will changing the curricu-lum address the lack of basic mathematical literacyof SA pupils or does it further attract the attentionof practitioners and policymakers away fromaddressing other more fundamental problems suchas the shortage of qualified teachers teaching math-ematics?

Further, what level of proficiency and skills inmathematics are expected of pupils? Naturally,there must be concerns about a system where 48%of the pupils fail the national final-year examin-ations (Education Foundation, 1999) and furtherwhere there is little difference between grade 8 andgrade 12 in the pupils’ basic mathematical literacylevel despite the fact that more than 80% of thepupils received four additional years of tuition inmathematics. This lack of growth reflects a lack ofquality of teaching and therefore of teachers whichmay be a more important problem to address.Finally, these concerns must be heightened by theknowledge that 50% of the mathematics teachersin the class are not formally qualified to teachmathematics.

This paper has addressed a number of researchquestions and has also raised a number of issues.On the whole, SA pupils do not seem to be wellprepared for entering the workplace as mathemat-ically literate employees and this has several impli-cations for pupils finding employment, for pro-spective employers in terms of training, and for thegovernment in terms of job creation projects. Lan-guage and communication skills are critical in theworkplace and it is worrying that so many pupilsdid not even reply to the questions requiringextended responses (e.g. for item A12, 44% of thepupils did not reply) and those that exhibited lang-uage problems.

In conclusion, it should be said that the investi-gation into possible causal factors was seriouslyhampered by the lack of school and teacher datafrom 1995. The lack of multilevel data narrow thescope of the type of exploratory secondary analysisneeded to probe complex issues dealing with pupil

achievement. This is an aspect that will receivebetter attention in the TIMSS-Repeat study carriedout in 1998 where data was successfully collectedand captured on all three levels. What is also clearfrom TIMSS is the pupils lack of experience withmultiple choice questions and although the per-formance was higher for this kind of item than theopen response items, greater experience withanswering this kind of question may have led tohigher scores. The poor performance on the open-ended items does the highlight issues related to lan-guage and apparent difficulties for second languagepupils in the reading and comprehension of ques-tions and the communication and expression ofideas, perhaps further exacerbated by the lack ofbasic knowledge of the subject. What is alsounknown in the SA context is the possible effectthat the combination of biology, physical science,geography and mathematics items in one test hadon the pupils. While it appears internationally thatpupils were not too negatively affected by this, per-haps this could have had a negative effect on thelow ability pupils in particular. Nonetheless theresults are so low that it is extremely difficult todetect trends with a certainty or to undertakefurther analysis in this vein.

Finally, international studies such as TIMSSmay serve to reflect the current situation within acountry such as South Africa. They can also helpprovide comparative data to set national goals aswell as assist in monitoring the introduction of sys-temic and curricula changes in national educationalsystems, thereby allowing countries to plan anddevelop their education systems to address nationalneeds and priorities.

Acknowledgements

The authors would like to thank Ms Elsie Venterfor her invaluable assistance with regard to the dataanalysis and to Ms Hannelie Knoetze for her helpin tracking the sources for this paper.

References

Asmal, K., 1999. Statement in the National Assembly by theMinister of Education on the Occasion of International Lit-eracy Day, 8 September 1999.


Blankley, W., 1994. The abyss in African school education inSouth Africa. South African Journal of Science 90, 54.

De Villiers, A.P., 1997. Inefficiency and demographic realitiesof the South African school system. South African Journalof Education 17 (2), 79–80.

Education Foundation, 1994. Edusource Data News no. 7,December.

Education Foundation, 1999. Edusource Data News no. 24,March.

van Eldik, P., 1998. The role of higher education in technologi-cal innovation in economically developing countries. Paperpresented at the 38th Annual forum of the Association forInstitutional Research, 17–20 May, Minneapolis.

Gouws, E., 1997. Entrepreneurship education: an educationalperspective. South African Journal of Education 17 (3), 143.

Hawkins, D.M., 1995. Formal inference-based recursive model-ling, Release 2.0. Technical Report no. 546, University ofMinnesota, School of Statistics.

Howie, S.J., 1997. Mathematics performance in the middleschool years in South Africa. A summary report on the per-formance of the South Africa pupils in the Third Inter-national Mathematics and Science Study. Human SciencesResearch Council, Pretoria.

Howie, S.J., Hughes, C.A., 1998. Mathematics and science liter-acy of final-year school pupils in South Africa. In Third

International Mathematics and Science Study. HumanSciences Research Council, Pretoria.

Howie, S.J., Pietersen, J.J., 1998. Mathematics literacy of final-year pupils: South African realities. Paper presented at theEuropean Conference on Educational Research (ECER98),Ljubljana (Slovenia).

HSRC, 1997. School needs analysis, Human Sciences ResearchCouncil, Pretoria.

IEA, 1998. IEA Guidebook 1998: Activities, Institutions andPeople. IEA-Secretariat, Amsterdam.

Mullis, I.V.A., Martin, M., Beaton, A.E., Gonzalez, E., Kelly,D.L., Smith, T., 1998. Mathematics and Science Achieve-ment in the Final year of Secondary School. Boston College,Chestnut Hill.

Orpwood, G., Garden, R.A., 1998. Assessing Mathematics andScience Literacy. TIMSS Monograph, Pacific EducationalPress, Vancouver.

Plomp, TJ., 1998. The potential of international comparativestudies to monitor the quality of education. ProspectsXXVIII (1), 45–59.

van Eldik, P., 1998. The role of higher education in technologi-cal innovation in economically developing coutries. Paperpresented at the 38th Annual forum of the Association forInstitutional Research, May 17–20, Minneapolis.

mathematical literacy of school leaving pupils in south africa

Documents