Page 1 of 13

4 Copthall House,

Station Square

Coventry CV1 2FL

+44 (024) 7652 4036

+44 (024) 7663 1646

info@curee.co.uk

What are the most effective forms of maths study and assessment post 16

(for different groups of learners)?

Page 2 of 13

What are the most effective forms of maths study and assessment post 16

(for different groups of learners)?

Introduction

As highlighted in the report in question 1, there are many concerns with mathematical

disengagement at 16. Whilst the reasons for this disengagement are complex there is evidence that

study and assessment methods play a role in this and that effective methods could engage learners

and help them achieve higher standards. This report firstly outlines the evidence for effective study

methods in maths and then considers the literature relating to effective assessment in post-16

maths. For the purposes of this review, studies that identified forms of assessment which were

effective in contributing to learner engagement and learning were explored.

Effective study in post-16 maths

The international evidence highlighted a complex picture of the links between the forms of

pedagogy and learner outcomes. Studies from the UK found a number of features of effective study

in maths and particularly highlighted the benefits of a ‘connectionist’ approach to study (which

emphasises collaborative learner-centred learning) as opposed to a ‘transmission’ approach (which

emphasises demonstration and individual practice). However, this more ‘traditionalist’ approach

seems to be in common use. Ofsted (2008) found that in many secondary school lessons, teachers

demonstrated rules and procedures and then asked learner to complete exercises practising the

skills. A study which looked at practice in adult numeracy found that teachers tended to show

learners procedures and demonstrate examples, and rarely used group work (Coben, 2007).Despite

the apparent widespread use of ‘transmission’ or ‘traditionalist’ teaching methods, our searches

retrieved no studies which found these methods effective or offered an explanation of their

popularity. The next section of this report outlines the international evidence in more detail and

then considers the evidence from the UK, in particular the connectionist approach and evidence

about linking mathematical learning with vocational and real-life contexts.

Whilst the review did include two studies which looked at post-16 maths provision in high-

performing countries (Hodgen et al, 2009, 2010) these focused on whether participation was

compulsory and the content and structure of the courses rather than the study and assessment

methods. However, two sources, whilst not focusing on post-16 study, looked at maths in countries

which had performed well in international assessments. One large-scale literature review found that

there was a great deal of variety in the teaching practices used within high-performing countries.

Moreover, there seemed to be a greater link between cultural factors and high attainment than

between the teaching approaches and attainment. Countries that performed well such as Finland,

Singapore and Korea tended to have traditional approaches which also featured in low performing

countries (Askew et al, 2010).

Page 3 of 13

The Askew et al (2010) study did find some evidence that a feature of effective approaches was

teachers engaging their learners in problems which enabled them to make connections between the

areas of their learning. Teaching in Pacific Rim countries emphasised both procedural fluency and

conceptual understanding through carefully constructed examples. Textbooks supported this and

could offer an opportunity for learners to work independently out of class. There was also some

evidence to indicate that teachers in higher attaining countries paid attention to developing learners' personal qualities (particularly perseverance) in their maths lessons (Askew et al, 2010).

An analysis of the connection between teaching strategies (as reported by learners) and

performance in PISA assessments also found no clear correlation between a set of teaching and

learning conditions and strong learner performance. The one factor that seemed to have a

universally strong association with performance was the disciplinary climate. learners who

experienced disorderly classrooms were less likely to achieve well (OECD, 2010).

Interestingly, a UK study (Coben et al, 2007) which specifically investigated effective teaching in

numeracy for adult learners found only a few correlations between particular teaching and learning

characteristics and learner progress. The researchers tested 250 learners, carried out interviews and

observed lessons. The researchers did find a significant positive correlation between progress and

procedural teaching. So classes where teachers showed learners discrete techniques made more

progress. However, there was no greater progress in classes where procedural teaching was more

extensively used. A small negative correlation was found where there was a great deal of learner individual work. The researchers classified the approaches used by the teachers into 3 types;

connectionist, transmission and constructivist/scaffolder. When they analysed the approaches of the

5 most effective teachers they found they used a range of approaches from all 3 types.

However, an Ofsted report (2006) which observed practice and scrutinised learners’ work in 26

schools and colleges found that a number of factors were linked with higher achievement in 14-19

mathematics. Effective teaching focused on:

developing learners’ understanding of mathematical concepts;

creating a positive atmosphere to build learners’ confidence;

promoting collaboration and discussion;

using ICT and other resources to promote learning; and

creating relevant contexts for learning.

The teachers developed leaners’ understanding of maths concepts through giving clear

explanations and requiring learners to articulate and refine their own ideas. Learners were

encouraged to develop their critical thinking and reasoning. The teachers made connections

between previous and forthcoming work on a topic, and highlighted recurring mathematical themes.

Page 4 of 13

Teachers built learners’ confidence by carefully assessing the learners’ starting points and

challenging learners to think for themselves. They used questioning to expose misconceptions and

created a positive atmosphere where the learners could talk these through in a non-threatening

way. The teachers also focused on ideas rather than just procedures.

Collaboration and discussion was used to promote learning. Learners were given opportunities to

work in groups to solve problems and discuss their ideas with other learners. Teachers used ICT to

present graphs and diagrams effectively but also enabled Learners to recognise the limits of these

tools. Effective teaching embedded mathematical work in compelling and relevant vocational

contexts (Ofsted, 2006).

Other UK studies provided further evidence that there were certain study methods which were

particularly effective for post-16 learners. These reinforce many of the features of effective teaching

included in the Ofsted (2006) report and particularly highlight the value of:

a learner-centred connectionist approach; and

linking mathematical learning with vocational and real-life contexts.

The connectionist approach which makes links between mathematical learning with vocational and

real-life contexts are considered in more detail in the following sections.

Learner-centred ‘connectionist’ approaches

Several studies emphasised the effectiveness of a learner-centred or ‘connectionist’ approach in

contrast to a transmission approach (Swan, 2005 and 2006; Swain and Swan, 2009; Williams et al,

2008; Callendar, 2008). With a transmission model, methods are explained to learners one step at a

time. Teachers question the learners to check that they have understood the steps. Learners then

practice the methods they have been shown in a series of graded exercises. Learning tends to be an

individual activity based on imitating the teacher. However, a connectionist model sees learning as a

much more collaborative activity where learners are actively involved in developing their

understanding through discussion.

This approach has been used with a range of learners including those in colleges, schools, prisons

and work-based learning environments in the learning and skills sector (Swan, 2005). There was

evidence of the approach benefiting:

adult learners (Swain and Swan, 2009);

learners re-taking GCSE maths at FE colleges (Swan, 2006); and

A level learners particularly those with lower grades (Williams et al, 2008).

The benefits included learners:

Page 5 of 13

developing increased understanding and transferable learning that could be applied to

situations out of the classroom (Swain and Swan, 2009);

achieving improved scores in algebra tests (Swan, 2006);

increasing their motivation and enjoyment (Swan, 2006;Williams et al); and

having decreased anxiety (Swan, 2006).

Retention rates on courses were also increased and there was some evidence of more learners

intending to study maths at a HEI.

Details of the connectionist approach

One study which involved working with over 200 teachers across 40 organisations to improve the

quality of mathematics in the learning and skills sector highlighted the research-based principles

associated with this approach (Swan, 2005). These included teachers:

building on the knowledge learners bring to lessons;

changing teaching to take account of assessment;

exposing and discussing common misconceptions;

developing effective questioning;

emphasising methods rather than answers;

using rich collaborative tasks which are accessible and extendable and enable learners to

make decisions;

creating connections between mathematical topics; and

using technology in appropriate ways.

A large scale study which developed these learner-centred and collaborative learning approaches

with learners taking GCSE maths at FE colleges (using a range of tools to collect evidence of impact)

emphasised:

the active role of learners;

mathematical methods and reasons rather than answers; and

mistakes as a positive aspect of the learning process.

learners were set rich, challenging tasks and questions and were encouraged to develop their own

varied methods and justifications for solving these. Mistakes were seen as learning points. learners

were encouraged to support each other through creating problems for their peers to solve and were

helped to create their own links between different mathematical topics. A supportive social context

was an important element of this (Swan, 2006).

Page 6 of 13

Interestingly, one study found that connectionist practices were more prevalent in classes where

learners had lower GCSE grades and the teachers thought that learners needed to build their

understanding (Williams et al, 2008).

Linking mathematics with vocational and real-world contexts

The importance of linking mathematical learning with the learning that was taking place on

vocational courses was emphasised in several studies (Stone et al, 2005, Casey et al, 2006 and

Hoachlander, 2008). There was some evidence that linking learning with real-world contexts played

an important role for those on A level courses in terms of their performance and attitudes (Little and

Jones, 2010).

The benefits associated with linking mathematical and vocational learning were that learners:

in most cases performed better than learners in traditional classes (Stone et al, 2005);

developed the same level of knowledge and skills in the vocational subject itself (Stone et al,

2005);

had improved attitudes, motivation and confidence in learning maths (Stone et al, 2005);

developed an increased understanding of the theory (Stone et al, 2005); and

were more likely to stay on the course (Casey et al, 2006).

Details of how maths and vocational learning was linked

In one large-scale study (involving randomised control trials), teachers embedded the maths learning

into the vocational lessons. They explicitly told pupils when they had reached the part of the lesson

where maths was embedded (Stone et al, 2005). They then:

used open questions to find out what pupils already knew;

worked through the steps of an example problem;

identified and shared maths concepts/principles;

worked through real-life and related abstract maths problems;

used open questions again to assess what learners had learned in the lesson; and

asked learners to create their own examples.

Another study which set out to explore the impact of embedded provision on learner’s literacy and

numeracy achievement (Casey et al, 2006) also found that it was important to closely link the

numeracy and vocational teaching so that learners could see the relevance of what they were

learning. It was also important to the learners that their teachers understood the demands of their

vocational subject.

Page 7 of 13

A literature review and analysis of official data releases in the US included an example of how the

maths and vocational learning were linked. On a building and environmental design course learners

learned the concepts and skills from geometry which would enable them to build roofs and frame

walls that can withstand strong winds (Hoachlander, 2008).

Linking A level learning with the real world

Whilst the present literature review was asked not to focus on curriculum models the research from

this area does have implications for this. A study which compared the experiences of learners on a

‘traditional maths’ A level course with those on a ‘use of maths’ course found that there was a lower

drop-out rate on the ‘use of maths’ course and learners were more likely to express their

appreciation of maths as a useful subject (Williams et al, 2008).

A study which compared about 500 learners’ responses and attitudes to questions presented both

in and out of context found that on the whole questions presented in context were not answered as

correctly as explicit questions. However, 55% of the learners indicated that they felt that questions

presented in this way showed how maths is useful. Learners varied in the way they viewed the

difficulty and level of interest of these questions. Boys tended to feel that they added interest, whilst

more girls thought that the real-life context made the maths hard (Little and Jones, 2010).

Effective assessment methods in post-16 maths

The research for effective assessment methods mainly came from the UK. One study considered

practice in Scotland and Hong Kong because of their attainment and progression records. They

found that in both countries there was a greater homogeneity in the qualifications types but greater

diversity in the curriculum content (Norris, 2012).

There was evidence from the UK about summative assessment that showed:

criterion and competency-based assessment was effective in terms of helping learners stay

on courses and achieve awards (Torrance, 2005);

adult learners preferred ongoing assessment based on discussion and portfolio-building

supported by teacher feedback and individual reflection (Ward and Edwards, 2002);

adult learners were most satisfied with their progress when they were involved in its

assessment (Ward and Edwards, 2002); and

learners on a ‘Use of Maths’ A level course found coursework motivating which may have

contributed to the course retention rates (Hernandez-Martinez et al, 2011) .

In addition to the literature about the more summative aspects of assessment, there was also

evidence about the important role of formative assessment.

Page 8 of 13

Summative Assessment

A study which set out specifically to explore the impact of different modes of assessment on

achievement and progress in the learning and skills sector (using case studies and interviews with

about 200 learners) found:

there was a difference between assessment methods on academic and vocational courses.

More practical methods were favoured in vocational contexts but achievements by these

methods were regarded as lower status;

key skills assessments were judged to not serve the purposes for which they had been

designed;

assessment methods did not directly affect learners’ choice of award or likelihood of success

but the association of certain awards with particular methods did. So whilst practical

learning and multiple choice questions were accepted by most learners, extensive written

methods tended to be avoided by all but A level learners;

electronic online assessment was popular and effective; and

criterion-referenced and competency-based assessment had helped learners stay on courses

and achieve awards.

The researchers particularly highlighted the link between assessment and teaching and study

methods. They concluded that whilst greater transparency in assessment criteria helped learners as

tutors gave them explicit support in meeting the criteria, it had also led to instrumentalism.

Assessment procedures and practices had come to dominate the learning experience (Torrance,

2005). Ofsted (2006) also noted that learners in 14-19 education were being taught to pass

examinations without necessarily gaining the skills and understanding to be able to apply their

knowledge. The impact of assessment on teaching and learning practices was highlighted in a large-

scale study which connected the assessment system with transmission teaching approaches (Wake,

2011).

A qualitative study which interviewed 70 adult learners from literacy, language and numeracy

classes found that they preferred ongoing assessment based on discussion and portfolio building,

supported by teacher feedback and individual reflection. The adults were more satisfied with their

progress when they had been involved in recognising learning gains. They used their own records of

progress and files to inform themselves of how far they had travelled and this enabled them to

persist with their learning. Some learners did want formal assessment through tests and external

exams as they saw them as motivating and confidence building. Most learners described their

learning gain in terms of what their learning enabled them to do in real life situations (Ward and

Edwards, 2002).

Interviews with learners (with lower GCSE grades) on a Use of Maths course indicated that

coursework allowed them to develop greater depth of understanding. They found it motivating and

Page 9 of 13

particularly appreciated that it allowed them to apply their maths to other subjects and use

technology to complete tasks. For many learners, coursework was seen as something which was

easier than exams and allowed them to achieve better grades than other methods of assessment

(Hernandez-Martinez et al, 2011).

Formative assessment

Just as the literature highlighted the importance of learners taking an active role in their study, it

also highlighted the importance of learners taking this active role in formative assessment. There is a

strong evidence base for the positive impact of Assessment for Learning (AfL) approaches in

compulsory education (e.g. Black et al, 2003).Ofsted’s (2006) report on 14-19 maths provision found

that teachers’ use of assessment to inform their teaching was linked with high learner achievement.

Effective teachers used questioning to probe their learners’ understanding and build on the

misconceptions which they uncovered. A study which focused on using this approach in post-16

maths highlighted the importance of: divergent assessment techniques which allowed for a range of responses;

self and peer assessment; and

useful detailed feedback that focuses on the task (Swan, 2005).

An action research project which developed formative assessment practices with adult learners

found that it was perceived to have a positive impact on teaching and learning. A number of

strategies were found to be useful including:

developing talk through encouraging learners to raise questions in group activities and talk

amongst themselves;

encouraging learners to assess and evaluate each other’s methods and ideas;

teachers saying less, by encouraging questions, contributions and interruptions from the

learners;

using ‘planned’ teacher mistakes;

helping learners to connect with their life experiences and use these as a resource to

support each other;

valuing contributions; and

giving time to talk through representations and tools.

The researchers found that learners with ESOL could find language a particular barrier. Observing

non-verbal as well as verbal activity, sometimes encouraging learners to talk in languages other than

English and using translators to interpret this, were strategies for overcoming the barriers.

Implementing practices in family numeracy and embedded courses could also be a challenge. Family

numeracy courses were often short so it could be difficult to develop a culture of participation but

Page 10 of 13

asking questions such as ‘how would you do with your child?’ offered a chance to explore

understanding. Making the connections with the vocational use of maths was key to effective use of

the strategy in embedded settings (Hodgen et al, 2009).

Summary Findings

International studies gave a complex picture of the links between pedagogy and outcomes in

pre-16 maths. ‘Traditional’ teaching methods were used in both high and low performing

countries.

A UK study of post-16 maths found that effective teachers used a range of teaching

approaches. There was evidence that lessons which included showing learners discrete

techniques were associated with greater progress and that large amounts of individual work

were associated with less progress. Another study highlighted the benefits for learner

achievement of developing learners’ understanding of mathematical concepts, building learners'confidence, promoting collaboration and using ICT to promote learning.

Traditional teaching styles were widely used but several studies highlighted the

effectiveness of a connectionist rather than transmission approach for post-16 maths. This

approach emphasised conceptual understanding, connections between areas of maths and

collaborative learning. Benefits included learners having increased understanding, higher

scores in algebra and increased motivation and enjoyment. Several studies highlighted the importance of linking mathematical learning with vocational

and real-world contexts. Benefits included learners performing better, having improved

motivation and being more likely to stay on the course. There was evidence that summative assessment which was criterion and competency-based

was effective in helping learners stay on courses and achieve awards. Ongoing assessment

was preferred by adult learners and learners on a ‘Use of Maths’ A level course and they

found coursework motivating. Some studies highlighted the beneficial role of formative assessment in post-16 maths. The

wider evidence base about effective teaching and learning supports this.

Methodology

The research was systematic and connoisseurial but pragmatically limited by time and resources. An

in-house search of CUREE’s database of international research was carried out. This was

supplemented by conducting BEI, ERIC, AEI and Google Scholar searches using the terms ‘effective

strategies’, ‘learners’, ‘post-16’, ‘maths ’. An ERIC search specifically used the terms ‘maths’,

‘assessment’ and ‘post secondary education’. In addition, research resources on the Excellence

gateway, NCTEM and NRDC websites were hand searched. These studies were initially filtered for

relevance and coverage of the key issues. Only those studies with an explicit and plausible

methodology were identified for data extraction.

Page 11 of 13

References

Askew, M., Hodgen, J., Hossain, S. & Bretcher, N. (2010) Values and variables: Mathematics

education in high- performing countries [online]. London: Nuffield Foundation. [Accessed 17 March

2012]. Available at:

<http://www.nationalstemcentre.org.uk/res/documents/page/Values_and_Variables_Nuffield_Fou

ndation_v._web_FINAL.pdf >.

Black, P., Harrison, C., Lee, C., Marshall, B. & William, D. (2003) Assessment for Learning: Putting it

into Practice. Maidenhead: Open University Press.

Callendar, S. (2008) Raising achievement in Mathematics by enhancing the learning experience

[online] [Accessed 17 March 2012]. Coventry: National Teacher Research panel. Available at:

<http://www.ntrp.org.uk/sites/all/documents/Sarah%20Callender_FINAL.pdf>.

Casey, H., Cara, O., Eldred, J., Grief, S., Hodge, R., Ivanic, R., Jupp. T., Lopez, D., & McNeil, B. (2006)

“You wouldn’t expect a maths teacher to teach plastering…” Embedding literacy, language and

numeracy in post-16 vocational programmes – the impact on learning and achievement. National

Research and Development Centre for adult literacy and numeracy [NRDC]. [Accessed 17 March

2012]. London: NRDC. Available at: <http://www.nrdc.org.uk/uploads/documents/doc_3188.pdf>.

Coben, D. Brown, M., Rhodes, V., Swain, J., Ananiadou, K., Brown, P., Ashton, J., Holder, D., Lowe, S.,

Magee, C., Nieduszynska, S. & Storey, V. (2007) Effective Teaching and Learning Numeracy [online].

National Research and Development Centre for adult literacy and numeracy [NRDC]. London: NRDC.

[Accessed 23 March 2012]. Available at: <http://www.nrdc.org.uk/publications_details.asp?ID=86 >.

Hernandez-Martinez, P., Williams, J., Black, L., Davis, P., Pampaka, M. & Wake, G. (2011) Students’

and teachers’ views on coursework: formative assessment, student-centred activity and

understanding in mathematics. Research in Mathematics Education [online]. 13 (2). Pp.197-212

[Accessed 13 March 2012]. Available at:

<http://www.tandfonline.com/doi/abs/10.1080/14794802.2011.585830>.

Hoachlander, G. (2008) Bringing industry to the classroom. Reshaping High Schools. 65 (8). Pp. 22-27.

Hodgen, J., Coben, D., & Rhodes, V. (2009) Feedback, talk and engaging with learners: Formative

assessment in adult literacy: Research report [online]. National Research and Development Centre

for adult literacy and numeracy [NRDC]. London: NRDC [Accessed 29 Feb 2012]. Available at:

<http://www.nrdc.org.uk/publications_details.asp?ID=180>.

Hodgen, J. & Pepper, D. (2010) Is the UK an outlier in upper secondary maths education? [online]

London: Nuffield Foundation. [Accessed 28 Feb 2012]. Available at:

<http://www.nuffieldfoundation.org/sites/default/files/files/Is%20the%20UK%20an%20Outlier_Nuf

field%20Foundation_v_FINAL.pdf>.

Page 12 of 13

Little, C. & Jones, K. (2010) The effect of using real world contexts in post-16 mathematics questions.

Proceedings of the British Congress for Mathematics Education April 10 [online]. Pp. 137-144

[Accessed 28 Feb 2012]. Available at: <http://www.bsrlm.org.uk/IPs/ip30-1/BSRLM-IP-30-1-18.pdf>.

Norris (2012) Solving the maths problem: international perspectives on mathematics education

[online]. Accessed 6 March 2012. London: The Royal Society for the encouragement of Arts,

manufactures and Commerce. Available at: <

http://www.thersa.org/__data/assets/pdf_file/0011/568181/RSA_Maths_report_10_2_12.pdf>.

Ofsted (2006) Evaluating mathematics provision for 14-19 year olds. Available at:

<http://www.ofsted.gov.uk/resources/evaluating-mathematics-provision-for-14-19-year-olds>

Ofsted (2008) Mathematics: understanding the score. Available at:

<http://www.ofsted.gov.uk/resources/mathematics-understanding-score

Organisation for Economic Development [OECD] (2010) Mathematics and Teaching and Learning

Strategies in PISA [online]. Paris: OECD. [Accessed 17 March 2012]. Available at:

<http://www.pisa.oecd.org/document/11/0,3746,en_32252351_32236173_46089739_1_1_1_1,00.

html#3>.

Stone, J., Alfeld, C., Pearson, D., Lewis, M., & Jenson, S. (2005) Building academic skills in context:

testing the value of enhanced maths learning in CTE. Minnesota: National Research Center for Career

and Technology Education.

Swain, J., & Swan, M. (2009) Teachers' attempts to integrate research-based principles into the

teaching of numeracy with post-16 learners. Research in Compulsory Education [online]. 14 (1).

Pp. 77-92 [Accessed 17 March 2012]. Available at:

<http://www.tandfonline.com/doi/abs/10.1080/13596740902717424>.

Swan, M (2005) Improving Learning in Mathematics: challenges and strategies [online]. [Accessed 24

Feb 2012]. London: Department for Education and Skills Standards Unit. Available at:

<https://www.ncetm.org.uk/public/files/224/improving_learning_in_mathematicsi.pdf>.

Swan, M (2006) Learning GCSE mathematics through discussion: what are the effects on students?

Journal of Further and Higher Education [online]. 30 (3). Pp.229-241 [Accessed 20 March 2012].

Available at: <https://www.ncetm.org.uk/public/files/27892/Malcolm+Swan's+paper.pdf>.

Torrance, H., Colley, H., Garratt, D., Jarvis, J., Piper,H., Ecclestone, K. & James, D. (2005) The impact

of different modes of assessment on achievement and progress in the learning and skills

sector. London: Learning and Skills Research Centre.

Wake, G. (2011) Introduction to the special Issue: Deepening engagement in mathematics in pre-

university education. Research in Mathematics Education - Special Issue: Deepening engagement in

Page 13 of 13

mathematics in pre-university education [online]. 13 (2). Pp. 109-118 [Accessed 29 Feb 2012].

Available at: <http://www.tandfonline.com/doi/abs/10.1080/14794802.2011.585822>.

Ward, J. & Edwards, J. (2002) Learning journeys, Learner's voices. Learners' views on progress and

achievement in literacy and numeracy [online]. London: LSDA. [Accessed 29 Feb 2012]. Available at:

< http://dera.ioe.ac.uk/12139/ >.

Williams, J., Black, L., Davis, P., Hutcheson, G., Nicholson, S., Wake, G., Hernandez-Martinez, P. &

Pampaka, M. (2008) Keeping open the door to mathematically demanding programmes in Further

and Higher Education, Research Briefing 38 [online]. London: Teaching and Learning Research

Programme [TLRP]. [Accessed 6 March 2012]. Available at:

<http://www.tlrp.org/dspace/retrieve/3587/Williams+RB+38+FINAL.pdf>.