http://ltj.sagepub.com/Language Testing

http://ltj.sagepub.com/content/30/4/557The online version of this article can be found at:

 DOI: 10.1177/0265532212475126

2013 30: 557Language TestingAndy Curtis

Book review: Testing the Untestable in Language Education  

Published by:

http://www.sagepublications.com

can be found at:Language TestingAdditional services and information for    

  http://ltj.sagepub.com/cgi/alertsEmail Alerts:

 

http://ltj.sagepub.com/subscriptionsSubscriptions:  

http://www.sagepub.com/journalsReprints.navReprints:  

http://www.sagepub.com/journalsPermissions.navPermissions:  

http://ltj.sagepub.com/content/30/4/557.refs.htmlCitations:  

What is This? 

- Sep 25, 2013Version of Record >>

at UNIV OF SOUTHERN CALIFORNIA on April 4, 2014ltj.sagepub.comDownloaded from at UNIV OF SOUTHERN CALIFORNIA on April 4, 2014ltj.sagepub.comDownloaded from

Language Testing30(4) 557 –563

© The Author(s) 2013Reprints and permissions:

sagepub.co.uk/journalsPermissions.navDOI: 10.1177/0265532212475126

ltj.sagepub.com

Book reviews

A. Paran and L. Sercu (Eds.)Testing the Untestable in Language Education. Bristol, UK: Multilingual Matters. 2010. 269 pp. £27.95 or US$ 49.95 (pbk). ISBN 9781847692658

Reviewed by: Andy Curtis, Anaheim University, USA

This book, which is part of the New Perspectives on Language and Education series, is divided into four main parts, with three chapters in each, plus an introduction and over-view chapter. Part 1 focuses on Intercultural Competence, Part 2 on Autonomy, Part 3 on Literature, and Part 4 on Language and Content.

The book starts with Paran advocating going beyond language, to ‘more than lan-guage … through a focus on additional aspects of language testing, either the content of what is being tested, or additional skills that are related to language learning’ (p. 5), and criticizing the ‘narrowing vision of language tests’, which ‘circumscribe the content that is taught in the language classroom’ (p. 3). Paran is also concerned about what he believes is ‘the continuing tension between assessment of learning and assessment for learning’ (p. 8), and it is these tensions that lie at the heart of the book and connect many of the parts and the chapters together.

As reflected in the title of Chapter 2, co-editor Sercu acknowledges that in relation to assessing intercultural competence, there are ‘more questions than answers’ (p. 17). Indeed, later he notes that ‘as yet, such a holistic measure [of intercultural competence] is not avail-able’ (p. 25). This is one of the recurring themes of the book: while most of the contributors argue the importance of testing such language-related skills and competencies, they also point out the difficulties associated with developing, administering and marking such tests, and few of them are able to give details of any tests. Hence, despite the title, the book is more of a ‘call to arms’ than a blueprint for how these skills might be tested.

In Chapter 3, Korhonen reports on the results of an intercultural training course with a group of Finnish Bachelor of Engineering students taking compulsory English studies. Korhonen used a series of pre- and post-questionnaires to assess the change in the stu-dents’ intercultural sensitivity as a result of the course, concluding that ‘[t]he intercul-tural training process … is slow, gradual and difficult to assess’ (p. 49).

In the final chapter in Part 1, Liddicoat and Scarino present two case studies focusing on the elicitation of intercultural behaviours in language teaching and learning situations. They conclude by recommending the use of portfolios for assessment as these provide ‘multiple exemplars of various aspects of the intercultural’ which ‘capture diverse instan-tiations of the intercultural over time’ (p. 71).

475126 LTJ30410.1177/0265532212475126Language TestingBook reviews2013

at UNIV OF SOUTHERN CALIFORNIA on April 4, 2014ltj.sagepub.comDownloaded from

558 Language Testing 30(4)

Part 2 of the book, on Autonomy, starts with a chapter by Benson, who proposes ‘three poles of attraction in regard to control over learning: student control, other control or no control’ (pp. 79–80). In an interesting extension of some of his earlier work on learner autonomy (see, e.g., Benson, 2001 and 2007), Benson claims that the three poles consti-tute ‘a possible framework for the measurement of autonomy based on observable behaviours’ (p. 82). However, it is not clear how such a model would translate into a measurement tool, and Benson also expresses serious reservations about the whole idea of measuring autonomy, as a result of difficulties that he describes as ‘intractable’ (p. 95).

In Chapter 6, Lamb argues for formative assessment of autonomy, and compares and contrasts Assessment for Learning with Assessment for Autonomy (pp. 99–103). The main focus of the chapter is on the relationships between autonomy, motivation and metacognitive knowledge, based on which Lamb concludes that ‘the framework of meta-cognitive knowledge and beliefs about learning and the tool of the FGC [Focused Group Conversations] offer a way forward in facilitating assessment for autonomy’ (p. 109).

In relation to the title of the book, Dam and Legenhausen start their chapter by stating ‘It is clearly a paradox to claim that one can test the untestable’ (p. 120). The basis of their argument lies in the distinction between testing and evaluation: they note that, when it comes to assessing learner autonomy ‘the obvious and only answer for teachers to get a real insight into the learning process is constant and recurring evaluation where the learners’ voices are heard and taken seriously’ (p. 137). They also add that ‘The history of testing in language pedagogy has shown that testing objectives that seem most worth-while turn out to be recalcitrant to testing’ (p. 121). These kinds of statements may sur-prise or even frustrate some language testers, and although such claims are not as well-supported as the certainty of their position appears to indicate, this book should nevertheless be of interest to language testers, if only so they might respond to, or even dispute, such claims.

One of the chapters that does attempt to give some concrete details of how these ‘untestable’ language skills and competencies might be assessed, if not tested, is Chapter 8, by Paran, which is the first chapter of Part 3, on Literature. Reminiscent of Bailey’s Learning About Language Assessment: Dilemmas, Decisions and Directions (1998), Paran first presents six dilemmas for testing literature in foreign language testing, includ-ing, inter alia, Testing language or testing literature? and Testing knowledge or testing skills?

Paran’s suggestions for the testing and assessment of literature in the language class-room include: using a variety of tasks, giving the test takers choices, providing texts, using portfolio assessment, and making the criteria transparent. These are all useful sug-gestions, although many language testers would claim that they have already been incor-porated into many language tests. Paran also suggests minimizing the weighting of language, although he does accept that might look an odd principle in a discussion of language teaching.

In Chapter 9, Spiro presents a concise and thoughtful consideration of creativity and language learning, in relation to a Language through Literature course co-taught with a colleague, and using data from approximately 100 students in two cohorts. Spiro dis-cusses the value of rubrics in establishing transparency, and the importance of ‘clarity of criteria, congruence and appropriacy of the feedback’ (p. 176). Also, as Spiro explains in

at UNIV OF SOUTHERN CALIFORNIA on April 4, 2014ltj.sagepub.comDownloaded from

Book reviews 559

her introduction, the chapter ‘explores the process of arriving at an assessment of creativ-ity’ (p. 165), so the focus of this chapter, like that of many chapters in this collection, is not so much on the testing of specific language skills, but more on the broader assess-ment of language-related competencies.

In the last chapter in Part 3, Lin describes how he developed ‘a language awareness test’ (p. 192), which he clarifies by explaining ‘As this test was aimed at assessing lan-guage awareness, not testing grammatical knowledge or language proficiency, students were free to respond to the test items in either Chinese or English as they preferred’ (p. 197). Lin draws on Bachman (1990) and Bachman and Palmer (1996) in the creation of his test, and makes the only attempt in the book to describe a test in some detail. However, he also notes that ‘a number of test methods have gone unexplored in this project’ and there is a ‘need to include more distinct elicitation procedures (e.g. fill-in-the-blanks, matching, sentence completion, multiple-choice, short essay) that can tap … the learner’s capacity to handle responses … to literary texts’ (p. 208).

In the first chapter of Part 4 of the book, Language and Content, Mohan, Leung and Slater discuss what they refer to as ‘the integrated assessment of language and content (IALC), with particular reference to second language learning and use’ (p. 217). In rela-tion to ‘testing the untestable’, the authors state clearly and concisely another one of the recurring themes in the book ‘We are talking about assessment in a broad sense, which includes situated classroom assessment processes; we are not talking about “tests” ’ (p. 218). Although Mohan, Leung and Slater’s directness is commendable, this recurring theme does question the validity of the book’s title.

In the next chapter, Low presents an account of ‘classroom assessment practices through teacher talk’ (p. 241). Her study is a discourse analysis of teachers’ discussions, drawing on data gathered from an international college in Canada and from an elemen-tary school in American Samoa, both of which are linguistically interesting and complex language teaching and learning contexts. While the extracts from the teachers’ discourse do shed some light on teacher decision-making processes, and the dilemmas they feel they face, especially when they are evaluating written texts, it is not clear how the discus-sion contributes to a consideration of ‘testing the untestable in language education’.

The last chapter (13), by Mohan, and Slater, presents arguments in favour of a ‘sys-tematic and sustained formative assessment of causal explanations in oral interactions’ (p. 256). This brief concluding chapter balances Low’s by focusing on the discourse of students, who are trying to describe cause and effect relations orally. The chapter focuses on the students’ abilities or inabilities to use grammatically metaphorical constructions, and presents a ‘general path for development of oral causal meanings’ (p. 268), which are important aspects of first- and second-language development. However, the claim that ‘this model has important implications for judging validity in standardized written tests as well as academic oral proficiency interviews’ (p. 268) would need to be more solidly supported.

To sum up, most language testers would probably not consider this to be a book on language testing, in the traditional sense, or even a book on testing given the acknowl-edgement by most authors of the impossibility of using formal testing methods. It is, nonetheless, an interesting and potentially useful collection of international writings ‘about assessment in a broad sense’, as Mohan, Leung and Slater put it. While it appears

at UNIV OF SOUTHERN CALIFORNIA on April 4, 2014ltj.sagepub.comDownloaded from

560 Language Testing 30(4)

as though classroom-based language assessment may have been confused with large-scale language testing, the book should still be of value to language testers who are interested in the assessment of language-related knowledge, skills and competencies, such as intercultural competence, learner autonomy, understanding literature, and con-tent-based instruction.

References

Bachman, L. (1990). Fundamental considerations in language testing. Oxford, UK: Oxford Uni-versity Press.

Bachman, L., & Palmer, A. (1996). Language testing in practice: Designing and developing useful language tests. Oxford, UK: Oxford University Press.

Bailey, K. M. (1998). Learning about language assessment: Dilemmas, decisions and directions. Boston, MA: Heinle Cengage Learning.

Benson, P. (2001). Teaching and researching autonomy in language learning. London: Longman.Benson, P. (2007). Autonomy in language teaching and learning. Language Teaching, 40(1),

21–40.

Barry O’Sullivan (Ed.)Language Testing: Theories and Practices. Palgrave Advances in Linguistics. Basingstoke, UK: Palgrave Macmillan. 2011. 328 pp. £19.99 or US$32.00 (pbk). ISBN 9780230230637

Reviewed by: Sari Luoma, Ballard & Tighe, Publishers, USA

Q: What kind of a reader reads the series editor’s introduction to an edited volume of academic articles?

A: A review writer.

In the case of O’Sullivan’s edited volume, the series editor’s introduction is essential for orienting the potential readers of the volume, including this reviewer. It identifies the target audience – postgraduate and research students and researchers – and states that the book is intended to ‘offer the informed readership a conspectus of perspectives on key themes’ related to the present and future of language testing, as seen through the editor’s eyes’. Furthermore, Candlin’s preface also presents the salient disclaimer that the book is not intended to present an overall guide or provide exhaustive coverage of subfields of language testing. In other words, hardly anyone (bar a review writer) would likely be expected to read the volume cover to cover. On the other hand, the book has something to offer to many language testing students and professionals, thus making it essential for the libraries of institutions that offer courses in language testing at the postgraduate level, and probably the professors who teach such courses. Individual chapters will also be of interest to researchers and postgraduate students who work on topics related to the key themes of the book: test validation, and the development and use of language tests.

The book is divided into four sections. The first is theoretical, the second covers appli-cation of theory to practice, the third includes academic studies of practical test develop-ment projects, and the fourth covers studies of language test use. The first three sections include four articles each, while the last section includes two.

at UNIV OF SOUTHERN CALIFORNIA on April 4, 2014ltj.sagepub.comDownloaded from