c lil 2006 article

The effect of prociency on CLIL benets inEngineering students in Spain

Marta Aguilar Universitat Politcnica de CatalunyaCarmen Muoz Universitat de Barcelona

This paper reports on a follow-up study of a Content and Language IntegratedLearning (CLIL) pilot experience with bilingual postgraduate engineeringstudents at a Spanish university. It aimed at examining learners gains inlistening and grammar skills after a CLIL course in English for a semester, inparticular whether students listening and grammar skills were affectedsimilarly and whether participants proficiency level played a role. Paired-sampled t-tests showed the difference between the mean scores in the pre-andpost-listening test was significant but it was not for the pre-and post grammartests. When students were distributed into three groups on the basis of theirpre-test scores, a repeated measures ANOVA showed that less proficientstudents obtained higher gains in listening and grammar skills than moreproficient students.

Keywords: CLIL, proficiency level, tertiary education, listening skills, grammarskills

Este artculo presenta los resultados de un estudio de seguimiento de unaexperiencia piloto de AICLE con estudiantes bilinges de ingeniera(postgrado) en una universidad espaola. El objetivo del estudio era examinarlas mejoras en las destrezas auditivas y gramaticales despus que losestudiantes hubieran cursado un curso AICLE en ingls durante un semestre;en particular, se trataba de examinar si las destrezas auditivas y gramaticalescambiaban de forma similar y si el nivel de competencia lingstica era unfactor importante. Los t-tests de muestras pareadas mostraron diferencia entrelos valores medios de la comprensin auditiva en el pre-test y el post-test perono en los valores de la destreza gramatical. Cuando se distribuyeron losestudiantes en tres grupos segn sus resultados en el pre-test, la ANOVA demedidas repetidas mostr que los estudiantes con dominio ms bajomejoraban ms que los que tenan un dominio superior tanto en las destrezasauditivas como en las gramaticales.

Palabras clave: AICLE, nivel de competencia, educacin terciaria, destrezasauditivas, destrezas gramaticales

bs_bs_banner

International Journal of Applied Linguistics Vol. 24 No. 1 2014

2013 John Wiley & Sons Ltd doi: 10.1111/ijal.12006

Introduction

The approach known as Content and Language Integrated Learning (CLIL)has been used in the past few years to enhance multilingualism and amultilingual educational approach whereby one or two additional languagesare used as the medium of instruction. However, the term CLIL is used veryoften as an umbrella term for very different teaching experiences, from totaland partial immersion to simply foreign language-medium instruction.

The use of an additional language, and in particular the use of English ininstruction at tertiary level, has a long tradition in many parts of the world fromIndia to someArab countries. In recent times, the use of English as the languageof instruction in European university courses has also increased, motivated bythe internationalization process undertaken within the European EducationalSpace (Wchter and Maiworm 2008). Many universities are increasinglydependent on incomes generated from international programmes, yet notmany incorporate clear and explicit communication and language learninggoals (Fortanet-Gmez and Risnen 2008; Doiz, Lasagabaster and Sierra2011). In fact, at tertiary level informal or tacit inclusion of language goals isreported to be more usual than a CLIL approach proper (Wilkinson and Zegers2008). Greere and Rsnen (2008) classified five major variants of CLIL, rangingfrom LSP courses given by language specialists to (in-between) pre-CLILcourses and adjunct-CLIL courses to fully dual CLIL approaches on thegrounds of main aims, target group, main actors, pedagogical approach, mainview of language, and learning outcomes expected. Grin (2005), for example,using variables like intensity, compulsory status or starting age, identified up to216 types of CLIL programmes.

To make things more complex, CLIL research seems to neglect that itborrows from Content-Based Instruction (CBI) influenced by Swiss andCanadian policies and common in the USA in the late 1990s. In this sense,pioneer studies at tertiary level relate to CBI (Brinton, Snow and Wesche1989; Kasper 1997; Rodgers 2006; Song 2006) and the ecological approach tolanguage (Garner and Borg 2005). Following Greere and Rsnen (2008)classification, CBI could be regarded as a Pre-CLIL approach in the sense thatlanguage specialists, not content specialists, teach and set the goals. Withinthis amalgam of approaches, the following three are often found at tertiarylevel: CLIL (with a fully content and language integration), English-MediumInstruction (EMI) (a content-oriented approach with no linguistic goals) andCBI (a language-oriented approach). In this paper CLIL is used as a coverterm, but strictly speaking EMI is the approach followed in the university weare analysing.

A wealth of research exists at primary and secondary levels. Primary andsecondary school results seem to point to a better language proficiency of thelanguage of instruction if compared with results in formal (traditional)instruction (Ruiz de Zarobe and Jimnez Cataln 2009; Prez-Vidal 2011).Research has shown that CLIL students perform better in receptive tasks than

2 Marta Aguilar and Carmen Muoz

2013 John Wiley & Sons Ltd

in productive tasks (Dalton Puffer 2007), in particular in listening skills, oralproduction and organization skills (Httner and Rieder-Bnemann 2010;Navs and Victori 2010; Ruiz de Zarobe and Lasagabaster 2010, Ruiz deZarobe 2011). CLIL students are seen to outperform non-CLIL students invocabulary acquisition and use (Jimnez Cataln and Ruiz de Zarobe 2007;Mrquez 2007), lexical transfer (Agustn Llach 2009; Celaya and Ruiz deZarobe 2010) or higher lexical richness (Vrkuti 2010). This is in line with theimmersion outcomes described by Swain (1985). Swain noted that immersionstudents progressed in content knowledge and in interpretive skills morethan in expressive (oral and written) skills and hypothesized that the reasonfor this discrepancy was that students processed input semantically (ratherthan syntactically), which implied that the key was making learners modifytheir output (Swain 1995; Swain and Lapkin 1995). However, later researchpoints to secondary CLIL students outperforming Non-CLIL students inwritten production as well (Jexenflicker and Dalton-Puffer 2010; Ruiz deZarobe 2010). Ruiz de Zarobe (2010) for example found out in a longitudinalevaluation that CLIL students outperformed their counterparts in the writtencompetence despite differences in hours of exposure (80 hours in a 3-yearspan) and age differences (CLIL students were 3 years younger), and that thewritten outcomes of younger learners in CLIL were higher than those ofolder Non-CLIL learners, hinting at the conclusion that under CLIL the rateof acquisition is faster. Hence grade, linguistic outcomes and educationalprogrammes seem to be positively related even though the longitudinalsample in the study was small and results have to be taken with caution.As regards listening skills, Muoz (2001) reports on a survey withstudents of English Philology which used a written questionnaire and oralinterviews. Students perceived they had improved particularly in listeningcomprehension after having followed a number of CLIL courses (e.g.literature and culture). The answers to the written questionnaire revealed thatstudents perceived higher gains in receptive skills than in productive skills,and that gains in self-confidence were also high. In Aguilar and Rodrguez(2012), engineering students most important self-reported gains pointed toimprovements in listening skills and acquisition of specialized vocabulary.As Airey (2004) acknowledged out, however, many studies do not assesslanguage competence but rather base their conclusions on the participantsself-reported opinions. He also questions the reliability of many studies inthat researchers usually asked people involved in the experience to expresstheir opinions, and these people were naturally positive to CLIL.

Incidental learning in language outcomes has been acknowledged(Pecorari, Shaw, Irvine and Malmstrm 2011a ; Pecorari, Shaw, Malmstrm andIrvine 2011b), yet CLIL is fraught with problems. The areas that do not seem tobe favourably affected by CLIL are morphosyntax, pronunciation (foreignaccent), informal/non-technical language (Ruiz de Zarobe 2011) and sentence-level dimensions in writing like cohesion, coherence, paragraphing, registerawareness, genre or style (Dalton-Puffer, Nikula and Smit 2010). Some

The effect of prociency on CLIL benets 3


researchers report no negative effect on content learning (Seikkula-Leino2007; Bonnet 2012), while for others content knowledge is sacrificed (Jochems1991;Vinke, Snippe and Jochems 1998; Airey 2004), in particular due tolecturers slower pace of delivery (between 23 and 23.4% slower, Hincks 2010;Thgersen and Airey 2011, respectively) and to increasing lecturer and studentworkload (Vinke et al. 1998, Tatzl 2011). Lecturers inadequate qualification,skills and language proficiency have repeatedly been pointed out (Airey andLinder 2009; Hellekjaer 2010; Aguilar and Rodrguez 2012; Tatzl 2011).University lecturers become less redundant, less expressive and clear inEnglish (Klaassen 2008) and seem to be quite unaware of students linguisticdeficiencies, deploying a limited stylistic lecturing variety, making a poorer/different use of metadiscursive devices (Dafouz and Nez 2010) and seldomrelying on repetition or glossaries of key terms, which is why Tatzl (2011)concludes that the three pillars on which teaching in English-medium highereducation rests are language proficiency, effective lecturing behaviour andpersonal attitude. On the other hand, students are seen to have lecturecomprehension problems caused by poor pronunciation and unfamiliarvocabulary (Hellekjaer 2010) and to understand content more superficially(Airey and Linder 2006; Shaw et al. 2008). Finally, other hurdles of English aslingua franca settings are that textbooks can inhibit rather than enhance contentlearning due to the extra workload (Pecorari et al. 2011a) and that sometimesuniversity students problems in English do not differ much from theirproblems in L1 lectures, basically due to their lack of academic literacy in L1(Airey and Linder 2009; Airey 2010; Hellekjaer 2010).

Sometimes contradictory and inconclusive results have been obtainedacross schools and across countries at all educational levels (Bonnet 2012). Onthe other hand, it is important to note that in some studies reporting highergains, CLIL students have had more exposure to the target language thannon-CLIL students, so it is not clear whether these are effects of increasedexposure or of the CLIL intervention itself. An exception is the study byVillarreal (2011) comparing groups of CLIL and non-CLIL secondary studentsin the oral production of some features of verbal morphology. In a firstcomparison with age-matched groups and different amounts of instructionhours, the CLIL students showed a slight advantage over the non-CLILstudents. But in a second comparison in which CLIL and non-CLIL studentswere matched for amount of instruction hours, the latter showed a clearadvantage over the former. Since the non-CLIL students were 3 years older(1718 vs. 1415), Villarreal suggests this result may be attributed to theirsuperior cognitive maturity (see Muoz 2006). This contrasts with thefindings in Ruiz de Zarobes study (2010) mentioned above and at the sametime partly confirms the areas where CLIL has no or little effect, as animportant finding of this study is that CLIL effects seem weaker than ageeffects, at least as far as verbal morphology is concerned. Another approach tothe time issue is to consider how much extra exposure needs to be providedby CLIL courses, since it might be argued that slight differences in the number



of hours (in both CLIL and non-CLIL courses) might be insufficient for CLILbenefits to clearly emerge among university students. In secondary education,it has been demonstrated that types of instruction with more exposure toEnglish achieve higher levels of proficiency than instruction with lessexposure, provided students receive sufficient exposure to the target language(Ruiz de Zarobe 2010), yet research on what type of instruction is mostbeneficial and how intense exposure at tertiary level should be is to ourknowledge still limited. Studies point out that stay abroad allows for greaterbenefits than CLIL and that CLIL yields better results than formal instructionamong children and teenagers (Prez-Vidal 2011). Evidence concerning theperiod of time (and the intensity) needed for fully benefiting from long-termuniversity CLIL programmes is still under way.

All in all, little is known about the effect of CLIL on university studentsoverall proficiency. Given that learners in tertiary education in particularcome equipped with different language competencies and backgrounds,learners proficiency level deserves consideration. Magnan (1986) probed therelationship between speaking rates and level of study at tertiary level. Afterinvestigating the level of speaking proficiency in French and the level of studyin undergraduates, he proposed sound foreign language requirements andteacher certification. Nevertheless, few researchers have tackled the issue ofwhether there is a threshold level that pupils should have attained beforeoptimally benefiting from CLIL classes (Muoz 2010), although the issue hasbeen addressed in reference to university immersion and study abroadcontexts. First, in the context of university immersion, research (Klee andTedick 1997; Lynch, Klee and Tedick 2001) has found that an initial proficiencylevel of Intermediate High on the OPI1 is the minimum optimal level forpredicting social and academic success and target language proficiencygrowth. Second, in the context of study abroad, DeKeyser (2010: 90) hasargued that students must have adequate basic knowledge of the structure ofthe language if they are to have any meaningful practice experience abroad,though he has not suggested a specific optimum proficiency level. Finally,another important aspect in the CLIL approach that has not been sufficientlyresearched is the relationship or interaction between students gains andteachers proficiency, and the optimal (or minimal) teachers proficiency levelfor linguistic gains to emerge. At tertiary levels, it is often the case thatstakeholders linguistic and cognitive levels differ. Adapting Cumminsmatrix, Coyle (2007: 554) posits that in situations with a mismatch (thelanguage level of learners is lower than their cognitive level), the learningenvironment must ensure that cognitive progression is maintained byaccessing content through a lower linguistic level (Quadrant 3), graduallyworking towards higher linguistic demands (Quadrant 4). At universitylevel, therefore, the mismatch between lecturers and students linguisticproficiency and the complexity of the content to be conveyed should bebrought to the fore in relation to student gains and to how lecturers speechdiverges from native and disciplinary speech. At both secondary and tertiary



levels, as mentioned above, studies point to teachers pragmatically lessvaried and less subtle language (Dafouz and Nez 2010; Nikula 2010; Dafouz2011) and to inadequate materials, either because they are inadequatelyadjusted or rediscursified materials (Lorenzo 2008) or because they are highlydemanding university textbooks (Pecorari et al. 2011a, b).

The present study aims at exploring the benefits of a CLIL experience onthe language outcomes of a group of engineering students. Specific attentionwill be paid to the role of students proficiency level in English (the targetlanguage), and to how proficiency may have an effect on the language gainsobtained through a CLIL course. Hence, with the aim of gaining insight intothe overall impact of the experience as well as of furthering knowledge aboutthe skills that seem to benefit the most, and about the role played by learnersgeneral proficiency in the target language, the following research questionswere established:

1. What is the effect of following a CLIL course in English for a semester ona group of Engineering students listening and grammar skills? Are thereany statistically significant gains after the CLIL experience?

2. Are the students listening and grammar skills similarly affected by theCLIL experience?

3. What is the role played by the participants proficiency level?

Background of the study

This study was conducted as a follow-up of a CLIL experience in a school ofengineering at a Spanish university. In fact, this was the schools secondattempt to implement CLIL. A general call for volunteers made two yearsearlier had been a total failure only seven lecturers out of 112 volunteered toteach in English. This time it was deemed more cautious to implement CLILgradually, so the second step consisted in choosing a few courses at a pilotstage. The criteria adopted to choose the courses were the following: (i)courses where the lecturer agreed and felt prepared to teach in English; (ii)courses with different groups so that students could choose the language ofinstruction; (iii) postgraduate courses; and (iv) practical and interactivecourses where problem-solving, case studies or projects were usual teamworkactivities, rather than highly theoretical lessons. Accordingly, seven coursesfrom different disciplines engineering projects (two groups), businessorganization, energy technology, planning and programming of logisticactivities, manufacturing technologies, world economy, and statisticalmethods were formally offered in the autumn semester of 20082009.Another decision made at that stage was to ask the university to collaborate intwo aspects: (i) provide internships (students) that would support lecturers inthe translation of their powerpoint material; and (ii) perform a follow-up



study of the experience. It was not until the following year that teachers wereoffered teacher training courses.

Information about the kind of classroom activities was necessary todocument the type of instruction. Classroom observation was not feasible, sodiscussion with the lecturers and some students shed some light. It is worthmentioning that there was no CLIL methodological adaptation andthat materials were therefore not grounded in any language teachingmethodology: powerpoint slides usually shown in class had been merelytranslated to English. Similarly, lecture instruction basically consisted in theteacher explaining with the help of powerpoint slides and students takingnotes during the first part of the lessons; then students worked in pairs or inteams, doing some related problem-solving or discussion tasks. Studentsoften had some recommended readings, and in three courses they worked inteams on a given topic for some weeks and at the end of the semesterdelivered an oral presentation and submitted a written summary (poster) oftheir presentation. In the rest of the courses, every group had to gatherinformation on a specific topic and deliver an oral presentation.

Method

Participants

The original sample of students was composed of 205 students in anengineering school, with an average age of 21.4, and with an upper-intermediate level of English (B2) at the pre-test. However, not all of thestudents took the post-test and thus the sample was reduced to 63 students.

Out of these 63 students, 82.6% were Spanish students and 17.4% wereforeign students in an Erasmus exchange. All of the Spanish students werebilingual (Catalan-Spanish). The Erasmus students came from Germany,France, Italy, Poland, the Netherlands, Denmark, Portugal and Slovenia.Erasmus students were initially separated in a subgroup but since theiraverage pre-test results did not differ from those of Spanish students, theywere merged together with the latter in the same group.

The participants were distributed in six 15-week, 4 hour/week courses(which is a total of 60 hours). Students followed the same degree programmeand had the same number of CLIL lectures.

Instruments

The test (Oxford Placement Test, OPT) consisted of a listening subtest and agrammar subtest. Two equivalent forms (Allan 2005a, b) were used as pre-test(first week of the semester) and post-test (last week of the semester),respectively.



Results

The results obtained by the whole group on the two subtests of the OPT atboth times (pre-test and post-test) are displayed in Table 1. The description ofthe scores obtained at the beginning and at the end of the CLIL course showsthat the mean score in both the listening test and the grammar test is slightlyhigher at the post-test. A paired-sampled t-test showed that the differencebetween the mean scores in the listening test was significant, with a smalleffect size (t = -3.468, p < 0.01, d = 0.30). In contrast, the comparison of themean scores in the grammar test by means also of a paired-sampled t-testshowed that the gains experienced by these learners during the course weretoo small to be statistically significant.

In order to answer the second research question concerning the role playedby proficiency in the language gains, students were distributed into three groupsof similar size on the basis of their scores at the pre-test, both on the listening testand on the grammar test. For the listening test, the lowest level groups scoresranged from 53 to 68, the intermediate group scored between 70 and 77, and thethird group ranged in scores between 79 and 92. For the grammar test, the lowestlevel group scored between 37 and 60, the intermediate group between 61 and 72,and the more advanced between 74 and 96. The respective sub-groups wereseparately entered in the analyses that explored the effects of proficiency onlistening and grammar skills, respectively.

First, a repeated measures ANOVA was conducted with the scores at Time1 and at Time 2 in the listening test as the within-subjects factor, and theproficiency group with three levels as the between-subjects factor. Table 2displays the descriptive statistics (mean and SD) of the three groups intowhich the sample was divided. While group 1 and group 2 can be observed toexperience some gains, higher in the former than in the latter, group 3 scoresslightly lower at Time 2.

The tests of within-subjects effects show a significant main effect for thevariable Time with a medium effect size (F1,60 = 13.78, p = 0.000, partial etasquared = 0.19, power = 0.95). The interaction between the variable time andthe variable proficiency is also significant and with a large effect size (F2,60 =7.9, p = 0.006, partial eta squared = 0.16, power = 0.85). This result indicates thatparticipants from the different proficiency groups did not obtain the same

Table 1. Descriptive statistics

N Minimum Maximum Mean Std. deviation

Pre-listening 63 53 92 72,75 9,670Pre-grammar 63 37 96 66,30 14,166Post listening 63 59 97 75,57 9,129Post-grammar 63 36 96 67,16 12,746



Table 2. Descriptive statistics: listening scores

Group Mean Std. Deviation N

Pre-listening 1 62,77 4,231 222 72,24 2,221 213 84,25 5,046 20

Total 72,75 9,670 63Post-listening 1 68,41 5,124 22

2 75,33 6,256 213 83,70 8,529 20

Total 75,57 9,129 63

Time21

Est

imat

ed M

arg

inal

Mea

ns

90

85

80

75

70

65

60

321

groups

Figure 1. Evolution of the three prociency groups in listening skills



gains in listening skills after the course. Pairwise comparisons among theestimated marginal means show that the effects of time are significant forgroup 1, with the lowest level of proficiency (F = 23.70, p = 0.000, partial etasquared = 0.26, power = 0.998), marginally significant for group 2 (F = 3.58, p= 0.063, partial eta squared = 0.05, power = 0.46), and non-significant for group3. Figure 1 shows the evolution of the three proficiency groups in listeningscores from Time 1 to Time 2.

A repeated measures Anova was conducted with the scores from thegrammar tests at time 1 and time 2, as above. Table 3 displays the descriptivestatistics (mean and SD) of the three groups into which the sample wasdivided. Groups 1 and 2 experience some gains, larger in the former than inthe latter, but in group 3 the mean score at the pre-test is higher than at thepost-test.

The tests of within-subjects effects show no main effect for the variableTime, indicating again that participants gains were not statistically significant.However, a significant interaction is shown between proficiency group and

Time21

Est

imat

ed M

arg

inal

Mea

ns

90

80

70

60

50

321

groups

Figure 2. Evolution of the three prociency groups in grammar skills



time with a large effect size (F2,60 = 7.9, p = 0.001, partial eta squared = 0.21,power = 0.94). This result indicates that participants from the differentproficiency groups did not obtain the same gains in grammar after the course.Pairwise comparisons among the estimated marginal means show that theeffects of time are significant for group 1, with the lowest level of proficiency(F = 12.97, p = 0.001, partial eta squared = 0.18, power = 0.94), non-significantfor group 2, and significant for group 3, with a medium effect size (F = -2.7,p = 0.05, partial eta squared = 0.06, power = 0.50). In other words, whereas thegains in group 1 are significant, so are the losses in group 3, and no significantchanges are observed in group 2. Figure 2 shows the evolution of the threeproficiency groups in grammar scores from Time 1 to Time 2.

Discussion

The first research question of this study inquired into the gains in Englishlanguage proficiency obtained by students that followed a 60-hour coursetaught in English at an engineering school. In order to answer this question,participants scores were compared before and after the course by means of alistening test and a grammar test. Results show that there was a smallimprovement in test performance.

However, and in relation to the second research question specifically,only gains in listening skills reached statistical significance, whereas theimprovement in grammar skills was too small. Therefore, the findingconcerning gains in listening skills provides confirmation that this is an areain which students may benefit from a CLIL experience. The evidenceregarding improvement in listening skills is also in line with universitystudents self-perception of improvement in listening comprehension, asshown in Muoz (2001) with students in an English degree, and in Aguilarand Rodrguez (2012) with a group of Engineering students that includedparticipants in this study.

The third research question inquired about the effects of studentsproficiency on their language gains after following the CLIL course. The

Table 3. Descriptive statistics: grammar scores

Group Mean Std. Deviation N

Pre-grammar 1 51,05 7,410 222 66,65 3,329 203 81,95 6,368 21

Total 66,30 14,166 63Post-grammar 1 55,82 7,391 22

2 66,95 8,587 203 79,24 9,027 21

Total 67,16 12,746 63



findings in this study clearly revealed an effect of proficiency. Students withan advanced initial level of proficiency in English (OPT Band 6) did notexperience any gains after the CLIL experience in either listening or grammarskills, as measured by the standardized tests used in this study. Further, thedecrease in grammar scores is significant, indicating that those participantswho were advanced users of English at the beginning of the course performedworse at Time 2. In contrast, participants with the lowest initial level ofproficiency in English (elementary level or OPT Band 3) obtained significantgains in both listening and grammar skills, which indicates that theybenefited from following the CLIL course. As for the participants with anupper intermediate level (OPT Band 5), their evolution in listening skills ispositive although it does not reach statistical significance, and their evolutionin grammar skills is extremely small.

This study has revealed the important role played by students proficiencyin university CLIL courses because the CLIL experience was moreadvantageous for less proficient students, hinting at the time horizon requiredto advance proficiency levels (less proficient students show greater gains inshorter periods than more proficient students). Previous research has focusedon the minimum initial proficiency level needed for learners to benefit fromuniversity immersion or partial immersion programmes (Klee and Tedick1997; Lynch et al. 2001). Specifically in the context of adult advanced-leveluniversity immersion, an initial proficiency level of Intermediate High(ACTFL guidelines) has been suggested as the minimum optimal level forpredicting target language proficiency growth (Klee and Tedick 1997; Lynchet al. 2001). However, in this study it has been shown that elementary levelstudents benefit more from the CLIL experience provided by just one 60-hourcourse than those students with an Upper Intermediate or Advanced level onthe OPT. This discrepancy may be attributed to the participants differentcharacteristics in the two settings. One may hypothesize that the absence ofgains in the more proficient learners in the present study may be attributableto the characteristics of the input received in class, probably more limited thanthe input received by students in the university immersion programmesreported above. As mentioned earlier, teaching materials (powerpointpresentations) had been translated into English for the teachers, who in somecases may have restricted their oral explanations to reading that material, thusproviding a poor linguistic environment. No adaptation along the lines ofCoyle (2007) was made.

In fact, the lecturers in this study had a relatively limited proficiency.Scores in the OPT they also took at the beginning of term placed them in fourdifferent bands: two in OPT Band 4 (lower intermediate), three in OPT Band5 (upper intermediate), one in OPT Band 6 (advanced), and one in Band 7(very advanced). This is lower than, for example, the recommended pre-requisite for university teaching assistants in many US universities, whichis advanced plus or superior (Magnan 1986). If lecturers proficiency isso limited that they cannot provide enriched input, students language



improvement may be endangered (Sercu 2004; Lasagabaster 2008). It may beargued that for those students whose proficiency level was higher than theirteachers, the experience may not have led to much improvement, as in thecase of the advanced students grammar. In that context, CLIL benefits may berestricted to students with a level of proficiency below the intermediate levels.However, research into different measures of English proficiency other thanthe OPT, with a greater sample of students, and with longer amounts of CLILexposure is necessary to corroborate this hypothesis.

When asked about the quality of delivered content, the lecturers in thisstudy stated that the same amount of topics had been covered in their Englishgroup as in the Spanish groups and that the failure rate in final exams did notvary significantly from that of the latter. It would be interesting to study thequality of content and types of examinations in depth. Informal interviewswith some of the lecturers revealed that exams were essentially numerical orfollowing a multiple-choice format. Such formats may not display the deepunderstanding of content as, for example, an exam requiring students toexplain the rich complexity of content in length and in depth. Research hascompared lecturers behaviour and experiences before and after receivingCLIL teacher training (Klaassen 2008), but it would also be interesting tocompare the effects of lecturer training on students gains, and on different-ability students gains.

However, and in spite of the shortcomings of the courses in this study, lessproficient students have been observed to benefit from the experience. It maybe argued that if benefits emerge even from a non-articulated CLILprogramme as the one in this study (and with not highly proficient lecturers)maybe even better results could be expected if students followed a carefullydesigned and planned CLIL course with a special emphasis on languagelearning (Prez-Vidal 2007; Navs 2011). In relation to the implementation ofCLIL courses at university level, further comparative research betweenuniversity approaches like CLIL, EMI and CBI comparing the advantages ofeach learning context could provide interesting insight.

This study has a number of limitations. One limitation is that the groupswere natural groups and there was no control of students allocation toteachers. Although teachers were asked about their methodology in class(mostly teacher-fronted lecturing with some variability in the opportunitiescreated for students oral contribution), classroom observations were notpossible and hence the actual use of the target language by students in classremains unknown. A second limitation concerns the fact that students wereself-selected, which indicates a degree of motivation that cannot be assumedto be generalizable to the whole population (Airey 2004). A third caveat isthe short length of the experience, since the period of 15 weeks per semestermay prove to be too short. It has been shown that, for example, two to threemonths of university-level instruction provide negligible change in theanalysed syntactic complexity measures of L2 writing (Ortega 2003). Finally,while this study has yielded interesting findings concerning the listening



comprehension skills of university students as well as their performance ona grammar test, further research should also examine how universitystudents with different entry proficiency levels benefit from CLIL, EMI, andother instruction types, which should always be embedded in a robustcontextualized framework with clear aims and projected outcomes (Coyle2007: 546).

Note

1. Oral Proficiency Interviews, following the American Council on the Teaching ofForeign Languages (ACTFL) guidelines.

References

Aguilar, M. and R. Rodrguez (2012) Implementing CLIL at a Spanish university.Lecturer and student perceptions. The International Journal of Bilingual Educationand Bilingualism 15.2: 18397.

Agustn Llach, M. P. (2009) The role of Spanish L1 in the vocabulary use of CLIL andnon-CLIL EFL learners. In Y. Ruiz de Zarobe and R. M. Jimnez Cataln (eds.),Content and language integrated learning. Evidence from research in Europe. Bristol:Multilingual Matters. 11229.

Airey, J. (2004) Can you teach it in English? Aspects of the language choice debate inSwedish higher education. In R. Wilkinson (ed.), Integrating content and language.Meeting the challenge of a multilingual education. Maastricht: Maastricht University.97108.

(2010) The ability of students to explain science concepts in two languages. Journalof Language and Communication Studies 45.1: 3549.

and C. Linder (2006) Language and the experience of learning university physics inSweden. European Journal of Physics 27.3: 55360.

(2009) A Disciplinary discourse perspective on university science learning:achieving fluency in a critical constellation of modes. Journal of Research in ScienceTeaching 40.1: 2749.

Allan, D. (2005a) Oxford placement test 1. Oxford: Oxford University Press. (2005b) Oxford placement test 2. Oxford: Oxford University Press.Bonnet A. (2012) Towards an evidence base for CLIL. International CLIL Research Journal

1.4: 6678.Brinton, D., M. Snow and M. Wesche (1989) Content-based second language instruction.

Rowley, MA: Newbury House.Celaya M. L. and Y. Ruiz de Zarobe (2010) First languages and age in CLIL and

non-CLIL contexts. International CLIL Research Journal 1.3: 606.Coyle, D. (2007) Content and language integrated learning: towards a connected

research agenda for CLIL pedagogies. The International Journal of BilingualEducation and Bilingualism 10.5: 54362.

Dafouz, E. (2011) English as the medium of instruction in Spanish contexts: a lookat teacher discourses. In Y. Ruiz de Zarobe and R. M. Jimnez Cataln (eds.),Content and Foreign Language Integrated Learning. Contributions to Multilingualism inEuropean Context. Bern: Peter Lang. 189210.



and B. Nez (2010) Metadiscursive devices in university lectures: a contrastiveanalysis of L1 and L2 teacher performance. In C. Dalton-Puffer, T. Nikula andU. Smit (eds.), Language use and language learning in CLIL classrooms. Amsterdam:John Benjamins. 21332.

Dalton-Puffer, C. (2007) Discourse in content and language integrated (CLIL) Classrooms.Amsterdam: John Benjamins.

T. Nikula and U. Smit (eds.) (2010) Language use and language learning in CLILClassrooms. Amsterdam: John Benjamins.

DeKeyser, R. (2010) Monitoring processes in Spanish as a second language during astay abroad program. Foreign Language Annals 43.1: 8092.

Doiz, A., D. Lasagabaster and J.M. Sierra (2011) Internationalisation, multilingualismand English-medium instruction. World Englishes 30.3: 34559.

Fortanet-Gmez, I. and C. Risnen (eds.) (2008) ESP in European higher education.Integrating language and content. Amsterdam: John Benjamins.

Garner, M. and E. Borg (2005) An ecological perspective on content-based instruction.Journal of English for Academic Purposes 4.2: 119134.

Greere,A. and A. Rsnen (2008) Lanqua subproject on content and language integratedlearning. Redefining CLIL. Towards multilingual competence. Year One Report.Retrieved July 2012 from (http://www.lanqua.eu/files/Year1Report_CLIL_ForUpload_WithoutAppendices_0.pdf ).

Grin, F. (2005) Added value of CLIL. Paper presented at The Changing EuropeanClassroom. The Potential of Plurilingual Education Conference. Luxembourg, 911March.

Hellekjaer, G. O. (2010) Language matters: assessing lecture comprehension inNorwegian English-medium education. In C. Dalton-Puffer, T. Nikula and U. Smit(eds.), Language use and language learning in CLIL classrooms. Amsterdam: JohnBenjamins. 23358.

Hincks, R. (2010) Speaking rate and information content in English lingua franca oralpresentations. English for Specific Purposes 20.1: 418.

Httner, J. and A. Rieder-Bnemann (2010) A cross-sectional analysis of oral narrativesby children with CLIL and Non-CLIL instruction. In C. Dalton-Puffer, T. Nikulaand U. Smit (eds.), Language use and language learning in CLIL classrooms.Amsterdam: John Benjamins. 6179.

Jexenflicker, S. and C. Dalton-Puffer (2010) The CLIL differential: comparing thewriting of CLIL and non-CLIL students in higher colleges of technology. In C.Dalton-Puffer, T. Nikula and U. Smit (eds.), Language use and language learning inCLIL classrooms. Amsterdam: John Benjamins. 16989.

Jimnez Cataln, R.M. and Y. Ruiz de Zarobe (2007) Does the type of instruction haveany bearing on EFL learners receptive vocabulary? Paper presented at the ELIA 10Conference. Issues in teaching, learning, and using vocabulary in an L2. Universityof Seville, Spain.

Jochems, W. (1991) Effects of learning and teaching in a foreign language. EuropeanJournal of Engineering Education 16.4: 30916.

Kasper, L. F. (1997) The impact of content-based instructional programs onthe academic progress of ESL students. English for Specific Purposes 16.4: 30920.

Klaassen, R. (2008). Preparing lecturers for English-medium instruction. In R.Wilkinson and V. Zegers (eds.), Realizing content and language integration in highereducation. Maastricht: Maastricht University: 3242.



Klee, C. and D. Tedick (1997) The undergraduate foreign language immersion programin Spanish at the University of Minnesotta. In S. B. Stryker and B. L. Leaver (eds.),Content-based instruction in foreign language education. Washington, DC: GeorgetownUniversity Press. 14173.

Lasagabaster, D. (2008). Foreign language competence in content and languageintegrated courses. The Open Applied Linguistics Journal 1.1: 3142.

Lorenzo, F. (2008) Instructional discourse in bilingual settings. An empirical study oflinguistic adjustments in content and language integrated learning. LearningLanguage Journal 36.1: 2133.

Lynch, A., C. A. Klee and D .J. Tedick (2001) Social factors and language proficiencyin postsecondary Spanish immersion: issues and implications. Hispania 84.3:51024.

Magnan, S. S. (1986) Assessing speaking proficiency in the undergraduate curriculum:data from French. Foreign Language Annals 19.5: 42938.

Mrquez, M. (2007) Models of CLIL: an evaluation of its status drawing on the Germanexperience. A critical report on the limits of reality and perspectives. RESLA.Volumen monogrfico: 95111.

Muoz, C. (2001) The use of the target language as the medium of instruction.University students perceptions. Anuari de Filologia 23.1: 7182.

(2006) The effects of age on foreign language learning: the BAF project. In C. Muoz(ed.), Age and the rate of foreign language learning. Clevedon: Multilingual Matters.140.

(2010) Challenges of bilingual education in Spain: CLIL from the perspective oflanguage acquisition. Plenary talk given at SEDLL (Sociedad Espaola de Didcticade la Lengua y la Literatura). Universidad de Jan, December.

Navs, T. (2011) How promising are the results of integrating content and languagefor EFL writing and overall EFL proficiency? In Y. Ruiz de Zarobe, J. M. Sierraand F. Gallardo del Puerto (eds.) Content and foreign language integratedlearning. Contributions to multilingualism in European context. Bern: Peter Lang.15586.

and M. Victori (2010) CLIL in Catalonia: an overview of research studies. In D.Lasagabaster and Y. Ruiz de Zarobe (eds.), CLIL in Spain. Implementation, results andteacher training. Newcastle upon Tyne: Cambridge Scholars Publishing. 3054.

Nikula, T. (2010) Effects of CLIL on a teachers classroom language use. In C. Dalton-Puffer, T. Nikula and U. Smit (eds.), Language use and language learning in CLILclassrooms. Amsterdam: John Benjamins. 10521.

Ortega, L. (2003) Syntactic complexity measures and their relationship to L2proficiency: a research synthesis of college-level L2 writing. Applied Linguistics24.4: 492518.

Pecorari, D., P. Shaw, A. Irvine and H. Malmstrm. (2011a) English for academicPurposes at Swedish universities: teachers objectives and practices. Ibrica 22.1:5577.

H. Malmstrm and A. Irvine. (2011b) English textbooks in parallel-languagetertiary education. TESOL Quarterly 45.2: 31333.

Prez-Vidal, C. (2007) The need for focus on form (FoF) in content and languageintegrated approaches. An exploratory study. RESLA Volumen monogrfico:3954.

(2011) Language acquisition in three different contexts of learning: formalinstruction, stay abroad, and semi-immersion (CLIL). In Y. Ruiz de Zarobe,



J. M. Sierra and F. Gallardo del Puerto (eds.) Content and foreign language integratedlearning. Contributions to multilingualism in European context. Bern: Peter Lang.10327

Rodgers, D. (2006) Developing content and form: encouraging evidence from Italiancontent-based instruction. The Modern Language Journal 90.3: 37386.

Ruiz de Zarobe, Y. (2010) Written production and CLIL. In C. Dalton-Puffer, T. Nikulaand U. Smit (eds.), Language use and language learning in CLIL classrooms.Amsterdam: John Benjamins. 191209.

(2011) Which language competencies benefit from CLIL. In Y. Ruiz de Zarobe, J. M.Sierra and F. Gallardo del Puerto (eds.) Content and Foreign Language IntegratedLearning. Contributions to Multilingualism in European Context. Bern: Peter Lang.12953.

and R. M. Jimnez Cataln (eds.) (2009) Content and language integrated learning.Evidence from research in Europe. Bristol: Multilingual Matters.

and D. Lasagabaster (2010) CLIL in a bilingual community: the Basque autonomouscommunity. In D. Lasagabaster and Y. Ruiz de Zarobe (eds.), CLIL in Spain.Implementation, results and teacher training. Newcastle upon Tyne: CambridgeScholars Publishing. 1229.

J. M. Sierra and F. Gallardo del Puerto (eds.) (2011) Content and foreign languageintegrated learning. Contributions to multilingualism in European context. Bern: PeterLang.

Seikkula-Leino, J. (2007) CLIL learning: achievement levels and affective factors.Language and Education 21.4: 32841.

Sercu L. (2004) The introduction of English-medium instruction in universities.A comparison of Flemish lecturers and students language skills, perceptionsand attitudes. In R. Wilkinson (ed.), Integrating content and language. Meetingthe challenge of a multilingual education. Maastricht: Maastricht University. 54755.

Shaw, P., C. Benson, S. Brunsberg, R. Druhs and D. Minugh (2008). Internationalmasters in Stockholm. In I. Fortanet-Gmez and C. A. Risnen (eds.), ESP inEuropean higher education. Amsterdam: John Benjamins. 26782.

Song, B. (2006) Content-based instruction: long-term effects and outcomes. English forSpecific Purposes 25.4: 42037.

Swain, M. (1985) Communicative competence: some roles of comprehensible inputand comprehensible output in its development. In S. Gass and C. Madden (eds.),Input in second language acquisition. Rowley, MA: Newbury House. 23553.

(1995) Three functions of output in second language learning. In G. Cook and B.Seidholfer (eds.), Principles and practice in applied linguistics: studies in honour of H.G.Widdowson. Oxford: Oxford University Press. 12544.

and S. Lapkin (1995) Problems in output and the cognitive processes theygenerate: a step toward second language learning. Applied Linguistics 16.3: 37191.

Tatzl, D. (2011) English-medium masters programmes at an Austrian university ofapplied sciences: attitudes, experiences and challenges. Journal of English forAcademic purposes 10.4: 25270.

Thgersen, J. and J. Airey (2011) Lecturing undergraduate science in Danish and inEnglish: a comparison of speaking rate and rhetorical style. English for SpecificPurposes 30.3: 20921.

Vrkuti, A. (2010) Linguistic benefits of the CLIL approach. International CLIL ResearchJournal 1.3: 6779.



Villarreal, I. (2011) Tense and agreement in the non-native English of Basque-Spanishbilinguals: content and language integrated learners vs. English as a school subjectlearners. Doctoral Dissertation, Universidad del Pas Vasco.

Vinke, A., J. Snippe and W. Jochems (1998) English-medium content courses in non-English higher education: a study of lecturer experiences and teaching behaviours.Teaching in Higher Education 3.3: 383394.

Wchter, B. and F. Maiworm (2008) English-taught programmes in European highereducation. The picture in 2007. Bonn: Lemmens.

Wilkinson, R. and V. Zegers (eds.) (2008) Realizing content and language integration inhigher education. Maastricht: Maastricht University.

email: [email protected]@ub.edu

[Received 7 October 2012]



c lil 2006 article

Documents