culi’s annual research seminar

CULI’s Annual Research Seminar

Pisamai Supatranont, Ph.D.

Rajamangala University of Technology Lanna [email protected]

November 28, 2007 at Chulalongkorn University

Background of the Study

Rajamangala University of Technology Lanna Tak(RMUTL Tak)

Engineering students at an undergraduate level

Students’ need = Reading academic texts in English

Due to high technology transfer from the West

To solve a problem of

students’ insufficient vocabulary size for academic reading

An Action Research

Problem Teaching reading skills & strategies is unsuccessful

Students’ inadequate knowledge of vocabularyCause

Evidence

Teacher’s observation

Students’ scores from Vocabulary Level Tests (Developed by Nation and Schmitt, Schmitt & Clapham in Nation 2001)

= under desirable criteria in all 3 levels regarded as a critical basis for academic reading

I. Background of the Study


Why? Lexical threshold to academic reading

To read academic texts comprehensibly, 95% coverage of words known in that text is the minimum point (Laufer, 1988).

Around 3,000 high frequency words are estimated to provide over 90% coverage of academic texts in all disciplines. (Nation, 2001; Coxhead & Nation, 2001; Cobb & Horst, 2001; and Nation & Waring, 1997)

General service list (GSL) = 2,000 high frequency words (West, 1953)

(and) University word list (UWL) = 800 academic words (Xue & Nation 1984)

(or) Academic word list (AWL) = 570 academic words (Coxhead, 1998)

2,0002,000 words words (GSL)(GSL) = about = about 8080%% coverage in most texts coverage in most texts

3,0003,000 words words (GSL & UWL/AWL) (GSL & UWL/AWL) = about = about 90%90% in most academic texts in most academic texts (Cobb & Horst, 2001)(Cobb & Horst, 2001)

i.e.

At RMUTL, Tak

Ss’ existing knowledge

About 1000 words(based on vocabulary level

tests)

Lexical threshold to academic reading

About 3000 words e.g. the GSL & AWL

1000 kn. words

3000 needed words

Knowledge gap= 2000 words

Therefore To improve reading proficiency, students’ lexical knowledge gap must, first and foremost, be bridged.


Only 1 semester

However,There are a few EAP courses &no courses focus on vocabulary.

Vocabulary component has to be integrated into an existing reading course.

As a result, Vocabulary learning has to be accelerated. A conventional method cannot prepare students properly

in a short time.

Classroom Concordancing = the concordance-based method

A potential solution to such a problem !!!

The method is applied from a corpus technique widely used for linguistic analysis in lexicography, linguistics, and computational linguistics. When being introduced to language instruction a few decades ago, the method was mostly used among designers of curricula, syllabuses, & materials.

At present, it is increasingly encouraged to be used directly in language classrooms.

Since a corpus and a concordancer are always used with this method, the term ‘corpus-based’ and ‘concordance-based’ method usually co-occur in related literature.

In language pedagogy, the practical aspect of the approach is often referred to as ‘classroom concordancing’.

Classroom Concordancing

Equipped with a corpus and a concordancer:

Corpus compilation

in an electronic form

Methods of text collection

OCRStudy a corpus with

a concordancer

Corpus / Corpora = A collection of texts compiled for linguistic purposes

Concordancer = A software program to study language in corpora

Concordance = A display of data in a KWIC (keyword-in-context) format

Description Around 500,000 running words Compiled from academic texts in engineering:

• textbooks & handouts• manuals & instructions• advertisements• news & articles• abstracts of articles / research

OCR

Corpus compilation

• Mostly from webpages

• Some from wordprocessing programs

In the Study

The Engineering CorpusA purpose-built corpus

A Concordancer => WCONCORD

• Freeware downloadable from http://www.kapohl.de/ • Developed by Zdenek Martinek & Leslie Siegrist (1996)

Basic Functions of WCONCORD

• Build word frequency list• Find frequency of each word• Display statistic information of a corpus

Basic Functions of WCONCORD

• Search & sort words• Display word behaviours in various contexts

Concordances in a KWIC (keyword-in-contexts) format

Keyword

Immediate contexts

Full sentence

Source text

was applied as a means:

To enable students to learn words in multiple authentic contexts.

To train them necessary skills for independent learning.

Classroom concordancing

In the Study

II. Research Questions

Can the concordance-based method significantly increasestudents’ vocabulary size?

1

Can the concordance-based method significantly increasestudents’ ability to transfer lexical knowledge to new contexts?2

What are students’ attitudes towards the application of the concordance-based method?5

Can the concordance-based method significantly increasestudents’ retention rates of vocabulary knowledge?3

What are the processes used by the students when dealingwith the concordance output?4

Objectives of the Study

To compare the learning effects of the classroom concordancingand the conventional teaching method on vocabulary learningin the following areas. 1.1. Students’ vocabulary size 1.2. Students’ ability to transfer lexical knowledge to new contexts 1.3. Students’ retention of vocabulary knowledge

To explore students’ processes in dealing with classroom concordancing.

To explore students’ attitudes towards the application ofclassroom concordancing.

IV. Statements of Hypotheses

Students’ scores on the measure of vocabulary size in the experimental group are significantly different from those in the comparison group.

Hypothesis 1:

Students’ scores on the measure of students’ ability to transfer lexical knowledge to new contexts are significantly different from those in the comparison group.

Hypothesis 2:

Students’ retention rates in the experimental group are significantly different from those in the comparison group.

Hypothesis 3:

Research Methodology

Population Engineering students

Samples Two intact groups of RMUTL Tak Engineering undergraduate students

Each group = 26 - 28 students

To equate the groups: - Students were matched in pairs according to their scores on the pretest (= 26 pairs). - The mean scores of 2 groups were compared with t-test.

Research Design

A quasi-experimental researchA matching-only pretest-posttest comparison group design

(Fraenkel & Wallen, 2000)

Experimental group O1 M X1 O2 O3

Comparison group O1 M X2 O2 O3

O1 = PretestO2 = PosttestO3 = Delayed testM = Matched participants in pairs according to the scores of O1

X1 = Classroom concordancingX2 = Conventional teaching method

Research Process

In Preparatory Phase:

Compiling a corpus

Designing research instruments

Selecting target words

Word Selections

Corpus500,000 words

Academic texts in engineering

Target Words480 words

• words in GSL & AWL• high frequency in corpus

= at least 15 occurrences

12 Lessons 2 versions40 w/lesson

Weekly Wordlists12 lists

40 w/list

4 Review Tasks Every 3 weeks

30 w/task, 120 w in total50 occurrences in a corpus

Testpre-/post-&delayed

101 w in total80 occurrences in a corpus

A frequency-based method

Research Process

Both groups were treated similarly except the teaching method.

The similarities between both groups

Syllabus & lessons

Teacher

Period of time

Review tasks

Pretest / Posttest& Delayed test

In Experimental Phase:

Research Process

The only difference was due to the requirements of the used methods.

Comparison groupWithout classroom concordancing

Experimental groupWith classroom concordancing

Classroom Normal classroom Language laboratory

Material Handouts

without concordancesHandouts

with concordance info

Activities Paper-based only Hands-on & paper-based

Teaching / Learning Methods

In each week, students will learn the same 40 target words.

Comparison groupWithout classroom concordancing

Students are taught through intensive reading i.e., studying some short passages together with reading and vocabulary exercises. The target words are contextualized and presented in the reading texts or exercises.

Experimental groupWith classroom concordancing

Students are trained: To access a corpus for vocabulary learning and reading practice To observe contexts of to-be-studied words in a concordance format. To find linguistic information from the corpus to complete the assigned tasks.

A Sample Lesson

Objective

term, mean, define, refer, such as etc.

To study words used for giving definitions & examples

Target Words

Comparison GroupUsing the conventional method

1. What is a machine?2. What is a simple machine?3. What are examples of a simple machine?4. What is a machine tool?5. What are examples of a machine tool?6. What is CNC machine?7. How many words are defined in the passages? What are they?8. What words or phrases are used to define these words?9. What words or phrases are used to give examples of them?

Activity 1: Read the following passage and answer the questions below.

Machines(Adapted from corpus information)

The terms ‘machine’ means an assembly of parts operating together to perform work. A machine is generally referred to as any mechanical or electrical device that transmits energy to perform tasks. A ‘simple machine’ is defined as a mechanical component such as bearing, gear, lever, screw whereas a ‘machine tool’ is defined as a powered mechanical device such as lathe, mill, drill etc. The term ‘machine tool’ usually refers to tools that use a power source. A computer-numerical-controlled (CNC) machine means the machine tools which are controlled by computers in manufacturing work.


Activity 2: Study the following sentences.• A machine means an assembly of parts operating together to perform work.• A machine is defined as an assembly of parts operating together to perform work.• A machine refers to an assembly of parts operating together to perform work.• A machine is referred to as an assembly of parts operating together to perform work.

Match the words in column A with their definitions in column B. Then make up sentences as in the above sentences.

A B…… Energy a. movement energy…… Kinetic energy b. an ability to do work…… A semiconductor c. the flow of electron…… Electric current d. a material that may act as a conductor

or as an insulator.


Activity 3: Complete the following sentences with the given words.

means defined refers referred

1. ‘Binary’ …………………. twice or two.2. In this context, data is ……...…. as a collection of numbers or characters.3. A device from the industrial revolution was ……………….. to as an engine. 4. Technology ………………….. to the study and science of techniques.5. Random Access Memory (RAM) ……………… that the memory cells can

be accessed in any order.6. Artificial Intelligence is …………………. as intelligence shown by anything

manufactured by humans.7. One AMP is …………… as 625,000,000,000,000,000,000 electrons moving

across a circuit every second.

Experimental GroupUsing Classroom Concordancing

Activity 1: Search the given words from the corpus to find the answers to the following questions.

a). Search the word ‘refer*’.1. Which form of ‘refer’, active or passive , is mostly used?2. Which word or words usually come after ‘refer’?

3. When does ‘as’ come after ‘refer to’? 4. What are typical collocations of ‘refer’?

b). Search the word ‘mean*’. c). Search the word ‘define*’.


Activity 2: Search the corpus to find the definitions of the italic words in the given concordances.

1 In RF circuits, Fo means ……………………………………………..

2 Microfarad means millionths of a Farad.

3 Power is defined as …….................………………... per unit time.

4 Energy is defined as ‘…………………………………………….’.

5 The flow of electrons is referred to as …………………………………………… .

6 A family of CPU designs is often referred to as .................................................................. .


Activity 3: Each set of the given concordances has the same keywords which are missing from the lines. From the contexts of each line in the set, determine which given keyword is missing from each set.

1 One AMP is as 625,000,000,000,000,000,000 (6.25 x 1018) electrons

One horsepower was as the amount of power needed to lift 33,000 pounds

Q is as the frequency divided by the bandwidth, measured fro

The volt was as the potential difference across a conductor when a curre

The potential difference is as the amount of work per change.

2 "Q = 0.5 C" the quantity of electric charge is 0.

An intangible thing a thing you can’t grab it and throw it against the wall.

Technology literally “the study and science of techniques.”

Work moving something, lifting something, warming something,

The term ‘ground potential’ there is no difference in potential (voltage) between a circuit

3 Historically, ‘memory’ to “magnetic core memory” in the 1950s.

Engineering drawing are often to as “blue prints’.

Such circuits are to as ‘conventional’ current as opposed to electron flow.

A family of CPU designs is often to as a CPU architecture

Section lines are commonly to as opposed to electron flow.

means defined referred

There are six main types of research instruments

Research Instruments

Pretest / Posttest & Delayed testStudents’ logs

Teachers’ field notes

Questionnaires

Interview

Classroom materials

Duration: 1 academic semester English class = once a week = 150-minute session deducting 2 weeks for midterm & final exams

total = 16 weeks

Data Collection

Stages

Experimental Phase Post-experimental Phase

Pretest Posttest

Delayed Test

Teacher’s Field notes &

Students’ LogsQuestionnaire 1

Questionnaire 2 &

Interview

Findings

I. Learning effects:

Can the concordance-based method significantly increasestudents’ vocabulary size?

1

Can the concordance-based method significantly increasestudents’ ability to transfer lexical knowledge to new contexts?2

Can the concordance-based method significantly increasestudents’ retention rates of vocabulary knowledge?3

According to the results from conducting MANOVA: Average scores in the experimental group

=> significantly higher in all measures of definitional knowledge, transferable knowledge and retention rates

=> with large effect size, especially in the measures of transferable knowledge.

H1: def. kn. scores (E) def. kn. scores (C)X X

9.73 10.92

34.8830.31

0

10

20

30

40

Average Scores

Groups & Measures

Average Scores on Definition Part in the Pretest and posttest

Experimental Group 9.73 34.88

Comparison Group 10.92 30.31

Pretest Posttest

Posttest: Definition Part

Paired Differences

t df Sig. (2-tailed) Mean Std. Deviation

Pair 1Experimental Group & Comparison Group

4.58 6.338 3.682 25 .001*

t-value > 2.086: p < 0.05 0.72 nearly large

Findings on Definitional Knowledge

Findings on Transferable Knowledge

t-value > 2.086: p < 0.05 1.42 very large

H2: tran. kn. scores (E) tran. kn. scores (C)X X

8.156.65

27.46

20.04

0

5

10

15

20

25

30

Average Scores

Groups & Measures

Average Scores on Cloze Part in the Pretest and posttest

Experimental Group 8.15 27.46

Comparison Group 6.65 20.04

Pretest Posttest

Posttest: Cloze Part

Paired Differences


Pair 1Experimental Group & Comparison Group

7.42 5.217 7.256 25 .000*

Findings on Retention Rates

t-value > 2.086: p < 0.05 of def. 0.44 medium of tran. = 1.53 = very large

H3: retention rate (E) retention rate (C)X X

34.88

100%29.35

84.15%

30.31

100%24.46

80.70%

27.46

100%

12.81

46.65%

20.04

100%

8.38

41.82%

0

5

10

15

20

25

30

35

Average Scores &% of Retention

Measures & Groups

Average Scores & Retention Percentage on the Posttest and the Delayed Test

Posttest 34.88 30.31 27.46 20.04

Delayed Test 29.35 24.46 12.81 8.38

Definition: Experimental

Definition: Comparison

Cloze: Experimental

Cloze: Comparison

Delayed Test Groups

Paired Differences


Pair 1:Definition Part

Experimental Group & Comparison Group

4.88 11.183 2.227 25 .035*

Pair 2:Cloze Part


4.42 2.887 7.812 25 .000*

Findings on Overall learning effects

t-value > 2.086: p < 0.05 of posttest 1.68 very large of delayed test = 0.81 = large

17.88 17.58

62.35

50.3542.00

67.36% 32.8565.24%

0

10

20

30

40

50

60

70

Average Total Scores

& % of Retention

Groups & Measures

Average Total Scores and Retention Percentage on the Pretest, the Posttest and the Delayed Test

Experimental Group 17.88 62.35 42

Comparison Group 17.58 50.35 32.85

Pretest Posttest Delayed Test

Tests & Groups

Paired Differences


Pair 1:Posttest


12.00 7.150 8.558 25 .000*

Pair 2:Delayed Test


9.15 11.170 4.179 25 .000*

Findings

II. Learning Process:

What are the processes used by the students when dealingwith the concordance output?4

Analytic and interpretative skills gradually improved satisfactory at the subsequent stages because of:

Students’ expansion of lexical knowledge Strategy used to cope with a vast amount of corpus data

Although the students could not adequately master some necessary skills, the findings on learning development signified high possibility of such potential.

These evidences were concluded that the concordance-based method could not only enhance effective learning but it could also promote

learning independence.

Findings

III. Learner’s Attitudes:

The students considered that: * The method was much useful for studying language. * Its difficulty level was average. * They regarded such difficulties as challenging and interesting, rather than completely discouraging. * Despite being moderately confident, the students liked dealing with the method a lot. * Appreciating its usefulness, the students mentioned that they would continue to practise utilizing the method for their self-study.

What are students’ attitudes towards the application of the concordance-based method?5

= Much positive towards the method

Advantages of classroom concordancing

A corpus is a fantastic resource for providing authentic

language input.

Corpus information reflects language as it is actually used

in real target situations.

Target words can be selected in a much more confident way.

Classroom concordancing demands ss’ active involvement

in learning and this likely consolidates a learning process.

Ss’ motivation is increased with authenticity of language

closely relevant to their needs as well as with the preference

in using a computer.

Limitations of classroom concordancing

A vast amount of corpus data may be overwhelming for students to cope with.

The concordances typically appearing as fragments may

look confusing and discourage students.

Many of words in contexts are mostly unknown for students

with low language proficiency.

The difficulty of linguistic analysis may also discourage

students.

Suggestions

While dealing with a vast amount of corpus data, students must be trained to screen out irrelevant information and focus their attention to particular language points at a time.

The increase in vocabulary size likely increases the number of known words in contexts for learning new sets of target words.

With low proficient students, the method of linguistic analysis should be simplified and conducted at a level conforming to students’ proficiency level since the purpose of students’ accessing a language corpus is clearly different from that of a linguist.

Suggestions

Training students to deal with classroom concordancing must be gradually conducted to prevent confusion and discouragement.

Variety in classroom activities and exercises are necessarily included in designing lessons and materials.to increase students’ motivation.

Despite taking time, the training is worth in moving students towards independent learning.

Conclusion

Empirical evidence from this classroom-based study can provide significant implication for other similar settings in solving the problem of students’ inadequate lexical knowledge. Although the process of the concordance-based training takes time, the goal of independent learning is really worth being achieved. In most educational settings, English courses have limited time. Therefore, there is an obvious need for teachers to help learners become independent so that they can continue to learn effectively, even after the courses are over. Training the students how to learn is certainly more rewarding and longer-lasting than only teaching them to learn something, but being unable to explore the new ones.

References

Coxhead, A. (1998). An Academic Word List. ELI Occasional Publication No.18, Victoria University of Wellington, New ZealandLaufer, B. 1988. What percentage of text-lexis is essential for comprehension? In Lauren, C., and Nordman, L. (eds.). Special language: From humans thinking to thinking machines, 316-323. Clevedon: Multilingual Matters. Martinek Z. & Siegrist L. (1996). WCONCORD. A downloadable programs available at http://www.kapohl/deNation, P. (2001). Learning Vocabulary in Another Language. Cambridge: Cambridge University Press.

West, M. (1953). A General Service List of English Words. London:

Longman, Green & Co.

Thank you for your attention.

culi’s annual research seminar

Documents