assessing writing prepared by olga simonova, maria verbitskaya, elena solovova, inna chmykh based on...
Post on 16-Jan-2016
229 Views
Preview:
TRANSCRIPT
ASSESSING WRITING
Prepared by Olga Simonova, Maria Verbitskaya, Elena Solovova, Inna ChmykhBased on material by Anthony Green
Outline of the lecture
1. Why teach writing? 2. Stages of teaching writing. 3. Writing: process vs. product.4. Why test writing? Types of tests & their
purpose for different test users.5. Writing test specifications.6. Rating scales in assessing writing skills.
WHY TEACH WRITING?
Why teach writing?
1. Children (school-based writing):– writing as a means of learning;– writing as a process;– writing as a product.2. Adults:– socio-cultural experience (travelling, migration,
etc.);– academic experience (furthering one’s education);– professional experience (career development).
STAGES OF TEACHING WRITING
6
Productive language processing
Stages of learning and teaching writing:
• graphics (level of letters/ their combinations);
• spelling (word level);• note-taking
(words/chunks/sentences/plan, etc.);• writing as a communicative skill.
Micro skills:
• produce graphemes and orthographic patterns• produce writing at an efficient speed to suit the purpose• produce an acceptable core of words and use appropriate
word-order patterns• use an appropriate grammar system (tense, agreement,
pluralisation, etc.)• express a particular meaning in different grammar forms.• use cohesive devices in written discourse.
(Brown 2003, p.221)
• use the rhetorical form and conventions of written discourse;
• appropriately accomplish the communicative functions of written texts according to form and purpose;
• convey links and connections between events;• communicate the main idea and supporting ideas;• communicate given information and the new
information, generalization and exemplification;
Macro skills:
• distinguish between literal and implied meanings when writing;
• correctly convey culturally specific references in the context of the written text;
• develop and use a battery of writing strategies such as:– assessing the audience’s interpretation;– using pre-writing devices;– writing the first draft:– using paraphrases and synonyms;– seeking peer and instructor’s feedback;– using feedback for revising and editing.
Macro skills:
Types of tasks
Copying Dictations Gap filling (word/ word
combination/ sentence level)
Scrambled words/ sentences
Sequencing/ ordering Find and correct the
mistakes
Short answer and sentence completion
Multiple choice questions Multiple matching Paraphrasing Grammatical
transformations Vocabulary exercises Cloze Picture-cued tasks
WRITING AS PROCESS AND PRODUCT
Writing: Process vs Process
Process:•drill / controlled / guided ex-s; •brainstorming, planning, all draft materials.
Product:•different types of written texts within different types of language course; •letters / e-mails / questionnaires, transactional letters; memos / minutes / business plan;•essays, reports, articles, presentations, etc.
Genres and text types
1. Academic writing: general subject reports, essays, compositions, term/ course papers, academically focused journals, short-answer test responses; technical reports (lab reports); theses, dissertation, etc.
2. Job-related writing: messages/phone messages; letters/e-mails/ transactional letters; memos; minutes; reports (job evaluation, project reports); schedules, labels, signs; announcements, advertisements; manuals, etc.
3. Personal writing: letters, emails, greeting, cards, invitations; messages, notes, reminders, shopping lists; financial documents (checks, tax forms, loan applications); forms, questionnaires, medical reports, immigration documents; diaries, personal journals; fiction, etc. (Brown 2003, p.221)
WHY ASSESS WRITING?
Construct Purpose
Task types Scoring criteria
Assessing writing: key questions
Why assess writing?
No single answer:• different groups of language learners have different needs, such as:
– international travellers: language for travel, leisure;– migrants: survival skills, access to employment;- students: notes, essays, dissertations;- professionals: emails, reports, letters, memos.
• different users have different purposes when they seek information from tests;
• but most users of language do need to write.
1. SOCIAL NEEDS:• international travellers: language for travel, leisure;• migrants: survival skills, access to employment;• professionals: emails, reports, letters, memos.
2. ACADEMIC NEEDS:• students: mobility, exchange programs, notes, essays,
course work, dissertations, national exams.3. EDUCATIONAL NEEDS:
• teachers: monitoring, control, assessment;• learners: progress check, skills development.
18
Why assess writing?
Bachman and Palmer (2010) Language Assessment in Practice
Performance testing in writing:
samples of written language that simulate real-life behaviour (are authentic in situation and interaction);
complex events involving unpredictable variables;
observed & evaluated by agreed judging process, where facets of measurement interact to produce score.
McNamara (1996)
WRITING TEST SPECIFICATIONS
Construct Purpose
Task types Scoring criteria
Assessing writing: key questions
Writing test specifications:
• topic• text type• length• areas to be assessed• timing• weighting• pass level.
(Coombe, 2011)
Parts of writing test tasks:
• rubric
• prompt / input
• expected response
• post-task evaluation.
Rubric: instructions and criteria
The instructions and the set of criteria for evaluation of the writing task.
It is supposed to:• specify a rhetorical pattern, length, time;• indicate the resources available for students
(dictionaries, spell checker, etc.) and the delivery method (paper and pencil, etc.);
• indicate whether a draft or an outline is required;• include the overall weighting of the writing task in
the exam.(Coombe, 2010)
Writing prompt
‘the stimulus the students must respond to’ Hyland (2003 p.221)3 types of prompt patterns:•base prompt (see the handouts);•framed prompt (see the handouts);•text-based prompt (see the handouts).
Kroll and Reid (1994 p. 233)
26
Expected response and relationship between input and
response:• length• purpose• text type• functions• linguistic features• interactiveness
• reactivity: degree of reciprocity;• scope: amount of input needed to respond;• directness: direct / indirect correlation / real world
knowledge involved apart from the input.(Bachman & Palmer, 1996)
RATING SCALES
Construct Purpose
Task types Scoring criteria
Assessing writing: Key questions
Task analysis
IELTS Academic Writing Task 2: UpbringingWRITING TASK 2You should spend about 40 minutes on this task,Write about the following topic:Children who are brought up in families that do not have large amounts of money are better prepared to deal with the problems of adult life than children brought up by wealthy parents.To what extent do you agree or disagree with the opinion?Give reasons for your answer and include any relevant examples from your own knowledge and experience.Write at least 250 words.
Rating scales
Holistic (see handouts)
+ fast (practicality) – not easy to interpret results/ remember all to arrive at score Analytical (see handouts)
parameters (task fulfillment, content, organization, cohesion, register, accuracy, etc.) and criteria (3-2-1-0 and their description)
+ clear, explicit, detailed diagnostic info, reliable, good for guiding learners
- time consuming
Holistic rating scales
• Positive features:– practicality – fast;– rating holistically may be more naturalistic.
• Disadvantages:– no useful diagnostic information - single score;– not always easy to interpret - raters not required
to use same criteria to arrive at score.
Analytic rating scales
Positive features:•can provide diagnostic information if scores reported separately;•potentially clear, explicit and detailed;•usually more reliable (multiple scores);•useful in training raters to focus on our construct;•potentially useful in guiding learners.Disadvantages:•time-consuming;•may overburden raters.
How to reduce subjectivity ?
• specification of the content of the assessment;• adopting standard procedures;• basing judgments on specific defined criteria;• requiring multiple judgments;• undertaking appropriate training;• moderation of judgments;• checking validity and reliability by analyzing
assessment data.
top related