Using Methodology Feedback to Improve Courses: A Case StudyPaul C. Corrigan
Introduce ILOs
Assessment (a.k.a. feedback) Group Activity 1: Warm Up Purposes and Types of Assessment Group Activity 2: Effectiveness of Assessment
Methodology Feedback Group Activity 3: Warm-up What is Methodology Feedback? Group Activity 4: Critical Analysis and Application
Case Study: Methodology Feedback
Summary and Conclusion
Outline of Today’s Seminar
22
Intended Learning Outcomes ILO 1
To review and gain further knowledge about assessment/ feedback
ILO 2 To critically examine the concept of ‘methodology
feedback’ ILO 3
To consider how methodology feedback might be usedfor your own teaching
ILO 4 To examine a case study of methodology feedback
Introduction
33
ASSESSMENT
4
In groups of about 3-4, discuss these questions and be ready to share your ideas with the rest of the seminar participants
In your courses:1. What types of assessment/feedback do you use?2. What are your purposes of assessment?
Activity 1 – Warm Up
55
Do you recognize these from your discussion of assessment?Some definitions of assessment terminologyDiagnostic Indicates how current performance differs from expected
performance. Can be used to identify problems that a pupilmay be experiencing.
Formative An assessment that helps pupils learn; results in actions thatare successful in closing the gap between current andexpected performance.
Summative An assessment that is used to certify or record end of courseperformance or predict potential future attainment; the finalproduct of a unit or course; an examination grade.
Evaluative Assessment information that is used to judge the performanceof schools or teachers; league tables.
6
Purposes and Types of Assessment It is not the assessment itself that is diagnostic,
formative, summative or evaluative but the waythat the information collected is used. For example, descriptive or prescriptive – but could
be others
As William and Black (1996, in Weeden, Winterand Broadfoot, 1996) suggest, ‘These terms aretherefore not descriptions of kinds of assessmentbut rather of the use to which information arisingfrom the assessments is put.” 7
Two broad categories of purpose areDescriptive and non-judgmental ORPrescriptive and judgmental
Some Purposes of Assessment & Type
88
Teachers monitor students’ learning For example, a quiz for this purpose
Could be judgmental/prescriptive Could be descriptive and non-judgmental to give
teacher an idea of what needs to be re-taught
Some Purposes of Assessment & Type
99
Students monitor their own learning For example, answering questions at the end of a
chapter for this purpose; Probably descriptive and non-judgmental
Some Purposes of Assessment & Type
101010
Teachers’ effectiveness is monitored For example, a teaching feedback form for this
purpose Could be descriptive to show areas where improvement
could be made Could be judgmental – e.g., contract renewal
Social purposes Institutions and students are assessed and described
for the other benefits for the society For example, university league tables, recruitment of
graduates
Some Purposes of Assessment & Type
1111
Activity 2 In groups of about 3-4, discuss this question
and be ready to share your ideas with the rest of the seminar participants:
How can you tell if that type of assessment is providing the information you need for a purpose?
12
Do you recognize any of your ideas from the discussion in these concepts?Definition of validity, reliability and manageability
Validity of assessment method
To what extent does an assessment measure whatit sets out to measure?
Reliability of assessment
How consistent is the measurement of resultsbetween different teachers, between differenttest situations, etc.?
Manageability of assessment
Can the assessment be conducted without toomuch disruption to normal teaching?
1313
Course/Programme Assessment Purpose
Descriptive: to understand what is taking place, perhaps to make changes without judging a person
Prescriptive: to judge quality and prescribe changes Etc.
Type survey forms oral feedback from tutors hearsay focus groups with students focus groups with other stakeholders Etc. 14
Course/Programme Assessment How can you tell if the types of assessment just
shown are valid, manageable and reliable? Are some more so than others?
15
METHODOLOGY FEEDBACK
16
Activity 3 In groups of about 3-4, discuss these
statements: which of them do most persons inyour group believe to be true? Materials are only as good as the teachers who use
them The attempt to produce ‘teacher proof’ materials is
both futile and undesirable Materials should be ‘neutral’; that is, they should be
free of any bias in approach, method, or technique
17
Methodology Feedback Methodology Feedback assumes that the same
materials could be taught in different ways But effectively or not is another question
In collecting feedback, both materials andteaching methods thus should be taken into account
Two major factors to take into account forcourse/programme quality are materials andmethods
18
Methods Techniques
Materials
Methodology
Methodology: Methods and Materials
1919
Prescribed WHAT Within an OBTL framework the content within teaching
and learning activities (TLA) which are constructivelyaligned with course intended outcomes
Actual WHAT The actual content that students were exposed to
during the course
Effective HOW Effective methods used to teach / students to learn the
content Is linked to students learning strategies
Methodology: The WHAT and HOW
20
X
X
X X
Methodology Feedback Table
21
Actual “What”
Effective “How”
Prescribed “What”
Methodology
Prescribed “What”
Prescribed “What”
Prescribed “What”
X
X
X X
Methodology Feedback Table
22
Actual “What”
Effective “How”
Prescribed “What”
Methodology
What is supposed to be taught isbeing taught and in an effectiveway
What is supposed to be taught isbeing taught but not in aneffective way
What is supposed to be taught isnot being taught but what is beingtaught is taught effectively
What is supposed to be taught isnot being taught but what isbeing taught is not being taughteffectively
Prescribed “What”
Prescribed “What”
Prescribed “What”
1. The prescribed content is being covered2. The prescribed content is being covered, but the
methods are not consistent with students’ learningstrategies
3. The prescribed content is not being taught, butthe methods are consistent with students’ learningstrategies
4. The prescribed content is not being taught, andthe methods are not consistent with students’learning strategies
What can a Methodology Feedback Table can tell us?
23
Remediation1. Change the teaching methods to coordinate with students’
learning strategies (unless you want the students to learn in anew way)
2. Re-examine content for relevancy If relevant, then explain to tutor(s) changes that should be
made in the actual content presented If not relevant, revise the content according to appropriate
academic procedures3. The third & fourth situations Is the content relevant or not relevant? (See situation above) Are the methods preventing the content from being
covered?
What can a Methodology Feedback Table can tell us?
24
Sometimes, the HOW is part of the WHAT that is to say the methods is part of the content
Why is this so? In teacher training or professional development
courses, methods are modeled for the participants That is, methods such as role play, teacher fronted,
student-centered methods are modeled so that thestudents can learn them experientially and laterexplicitly reflect on them
Methodology
25
Literature Support for Methodology Feedback Sax (1997) implies that feedback on both
materials and methods should be obtained forprogramme evaluation
Nunan (1988 ) says a key question in assessmentof a programme is “Are the materials, methods,and activities consistent wit the pre-specifiedobjectives?” which he places this under the heading“Methodology”
From these writers it is inferred, therefore, thatmethodology is comprised of two majorcomponents - materials and methods
26
Other Support for Methodology Feedback Our teaching methods are not ‘neutral’ They reflect conscious and/or unconscious
assumptions about teaching and education Classical humanism Approach
Content is most important
Reconstructionism Approach The teacher is just a curriculum implementator
Progressivism ApproachMethod is seen as very important
27
Edward Anthony (1963) described the relationship among approach, method and technique
28
Approach (Education Philosophy) / Method / Technique
28
Method• Techniques• Techniques• Techniques
Method• Techniques• Techniques• Techniques
Method• Techniques• Techniques• Techniques
Approach (Education Philosophy) / Method / Technique
29
APPROACH
29
Approach (Education Philosophy) / Method / Technique
For example, in a given situation: The Progressive Approach is in use Discussion is a method used in that approach Requesting students to first discuss with one partner in
English before talking to the class is a discussion technique
30
Activity 4 Discuss what are the pros and cons of Methodology
Feedback? Discuss the main implicit or explicit approach in one of
your courses. Are the teachers seen mostly as implementors?Or is method given a lot of importance?Or is content the most important thing?
Within that main approach, what methods andtechniques are used?
How would you gather valid, reliable, andmanageable feedback on both the content andmethods?
31
CASE STUDY: METHODOLOGY FEEDBACK
32
Case Study Outline Background Pilot Programme Preliminary Analysis of Feedback for each Course
33
Background:Hong Kong’s Quality Enhancement Grant Scheme Promoting Teaching Effectiveness in English as the Medium of
Instruction in Hong Kong’s Self-financing, Post-secondary Sector
1
• Research the general and special needs of teachersusing English as the Medium of Instruction in the self-financing, post-secondary education sector in Hong Kongand interpret the results within a critical literature review
2
• Develop printed and web resources for a pilotprogramme of four mini-courses in teaching in themedium of English for teachers in the self-financing post-secondary education sector.
34
34
Background: Hong Kong’s Quality Enhancement Grant Scheme
3• Offer the pilot programme to teachers in the sector
and assess the results to make improvement
4• Offer the revised programme on a wider scale
and assess the results
5• Offer the revised programme on a wider scale
and assess the results
35
35
Pilot Programme:Structure, Registrants and Dates Four Courses
Second Language Learning and the Classroom Communicative Teaching Methods and Techniques Language Awareness Developing Curriculum and Materials in English
20 registered attendees Post-secondary teachers in different subject areas
August 2010
36
Pilot Programme: CILOs Second Language Learning and the Classroom
1. Demonstrate knowledge of recognized approaches in SLA
2. Analyze features of those approaches which are implicit or explicit in their classroom methods and techniques
3. Synthesize a coherent approach to second language learning for their classroom
4. Develop a strategy for applying a coherent approach.
37
Pilot Programme: CILOs Language Awareness
1. Demonstrate awareness of British English, American English, and World Englishes, including ‘Hong Kong English’
2. Apply vocabulary learning skills3. Demonstrate awareness of common writing errors and
their repair at the sentence and discourse levels4. Demonstrate awareness of common pronunciation and
intonation errors and their repair5. Describe and apply sociolinguistic appropriateness in
different communicative contexts.38
Pilot Programme: CILOs Communicative Teaching Methods and Technique
1. Describe a range of communicative teaching methods and techniques
2. Identify which methods and techniques are suitable for their subject.
39
Pilot Programme: CILOs Developing Curriculum and Materials in English
1. Describe how to apply existing communicative methods and techniques to curriculum and materials development in their subjects
2. Describe how to synthesize new communicative methods and techniques to curriculum and materials development in their subjects
3. Develop lesson plans which employ, and justify the use of, such methods and techniques
40
Pilot ProgrammeEmbedded Methods: Second Language Learning and the Classroom
Teacher fronted PowerPoint presentation Preview questions, presentation of materials
Small group discussions Consolidation, elaboration, critical thinking, application
Debate
Small group discussions Consolidation, elaboration, critical thinking, application
Small group discussions and presentations with visual aids Consolidation, reformulation, cooperative learning
4141
Pilot ProgrammeEmbedded Methods: Second Language Learning and the Classroom
Jigsaw reading
Quizzes by students Review materials, draft quiz, administer / take it, review scores,
seek clarification
Student presentations of course units
Written reflection
4242
Pilot ProgrammeEmbedded Methods: Communicative Teaching Methods and Techniques
4343
Teacher fronted PowerPoint presentation Preview questions, presentation of materials
Survey questionnaire Read survey, complete survey, discuss answers in small group to
raise self-awareness
Quizzes by students Review materials, draft quiz, administer / take it, review scores,
seek clarification
Pilot ProgrammeEmbedded Methods: Communicative Teaching Methods and Techniques
4444
Groups discussion Discuss, evaluate and apply
Debate Reading Extrapolating and applying
Short essay Group presentation on course units Short reflective essay
Pilot ProgrammeEmbedded Methods: Language Awareness
4545
Teacher fronted PowerPoint presentation Preview questions, presentation of materials
Video Preview, watching the video, review and application
Debate Group writing Look-and-Describe Experiential method: Reading Determining the meaning of vocabulary
Pilot ProgrammeEmbedded Methods: Language Awareness
4646
Video Teacher summary, student prediction, sequencing of actions,
cloze activity, role play
Imagine-and-speak Read-and-lookup Student presentation of course units Short reflective essay
Pilot ProgrammeEmbedded Methods: Developing Curriculum and Materials in English
4747
Teacher fronted PowerPoint presentation Preview questions, presentation of materials
Survey questionnaire Read survey, complete survey, discuss answers in small group
to raise self-awareness
Group Discussion Discuss, evaluate and apply
Practicum Students develop and deliver lessons in cooperating what
they learnt in the programme
Student developed quiz Short reflective essay
Methodology Feedback Form / Sample
48
Case Study: Preliminary Analysis
of Feedback for each Course
49
Second Language Learning and the Classroom Quantitative Feedback
Materials: Satisfactory to Good Teaching: Satisfactory to Good
Qualitative Feedback
Too much material Too theoretical More practical ideas for EMOI Discussions and Peer sharing were helpful
Remediation
Remedy1. Revise course materials to proceed from concreteclassroom cases to theory
Remedy2. Make certain students understand the Intended LearningOutcomes of this course, and not confuse them with other courses50
Quantitative Feedback Materials: Good Teaching: Good to Very Good
Qualitative Feedback Materials were fine, but too many High praise for the teacher
Remediation Remedy1.
Make materials more concise
51
Communicative Teaching Methods and Techniques
Quantitative Feedback
Materials: Satisfactory to Good Teaching: Good to Very Good
Qualitative Feedback
Good interactive methods but not enough content Liked specific strategies but would like more strategies Material appropriate for some, too easy for others
Remediation
Remedy1. Introduce more material with additional strategies Remedy 2. Possibly stream participants by language proficiency or
scaffold stronger with weaker for peer-teaching/learning
52
Language Awareness
Quantitative Feedback
Materials: Satisfactory to Good Teaching: Good to Very Good
Qualitative Feedback
Materials not used much Materials not enough High Praise for teacher’s method
Remediation
Remedy1. Provide and review with tutor the material earlier and how to use it.
53
Developing Curriculum and Materials in English
Thoroughly explain the rationale of the entireprogramme curriculum to teachers of the coursesduring the interviews
Ensure, when hired, that teachers understand howthe specific courses fit together in the entireprogramme
Thoroughly explain to teachers how to use thematerials for each course, including the importanceof modeling methods as part of the content
54
Case Study: Preliminary Analysis of Feedback for the Programme
Summary & Conclusion Methodology Feedback includes feedback on both
course teaching methods and materials It provides richer information for making decisions
about how to revise courses than traditional feedback
It might be beneficial to try it, at least informally, for a course and see how useful it is
Thanks to: The participants in the pilot programme EDGE for inviting me You for attending Ms. Judy Lip for assisting with the research
Anthony, E. M. (1963). Approach, method and technique.English Language Teaching, 17, 63-67.
Broadfoot P. M. (1996). Education, Assessment andSociety. USA: Open University Press.
Broadfoot, P. M., Weeden, P. & Winter, J. (2002).Assessment: What’s in it for schools?. London:RoutlegdeFalmer.
Nunan, D. (1988). The Learner-Centred Curriculum.Cambridge: Cambridge University Press
Sax, G. (1997). Principles of Educational andPsychological Measurement and Evaluation. USA:Wadsworth Publishing Company.
References
5757
Tables from Slide 6 & 13 cited from Broadfoot, P. M.,Weeden, P. & Winter, J. (2002). Assessment: What’s in it forschools?. London: RoutlegdeFalmer.
Slide 23 adapted from Nunan D. (1988). The Learner-Centred Curriculum. Cambridge: Cambridge UniversityPress.
Statement no. 1&2 from Slide 17 cited from Nunan D.(1988). The Learner-Centred Curriculum. Cambridge:Cambridge University Press.
Citations
58
