OBTL at CityU in the Hong Kong Context

7th December 2006

        Professor John Biggs

Principal Consultant OBTL Project

Outcomes-based Education (OBE)

There are three main forms of “OBE”. All focus on educational outcomes but each is based on a differentphilosophy. It’s easy to confuse them.

1. Outcome-based Education at school level. Originally for disadvantaged children, but later used generally e.g. Target Oriented Curriculum (TOC) for individualising teaching.

2. Outcomes-based Education. Outcomes at institutional level, used for benchmarking, credit-transfer.

3. Outcomes-based Teaching and Learning (OBTL). Defining learning outcomes at course and subject level,to enhance teaching and learning.

Institutional, Progamme and Course Outcomes

OBE OBTL

Outcomes at the institutional level: Graduate attributes, General competencies

for use within and across institutions.

Outcomes at the programme level:

Academic and professional expertise

and values

Outcomes at the course level:where the teaching, learning and

assessment actually occur

What the UGC said “The UGC’s goal in promoting outcome-based

approaches is simple and straightforward—improvement and enhancement in student learning and teaching quality.” (Alice Lam, May 06)

OBTL, in other words.

“We think that the curriculum revision under '3 + 3 + 4' will be a good opportunity to weave 'outcomes' into the new curriculum.”

The UGC is providing extra funding to promote early adoption of OBTL and a supportive Task Force.

Outcomes-based Teaching and Learning (OBTL): Classroom Based

Intended LearningOutcomes:What the

student has to do

Teaching:To facilitate attaining the

ILOs

Assessment:How well hasthe student has attained

the ILOs

Implementing OBTL using Constructive Alignment

ILO:

What the studenthas to learn, e.g.'to analyse X'

Teaching/LearningActivities:

Engaging the Student to perform

verb in the ILO

Assessment Task:

How wellthe student has met the

ILO

Outcomes-based Teaching and Learning through Constructive Alignment

There are four steps in designing such teaching:

1. describe intended learning outcomes in the form of verbs in a context, indicating what is to be attained and the desired quality of attainment.

2. create a learning environment likely to bring about the intended outcomes.

3. use assessment tasks enabling you to judge if and how well students’ performances meet the criteria.

4. transform these judgments into standard grading criteria.

Constructive Alignment is a Framework for

Reflective PracticeConstructive alignment is not a prescriptive “method”: each step provides teachers a focus for reflective practice.

Teachers can then design ILOs, select TLAs and assessment and grading procedures that are the most effective and practicable given the particular context and subject matter of each.

Several teachers have told us that this simply sharpens what they have been doing already.

Absolutely! Good teachers naturally think in terms of what their students have to learn, how they might best learn it, and how they might best confirm that students have indeed learned what was intended.

Constructive Alignment

Intended Learning Outcomes (ILOs) expressed as verbs students have to enact

The very best understanding that could be reasonably expected: verbs such as hypothesise, apply to “far” domains, generate, relate to principle, etc.

Highly satisfactory understanding: verbs such as explain, solve, understand main ideas, analyze, compare, etc.

Quite satisfactory learning, with under- standing at a declarative level: verbs such as elaborate, classify, cover topics a to n,

Understanding at a level that would warrant a Pass: low level verbs, also inadequate but salvageable higher level attempts.

Teaching / Learning Activities

Designed to elicit desired verbs

May be:

Large class activities

Small class activities

Teacher-managed

Peer-managed

Self-managed

as best suits context

Assessment Tasks

Format such that the target verbs areelicited and deployedin context.

Criteria clearly allowjudgement as to thequality of the student'sperformance

Intended Learning Outcomes (ILOs)

. Statements of what students are expected to be able to do as a result of studying a course or programme. . Expressed in the form of learning verbs needed to achieve the outcome. The teaching/learning activities and assessment tasks also address these verbs.

The Verbs in the ILOs

Generic high level verbs include: reflect, hypothesise, solve unseen complex problems, generate new alternatives.

Lower level verbs include: describe, identify, memorize. Often the these lower level ILOs can be subsumed under higher level.

Some ILOs will contain ‘large’ verbs like ‘write’ with focus on a particular task (write a lab report with specifications X, Y, Z …)

The SOLO Taxonomy with sample verbs indicating levels of understanding

Competence

Prestructural Unistructural Multistructural Relational Extended Abstract

one relevant several relevant integrated into generalized to aspect independent aspects a structure new domain

IdentifyNameFollow simple procedure

CombineDescribeEnumeratePerform serial skillsList

AnalyzeApplyArgueCompare/ contrastCriticizeExplain causesRelateJustify

CreateFormulateGenerateHypothesizeReflectTheorize

Incompetence

FailIncompetentMisses point

. . . .

Distinguish the kind of knowledge you want

Declarative knowledge:. Knowing about things. Knowledge we can declare to someone in writing or telling

e.g. ‘Distinguish between OBE and OBTL’

Functioning knowledge:. Knowledge we put to work in solving a

physics problem, analysing a case study, designing a building, making an argument

e.g. ‘Write an ILO for a subject you are currently teaching’

Designing Teaching/Learning Activities to Align with Intended Learning Outcomes

Having designed the ILOs, we now needto activate the verbs or learning activitiesembedded in the ILOs by designingsuitable Teaching/Learning Activities thatwill facilitate students achieving the ILOs.

Typical ILO Possible TLAs

Describe set reading, lecture, field trip

Explain tutorial, written essay

Integrate project, assignment

Applyproject, case study

Solve problem PBL, case studyDesign, create project, creative

writing Hypothesise experiment,

projectReflect reflective

diary

The point is not how you are going to teach but how and what you want your students to learn.

NOTE! Many of these TLAs can be assessments tasks as well. Then you have excellent alignment.

Four common teaching situations and associated teaching and learning activities

Situation Teaching activities Learning activities

LECTURE talk, explain, clarify listen, take notes, accept, query,

discuss with peers, one-minute paper

TUTORIAL set/answer questions pre-read, prepare questions, provide feedback learn from

peers, critique, analyse

PROJECT set brief, ongoing apply, create, self-monitor feedback communicate, teamwork

PBL set problems set learning goals, design, apply,

accessing desired integrate, solve problems content, skills

What teaching /learning activities will best facilitate your ILOs?

Designing Assessment Tasks (ATs)

Steps:

1. Selecting a practicable task that embodies the target ILO verb. (Try the TLA first).

2. Making a judgment on how well the ILO has been met by the students' performance on the ATs –

developing grading criteria.

Common ILOs Possible Assessment Tasks

Describe assignment, essay question examExplain assignment, essay question exam,

oral, letter-to-a-friendIntegrate project, assignmentAnalyse case study, assignmentApply project, case study, experimentSolve problem case study, project, experimentDesign, create project, experimentReflect reflective diary, portfolio,

self-assessmentCommunicate a range of oral, writing or

listening tasks addressing the ILOs, e.g. presentation, debate,

role play, reporting, assignment, precis, paraphasing, answering

questions etc.

Examination is a very commonly used assessment task especially for large classes. We need to consider if

1. examinations involving answering essay type of questions under invigilated conditions is able to assess students' performance in some high level ILOs, e.g. apply, reflect, create etc.

2. there are other alternative assessments tasks which will more appropriately addess those high level ILOs.

Assessments Tasks for Large Classes

Task-format Useful for

Final exam Ensuring work is student’s own, overview of whole course

Multiple choice Recognition, strategy, coverage Ordered outcome Hierarchies of understanding Poster integration, application, creativity Concept maps, Coverage, relationships Flow charts Three minute essay Different levels of understanding, sense of relevance Gobbets Interpret significant detail, explain Short answer Recall units of information, coverage Letter to a friend Integration, application, reflection

Assessment Issues

1. Assess the ILO or the Assessment Task itself?

2. Assess the task quantitatively (marking) or qualitatively (holistically)?

3. How to combine AT performances to obtain a final grade: quantitatively or holistically?

The most important thing is that the assessment task is aligned to the ILO. The methods of assessing the task and of combining results into a final grade grading are important but secondary issues.

Example of Grading Criteria for an ILO

Marginal Pass Satisfactory Good Excellent D C- C C+ B- B B+ A- A A+ Grade 1.00 1.70 2.00 2.30 2.70 3.00 3.30 3.70 4.00 4.30 point/unit

ILO Reflect

Able to use available Able to use available Able to use available As in “Good”. Able information to self- information to self- information to self- to generalize self-

evaluate and identify evaluate and identify evaluate and identify evaluation to beyond limited aspects of own more aspects of own the full range of owm existing context. strengths and weaknesses strengths and weaknesses strengthes and weak- Suggest ways of in a general sense. No in a general sense. Little nesses. Self-evaluation improving perform- evidence of suggestions application of theory in is based on theory. ance to real-life of ways to improve self-evaluation and limited Increasingly able to professional performance. No evidence suggestions of ways to suggest ways to contest. of theory being used in improve performance. improve performance

self-evaluation. in a specific context.

Grading the AT

In other courses it may be more relevant to grade the task, which is in effect an ILO.

For example, in language courses the ILO is that students can write an effective argument in English/Chinese, or in Science, that they can write a lab report.

Assessing quantitatively

For:

. Used to it.

. Seems to be the logical way to assess in certain courses.

. Logistically easy.

Against:

. Defines quality in terms of accumulating small quantities.

. Measurement error also accumulates, thus invalidating fine discriminations. E.g. there is no valid difference between

74 and 75, yet to the student it can make a BIG difference - an A or a B! Or worse, the difference between pass or fail.. Sends undesirable messages to students (backwash).

Assessing qualitatively

For:

• Student’s performance is appropriately assessed against what they are intended to learn – criterion-referenced.

• Backwash is positive.• The final grade tells students what they have achieved and

what they need for a better grade.

Against:

• Requires a different mind set for some teachers.• Initially more work in designing ILOs and suitable

assessment tasks, but once established is not extra work.

Qualitative assessment involves making judgments against criteria (rubrics), not by counting ‘marks’

If ILOs are to reflect workplace or ‘real world’ standards it is more appropriate to assess them qualitatively.

‘Real world’ outcomes are not stated in terms of marks obtained.

An example of a holistic way of deriving final grade

Curriculum and Instruction: A subject in a course for Ed. Psychlsts.

ILOs:

1. Apply the principles of good teaching and assessment to chosen contexts.2. Relate selected aspects of curriculum design and management to the

educational system in Hong Kong.3. Apply the content and experiences in this subject to enhance your effectiveness as an educational psychologist.4. Show examples of your reflective decision-making as an educational psychologist.

Final grade:

A Awarded if you have clearly met all the ILOs, provide evidence of original and creative thinking, perhaps going beyond established practice.B Awarded when all ILOs have been met very well and effectively.C Awarded when the ILOs have been addressed satisfactorily, or where the

evidence is strong in some ILOs, weaker but acceptable in others.F Less than C, work plagiarised, not submitted.

Assessment: Curriculum and InstructionShow evidence that you have learned according to the criteria in

the ILOs.

Keep a reflective journal to record useful insights as a data base.

A paper explaining how you would like to see the Hong Kong educational system implement a major educational reform. You should have ILO (2) in mind.

A report specifically addressing ILOs (3) and (4), a review of those aspects of the course that you think will probably enhance your work as an EP.

Your own rationale of your group presentation. You should have ILO (1) in mind.

A self-evaluation showing how you have addressed each of the ILOs.

Place in a portfolio of about 5,000 words.

Use of OBTL in Hong Kong based on the Constructive Alignment Model

At City U, 240 degree courses are committed and on the way:

Faculty of Business: 28Faculty of Humanities and Soc Sci 86Faculty of Science and Engineering 105School of Creative Media 14School of Law 4Dept. Building Science and Technology 3

At PolyU, well on the wayBaptist U startingHong Kong U, 4 faculties in PBL (a form of constructive alignment) and now moving more generally

Matching OBE and OBTL

It's important that the taxonomy used at the institutional level is not only compatible with that used in the classroom but can actually guide the design of programmes and courses.

Too many fragmented knowledge/ skill/ value/ societal outcomes make aligning TLAs and ATs to those outcomes difficult if not impossible.

A Suggested Taxonomy for Institutional level outcomes

Declarative Outcomes: e.g. content knowledge expert. Functioning Outcomes: e.g. creative/innovative professional, life-long learner, professional expert, good communicator, team-worker.

Value outcomes: e.g. ethical professional.

Graduate outcomes need to be stated at an appropriate level (extended abstract in SOLO terms): e.g. ‘able to solve unseen problems‘.

These graduate outcomes need to be teachable and assessable, usually in the Programme/Course ILOs.

Institutional, Progamme and Course Outcomes OBE OBTL Outcomes at the institutional level:

Declarative, Functioning, Value

Outcomes at the programme level: Above as appropriate to the programme.

Outcomes at the course level: Programme ILOs embedded where

appropriate as Course ILOs, to which

TLAs and ATs aligned.

Reference

Biggs, J.B. Teaching for Quality Learning at University.

Buckingham: Open University Press/McGraw Hill,

2003.

New edition by Biggs and Tang explicitly addressing OBTL is in process, due late 2007.