assessing student generic programme outcomes with final ... · elocution a % of students far exceed...
TRANSCRIPT
![Page 1: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/1.jpg)
Assessing Student Generic Programme
Outcomes with Final Year Project
3+3+4 Symposium
December 14, 2009
Prof. Esmond Mok
Department of Land Surveying & Geo-Informatics
![Page 2: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/2.jpg)
The Project
• One of the projects on promoting outcome-based
approaches in student learning
• Faculty-wide project
• Start date: July, 2008
• Expected completion date : end of 2010
![Page 3: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/3.jpg)
Programme Outcomes(professional & generic outcomes)
Generic Competences students are expected to achieve at graduation
(G1, G2, …G?)
Year 3/4 Subject -1
Generic Competences to be developed/reinforced in subject 1
….Year 3/4 Subject -N
Generic Competences to be developed/reinforced in subject N
Year 2 Subject -1
Generic Competences to be developed/reinforced in subject 1
Year 2 Subject -M
Generic Competences to be developed/reinforced in subject M
Year 1 Subject -1
Generic Competences to be developed/reinforced in subject 1
Year 1 Subject -L
Generic Competences to be developed/reinforced in subject L
….
….
General Structure of an Outcome Based Curriculum
![Page 4: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/4.jpg)
How to evaluate whether students haveattained the intended generic learning outcomes at graduation ?
![Page 5: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/5.jpg)
What learning outcomes can students demonstrate in the process and presentation of final year project(s) ?
Final Year Project emphasizing integration of knowledge and
independent thinking
Problem Solving Skills
Professional knowledge
Critical Thinking
Communication Skills
(written & oral)
Creative Thinking
Teamwork
![Page 6: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/6.jpg)
Aims of the project
1. to design and develop a mechanism to assessstudent generic programme outcomes withfinal year projects in the four departments ofFaculty of Construction and Land Use (FCLU):Building & Real Estates (BRE), Building ServicesEngineering (BSE), Civil and Structural Engineering(CSE) and Land Surveying and Geo-informatics(LSGI)
2. to evaluate the effectiveness of the assessmentmechanism
![Page 7: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/7.jpg)
Two Stages of the Project
12 Months Development of a mechanism
to assess student generic programme outcomes
12 Months Operation and evaluation on
the effectiveness of the mechanism
FIRST STAGE:
SECOND STAGE:
(Development)
(Operation &Evaluation)
![Page 8: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/8.jpg)
Select generic competences to be assessed, develop assessment mechanism for final year group/individual projects, develop assessment rubrics
Carry out a trial run of the assessment mechanism
Evaluate and improve the initial design of the assessment mechanism
Develop an operational version of the assessment mechanism/rubrics
FIRST STAGE
![Page 9: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/9.jpg)
Send invitation to students and teachers of BRE, BSE, CSE and LSGI
Operate the assessment mechanism in group/individual projects
Collect assessment results at the end of semester one/two 2009/10
Evaluate the effectiveness of the assessment mechanism, revamp the mechanism for future implementation, and disseminate results to parties
interested in assessing student generic outcomes
Collect feedbacks from teachers and students
SECOND STAGE
![Page 10: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/10.jpg)
What have been done so far ?
![Page 11: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/11.jpg)
Identify generic competences to be assessed
FIRST STAGE
• August 2008:
Discussions with project team, Students Affairs Office (SAO)
and Education Development Center (EDC)
Assessment of five generic outcomes identified:(i) critical thinking (ii) creative thinking (iii) problem solving (iv) communication(v) teamwork #
# Outcome (v) can only be assessed in group projects
![Page 12: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/12.jpg)
Develop assessment mechanism for final year group/individual projects
FIRST STAGE
• September 2008:
Design and development of assessment mechanism
Final Year Project PeriodPre-survey of thefive generic outcomes
Post-survey of thefive genericoutcomes
Students’self-assessmentat start of project
Students’self-assessmentat end of project
Assessments of studentperformance in the five generic outcomes
Pre- and post- survey would enable us to understandstudents’ perception on the generic competences, and thedifference between students’ perception and assessment results
![Page 13: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/13.jpg)
Pre- and Post- surveys:Indirect measurement using self-evaluation formdeveloped by SAO
Assessment of Final Year Projects (FYP):Direct measurement using well designed rubrics
![Page 14: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/14.jpg)
Examples of Definitions Defined in the Self-Evaluation Form
![Page 15: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/15.jpg)
Examples of Definitions Defined in the Self-Evaluation Form
![Page 16: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/16.jpg)
Student Self-Evaluation Form
Teamwork Q1-4
statements addressing various domains of a generic competence
![Page 17: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/17.jpg)
Teamwork Q5-10
![Page 18: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/18.jpg)
Critical Thinking Q1-6
![Page 19: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/19.jpg)
Critical Thinking Q7-12
![Page 20: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/20.jpg)
Develop assessment mechanism with final year group/individual projects
FIRST STAGE
Assessment of generic outcomes - two formats of assessments classified:
• Assessments based on students performance/participation in the process of carrying out the project. (through observations in informal discussions and in project meetings)
• Assessments based on presentations and submissions, e.g. oral presentation (oral communication), project proposal, intermediate and final dissertation submissions (written communication, critical thinking, problem solving)
![Page 21: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/21.jpg)
Develop assessment rubrics
FIRST STAGE
September- October 2008:
Development of Assessment Rubrics with assistance of EDC
![Page 22: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/22.jpg)
![Page 23: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/23.jpg)
![Page 24: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/24.jpg)
![Page 25: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/25.jpg)
![Page 26: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/26.jpg)
![Page 27: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/27.jpg)
![Page 28: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/28.jpg)
![Page 29: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/29.jpg)
![Page 30: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/30.jpg)
![Page 31: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/31.jpg)
Carry out a pilot test of the assessment mechanism
Evaluate and improve the initial design of the assessment mechanism
FIRST STAGE
2008/09:
• BSc in Geomatics (Geo-IT) selected for the pilot test of the assessment mechanism (group project with 29 students)
• Pre- and Post- student surveys carried out• Assessment results collected and student performanceof each generic outcomes compiled with the help of EDC
![Page 32: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/32.jpg)
Overall Results of the pilot test of the direct measure
Subject: LSGI 425
Sample size: 39 students
Benchmark: 70% of students getting Grade C or above
Result: “Creative thinking” fall short of target (only 20.5% of students meet expectations)
[this group project
is emphasized
on problem solving,
Project specification
was provided]
Outcomes assessment summary
0 25 50 75 100
Problem solving
Critical thinking
Oral communication
Written communication
Creative thinking
Teamwork (whole team)
Teamwork (individual)
Le
arn
ing
ou
tco
me
s
% of students
Far exceed expectations (A+ or A)
Exceed expectations (B+ or B)
Meet expectations ( C+ & C )
Threshold (70% students getting C or above)
Threshold: 70% studentsobtained C or above
![Page 33: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/33.jpg)
Oral communication
0 25 50 75 100
Organisation of presentation
Subject knowledge
Graphics
Accuracy
Elocution
Crit
eria
% of students
Far exceed expectations (A+ or A)
Exceed expectations (B+ or B)
Meet expectations ( C+ & C )
Threshold (70% students getting C or above)
An example showing students’ performance on different domains of “oral Communication”
![Page 34: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/34.jpg)
• A project handbook would help participating students andcolleagues to better understand the purpose and operationdetails of the project.
• Different weights (0-1) can be assigned to each criteria ofassessment rubrics to indicate their relative importance in the project. The total weight equals 1.
• Development of an online rubric system will help teachersto quickly carry out assessment and analysis of results.
Major Feedbacks on stage1’s work and pilot test:
![Page 35: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/35.jpg)
0.3
0.2
0.3
0.2
![Page 36: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/36.jpg)
Electronic rubrics under development to facilitate grading andanalysis of assessment results
![Page 37: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/37.jpg)
![Page 38: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/38.jpg)
Send invitation to students and teachers of BRE, BSE, CSE and LSGI
Operate the assessment mechanism in group/individual projects
Collect assessment results at end of semester one/two 2009/10
Evaluate the effectiveness of the assessment mechanism, revamp the mechanism for future implementation, and disseminate results to parties
interested in assessing student generic outcomes
Collect feedbacks from teachers and students
SECOND STAGE
![Page 39: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/39.jpg)
First & Second Semester 2009-2010
Department Programme No. of Students Project Types
LSGI BSc in Geomatics 20 Full-year individual project
CSE BEng in Civil Engineering
17 (FT) / 6 (PT) Full-year individual project
BSE BEng in Building Services Engineering
72 (UGC-funded)/ 48
(self-financed)
Full-year group design project
BRE * BSc in Property Management
130 Full-year individual project from Semester 2 in year 2 to Semester 1
in year 3 (including summer holiday)
• Students of BRE will participate in Semester 2 2009 - 2010 to start a new cycle of their Final Year Project.
![Page 40: Assessing Student Generic Programme Outcomes with Final ... · Elocution a % of students Far exceed expectations (A+ or A) Exceed expectations (B+ or B) Meet expectations ( C+ & C](https://reader033.vdocuments.site/reader033/viewer/2022042123/5e9dd00945eada01aa06795b/html5/thumbnails/40.jpg)
Thank You