evaluating programs
DESCRIPTION
Evaluating programs. Problem:. You have been given responsibility for the quality of a three year undergraduate program involving some 30 courses. What evidence do you have about the program and its outcomes that informs decisions about quality and quality improvement?. Feeling powerless????. - PowerPoint PPT PresentationTRANSCRIPT
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Evaluating programsEvaluating programs
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Problem:Problem:
You have been given responsibility for the quality
of a three year undergraduate program involving
some 30 courses.
What evidence do you have about the program
and its outcomes that informs decisions about
quality and quality improvement?
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Feeling powerless????Feeling powerless????
GCEQ – but………………
Commission an evaluation by some-one – but
what data can be collected, what standards
apply, who interprets the data against the
standards and how will evaluation be used in
decision making?
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Answer?????Answer?????
•Depends on what you consider to be relevant
data and how it might be used either ‘by itself’
or in conjunction with other information
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
The issues for you are:The issues for you are:
•Getting data that connects to academic work – eg how
do students regard my course?
•Having tools that allow you to generate data about
programs
•Being able to connect data and tools with
professional judgments to decide about quality
improvement?
•Getting others to understand the system and its
limitations in assisting decisions about quality
improvement
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Aggregated evaluation items
courses
Program characteristic 1
Program characteristic 2
Program characteristic n
Model 1
All students in all courses in the program Exit students
Program evaluation
Program characteristics
Model 2
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Program Evaluation at UniSAProgram Evaluation at UniSA
•Course evaluation aggregated to provide
program and sub program data
•Policy framework identifies Program Directors
to manage evaluation through management of
course evaluation
•Online delivery that uses data from student
information system
• Formal Program Evaluation Reports include
benchmarking based on data
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
System ComponentsSystem Components
•A workable view about quality
• Identified standards
•A means of measurement
• Judgment against standards
• Linkages with quality improvement
• Linkages with change agents
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Program Evaluation at UniSAProgram Evaluation at UniSA
•A workable view about quality
• Resolved through policy directions
• Shaped by national ‘tests’ and performance
tables
• Privileges student data within evaluation
process
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Program Evaluation at UniSAProgram Evaluation at UniSA
• Identified standards
• GCEQ scales related to teaching and generic
program outcomes
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Program Evaluation at UniSAProgram Evaluation at UniSA
•A means of measurement
• Course data that is aggregated rather than
specific program instrument
• Core items make up standardised survey –
items sample scales from the GCEQ and
include overall satisfaction
• Constructed and delivered online through
course home pages
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Program Evaluation at UniSAProgram Evaluation at UniSA
• Judgment against standards
• Intra program benchmarking according to
other high performance courses in relation to
items of the core
• External benchmarking – overall satisfaction
and GCEQ scales
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Program Evaluation at UniSAProgram Evaluation at UniSA
• Linkages with quality improvement
• Required reportage of evaluation of program
• Requirements regarding identification of
changes for quality enhancement
Educating Professionals - Creating and Applying Knowledge - Serving the Community
UNIVERSITY OF SOUTH AUSTRALIA
Program Evaluation at UniSAProgram Evaluation at UniSA
• Linkages with change agents
• Reward systems use evaluation data