quality review at irish universities - a tool for empowerment and reform?

Irish Universities Quality Board

Quality Reviewat Irish Universities-

A tool for empowerment and reform?

Dr Padraig WalshChief Executive


“Leadership through Change”DCU, 3 November 2005


• “continuous internal institutional reform”

• “the reform agenda”

• “refocusing of the organisation’s culture”

• “changing the organisation”

“Leadership through Change”


• Quality Assurance?

• QA in Irish Universities– Legislation– Practice

• Role of QA in University Reform– Ireland– Europe

Outline of Presentation



established 2003Website: www.iuqb.ie



• Independent, non-statutory body– 17 member Board

• 7 nominees from IUA (universities)• 10 other nominees

– HEA, NQAI, IBEC, ICTU, USI– European, American university system

• Funded by– Universities, HEA

• Currently being incorporated


???

Quality


• Quality Review (Evaluation)– Improvement– Enhancement– Formative

• Accreditation– Judgement-based

The Quality Assurance Lexicon


• Accreditation is likened to …– doing an examination

• Quality Review is likened to …– going to confession

The Quality Assurance Lexicon


• Quality as ‘fitness for purpose’

• Balancing Improvement and Accountability

– “Navigating between Scylla and Charybdis”

Quality Review


Principles of Quality Assurance

• System should be built on excellence

• System should be built on trust


Universities Act (1997)

Quality Assurancein Irish Universities


Legislation that requires the University to put in place

“quality assurance procedures aimed at improving the quality of education and related services provided by the university”

Universities Act 1997


“The (QA) procedures shall include

the evaluation of .. each department (of the university) .. and each faculty (of the university)

.. and any service provided by the university

by employees of the university and by persons .. who are competent to make

national and international comparisons on the quality of teaching and research and the provision of services at university level”



“The procedures shall include

assessment by those, including students, availing of the teaching, research and other services provided by the university”



System of QA based on primary legislation that defines basis for:

• Internal QA procedures for individual universities

• External QA of individual universities– Periodic audits (organised by IUQB) of the

effectiveness of QA procedures in individual universities

• External QA of university sector– Review (organised by HEA) of QA procedures in

university sector



Principles of Irish QA Legislation

• Primary responsibility for Quality Assurance rests with the University

• Internal QA: with the active contribution of students, universities must evaluate all their activities, including service departments

• External QA: should focus on checking through institutional audit that internal monitoring has been effectively done


What happens in practice?



• Self-assessment– of Academic/Administrative Departments– assessment by students/stakeholders

• Peer Review (from list of nominees)– National experts– International experts

• Publication of Findings

• Implementation of Recommendations– Follow-up

Legislation to Methodology


Self-Assessment / Self-Evaluation



• What are we trying to do?

• How are we trying to do it?

• How do we know it works?

• How do we change to improve?

Self Assessment –4 questions


Academic Department Review



• Easily identified “products”– Teaching Programmes– Research Inputs and Outputs– Service (Committees / Societies )

• Easily identified “stakeholders”

• Familiarity with Peer Review– Research Grants– Manuscipts for Journals– Examination Papers

Academic Department Review


Administrative Department Review



• Not easy to identify “products/processes”

• More complex stakeholder system– Student Affairs (mainly students)– Human Resources (mainly staff)– Computer Services (everybody)

• Less familiar with Peer Review

• Must be reviewed themselves

• Are reviewed by everybody else!!

Administrative DepartmentReview


• In Ireland, we use the same methodology as when reviewing academic departments

• In other countries, Quality reviews of HEIs tend to focus on:– Subject review– Accreditation of programmes

• In such systems, Administrative Departments get evaluated only insofar as they serve or support the academic function

Self-Assessment Methodolgy


• Report should provide a succinct, comprehensive statement of Unit activities

• Report should be a self-criticial analysis of the Unit, including how the Unit sees itself contributing to the Mission of the Organisation and the Strategic Plan

Principle of Self Assessment


Lessons learned since 1995

Self Assessment of Units


• A statement consisting solely of functions, activities and outputs is not a self-assessment Report

• A good self-assessment report implies no surprises for Review Team

• Review visit is a validation of the self-assessment report

Self Assessment Report


• Co-ordinating committee of Unit representative of all staff at ALL grades

• Involvement of the Head

Self Assessment (of unit)recipes for success


• Have a small but representative Quality committee

• Regular communication with all staff not on the Quality Committee

Self Assessment (of unit)recipes for success


Good Peer Review Report!

Good Self Assessment


Quote from a (good) self-assessment report

“ . . . during the course of the two days of meetings, it was notable that no significant areas of criticism were identified which had not already been noted in the (self-assessment) report . . . ”

Good Self AssessmentGood Peer Review


• Benchmark. Aim high!

• Aim high! Nominate first-class reviewers

Good Self AssessmentGood Peer Review


Follow-up / Implementation



Demonstrate leadership!

• Circulate Review Report to all staff

• Agree a plan of action (Quality Improvement) with timescales and deliverables– 1 year– 5 year

Follow upat Unit level


Demonstrate leadership!

• Publish the Review Report (visibly!)

• Publish the Quality Improvement Plan

• Agree to report on progress with the 1 year plan 1 year later– Perhaps invite members of review group back

Follow upat Unit level


• Remember who you are assuring– Students– Staff– External stakeholders

• QR process is not simply a management tool

• Each level is being judged on its response to recommendations– Department– Faculty– Management Team

Follow-upLessons for Leadership


Review and reform

Reviews of Irish Higher Education


OECD Review of Higher Education in Ireland (2004)

– By Politicians/Civil Servants for Politicians/Civil Servants!

– Diagonal slice look through Irish Higher Education

Reviews of Irish HE


EUA Review of Quality Assurance in Irish Universities (2004-05)

– By universities for universities!

– Individual reports on 7 Irish Universities

– Cross-cutting report on university sector

Reviews of Irish HE


European University Association (EUA) were jointly commissioned by IUQB and the Higher Education Authority to:

• Review the quality assurance procedures in the Irish University sector– 1 report

• Review the effectiveness of quality assurance procedures in the individual Irish Universities– 7 reports

EUA Review of QA in Irish Universities 2004-05


“ ... in the context of (each university’s) overall institutional decision making and strategic planning”

EUA Review of QA in Irish Universities

2004-05


Strengths

EUA Review of QA


• “(the) systematic organisation and promotion of QA at the initiative of the universities themselves .. is unparalleled in any other country in Europe, or indeed in the United States and Canada”

• “A common framework for QA has been agreed collectively by all the Irish Universities through IUQB”

EUA Review of QA - strengths


Irish Universities Quality Board (IUQB)

IUQB was established voluntarily by the Irish Universities in 2003 with the aim of increasing the level of inter-university co-operation in developing quality assurance processes


• “The system would appear to strike the right tone and combination of public interest, accountability and university autonomy”

• “The agreed IUQB focus on quality improvement is particularly to be commended”



• “(system) encourages a greater focus on quality and improvement than some systems worldwide, while .. being less intrusive than some other systems in Europe”

• “(this activity) has taken place without the existence of a government agency”



• “not linked to the distribution of core governmental funding”

• “concept of quality is being built into discussions at many levels across each university campus”



QA in a university

Required Practice v Good Practice

Required by law A university

Departmental Review

Every 10 years 6 year cycle

Institutional Audit

Every 15 years Within 7 years

Publication “Outcome of evaluations”

•Full PRG Report•Full QuIP•Synthetic Evaluation•1 year follow-up


Weaknesses

EUA Review of QA


• “There is no system in place in the Irish Universities for the performance appraisal of individual staff members, with the possibility of incentives or sanctions attached to this”

EUA Review of QA - weaknesses


• “Given that widespread and sustainable quality improvement in an organisation is unlikely without the opportunity regularly to review and discuss the quality and performance of individuals, the (review) teams were pleased to learn that such a system has recently been agreed with the labour unions and will soon be introduced across the universities.

• “This potentially powerful tool should be closely articulated with the QA framework”



• “Focusing the review process on academic departments (tends) to reinforce existing academic structures and boundaries”

• “Quality review cycle is not linked to (for example) the strategic planning (cycle)”



• “QA process has until recently not appeared consistently to have enjoyed the necessary support of senior management and governance at all universities”

• “Little substantial evidence of systematic follow-up or monitoring at university level to the quality improvement plans of the various units”



Lessons learned

EUA Review of QA


• A review of the effectiveness of QA procedures must begin with a critical institutional self-evaluation

• Unit level reviews are a convenient way of “covering all the bases”, not ends in themselves

Lessons learned fromEUA Institutional Reviews


• Similar recommendations in many unit review reports

• Responses in unit quality improvement plans feed into institutional strategic planning and institutional change

Lessons learned fromEUA Institutional Reviews


Bologna!

University Reformin Europe


• Bologna Declaration (1999)

• Bologna Process (2000-2010)

University Reformin Europe


• Now signed by 45 European Countries

– Adoption of a system of easily readable and comparable degrees based on two main cycles: UG and PG

– Establishment of a system of credits

– Promotion of mobility for students and staff

– Promotion of European co-operation in quality assurance

Bologna Declaration 1999


• EUA ‘Trends’ Surveys

– Biannual survey of state of the Bologna Process

Bologna Process


EUA “Trends IV” survey

2005

• Prepared by the European University Association (EUA) to evaluate the ways in which many higher education institutions are responding to the implementation of the Bologna Reforms

• Based on 62 site visit to HEIs in 29 European Countries (including Ireland)


EUA “Trends IV” survey

2005

Findings in relation to Quality


EUA Trends IV survey - findings

• “Among a considerable number of HEIs, which had developed a reform agenda, the most limiting factors to their quality enhancement was not the nature of their internal quality processes but the limits of available resources and the scope of their autonomy, i.e. the extent to which they can decide and plan their own institutional future”



• “Institutions with the most systematic approach to quality are also those that benefit from the greatest institutional autonomy”

• “Institutions find that a well developed internal quality culture should be associated with a light external quality control”



• “Generally, institutions had experienced internal quality processes to be more improvement oriented than external quality assurance processes, which were felt to be either more geared towards control and compliance or less attuned to the aims, priorities and conditions under which the institutions or evaluated unit was developing”


Final thoughts

EUA Sectoral Report on Irish Universities


• “While the Irish universities are pioneers in developing and implementing a systematic quality review and quality improvement system by themselves, .. they are not yet leaders in using the outcomes of the system to stimulate strategic change inside the universities”

• “This should be a priority during the next phase of developing the QA system”

quality review at irish universities - a tool for empowerment and reform?

Documents