qa policy from vision to kpis - eurashe · simon sinek (2009) 13 from vision to kpis development of...

QA POLICY

from VISION to KPIs

Lucien Bollaert

International independent QA expert

Vice-Chair EQAR Register Committee

Member of boards of AEQES, QQI, EQ-Arts

Visiting professor & lecturer & researcher

EURASHE – ARES Seminar on QA

6 & 7 February 2017

Brussels

future QA in HE contents

Intro: no KPI without vision

Development of a vision

From vision to mission

Turning mission into strategic objectives

Realizing strategy via action plans

Measuring realisation of strategy with KPIs

Conclusions: pitfalls & recommendations

E

X

E

R

C

I

S

E

Look critically to the global context

with your external & internal stakeholders

Renew your vision

Use a SWOT looking from outside to inside

Engage all your stakeholders

Renew your mission

Identify your existing organisational (Q)C

Identify your wished organisational QC

Engage all your external & internal stakeholders

Renew your (strategic) policy

Use SMART quantitative& qualitative KPIs & indicators

Communicate top-down & bottom-up

(Re)design your programmes & other processes

(re)design your organisation its governance &structures

Create independent facili- tating QA units

“DO” of PDCA

WHY?

this future of HE & Q(A)?

this vision of HE & Q(A)?

WHAT?

do we stand for in HE & its Q?is our mission in Q of HE?

HOW?Are we going to realize our mission?

is our mission in Q of HE?

WHICH?People (leaders & staff)?

Governance QA structure?

HOW?Which (new) strategic policy?

Which action plans?

HOW TO DO?

REAL PRACTICE

from vision to KPIs contents

QA POLICY

from VISION to KPIs

Intro: no KPIs without vision

The Future of HE in Europe :

“Currently, these policies and tools might not represent

the best answer to the question “are we ready for the

future?” This situation makes a fundamental re-thinking

of how a European common space for higher ecuation

could continue necessary. For this, we could build on

the positive experiences and achievements to date.”

“For that, however, a new vision is needed, not just

technical adjustments.”

“(...) inform decision-making and practical action,

coordinated or individually, in order to address new

and emerging national challenges, first, and then also

European challenges.”

“It is important to acknowledge that a European

common space for higher education can continue to

exist and play a positive role in the future even though

the European integration process might be stalled or even in the same ways reversed.”

from vision to KPIs intro: no KPIs without vision


Standard I.1:

“Institutions should have a policy for QA that is made public and forms part of their strategic management. Internal stakeholders should develop and implement this policy through appropriate structures and processes, while involving external stakeholders.”

In a changing world characterized by …

Globalization of economy & society;

Climate change;

Informatics turning into ‘internet in internet’ automatization

& robots …

… making use of big data;

Technological innovation;

Demographic changes turning the population older;

A widening gap between rich & poor;

The need of transferable competences for future jobs we

don’t know yet;

International tensions causing massive immigration;

Changing (social) communication, values and beliefs.


How does the world look like in 2030?

(Between 3 disasters & 2 utopia)

Automation (computers & robots) & ageing : no

new jobs

Climate change & hunger : new conflicts

Financial (bank) crisis (bis) : local economy vs

protectionism

Sustainability, ecological engineering,

cybersecurity : new high-education jobs

Creative society : new, local, ecological balance

between (part-time) work & life

OECD (2015), Securing livelihoods for all8


Challenges of HE in EHEA DIVERSITY in (historical & national) context, in vision & mission, in strategy,

in teaching & research, in practice & culture vs globalisation of knowledge society;

In majority PUBLIC, but with fixed 3 means of income: state/region + tuition + contracts (in competition) vs prolonged crisis & lump sum;

Academic AUTONOMY vs NATIONAL policy;

National external quality (EQA) system inspired by revised ESG andswinging to institutional level;

STAKEHOLDER MODEL, internal (students & staff), external (society, employers, parents, …) vs world of work;

Teaching & research & social impact vs learning & co-creation;

Democratically elected leadership with “blended” new public management vs CEO directives;

In need of new “transversal” competences vs immediate employment;

With student-centred learning strategies & processes vs massification of HE and exponential growth of knowledge;

Equal opportunities based on talents vs selection for excellence;

Underpinned by a culture of shared values & beliefs vs rankings:

individual development to citizenship,

key role in human, social and economic development via innovation;

knowledge co-creation & sharing;


Underlying principles & values

Leading with vision, inspiration and integrity by leaders acting as a role model for

their values and ethics (responsibility, people management…)

Managing through structured and strategically aligned processes using fact-

based decision making

Integrated system interconnecting all processes and measuring theirperformances;

Trust in continuous improvement;

Succeeding by people who are valued in a culture of empowerment for

balanced achievement of organisational and personal goals (democracy)

future QA in HE quality culture

QA POLICY

from VISION to KPIs

Development of a vision



Renew your vision

WHY?



from vision to KPIs development of a vision

START WITH THE QUESTION “WHY ?”

THEN ASK “HOW ?”

FINALLY ASK “WHAT ?”

Simon Sinek (2009)

13


Why are we in (higher) education?

Why do we want a better society?

Why do we need a better world?

Why do we want a better life?

Why do we exist? What is the added

value we provide to life on the world?

Why do we need quality (assurance)

management?

Why do we want quality? 14

From vision to KPIs development of a vision

New vision, mission & (strategic) policy of HE(I)

• Traditionally :

Education : gone global with new stress on global

competences fit for global knowledge society of the 21st

century

Research : more global than ever through rankings new

continuum from fundamental to applied fit for global

challenges & innovation

Social services/impact : from regional to global


“We prepare the leaders of tomorrow.”

“We nurture lifelong learners.”

“We aim to have a global impact, while serving our local

community.”

Gallup (2015) found that more than 50% of vision or mission statements of HEI share striking similarities, regardless of size, public or private, land-grant status or religious affiliation, or for-profit or not-for-profit.

They may accurately represent the broad views andaspirations of education leaders and their institutions, andthey probably differentiate the institutions from financial services and retail companies…

BUT THEY OFFER LITTLE GUIDANCE TO CURRENT AND FUTURE STUDENTS (and staff).


GALLUP’s RECOMMENDATIONS :

Establish a clear and differentiated purpose by answering

the questions: “Why do we exist?” and “What value do we

provide to the world?”.

Align the brand by telling the outside world what the

institution is and what it will deliver. (see mission)

Support identity with engaged culture primarily including

the student experiences that should support the HEI’s

purpose and brand.

Gallup (2015)

www.gallup.com/businessjournal/184538/hard-differentiate-one-higher-brand.aspx


http://www.gallup.com/businessjournal/184538/hard-differentiate-one-higher-brand.aspx

New vision, mission & (strategic) policy of HE(I)

• Traditionally :

Education : gone global with new stress on global

competences fit for global knowledge society of the 21st

century

Research : more global than ever through rankings new

continuum from fundamental to applied fit for global

challenges & innovation

Social services/impact : from regional to global

• New vision & mission?

HE(I) as a open & global community of co-creation with

all the stakeholders underpinned by a quality culture


Possible classification of HEIs

1. Research driven university;2. Profession-oriented HEI;3. Civic college to become critical citizen;4. Open and Distance Learning;5. “Humboldt” model developing character;6. Liberal arts (and sciences) college;7. Technical university;8. Business/arts school;9. International;10. Excellent world-class university;11. Or any topic you think is essential for the future of HE in

your country/in the world.


Dirk Van Damme (OECD)

Q

U

A

L

I

T

Y

EDUCATIONAL STRATEGIC CHOICE and REALITY

secondary

secondary

QF minimum standard/LO

excellence

HE a

HE b

secondary

HE c

Secondary minimum leaving standards/LO


I

N

T

E

R

N

A

L

O

W

N

E

R

S

H

I

P

Reactive Quality Culture: sees quality as something that is

‘imposed’ from the outside environment and, thus, focuses on

individual aspects of quality and avoiding external threats;

Reproductive Quality Culture: emphasisthe maintenance of the

status quo, because changes lead to internal resistance;

Responsive Quality Culture: primarily evaluates its own practice

in the light of external quality requirements and contributes to

an improvement agenda;

Reproductive Quality Culture: typical of a ‘learning

organisation’ in which quality consciously is embedded in daily

operations;

4 types of quality culture

Harvey & Stensaker (2008)


identification of Quality Culture (existing & desired)

dr. Dries Berings (2011)


After the meeting they realized the new manager

suffered from severe adaptive difficulty



Why do we measure ?

To understand

To monitor

To manage

To improve

a best practice

How do you create strategic objectives?

NEW

STR

ATE

GY

CU

RR

EN

T

REA

LITY

vision misionStrategic

objective

LLL

competen

ces

Local-

oriented

Tknowledge

transfer

KPI

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

Actions

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

Local-

oriented


Tpeople-oriented

va

lue

Tcreative

QA POLICY

from VISION to KPIs

From vision to mission



Renew your vision



Renew your mission

WHY?



WHAT?



from vision to mission

vision mission

• WHY do we exist? WHERE do we want to be?

WHY should we exist in HOW you will get there?

the (desired) future

• the FUTURE purposes PRIMARY purposes &

& values objectives related to your

STAKEHOLDERS’ needs &

desires

• Where do we AIM What do we DO?

to be? What makes us different?

from vision to KPIs from vision to mission

from vision to mission

vision mission

• Answers the WHY? & WHAT? Answers the HOW?

• Deals only with the FUTURE Deals with PRESENT leading to

the FUTURE

Answers WHEN? (10 y) Answers WHEN? (5 y)

• Should be INSPIRING Lists the broad GOALS

• For the leadership, all staff & Prime audience is the

potential clients leadership & stakeholders

• Tells what you stand for in Ties back to stakeholders’

the FUTURE (10 y) needs & desires

• WHERE do we want to go to? WHAT do we do today?


desired future vision = answer to the question ‘why?’

to want

mission mission

future needed achievable future

to need to be able

Why?

What is our added

value?

What do we want

to?

What does our

(political) context

wish?

How will we do it?

What do we need

Which means do

we need?

How will we do it?



Why do we measure ?

To understand

To monitor

To manage

To improve

a best practice


NEW

STR

ATE

GY

CU

RR

EN

T

REA

LITY


objective

LLL

competen

ces

Local-

oriented

Tknowledge

transfer

SCL co-

creation

International

TC lectures

Locally

involved

KPI

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

Actions

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

Local-

oriented


Tpeople-oriented

va

lue

Tcreative

QA POLICY

from VISION to KPIs

Turning mission into strategic objectives



Renew your vision



Renew your mission





WHY?



WHAT?





from vision & mission to (strategic) policy future

WHY

vision WHY?

Mission WHY? & WHAT?

WHOare we?

do we want to be?

WHEREdo we want

to be active?

Which

ACTIVITIES?

WHATdo we want to reach

for our stakeholders?

HOWdo we want to reach

our objectives?

OBJECTIVES &

STRATEGY

HOWdo we want to deal

with our situation?

HOWdo we want to deal

with ourselves?

VALUES, BELIEFS &

CULTURE

Have our

activities

changed?

Do we

need a

new

strategy?

Has our

situation &

culture

changed?

From vision to KPIs turning mission into strategy

SWOT analysis


future

turn W into S

turn T into O


http://argowiki.com/images/2/26/SWOT_analysis_example.png

http://argowiki.com/images/2/26/SWOT_analysis_example.png

from vision to KPIs turning mission into strategy

Why do we measure ?

To understand

To monitor

To manage

To improve

a best practice

from vision to KPIs turn the mission into a strategy

Why do we measure ?

To understand

To monitor

To manage

To improve

From vision to KPIs: an example


NEW

STR

ATE

GY

CU

RR

EN

T

REA

LITY


objective

LLL

competen

ces

Local-

oriented

Tknowledge

transfer

SCL co-

creation

International

TC lecturesBuild more

auditoria

Locally

involved

Recruit more

locals

Move from

local to

international

Move from

TCL to SCL

KPI

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

Actions

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

Local-

oriented

from vision to KPIs turn the mission into a strategy

Tpeople-oriented

va

lue

Tcreative

VERB+ ADJECTIVE +NOUN

QA POLICY

from VISION to KPIs

Realizing strategy via action plans



Renew your vision



Renew your mission








WHY?



WHAT?





Which action plans?

from vision to KPIs realizing strategy via action

plans

intended LO

LO new students study programme achieved LO

mission strategy & policies results

input outputprocesses

transformation = value added

From vision to KPIs realizing strategy via action plans

Nine out of 10

organizations fail

to execute

strategy !

Building KPI Tree

1st level

Total score of HEI

2nd level

The criteria (e.g.) teaching | research

supporting

3rd level

The rating scale

contains KPIs related to each criteria, and its rating scale

Suryadi, Kadarsah (2007), “Key Performance Indicators Measurement Model Based on Analytic Hierarchy Process and Trend-Comparative Dimension in Higher Education Institution”, Bandung Institute of Technology, Chile

The Balaced Scorecard Tree


Building KPI Tree

1st level

Total score of HEI

2nd level


supporting

3rd level

The rating scale




The Balanced Scorecard is the

framework that links your purpose

statement to the execution of your

strategy.


Is the objective precise and well-defined?

Is it clear? Specific S

Can everybody understand it?

How will the individual know when the task

has been completed? Measurable M

How will that be judged?

What evidence is needed to confirm it?

Is the objective achievable?

Is it within their capabilities? Action-related A

Are there sufficient resources?

Is it possible for the individual(s) to perform?

How sensible is the objective in the current Realistic R

Does it fit?

Is there a deadline? Is it feasible to meet?

Are there review dates? Time-related T

Is it appropriate to undertake the work now?

intended LO



influenced by

development phase


(Internal) Quality Assurance (QA)


methodology

tool tool tool tool tool

“QA is a management approach to focus on the quality of the

organisation and is based on participation of all stakeholders in order to

satisfy their expectations and aims as long as possible” (ISO)



Internal Quality Assurance (QA)


methodology





underlying principles

underlying (hidden) values

intended LO



influenced by

development phase

influence on

development

phase




Why do we measure ?

To understand

To monitor

To manage

To improve




NEW

STR

ATE

GY

CU

RR

EN

T

REA

LITY


objective

LLL

competen

ces

Local-

oriented

Tknowledge

transfer

SCL co-

creation

International


auditoria

Locally

involved

Recruit more

locals

Move from

local to

international

Move from

TCL to SCL

KPI

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

Actions

-

-

-

-

-

-

-

-

-

- Build new building

- Renovate old buil-

dings with auditoria

- Go to high schools

- Recruit local profs

-

Local-

oriented

Tpeople-oriented

va

lue

Tcreative


QA POLICY

from VISION to KPIs

Measuring realisation of strategy with

KPIs



Renew your vision



Renew your mission










“DO” of PDCA

WHY?



WHAT?







Which action plans?

HOW TO DO?

REAL PRACTICE

from vision to KPIs measuring strategy with KPIs

intended LO



use standards

& indicators

influenced by

development phase




methodology





influence on

development

phaseunderlying (hidden) values

underlying principles


Standards & indicators

Standard : a (kind of) description that describes

how the organisation and its people should

develop and manage materials, products, services,

processes & systems in order to satisfy the needs &

expectations of its stakeholders

Threshold or excellence standards?

Guideline : helps to explain how processes &

procedures should/can help in order to meet the

standard (good practice)


Indicator : breaks up standards into measurable and

appropriate units in order to measure how much the

organisation has achieved or is compliant with the

standards

Indicators should give information on the

organisation’s performance as far as quality of its

requirements are concerned and help the

organisation to improve its performance

3 functions of indicators : measure, signal &

communicate (internal & external)


Why do we measure ?

To understand

To monitor

To manage

To improve


(sub-)categories of indicators

Quantitative indicators with a number

Qualitative indicators can’t be presented as a number

Input indicators measure the situation at the start of a process or the

amount of resources consumed during the generation of the outcome

Process indicators represent the efficiency or productivity of the addedvalue during the process

Output indicators reflect the outcome of results of the process activities

Directional indicators indicate whether the organization is getting

better

Actionable indicators measure the organization’s control to effect

change

(non-)Financial indicators (don’t) measure the finances of operations

Strategic indicators measure the realisation of strategic goals


Admission

Advancement

Alumni

Athletics

Business connections

Community connections

Course measures

Connections with other

educational institutions

Employee & HR

Enrollment

Facilities

Financial Aid

Graduation Measures

Grants & Research

Library

Other

Peer comparisons

Retention

Satisfaction

Strategic planning

Student engagement

Student success

Technology

Areas Measured

Ballard, J. Paul (2013), Measuring Performance Excellence:

Key Performance Indicators for Institutions Accepted into the Academic Quality Improvement

Program (AQIP), Western Michigan University dissertation


Building KPI Tree

1st level

Total score of HEI

2nd level


supporting

3rd level

The rating scale





Building KPI Tree

1st level

Total score of HEI

2nd level


supporting

3rd level

The rating scale



The Balaced Scorecard TreeBuilding KPI Tree

1st level

Total score of HEI

2nd level

The criteria (e.g. teaching | research

supporting

3rd level

The rating scale




Building KPI Tree

1st level

Total score of HEI

2nd level


supporting

3rd level

The rating scale



The Balaced Scorecard TreeBuilding KPI Tree

1st level

Total score of HEI

2nd level

The criteria (e.g. teaching | research

supporting

3rd level

The rating scale




KPIs are

performance indicators

to measure the realisation

of an organization’s

particular activity or

strategic goal.

From strategy goals to KPIs: an example

prio

ritie

so

bje

ctiv

es

KP

Is

Enhance

quality

academic

provision

Improve

student

satisfaction

Grow

income

across all

areas

Improve

productivity

Develop teaching portfolio with stronger emphasis on science &

technology

Enhance learning & working environment

Build productive partnerships

Improve marketing effectiveness

Implement research, enterprise & endowment

strategies

QA Audits Student surveys RAE outcomes Overheads

% staff cost

Property cost

per student

FTE

Market share

Research

income

Business

income

Proportion of 1st & 200

Retention Student

employment



Why do we measure ?

To understand

To monitor

To manage

To improve




NEW

STR

ATE

GY

CU

RR

EN

T

REA

LITY


objective

LLL

competen

ces

Local-

oriented

Tknowledge

transfer

SCL co-

creation

International


auditoria

Locally

involved

Recruit more

locals

Move from

local to

international

Move from

TCL to SCL

KPI

-

-

-

-

-

-

-

-

-

- # buildings

- # auditoria

-

- Ratio local

vs non-local

student/prof

Actions

-

-

-

-

-

-

-

-

-

- Build new building

- Renovate old buil-

dings with auditoria

- Go to high schools

- Recruit local profs

-

Local-

oriented

Tpeople-oriented

va

lue

Tcreative




NEW

STR

ATE

GY

CU

RR

EN

T

REA

LITY


objectiveKPI

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

Actions

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

va

lue

Strategic monitoring and follow-up form

NEW

STR

ATE

GY

TIP

S F

OR

FU

TUR

E

IMP

RO

VEM

EN

T

Strategic

objective

-

-

-

-

-

-

-

-

-

KPI

-

-

-

Actions

-

-

-

-

-

-

-

-

-

-

-

-

va

lue

By whom?When?

(deadline)

realisa

tionComment

(lesson learnt)

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-

-


QA POLICY

from VISION to KPIs

Conclusions: pitfalls & recommendations

KPIs and metrics

Metrics are nationally imposed quantitave indicators that

are supposed to indicate the degree of quality or of

performance of a HEI

They are mostly linked to policies on subsidy from the

state/government

Sometimes linked with risk-based approach in national

EQA

Do not take into account the differences in mission,

strategy, resources, locations, etc of HEI

“Don’t track traditional metrics. Instead of worrying about typical customer

satisfaction measures (…), organizations should look at the number of value-

adding service ideas put into practice. It’s not that conventional metrics are

unimported, the researchers say, but because they are “lagging indicators”,

they can bog down efforts to achieve rapid, dramatic change.”

Mark Pernice, “Revolutionizing Customer Service”, HBR April 2016, p.27

from vision to KPIs conclusions: pitfalls & recommendations


“IN GOD

WE TRUST,

ALL

OTHERS

BRING

DATA.”

W. Edwards

Deming

7 deadly sins of

measuring and how

to avoid themMichael Hammer (2007)

1. Vanity: only measure to look

good.

2. Provincialism: organizational

boundaries and concerns

dictate the metrics.3. Narcissism: measure from

one’s own point of view rather

than from the customer.

4. Laziness: assuming one knows

what is important to measure

without giving it adequate

thought or effort.

5. Pettiness: measure only a small

component of what matters.

6. Inanity: not thinking about the

consequences on human

behaviour & performance.

7. Frivolity: not being seriousabout measurement in the first

place.



Renew your vision



Renew your mission










“DO” of PDCA

WHY?



WHAT?







Which action plans?

HOW TO DO?

REAL PRACTICE



Improving through measuring (IQA & EQA)

By setting goals

measurable

through KPIs

By measuring

the realisation

using KPIs

By setting new

measurable goals

having learnt from

the previous

measurement

Measuring with wisdom

Knowing what needs to be measured is just the first

step …

finding the right way to do so is the next.

Deciding what needs to be measured is something

of a science;

deciding how to measure remains an art.

Organizations often construct complex mechanisms

for calculating a metric, when, in many cases, a far

simpler one would suffice.


Recommendations (1)

Develop a future-oriented vision with all your stakeholders

Translate in into an inspiring but achievable mission

Turn that mission into a strategy developed by all

stakeholders

Formulate (strategic) objectives in relation to your mission in

SMART terms

Use KPIs and indicators that are related to your own vision,

mission and strategy

Use KPIs and indicators that really tell something about the

degree of quality or of performance of a HEI

Use both quantitative and qualitative indicators


Recommendations (2)

Communicate them, their resulting measurements, their

analyses and lessons with all your stakeholders

Use indicators to monitor and to improve

Some indicators of some rankings (primarily on research)

may be inspiring or useful

(Inter)national metrics can only be used in connection with

your own indicators linked to your mission and strategy

Learn lessons by the results of the realisation and give tips

(out of the box) to change for the better


THANKS again

Q & A

qa policy from vision to kpis - eurashe · simon sinek (2009) 13 from vision to kpis development of...

Documents