This article was downloaded by: [Swinburne University of Technology]On: 26 August 2014, At: 08:22Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number:1072954 Registered office: Mortimer House, 37-41 Mortimer Street,London W1T 3JH, UK

Journal of Higher EducationPolicy and ManagementPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/cjhe20

Outside BenchmarkExpectations? Variation innon-completion rates inEnglish higher educationMantz YorkePublished online: 03 Aug 2010.

To cite this article: Mantz Yorke (2001) Outside Benchmark Expectations? Variationin non-completion rates in English higher education, Journal of Higher EducationPolicy and Management, 23:2, 147-158, DOI: 10.1080/13600800120088643

To link to this article: http://dx.doi.org/10.1080/13600800120088643

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of allthe information (the “Content”) contained in the publications on ourplatform. However, Taylor & Francis, our agents, and our licensorsmake no representations or warranties whatsoever as to the accuracy,completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views ofthe authors, and are not the views of or endorsed by Taylor & Francis.The accuracy of the Content should not be relied upon and should beindependently verified with primary sources of information. Taylor andFrancis shall not be liable for any losses, actions, claims, proceedings,demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, inrelation to or arising out of the use of the Content.

This article may be used for research, teaching, and private studypurposes. Any substantial or systematic reproduction, redistribution,reselling, loan, sub-licensing, systematic supply, or distribution in any

form to anyone is expressly forbidden. Terms & Conditions of accessand use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Swin

burn

e U

nive

rsity

of

Tec

hnol

ogy]

at 0

8:22

26

Aug

ust 2

014

Journal of Higher Education Policy and Management, Vol. 23, No. 2, 2001

Outside Benchmark Expectations? Variation in non-completionrates in English higher education

MANTZ YORKE, Liverpool John Moores University, UK

ABSTRACT The performance data for UK higher education institutions which were published by theHigher Education Funding Council for England in the year 2000 are sophisticated in that, in theprovision of benchmark data, they take account of institutional variation. However, the non-completiondata do not take into account a number of signi�cant parameters that, when taken into accountqualitatively, suggest that these might account to a considerable extent for performance deviation above andbelow benchmark level. The analysis presented in this article suggests that an uncritical use of theperformance data (for instance, in league tables of institutions) might not give adequate credit toinstitutions that are succeeding in relatively unpropitious demographic environments.

Student Completion Statistics and Benchmarks

Why do some institutions achieve better completion rates for students than calculatedbenchmarks would suggest, whereas others achieve worse? The publication of perform-ance indicators in which benchmarks tailored to institutions’ pro� les have been calcu-lated (HEFCE, 2000)1 allows some tentative steps towards an answer. The argument,which points to the need to scrutinise performance data with some care, is presented withreference to English higher education institutions, since the inclusion of other countriesin the UK would introduce complications stemming from the diversity in the provisionof higher education.

The issues of retention and completion have risen up various nations’ political agendasas governments seek to maximise the return on their investment in higher education.Assessments of cost and bene� t include not only what some term (often pejoratively)‘wastage’, but also the employability of graduates.2 The political concern is greatestwhere a signi� cant amount of the public money is used to underwrite higher education,and is discernible in a report on completion rates in the seven universities in the Republicof Ireland (Morgan et al., 2001), McInnis and colleagues’ (2000a) survey of the literaturefor the Australian government’s Department of Education, Training and Youth Affairs,Yorke and co-workers’ surveys of ‘non-completers’ in England (Yorke et al., 1997; Yorke,

ISSN 1360-080X print; 1469–9508 online/01/020147-13 Ó 2001 Association for Tertiary Education ManagementDOI: 10.1080/1360080012008864 3

Dow

nloa

ded

by [

Swin

burn

e U

nive

rsity

of

Tec

hnol

ogy]

at 0

8:22

26

Aug

ust 2

014

148 M. Yorke

1999b), and the report on retention produced by the UK House of Commons Educationand Employment Select Committee (Education and Employment Committee, 2001).

The performance indicators published by the funding councils in the UK (HEFCE,1999, 2000) show a considerable variation between institutions in their projectedcompletion rates. Completion rates are generally higher in the ‘old’ universities than inthe new universities and colleges,3 as is shown in Table 1.

Various factors in� uence completion rates, as is acknowledged in the HEFCE tables.Two factors that bear on non-completion are whether or not the student is a ‘mature’student on entry, and the social class of the entering student: summary data are alsoprovided in Table 1, from which it can be seen that the old universities are clearlydifferentiated from the rest of the higher education sector. The old universities tend todraw a greater proportion of their intake direct from school (entry typically at age 18)and from those with higher points scores in the Advanced Level examinations. They arebetter resourced than most other UK institutions by virtue of greater research incomeand—particularly in some cases—fundraising. The new universities and large collegestend to have a greater proportion of mature students (those entering full-time highereducation after a period spent outside the education system, and aged at least 21) andentrants from the lower end of the socioeconomic spectrum. The small, specialistinstitutions tend to be atypical because of their narrowly focused cohorts.4 It is noticeablethat the college sector, in which institutions are generally smaller than the universities,has lower rates of non-completion than the new universities and that the contrast isgreater for the specialist institutions.

The close association between non-completion, percentage of mature entrants to� rst-degree programmes, and the percentage of � rst-degree entrants from socioeconomicclasses IIIm, IV and V (loosely, working-class entrants) is shown in Table 2. A regressionanalysis of the October 2000 institutional-level data for English university institutionsshows that maturity of entry and social class accounted for the bulk of the variance innon-completion (Table 3)5. The outcomes of these analyses closely parallel a similaranalysis conducted on the previous year’s data (Yorke, 2001).

The ratio of mature to young students’ non-continuation following the year of entryis roughly a factor of two, except for creative arts & design and for education, where theratio is lower (see HEFCE, 2000, pp. 132–133). Although there is a gradient of teachingquality assessment scores (highest in old universities, lowest in the colleges) the differencehas narrowed over the years (see Baty, 1999a, 1999b, 2001a, 2001b) and a crudeindex—the proportion of institutional provision taken as ‘excellent’6—had no extraexplanatory power in the regression analysis of the December 1999 performanceindicator data (see Yorke, 2001), and has therefore been disregarded in this article.

The regression analysis of the October 2000 data is confounded by other variables thatwere not available for inclusion but which are correlated with the variables that wereused. One is ‘subject mix’, identi� ed by Johnes and Taylor (1990) as accounting formuch of the variation between the then universities (in modern UK parlance, the ‘old’universities) as regards completion rates. Some old universities have medical, dental andveterinary schools (for which subjects the completion rate is high7), so this will tend toease their overall completion rates upwards.

The biggest confounding variable, though, is entry quali� cation, to which the fundingcouncils can obtain access.8 The entry pro� le of an institution can therefore be producedas a matrix in which various parameters can be related. Twenty-one entry quali� cationswere identi� ed by the Higher Education Funding Council for England (HEFCE), themajority relating to points scores at Advanced Level. These were used in the perform-

Dow

nloa

ded

by [

Swin

burn

e U

nive

rsity

of

Tec

hnol

ogy]

at 0

8:22

26

Aug

ust 2

014

Outside Benchmark Expectations? 149

TA

BL

E1.

Perf

orm

ance

data

for

four

cate

gori

esof

Eng

lish

high

ered

ucat

ion

inst

itutio

n

Stud

ents

from

soci

alcl

ass

Proj

ecte

dno

n-co

mpl

etio

nM

atur

est

uden

tsII

Im,I

V,V

Stan

dard

Stan

dard

Stan

dard

HE

IT

ype

NM

ean

%de

viat

ion

Ran

geM

ean

%de

viat

ion

Ran

geM

ean

%de

viat

ion

Ran

ge

Old

univ

ersit

ya43

10.6

5.9

1–29

13.4

8.6

3–42

18.8

6.0

8–33

New

univ

ersi

tyb

3522

.66.

115

–38

33.4

11.4

17–6

232

.46.

018

–43

Gen

eral

colle

gec

1916

.05.

210

–32

29.5

11.5

12–5

432

.55.

925

–46

Spec

ialis

tco

llege

d18

13.6

4.2

8–20

31.1

7.7

19–4

426

.37.

412

–42

Sour

ce:

HE

FCE

(200

0).

aSo

me

smal

land

larg

ely

mon

otec

hnic

and/

orpo

stgr

adua

tein

stitu

tions

have

been

excl

uded

from

the

old

univ

ersit

yca

tego

ry.

bN

534

for

new

univ

ersit

ypr

ojec

ted

non-

com

plet

ion

stat

istic

sdue

tola

ckof

data

for

De

Mon

tfort

Uni

vers

ity.c

Uni

vers

ityof

Surr

eyR

oeha

mpt

onis

trea

ted

here

asa

gene

ralc

olle

gesin

ceth

epr

ojec

ted

non-

com

plet

ion

data

ante

date

the

mer

ger

ofth

eR

oeha

mpt

onIn

stitu

tew

ithth

eU

nive

rsity

ofSu

rrey

.d

Ver

ysm

alls

peci

alis

tco

llege

sha

vebe

enex

clud

ed.

Dow

nloa

ded

by [

Swin

burn

e U

nive

rsity

of

Tec

hnol

ogy]

at 0

8:22

26

Aug

ust 2

014

150 M. Yorke

TABLE 2. Upper triangle correlation matrix (Pearson r) forperformance data from the largest English university

institutions’ performance data

Variable Mature entry Class

Non-completion 0.89 0.78Mature entry 0.72

Note: No non-completion data are recorded for De MontfortUniversity in HEFCE (2000). Correlations involving non-completion are based on 77 institutions whereas thecorrelation of class with mature entry is based on 78. Thistable excludes a number of small institutions in the olduniversity category which are largely monotechnic and/orpostgraduate in character.

ance indicator tables to allow for entry quali� cation in the construction of ‘adjustedsector benchmarks’ for students’ completion of their programmes.

The adjusted sector benchmarks for completion also incorporate data pertaining tothe institution’s subject mix and the proportions of young and mature entrants. Thebenchmarks do not include any weighting for socioeconomic status (nor for thecatchment of the institution, which could be captured by postcode analysis, and whichis broadly correlated with socioeconomic status). However, data are provided in separatetables by HEFCE (1999; 2000).

The projected completion rates for full-time students starting � rst-degree courses in theacademic year 1997–1998 are given by HEFCE (2000, p. 92ff). Benchmark perfor-mances have been calculated for the percentages of students who neither gain an awardnor transfer to another institution. The percentage rates are, in effect, estimates of thelevel of non-completion. The assumption is made in this article that those who transferto another institution do in fact complete their new programme of study: in practice, asmall proportion of these will not, but the error involved is judged to be negligible in thebroad analysis that follows.

The estimated level of non-completion is set against the calculated benchmark � gurefor the institution, and a � ag is raised wherever the estimate is three standard deviationsdiscrepant from the benchmark and the discrepancy exceeds three percentage points.Twenty-eight English institutions ‘did better’ than their benchmark suggested, and 14‘did worse’.

TABLE 3. Outcome of regression analysis of institution-level performance data from 77 Englishuniversities. The dependent variable is projected institutional percentage non-completion.

Variables B Standard error B Beta Adjusted R2

Percentage mature entry 0.40 0.04 0.67 0.82Percentage working class 0.27 0.07 0.29[Constant] 0.02 1.22 —

Note: This table excludes De Montfort University, for which no non-completion data are given inHEFCE (2000), and a number of small institutions in the old university category which are largelymonotechnic and/or postgraduate in character.

Dow

nloa

ded

by [

Swin

burn

e U

nive

rsity

of

Tec

hnol

ogy]

at 0

8:22

26

Aug

ust 2

014

Outside Benchmark Expectations? 151

TA

BL

E4.

Inst

itutio

nsab

ove

and

belo

wth

eir

benc

hmar

ksfo

rpro

ject

edno

n-co

mpl

etio

n,sh

owin

gth

eir

perf

orm

ance

rela

tive

tobe

nchm

arks

onfo

urfu

ther

dem

ogra

phic

vari

able

s YO

UN

GFI

RST

DE

GR

EE

MA

TU

RE

EN

TR

AN

TS

EN

TR

AN

TS

NE

ntra

nts

%yo

ung

Prop

’nfr

omSo

cioe

con.

Low

part

icip

nopr

ior

HE

1998

–9en

tran

tsst

ate

scho

olst

atus

neig

hbou

rhoo

d&

low

part

.nei

gh’d

Bette

rth

anpr

ojecte

dno

n-co

mpl

etion

Uni

vof

Ast

on14

0689

abov

eB

ath

Spa

Uni

vC

oll

813

59ab

ove

Buc

king

ham

shir

eC

hilte

rns

UC

1583

67U

niv

ofC

ambr

idge

2808

97be

low

belo

wC

ante

rbur

yC

hris

tC

hurc

hU

C11

0969

belo

wC

hest

erC

olle

ge95

485

abov

eU

CC

hich

este

r81

367

Uni

vof

Eas

tA

nglia

1682

81be

low

Uni

vof

Exe

ter

1988

89be

low

belo

wbe

low

belo

wK

ings

ton

Uni

v31

9867

belo

wbe

low

Uni

vof

Lei

cest

er18

8294

abov

eU

niv

ofL

inco

lnsh

ire

and

Hum

bers

ide

2162

70ab

ove

abov

eL

ough

boro

ugh

Uni

v25

0893

Uni

vof

New

cast

le28

2695

belo

wbe

low

New

man

Col

lege

380

70ab

ove

Nor

thR

idin

gC

olle

ge41

768

Nor

wic

hSc

hool

ofA

rt&

Des

ign

190

64U

niv

ofPl

ymou

th36

2570

belo

wbe

low

UC

Rip

onan

dY

ork

StJo

hn99

081

abov

eC

olle

geof

StM

ark

and

StJo

hn63

565

StM

artin

’sC

olle

ge97

478

abov

eU

niv

ofSh

ef�e

ld30

7991

abov

eSh

ef�e

ldH

alla

mU

niv

5058

75ab

ove

abov

eSt

affo

rdsh

ire

Uni

v34

2978

abov

eab

ove

abov

eU

niv

Col

lege

Lon

don

2600

86be

low

belo

wbe

low

Uni

vof

Wes

tmin

ster

3194

59ab

ove

belo

wbe

low

Dow

nloa

ded

by [

Swin

burn

e U

nive

rsity

of

Tec

hnol

ogy]

at 0

8:22

26

Aug

ust 2

014

152 M. Yorke

TA

BL

E4.

Con

tinue

d

YO

UN

GFI

RST

DE

GR

EE

MA

TU

RE

EN

TR

AN

TS

EN

TR

AN

TS

NE

ntra

nts

%yo

ung

Prop

’nfr

omSo

cioe

con.

Low

part

icip

nopr

ior

HE

1998

–9en

tran

tsst

ate

scho

olst

atus

neig

hbou

rhoo

d&

low

part

.nei

gh’d

UC

Wor

cest

er76

961

abov

eU

niv

ofY

ork

1464

92

Wor

seth

anpr

ojecte

dno

n-co

mpl

etion

Ang

liaPo

lyte

chni

cU

niv

2260

59ab

ove

belo

wbe

low

Bol

ton

Inst

itute

1215

49ab

ove

abov

eab

ove

abov

eU

niv

ofC

entr

alE

ngla

nd29

4670

abov

eab

ove

Uni

vof

Cen

tral

Lan

cash

ire

3447

61ab

ove

abov

eab

ove

abov

eU

niv

ofD

erby

3039

68ab

ove

Uni

vof

Eas

tL

ondo

n26

2244

abov

eab

ove

Uni

vof

Gre

enw

ich

2396

59U

niv

ofH

udde

rs�e

ld28

7672

abov

eab

ove

abov

eab

ove

Lon

don

Gui

ldha

llU

niv

2145

58be

low

belo

wU

niv

ofL

ondo

n21

0365

abov

eab

ove

UM

IST

1294

92be

low

SOA

S40

458

belo

wbe

low

belo

wU

niv

ofSu

nder

land

2762

69ab

ove

abov

eab

ove

abov

eU

niv

ofW

olve

rham

pton

3488

63ab

ove

abov

eab

ove

abov

e

Not

e:N

ono

n-co

mpl

etio

nbe

nchm

arks

calc

ulat

edfo

rth

efo

llow

ing

inst

itutio

ns(h

ence

they

coul

dno

tbe

cons

ider

edfo

rin

clus

ion

inth

isT

able

):

Cra

n�el

dU

niv

Roy

alC

olle

geof

Nur

sing

Inst

itute

StG

eorg

e’s

Hos

pM

edic

alSc

hool

Cum

bria

Col

lege

ofA

rtan

dD

esig

nR

oyal

Aca

dem

yof

Mus

icSo

utha

mpt

onIn

stitu

teD

eM

ontfo

rtU

niv

Roy

alC

olle

geof

Mus

icSo

uth

Ban

kU

niv

Uni

vers

ityof

Nor

thL

ondo

nR

oyal

Nor

ther

nC

olle

geof

Mus

icT

ham

esV

alle

yU

niv

Uni

vof

Oxf

ord

Roy

alV

eter

inar

yC

olle

geT

rini

tyC

olle

geof

Mus

ic

Dow

nloa

ded

by [

Swin

burn

e U

nive

rsity

of

Tec

hnol

ogy]

at 0

8:22

26

Aug

ust 2

014

Outside Benchmark Expectations? 153

Why the Discrepancies?

The adjusted sector benchmarks take into account subject mix, entry quali� cations andage on entry. They do not take into account other demographic variables such as theproportion of entrants from state schools, socioeconomic status and geographical location(for which separate data are provided) and for ethnicity (for which data are not provided,although ethnicity may to some restricted extent be correlated with geographical locationdue to the tendency of ethnic groups to cluster).

The reasons for the discrepancies can be approached by taking those institutions thatdeviated (in either direction) from the benchmark estimate of non-completion and seeingif any of these other variables might begin to account for the observed differences. Thetwo extremes of the English institutional population are thus being used to point towardshypotheses for possible future testing. The data in the various tables are not suf� cientlyaligned to permit a secure quantitative analysis, but they do allow a qualitative analysisto be undertaken. Table 4 brings together the deviating institutions and the availabledemographic data.

The four demographic variables in Table 4 are all taken from HEFCE (2000) andcomprise the following:

· the proportion of young � rst-degree students from state schools (as opposed to private,fee-paying schools);· the proportion of young � rst-degree students from socioeconomic groups IIIm, IV andV (as noted earlier—loosely, the working class);· the proportion of young � rst-degree students from neighbourhoods with traditionallylow participation rates;· the proportion of mature entrants to � rst-degree programmes who have no knownprevious higher education experience and who come from neighbourhoods with tradi-tionally low participation rates.

Taken together, these variables are, broadly, indicators of students’ relative lack both of� nancial strength and of a tradition of engagement in higher education. Students withthese characteristics are those whom the UK government is concerned to bring intohigher education through access initiatives of varying kinds.

The correlation matrix in Table 2 shows that lower social class is associated withhigher levels of non-completion. Yorke (1999b, p. 148 and passim) found that workingclass ‘non-completers’ cited, more than other non-completers, � nancial problems ashaving been an important in� uence on their leaving. There is no empirical evidenceavailable that links the lack of a familial or peer tradition of entry to higher educationwith non-completion. However, it can be hypothesised that the lack of such a traditionwould mean that informal support would be less likely to be available from relatives andfriends, with the further consequence of a higher likelihood of premature departure.

It is implicit in Table 4 that, if an institution’s entry pro� le is above the benchmarkon the identi� ed demographic variables, it is attracting students with a correspondinglylower propensity to complete their programmes of study. It would therefore be nosurprise to � nd the institution performing worse in terms of student non-completion thanthe sector-adjusted benchmark would suggest. Five institutions (Bolton Institute, Univer-sity of Central Lancashire, University of Hudders� eld, University of Sunderland andUniversity of Wolverhampton) score above benchmark level on all four variables, andthree (University of Central England, University of East London and University ofLuton) score above benchmark on two.9 Thus the demographic factors offer a tentative

Dow

nloa

ded

by [

Swin

burn

e U

nive

rsity

of

Tec

hnol

ogy]

at 0

8:22

26

Aug

ust 2

014

154 M. Yorke

contribution to an explanation of why these institutions have non-completion levels thatare higher than their benchmark levels: there may, of course, be other causal factors.

Where the institution scores lower than the benchmark on the demographic variables,it suggests that the reason for the discrepancy lies in variables that appear neither in thecalculation of the sector-adjusted benchmark nor in the demographic data.

Two institutions from the old university grouping exhibit levels of non-completion thatare lower than their benchmarks and that do not appear to be in� uenced by thedemographic variables for which HEFCE was able to publish data. Both the School ofOriental and African Studies (SOAS) and the University of Manchester Institute ofScience and Technology (UMIST) have particularly high percentages of overseasstudents: HESA (2000) gives 44% and 34%, respectively. A cluster of variables could becontributing to the relatively high levels of non-completion: among these might be foundentry quali� cations, � uency in English, and the quality of the student experience (bothacademic and social). However, the intakes in a number of other old universityinstitutions include at least 30% of overseas students (the London School of Economicsand Political Science having the highest proportion, 61%, according to HESA [2000]),and yet these institutions do not perform below benchmark.10

Where the institution achieves a lower level of non-completion than the benchmarkwould suggest, low scores on the demographic variables are likely to contribute to theexplanation. A number of institutions fall into this category. The University of Exeterscores low on all four demographic variables, University College London on three, andthe University of Cambridge, Kingston University, University of Newcastle and Ply-mouth University on two. The University of Westminster is in an ambiguous position,scoring below benchmark on two demographic variables and above on one.

In this group that is performing ‘better than benchmark’ for non-completion can befound a number of smaller colleges—Bath Spa University College, Canterbury ChristChurch University College, Chester College, University College Chichester, NewmanCollege, North Riding College, Norwich School of Art and Design, the College of Riponand York St John, the College of St Mark and St John, St Martin’s College andUniversity College Worcester. None has particularly dramatic demographic data thatwould account for their presence in the group. Two factors may be in� uential. First, therelatively small size of these colleges is likely to give students a greater sense of belongingthan might be developed in a larger institution. Second, some of these colleges werefounded by churches and because of this have a particular ethos that also assists thedevelopment of a sense of belonging. Two principals of smaller colleges (one secular, onechurch based), in giving evidence to the House of Commons Education and EmploymentSelect Committee, referred speci� cally to the value that a smaller institution can bringto the student experience.11

There are three institutions, all new universities, that have non-completion rates thatare better than benchmark but which also have higher than benchmark scores on twoor more of the four demographic variables. For this reason, these institutions can be saidto be doing particularly well in retaining their students. The institutions are StaffordshireUniversity (three demographic variables), University of Lincolnshire and Humberside(two), and Shef� eld Hallam University (two). All three are around the new universitymedian for the amount of funding allocated per student, so the key is not the level offunding per se but rather how the funding is being used by the institution.

Shef� eld Hallam University and Staffordshire University have both made particularefforts to widen participation and, crucially, to work through the implications in termsof the student experience. At the former, there has been a particular focus on retention

Dow

nloa

ded

by [

Swin

burn

e U

nive

rsity

of

Tec

hnol

ogy]

at 0

8:22

26

Aug

ust 2

014

Outside Benchmark Expectations? 155

(which is used as one of the key indicators of schools’ performance), and on providinga supportive Student Services Centre. At the latter, a set of initiatives have beenimplemented in respect of student support, and of learning and teaching, as a sustainedcommitment to the ‘widening access’ agenda. The critical aspect of these institutions’work is probably that both have been prepared to make long-term investment in thesedevelopments, rather than be dominated by the short-termism that can sometimes be aconcomitant of an external environment that is labile with respect to both policy andfunding. Sustaining, and from time to time refreshing, a particular kind of educationalcommitment can lead to the level of recognition and reputation gained by AlvernoCollege in the United States, whose approach to student learning (based on theprogressive development of eight generic learning outcomes) has been implemented overmore than a quarter of a century (see Mentkowski et al., 2000).

Looking Beneath the Surface

The performance indicators for higher education that were produced by HEFCE inOctober 2000 are the most sophisticated that have been produced to date in the UK.Yet there is a need to interrogate them closely if the limits to what they say are to beproperly appreciated. The analysis in this article shows that some important variableshave not been taken into account in the production of benchmarks (to be fair, it mayhave been impossible for HEFCE to have done this with statistical soundness). Theseparately published demographic variables could be accounting for some of theinstitutional non-completion performances if they are qualitatively factored in, but—asthe cases of SOAS and UMIST, for example, showed—variables not contained any-where in the HEFCE indicators could also have explanatory power.

There is, here, a message for the policymaker in the UK and further a� eld. There isa danger of using performance data too simplistically,12 even when the data themselvesare sophisticated and subtle. The analysis contained in this article illustrates the point.Policymaking is usually a complex process of optimising competing objectives. In theUK, non-completion (note, though, that the concept becomes increasingly suspect aspolitical moves are made to encourage lifelong learning) is in tension with the desire toraise the level of engagement in higher education from under-represented groups, sincethese groups have the greatest propensity not to complete nor to re-engage in highereducation. Ideally the government would want non-completion to be low (crudely, toreduce waste in the spending of public � nances) at the same time that it would wantaccess to be high (for various economic reasons, but at some risk that the students mightnot complete and therefore contribute to ‘waste’ in public spending). The performanceof some institutions shows that they may be nearer an optimal resolution of this tensionthan others: should not they be praised and—dare it be suggested—rewarded for theirachievements?

Towards Hypotheses Relating to Institutional Success in Retaining Students

Those institutions that performed particularly strongly with reference to their bench-marks seem to have done so for one or more of the following reasons:

· demographic variables running in their favour;· a collegiate ethos that encourages a sense of belonging in students;· a determination to make the student experience as rewarding as possible.

Dow

nloa

ded

by [

Swin

burn

e U

nive

rsity

of

Tec

hnol

ogy]

at 0

8:22

26

Aug

ust 2

014

156 M. Yorke

In practice, the second and third reasons overlap, and the difference between them maybe more a matter of the perspective that one brings to the issue of non-completionthan anything else. A smaller institution prompts thoughts about collegiality, whereas alarger institution prompts thoughts about structures and processes—but these are farfrom being mutually exclusive conceptualisations. These two reasons do, however,connect with Tinto’s (1993) theorising. Tinto, in brief, argues that academic and socialintegration are powerful factors in determining whether a student persists or withdraws.Institutions strong in both (and other things being equal) would, in Tinto’s view, be likelyto exhibit the highest completion rates. If a student’s academic integration is weak, itwould seem logical to conclude that the academic commitment of that student wouldsuffer (if it had not already done so) and that he or she would be more likely to withdrawor fail.

For some students institutional social integration beyond the boundaries of theprogramme can be vestigial if they are having to work signi� cant numbers of hours inorder to fund themselves through higher education,13 or if they are older students withdependants. Such students may have to make the most of the socialising that is possiblewithin the programme (or, as is sometimes the case, within those parts of the programmethat they actually attend). This points attention towards the nature of the studentlearning experience—lecturing and resource-based learning may provide academicsustenance, but do not by themselves provide the social climate that might help to givethe student that extra edge of determination to persist in the face of adversity.

Hypotheses about institutional success in retaining their students, therefore, need tore� ect not only demographic variables and matters such as the subject mix within theinstitution, but also how well the institution is able to create an engaging and supportivelearning experience for its students. Following Tinto (1993), and extending his theorisinga little, the hypotheses need to be couched in terms of different kinds of student—forexample, the students who are able fully to enjoy the social opportunities availablethrough the institution being compared with those who, for various reasons, have nosuch luxury.

Performance Indicators in the World beyond Academe

The successful institution, as far as non-completion is concerned, has increasingly to beimaginative in its approach to the student experience. Yet there is a lack of reliableevidence to indicate where imaginative and engaging experiences are to be found. Thescores produced by the Quality Assurance Agency’s (QAA) reviews of subject provisionrefer to institutionally determined aims and objectives, and in any case have lost anydiscriminatory power that they might once have had as the upward trends of scores fordiffering types of institution converge on the ceiling.

League tables of institutions that are produced in newspapers and magazines need tobe taken with a full cellar of salt. Their technical � aws have been well documented (see,for example, McGuire, 1995; Morrison et al., 1995; Yorke, 1997; 1998; Bowden, 2000),but above all these lies the fact that they cannot give intending students any meaningfulappreciation of the nature of the programme that they may be thinking about joining.Sophisticated as they are, the HEFCE performance indicators with their benchmarkingof non-completion can only provide hints as to where intending students might � nd aprogramme with a rewarding learning experience. In the end, there is no substitute fordirect engagement with the institution(s) that they are considering.14A league tableconstructed on the basis of the variables discussed in this article would look very different

Dow

nloa

ded

by [

Swin

burn

e U

nive

rsity

of

Tec

hnol

ogy]

at 0

8:22

26

Aug

ust 2

014

Outside Benchmark Expectations? 157

from those that are published in the press, which are dominated by research-relatedparameters. If, for example, English higher education were segmented in a manner suchas that of Table 1 (as is done in the tables of institutional rankings produced annuallyby US News and World Report), some relatively unsung institutions would be vying for the‘best in class’ rosettes for student completion.

Correspondance: Mantz Yorke, Centre for Higher Education Development, LiverpoolJohn Moores University, I. M. Marsh Campus, Barkhill Road, Liverpool L17 6BD. Tel:1 44 151 231 5281; Fax: 1 44 151 231 5346; E-mail: m.yorke@livjm.ac.uk

NOTES

1. These are the second set of indicators published by the Higher Education Funding Council for England.The � rst set was published in 1999.

2. Employability is now being seen in the UK, in performance indicator terms, as whether graduates geta–any–job, but is preferably seen from an educational perspective in terms of the graduates’ ability to do‘graduate jobs’.

3. The college sector is heterogeneous and includes some large institutions little different from newuniversities and some small, specialist institutions dealing with subjects such as art, music, and agriculture.On the whole, completion rates for specialist institutions tend to be higher than for the larger, generalinstitutions.

4. ’Specialist’ is used here to indicate that the institution’s portfolio of programmes is strongly (though inmany cases not exclusively) focused on a single subject area, such as art & design or teacher education.

5. The colleges were not included in this analysis because of their comparative heterogeneity. If they hadbeen included, the outcomes would have been broadly similar (though of slightly lower intensity).

6. Such an index is dubious anyway. For example, the assessment methodology is referenced against theinstitution’s own aims, it (and the assessment categories) have varied across time, and issue might be takenwith the treatment of the six components as being of equal weight in the method run in England betweenabout 1996 and 2001.

7. See Tables B13 and B14 in HEFCE (2000), which refer to non-continuation the year after entry foryoung and mature entrants, respectively. The non-continuation rate for medicine, dentistry and veterinary science is a mere 2% in each case.

8. Data for speci� c institutions are treated as con� dential.9. In this and subsequent analyses a single instance of above or below benchmark performance is considered

to be insuf� ciently strong for comment.10. The University of Essex, Imperial College, and Wye College are the other institutions with 30% or more

of their intakes from overseas, according to HESA (2000).11. See the minutes of evidence given to the committee by Dorma Urwin and Michael Wright (Education

and Employment Committee, 2001, Question 47ff).12. The reaction of the press to the publication of the December 1999 data was decidedly simplistic (and,

in some instances, clearly biased in political terms)–see Yorke (2001) for an analysis.13. Callender and Kemp’s (2000) study suggested that students who worked during term time were in

employment for 11 hours per week on average. McInnis et al. (2000b ) noted an increase, over a � ve-yearperiod, in the number of hours that Australian students devoted to paid work while enrolled on aprogramme of study, with the mean number of hours in 1999 appearing to be slightly higher than thatfound by Callender and Kemp.

14. Yorke (1999b) drew on his empirical studies to produce guidance for intending students which covers notonly the learning experience but also other matters affecting non-completion, such as the nature of thecommunity in which the student might choose to live.

REFERENCES

BATY, P. (1999a ) A quality game where cheating is allowed? Times Higher Educational Supplement, 12 March,pp. 4–5.

BATY, P. (1999b) Encounters of an unfair kind, Times Higher Education Supplement, 19 March, pp. 6–7.BATY, P. (2001a) TQA devalued by grade rises, Times Higher Education Supplement, 2 March, p. 5.BATY, P. (2001b) Worthy project or just a game? Times Higher Education Supplement, 30 March, p. 7.

Dow

nloa

ded

by [

Swin

burn

e U

nive

rsity

of

Tec

hnol

ogy]

at 0

8:22

26

Aug

ust 2

014

158 M. Yorke

BOWDEN, R. (2000) Fantasy higher education: university and college league tables, Quality in Higher Education,6(1), pp. 41–60.

CALLENDER, C. & KEMP, M. (2000) Changing Student Finances: income, expenditure and the take-up of student loans amongfull-time and part-time higher education students in 1998–99. London: Department for Education and Employment.

EDUCATION AND EMPLOYMENT COMMITTEE (2001) Higher Education: student retention, 6th report. London:Stationery Of� ce.

HEFCE (1999) Performance Indicators in Higher Education, 1996–97, 1997–98. Bristol: Higher Education FundingCouncil for England.

HEFCE (2000) Performance Indicators in Higher Education, 1997–98, 1998–99. Bristol: Higher Education FundingCouncil for England.

HESA (2000) Insight 2000: a statistical guide to undergraduate study. Cheltenham: Higher Education StatisticsAgency.

JOHNES, J. & TAYLOR, J. (1990) Performance Indicators in Higher Education. Buckingham: SRHE and OpenUniversity Press.

MCGUIRE, M. D. (1995) Validity issues for reputational studies, in: R. D. WALLERI & M. K. MOSS (Eds)Evaluating and Responding to College Guidebooks and Rankings, pp. 45–59. San Francisco: Jossey-Bass.

MCINNIS, C., HARTLEY, R., POLESEL, J. & TEESE, R. (2000a) Non-completion inVocational Education and Training and Higher Education. Canberra: Department of Education, Training and Youth

Affairs.MCINNIS, C., JAMES, R. & HARTLEY, R. (2000b) Trends in the First Year Experience in Australian Universities.

Canberra: Department of Education, Training and Youth Affairs.MENTKOWSKI, M. ET AL. (2000) Learning that Lasts: integrating learning, development, and performance in college and beyond.

San Francisco: Jossey-Bass.MORGAN, M., FLANAGAN, R, & KELLAGHAN, T. (2001) A Study of Non-completion in Undergraduate University Courses.

Dublin: Higher Education Authority.MORRISON, H. G., MAGENNIS, S. P. & CAREY, L. J. (1995) Performance indicators and league tables: a call for

standards. Higher Education Quarterly, 49(2), pp. 128–145.TINTO, V. (1993) Leaving College: rethinking the causes and cures of student attrition, 2nd edn. Chicago, IL: University

of Chicago Press.YORKE, M. (1997) A good league table guide? Quality Assurance in Education, 5(2), pp. 61–72.YORKE, M. (1998) The Times ‘league table’ of universities, 1997: a statistical appraisal, Quality Assurance in

Education, 6(1), pp. 58–60.YORKE, M. (1999a) Getting It Right First Time. Cheltenham: Universities and Colleges Admissions Service.YORKE, M. (1999b) Leaving Early: undergraduate non-completion in higher education. London: Falmer.YORKE, M. (2001) Telling it as it is? Performance indicators, massi� cation and the press, Tertiary Education and

Management, 7(1), pp. 57–68.YORKE, M. WITH BELL, R., DOVE, A., HASLAM, L., HUGHES JONES, H., LONGDEN, B., O’CONNELL, C., TY-

PUSZAK, R. & WARD, J. (1997) Undergraduate non-completion in England, in: Undergraduate Non-completion inHigher Education in England. Bristol: Higher Education Funding Council for England.

Dow

nloa

ded

by [

Swin

burn

e U

nive

rsity

of

Tec

hnol

ogy]

at 0

8:22

26

Aug

ust 2

014