hesa for planners. objectives identify best practice around quality assurance and use of data...

129
HESA for Planners

Upload: timothy-crawford

Post on 23-Dec-2015

220 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

HESA for Planners

Page 2: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Objectives

• Identify best practice around quality assurance and use of data

• Improve our understanding of check documentation and how it can be utilised

• Introduce the downloadable files and how they can be used

• Outline the future information landscape• Better understand the IRIS outputs• Learn from each other

Page 3: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how
Page 4: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

“You never finish HESA, you abandon it”

Utilisation of time – ‘opportunity cost’

Best practiceCollaborative approach to data

Efficient and cost-effective procedures

Systems that work for the organisationResource

Page 5: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

How to be good…

• Data ownership:- Systems (storage issues)- People

• Translation:- From HEIs internal data language to an external data language- The extent to which these match- The variety of external languages that an HEI has to work with

• Documentation- How, who, when

• Education- Value of data and transparency

Page 6: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Evidence (or anecdote) from the KIS

• New requirement – high profile• Data spread across institutions

– No documentation– Little/no control– No standardisation/comparability– Variable quality– Variable approaches to storage

• …being assembled and managed in spreadsheets

Page 7: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how
Page 8: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Spreadsheets

• Often created by people who don’t understand principles of sound data management

• Conflate data and algorithms• Almost impossible to QA• Spread and mutate like a virus

Search “Ray Panko spreadsheets”

Page 9: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

The institutional perspective

Page 10: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

A Planning Perspective on HESA ReturnsFidelma Hannah, Director of Planning

Loughborough University

Page 11: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Overview

Responsibility for completing HESA returns lies with relevant sections of the University but Planning has the role of:

co-ordinating the returns ensuring appropriate governance, data assurance and consistency

between all HESA returns disseminating HESA data across the University

Page 12: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Co-ordination

The Planning Office produces a schedule of Statutory Returns listing all HESA, HEFCE and other Funding Agency returns, identifying: Submission dates Ways in which data is used Process for completion Independent checking and sign-off process

The Planning & Finance Offices are accountable to the Vice-Chancellor and Audit Committee for the verification and accuracy of the data returns.

The Planning Office liaises with all relevant sections of the University to ensure that returns are completed, checked and signed off in accordance with the schedule.

Page 13: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Co-ordination, contd.

Planning is: Involved most directly with preparation and checking of HESA student

return

BUT Has a significant and increasing input into the processes used for other

HESA returns

Page 14: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Responsibility for Completion of HESA Returns

Student Return – Student Office, Academic Registry Staff Return – Human Resources Finance Return – Finance Office HEBCI – Enterprise Office/Planning Office Destination of Leavers - Careers Estates Management Statistics – Facilities Management Institutional Return – Planning Office

Page 15: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Other Student - Related Returns

HESES TRAC OFFA Teaching Agency Skills Funding Agency Education Funding Agency REF

All of these returns incorporate HESA data

Page 16: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Governance and the role of Audit Committee

The University’s Audit Committee must provide assurance about the management and quality assurance of data provided to HEFCE, the Higher Education Statistics Agency (HESA) and other public bodies.

This is a requirement of the HEFCE Financial Memorandum and Accountability and Audit Code of Practice introduced on 1 August 2008.

Audit Committee reviews the schedule of statutory returns annually and also receives regular reports from internal and HEFCE auditors on the various returns.

Page 17: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Data Assurance – in year

Planning: Liaises closely with Student Office during preparation of HESES return as

this helps to ensure data quality in year

Co-ordinates monthly Data Management Group meetings Membership : IT Services, Planning, Student Office, Research Student Office,

Careers and Admissions

Reviews the funding and monitoring data produced by HEFCE after HESA return has been submitted

Page 18: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Data Assurance during HESA preparation

Planning: Maintains regular contact with Student Office during preparation of HESA

return Uses the HEFCE recreation files extensively to check data quality before

HESA student return is finally committed (This includes detailed examination of individualised student files)

Undertakes a comprehensive review of check documentation at commit stage with cross-checking by Finance Office

Retains comprehensive records and an audit trail of the checking processes

Joins the briefing meeting with VC before sign-off

Page 19: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Consistency across HESA Returns

Vital to ensure that data is consistent across HESA Staff, Student and Finance returns because data will be combined

Important to align JACS, Cost Centres and UOAs Implications for subject mapping must be considered Implications for funding must be considered ,e.g. JACS codes and cost

centres both used to determine additional funding for very high cost subjects

Added complexity of Key Information Sets

Page 20: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Disseminating Bench-marking Data and Comparisons

Use of HEIDI to generate bench-marking data at subject level including: Student: Staff Ratios NSS Employability Degree Classifications International/UK/EU students Completion rates

Production of institutional profile data such as: Student profile Income & Expenditure profile Cost Centre profile

Page 21: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Final Comments

Understanding HESA data is becoming even more critical in current HE environment

Ensuring the accuracy of HESA data is important for future funding streams (SNC monitoring, additional funding for high-cost subjects, WP indicators, etc.)

Effort invested will make future income streams more reliable, avoiding claw-back in later years.

BUT Complexity and cross-checking is increasing demands on Universities.

Page 22: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

HESA – Living and Learning

Becs Lambert

Senior Assistant Registrar Strategic Planning and AnalyticsUniversity of Warwick

Page 23: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Outline

1. The Warwick context

2. Warwick’s HESA process

3. Sign-off, Verification and Quality Assurance

4. Issues

5. Positives

6. Challenges moving forward

7. Using HESA data – the HEIDI API

Page 24: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

My context…

Maternity leave cover

Planning: responsibility for Enrolment, HESES, HESA, KIS, student numbers…

…October crunch point for key Planning activities

Student reporting and HESA experience = HEIDI (basically nil)

= …baptism of fire

+

+

+

Page 25: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Warwick context…

Student number related returns (HESA, HESES, KIS)

Located in the Deputy Registrar’s Office, but close liaison with Academic Registry re: data input, quality, implications.

One key member of staff (data input, liaison, query resolution, data quality management, Minerva, etc. etc.)

Page 26: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

The HESA process- Warwick SITS update schedule (positives and negatives)

- Prep and housekeeping from April – address learning points from previous year,

implement procedures for HESA changes, check ‘usual suspects’

- Strong use of validation kit to identify issues

- Use of internal access databases to cross-check HESA return data and ensure

comprehensive data checks

- Aim for early as possible submit/commit schedule to front load schema and

business validation issues

Page 27: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Sign-off, verification and data quality

• Validation kit is a good early prep tool (though limitations)• Check docs and Minerva are key tools (post markers for further

internal analysis)• Two year historical comparison of return data – explain or check • Student level data checks (targeted)• Scrutiny of check docs by Assistant Registrar (close to student data,

highlight potential issues, discrepancies)

Verification and Data Quality

• Senior level oversight and final sense check of numbers• VC involvement• Understanding of downstream implications of the return

Sign-off

Page 28: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Issues

Workload in ‘peak season’

Reconciliation reports (HESA/HESES)

Ownership

Mis-match of needs (HESA rules v internal processes)

Page 29: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

StrengthsStrong HESA and institutional expertise (also a

negative??)

Collegial spirit

Established and clear process for generation, checking, verification and submission of HESA return

Strong data quality focus throughout the year given BI focus of office

Minerva

Page 30: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Challenges moving forward

Look to be less reactive to HESA data quality issues

More structure understanding of HESA implications and responsibilities across data owning departments

Increased use of FAMD docs

Re-vamp of process documentation (Business Continuity)

Page 31: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Before…

• Flexible report writing with drag and drop interface for usability but can be slow to build large reports

• Direct output to Excel or XML file

• Limited to 125 columns for extracts (eg. Finance Table 5b has 490 columns of data times 3 years = 1470 columns = 12 separate extract files)

• People like cross tabular reports, but data warehouses need flat data files so the extracts need to be transformed prior to loading

• Our data transformation was based on a VBA script in Excel, but needed to be customised for each extract (different numbers and column groupings)

• Depending on the extract size many files may need to be processed and concatenated together

Using HESA data – the HEIDI API

Page 32: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

• Turning this:

into this:

is relatively slow and painful!

HEIDI > Data Warehouse

Page 33: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

• API permits rapid extraction of large volumes of data in warehouse-friendly format

• Based on standard web services technology

• Difficult to use and requires specialist technical skills but very powerful and fast

• Generate a custom url to produce a response (eg.

https://heidi.hesa.ac.uk/api/1.0/datareport?rowtype=3297&year=61422&domain=3311&valuetype=4008&field=61432

produces a report of UCAS Accepted Applicants for 2011/12 by Institution and Gender)

• Extracts produced as single files containing data and metadata (field descriptions)

• Simple direct loading into warehouse

• XML shredded (transformed) into data tables using the native query language capabilities

HEIDI API > Data warehouse

Page 34: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

• Lessons Learnt:

• API is not a “magic bullet” but is a useful additional tool

• Harvesting HESA data for BI analysis now down from days to hours, but specialist skills and knowledge still required

• Current API needs simplifying and extending to allow multi-year and multi-value extracts

• Next steps for Warwick:

• Use of the API still requires a number of steps – plan is to more tightly integrate the extract and loading of data using SQL Server Integration Services

• Provision of standardised self-service reporting capability for power users to extract and analyse HESA data contained in the warehouse

HEIDI API > Data warehouse

Page 35: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Discussion

• Discuss the following as a table:• How good are you at HESA?- Consider factors such as data ownership, documentation,

staffing, knowledge, resilience, training, systems, data quality process – how extensive and sophisticated it is. How often you use HESA data and what for and how is the process and data managed/structured internally. What are the barriers and how do you overcome them?

• Now consider and rate your own institution:

1st class 2:1 2:2 3rd Unclassified

Page 36: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Using check documentation

Page 37: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

What is check documentation?

An Excel workbook which displays the data in a series of tables

Used by analysts at HESA for quality assurance

Available after any successful test or full commit

Page 38: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Why should I use it?

Check documentation gives an overview of the submitted data which can help identify potential issues

Provides context to the queries raised by HESA

The institution will be able to spot anomalies that HESA would not

Comparison feature also useful for later commits/test commits to monitor changes

Check doc is one of many reports and is best used in conjunction with other reports

Page 39: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Task

1. In your groups, or individually, complete check documentation tasks 1-4

Page 40: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

How can check doc be used?

• Use the check documentation guide produced by HESA as a starting point

• Many of the items provide year on year comparisons:

Page 41: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Using check documentation

Different populations and groupings are used for each item in the check documentation, including derived fields

For 2012/13 the definitions sheet has moved to the coding manual

Page 42: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Who are those 5 students??

• To get the most out of check documentation and work out whether something is an error, you need to identify the records behind the table

• To do this you can use Data Supply which contains much of the raw data submitted alongside the derived fields used by HESA

• Pivot tables can be used to recreate items and identify particular cells

Page 43: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Identifying students:

• The HESA for Planners manual contains instructions on recreating the populations and conditions used in check documentation

• As an example we will recreate item 6a ‘Student cohort analysis’….

Page 44: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Check doc changes for 2012/13

• Revised tolerances• Items 1, 2 & 3 will now highlight year on year changes of

+/- 10%/50 students• Item 11 will look at sector averages rather than just the

previous year

• Move to JACS3 and new cost centre coding frame• New Fees tab• More detailed breakdowns, summations and

percentage changes added to enable checking

Page 45: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Item 2a - Qualifications awarded

What is the difference between 2 and 2a?Item 2 Item 2a

Shows the qualifications awarded to students in the format that will be published in the student volume

Displays the year on year differences using the qualifiers field of XQLEV501 (including the split out of PGCE and Post grad cert in Education.

Used to check that the qualification awarded are, in the main, those which they were aiming for

More consistent with the SFR variance figures

Item 2a - Qualifications obtained by students on HE course by level of qualification obtained and mode of study (2012/13 and 2011/12)

Page 46: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Item 7 – Highest qualifications on entry

• Now split into 7a & 7b ‘proportion of highest qualification on entry for first years’

• Subtotals also added to item 7a

Page 47: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Item 12 – average instance FTE

• This item has been broken down further to provide a three way split of starters, leavers and ‘others’.

• The different groups may have very different FTE values that impact the average

Page 48: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Other reports

Page 49: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how
Page 50: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Minerva

…is the data query database operated by HESA

• During data collection HESA (and HEFCE) raise queries through Minerva and institutions answer them

• These responses are then reviewed and stored for future use by HESA and the institution

Data submitted

Quality assurance

Queries raised

Queries resolved

Sign-off

Page 51: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Using Minerva for quality assurance

• Responses from previous years are retained in the Issue Report

• Review targets set for the current year

• Queries raised by HESA are prioritised:

Page 52: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Contextual Intelligence

• At the request of the National Planners Group HESA have formulated a ‘public’ version of Minerva

• Designed to give users of the data additional context• HESA has published a query to Minerva to which HEIs

can add notes about their institution e.g. ‘we recently opened a new department’

• HESA will not interact with what is added• Will remain open throughout the year • HESA will extract and send the information to

accompany data requests

Page 53: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Using downloadable files

Page 54: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Downloadable files

• Data Supply (Core, subject, cost centre, module and qualifications on entry tables)

• NSS inclusion (person and subject) and exclusion files• POPDLHE• TQI/UNISTATS• All available after every successful full and test

commit

Page 55: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Using downloadable files

• The files should be utilised to:- Carryout additional DQ checks- Benchmarking- Planning/forecasting- Improve efficiency (recreating data from scratch

unnecessary)

Page 56: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

League tables

• Student staff ratios by institution and cost centre • First degree (full-time for Guardian) qualifiers by institution and league table subject

group• Average total tariff scores on entry for first year, first degree students by institution

and league table subject group. • Data is restricted to tariffable qualifications on entry (QUALENT3 = P41, P42, P46,

P47, P50, P51, P53, P62, P63, P64, P65, P68, P80, P91) (Times applies ‘under 21’ restriction, Guardian applies ‘full-time’ restriction)

• Full-time, first degree, UK domiciled leavers by Institution, League table subject, Activity

• Full-time, first degree, UK domiciled leavers entering employment• Graduate employment/Non graduate employment/Unknown• Positive destinations /Negative destinations • Expenditure on academic departments (Guardian)• Expenditure on academic services (Guardian, Times, CUG)• Expenditure on staff and student facilities (Times, CUG)

Page 57: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Who cares? Why bother?

• …because you can’t afford not to care• In what space does student recruitment take place?• Extended coverage…• Subject based…• Supply and demand are linked to measures of quality• Internationalisation of Higher Education

Page 58: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

But be aware of the tail wagging the dog…

• Collectively we can become obsessed about specific measures…

• …and dangerously on the wrong type of measures…• …and instead of good (or accurate) ranking being

born out of doing your day to day business well, it becomes fuelled by quick fixes

• Measurement culture tends to trade long-term value for short-term gains…

• …this holds true in ‘data world’

Page 59: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how
Page 60: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

But there are gains to be sought – both in terms of quality and benchmarking“Firstly, you need a team with the skills and motivation to succeed. Secondly, you need to understand what you want to achieve.Thirdly, you need to understand where you are now.Then, understand ‘aggregation of marginal gains’. Put simply….how small improvements in a number of different aspects of what we do can have a huge impact to the overall performance of the team.”

Dave Brailsford, Performance Director of British Cycling

Page 61: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

But there are gains to be sought – both in terms of quality and benchmarking“Firstly, you need a team with the skills and motivation to succeed. Secondly, you need to understand what you want to achieve.Thirdly, you need to understand where you are now.Then, understand ‘aggregation of marginal gains’. Put simply….how small improvements in a number of different aspects of what we do can have a huge impact to the overall performance of the team.”

Dave Brailsford, Performance Director of British Cycling

Page 62: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Is there a correlation between this spread and league table positioning?

Page 63: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Before you begin…

• Remember the different populations• Use derived fields (those beginning with an X!)• The INSTCAMP field can be used to better analyse

and understand your data

Page 64: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Demonstration…

Page 65: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Performance indicators

• The PI tables (available from the HESA website) give sector wide data on:

- Non-continuation rates- Widening participation of under-represented groups

and those in receipt of DSA- Research output- Employment of leavers• Included are benchmarks and definitions

Page 66: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

DDS

• http://www.hesa.ac.uk/content/view/2664

Page 67: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Scenario planningThe impact of fee increases on applications

Page 68: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Total applications 05/06 – 11/12

04/05 05/06 06/07 07/08 08/09 09/10 10/11 11/120

10,000

20,000

30,000

40,000

50,000

60,000

The Nottingham Trent UniversitySheffield Hallam University

heidi

Page 69: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

% change of total applications

05/06 06/07 07/08 08/09 09/10 10/11 11/12

-25.0

-20.0

-15.0

-10.0

-5.0

0.0

5.0

10.0

15.0

20.0

The Nottingham Trent University Sheffield Hallam University

heidi

Page 70: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Total applications by regions

04/05 05/06 06/07 07/08 08/09 09/10 10/11 11/120

50,000

100,000

150,000

200,000

250,000

300,000

Total Yorkshire & the HumberTotal East Midlands

heidi

Page 71: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

NSS 05/6 results versus 06/7 applications heidi

Page 72: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Student:Staff ratios 2005/06

Page 73: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

% change in applications by subject for sector

-12

-10

-8

-6

-4

-2

0

2

4

6

Total applications % change 2006/07

Nursing

Mass communications

Creative arts

Education

Law

Page 74: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

2006/07 Subject profile

Nottingham Trent University Sheffield Hallam University

Check documentation

Page 75: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Performance indicators 2006/07

Page 76: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

The Nottingham Trent University Sheffield Hallam University0

10

20

30

40

50

60

70

80

90

2005/06 Building condition Total Non-residential - condition A & B

2005/06 Building condition Total Non-residential - condition A & B

Page 77: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how
Page 78: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

What we ‘know’…

- Some subject areas are more price elastic than others?

- Applicants take note of NSS?- Condition of the estate matters to applicants? - Some socio-economic groups are more affected by

fee increases than others• Each of these variables might have a value of x

number of applicants

Page 79: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

What we don’t know…

• …but could scenario plan for…- Future government policy on HE funding- Social/cultural/economic impact- The power of perception

Page 80: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

NSS 2011/12

Nottingham Trent Unviersity Sheffield Hallam University0

10

20

30

40

50

60

70

80

90

Overall satisfaction

Page 81: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Student:Staff ratios

Page 82: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

2011/12 Subject profile

Nottingham Trent University Sheffield Hallam University

Check documentation

Page 83: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Performance indicators 2011/12

• Was 2.3% difference, now 0.2%

Page 84: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

2005/06 Build

ing condition To

tal Non-Resi

dential

- Condition A &

B

2010/11 Build

ing condition To

tal Non-Resi

dential

- Condition A

0

10

20

30

40

50

60

70

80

90

100

The Nottingham Trent University Sheffield Hallam University

Page 85: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

heidi - % change of total applications

05/06 06/07 07/08 08/09 09/10 10/11 11/12

-25.0

-20.0

-15.0

-10.0

-5.0

0.0

5.0

10.0

15.0

20.0

The Nottingham Trent University Sheffield Hallam University

Page 86: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

heidi - % change of total applications

05/06 06/07 07/08 08/09 09/10 10/11 11/12 12/13

-25.0

-20.0

-15.0

-10.0

-5.0

0.0

5.0

10.0

15.0

20.0

The Nottingham Trent University Sheffield Hallam University

Page 87: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Aggregation of marginal gains

Page 88: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

The power of perception

• …can be influenced by the power of datahttp://www.youtube.com/watch?v=ZWTJ_TPraLQ

• If you don’t like what they’re saying, change the conversation…

• …what data are you using on the website and is it the right data?

• …repositioning - find what you are good at and sell it (both internally and externally)….

• …but never neglect what you need to improve

Page 89: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

• http://www.ucl.ac.uk/about-ucl

Page 90: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

`

Page 91: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Higher Education Information Database for Institutions

Estates Destinations

Finance HE-BCI

Student Staff

Applications Equality

National student survey

Derived statistics

heidi.hesa.ac.uk

Available to all HEIs

Page 92: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Capabilities

Collate, cross-reference and interpret

information

View, create and export reports, charts

and custom tabulations

Generate aggregations, ratios and percentages

Benchmark the performance of your

institution against others

Use heidi data within your own business

intelligence software

Embed heidi reports and charts into your

own website(s)

Page 93: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Notes and definitions provided

Adjust the year or the group from within the report

Page 94: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Sharing

• New to heidi• Allows reports and

charts to be shared with others – Including non-heidi users

• Share by email using the ‘send to’ link

• Use the HTML code to embed reports or charts into websites

Page 95: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Use groups and sub-groups to compliment

analysis of charts

Adjust the year or the groups from within the chart

Download to any version of PowerPoint

Page 96: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Charts

New Radar Chart

Page 97: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Application Programming Interface

• Does your HEI have it’s own data warehouse or BI system?

• Use API to specify wanted heidi data and retrieve in a usable format

• API is aimed at users familiar with writing and understand programming code. HESA can provide support to colleagues involved with API at your institution

Page 98: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

The future

Re-designed DLHE data set

Federated user accounts

Benchmarking functionality

We are always keen to hear feedback, especially ideas and suggestions for

future release of heidi. Please send any

comments to [email protected]

Page 99: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Sign up

heidi account• Contact your Local

Administrator for access to heidi – Expert account allows

access to all features– Standard account allows

access on a view mode

[email protected]

JISC • Mailing service which allows

heidi users to make new contacts, ask questions and share knowledge and best practice

www.jiscmail.ac.uk

Page 100: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Welcome slide

Page 101: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

HESA for Planners (Student Record) Seminar

May 2013

Anthea Beresford – Data Assurance Consultant, HEFCE

Page 102: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

The aim of this session is to advise you of:• areas of HEFCE’s Data Assurance Team’s coverage from April 2013

to March 2014.

HEFCE’s Data Assurance Team activity April 2013 to March 2014

Page 103: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Data assurance activity of the Data Assurance Team• team of 3, supplemented by a consultant;

• we are part of the overall data assurance framework;

• annual audit plan agreed by our Funding Round Process Board (internal Executive oversight) and our Audit Committee to which we report regularly and provide an annual report on audit outcomes;

• ever changing activity, for example, as funding rules change and new areas become important;

• interested in both funding and non-funding issues with data.

Page 104: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

3 distinct types of activity:

• data audit;

• data verification;

• data reconciliation.

Core roles of the Data Assurance Team

Page 105: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Areas identified for attention from April 2013 to March 2014• Student funding data work:

• HESA student data verification work pre-sign-off;

• HESES verification work pre-sign-off;

• 2011-12 outturn review of FT UG HEFCE-funded student non-completion rates.

• Research Funding data work:

• Research income from Charities;

• Research income from Business;

• Research HESA student data exploratory work.

• Key Information Set (KIS) 2013/14

• Destination of Leavers from Higher Education (DLHE) 2011/12

Page 106: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Areas identified for attention from April 2013 to March 2014 (cont.)• National Scholarship Programme (NSP)

• 2011-12 Funding and Monitoring Data (FAMD) (reconciliation) exercise

• BIS Service Level Agreement work:

• Access to Learning Fund (ALF).

• Higher Education Business and Community Interaction Survey (HE-BCIS)

• Student Number Control (SNC)

• Equivalent and Lower Qualifications (ELQ)

Page 107: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

• 2011-12 outturn review of FT UG HEFCE-funded student non-completion rates:• desk based review of a 5% random sample of FT UG HEFCE-funded students.

• National Scholarship Programme (NSP):• desk based request for explanations of differences between HESES11, HESA

2011-12 and HESES12 new entrant student numbers.

• Research income from Charities and Business; Key Information Set (KIS) 2013/14; Destination of Leavers from Higher Education (DLHE) 2011/12; Access to Learning Fund (ALF):• Pre-audit visit review of data; on-site visit; post visit follow-up; issue of audit

report with recommendations; completed action plan for approval; closure of audit; implementation of funding adjustments following the Appeals process, where relevant.

Data audit – established work

Page 108: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

• Current audit programmes can be found at:http://www.hefce.ac.uk/whatwedo/invest/institns/funddataaudit/dataaudit/

• Current audit reports can also be at that link. Note the new KIS 2012/13 report.

Data audit – established work

Page 109: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

• Research HESA student data exploratory work; Higher Education Business and Community Interaction Survey (HE-BCIS); Student Number Control (SNC) and Equivalent and Lower Qualifications (ELQ):• Pre-audit visit review of data; on-site visit; post visit follow-up; issue of audit

report with recommendations; completed action plan for approval; closure of audit; implementation of funding adjustments following the Appeals process, where relevant.

Data audit – developmental work

Page 110: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

• HESA student data verification work pre-sign-off:• Desk based;

• Working in conjunction with HESA;

• Querying institutions on their data during the student data collection period to assist institutions in identifying potential data issues for correction before sign-off;

• We will publish guidance nearer the time at:

http://www.hefce.ac.uk/whatwedo/invest/institns/funddataaudit/dataverification/

• HESES verification work pre-sign-off:• Desk based;

• Working with institutions between initial submission and sign-off, obtaining explanations for data differences or changes to data.

Data verification

Page 111: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

• Reconciliation between HESES11 and HESES11 re-creation based on HESA 2011-12 data:• Desk based;

• Thresholds for selection;

• 2 stage process this year due to students with undetermined completion status (FUNDCOMP=3). The ‘Completion Status Survey’ is currently underway where we are asking institutions to update their completion status information for those who were returned as FUNDCOMP=3 in their 2011-12 return. The deadline for sign-off for this is 19 July 2013. We have therefore selected institutions who currently break the selection criteria thresholds. All institutions will be looked at again following submission of the Completion Status Survey;

• Gain explanations of differences;

• Action plan and amendment of data as necessary;

• Formal sign-off by institution;

• Implementation of any funding adjustment following an Appeals process.

Data reconciliation

Page 112: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

• The link to guidance on our website concerning this area of activity can be found at:http://www.hefce.ac.uk/whatwedo/reg/assurance/datareconciliations/

Data reconciliation (cont.)

Page 113: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Any questions?

Finally….

Page 114: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Thank you for [email protected]

Page 115: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

The HE information landscape

Update

Page 116: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how
Page 117: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Recommendations to RPG

• Governance for data and information exchange across the sector

• Development of a common data language– Data model, lexicon, thesaurus

• Inventory of data collections• Specific data standards work

– JACS– Unique Learner Number

Page 118: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

2. Data model, lexicon and thesaurus

• Review of existing collections/definitions – the as is• Better understanding of differences/similarities

– In definitions– In terminology

• Coming from both angles:– What are collectors asking for? – What are institutions supplying?

• To inform future standardisation and data sharing discussion• Deliverables:

– Data model, lexicon and thesaurus– Maintenance plan

Page 119: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

3. Inventory of data collections

• HEBRG survey identified 550 lines of reporting• Very little detail (width)• Is it complete? (length)• We need a solid understanding of the current burden

– To help HEIs become more joined up in their reporting– To challenge data collectors to reduce duplication

• Deliverables:– Database of collections– Maintenance plan

Page 120: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

4a JACS development

• Problems:– JACS could have far broader use– Current structure has run out of space

• Analysis of requirements• Exploration of coding options• Deliverable:

– Road map for future development

Page 121: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

4b ULN implementation

• ULN widely accepted as a Good Thing– Reducing burden by replacing existing IDs– Adding value through better data linking/sharing

• What are the real barriers to adoption?• What would it take to resolve these issues?• Deliverable

– An assessment of where we currently are with ULN– Commitment to a roadmap?

Page 122: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Governance?

• What will it do?

• What authority does it have to progress actions?

• How will the work be delivered and coordinated?

Page 123: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Proposal

• A programme of work…• …made up of specific projects…• …overseen by a Programme Board…• …and reporting to a Sponsoring Group

• Utilising best practice from Managing Successful Programmes

Page 124: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Programme board

SRO/Programme Director

Chair of the Programme board

Sponsoring group

Project A

Project B

Project C

Programme Management Office

Page 125: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Programme board

SRO/Programme Director

Chair of the Programme board

RPG

Project A

Project B

Project C

Programme Management Office

Page 126: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Programme board

SRO/Programme Director

Chair of the Programme board

RPG

Project A

Project B

Project C

Programme Management Office

Page 127: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

www.hediip.ac.uk @hediip

Page 128: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

HEDIIP

• Enhance the arrangements for the collection, sharing and dissemination of data and information

• Programme management office based at HESA• Publishing and maintaining the inventory of data

collections• Carry forward the established projects

– Common data language– Replacement for JACS– Implementation of the ULN

• Other strategic developments

Page 129: HESA for Planners. Objectives Identify best practice around quality assurance and use of data Improve our understanding of check documentation and how

Keep in touch

If you require additional training help, including bespoke visits to your institution, get in touch with the training department…w: www.hesa.ac.uk/traininge: [email protected]: 01242 211472

Follow us on Twitter: @HESATraining