bob gilworth
TRANSCRIPT
Measuring Teaching Quality
Careers Registration and
Learning Gain
14 July 2016
Dr. Bob Gilworth - Director of The Careers Group, University of London
& AGCAS Director of Research
2
Careers Registration
- Employability-related questions included in student registration data collection
- Completed by all new and all re-enrolling students at the start of each year
- Pioneered at University of Leeds in 2012
- Both cross-sectional and longitudinal
- Already scaled and sustainable
3
Career Decidedness (CD)
Readiness to engage with career management- Decide, Plan, Compete
Please select the statement which best represents your current careers position: I am not ready to start thinking about my career yet (Decide) I have no career ideas yet but want to start thinking (Decide) I have some ideas about my career & am ready to start planning (Decide) I have a career in mind & intend to gain relevant work experience (Plan) I know what I want to do but not sure how to get there (Plan) I want to spend a year gaining experience (Plan) I am ready to apply for graduate level / professional opportunities (Compete) I am ready to apply for further study (Compete) I have been applying for opportunities & have not been successful (Compete) I have a job, further study or my own business plan confirmed (Other)
4
Employability Experience (EE)
I have gained work experience through (select all that apply):
a placement year during my degree
a summer internship with an organisation
a vacation internship (not summer) with an organisation
work shadowing
a short placement as part of a University module (e.g. 10 or 20 credit module)
part time work alongside my studies
a holiday job
volunteering
a position of responsibility in a club or society
full time work prior to my course (two years or less)
full time work prior to my course (more than two years)
self-employment / running my own business
I have no work experience to date
5
Benefits of CR
- Current: Live data vs DLHE data six months after graduation- Comprehensive: Extent and nature of students’ work
experience vs limited information from formal placements- Consistent: Same data on all students vs patchy data gathered
through departments- Connected: Links employability data to the student record- Comparable:
• Individual progress (starting points and trajectory) — what support students’ need and when
• Institutional impact —cohorts, departments, backgrounds— where to target scarce resources
6
Visualising the ‘journey’All students
Decide
Plan
Compete
Other
1st year
Decide
Plan
Compete
Other
2nd year
Decide
Plan
Compete
Other
Final year
Decide
Plan
Compete
Other
7
Project partnersInstitution Status
University of Bristol Implemented 2014
University of Exeter Implemented 2014
Goldsmiths, University of London Implemented 2015
King’s College London Implemented 2015
Lancaster University Implemented 2014
Liverpool John Moores University Implemented 2014
Queen Mary, University of London Aiming for implementation 2017
Royal Veterinary College Negotiating implementation, other measures
St George’s University of London Implemented 2015 (variations)
St Mary’s University Pilot from 2015, full implementation 2017
School of Oriental and African Studies Implemented 2015 (non-mandatory)
University College London Aiming for implementation 2016
University of Edinburgh Pilot in 2014, wider roll-out 2015
Ulster University Implemented 2015
8
Primary Research Question
Does Careers Registration provide a viablemethod for obtaining data that provides anindication of learning gain related to work
readiness?
Correlating CR data with existing measures ofwork related learning gain
9
Secondary Research Questions
- What’s the best way to implement CR?
• Statement wording
- Can we use CR to highlight employability needs of different groups?
• Links to WP factors
- Can we use CR to evaluate different employability activities?
• Stakeholder understanding of data
10
Suggested success criteria
1. Three cross-sectional analyses and three partial longitudinal analyses of CR data across three institutions
2. One comparison of CR data with DLHE, one smaller-cohort comparison with other measures of employability, one comparison of CR data with other data such as NSS scores, retention rates or academic record
3. An evaluation of the effectiveness of data collection and the quality of the data and monitoring roll-out of CR within three new institutions
4. An examination of the usefulness of CR data for informing student engagement, needs of specific student groups and collaboration with academic departments and employers
5. Recommendations for the implementation of CR in institutions and a standardised methodology
11
The TEF
Source: Fig 2. TEF assessment framework conceptual model
12
The TEFStudent Outcomes and Learning Gain
Indicative list of additional evidence
Aspect Possible examples of evidence
Student Outcomes and Learning Gain
• Learning gain and distance-travelled by students• Evidence of longer-term employment outcomes and progression
of graduates including into highly-skilled employment• Evidence and impact of initiatives aimed at maximising graduate
employability• Extent of student involvement in enterprise and
entrepreneurship• Impact of initiatives aimed at closing gaps in development,
attainment and progression for students from different backgrounds, in particular those from disadvantaged backgrounds or those who are at greater risk of not achieving positive outcomes.
Source: TEF Technical Consultation for Year Two, May 2016