tracking, monitoring and evaluating
DESCRIPTION
A presentation to Teach First teachers on the 3rd November 2015TRANSCRIPT
![Page 1: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/1.jpg)
www.derby.ac.uk/iCeGS
Tracking, monitoring and evaluating
Why counting is an important part of your school’s career development programme
Tristram Hooley
![Page 2: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/2.jpg)
www.derby.ac.uk/iCeGS
Overview
By the end of this session participants will have:• Considered what information they should be keeping to
support the effective deliver of their school’s career development programme.
• Discusses the difference between tracking, monitoring and evaluation.
• Identified a range of different kind data that they can use to evaluate their careers programme.
• Developed an action plan to improve their school’s use of data.
![Page 3: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/3.jpg)
www.derby.ac.uk/iCeGS
What is tracking?
![Page 4: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/4.jpg)
www.derby.ac.uk/iCeGS
What to track
• Progress• Engagement• Interventions• Decidedness• Follow through• Destination
Which of these do you track?
How do you track them?
![Page 5: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/5.jpg)
www.derby.ac.uk/iCeGS
Approaches to trackingWho How WhenTeachers
Careers lead
All school staff
Admin/temp staff
Notes and card
Databases
School systems
Throughout school
At key decision points
After school
![Page 6: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/6.jpg)
www.derby.ac.uk/iCeGS
Tracking software
• Schools Intelligence http://www.schoolsintelligence.co.uk/
• Pixl Edge http://www.pixl.org.uk/edge
• UCAS Adviser Track https://www.ucas.com/advisers/managing-applications/adviser-track-and-schools-reports-2015-16
• Bespoke system development
![Page 7: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/7.jpg)
www.derby.ac.uk/iCeGS
Approaches to practice
• Integration into personal development planning (e-portfolios)
• Form tutor responsibilities• Central administration and flagging• Career interviews (including group interviews)• Triage processes• Integration with existing school processes e.g. VLE,
school reports• RONIs
![Page 8: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/8.jpg)
www.derby.ac.uk/iCeGS
Your tracking approach
Discuss what you are going to change/develop in your school.
![Page 9: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/9.jpg)
www.derby.ac.uk/iCeGS
Monitoring and evaluation
• Monitoring allows you to check that you are doing what you said that you were going to do.
• Formative evaluation allows to you inform what is being done while the activity is still in progress.
• Summative evaluation creates a summary of what has been achieved and what the impacts have been.
![Page 10: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/10.jpg)
www.derby.ac.uk/iCeGS
Why evaluate?
Evaluation enables us to:
– examine what we do
– think about how we can improve it
– decide on whether it was worth doing
– provide others with a summary to help them to understand what was done.
www.derby.ac.uk/icegs
![Page 11: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/11.jpg)
www.derby.ac.uk/iCeGS
Discussion: Evaluation data
• What data do you have which could allow you to make a judgement about the impact of your practice?– What level of impact would it describe?
• What data could you collect which could allow you to make a judgement about the impact of your practice?– What level of impact would it describe?
www.derby.ac.uk/icegs
![Page 12: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/12.jpg)
www.derby.ac.uk/iCeGS
Key principles
• Granular
• Linkable
• Analysable
What is your counter-factual?
![Page 13: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/13.jpg)
Levels of impactReturn on investment
Results
Behaviour
Learning
Reaction
Take-up
Investment
![Page 14: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/14.jpg)
www.derby.ac.uk/iCeGS
Publish or perish
• Too little evaluation and impact work on careers work is published.
• Writing up your evaluation for broader circulation is an important way to support the development of the sector. • Self publication• Journal publication• Partnership with academics• Using external consultants
www.derby.ac.uk/icegs
![Page 15: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/15.jpg)
www.derby.ac.uk/iCeGS
Write it down
“My main reason for writing is simple: I do not know what I think until I have written it. In conversation one can get away with loose, exploratory thinking, but in writing it down one has to weigh up the arguments and the evidence, and decide what it all means and where one stands. It is hard work, but important; and if published, it adds to the body of knowledge on which others can draw.” Tony Watts
![Page 16: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/16.jpg)
www.derby.ac.uk/iCeGS
On working with academics and researchers• What matters to them
– Time– Money– Authorship– Impact
• What you can give them– Your time– Funding– Authorship or co-authorship– Data– Access to interventions and research populations– The change to impact on practice
![Page 17: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/17.jpg)
www.derby.ac.uk/iCeGS
Your monitoring and evaluation approach• Discuss what you are going to change/develop in your
school.
![Page 18: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/18.jpg)
www.derby.ac.uk/iCeGS
Useful resources
• Dent, P., Garton, E., Hooley, T., Leonard, C., Marriott, J. and Moore, N. (2013). Higher Education Outreach to Widen Participation: Toolkits for Practitioners. Evaluation, 2nd. Edition. Bristol: HEFCE.
• Hooley, T. (2014). The Evidence Base on Lifelong Guidance. Jyväskylä, Finland: European Lifelong Guidance Policy Network (ELGPN).
• Hooley, T., Marriott, J. and Wellens, J. (2012). What is Online Research?: Using the Internet for Social Science Research. London: Bloomsbury Academic.
• Hughes, D., Bowes, L., Hartas, D. and Popham, I (2001). A Little Book of Evaluation. Sheffield: CSNU.Hughes, D., Lang C. and Popham I. (2001).
• Taylor, A.R. & Hooley, T. (2014). Evaluating the impact of career management skills module and internship programme within a university business school. British Journal of Guidance & Counselling, 42(5): 487-499.
![Page 19: Tracking, monitoring and evaluating](https://reader033.vdocuments.site/reader033/viewer/2022042608/5695d52b1a28ab9b02a44d98/html5/thumbnails/19.jpg)
www.derby.ac.uk/iCeGS
About me
Tristram HooleyProfessor of Career EducationUniversity of Derbywww.derby.ac.uk/icegs
[email protected] @pigironjoe https://adventuresincareerdevelopment.wordpress.com/