curtis cartwright programme evaluation jan2010
DESCRIPTION
TRANSCRIPT
www.curtiscartwright.co.uk CC382D006 – 1.0 - 1
Evaluation
Dr Geoff CurtisDr Rob Hawtin
29 January 2010
Institutional Innovation Programme
www.curtiscartwright.co.uk CC382D006 – 1.0 - 2
Overview
• Institutional Innovation Programme (IIP)
• Evaluation questions
• Headlines from evaluation of Round 1 projects
• Initial findings
• Lessons identified
• Evaluation of Round 2 projects
• Approach
• Your contribution
• Self-evaluation guidance
www.curtiscartwright.co.uk CC382D006 – 1.0 - 3
Institutional Innovation Programme
• Issues with JISC approach to ICT in education and research
• Often applies to specific problems of department / institution
– Not the sector
• Solution often technology- rather than business-led
• Not necessarily sustainable
www.curtiscartwright.co.uk CC382D006 – 1.0 - 4
IIP concept
• Institutional approach
• Build on advances from JISC innovation activities
– Proven practices, technologies, standards and services
– Often substantial internal business change for institution
– Supported by significant senior management involvement
• Fund range of substantial ICT deployments/business change activities
• Use as exemplars to demonstrate innovation and good practice
• Extend benefits to sector
– Benefits realisation activities
• Problem areas identified
– professional bodies
– other initiatives
www.curtiscartwright.co.uk CC382D006 – 1.0 - 5
Top-level evaluation questions
• 1: Is IIP concept useful as general approach for institution-wide
problems?
• 2: Is IIP delivering desired outcomes at institutional level for key
areas of institutional ICT concern?• Improved community engagement• Improved and sustained JISC leadership to the sector• Improved capacity, knowledge and skills and diversity of background;• Sustainable technological solutions• Development of good practice approaches• Evidence base for good practice and dissemination• Overall sustainability of outcomes
• 3: Is IIP being managed well?
www.curtiscartwright.co.uk CC382D006 – 1.0 - 6
Evaluation
• Assessment of evaluation questions
• Lessons identified?
• Improvements?
• What else?
• Evaluating programme not the projects
www.curtiscartwright.co.uk CC382D006 – 1.0 - 7
Initial findings –IIP concept
• Assessment
• Useful general approach for creating sustainable change
• More general applicability
• Lesson identified
• Need to select/manage projects to meet IIP characteristics
• Areas for improvement
• Benefits realisation activities
• More stringent selection of areas of institutional concern and projects
• Training on concept and implementation
• Review approach to planning/funding of benefits realisation activities
www.curtiscartwright.co.uk CC382D006 – 1.0 - 8
Initial findings – IIP delivery
• Assessment
• Too early for definitive assessment
– SSBR had only just started
– Likely to mature post project completion
– Aspirational nature of outcomes
• Continue to focus on desired outcomes
www.curtiscartwright.co.uk CC382D006 – 1.0 - 9
Initial findings – IIP management
• The programme is generally well managed
• Lack of programme management resource and overall scale of Round 2
projects might be a risk to programme
• Focus role of SSBR
www.curtiscartwright.co.uk CC382D006 – 1.0 - 10
Initial findings – summary
Question After Round 1 Potential
1 IIP concept
2 IIP delivery
3 IIP management
Overall
www.curtiscartwright.co.uk CC382D006 – 1.0 - 11
Evaluation of IIP – phase 2
• Approach
• Phase 1 was by interviews against detailed questions
– Many more Round 2 projects
• Evaluation based on
– Questionnaires– Detailed evaluation questions as before– Quantitative and qualitative– Stakeholders– Project teams
– Follow-up interviews, where necessary
– Other available information– Programme meetings– Final and completion reports
– Self-evaluation
www.curtiscartwright.co.uk CC382D006 – 1.0 - 12
Evaluation of Round 2 projects
• Your contribution
• Please find time to respond to questionnaires
– End February
• Please do your final and completion reports
– Good things
– Lessons identified
www.curtiscartwright.co.uk CC382D006 – 1.0 - 13
Self-evaluation
• JISC evaluation handbook
• Based on your evaluation plans
• Tips on self-evaluation
• Outcomes
www.curtiscartwright.co.uk CC382D006 – 1.0 - 14
Contact us
• [email protected] 01483 685023
• [email protected] 01483 685024