www.monash.edu.au
Platform Influences on Survey Delivery and Response Rate
Presenter
Phillip AdamsEvaluations Manager
Australasian Higher Education Evaluation Forum 2008Friday 2 October, 2008
www.monash.edu.au
Presentation Outline
• CHEQ (Centre for Higher Education Quality)• The Way We Were• The Response• The Outcomes
www.monash.edu.au
CHEQ
• Established in September 2000
• Lead and support quality assurance and improvement
• various areas of teaching, research, research training and support services
www.monash.edu.au
The Evolution
Two drivers for the change –
• Quality was an integral part of the higher education sector
• Need for institutional measures of academic activity
www.monash.edu.au
Unit EvaluationsThe way we were
• 1998 – 2002
• item bank
• academics create own survey
• 2002 – 2004
• faculty takes responsibility
• faculty-wide questionnaire
• conduct at least every five years
www.monash.edu.au
The issues
• 1998 – 2002
• singular, aggregated report per unit/subject
• no university benchmarking capacity
www.monash.edu.au
The issues
• 2002 – 2004
• patchy take up
• saw mix of academic & faculty reports
• still no university benchmarking capacity
• few opportunities for monitoring improvement
www.monash.edu.au
Major change in 2005
• 10 common items across university
• Up to 10 faculty items
• To be taken of each unit each year it is offered
• Results posted on a common website
• Results to be systematically considered by faculty
www.monash.edu.au
Re-engineering Technology And Processes
• both paper-based and online surveys for all units
• large volume of survey responses in short turnaround time
• system capable of storing data in any survey, online or paper, in single location
• improved access to results for staff and key stakeholders
• reduced cost of survey development and processing
• allowed automated reporting on WWW
2005, CHEQ introduced Survey Management System (SMS). Significant benefits from change in technology and processes:
www.monash.edu.au
PDF survey
• Benefits
• able to fill in without downloading
• presentation appealing
• Drawbacks
• practically clumsy
• poor accessibility for visually impaired
www.monash.edu.au
PDF survey limitations
• Three main types of student queries/complaints
• students unfamiliar with a fillable PDF
• browser version incompatibilities
• web browser setup
• Plus
• lack of accessibility for visually impaired
• disenfranchised students
www.monash.edu.au
Our response
• prepared “how-to” information sheets
• initially assisted VI over phone and with customised survey
• shifted from PDF to HTML form
www.monash.edu.au
Outcomes
• student familiarity
• accessible for visually impaired
• assisted over phone
• special form design considerations
• No browser compatibility issues
www.monash.edu.au
Outcomes
• Zero complaints about accessAccess Problems
157
109
0 00
20406080
100120140160180
2005 2006 2007 2008
Years
Nu
mb
er o
f co
mp
lain
ts
www.monash.edu.au
Outcomes
• Increase in web based response rate Annual Online Response Rate
26.330.8
33.8
42.8
0
10
20
30
40
50
2005 2006 2007 2008 (s1)
Evaluation period
Res
po
nse
rat
e (%
)
www.monash.edu.au
Conclusion
• Quality cycle relies on the effectiveness of the evaluation system
• Better meet the needs of stakeholders
• Survey more user friendly
•Positive influence on response rate