developing a system function future state for online management of assessment
TRANSCRIPT
Developing a system function future state for online management of assessment
Stuart Hepplestone and Scott PorterSheffield Hallam University, UK1
2
Workshop schedule
13:40-13:45 (5 mins) Welcome
13:45-13:55 (10 mins) Presentation: Assessment Journey Programme
13:55-14:10 (15 mins) Activity: vision and lifecycle
14:10-14:25 (15 mins) Presentation: technical model
14:25-14:45 (20 mins) Activity: key developments
14:45-14:50 (5 mins) Presentation: stakeholder benefits
14:50-15:05 (15 mins) Activity: implications for your role
15:05-15:10 (5 mins) Final thoughts and questions
15:10 Session end
3
Workshop outcomes
By the end of this workshop, participants will:• learn how the Assessment Journey Programme at
Sheffield Hallam University is delivering a seamlessand improved assessment experience
• consider how the accompanying assessment lifecycle and technological future state model reflects that oftheir own institutions
• deliberate how Blackboard might realise the future state for online management of assessment
• consider the potential implications of online management of assessment for their own role
4
About Sheffield Hallam University
• One of largest universities in the UK• Over 33,000 students (26,000 undergraduate and 7,000 postgraduate students)• Over 2,100 academic staff• 4 teaching faculties covering 18 academic departments• Over 700 courses
5
Assessment Journey Programme
• Will investigate, define and deliver the changes needed to provide a seamless and improved assessment experience for students and staff, as well as improving the operational effectiveness of assessment delivery
• Delivered over 3 years• Two deliverables:
– Assessment Design and Delivery (ADD)
– Online Management of Assessment (OMA)
• Executive sponsors: – Deputy Vice-Chancellor
– Secretary and Registrar
6
Assessment Journey Programme
Will support the University Strategyand delivery of an excellenteducational experience by:• delivering clear and accessible
guidance and information aroundassessment design, marking and feedback
• providing a view of upcoming assessments for the academic year
• online submission and receipting of assessments• publishing feedback and marks online• delivering an integrated technical model for single mark entry• providing access to a course view of feedback and assessment outcomes
7
Vision and lifecycleAssessment is recognised as an integral part of
the University strategy
Assessments are designed around clear learner focused principles
Expectations for assessment are clearly set around what is required, how work will be assessed and when this will be submitted
The assessment experience for students and staff is clear and coherent, providing integrated processes
and one point of access for assessment related information
Students will primarily submit work online and will have access to all feedback and their marks,
online from a single point of access
Feedback tools and resources are made available that support staff to generate useful feedback for
students quickly and consistently
Marking criteria and practice is clear, fair and transparent
Assessment processes are effective and efficient and activities are automated as far as possible
There is single mark entry and a single authoritative data source for marks and
assessment data
Systems and processes should be intuitive and guide students, academic and professional
services staff through the entire assessment cycle
Lifecycle adapted from, and attributed to,Manchester Metropolitan University
http://www.celt.mmu.ac.uk/assessment/index.php
8
Timescale for implementation
Academic Year 2015/16Formal introduction of the Assessment Design and
Delivery Framework across the University: this will be implemented in advance of any formal policy on online
submission and feedback to enable staff to align assessment practice with the framework's guidance
and move towards managing assessment online.
Academic Year 2016/17
Implement online management of assessment including submission and feedback to levels 4 and 7
(plus existing/early adopters for levels 5 and 6).
Academic Year 2017/18
Implement to all other levels.
• Implementation strategy considers any changes in policy needed to deliver online submission and feedback
• Policies to be developed in tandem with changes to practice, delivery and management of assessment – ‘pathway to policy’
9
Activity 1: vision and lifecycle
• In small groups, discuss how the vision and lifecycle reflects the assessment process at your institution: (10 mins)
1. is there anything in our vision and lifecycle that is different from your process?
2. is there anything in your process that is missing from our vision and lifecycle?
• Share any comments or thoughts(5 mins)
10
Technical model
• SITS and Blackboard-based architecture. Additional reporting elements via the data warehouse
• Incremental developments
• Work is also dependent on more detailed data being set up in SITS
• Main functions of the model will be developed in partnership with students and staff and will consider practice, policy, process and organisational design
11
Technical model: key developments
• Assessment portal:– students should be able to see upcoming tasks and results of completed tasks in a single view
• Data into SITS:– determine data required to update systems and automate processes
– auto-creation of Blackboard sites and assignment submission points. What about extensions management, referrals and deferrals?
• Integrated text-matching:– all text-based summative submissions to be automatically checked for originality
• Receipting:– online submissions receipted using Advanced Receipting Building Block
– but for offline?
12
Technical model: key developments
• Print to mark– service provided for academic staff who do not wish to read student work on-screen
• Batch upload– tool to bulk upload electronic feedback files to the Blackboard Grade Centre
• Mark transfer– marks inputted into the Blackboard Grade Centre are automatically transferred to SITS
• Reassessment– transfer of information from SITS to create a new submission point in Blackboard for
affected students
13
Activity 2: key developments
• Consider the key development from the technical model given to your table
• Discuss and describe: (15 mins)
1. The requirements
2. The benefits
3. The risks or issues
4. Any decisions to be taken
• Share the main requirements, and one key benefit, risk and decision (10 mins)
14
Activity 2: key developments
• Assessment portal:– students should be able to see upcoming tasks and results of completed tasks in a single
view
• Recording offline submissions:– confirmation process that work submitted offline has been received
• Print to mark:– service provided for academic staff who do not wish to read student work on-screen
• Batch upload:– tool to bulk upload electronic feedback files to the Blackboard Grade Centre
15
Benefits for students
• Consistent assessment experience between modules
• Improved visibility of upcoming assessments and electronic deadlinereminders
• Clear guidance and framework to provide understanding of, andconfidence in, the assessment process
• Reduced printing and travel costs
• Eliminates dual submissions
• Access to marks and feedback from a single location
• Automatic receipting of online submission of assessments
• Work is stored securely and backed-up
• Improved clarity and understanding of feedback
• Meeting student expectations – everything online
16
Benefits for academic staff
• Assessment guidance to support assessment design anddelivery across modules, courses and programmes
• Principles, guidance and tools to enable innovation of practice
• Automated Blackboard site set up and creation of assignmentsubmission points
• Immediate access to submitted assessments
• Convenience of not having to collect or carry large quantities of paper
• Convenience of electronic filing and security of having work backed up online
• Ability to moderate marks online
• Being able to perform text-matching checks in the same environment as marking
• Provision of appropriate equipment and technologies to support electronic marking:– Marking Practices and Preferences Survey
17
Benefits for professional services staff
• Reduction in managing, receipting, sorting and returningphysical submissions
• Reduction in student queries resulting from clear andaccessible assessment guidance and framework, andconfidence automatic receipting of assessment submissions
• Single mark entry via Blackboard reduces time, and potentialerror, in entering marks into the student record system (SITS)
18
Activity 3: implications for your role
• Considering the vision and lifecycle, the technical model, and the benefits ...
... what would be the implication of implementing online management of assessment for your role(5 mins)
• Discuss with your table ... (5 mins)
• ... and for each role identified on your table share the key implication with the wider group (5 mins)
19
Communication and engagement
• Our attendance at departmental, subject group/course meetings ...• ... their attendance at standing meetings and groups• Regular newsletter• Contribution via faculty programme and project
contacts• Engagement in topic-specific workshops and
discussion papers• Student focus groups• Academic champions and case study contributions• Marking practices and preferences survey
