designing work integrated assessment- tools & techniques for creating 'authentic'...
DESCRIPTION
Copy of a keynote/workshop I gave at Queen's University Belfast on 26th of June, 2014TRANSCRIPT
Designing work-integrated assessment: tools & techniques for creating 'authentic' assessments
Running order
The Collaborate project (20 mins)Designing a Work-integrated Assessment (Group activity - 15 mins)
Tea break (15 mins)
Affordances: What are they? (20 mins)Aligning digital technologies (Group activity - 10 mins)
Evaluating a redesigned assessment (10 mins)What can we do? (Group activity - 10 mins)
Where next / final questions (5 mins)
"Bringing together staff, students and employers to create
employability focused assessments enhanced by technology“
July 2011 - July 2014
JISC Institutional Change, Assessment & Feedback Strand
The Collaborate Project
Key threads, key questions
Assessment
Can current forms of assessment close the gap?
Technology
What role can technology play to support this?
Employability
Is there a gap in current provision?
Where to find answers?
Literature Interviews Analysis
Employability is potentially the biggest force driving people to
University ...
Amongst the foremost reasons for those 22+ considering a
university education (Pollard et al, 2008)
Almost 80% of students stated improving job opportunities
was the most important reason to go to university (CBI/NUS,
2011)
Driven by Employability
Wilson Review (2012) suggested a gap between business
and higher education.
One recommendation of many:
“Universities should reflect on the opportunities that are
provided for students to develop employability skills through
the formal learning methodologies used within the university”
(Wilson, 2012, p.10)
Driven by Employability
Career Zone
Many Employability Initiatives
Enterprise and self-employment
CVs, applications, interviews
Graduate jobs
Part-time jobs
Skills training and events
Volunteering
Placements
The Exeter Award
Global employability
Internships
Career research
Embedded Curriculum Approach
Curriculum EmploymentEmployability
Internship
Key skills training
Work experience
module
New forms of assessment
Assessment Drives Behaviour
What students attend to, how much work they do and how they go
about their studying is completely dominated by assessment
(Gibbs, 2004)
But can exams really prepare
students for the office?
Other forms of assessment
exist which may be better
suited ... e.g. ‘authentic’ assessment
Why Authentic Assessment?
“The types of assessment we currently use do not promote
conceptual understanding and do not encourage a deep
approach to learning” [encourage] “strategic or mechanical
approach”
Newstead (2002, p. 72)
Newstead, S. (2002). Examining the examiners: Why are we so bad at assessing students. Psychology Learning and
Teaching, 2(2):70-75.
Authentic Assessment
Originated in US K-12 system in the early 1990’s, focused on real
world problems in real world contexts.
“an assessment requiring students to use the same competencies,
or combinations of knowledge, skills, and attitudes, that they need
to apply … in professional life”
J. T. M. Gulikers, et al. (2004). `A five-dimensional framework for authentic assessment'. Educational
Technology Research and Development 52(3).
Authentic Assessment
Can be challenging in higher education - 'authentic' has multiple
meanings:
“the term inappropriately implies that some assessment
approaches are superior to others because they employ tasks that
are more ‘genuine’ or ‘real’”
J. Terwilliger (1997). `Semantics, Psychometrics, and Assessment Reform: A Close Look at "Authentic"
Assessments'. Educational Researcher 26(8):24+.
Work-Integrated Assessment
Combines concepts of authentic assessment with local research.
Defined by six 'dimensions':
‘Real World’ Problem / DataSet an overall real world
problem, supported by real world data
Collaborative WorkingCreate teams of students from
the outset, encourage collaboration
Varied AudiencesAim to set explicit audiences for each assessment point
Multiple Assessment PointsMove to a more distributed
pattern of assessment; consider introducing ‘surprise’
points
Light StructureLightly structure the overall assessment; reward student
approaches
Peer / Self ReviewInclude peer and/or self review
explicitly in the assessment process
Visualising dimensions
1 Analyse
2 Design
3 Evaluate
PSY1301: Professional Issues and Development
Original Assessment
75% written examination (1½
hours)
25% oral presentation
(undertaken in pairs, 10
minutes each)
Redesigned Assessment
40% written project: 750 word
patient information leaflet in
groups of up to 6 students
20% oral presentation
(undertaken in groups)
40% MCQ
Assessment
Problem / Data: Creation of a patient information leaflet
Time: Across the module, depending in part on the lectures
Audience: Two externals as new audiences / markers
Collaboration: Students in assigned teams of six
Review: Team working for informal review
Structure: Signposts to relevant guidance
1 Analyse
2 Design
3 Evaluate
Small groups of 3-4Dummy/own assessment
Analyse & redesign
Discussion
How was the model useful in supporting thinking about the assessment?
Could you see yourself using this in your own context?
What would you change about the model?
Affordance Theory
What do these mean to you?
Affordance: Gibson’s definition
“The affordances of the environment are what it offers the animal,
what it provides or furnishes, either for good or ill ... It implies the
complementarity of the animal and the environment.”
Gibson (1979, p. 127)
Gibson, J.J. (1979) The Ecological Approach to Visual Perception, Houghton Mifflin Harcourt (HMH).
Gibson vs Norman: Wikipedia’s version
“An affordance is a quality of an object, or an environment, which
allows an individual to perform an action. For example, a knob
affords twisting, and perhaps pushing, while a cord affords pulling.
...
Psychologist James J. Gibson originally introduced the term ...
defined affordances as all ‘action possibilities’ latent in the
environment.”
Integrating Technologies
We align our learning outcomes, teaching methods and assessments - but what about technologies?
Can Affordances theory help?
“The affordances of the environment are ... what it provides or furnishes” Gibson (1979, p. 127)
What does Blogger provide?
What does LinkedIn provide?
What does Skype provide?
Rating technologies against dimensions
e.g. Time Question: Does the technology provide the learner with a
framework for constructing and managing multiple contributions
extended over time?
e.g. Collaboration Question: Does the technology provide social
presence in ways which allow multiple learners to work together
digitally, sharing ideas and content creation?
“Tech Trumps”
How do we use technology?
"There is no shortage of technology, only a shortage of the educational vision necessary to use the technology to create new educational environments"
Berge and Collins (1995, p. 5)
“I just need to know what programmes do what .... they could be very useful to me.”
Politics undergraduate studentBerge, Z. L., & Collins, M. P. (Eds.) (1995). Computer Mediated Communication and the Online Classroom Vol. III. Distance learning. Cresskill, NJ.: Hampton Press.
PSY1301 "Tech Trumps"
Technology demonstrations
Normal scheduled lecture
Brief description of project background
10 minute overview of potentially useful technologies
The iTest
What is the iTest?
Thirty categorised
questions/statements
I make use of digital technology that few
other colleagues are using
I tend to avoid digital technology as I do not
trust it to work as it should
I can apply technology in novel ways to
meet specific needs
Students have the hardware, but can they use it in an
academic context?
High expectations for students coming to university to be
digitally literate ... not always the case
Is there a digital divide that needs to be managed?
Kennedy, G., Judd, T., Dalgarno, B., and Waycott, J. (2010). Beyond natives and immigrants: exploring types of net
generation students. Journal of Computer Assisted Learning, 26(5):332-343.
Where does the iTest fit?
Back in small groupsRedesigned assessmentNow align technologies
http://goo.gl/HA0Y05
Discussion
Could you see technologies that might suit your design?
What kind of thinking did the tech trumps provoke?
What's missing?
Evaluation package: student centered evaluations
"Mark on the Line" activity
"Mark on the Line" activity
Open answer postcards
Open answer postcards
Drag and drop skills evaluation
Drag and drop skills evaluation
Group activity
http://goo.gl/vRQo8M
Back in small groupsSkill cards drag and drop
What can you do?
Contact:
Richard Osborne
Education Advisor (Technology Enhanced Learning)
University of Exeter
01392 72 4951
Questions?
Collaborate websitehttp://www.exeter.ac.uk/projects/collaborate/