formative e-assessment developers day
DESCRIPTION
TRANSCRIPT
Formative e-Assessment PED 5, Dec 2008
Scoping a vision for formative e-assessment:
Cases and design patterns
Formative e-Assessment PED 5, Dec 2008
Learners?
Problem I: common language
DesignKnowledge
in TEL
Developers
Policy makers
Teachers
Researchers
Formative e-Assessment PED 5, Dec 2008
Problem II: The void
The Prophetswill tell you what should be done
The Explorerswill tell you what they did
Current discussion of learning and technology alternates between the abstract theoretical and the anecdotal.In between there is a shortage of design-level discourse.
?
Formative e-Assessment PED 5, Dec 2008
Wanted: a design science of learning
A science of design has - A value dimension A functional axis of decomposition Attention to representation
(Mor & Winters, 2007)
Herbert Simon (1969): we need a scientific study of the man-made. At its core, the science of design. “everyone designs who devises courses of action aimed at changing existing situations into desired ones”
Formative e-Assessment PED 5, Dec 20085
The Design Knowledge Problem
Expert := one who solves problems in a
particular domain
Expert := has domain design knowledge
Experts do,Consultants talk
Experts talk in jargon
But..
Formative e-Assessment PED 5, Dec 2008
Solution: sharing stories
Stories (narratives) are a fundamental form of generating / sharing knowledge. (Bruner)
Thick descriptions of problems & solutions. Everyone likes a good story.
http://www.slideshare.net/yish/case-study-how-to-presentation
Formative e-Assessment PED 5, Dec 2008
But..
Narratives are not enough: The Aha! Factor
How do we identify the key design element in a story?
The fantasy factor How do we know its true?
The (cognitive) load factor The world is changing too fast for us to take in all
the good stories.
Formative e-Assessment PED 5, Dec 20088
Design patterns
[describe] a problem which occurs over and over again in our environment, and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it the same way twice(Alexander et al., 1977)
C o n t e x t
Problem Solution
Formative e-Assessment PED 5, Dec 20089
ProblemKeep the rain out
ContextCold, wet, poor.
Method of solutionThatched roof
RelatedTimber frame, Slanted roof,Chimney
Formative e-Assessment PED 5, Dec 200810
example: activity nodes
Design problemCommunity facilities scattered individually through the city do nothing for the life of the city.
Design solutionCreate nodes of activity throughout the community, spread about 300 yards apart.
http://www.uni-weimar.de/architektur/InfAR/lehre/Entwurf/Patterns/030/ca_030.html
Formative e-Assessment PED 5, Dec 2008
Scenarios
The ultimate proof of a pattern language is in its effectiveness as a tool for design.
Ask participants to tell as “fantasy story”: a current design challenge as an “I wish” case story.
Apply patterns to derive solution.
Formative e-Assessment PED 5, Dec 2008
A few cases
Creature of the week CoMo Post 16 String Comparison Open Mentor Academic writing Audio files CLICK ...
Formative e-Assessment PED 5, Dec 2008
Creature of the week Situation
large class (138) first and second year computer science students. assignment: create a virtual pet in Second Life.
Task Engage and motivate the students show examples of good work which others could
learn from show students their work is valued. build a sense of community.
Formative e-Assessment PED 5, Dec 2008
Formative e-Assessment PED 5, Dec 2008
CoMo
Situation Royal Vet College. Hospital rotations as part of their training.
Task Allow students to capture critical incidents in text
and image. Support sharing of clinical experiences and co-
reflection.
Formative e-Assessment PED 5, Dec 2008
Formative e-Assessment PED 5, Dec 2008
Post 16 string comparison
Situation Grammar school been piloting the ‘string
comparison’ approach to language teaching at post-16 for AS and A2 level students.
Sixth Form level, grammatical consolidation and whole-sentence translation.
Task Allow students to practise written language
independently and receive feedback on errors in order to improve their language skills.
Formative e-Assessment PED 5, Dec 2008
Solution
A bespoke string (sequence) comparator was designed; uses fine-granularity sequence comparison to compare correct language strings to a user’s answer. Students answer questions and the comparator marks up errors in their input using colour coding (and font style) to highlight the different types of error. If an answer contains errors the student is given a second attempt in which to correct the submission based on the feedback received.
Formative e-Assessment PED 5, Dec 2008
Open mentor
Formative e-Assessment PED 5, Dec 2008
A few patterns..
Try Once, Refine Once Feedback on Feedback Classroom display Use my Stuff Round and Deep Showcase Learning Three Hats
Formative e-Assessment PED 5, Dec 2008
where the learner is going where the learner is How to get there
teacher
PeerActivating students as learning resources for one another
LearnerActivating students as owners of their learning
Try once refine once; Three hats; Narrative spaces (***)
Clarify and share learning
intentions
Engineering effective
discussions, tasks and
activities that elicit
evidence of learning
Providing feedback that
moves learners forward
Three hats; Narrative spaces (*)
Feedback on Feedback; Soft Scaffolding
Understand and share
learning intentions
Round and deep; Classroom display; Use my stuff; Showcase learning; Three hats; Objects to talk with;
Narrative spaces (*)
Understanding learning
intentions
Narrative spaces
Wear your skills
Formative e-Assessment PED 5, Dec 2008
Try Once, Refine Once
http://patternlanguagenetwork.myxwiki.org/xwiki/bin/view/Patterns/TryOnceRefineOnce
Formative e-Assessment PED 5, Dec 2008
Problem
Lack of immediate feedback for students leads to fossilisation of errors and misconceptions
providing immediate feedback in an iterative fashion can also hinder effective learning since students are able to "grope their way" step-by-step to a correct solution without necessarily having to think about each answer as a whole.
Formative e-Assessment PED 5, Dec 2008
Context
Class size Large (30-300)
Content Skills \ facts
Mode of instruction Blended / on-line. Computer tested.
Formative e-Assessment PED 5, Dec 2008
Solution
Formative e-Assessment PED 5, Dec 2008
Feedback on
Feedback
http://patternlanguagenetwork.myxwiki.org/xwiki/bin/view/Patterns/FeedbackonFeedback
Formative e-Assessment PED 5, Dec 2008
ProblemGood feedback should -
Alert learners to their weaknesses.Diagnose the causes and dynamics of these.Include operational suggestions to improve the learning experience.Address socio-emotive factors.
Tutors know this, but are pressed for time. Or not aware of their feedback strategies
Large teaching organisations are not equipped to provide tutors with personal feedback on their teaching
Formative e-Assessment PED 5, Dec 2008
Context
Large scale, technology supported, graded courses many tutors instructing many students.
Feedback is mediated by technology that allows it to be captured and processed in real time
Topic of study is subject to both grading and formative feedback.
Formative e-Assessment PED 5, Dec 2008
Solution
Embed a mechanism in the learning and teaching system that regularly captures tutor feedback, analyses it, and presents them with graphical representation of the types of feedback they have given. Ideally, this should also include constructive advice as to how to shift from less to more effective forms.
In computer supported environments (e.g. VLEs), this mechanism could be integrated into the system, providing tutors with immediate analysis of their feedback, as well as long-term aggregates.
Formative e-Assessment PED 5, Dec 2008
Classroom Display
http://patternlanguagenetwork.myxwiki.org/xwiki/bin/view/Patterns/Classroomdisplay
Formative e-Assessment PED 5, Dec 2008
Problem
Rewards participation.Relates to learner's personal experiences.Window on student conceptions.
Using learner generated Using learner generated content..content..
Needs to collate works in a single easy to access location.Learners uncomfortable about presenting their work in publicLegal or other restrictions on sharing work.
Formative e-Assessment PED 5, Dec 2008
Context
Class size: Small / medium (6-60)
Mode of instruction: Blended (preferable)
Time frame Continuous, over a period
Pedagogy Involves construction / media production
Formative e-Assessment PED 5, Dec 2008
Solution
Formative e-Assessment PED 5, Dec 2008
Task 1: highlight concepts
Hazard: Hazard: needs definition
Asset: Asset: Key common concept
Formative e-Assessment PED 5, Dec 2008
Task 2: design game – add formative assessment to e-Learning
Agree on a scenario (who, what, where) Each team member takes on a role (teacher,
student, mgmt, admin) State desires Find appropriate patterns Propose solution
Formative e-Assessment PED 5, Dec 2008
A scenario is..
Preferably, a real problem you are familiar with. e.g. extending a system you have in development /
in use Define the context of use
Environmental constraints Define the primary roles and workflows
What is the problem to be solved?
Formative e-Assessment PED 5, Dec 2008
Coming up..
Formative e-Assessment PED 5, Dec 2008
Deadline: December 23rd 2008
http://www.iwm-kmrc.de/workshops/e-learning-patterns/
Formative e-Assessment PED 5, Dec 2008
Review on formative e-Assessment Paper on the CoMo case study Symposium on pattern methodologies
http://www.cal-conference.elsevier.com/
Formative e-Assessment PED 5, Dec 2008
July 8-12, 2009, Irsee Monastery, Bavaria
Deadline: February 14, 2009
http://www.hillside.net/europlop/
http://flickr.com/photos/ademaraguiar/