innovation in action: a simpler path for meaningful assessment … · 2018. 1. 22. · a simpler...
TRANSCRIPT
assessment ∙ accreditation
e-portfolios
Innovation in Action: A Simpler Path for Meaningful Assessment Data
Ben Coulter, Ed.D.Senior Director, Campus Solutions Taskstream
Outcomes
• Understand the goals and scope of the Multi-State Collaborative (MSC) to Advance Learning Outcomes Assessment
• Understand the benefits and experiences for participants
• Appreciate how simple technology can reduce barriers to doing direct assessment of student learning outcomes (SLO’s)
• Consider how the technology could be used on your campuses to engage faculty and start or improve assessment initiatives
• Founded in 2000
• 500+ client institutions
• Proven, reliable technology
• Experienced, caring team
• Dedicated to client success
Taskstream advances effective assessment to improve student learning and institutional quality.
AMS
Assessment Planning & Documentation
LAT
ePortfolio-Based Assessment
Aqua
Streamlined Direct Assessment
The Multi-State Collaborative to
Advance Learning Outcomes
Assessment
MSCGates-funded project to create a scalable model to assess Essential Learning Outcomes based on authentic student work using VALUE rubrics
VALUE Rubrics
• ACC&U initiative in 2007-2009
• Faculty and other professionals from over 100 higher education institutions developed common rubrics
• 16 Essential Learning Outcomes
• Available for institutional use
Inquiry and Analysis
Critical Thinking
Creative Thinking
Written Communication
Oral Communication
Reading
Quantitative Literacy
Information Literacy
Teamwork
Problem Solving
Civic Engagement
Intercultural Knowledge
Ethical Reasoning
Lifelong Learning
Global Learning
Integrative Learning
The MSC Challenge: Scaling Direct Assessment
• Most participants were first first-time users of Taskstream
• Each had a specific role and goals
• All with day jobs and other responsibilities
• Limited opportunity to train
• Limited time to complete their work
• Focused on one activity at a time
PROJECTLEADS
DATALEADS
FACULTYSCORERS
The Technical Challenge: Engaging Distributed Users
We asked the question:Why is it so difficult for so many institutions
to engage in meaningful, sustainable assessment of student learning?
Why is it so difficult for many institutions to engage in meaningful, sustainable
assessment of student learning?
motivation
outcomes
rubrics
exams
papers
portfolios
surveys
calibration
reliability
gradesculture
politics
reputation accreditation status
mission
governance
improvement
students
faculty
administrators
fearvalue
abilitytime
funding training
identity
recognition
trustautonomy
leadership
Outcomes assessment is complex.
And…
Software doesn’t do assessment.People do assessment.
If people aren’t engaged in the work, software isn’t going to help.
We asked another question:Why is it so difficult for so many institutions
to engage in meaningful, sustainable assessment of student learning?
How easy can we make this?
Visual tracking reports to show progress of collection and scoring
PROJECTLEADS
DATALEADS
FACULTYSCORERS
Engaging the MSC users: How we did it
use space for screenshots or other large images
Track progress of collection across the project and by statePROJECTLEADS
use space for screenshots or other large images
Track progress of scoring overall and by outcomePROJECTLEADS
Visual tracking reports to show progress of collection and scoring
PROJECTLEADS
DATALEADS
FACULTYSCORERS
Engaging the MSC users: How we did it
Simple tools for tagging assignments and bulk uploading artifacts
Create and tag assignments with key metadataDATALEADS
Bulk upload student artifacts (papers)DATALEADS
Visual tracking reports to show progress of collection and scoring
PROJECTLEADS
DATALEADS
FACULTYSCORERS
Engaging the MSC users: How we did it
Basic tools for tagging assignments and bulk uploading artifacts
Simple, intuitive and tablet-friendly evaluation experience to make scoring intuitive
A simple queue of workFACULTYSCORERS
Easy interaction with the contentFACULTYSCORERS
Quick access to the rubricFACULTYSCORERS
Clear way to review scoresFACULTYSCORERS
Minimal User Training Needed
• Demonstrated the “VALUE Rubric Scoring System” to MSC participants
• Training was a 30-minute demo and 15 minutes of questions
How easy did we make it?
“I was impressed by the design of this process. Functions were intuitive, everything was easy to find and use. The interface made
the most of screen real estate for easy reading and referencing of criteria.”
MSC pilot study participant feedback
“Taskstream took any concerns about technology out of the
scoring experience.”
“Taskstream was a useful system that allowed me to assess the student artifacts quickly.”
“I was pleased with the ease of using and dependability of Taskstream.”
“Taskstream was great. It was an easy system to use, understand, and navigate.”
“Taskstream is very user-friendly.”
MSC pilot study participant feedback
Reflections and Evolution
• Improve user experience for uploading student work.
• Present assessment results with visually engaging reports.
• Offer additional ways to bring work into the system.
• Support multiple artifacts per student, assessment of multiple outcomes for each assignment.
• Provide the means for institutions to engage in similar initiatives locally.
Headwaters of Aqua…Why is it so difficult for so many institutions
to engage in meaningful, sustainable assessment of stunt learning?
Developmental TrajectoryWhy is it so difficult for so many institutions
to engage in meaningful, sustainable assessment of stunt learning?
Motivation = Expectancy x Value - Cost
Expectancy-Value Theory (Eccles, 1983)
Motivation = Expectancy x Value - Costan individual’s anticipated ability to successfully accomplish a task
Motivation = Expectancy x Value - Costan individual’s perceived importance for the task
Motivation = Expectancy x Value - Costhow much an individual perceives that he/she has to sacrifice
to accomplish the task
Expectancy x Value – Cost
Motivation
=
A breakthrough response to the complexities ofoutcomes assessment that provides a simpler pathto meaningful assessment and lets you focus moreclearly on improving student learning.
A simple path for direct assessment of learning
Critical Thinking
Written Communication
Intercultural Knowledge
What?Which outcome(s)
will you assess?
Where?Where will you get
the evidence?
course assignments
Who?Who will score
student work?
SimpleSetup
A simple path for direct assessment of learning
What?Outcome(s)
Where?Assignments
Who?Scorers
SimpleSetup
Data!Streamlined
Scoring Experience
Create Projects Simply
Easily create and track your assessment projects, and provide context to faculty about the project goals and the outcomes to be assessed
Manage Multiple Projects
View the status and progress of all of your assessment projects to see where things stand and what actions you need to take
Control How Scoring Will Take Place
Designate evaluators by outcome and distribute student work to calibrated evaluators, with or without student anonymity
Collect Work In The Way That Makes Sense For Your Community
Assemble student work for scoring through a variety of collection methods
Expectancy
Streamlined, intuitive workflows make it clear what you need to do and
how to do it
Value
Clean displays of outcomes data make it easier to focus on results
Cost
User-centered design improves efficiency, saves faculty time
Benefits of Aqua
1. Gets to actionable data more easily
2. Makes the assessment process more efficient, saves time
3. Allows you to start small and grow as it makes sense for your community
4. Offers an intuitive user experience for even the most technologically-challenged users
5. Does not require student interaction with the system
6. Cost/Pricing…
Live Site Walkthrough
assessment ∙ accreditation
e-portfolios
Innovation in Action: A Simpler Path for Meaningful Assessment Data
Ben Coulter, Ed.D.Senior Director, Campus Solutions Taskstream