assessing co-curricular learning robert mundhenk visiting scholar the higher learning commission
TRANSCRIPT
![Page 1: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/1.jpg)
Assessing Co-curricularLearning
Robert MundhenkVisiting Scholar
The Higher Learning Commission
![Page 2: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/2.jpg)
Traditional Approaches to Assessment of Learning
Ask Institutional Research about graduation, retention, GPAs, and the like
Ask faculty about their teaching and the learning it produces—but not necessarily whether they know they’re producing learning
Ask faculty how they know they’re producing learning
Rescheduled for Jan. 19 at 3:30 – 5:00 p.m. YU 204•Co-curricular Assessment workshop presented by Bob Mundhenk Yellowjacket Union 203 While most institutions have devoted effort and resources to the effectiveness of the co-curricular services and programs they offer, only recently have they started to assess the learning these areas produce. This session will explore ways to assess the impact of co-curricular efforts on student learning and development.Targeted Audience: Faculty and staff in non-academic units interested in assessment.
![Page 3: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/3.jpg)
The Institutional Mismatch
Traditional wisdom: Learning outcomes need to be aligned at course, program, and institutional levels
But where are many general education goals, like “tolerance” and “teamwork” and “the ability to function in an increasingly diverse world” and “inclination” taught and assessed?
Or is “taught” the right word???
![Page 4: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/4.jpg)
Shifting Perspectives
What happens if we substitute the word “learned” for the word “taught”?
What are the implications of “Where are learning outcomes learned and assessed”?
Emphasis on student demonstration, not topic-covering
Ability to do or apply supersedes knowing Responsibility for learning is shared Site of learning becomes less specific, and
boundaries become more fungible
![Page 5: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/5.jpg)
After Learning Reconsidered
“Learning” is not exclusively classroom-basedMany valued outcomes are not taught
exclusively in the classroomMany valued outcomes are the result of
processes outside the classroom “Learning” is a process based on three
interdependent student experiences: Understanding academic content and processes Student development Identity formation
![Page 6: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/6.jpg)
After Learning Reconsidered
Responsibility for “learning” exists outside the classroom
Responsibility for “learning” doesn’t always take the same form; some entities on campus produce it, some facilitate it, some support it
Responsibility for assessing learning exists outside the classroom as well
![Page 7: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/7.jpg)
Some Post-LR Examples of Learning
Civic Responsibility AA: Service learning SA: Student government, voter registration,
student judicial boards
Think and Engage as a Global Citizen AA: Language courses, Anthropology, Sociology SA: International experiences, culture days,
residence halls
![Page 8: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/8.jpg)
UW- Superior’s Five Institutional Goals
Ability and inclination to:◦Think and make connections across disciplines◦Express oneself in multiple forms◦Analyze and reflect upon multiple perspectives
to arrive at a perspective of one’s own◦Think and engage as a global citizen◦Engage in evidence-based problem-solving
![Page 9: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/9.jpg)
Ability and Inclination
How is ability made evident?Where and how is ability developed?Where is ability assessed?
What is “inclination”?How is it made evident?Where and how is it developed?Where is it assessed?
![Page 10: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/10.jpg)
How Do These Outcomes Connect with Undergraduate Learning Outcomes?
Campus Life learning outcomes:◦Recognizing and enhancing leadership potential◦Developing an appreciation of human
differences◦Seeking opportunities to engage in the campus
community and beyond◦Expanding desire for life-long learning
![Page 11: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/11.jpg)
What Ability or Inclination Do They Develop?
Campus Life learning outcomes:◦Recognizing and enhancing leadership potential◦Developing an appreciation of human
differences◦Seeking opportunities to engage in the campus
community and beyond◦Expanding desire for life-long learning
![Page 12: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/12.jpg)
How Do We Know They’re Achieved?
Campus Life learning outcomes:◦Recognizing and enhancing leadership potential◦Developing an appreciation of human
differences◦Seeking opportunities to engage in the campus
community and beyond◦Expanding desire for life-long learning
![Page 13: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/13.jpg)
Co-curricular Outcomes
Need to be intentionalNeed to be planned Need to be part of the structure of a
student’s experienceNeed to be assessed
![Page 14: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/14.jpg)
BEING INTENTIONAL
![Page 15: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/15.jpg)
Traditional Co-curricular Assessment
Efficiency models: Focus on process How well is this office/service functioning? Focus on numbers:
Clients served Graduation rates Tutorial visits Attendance at activities Student/staff ratios
![Page 16: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/16.jpg)
Newer Co-curricular Assessment
Effectiveness Models: Indirect◦Based on surveys and other indirect indicators,
like NSSE◦Often rely on student self-reporting◦Tend to skew positively on outcomes, if not
always on the processes that led to them
![Page 17: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/17.jpg)
Newer Co-curricular Assessment
Effectiveness Models: Direct◦Focuses on student performance◦Can be based on observation or objective
measures◦Require carefully designed and consistent
measuring practices
![Page 18: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/18.jpg)
How to Assess Co-curricular Learning
Apply external standards, like CAS
Use surveys and questionnaires
Develop direct measurement strategies
All of the above
![Page 19: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/19.jpg)
CAS Standards
Standards for 40 functional areasThirteen component parts:
Mission Program Leadership Organization and management Human resources Financial resources Facilities, technology, and equipment Legal responsibilities Equity and access Campus and community relations Diversity Ethics Assessment and evaluation
![Page 20: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/20.jpg)
CAS’s Six Outcome Domains
Knowledge acquisition, construction, integration, and application
Cognitive complexityIntrapersonal DevelopmentInterpersonal competenceHumanitarianism/Civic EngagementPractical Competence
![Page 21: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/21.jpg)
Using CAS Standards
Can easily document the efficiency of processes and organization
Can be used as well (through an emphasis on the domains) to chart the effectiveness of outcome development efforts—depending on outcome definition and quality of evidence
![Page 22: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/22.jpg)
Outcome Definition
Career Services: As a result of interactions with the Career Services Office, students and alumni will:
Identify their skills, abilities, and strengths in order to make knowledgeable career decisions
Have the necessary resources and skills to prepare for life-long post-graduate experiences
How are these outcomes connected to institutional learning outcomes?
What are “resources and skills”?
![Page 23: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/23.jpg)
Outcome Definition
College Unions: As a result of experiences in the Yellowjacket Union, students will:◦ Identify and utilize the opportunities and services
available to them◦Demonstrate a sense of ownership for the campus
community and civic involvement◦ Interact with and value individuals from diverse
backgrounds and lifestylesHow do you know you’ve achieved the second
and third outcomes?How do they overlap with institutional
outcomes?
![Page 24: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/24.jpg)
Outcome Definition and Evidence
Clearly defined outcomes help determine the nature of evidence to be collected
Clearly defined outcomes focus on student performance and development, thus calling for both direct and indirect forms of evidence
![Page 25: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/25.jpg)
Direct and Indirect Measures
DIRECT: uses performance or product, created by students, that can be compared to expected outcomes--Capstone courses, projects, internships, clinical experiences, etc.
INDIRECT: uses information that does not directly link the learning to the outcomes--graduation rates, grades, surveys, “usage” rates
![Page 26: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/26.jpg)
Questions about Evidence
Is it relevant to the area’s stated mission and function?
Does it measure what we want it to measure?
Does it deal in some way with outcomes?Is the information derived useful?Can the information be used to improve
either function or learning?
![Page 27: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/27.jpg)
Surveys and Questionnaires: Some Sample Questions
Did you accomplish what you hoped to accomplish in your meeting with your advisor?
How well did your experience at X prepare you for employment?
As a result of this First-Year program, do you feel better prepared for college?
Write a short essay in which you describe the ways in which your attitudes and values have changed as a result of your semester in Argentina.
![Page 28: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/28.jpg)
Surveys and Questionnaires: Indirect Evidence
Traditional, indirect source of information on effectiveness
Limitations: Self-reporting Unvalidated opinion Response rates Opportunistic data Skewed samples
![Page 29: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/29.jpg)
Surveys and Questionnaires
Kinds: Satisfaction Reflective Post-experience experience (alumni and
employers)Value:
True “customer” response Can indicate areas for improvement and
ratification Provides data for planning and review
![Page 30: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/30.jpg)
Surveys and Questionnaires
Making them tools to assess learning: Use learning outcomes as basis for at least some
questions Validate by cross-referencing outcomes with
different populations (employers, alumni, graduate, current students)
Emphasize the learning outcomes in design and analysis of surveys and questionnaires
![Page 31: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/31.jpg)
Direct Effectiveness Measures
If Learning Reconsidered made the case for cross-campus responsibility for learning, then assessment of learning outcomes is also a cross-campus responsibility
Adaptation of practices and devices already in use in academic settings
Standardized Judgment-based
![Page 32: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/32.jpg)
Developing Direct Measures of Effectiveness
Intentional Planning:◦Determine areas of responsibility: what
office/function might be a logical place to contribute to particular learning outcomes?
◦Plan the outcome-based purpose of the activity◦Aim at the appropriate level of Bloom’s taxonomy◦Design non-passive activities (watching a film plus
discussion; International Days as more than food, costumes, and dance)
◦Design outcome-focused opportunities for processing
![Page 33: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/33.jpg)
Understanding the Outcomes
What is “inclination” and how is it developed in co-curricular activities?
How does a student “express oneself in multiple forms”?
What is a “global citizen”?What does it mean to “make connections
across academic disciplines” and how might co-curricular activities have a role in developing this outcome?
![Page 34: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/34.jpg)
Being Intentional
Choose one of the outcomes below and determine how your co-curricular area might have some responsibility for developing it. Name specific activities that might help develop the outcome and specify what their effect on the student should be.
Think and make connections across academic disciplines Express oneself in multiple forms Analyze and reflect upon multiple perspectives to arrive at a
perspective of one’s own Think and engage as a global citizen Engage in evidence-based problem-solving
![Page 35: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/35.jpg)
Planning for the Long Term
Mapping:◦If the learning outcome is important, single
exposure isn’t enough◦How do first-year experiences differ from last-
year ones—or what difference is expected in student response?
◦How to assure student’s development of outcomes from first year to last?
![Page 36: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/36.jpg)
Regular Assessment
One-shot assessment produces haphazard results that are usually insufficient for planning improvement
Tie assessments to logical stages of development, based on an outcome map
Be consistent in approach to assessingOptions:
Standardized instruments Self-generated tools
![Page 37: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/37.jpg)
Self-generated Tools
ObservationsExpert judgmentsStudent self-reflectionEmployer/supervisor judgments
![Page 38: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/38.jpg)
Using Self-Generated Tools
Consistency across observers is crucial, so a rubric of some kind is essential
Holistic rubrics: broad judgments (Acceptable/Not Acceptable/Needs Improvement)
Descriptive rubrics: defined criteria and measures
![Page 39: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/39.jpg)
Descriptive Rubrics
First, determine the aspects of student performance that would indicate he/she has achieved an outcome (e.g., one aspect of a “social justice” outcome might be “the student’s writing demonstrates sensitivity to issues of class and power”)
Second, define the specific things a student would have to do to show he/she has mastered that aspect (e.g., “Clear understanding of the ways in which economic status affects behavior.” (Criteria)
Finally, describe degrees of achievement for each criterion (Measures)
![Page 40: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/40.jpg)
Developing a Rubric
Using the outcome and the functional area you chose earlier, develop a rubric to measure student achievement, defining one performative aspect of the outcome, one criterion for measuring that aspect, and a system (holistic, descriptive, whatever) for describing degrees of attainment
![Page 41: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/41.jpg)
Institutional Assessment
Assessments done across campus should ideally use the same rubrics or measures
When using the same tools is not possible, it is essential that there be a way to extract information that is usable at the institutional level while still serving the needs of the functional area
![Page 42: Assessing Co-curricular Learning Robert Mundhenk Visiting Scholar The Higher Learning Commission](https://reader033.vdocuments.site/reader033/viewer/2022051618/56649cf15503460f949c0d8f/html5/thumbnails/42.jpg)
To conclude:◦Understand the meaning of the desired
outcomes and your role in developing them◦Separate efficiency from effectiveness◦Plan experiences and assessments carefully◦Focus on using assessment information to
improve learning, not to justify your existence◦Collect information that is relevant, meaningful,
and useful◦Design systems that are reasonable and
manageable