an adventure in assessment

An Adventure in Assessment Stephen “Gavin” Weiser, M.Edstephen.weiser@gmail.com

A Lil About Me

Gavin Weiser

Residential Experiential Education Facilitator Princeton-Blairstown Center

Recent M.Ed from U. South Carolina

Let’s Hear About You

Find someone from a similar type of program as folks are talking.

Outline

What is Assessment

Purposes of Assessment

Some Pitfalls

Methods

Learning Outcomes - Creation

Tool Selection & Creation

What is assessment?

A. What we do to people we don’t like

B. An old wooden ship

C. Something we all hate and loathe

D. A sustainable process of collecting data to improve upon services & programs

Purposes

To collect data in order to better understand what we are doing

What can this look like?

Pitfalls

“teaching to the test”

The politics

Fatigue – particular to certain methods

Some bad ways to use assessment.

Methods

Qualitative vs. quantities

Mixed Methods

Literature review

Learning Outcomes

Why are they important?

What are they? Measureable & meaningful

Examples: To gain leadership abilities.

Good or bad?

Tool Selection

Based upon what we are trying to measure

Different Types: Surveys Focus Groups Interviews

Structured vs. Unstructured Protocols Field Observation

Create a“Culture of

Assessment”What does this look like?

Vital to it’s and the programs success!

Example Project

Background of Outdoor Recreation at Carolina

Opened Fall 2008, Director: Katie Coley

Information from Katie, annual report, focus groups with current student leaders

Annual Report Strengths Weaknesses

5 year plan

Programs of Outdoor Recreation

Climbing Wall

Clinics

Adventure Trips

Bike Shop

Student Leadership

Outdoor Leadership Training

Focus Groups

8 Students Leaders; 2 groups of 4

Methodology

Provided information based on experiences and goals with outdoor recreation

Information received assisted in guiding what learning outcomes should be created

Previous Assessments in Outdoor Recreation

Office had “learning outcomes” but they were immeasurable

Same tool was used to assess all programs

Tool focused on satisfaction

Purpose of Assessment

New office which allowed for flexibility

To create a process that can be used for future assessments within Outdoor Recreation

Outdoor Recreation programs nationally have a need for quality assessments

Provide feedback on some of the learning outcomes related to the Rock Wall

Methodology

Students that have completed climbing the rock wall

University 101 and walk-up participants

A qualitative assessment was distributed post wall climb

2 week period in November – only one week is included in this analysis

Learning OutcomesStarted from focus group, Katie’s feedback and

goals

Took two goals: Teambuilding and Trust and developed outcomes from them

Created measurable outcomes

U101 Outcomes

Recognize individual’s strengths within a working group.

Understand how to apply strengths in an existing challenge. 

Demonstrate a willingness to rely on others during a challenge.

Participants will learn technical skills related to the activity in which they participate.

Individuals’ Outcomes

Demonstrate an increase in willingness to go beyond limitations and boundaries.

Participants will learn technical skills related to the activity in which they participate.

ToolSelected a qualitative instrument based on:

feasibility, rationale, design, research questions

Technically mixed method

Local instrument – developed by us

Open-ended questionnaire

Challenge: probing but not leading and receiving the information that was necessary

Katie’s feedback to make improvements to the tool Created a second tool

Analysis

Used a coding procedure “Focused coding”

Codes were guided by outcomes and data

U101: coded as whole system

Individual: coded by question

Used SPSS to develop codebook and pie charts for representations

Results

Solid process of developing a survey based on targeted outcomes – Major Result

Most applicable to outcomes will be discussed

Others are informative but not directly tied to outcomes

Individual: Expectations and Challenges

Technical Skills

Cohesieveness

Team Support

Technical SkillsCohesievenessTeam Support

Challenges

Reasons For Visiting

40%

46.70% 41%

5.90%

52.9%

13.3%

Skills Learned

Individual U101

Tech-nical Skills

Co-hes-ieve-ness

Technical SkillsCohesievenessTeam SupportCommunica-tion

U101: How were challenges overcome?

Interesting to note, this came as a surprise

Technical SkillsCohesievenessTeam SupportCommunication

Points of Improvement

Tool did not always provide needed data

Not a large sample size

Difficult to know if technical skills were learned during activity

Time constraints

Strengths

Informed by focus group data

Process for developing outcomes and tool was well documented and could be replicated

Tool design – well-sequenced

Informed support from whole office

Code and analysis were manageable and clear

Recommendations for Future

Use process to develop a comprehensive tool for each activity

Connect outcomes to Campus Recreation outcomes

Use satisfaction data and skills data to publicize the wall, attract non-climbers

Be intentional in educating student participants about outcomes

Expand office staff

Partner, Pair, Share

Find that special someone from earlier

Develop outcomes and the beginnings of a tool.

Thank You!

Contact: Gavin Weiser @ stephen.weiser@gmail.com (609) 458-2151