disability resource center

Culturally Responsive Evaluation: A Tool for Assessing Faculty Understanding of Serving Students with Disabilities

Disability Resource Center

2014 AHEAD Conference

Disability Resource Center

2014 AHEAD ConferenceSacramento, CA

Susan A. Aase, J.D., M.S.Ed.Donna Johnson, M.A., M.S.

Disability Resource Center (DRC)University of Minnesota – Twin Cities

July 18, 2014

Agenda

Introductions/Ground Rules

Learning Objectives

Culturally Responsive Evaluation

Application

Discussion

Wrap Up

Cautionary Tale

Participating in a 90-minute session on evaluation does not make one an expert but…..

Learning Objectives

Participants will learn the differences between Culturally Responsive Evaluation and traditional Western models.

Participants will learn how Culturally Responsive Evaluation may be a useful approach in evaluating faculty understanding of serving students with disabilities using the lens of strengths of the communities served.

Participants will learn practical strategies for conceptualizing, developing, and implementing a component of Culturally Responsive Evaluation that may be implemented by a small office at minimal cost.

Evaluation in the Context of the Office for Equity and Diversity (OED)At the University of Minnesota, the following offices report to OED:

•Business Community Economic Development•Conflict Resolution•Disability Resource Center (DRC)•Diversity in Graduate Education•Equal Opportunity and Affirmative Action•Gay, Lesbian, Bisexual, Transgender, and Ally Programs•Institute for Diversity, Equity, and Advocacy•Multicultural Center for Academic Excellence•Women’s Center

The OED Philosophy

By including disability as an aspect of human diversity, OED views disability through a “cultural lens.”

OED selected Culturally Responsive Evaluation as the methodology that worked for all OED units.

Your Program’s Philosophy

What is your program’s philosophy?

How is disability viewed?

Culturally Responsive Evaluation: What Is It?

Culturally Responsive Evaluation is a collection of practical strategies and frameworks that attend to culture and context when preparing for an evaluation, conducting it, and disseminating and using the results of the study.

(Samuels & Ryan, 2011)

Culturally Responsive Evaluation

Is new paradigm of evaluation

Utilizes cultural strengths in a community

Aligns assessment perspectives and methods with traditional knowledge, values, worldviews, lived experience and ways ofknowing

Is strengths-based and culturally meaningful(White Shield, Justilien & Acharya, 2012)

Western Paradigm Methods

Quantitative: counting, checklists, surveys, pre-post tests, analysis of statistics

Qualitative: case studies, anecdotes, focus groups, observations, analysis of files

(White Shield & Acharya, 2012)

Non-Western Paradigm Methods

Focus on discovery of knowledge already present within the community

Congruent with communities’ worldviews and intrinsic cultural strengths

Rely on research/evaluator’s lived experience to help inform the process

(White Shield & Acharya, 2012)

Non-Western Paradigm Methods

In what other settings might you operate using a Non-Westernparadigm?

Created a Logic Model

A logic model is a tool used most often by program managers to evaluate the effectiveness of a program. Logic models are usually a graphical depiction of the logical relationships between the resources, activities, outputs and outcomes of a program.

What Is a Logic Model?

Developed A Purpose Statement

A purpose of the Disability Resource Center is to facilitate the University of Minnesota’s recognition of disability as a natural part of the human experience.

Purpose Statement

What is a purpose of your disability services?

Explored Cultural Strengths of People with Disabilities

•Resiliency – Ability to adapt to situation

•Flexibility – Ability to do tasks in different ways

•Diversity – People with disabilities are not a monolithic group

•Ability to move along a spectrum of cooperation to get things done

•Ability to tap into unique perspectives on and experiences with defining success and failure

Developed Assumptions (If/Then Statements)

If the Disability Resource Center (DRC) approaches service delivery from a continuous improvement standpoint, then information is sought from its many stakeholders on what it can do to better meet the needs of university students, staff, faculty, and guests with disabilities.

If the DRC considers disability to be an equity issue, then the DRC will collaborate with campus partners to create an accessible campus.

Developed Overarching Outcomes

The expectation is that the Disability Resource Center

will:

gather qualitative and quantitative data to modify our decision-making based on evidence.

• be responsive to the needs of its users in the university community.

assist qualified people with disabilities to fully participate in and contribute to the university community.

Overarching Outcomes

What might your overarching outcomes

be?

Selected the Following Output/Activities

1. Develop Faculty Advisory Committee

2. The Disability Resource Center (DRC) will gather data to modify its decision-making based on evidence. The first priority addressed was the exploration of faculty understanding of the student role, the faculty role, and the DRC’s role.

Selected the Following Inputs

1. UM Faculty with disabilities

2. UM Faculty without disabilities

Data Collection Tool Development

Faculty Advisory Committee members developed the questionnaire using an iterative process.

Created tool to assess faculty understanding of their role in accommodating students with disabilities.

Met in person and then worked together remotely over the course of several weeks.

Data Collection Methods

The Disability Resource Center (DRC) used a

community expert (kinship) model in which

members of the DRC Faculty Advisory Committee

were asked to recommend 1-2 faculty to complete

an e-mail questionnaire.

Data Collection Methods

These were sent to thirty-three (33) faculty in the

College of Liberal Arts, College of Science and

Engineering, Humphrey Center, College of

Biological Sciences, Academic Health Center,

College of Education and Human Development, and

the School of Public Health, Twenty-two (22)

faculty completed surveys for a 66 percent

response rate.

Sample Faculty E-mail

Dear Professor Aase,

Professor Fuller, a member of University of Minnesota-Twin Cities Disability Resource Center’s advisory committee, recommended you to participate in a brief questionnaire to assist the Disability Resource Center in assessing how faculty perceives the disability accommodation implementation process. The questionnaire takes about 10 minutes to complete. Your answers do not need to be lengthy. We are looking for key thoughts around each of the questions. Please type your answers in the attached document and return it to Donna Johnson, Director, Disability Resource Center at johns042@umn.edu by Tuesday, January 16, 2013. If you have any questions, please contact me at 612-624-4120.  Sincerely, Donna JohnsonDirectorDisability Resource CenterUniversity of Minnesota-Twin Cities

Sample Questions

1. Do you understand your role in implementing disability-related accommodations in the classroom, labs, testing environment, and/or course related experiences?

2. If yes, briefly explain your role.

3. If you are unsure as to how to support students with disabilities, how might the Disability Resource Center assist you in implementing disability-related accommodations in the classroom, labs, testing environment, and/or in course-related field experiences? Please be specific.

Sample Questions

4. How do you facilitate students with disabilities informing you of their accommodation needs? (Check all that apply):

• I include a syllabus statement inviting students to discuss their disability-related needs with me.

• I discuss disability-related accommodations as part of my introductory lecture on the first day of class.

• I include a statement on my course Moodle site inviting students to discuss their disability-related needs with me.

Sample Questions

5. On average, how many students with disabilities self-identify as needing accommodations in a given semester? (Check the appropriate response):

•1-2 students per semester

•3-4 students per semester

•5-6 students per semester

•7 or more students per semester

6. Of the students who identified themselves as needing disability accommodations, what types of accommodations have you implemented?

Sample Questions

7. Do you feel comfortable contacting the Disability Resource Center regarding a question about a disability-related accommodation listed in a student accommodation letter? Why/Why not?

8. Please briefly describe what has worked when when implementing accommodations for students with disabilities at the University of Minnesota?

9. Please briefly describe what has not worked well when implementing accommodations for students with disabilities at the University of Minnesota?

10. What recommendations do you have to improve communication between faculty and the Disability Resource Center?

Sample Questions

Based on your overarching outcomes, what questions might you ask?

Analysis Methods

A thematic analysis was used. Disability Resource

Center (DRC) staff compiled and reviewed the data

to identify themes. Peer and member checks were

used to validate the process by reviewing the

findings with members of the DRC Faculty Advisory

Committee.

Key Findings

1. Faculty are unclear regarding the DRC’s role – access vs.

success.

2. Faculty need multiple methods of communication, regarding

providing accommodations to students with disabilities in

their courses (from DRC, Dept. Chairs, and Provost).

3. Faculty consider DRC as a “just in time” service unit – contact

with DRC is initiated only when a problem or crisis occurs.

4. Faculty have observed more students with mental health

issues, behavioral issues, and unrealistic expectations.

Key Messages

1. Faculty want to know how many and what types of

disability conditions are present on campus.

2. Faculty want a flow chart that outlines the faculty role, the

student role, and DRC’s role in providing accommodations.

3. Faculty want additional information on how to approach

students who may have not disclosed the disability.

4. Faculty want a list of frequently asked questions around

disability issues that are modified for specific academic

audiences.

Key Recommendations

1. DRC should tailor messaging for faculty based on faculty

needs/interests.

2. DRC outreach efforts should include basic elements of

what is a disability, what is the purpose of the

accommodation letter, what is the student’s role, the

faculty role, and DRC’s role in implementing the

accommodation, and what is the process to address issues

if the accommodation is not working?

Key Accomplishments

• Created Disability Toolkit for Faculty on DRC

Website.

FAQs on testing accommodations.

Accommodation process flowchart.

• Revised accommodations letter.

• Established practice for emailing faculty.

• Participate in new faculty orientation.

Key Accomplishments

• Participate in Provost’s New Chairs Training.

• Developed video on the Interactive Process.

• Developing “Who is in Your Classroom video.

• Developing On-Line Training for Faculty with Data

Tracking Capabilities

The Culturally Responsive Evaluation Process

1. Shifted the focus on understanding the “lived experience” of faculty –

more inclusive than asking faculty to complete the questionnaire.

2. Community expert (kinship) method involved asking faculty to

contribute questions. Faculty reviewed the final questionnaire to

validate that the questions were reflective of their lived experience.

Faculty took ownership of the data gathering.

3. Faculty Advisory Committee members completed the questionnaire

and then asked 1-2 other faculty colleagues to complete the

questionnaire – This was faculty led, using faculty voice, faculty-

focused analysis with peer member checks and outcomes.

The Culturally Responsive Evaluation Process

4. Faculty could see how their work lead to meaningful action

to improve communication around the role of student,

faculty, and DRC staff in the interactive process.

5. Faculty Advisory Committee serves as a Community of

Practice (Non-Western concept of shared knowledge).

6.The process of implementing Culturally Responsive

Evaluation strengthened the DRC’s reframing of disability as a

aspect of diversity and as an equity issue.

The Culturally Responsive Evaluation Process

7. Helped build trust and buy-in and was a way for faculty to

feel heard.

8. Helped the DRC have a better shared understanding of

faculty concerns.

9. Faculty advisory committee members had a vested

interest in “harvesting” information.

Evaluation

Thank you for your participation!

Please complete the evaluation.

Thank You!

Susan A. Aase, J.D., M.S.Ed.Associate to the Director, Disability Resource Center612-624-2993aase0020@umn.edu

Donna Johnson, M.A.. M.S.Director, Disability Resource Center612-624-4120johns042@umn.edu