designing and implementing a sustainable assessment process
TRANSCRIPT
11/18/18
1
Designing And Implementing a Sustainable Assessment Process
Practical Strategies for Balancing Accountability and Improvement
Felix WaoAssessment Institute Pre-Conference Workshop, Indianapolis, IN
September, 2018
Workshop Outcomes
Upon completion of this workshop, participants should be able to the following at their institutions:v Design and implement effective assessment initiatives.v Develop strategies for balancing accountability and
improvement.
11/18/18
2
Activity 1__________________
What does assessment mean to you… in one word?
Assessment is…“…is a systematic process ofunderstanding and improving
student learning.”
Angelo, 1995.
Assessment is…
11/18/18
3
Levels of Assessment
Activity 2__________________
Working in groups of two or three, review Characteristics of Effective Assessment
in the handout provided.To what extent are they currently reflected at
your institutions?
11/18/18
4
Elements of Effective Assessment Planning
Key Elements in Planning Effective Assessment
Effective Assessment
Identify & Engage Stakeholders
Determine Appropriate Timing
Use Evidence to Cultivate Assessment
Culture
Design an Assessment Plan
Align Assessment with Strategic
Planning
11/18/18
5
Identify & Engage your stakeholders…
Staff
Stakeholders
students
Academic Administrators
Faculty
Student Affairs Professionals
Design a Plan…
“…written guidance on who
does what when.”
Suskie, 2004
11/18/18
6
Needs Assessment
No assessment process in place
Developed a simple 4-step process: 1=plan, 2=measure3=analyze4=adjust
Developed guidelines detailing each of the four components of the assessment process.
Significant (structural) variation among assessment reports. No guidelines.
Focus primarily on Compliance, not student learning
Refocused attention to student learning (template divided into two parts).
Resulting Actions…
11/18/18
7
Some programs excluded from the process.
Developed an outreach campaign to all department and provided examples of assessment reports.
Developed a detailed and informative color-coded review rubric.
No feedback to departments
No opportunities for dialogue among Liaisons
Created the “OU Assessment Conference”, facilitated workshops and frequent outreach to departments.
Resulting Actions…
No Institutional Assessment Office
Provost Office Created Office of Academic Assessment.
Formed Provost Advisory Council for Learning Outcomes Assessment (PACLOA).
No Assessment Committee
Resulting Actions…
11/18/18
8
Sample Assessment Process…
Programs have the greatest difficulty in the assessment
cycle of “closing the loop”
Banta & Blaich, 2011
Step 1
Hint: Use of Bloom’s Taxonomy of active verbs.
Key Question…
11/18/18
9
Creating
Evaluating
Analyzing
Applying
Understanding
RememberingLower order learning
Higher order learning
Bloom’s Taxonomy
Examples of Weasel Words.[Can be used to define a Goal, but NOT a Learning Outcome]
imagine, realize, acquainted with, familiar with, understand, know,
believe, appreciate,conscious of, interested in,
awareness of, see.
11/18/18
10
STUDENTLEARNING
OUTCOMES
CORE COURSES
A D E F
1
2
3
WHERE?
Step 2
HOW?
Hint: v Undergraduate: Focus on core courses and other educational experiences required of all students.v Graduate: Center on culminating experiences and processes that prepare students for culminating
experiences.
Key Questionsq What do the data show about your students’ level of
mastery of intended learning outcomes?
q Are there areas where students are outstanding? Are they consistently weak on certain skills?
q Are there areas where performance is good, but not outstanding, and you’d like to see better/higher performance?
Step 3
Hint: Focus on (aggregate) student performance.
11/18/18
11
Key Questionsq What will you do to improve student learning? q Which program elements should be:
Ø Reinforced?
Ø Maintained?
Ø Strengthened?
Ø Modified?
Step 4
Tip: Focus on program level action plans based directly on evidence of student achievement.
Align Assessment Process with Strategic Plan…
Assessment Institutional Strategic Plan
11/18/18
12
Align Assessment Process with Strategic Plan… (Pace University)
Where are we? …And how are we
doing? retention
admission
funding
research
Graduation rates
Teaching & learning
Program review
accreditation assessm
ent
Timing is crucial…
11/18/18
13
Timing is crucial… (University of Houston)
Institutional Assessment Plan
Academic Programs
Administrative Units
Cultivate a Culture Based on EVIDENCE…
EVIDENCE
Care about student learning
Trust and Integrity
Curriculum Design & Pedagogy
Delivery and
assessment
Formative Summative
TeachingFaculty
Research
11/18/18
14
Activity 2 __________________
Working in groups of two or three, use the template provided to rate the level of
importance of each element in implementing assessment at your current institution.
11/18/18
15
Techniques for Successful Implementation of Assessment Plans
Leadership Matters…
11/18/18
16
Encourage Faculty to Actively Participate in Committees…
Provide Adequate Resources…
11/18/18
17
Provide Professional Development Opportunities
Faculty Learning
CommunitiesPeer Coaching
Needs Assessment Faculty
Mentoring Programs
Professional Development
Plan
Assess Both Processes and Outcomes…Assess Both Processes and Outcomes…
Processes Outcomes
improves
informs
11/18/18
18
Disseminate Results…
Faculty
Students
Staff
Commu
nity
Deans
Chairs
Suskie (2009)
Use Findings Fairly, Ethically and Responsibly…
11/18/18
19
Strategies for Sustaining Effective Assessment
Disseminate Credible Results to Multiple Stakeholders…
Faculty
Students
Academic Advisors
Community
Deans
Retention
Student Funding
Admission Standards
Course Revisions
Course Sequence
Faculty Hiring
Licensure Pass Rates
Accredito
rs
Student Affairs Professionals
Assessment Process
Admission Officers
11/18/18
20
MISSING INFORMATION
NEEDS REVISION
MEETS OR EXEEDS
EXPECTATIONS
Assessing Assessment…
Activity 3 __________________
Reviewing Assessment Reports
11/18/18
21
Resistance… how do you handle it?
Key to Balancing Accountability and Improvement…
ImprovementAccountability
Planning*, implementing and sustaining effective
assessment initiatives focused on continuous
improvement.
*Embed external requirements during the planning phase.
11/18/18
22
• What is the most valuable lesson that you learned from this session?• What is one question that you still have?• What is one thing you will implement on your campus
as a result of this presentation?
One-minute Eval…
?
11/18/18
23
Felix Wao [email protected]
University of OklahomaOffice of Academic Assessment
www.ou.edu/assessment
References
Banta, T. W., & Blaich, C. (2011). Closing the assessment loop. Change: The Magazine for Higher Learning, 43(1), 22–27.
Suskie, L (2004). Assessing Student Learning: A Common Sense Guide. Bolton, MA: Anker.
Suskie, L (2009). Assessing Student Learning: A Common Sense Guide (2nd Ed.). San Francisco, CA: Jossey-Bass.
Course 1 Course 2 Course 3 Course 4 Course 5 Course 6
BASIC Curriculum Mapping Template
Part AProgram student learning outcomes (SLOs).
SLO #1
SLO #5
SLO #3
SLO #4
SLO #2
Part BRequired Courses Part C
Indirect Measures
Part C
Map indirect measures.
Insert "X" under each course to align with the SLO it addresses. Insert program student learning outcomes (SLOs).
INSTRUCTIONS
Part B
Indirect measures ask students to reflect upon and report their perceptions of their gains in knowledge, skills, etc.
Examples of indirect measures include graduating surveys, exit surveys, alumni surveys, and focus groups.
If the program conducts surveys or focus groups with students, alumni, etc., place an "x" in the cell(s) in which a part or all of the survey's (or focus group's) findings may align with a program-level SLOs.
1. Insert program outcomes in the rows labeled "Program level student learning outcomes" . Insert additional rows if needed.
2. Insert an abbreviated course name, number, and units/sections (if applicable) in labeled columns. Insert additional columns if needed.
It is helpful to insert course information in the sequence in which they are typically taught.
1. Request that faculty teaching each course listed in the columns to determine which, if any, of the program student learning outcomes (SLOs) are addressed and/or assessed in their course.
2. Ask those same faculty members to identify alignment of each course with one or more SLOs. Select a course in which the SLO is addressed.
It's perefectly ok to have more than one course addressing a single SLO and/or more than one SLO addressed by a single course.
Part A
Course 1 Course 2 Course 3 Course 4 Course 5 Course 6
MAPPING KEY: I=Introduced , A=Advanced , M=Mastery
ENHANCED Curriculum Mapping Template
A. Insert units/sections (if applicable)
Program student learning outcomes (SLOs).
A. Insert course names & #'s in columns and program (SLOs) in
rows D. Indirect Measures
Required Courses
B. Insert "I," "A," or "M"
D. Insert potential assessments
B. Insert "I," "A," or "M"
D. Insert potential assessments
C. Insert instructional activities
C. Insert instructional activities
SLO #1
SLO #2
SLO #3
SLO #4
SLO #5
B. Insert "I," "A," or "M"
C. Insert instructional activities
D. Insert potential assessments
B. Insert "I," "A," or "M"
D. Insert potential assessments
B. Insert "I," "A," or "M"
D. Insert potential assessments
C. Insert instructional activities
C. Insert instructional activities
EMap indirect
measures.
INSTRUCTIONS
Insert course and program student learning outcomes (SLOs).
Insert expected knowledge or skill level of each course.
BA
1. Insert program outcomes in the rows labeled "Program level student learning outcomes" . Insert additional rows if needed.
2. Insert an abbreviated course name, number, and units/sections (if applicable) in labeled columns. Insert additional columns if needed.
It is helpful to insert course information in the sequence in which they are typically taught.
1. Request that faculty teaching each course listed in the columns to determine which, if any, of the program student learning outcomes (SLOs) are addressed and/or assessed in their course.
2. Ask those same faculty members to identify whether students--after taking the course--are expected to demonstrate:- introductory knowledge or skill (e.g., recall or explain facts, concepts), - advanced knowledge or skill (e.g., apply a procedure or analyze how parts relate to or contrast from one another), or - mastery (e.g., evaluate or make judgments based on criteria; create a novel approach, product, or artifact).
In order to represent each faculty member's determination, use the following key:
"I" for Introductory , "A" for Advanced , "M" for Mastery.
Indirect measures ask students to reflect upon and report their perceptions of their gains in knowledge, skills, etc.
Examples of indirect measures include graduating surveys, exit surveys, alumni surveys, and focus groups.
If the program conducts surveys or focus groups with students, alumni, etc., place an "x" in the cell(s) in which a part or all of the survey's (or focus group's) findings may align with a program-level SLOs.
1. Instructional activities occurring in and out of class reinforce learning objectives and prepare students for assessments. Request that faculty teaching each course list the instructional activities for each course.
Frequently used instructional activities are listed below. (This list is not all-inclusive; faculty should insert any activities not found in this list.)
--case study--client project--debate--discussions (online and face-to-face)--exhibition of work--group work--lab reports--lecture (e.g., by faculty, guests)--oral presentation--performance--public review--recitation--service learning--studio--video clips--Book review and discussions
Insert instructional activities used in courses
1. Request that faculty teaching each course state how student knowledge or skill is assessed in their course, as aligned with the program-level SLOs.
2. Insert potential assessment methods into the appropriate cells.
Frequently used assessments are listed below. (This list is not all-inclusive; faculty should insert any activities not found in this list.)
--case study--client project--debate--exhibition of work--exam*--group work/project--lab reports--oral presentation--performance--problem sets--service learning--studio--written work--internship/practicum--reflection papers--journal article critiques
*Besides multiple-choice tests/exams, the quality of student work in the rest of the assessments require use of rubrics .
C DInsert how the outcome is assessed in the
course.
Perspectivesin Western Art
Perspectives in Ancient & World
Art
CriticalTheories in Art
FoundationStudio I
AdvancedStudio I
Future Media and Advanced
Techniques
Senior Studio, Exhibition, &
PortfolioGraduates of the AA in Art should be able
to: 2 3 5 4 4 5 8SLO 1: Appropriately conduct and incorporate research findings into their work
x x x x x x x
SLO 2: Evaluate art movements from various cultures and time periods x x x x x x x
SLO 3: Articulate a philosophical and aesthetic approach to their art and its place in the larger cultural and historical context
x x x x
SLO 4: Design and execute projects effectively x x x x
SLO 5: Use new tools and methods with facility x x x xSLO 6: Create a distinctive body of work that embodies their personal approach and their creative and technical mastery
x x x x
BFA in Art Example of BASIC Undergraduate Curriculum Map
Program Student Learning Outcomes (SLOs)
Required Courses
Indirect Measures
MAPPING KEY: I=Introduced , A=Advanced , M=Mastery
Perspectivesin Western Art
Perspectives in Ancient & World
Art
CriticalTheories in Art
FoundationStudio I
AdvancedStudio I
Future Media and Advanced
Techniques
Senior Studio, Exhibition, &
PortfolioGraduates of the AA in Art should be able
to: 2 3 5 4 4 5 8SLO 1: Appropriately conduct and incorporate research findings into their work
I I I I A A M
SLO 2: Evaluate art movements from various cultures and time periods I I I I A A M
SLO 3: Articulate a philosophical and aesthetic approach to their art and its place in the larger cultural and historical context
I A A M
SLO 4: Design and execute projects effectively I I A M
SLO 5: Use new tools and methods with facility I I I A
SLO 6: Create a distinctive body of work that embodies their personal approach and their creative and technical mastery
I I A A
BFA in Art Example of BASIC Undergraduate Curriculum Map (with Levels)
MAPPING KEY: I=Introduced , A=Advanced , M=Mastery
Program Student Learning Outcomes (SLOs)
Required Courses
Indirect Measures
Perspectivesin Western Art
Perspectives in Ancient & World Art
CriticalTheories in Art
FoundationStudio I
AdvancedStudio I
Future Media and Advanced
Techniques
Senior Studio, Exhibition, & Portfolio
Graduates of the AA in Art should be able to: A. Course units/sections 2 3 5 4 4 5 8
B. Insert "I," "A," or "M," I I I I A A M
C. Insert instructional activities
Virtual and in-person resource orientations,
lectures, group discussions, debates
Virtual and in-person resource orientations,
lectures, group discussions, debates
Lectures, group discussions, artist
seminars
Artistic work process, iterative critique and
dialogue w/peers& faculty
Artistic work process, iterative critique and
dialogue w/peers& faculty
Artistic work process, iterative critique and
dialogue w/peers& faculty
Exhibition & portfolio process, iterative critique
and dialogue w/peers& faculty
D. Insert potential assessments Written critiques Written critiquesMid-term andfinal papers
Artistic work product, oral presentation
Artistic work product, oral presentation
Artistic work product, oral presentation
Exhibition & portfolio presentation
B. Insert "I," "A," or "M," I I I I A A M
C. Insert instructional activities
Lectures, group discussions, debatesvirtual and in-person
museum tours, PowerPoint reviews
Lectures, group discussions, debatesvirtual and in-person
museum tours, PowerPoint reviews
Lectures, group discussions,
artists seminars
Artistic work process, iterative critique and dialogue w/peers &
faculty
Artistic work process, iterative critique and dialogue w/peers &
faculty
Artistic work process, iterative critique and dialogue w/peers &
faculty
Exhibition & portfolio process, iterative critique
and dialogue w/peers& faculty
D. Insert potential assessments Written critiques Written critiques Mid-term andfinal papers
Artistic work product, oral presentation, written
critiques of peer work
Artistic work product, oral presentation, written
critiques of peer work
Artistic work product, oral presentation, written
critiques of peer work
Exhibition & portfolio presentation, written
critiques of peer work
B. Insert "I," "A," or "M," I A A M
C. Insert instructional activities
Artistic work process, iterative critique and dialogue w/peers &
faculty
Artistic work process, iterative critique and dialogue w/peers &
faculty
Artistic work process, iterative critique and dialogue w/peers &
faculty
Exhibition & portfolio process, iterative critique
and dialogue w/peers& faculty
D. Insert potential assessments Oral presentation of artistic work product
Oral presentation of artistic work product
Oral presentation of artistic work product
Exhibition & portfolio presentation
Perspectivesin Western Art
Perspectives in Ancient & World Art
CriticalTheories in Art
FoundationStudio I
AdvancedStudio I
Future Media and Advanced
Techniques
Senior Studio, Exhibition, & Portfolio
I=Introduced , A=Advanced , M=Mastery
Program Student Learning Outcomes
(SLOs)
A. Insert course names & #'s in columns and program SLOs
in rows
Required Courses
D. Indirect Measures
Example of ENHANCED Undergraduate Curriculum Map BFA in Art
Program Student Learning Outcomes
(SLOs)
A. Insert course names & #'s in columns and program SLOs
in rows
Required Courses
D. Indirect Measures
MAPPING KEY:
Appropriately conduct and incorporate research findings into their work.
Focus Groups and
Alumni and Graduating
Surveys
Evaluate art movements from various culturesand time periods.
Focus Groups and
Alumni and Graduating
Surveys
Articulate a philosophical and aesthetic approach to their art and its place in the larger cultural and historical context
Focus Groups and
Alumni and Graduating
Surveys
Graduates of the BFA in Art should be able to: A. Insert course units 2 3 5 4 4 5 8
B. Insert "I," "A," or "M," I I A M
C. Insert instructional activities
Artistic work process, iterative critique and dialogue w/peers &
faculty
Artistic work process, iterative critique and dialogue w/peers &
faculty
Artistic work process, iterative critique and dialogue w/peers &
faculty
Exhibition & portfolio process, iterative critique
and dialogue w/peers& faculty
D. Insert potential assessments
Artistic work process, iterative critique and dialogue w/peers &
faculty
Artistic work process, iterative critique and dialogue w/peers &
faculty
Artistic work process, iterative critique and dialogue w/peers &
faculty
Exhibition & portfolio process, iterative critique
and dialogue w/peers& faculty
B. Insert "I," "A," or "M," I I I A
C. Insert instructional activities
Artistic work process, iterative critique and dialogue w/peers &
faculty
Artistic work process, iterative critique and dialogue w/peers &
faculty
Artistic work process, iterative critique and dialogue w/peers &
faculty
Exhibition and portfolio process, iterative critique
and dialogue w/peers& faculty
D. Insert potential assessments Artistic work product Artistic work product Artistic work product Exhibition, portfolio presentation
B. Insert "I," "A," or "M," I I A A
C. Insert instructional activities
Artistic work process, iterative critique and dialogue w/peers &
faculty
Artistic work process, iterative critique and dialogue w/peers &
faculty
Artistic work process, iterative critique and dialogue w/peers &
faculty
Exhibition & portfolio process, iterative critique
and dialogue w/peers& faculty
D. Insert potential assessmentsArtistic work product,
oral presentationArtistic work product,
oral presentationArtistic work product,
oral presentationExhibition & portfolio
presentation
MAPPING KEY: I=Introduced , A=Advanced , M=Mastery
Use new tools andmethods with facility
Focus Groups and
Alumni and Graduating
Surveys
Create a distinctive body of work that embodies their personal approach and their creative and technical mastery
Focus Groups and
Alumni and Graduating
Surveys
D. Indirect Measures
Design and executeprojects effectively
Focus Groups and
Alumni and Graduating
Surveys
ãFaculty Needs Survey, Office of Academic Assessment, University of Oklahoma
1
UNIVERSITY OF OKLAHOMA OFFICE OF ACADEMIC ASSESSMENT
NEEDS ASSESSMENT SURVEY FOR FACULTY
Part 1: YOUR VIEWS ABOUT ASSESSMENT
a) Please indicate your level of agreement with the following statements. My department...
Agree Strongly Agree Disagree Disagree
Strongly I Don't Know
Coordinates its student assessment activities annually in conjunction with campus administrators o o o o o Is recognized by faculty in other colleges/departments for its approach to program assessment. o o o o o Has an effective plan for monitoring student outcomes. o o o o o Demonstrates a great deal of consensus on its approach to student learning o o o o o Collects information about employer needs for specific skills and knowledge among our graduates o o o o o Is perceived as a campus leader on issues of student assessment. o o o o o From an educational standpoint, it is necessary for us to monitor what students learn. o o o o o Has influence on assessment techniques I use in my course(s) o o o o o
ãFaculty Needs Survey, Office of Academic Assessment, University of Oklahoma
2
b) Please indicate your level of agreement with the following views about assessment.
Agree Strongly Agree Disagree Disagree
Strongly I Don't Know
The results of student evaluations of my teaching influence my approach to assessing student work. o o o o o State or federally mandated assessment requirements improve the quality of undergraduate education. o o o o o Student assessment reduces the quality of education. o o o o o Student assessment limits the amount of time I have to devote to other academic activities. o o o o o Student assessment is more effective when determined by the faculty member rather than by the institution. o o o o o Student assessment has improved the quality of education at OU. o o o o o From an educational standpoint, it is necessary for us to monitor what students learn. o o o o o The effectiveness of teaching is enhanced when faculty regularly assess students. o o o o o What I learn by assessing student learning has immediate relevance to what takes place in the classroom. o o o o o Regular assessment of students accurately captures what they are learning in my classroom o o o o o Monitoring student assessment is a distraction and competes with essential academic work o o o o o Faculty have a professional obligation to regularly assess what students are learning in courses o o o o o
ãFaculty Needs Survey, Office of Academic Assessment, University of Oklahoma
3
Part 2: PROFESSIONAL DEVELOPMENT TOPICS a) The following section reflects important aspects of assessment at both course and program levels. Please
rate your perception of each aspect and indicate if you would be interested in attending workshops/sessions designed to address them.
Your perception of the importance of this aspect
Would you be interested in attending a
workshop designed to address this aspect?
Very
Important Moderately Important
Of little Importance
N/A
Definitely Yes
Probably Yes
Probably Not
Unsure
• Defining course level student learning objectives O O O O O O O O
• Defining program level student learning outcomes O O O O O O O O
• Aligning course objectives with program level student learning outcomes O O O O O O O O
• Determining appropriate assessment strategies for face-to-face courses O O O O O O O O
• Determining appropriate assessment strategies for blended or fully on-line courses (including Open Courses).
O O O O O O O O • Developing strategies for assessing
students' dispositions, virtues and attitudes
O O O O O O O O • Providing feedback to learners and
using feedback to enhance instruction O O O O O O O O • Organizing content in the best
sequence for learning O O O O O O O O • Using digital media to support learning
(wikis, blogs, social media) O O O O O O O O • Designing and assessing team-based
learning sessions O O O O O O O O • Making lectures engaging and
interactive O O O O O O O O • Using clickers to elicit student
participation and promote engagement O O O O O O O O • Developing service learning initiatives to
achieve learning goals O O O O O O O O • Designing competency-based
teaching/learning strategy O O O O O O O O • Designing problem-based learning
strategies O O O O O O O O • Promoting case-based instruction O O O O O O O O • Assessing large classes O O O O O O O O • Integrating and assessing academic
service learning into your course O O O O O O O O • Developing rubrics for assessing
assignments and projects O O O O O O O O • Writing effective multiple-choice tests
(that target higher order thinking skills) O O O O O O O O • Classroom Assessment Techniques
(CATs)/Informal, formative assessments
O O O O O O O O • Use and evaluation of portfolios O O O O O O O O
ãFaculty Needs Survey, Office of Academic Assessment, University of Oklahoma
4
b) What other topics/areas would you like to see presented?
________________________________________________________________
________________________________________________________________
________________________________________________________________ c) Would you be willing to join the staff of the Center for Teaching Excellence as a guest speaker or a facilitator for any of
the topics/areas mentioned above? Is yes, please write your name and e-mail address in the space below as well as and the topic(s) you'd be interested in facilitating.
________________________________________________________________
________________________________________________________________
________________________________________________________________ d) Can you suggest possible guest speakers or facilitators (on/off campus) for our workshops? Please provide names,
institutions and their areas of expertise.
________________________________________________________________
________________________________________________________________
Part 3: PREFERENCES FOR WORKSHOP DELIVERY METHODS AND FORMATS
Preferences for workshop formats often depend on the topic. However, it would be helpful if you could provide us with your general preferences for workshop formats, time, and length. a) Preferred formats (please check all that apply)
O Formal face-to-face presentations followed by discussion
O Combination of presentation, group/interactive work, and discussion
O Self-paced/self-directed materials (e.g., Web-based resources, video tapes/DVD's, handouts, etc.)
O Informal face-to-face events (e.g., presentations, brown bag meetings, etc.)
O Informal on-line sessions (e.g., web-based presentations, chat sessions, etc.)
O Other (please specify): ________________________________________________
b) Preferred day/time (please check 3 preferred starting times for each day of your choice) Day Time
8am 9am 10am 11am Noon 1pm 2pm 3pm 4pm Monday O O O O O O O O O Tuesday O O O O O O O O O
Wednesday O O O O O O O O O Thursday O O O O O O O O O
Friday O O O O O O O O O
ãFaculty Needs Survey, Office of Academic Assessment, University of Oklahoma
5
c) Preferred length (please check all that apply)
▢ up to 60 minutes
▢ up to 90 minutes
▢ up to 2 hours
▢ up to 2 1/2 hours
▢ up to 3 hours
▢ 1/2 day
▢ Other (please specify): _____________________
d) Preferred way to receive information about upcoming
workshops/sessions, programs, and program materials (please check all that apply)
▢ Office of Academic Assessment website
▢ Flyer in campus mail
▢ Listserv
▢ Posters
▢ Other (please specify) ______________________ e) Please provide any additional comments or suggestions below.
_________________________________________________________________________________________________
_________________________________________________________________________________________________
Part 6: DEMOGRAPHIC INFORMATION This section is optional. However, it will help us to identify needs of specific groups.
a) Your primary appointment O Department O College
b) Position
O Professor
O Associate Professor
O Assistant Professor
O Instructor
O Ranked Renewable Term
O Adjunct Professor/Visiting Lecturer
O Graduate Assistants Teaching Stand-alone Courses
O Other (Please Specify) ________________________________________________
c) What is your tenure status?
O Tenured (please enter year you received tenure)
O On Tenure Track
O Not on Tenure Track
d) Please indicate the type and format of courses you primarily teach.
Face-to-Face Blended or Hybrid 100% Online
• Undergraduate credit courses O O O
• Graduate courses O O O • Non-credit courses O O O
ãFaculty Needs Survey, Office of Academic Assessment, University of Oklahoma
6
e) Gender
o Female
o Male
o Do not wish to respond
f) Time at OU
o Less than 1 year
o 2-5 years
o 6-9 years
o More than 10 years
g) What is your race/ethnicity? (Select all that apply)
▢ American Indian or Alaskan Native
▢ Asian
▢ Black or African American
▢ Hispanic or Latino/Latina
▢ Native Hawaiian or Other Pacific Islander
▢ White
▢ Mixed Race
▢ Other (please specify) ________________________________________________
1 a) a)
a) a)
b) b)
a) a)
b) b)
a) a)
b) b)
©OFFICE OF ACADEMIC ASSESSMENT - UNIVERSITY OF OKLAHOMA
Report includes information on dissemination of results to faculty and an action plan for continous improvement.
One or more of the descriptions does not address gaps in student performance and is not aligned with the SLOs, direct measures (and, where applicbale, indirect measures), performance targets and/or assessment results.
Descriptions for ALL SLOs address gaps in student learning and are directly aligned with the SLOs, direct measures (and, where applicbale, indirect measures), performance targets and assessment results.
Report shows critical evaluation of past and current assessment, including strengths and possible opportunities for continous improvement.
Descriptions for one or more SLOs are: (1) based exlusively on end-of-course grades or indirect measures, (2) devoid of specific examples of planned and/or implemented use of assessment results for program improvement, (3) consistently indicating that no changes are needed without further explanation.
Descriptions for ALL SLOs reflect specific examples of planned and/or implemented use of assessment results primarily from direct and, where applicable, indirect assessments for program improvement. Explanations are provided where descriptions indicate that no changes are needed.
3 ASSESSMENT RESULTS
Briefly describe students' actual performance on each SLO. Report data on aggregate .
4 USE OF ASSESSMENT RESULTS
Describe in detail how faculty have used or plan to use results of student achievement and, where possible, augment outcomes of direct measures with students' perceptions about their learning, to make curricular changes . Include an action plan on how data on student achievement are or will be used to support program improvements tied directly to student learning.
Use of assessment results information is missing.
STUDENT LEARNING OUTCOMES (SLOs)
Articulate 3-5 statements that reflect specific knowledge, skills or abilities graduates of your degree program should be able to
SLOs are missing.
Trends or patterns over time are discussed to indicate progress of student performance. Assessment results indicate areas where students excelled, met standards, and fell short.Results for one or more SLOs does not indicate the number of
students assessed.Results for ALL SLOs indicate the number of students assessed.
REVIEW RUBRIC FOR PROGRAM ASSESSMENT REPORTSThisrubricwasdevelopedbytheOfficeofAcademicAssessmenttoprovidefeedbackregardingtheannualProgramAssessmentReports.Therubricreflectsbriefdescriptionsofthe4-stepOUProgramAssessmentProcessandColorCodesindicatingtheoutcomeofthereviewsofcontentsforeachstep.Forinstance,ifaStudentLearningOutcome(SLO)isnotstatedin"Measurable"terms,thenitem"a"under"NEEDSREVISION"willbehighlightedinYellowtosignifythattheSLOneedstoberevised.Inshort,onlythehighlightedtext/area(inGreen,YelloworRed)appliestoyourreport.
COLLEGE______________________
Assessment results or findings are missing.
Results for one or more SLOs: (1) rely exclusively on end-of-course grades or indirect measures, (2) are not clearly described and not reported on aggregate, and (3) are not aligned with methods or performance targets in SLOs.
Results for ALL SLOs: (1) are presented using appropriate direct assessments, (2) are clearly described and reported on aggregate and (3) are directly aligned with methods and performance targets in each SLO.
2 ASSESSMENT METHODS
Describe how each SLO is assessed using at least one direct measure (e.g., exams, Capstone projects, etc.) and, where possible, an indirect measure (e.g., exit surveys).
Assessment methods are missing.
At least one of the methods does not reflect description of data collection process and use of appropriate direct measure. Performance targets are not reported and no explanation is provided.
ALL methods clearly describe the data collection process and include at least one appropriate direct measure for each SLO. Performance targets are reported -- if none is reported, an explanation is provided.
Report includes attachment of sample rubric(s).
One or more methods reflect exclusive use of end-of-course grades and/or indirect methods (e.g., surveys).
ALL methods reflect evaluation of the quality of students' work using a rubric (where appropriate). Indirect measures (e.g., surveys), if mentioned, are used only to supplement direct measures.
Methods include information on psychometric properties of assessment instruments or tools (e.g., inter-rater reliability of rubrics).
Degree Program:________________________________
One or more SLOs is unclear and not stated in measurable and/or observable terms. SLOs statements are too broad and/or stated in terms of program effectiveness goals, not student learning outcomes.
ALL SLOs are clearly stated in measurable and/or observable terms. SLOs reflect specific knowledge, abilities or skills graduates of the degree program are expected to demonstrate.
SLOs include initiatives that resulted from previous assessments.
OFFICE OF ACADEMIC ASSESSMENT
STEPS OF PROGRAM ASSESSMENT PROCESS
REVIEW COLOR CODES FOR EACH STEP OF THE ASSESSMENT PROCESS
RECOMMENDATIONSMISSING INFORMATION NEEDS REVISION MEETS EXPECTATIONS EXCEEDS EXPECTATIONS
Information for ONE OR MORE of the Assessment Steps is missing OR the assessment Report was NOT submitted.
Assessment Report reflects ONE OR MORE of the following IN EACH STEP of the assessment process:
Assessment Report reflects ONE OR MORE of the following IN EACH STEP of the assessment process:
All of the "Meets Expectations" indicators are met and AT LEAST ONE of the following additional indicators is met (IN EACH STEP).