texas a&m university assessment workshop prairie view a&m university november 21, 2008 ...

Texas A&M University

Assessment WorkshopAssessment Workshop

Prairie View A&M UniversityPrairie View A&M University

November 21, 2008November 21, 2008

http://assessment.tamu.edu

Celebrating 25 Years of Excellence in Teaching and Learning - 1983-2008


Outcomes for Today’s Session:Outcomes for Today’s Session:By the end of this session, participants will be able to:• Describe the role of program assessment in SACS accreditation• Identify the steps of a program assessment plan and complete a

plan given a sample program learning objective• Describe the benefits of course embedded assessment used for

program assessment• Create part of a rubric that could be used in program

assessment• Self-assess your own knowledge with regards to program

assessment


Agenda Agenda • Introductions (5 min.)• Accreditation Overview (15 min.)• Writing Program Learning Objectives and the Program

Assessment Cycle (15 min.)• Break (10 min.)

• Course-embedded Assessment (30 min.)• Rubric Use and Development (40 min.)• Resources (10 min.)• Questions (15 min.)


Assessment Steps--Assessment Steps-- Where are You? Where are You? 1. Written Mission2. Developed program-level learning

outcomes3. Determined measures4. Gathered results5. Used the results for program

improvement

04/18/23

Accreditation“Within the framework of the mission….”


Assessment of student learning

What does accreditation mean for faculty?


“The institution identifies expected outcomes, assesses whether it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas:

SACS Standard: Institutional Effectiveness


Educational programs, including student learning outcomes

Administrative support servicesEducational support servicesResearch within its educational mission, if appropriateCommunity/public service within its educational

mission, if appropriate

Revisions to the Principles of Accreditation: Foundations for Quality Enhancement, December 12, 2006

SACS Standard:Institutional Effectiveness


FacultyAdministration

High

Med

Low

Institutional Program Course

Assessment


Adapted from: Trudy Banta, IUPUI

13

Program Assessment CycleProgram Assessment Cycle

What will the students know, be able to do and how will they think when they

graduate from your program?

What evidence do the students and instructors have of success?

What data will I select to makea judgment about the

program?

Are the students meeting the learning outcomes/succeeding?

What is working/not working in the program?

What changes will be incorporated in the next program cycle?

ALL ASSESSMENT:Capstone Course project or assignmentNat’l Normed ExamEval of PortfolioInternship Perf. Review Senior SurveyAlumni Survey

INTERPRETATION:Reflection/ DocumentationCourse Portfolio

SELECTED ASSESSMENT:CapstoneNat’l ExamPortfolioSr Survey

PROGRAM LEARNING OBJECTIVES

Prior Knowledge?

Program Mission


14

Form of Learning OutcomesForm of Learning Outcomes• Learning Outcomes must:

• Contain a verb describing an observable or identifiable action

• Focus on the student as the performer• What is the student expected to be able to know?• What is a student expected to be able to do?• How is a student expected to be able to think?


15

ActivityActivity

• Choose one program learning objective for your area• What will the graduate know, be

able to do and how will they think when they walk across the stage?

• How will you know?• Share with a partner, improve with

their feedback


Program Assessment Plan Program Assessment Plan

Plan Development Steps1. Specify the Program Learning Objectives2. Identify related course(s)3. Identify course embedded assessment techniques4. Establish assessment criteria


From Program Learning Objectives to From Program Learning Objectives to

Course Embedded AssessmentCourse Embedded Assessment1. Specifying Program Learning Objectives

• What will the graduate know, be able to do and how will they think when they walk across the stage?

• How will you know?2. Identifying Related Courses and Learning Outcomes

• What courses reveal student mastery and summative assessment to address this program learning objective?

• What must students do to prove that they have mastery?

18

Where will the Program Learning Outcomes be Assessed?Where will the Program Learning Outcomes be Assessed?

Course 1 Course 2 Course 3 Course 4

Course 5 Course 6 Course 7 Course 8

Course 9 Course 10 Internship ??????

StudentTeaching

Course 14 Capstone Course 16Senior

Junior

Sophomore

Freshman

Assess to determine achievement of program learning outcomes

Assess to determine where and how interventions to improve program learning outcomes might be accomplished


Advantages of Course Embedded Assessment Advantages of Course Embedded Assessment

You’re doing this already…• Relationship to program learning objectives is

obvious• Flexibility in assessment design and

administration—Faculty ownership• No additional cost for instrumentation or

assessment techniques • Efficient


Course Embedded Assessment TechniquesCourse Embedded Assessment Techniques

• Questions embedded in final exams/tests• Common exams/tests questions across

sections of a course• Use of rubrics, checklists, rating scales, etc.,

to evaluate student performance in applicable knowledge/skills areas on essays, research papers, case study reports, oral presentations

• Pre- & post-tests

Program Assessment Plan Sample 1 Program Assessment Plan Sample 1

The Department of Chemical Engineering, Texas A&M UniversityProgram Learning Objective: An ability to use the techniques, skills, and

modern engineering tools necessary for chemical engineering practice Course Learning Outcome: Use advanced chemical process simulation

software to design integrated unit operations at the chemical plant scale. Related Course: CHEN 426: Plant Design Assessment Measure: As a course requirement, students write a final design

report and produce an oral presentation. An industry panel evaluate students’ presentations using an evaluation rubric. One of the questions address students’ skill to use advanced chemical process simulation software to design integrated unit operations at the chemical plant scale.

Assessment Criteria: At least 85% of students will be rated at 2 (“meets expectations”) and above on ability to use advanced chemical process simulation software to design integrated unit operations at the chemical plant scale.


Program Assessment Plan Sample 2Program Assessment Plan Sample 2

Program Learning Objective: Students will recognize and apply Texas law.

Course Learning Outcome: Students will define elements of crimes and penalties using Texas statutes as illustrations.

Related Course(s): CMJU 401Assessment Measures: Students will complete a final

examination in which questions related to their understanding of the elements of crimes and penalties using Texas statutes have been embedded. Members of the program assessment committee will evaluate students’ answers to the embedded questions.

Assessment Criteria: At least 70% of students completing the final examination will respond correctly to 80% or more of embedded questions.


Examples from the ParticipantsExamples from the Participants

• Brainstorm several course embedded techniques that address your program learning objective.


What is a rubric?What is a rubric?• AKA: Rating scale or grading template• A chart or list that states the criteria

and/or characteristics upon which student work will be evaluated; and

• Defines and describes two or more levels of quality for each criteria.

• A tool to measure attainment of selected program learning outcomes

Poor So-So Good

Criteria 1

Description of doing a poor job on Criteria 1

Description of doing a so-so job on Criteria 1

Description of doing a good job on Criteria 1

Criteria 2

Description of doing a poor job on Criteria 2

Description of doing a so-so job on Criteria 2

Description of doing a good job on Criteria 2


Why use a rubric?Why use a rubric?• As an evaluation tool for instructors:

• At the program level:• Assists in making course outcomes and program

learning objectives more explicit and transparent• Ensures consistency and fairness in analyzing program

learning objectives through shared analysis• Saves time• Helps analyze program strengths and weaknesses.

• At the course level:• Helps define “A” work, “B” work, etc.• Increases level of trust between students and

instructor• Saves time• Helps analyze student strengths and weaknesses.• Assists in making course outcomes and program

learning objectives more explicit and transparent


Why use a rubric? – cont.Why use a rubric? – cont.• As an instructional tool with students

• Reveals the standards of the discipline/course/assignment

• Can be used to involve students in setting standards and criteria for performance

• To provide students with benchmarks for developing, judging, and revising their own work, thus developing their ability to self-monitor, self-direct and self-assess their own learning

Ex: Ex: Rubric – Article ReviewRubric – Article Review

Summary statement of article content

Article summary is clear and concise, showing complete understanding of the content

Summary is fairly thorough, though doesn’t clearly show a full appreciation of the arguments

Summary is confused or unclear, the student does not appear to understand the article

The article is not summarized at all

Clear grasp of the argument

The student clearly understands what the author was saying, and expresses this understanding well

The student gets the gist of the paper, but doesn’t fully understand the subtleties of the arguments

The student misses a lot of important details, and doesn’t seem to get the bigger picture

The student clearly does not understand what the author was trying to say


How to create a rubricHow to create a rubric1. Identify characteristics you use in evaluating student

work

2. For each characteristic, describe what exemplary student work would look like, do the same for the lowest level

3. Decide how many levels are appropriate or necessary for each characteristic

4. Describe the remaining levels

5. Assign a weight for each characteristic

6. Check for reliability, e.g., practice with sample student work, ask a colleague to review/use the rubric with the samples, revise, revise, etc.


Program Learning Objective: Your program area example…Course Learning Outcome:Related Course(s):Assessment Measures: Assessment Criteria:

Activity: Program Assessment Plan


Take-Home MessagesTake-Home Messages You do not have to assess everything every year Modify something already being done that is meaningful to the

program Be flexible—this is an iterative process This is a faculty-driven process (The outcomes you choose

should be important to you.)


References

Palomba, C.A. and Banta, T.W. (1999). Assessment essentials: Planning, implementing and improving assessment in Higher Education. San Francisco: Jossey-Bass.

Huba, M.A. & Freed, J.E. (2000). Learner-centered assessment on college campuses: Shifting the focus from teaching to learning. Boston: Allyn and Bacon. pp 151-200.

Andrade, H. G. (February, 2000). What do we mean by results?: using rubrics to promote thinking and learning, Educational Leadership, 57(5), pp13-18.


What do we want every undergraduate to be able to know and do What do we want every undergraduate to be able to know and do

when he or she graduates from Texas A&M University?when he or she graduates from Texas A&M University?

A TAMU graduate will: Master the depth of knowledge required of a discipline. Demonstrate critical analytical skills. Communicate effectively in writing and speaking. Provide ethical leadership in a global and diverse society.

04/18/23

February 22-24, 2009

College Station, TX

For more information on the conference and registration, visit http://assessment.tamu.edu/conference

texas a&m university assessment workshop prairie view a&m university november 21, 2008 ...

Documents

program assessment slide

university slide

program mission slide

program assessment cycle

program assessment plan

role of program assessment

program assessment self

program cycle