2015 Fall Teaching InstituteLouisiana State University at Alexandria

August 20, 2015

Why do We Assess? Improving Student Learning through Performance Based Measures

Dr. Kim Chavis, Ph.D.North Carolina A & T State

University

ktchavis@ncat.edu

Words to Consider

"Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance.” '

Dr. Tom Angelo, Reassessing (and Defining) Assessment. The AAHE Bulletin, 48(2), November 1995, pp.7-9.

THERE IS A DIFFERENCE BETWEEN WHAT IS DESIGNED, TAUGHT, AND LEARNED

Designed curriculum

Official course outlinesCourse catalog

Taught curriculum

Found in syllabi

Learned curriculum

What have they learnedWhat can they doWhat do they care about

NATIONAL MOVE TOWARD ASSESSMENT OF LEARNING

What is assessment?

“the methods that an institution employs to gather evidence and evaluate quality

(Accrediting Commission for Community and Junior Colleges)

What are student learning outcomes?

“statements which define what a student should be able to do after the completion of a course or program”

(http://online.bc.cc.ca.us/courseassessment)

In terms of the learning students are able to demonstrate Rather than

inputs such as number of credit hours or curricular content

There Has Been a Paradigm Shift in the Way That We View Educational Effectiveness

Learning Outcomes:

ASSESSMENT: WHY BOTHER?

Three main purposes for assessment

1. To demonstrate quality and excellence and ensure the same level of quality continues

2. To identify areas needing attention, support and development and decide how to improve those areas

3. To plan changes that will improve policies, procedures, services, curriculum, resources, teaching, campus climate and ultimately improve student learning.

IN THE PRINCIPLES FOR ACCREDITATION

Comprehensive standard 3.3.1

The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas: (Institutional Effectiveness)

ASSESSMENT STAIRS

WHY BOTHER WITH LEARNING OUTCOMES?

Supporters assert…

They help students learn more effectively Students know what to expect from a

particular course or program Learning outcomes help instructors

Design their materials more effectively Select the appropriate teaching

strategy and technology Design examinations which reflect

what was taught

10

PROGRAMS NEED TO BE ASSESSED

Programs are an aggregation of courses with a common program mission.

The heart of assessment is embedded in the courses.

The program’s product is the graduate.

ESTABLISH CLEAR PROGRAM LEARNING DOCUMENTS

Programs should clearly define

Program Mission

Program Goals and Objectives

Learning Outcomes for each Goal

Assessment Methods

Expected Results

Actual Results

Use of Results to Improve the Program

A Mission…A mission indicates the purpose of the program or its reason for being

A mission states what it hopes to achieve in the future

A mission affirms its principles and beliefs, nature, and values.

A Sample Mission…

The Forestry and Ecology program is designed to educate broad-based, ecologically sensitive resource managers, enabling them to succeed as professional managers and practice wise stewardship of forests or other natural resources for multiple uses. As part of an Historically Black College or University, the program addresses the needs of capable students who are underrepresented in the forestry profession as well as the needs of the minority, forest-landowner community that have historically been under-served by the forestry profession.

A program goal…

A goal is broadly stated

A goal is accomplished via objectives

A goal is not directly measurable

Example: To understand and apply the concepts of bioinformatics in professional life

A program educational objective…

Objectives describe knowledge, skills, abilities, attitudes, or dispositions

An objective is directly measurable

An objective is specific and tied to a series of linked learning outcomes

Educational objectives are sometimes considered synonymous with learning outcomes, but learning outcomes are more detailed, behavioral in nature, and stated using precise verbs.

A program learning outcome…

Describes what…

we want the students in the program to know

we want the students to be able to do

values or attitudes we want to instill in our students

A program learning outcome is capable of being assessed

ProgramSLOs

CourseSLOs Course

SLOs

CourseSLOs

CourseSLOsCourse

SLOs

CourseSLOs

Overlapping Course Student Learning Outcomes Become Program Student Learning Outcomes

COURSE MAPPING

Course mapping helps to develop a sense of how current course offerings in the program cover different learning outcomes

A course map is a table with the learning outcomes on one axis and the courses in the major on the other axis

ADVANTAGES OF CURRICULUM MAPPING IN ASSESSING STUDENT LEARNING OUTCOMESCurriculum Mapping causes…

Thought about LEARNING INPUTS – Things that are in place before learning begins [prerequisites, placement, class size, facilities & equipment.

Thought about LEARNING PROCESSES – Time on task, active learning opportunities.

Thought about LEARNING CONTEXT – Professional v. liberal arts program, professional expectations, attitudes towards learning.

http://www.bridgew.edu/AssessmentGuidebook/chapter4.cfm

Simple Course Map

http://www.bridgew.edu/AssessmentGuidebook/chapter4.cfm

More Complex Course

Map Indicating Level of

Emphasis on

Learning Outcome

MYTHS ABOUT ASSESSMENT

It is not my job

I already assess; I call it grading

You cannot assess what we do in academia

I do not have the time to do it

I do not have the expertise to do it

Assessment might reveal “bad” news

It is a passing fad

YOU CANNOT ASSESS WHAT YOU HAVE NOT DEFINED: ESTABLISH CLEAR LEARNING OUTCOMES

Are your learning outcomes clear?

Are they shared outcomes?

Can you talk about them easily?

What do the results of assessment mean?

How are we going to use the results to improve student learning?

Closing the Loop

UNTIL THE RESULTS OF ASSESSMENT OF STUDENT LEARNING ARE USED, THEY HAVE NO VALUE

This may take the form of revision of course content of individual courses, revision of curricula, changes in methods of teaching, inclusion of experiential approaches

The changes must in turn be assessedfor effectiveness themselves

Don’t forget – Assessment is

not useful until it is

used!

Learning Outcomes…

Behaviors that demonstrate

• desired knowledge (cognitive domain)

• attitudes or dispositions (affective domain)

• acquired skills (psychomotor domain)

Fundamentally, the key is what will students be able to do as a result of the instruction.

The Cognitive Domain

Knowledge ComprehensionApplication

Synthesis

Analysis Evaluation

Bloom’s Classification

Knowledge…

Recalling or remembering something without necessarily understanding, using, or changing it

Count Define Describe Draw Identify

Label List Match Name Point to

Outline Quote Read Recall Recite

Record Repeat Select State Write

Comprehension…

Understanding something that has been communicated without relating it to anything else

Associate Compute Convert Defend

Discuss Distinguish Estimate Explain

Extend Extrapolate Generalize Infer

Paraphrase Predict Rewrite Summarize

Application…

Using a general concept to solve problems in a particular situation or using learned material in new and concrete situations

Add Apply Calculate Classify

Complete Compute Divide Examine

Graph Interpolate Manipulate Modify

Operate Produce Show Solve

Analysis…Breaking something down into its parts (identification of parts, analysis of relationships among parts, recognition of organizing principles)

Analyze Arrange Breakdown Combine

Design Detect Develop Diagram

Illustrate Outline Point out Relate

Select Separate Subdivide Utilize

Synthesis…

Creating something new by putting parts of different ideas together to make a whole

Categorize Combine Compile Compose

Create Design Devise Generate

Group Integrate Modify Order

Organize Plan Prescribe Propose

Rearrange Reconstruct Reorganize Revise

Evaluation…Judging the value of material or methods as they might be applied in particular situations or judging with the use of definite criteria

Appraise Assess Compare Conclude

Contrast Criticize Determine Grade

Interpret Judge Justify Measure

Rank Rate Support Test

Knowledge Comprehension Application Analysis Synthesis Evaluation

Student remembers or recognizes information as communicated with little personal assimilation

Student grasps the meaning behind the information and interprets or comprehends the information

Student uses information to relate and apply it to a new situation with minimal instructor input

Student discriminates, organizes, and scrutinizes assumptions in an attempt to identify evidence for a conclusion

Student creatively applies knowledge and analysis to integrate concepts or construct an overall theory

Student judges or evaluates information based upon standards and criteria, values, and opinions

Cite LabelListEnumerateIdentifyImitateMatchNameQuoteRecallReproduceStateWrite

ConvertDefineDescribeDiscussEstimateExplainGeneralizeIdentifyIllustrateLocateParaphraseRestateSummarize

ApplyChartComputeDemonstrateDetermineDramatizeEstablishMakeManipulatePrepareProjectSolveUse

AnalyzeCompareContrastCorrelateDiagramDissectDifferentiateDistinguishInferInvestigateLimitOutlineSeparate

AssembleCreateConstructDesignDevelopFormulateGenerateHypothesizeInitiateInventModifyReframeSynthesize

AccessAppraiseConcludeCritiqueDecideDefendDiagnoseEvaluateJudgeJustifyRankRecommendSupport

Cognitive Domain – Learning Outcomes Related to Knowledge

From Assessing Student Learning Workshop, Janet Fulks, Bakersfield College, 2004

Observe Model RecognizeStandards

Correct Apply Coach

Studentstranslate sensory input into physical tasks or activities

Students are able to replicate a fundamental task or skill once shown

Students recognize standards or criteria important to perform a task or skill correctly

Students use standards to evaluate their own performances and make corrections

Students apply this skill or ability to real life situations

Students are able to instruct or train others to perform this skill in other situations

HearIdentifyObserveSeeSmellTasteTouchWatch*

AttemptCopyFollowImitateMimicModelReenactRepeatReproduceShow Try

CheckDetectDiscriminateDifferentiateDistinguishNoticePerceiveRecognizeSelect

AdaptAdjustAlterChangeCorrectCustomizeDevelopImproveManipulateModifyPracticeRevise

BuildComposeConstructCreateDesignOriginateProduce

DemonstrateExhibitIllustrateInstructTeachTrain

Psychomotor Domain – Learning Outcomes Related to Skills

From Assessing Student Learning Workshop, Janet Fulks, Bakersfield College, 2004

Receiving Responding Valuing Organizing Characterizing

Students become aware of an attitude, behavior, or value

Students exhibit a reaction or change as a result of exposure to an attitude, behavior, or value

Students recognize value and display this through involvement or commitment

Students determine a new value or behavior as important or a priority

Students integrate consistent behavior as a naturalized value in spite of discomfort or cost and the value is recognized as part of the person’s character

AcceptAttendDescribeExplainLocateObserveRealizeReceiveRecognize

BehaveComplyCooperateDiscussExamineFollowModelPresentRespondShowStudies

AcceptAdaptBalanceChooseDifferentiateDefendInfluencePreferRecognizeSeekValue

AdaptAdjustAlterChangeCustomizeDevelopImproveManipulateModifyPracticeRevise

AuthenticateCharacterizeDefendDisplayEmbodyHabituateInternalizeProduceRepresentValidateVerify

Affective Domain – Learning Outcomes Related to Attitudes, Behaviors, and Values

From Assessing Student Learning Workshop, Janet Fulks, Bakersfield College, 2004

TO DEVELOP A LEARNING OUTCOME, BUILD IT IN PARTS…

Students will be able to demonstrate

by

learning of…the ability to…competence with…knowledge of…understanding of…

Insert informationand/orskills to belearned

Insertverbfromlist

Insert whatstudent must doto demonstratethe learning

Example:

Students will be able to demonstrate knowledge of plant nutrient requirements by identifying deficiency symptoms and listing actions to alleviate them.

EXAMPLES…

Students will be able to demonstrate knowledge of correct syntax by editing written narratives from other students

Students will be able to demonstrate understanding of human circulatory systems by drawing diagrams illustrating the path that blood follows in the body

Students will be able to demonstrate the ability to use logic by constructing a deductive argument from a real life setting

POOR LEARNING OUTCOMES…

Students will get an introduction to bioinformatics and its practical applications.

Students will demonstrate proficiency by conducting a review of published research.

Students will demonstrate an understanding of the connections between religion and politics.

Students will demonstrate proficiency with conversational Dutch by the end of the semester.

Students will demonstrate an appreciation for music of the Band Era.

LEARNING OUTCOMES ARE NOT THE SAME AS COURSE OBJECTIVES OR GOALS

Outcomes Represent overarching

products of the course Expresses higher level

thinking as observed behaviors, skills, and useable knowledge

Can be assessed as an end product and evaluated against specific criteria

Objectives Present valuable skills, tool,

or content that enable a student to do well in the course

Focus on content that is important in the classroom or what the faculty will do (input into the course)

Can be numerous, specific, and detailed

SAMPLES FOR COMPARISONGoal (the purpose of the course in more general terms)

The goal of the Scientific Writing course is to teach the basic tenets of ethical research, sound scientific writing, presentation of data, and development of proposals in response to funding opportunities.

Objectives (details course contents or activities but not what the student will do to demonstrate learning)Learn how to organize the parts of a scientific paper that is to be submitted as a manuscript for publication.

Learning Outcomes (describes what a student will know or do at the end)Students will demonstrate competency in presenting data by developing graphs for both continuous and non-continuous data.

MINI-QUIZARE THESE GOALS, OBJECTIVES, OR LEARNING OUTCOMES?

This course introduces the concept of human nutrition and its relationship with health interventions, educational training, and policy.

Concepts of nutritional requirements and how they vary with age, gender, and activity will be described.

Students will demonstrate knowledge of nutrition by documenting a nutritional problem, determining a strategy to correct it, and drafting a set of preventative strategies.

GOOD LEARNING OUTCOMES ARE ASSESSABLE!

Define expectations Develop criteria Measure the

outcome

YOU CANNOT ASSESS WHAT YOU HAVE NOT DEFINED:

ESTABLISH CLEAR LEARNING GOALS

Are your learning goals clear? Are they shared goals? Can you talk about them easily?

STUDENT LEARNING OUTCOMES: QUESTIONS TO CONSIDER

How would you ensure students have the opportunity to learn this?

How would you evaluate this in a way that is NOT dependent on a course instructor’s opinion or grade?

How would you compile the results of your evaluation?

How would you share the results? If you were dissatisfied with your

findings, what would you do about it?

47

OUTCOME DEVELOPMENT EXERCISE Use the form and lists of active verbs in

each domain to describe the attitudes, skills, and knowledge that you would like your students to know or do as a result of your course.

Divide them into domains Affective Psychomotor Cognitive

Try to get at least three in each section

DRAFT YOUR OWN LEARNING OUTCOMES

Students will demonstrate…

learning of…the ability to…competence with…knowledge of…understanding of…

Insert informationand/orskills to belearned

Insertverbfromlist

Insert whatstudent must doto demonstratethe learning

of

by

Type of learningKnowledge, skill or attitude

Action or behaviorActive verb

Try to doten of them

WHY AREN’T GRADES ENOUGH?

Grades evaluate a limited set of objectives which may or may not be related to the program objectives

They tend to be inconsistent from section-to-section and from term-to-term to be a valid overall program assessment tool

There is a conflict of interest when the instructor is the only evaluator of whether students have met program objectives

Grades do not reflect long term learning and accomplishments

Many times they do little to capture perceptions, attitudes, and skills

Knowledge

Recalling or remembering something without necessarily understanding, using, or changing it

Good Assessment Tools:

Students will identify, describe, or recount in their own words…-Written tests and essays- Oral tests- Reports- Making lists- Role play focused on accurate information- One minute papers

Understanding something that has been communicated without relating it to anything else

Comprehension

Application…Using a general concept to solve problems in a particular situation or using learned material in new and concrete situations

Good assessment tools:

Students will apply, demonstrate, solve…- Problem scenario- Problem set- Demonstration of skill- Role play- Lab report- Field experience- Internship report

Analysis…Breaking something down into its parts (identification of parts, analysis of relationships among parts, and recognition of organizing principles)

Good assessment tools:

Students will categorize, examine, induce…- Case study- Journal- Field experience/internship reflection- Complex problems- Conduct experiments and test hypotheses- Observations- Examinations of an object, writing, process

Synthesis…

Creating something new by putting parts of different ideas together to make a whole

Good assessment tools:

Students will design, integrate, relate…- Project- Research thesis- Essay- Design an experiment- Annotated bibliography- Create a video or poster- Portfolio

Evaluation…Judging the value of materials or methods as they might be applied in particular situations or judging with the use of definite criteria

Good assessment tools:

Students will conclude, critique, justify…- Debate- Essay- Report- Journal- Write and editorial/position paper- Comment on author’s perspectives- Book review- Prepare a committee/team briefing paper