Dr. Timothy S. BrophyDirector of Institutional AssessmentUniversity of Florida

WRITING MEASURABLE

STUDENT LEARNING OUTCOMES

Describe and explain SACS accreditation expectations for academic program assessment

Identify and apply steps for developing measurable student learning outcomes

Develop and/or refine student learning outcomes for your degree program

TODAY’S GOALS

SACS-COC = the Southern Association of Colleges and Schools’ Commission on Colleges

SACS is the Federally-approved accrediting body for southern region of the US

SACS develops policies and standards that operationalize Federal Regulations

Federal Student Aid is tied to our reaffirmation – without accreditation we lose this important funding source

WHAT IS SACS AND WHY IS IT IMPORTANT TO BE ACCREDITED?

Selected Student Financial Aid Data for University of Florida Undergraduate Students for the

Three Most Recently Available YearsAcademic Year

2008-09 2009-10 2010-11All undergraduate students

Pell Grants $ 30,894,352

$ 41,996,944

$ 46,733,613

Federal Loans $ 60,969,318

$ 59,795,534

$ 59,470,923

Full-time, first-time degree/certificate-seeking undergraduate students

Federal Grants $ 6,512,300

$ 10,400,395

$ 11,826,734

Pell Grants $ 5,471,084

$ 8,097,580

$ 9,014,885

Other Federal Grants

$ 1,041,216

$ 2,302,815

$ 2,811,849

Federal Loans $ 6,651,786

$ 7,457,421

$ 7,362,376

Source: IPEDS Student Financial Aid Component

$106,204,536

THEN:2003 Self

Study

Volume 1: Six

required sections

Volume 2:Internation

al Focus report

THE TRANSITION

:2003-

presentEvolving Standards

and Expectatio

ns

Paradigm shift

NOW:Data–driven Continuous

Improvement

Compliance

Quality Enhancem

ent

Student Learning

HISTORICAL CONTEXT

SCOPE OF THE SACS PRINCIPLES

12 Core Requirement

s

35 subcompone

nts

14 Comprehensi

ve Standards

73 subcompone

nts

9 Federal Requirement

s

3.3.1 - The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas: (Institutional effectiveness)

3.3.1.1 educational programs, to include student learning outcomes

SACS STANDARD 3.3.1.1

Completion of at least one

complete SLO assessment

cycle

Documented evidence that

the “improvement

cycle” is complete

Evidence and documentatio

n of compliance

with all requirements and standards

WHAT SACS EXPECTS

Insti

tutio

nal

Effec

tiven

ess

Establish Goals and Outcomes

Assessment Planning

Implement the Plan and Gather Data

Interpret and Evaluate the

Data

Modify and Improve

THE UF ASSESSMENT PROCESSAssessm

ent

UF MISS

ION

Academic

Assessment Plan

Mission Alignment

Student Learning Outcomes

Curriculum Maps (UG)/ Assessment Timelines (Grad/Prof)

Assessment Cycle

Methods and

Procedures

Assessment Oversight

ACADEMIC ASSESSMENT PLAN

March 29, 2007 – Board of Governors regulation 8.016, “Academic Learning Compacts”

“Research indicates that university students are served best when students and faculty fully engage in a teaching-learning partnership, and this partnership is all the more meaningful if it is made as clear as possible to students what it is they will learn and how program faculty will assess that learning. Therefore, the Board has determined that universities must develop “Academic Learning Compacts” and related assessment processes to define and demonstrate student achievement in baccalaureate degree programs in the State University System.”

BOG Regulation 8.016, revised 1-19-12, “Student Learning Outcomes Assessment”

STUDENT LEARNING OUTCOMES AT UF

IDENTIFYING SLOS AND PROGRAM

GOALS

Student Learning Outcomes (SLOs)describe student learning – what students will know and be able to do as a result of completing a UF program

Program Goals do not describe student learning – instead, they describe programmatic elements, such as admission criteria, acceptance and graduation rates, etc

DEFINITIONS

OUTPUTS OR OUTCOMES?

Outputs describe and count what we do and whom we reach, and represent products or services we produce. Processes deliver outputs; what is produced at the end of a process is an output.

An outcome is a level of performance or achievement. It may be associated with a process or its output. Outcomes imply measurement - quantification - of performance.

DEFINITIONS

This distinction is important, especially in the development and review of Student Learning Outcomes.

We seek to measure outcomes as well as their associated outputs; however, SLOs focus on outcomes.

For example, while we produce a number of new graduates (the output), it is critical that we have a measure of the quality of the graduates as defined by the college or discipline (the outcome).

Effective Student Learning Outcomes describe, in measurable terms, these quality characteristics by defining our expectations for knowledge, critical thinking, and communication for UF undergraduates, and knowledge, skills, and professional behaviors for graduate and professional students,.

OUTCOMES AND OUTPUTS: WHAT IS THE DIFFERENCE?

Focus on what students will know and be able to do. All disciplines have a body of core knowledge that students must

learn to be successful as well as a core set of applications of that knowledge in professional settings.

Describe observable and measureable actions or behaviors. Effective SLOs present a core set of observable, measureable

behaviors. Measurement tools vary from quizzes and tests to complex rubrics.

The key to measurability: an active verb that describes a observable behavior, process, or product

A framework for developing SLOs: Bloom’s Taxonomy (see Table 3 in your Student Learning Outcomes handout)

CHARACTERISTICS OF EFFECTIVE STUDENT LEARNING OUTCOMES

Understand An internal process that is indicated by demonstrated behaviors – OK for

ALCs but not recommended for program or course SLOs Appreciate; value

Internal processes that are indicated by demonstrated behaviors closely tied to personal choice

Become familiar with Focuses assessment on “becoming familiar,” not familiarity

Learn about, think about Not observable; demonstrable through communication or other

demonstration of learning Become aware of, gain an awareness of

Focuses assessment on becoming and/or gaining – not actual awareness Demonstrate the ability to

Focuses assessment on ability, not achievement or demonstration of a skill

VERBS AND PHRASES THAT COMPLICATE MEASURABILITY

DIRECT OR INDIRECT ASSESSMENT?

Direct assessments of student learning are those that provide for direct examination or observation of student knowledge or skills against measurable performance indicators.

Indirect assessments are those that ascertain the opinion or self-report of the extent or value of learning experiences

DEFINITIONS

This model allows you to develop assessments that measure the outcomes, and that then connect directly to the program learning goals

Course-level Student Learning Outcome these are determined by the faculty and specify course-level, observable products or

demonstrations

Program-level Student Learning Outcomethese describe what students will do to demonstrate they have met the learning

goals

Program Learning Goal Level – programs establish learning goals for the degree these are described in the Academic Learning Compact, Program mission, or Catalog

entry

DEVELOPING MEASURABLE SLOS: A THREE-LEVEL MODEL (CARRIVEAU,

2010)

Learning Goals – these are found in the Academic Learning Compact in the description of the major

Example: Materials Science and Engineering

The major enables you to develop an understanding of materials systems and their role in engineering. Emphasis is placed on the ability to apply knowledge of mathematics, science and engineering principles to materials science and engineering; to design and conduct experiments, as well as to analyze and interpret data; and to design a system, component or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability.

LEVEL 1: ESTABLISHING LEARNING GOALS FOR THE

DEGREE

Source: 2012-13 UF Undergraduate Catalog, https://catalog.ufl.edu/ugrad/current/engineering/alc/materials-science-and-engineering.aspx

Students who complete the MSE degree will:

Understand materials systems and their role in engineering

Apply knowledge of mathematics, science and engineering principles to materials science and engineering to design and conduct experiments, as well as to analyze and interpret data

Design a system, component or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability

LEVEL 1: LEARNING GOALS BASED ON THE MSE ALC

Content Knowledge• Apply knowledge of mathematics, science and engineering

principles to materials science and engineering. • Design and conduct materials science and engineering

experiments and analyze and interpret the data.Critical Thinking• Design a materials science and engineering system, component

or process to meet desired needs within realistic economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability constraints.

Communication• Communicate technical data and design information effectively

in speech and in writing to other materials engineers.

LEVEL 2 – PROGRAM STUDENT LEARNING OUTCOMES FOR MSE

ALC Learning Goals: Understand materials systems and their role in engineeringDesign a system, component or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability

Student Learning Outcomes: Design a materials science and engineering system, component or process to meet desired needs within realistic economic, environmental, social, political, ethical, health and safety, manufacturability and sustainability constraints.Communicate technical data and design information effectively in speech and in writing to other materials engineers

MSE: CONNECTING GOALS TO OUTCOMES

Goal

SLO

ALC Learning Goals: Understand materials systems and their role in engineeringApply knowledge of mathematics, science and engineering principles to materials science and engineering to design and conduct experiments, as well as to analyze and interpret data

Student Learning Outcomes: Apply knowledge of mathematics, science and engineering principles to materials science and engineering. Design and conduct materials science and engineering experiments and analyze and interpret the data

MSE: CONNECTING GOALS TO OUTCOMES

Level 1

Level 2

SLOs            Additional Assess-ments

Content Knowledg

eEMA3050 EMA3066 EMA4714 EMA3080C EMA3513C EMA4714  

#1 I R A       Senior exit survey

#2       I R A Senior exit survey

Critical Thinking EMA3066 EMA4223 EMA4714        

#3 I R A       Senior exit survey

Communi-cation EMA3080C EMA3013C EMA3513C        

#4 I R A       Senior exit survey

CONNECTING PROGRAM SLOS TO COURSES MSE CURRICULUM MAP

Assessments in the boxes marked A are conducted using specific homework, exam, or assignment questions aligned with that SLO.Source: 2011-12 MSE Academic Assessment Plan

These are determined by the faculty to teach the course

However, these should directly relate to the program SLOs

LEVEL 3 – COURSE LEVEL SLOS

1. Review the Academic Learning Compact. List the learning goals for the program that are in the ALC.

2. Review the current SLOs for your area with your program faculty for recency, relevance, and rigor.

3. Examine the SLOs for the Knowledge Type (see Table 1) and Cognitive Processes level (see Table 2) they engage. The majority of the SLOs should be in the upper three levels of the Cognitive Processes Dimension – Analyze, Evaluate, and Create. The Taxonomy template in Figure 1 may help with this process.

4. Cross-reference your SLOs with the list of verbs/actions associated with their corresponding cognitive dimension levels (see Table 3), and replace any “verbs and phrases to avoid” with appropriate verbs from Table 3.

5. Write the SLO concisely and clearly.

WRITING STUDENT LEARNING OUTCOMES

(use with the UF Student Learning Outcome Guide)

Carriveau, R. (2010). Connecting the dots – Developing student learning outcomes and outcomes-based assessments. Denton, TX: Fancy Fox Publications

Suskie, L. (2009). Assessing student learning: A common sense guide (2nd ed.) San Francisco, CA: Jossey-Bass.

Walvoord, B. (2010). Assessment clear and simple: A practical guide for institutions, departments, and general education (2nd ed.). San Francisco, CA: Jossey-Bass.

You can always find our UF-specific Institutional Assessment resources at our website, http://assessment.aa.ufl.edu/

ADDITIONAL RESOURCES