interpreting data - unc student affairs · interpreting data assessment spotlight hosted by the...

Interpreting Data

Assessment Spotlight

Hosted by the Student Affairs Assessment Council

University of North Carolina at Chapel Hill

Learning Outcomes

Participants will be able to identify “making meaning” as a primary facet of data interpretation.

Participants will identify reflection and/or critical thinking as significant actions supporting data interpretation.

Participants will be able to name two schemas used to describe or classify data.

Participants will be able to name two frameworks applied to the process of data interpretation.

Hereafter, upon hearing the word “data”, participants’ first reaction will not be to cringe, sink down in their seat, or run shrieking from the room.

The Assessment Cycle

Planning

-identifying learning outcomes

-mapping programs to learning outcomes

-selecting the measurement method(s)

-providing learning opportunities

Measuring Student Learning

-collecting data

Using the Data

-interpreting data

-implementing changes based on data

What’s my Motivation?

Evaluation

- mission statement, goals, objectives

- unit and/or program functioning or effectiveness

Assessment

-learning outcomes

- student learning or development within a particular context

Embrace the relationship…

Interpretation

Data analysis and interpretation is the

process of assigning meaning to

collected information and determining

the conclusions, significance, and

implications of the findings.

OIRA, Syracuse University

Data Basics

Qualitative Data

Data that approximates or describes but does not apply

numeric measurement to define the characteristics, or

properties of a thing or a phenomenon. Examples of

qualitative data are gender identity, college

major, hometown.

Quantitative Data

Data that can be meaningfully expressed as a number,

or quantified. Examples of quantitative data are GPA,

service hours completed, or semesters enrolled full-time.

Organizing Frameworks

Differences

Relationships

Change

Competency

(Pieper, S. L. et al., 2008.)

Screening Data

Review your data:

- Missing data

- Miscoded or impossible responses

- Outliers or irregularities

Summarizing Qualitative Data

CONCEPTUALIZE Read through the data and look for patterns or themes.

CODE Identify “a code” for each unique theme

represented in the data. Read through the data a second time, and assign the appropriate code to responses or in some cases parts of responses

CATEGORIZE Identify broader patterns defined by the

codes. Groupings or Overlap? Conflicting examples? Outliers?

CONCLUDE Summarize or draw conclusions based on

the broader patterns you identify.

NOTE: Whenever possible, two or more people should code the same data and compare results. This increases the reliability & validity of the findings.

Qualitative Exercise

Read through the collection of student comments:

1) CONCEPTUALIZE - Read through the data and look for patterns or themes.

2) CODE – Identify “a code” for each unique theme represented in the data. Read through the data a second time, and assign the appropriate code to responses or in some cases parts of responses

3) CATEGORIZE - Identify broader patterns defined by the codes…Groupings or Overlap? Conflicting examples? Outliers?

4) CONCLUDE - Summarize or draw conclusions based on the broader patterns you identify.

Summarizing Quantitative Data

Aggregating & Disaggregating Data

Whole Group Trends

Group Comparisons

Describing Individual Variables

Frequency Counts & Percentages

Range of Responses

Averages

Skew

% of students meeting or exceeding a cut-off point

Describing a Relationship Between Variables

Correlation between two variables

How well a variable or set of variables can predict another

How a set of variables is associated with an outcome

Quantitative Exercises

Exercise 1: Most of the Time

Exercise 2: The F Test

Exercise 3: The Starburst Challenge

Key Questions

What patterns are evident?

What conclusions can we draw?

How does this inform what we are doing?

What are the limitations of the process?

Additional Resources

Office of Institutional Research

Dr. Bill Ware’s statistics courses in the SOE

Beginning /Advanced qualitative courses in the SOE

The Odum Institute Short Courses