interpreting data - unc student affairs · interpreting data assessment spotlight hosted by the...
TRANSCRIPT
Interpreting Data
Assessment Spotlight
Hosted by the Student Affairs Assessment Council
University of North Carolina at Chapel Hill
Learning Outcomes
Participants will be able to identify “making meaning” as a primary facet of data interpretation.
Participants will identify reflection and/or critical thinking as significant actions supporting data interpretation.
Participants will be able to name two schemas used to describe or classify data.
Participants will be able to name two frameworks applied to the process of data interpretation.
Hereafter, upon hearing the word “data”, participants’ first reaction will not be to cringe, sink down in their seat, or run shrieking from the room.
The Assessment Cycle
Planning
-identifying learning outcomes
-mapping programs to learning outcomes
-selecting the measurement method(s)
-providing learning opportunities
Measuring Student Learning
-collecting data
Using the Data
-interpreting data
-implementing changes based on data
What’s my Motivation?
Evaluation
- mission statement, goals, objectives
- unit and/or program functioning or effectiveness
Assessment
-learning outcomes
- student learning or development within a particular context
Embrace the relationship…
Interpretation
Data analysis and interpretation is the
process of assigning meaning to
collected information and determining
the conclusions, significance, and
implications of the findings.
OIRA, Syracuse University
Data Basics
Qualitative Data
Data that approximates or describes but does not apply
numeric measurement to define the characteristics, or
properties of a thing or a phenomenon. Examples of
qualitative data are gender identity, college
major, hometown.
Quantitative Data
Data that can be meaningfully expressed as a number,
or quantified. Examples of quantitative data are GPA,
service hours completed, or semesters enrolled full-time.
Organizing Frameworks
Differences
Relationships
Change
Competency
(Pieper, S. L. et al., 2008.)
Screening Data
Review your data:
- Missing data
- Miscoded or impossible responses
- Outliers or irregularities
Summarizing Qualitative Data
CONCEPTUALIZE Read through the data and look for patterns or themes.
CODE Identify “a code” for each unique theme
represented in the data. Read through the data a second time, and assign the appropriate code to responses or in some cases parts of responses
CATEGORIZE Identify broader patterns defined by the
codes. Groupings or Overlap? Conflicting examples? Outliers?
CONCLUDE Summarize or draw conclusions based on
the broader patterns you identify.
NOTE: Whenever possible, two or more people should code the same data and compare results. This increases the reliability & validity of the findings.
Qualitative Exercise
Read through the collection of student comments:
1) CONCEPTUALIZE - Read through the data and look for patterns or themes.
2) CODE – Identify “a code” for each unique theme represented in the data. Read through the data a second time, and assign the appropriate code to responses or in some cases parts of responses
3) CATEGORIZE - Identify broader patterns defined by the codes…Groupings or Overlap? Conflicting examples? Outliers?
4) CONCLUDE - Summarize or draw conclusions based on the broader patterns you identify.
Summarizing Quantitative Data
Aggregating & Disaggregating Data
Whole Group Trends
Group Comparisons
Describing Individual Variables
Frequency Counts & Percentages
Range of Responses
Averages
Skew
% of students meeting or exceeding a cut-off point
Describing a Relationship Between Variables
Correlation between two variables
How well a variable or set of variables can predict another
How a set of variables is associated with an outcome
Quantitative Exercises
Exercise 1: Most of the Time
Exercise 2: The F Test
Exercise 3: The Starburst Challenge
Key Questions
What patterns are evident?
What conclusions can we draw?
How does this inform what we are doing?
What are the limitations of the process?
Additional Resources
Office of Institutional Research
Dr. Bill Ware’s statistics courses in the SOE
Beginning /Advanced qualitative courses in the SOE
The Odum Institute Short Courses