![Page 1: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/1.jpg)
nsse.iub.eduNational Survey of Student Engagement
An Experience-Based View of University Quality in the USA:
The National Survey of Student Engagement
Alexander C. McCormick
Center for Postsecondary Research
Indiana University Bloomington, USA
HEPI-HEA ConferenceThe British Academy, London
22 May 2012
![Page 2: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/2.jpg)
• Context: Conceptions of quality in U.S. tertiary education
• The National Survey of Student Engagement (NSSE)
• Low stakes, high yield assessment• Student engagement trends• Controversies and cautions
Overview
![Page 3: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/3.jpg)
nsse.iub.eduNational Survey of Student Engagement
Context
Higher education in the USALimited understanding of quality
Student engagement as a new approach
![Page 4: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/4.jpg)
nsse.iub.eduNational Survey of Student Engagement
Higher Education in the USA
No national system of higher education Autonomy of institutions Diversity of institutions 4,600 degree-granting tertiary institutions Public and private Universities (undergraduate and
postgraduate) Colleges (undergraduate only)
• 4-year bachelor’s degree• 2-year associate’s degree (community colleges)
![Page 5: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/5.jpg)
nsse.iub.eduNational Survey of Student Engagement
Higher Education in the USA (2)
Diversity of students Full- and part-time Traditional college-age Older and returning students Widely varying levels of preparation
![Page 6: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/6.jpg)
nsse.iub.eduNational Survey of Student Engagement
What Do We Really Know About University Quality?
![Page 7: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/7.jpg)
nsse.iub.eduNational Survey of Student Engagement
Quality Assessment in US Higher Education
Accreditation Limited public understanding Limited disclosure Traditional emphasis on capacity and resources Limited attention to teaching & learning (until
recently)
Newsmagazine rankings Emphasize reputation, resources, other inputs
Research rankings Blind to undergraduate education
![Page 8: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/8.jpg)
nsse.iub.eduNational Survey of Student Engagement
Origins of the “National” Survey of Student Engagement (NSSE)
A reaction to the dominant discourse about university quality in the USA
Shift the focus to teaching and learning
Promote evidence-based improvement
![Page 9: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/9.jpg)
nsse.iub.eduNational Survey of Student Engagement
The Big Idea
Ask students about their experience
Focus on behavior: empirically confirmed effective practices
![Page 10: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/10.jpg)
nsse.iub.eduNational Survey of Student Engagement
Why a Student Survey?
Direct assessment of learning is problematic Measurement challenges Inferential challenges Cost challenges Motivation challenges
Survey approach offers advantages Cost-effective and efficient Diagnostic utility of behavioral data Students are the experts on their experience
![Page 11: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/11.jpg)
nsse.iub.eduNational Survey of Student Engagement
NSSE’s Key Aims
Enrich the impoverished national discourse about university quality Shift the focus to teaching & learning,
through the lens of effective practices
Provide diagnostic information that can be used to improve undergraduate education Actionable information based on
valid & reliable measures Meaningful comparisons to other institutions
![Page 12: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/12.jpg)
nsse.iub.eduNational Survey of Student Engagement
What is Student Engagement?
The extent to which students engage in and are exposed to effective educational practices Activities and practices
known to be related to desired outcomes
![Page 13: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/13.jpg)
nsse.iub.eduNational Survey of Student Engagement
OK, but what is Engagement?
Challenging academic work
Active learning Deep approaches to learning
High-impact practicesInvolvement with instructors
In a campus context that
promotes and supports success
![Page 14: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/14.jpg)
nsse.iub.eduNational Survey of Student Engagement
Conceptual & Empirical Foundations
Time on task (Tyler, 1930s)
Quality of effort (Pace, 1960s & 70s)
Student involvement (Astin, 1984)
Academic & social integration (Tinto, 1975 & 1987)
Good practices in undergraduate education (Chickering & Gamson, 1987)
College impact (Pascarella, 1985)
Student engagement (Kuh, 1991, 2005)
![Page 15: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/15.jpg)
nsse.iub.eduNational Survey of Student Engagement
Why Does Engagement Matter?
The impact of college is largely determined by individual effort and involvement in the academic, interpersonal, and extra-curricular offerings on a campus. It is important to focus on the ways an institution can shape its offerings to encourage student engagement.
Paraphrased from Pascarella & Terenzini (2005), p. 602
![Page 16: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/16.jpg)
How Does it Work?
What Do We Ask?
![Page 17: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/17.jpg)
nsse.iub.eduNational Survey of Student Engagement
How NSSE Works
• Paper or Web-based survey• First- & final-year
undergraduates• Uniform, centralized survey
administration• Institutions receive detailed
reports and an identified student data file
• Customizable comparison groups
• Results are confidential• Supported by institution fees
![Page 18: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/18.jpg)
nsse.iub.eduNational Survey of Student Engagement
Participation
550 to 770 U.S. & Canadian institutions per year
About 1,500 since NSSE’s launch in 2000
Most repeat every 1-4 years
![Page 19: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/19.jpg)
nsse.iub.eduNational Survey of Student Engagement
Survey Content
Academic activities & experiences• Academic
challenge• Reading and
writing• Active learning• Cognitive tasks in
courses• Enriching
educational experiences
Time use Study, work,
socializing…
Co-curricular activities
Institutional emphases
Quality of campus relationships
Cognitive and noncognitive gains
Satisfaction Demographic & enrollment characteristics
![Page 20: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/20.jpg)
nsse.iub.eduNational Survey of Student Engagement
Sample Questions
Asked questions in class or contributed to class discussions?
Made a class presentation?Prepared two or more drafts of a paper or assignment before turning it in?
In your experience at your institution during the current school year, about how often have you…
![Page 21: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/21.jpg)
nsse.iub.eduNational Survey of Student Engagement
Sample Questions
Put together ideas or concepts from different courses when completing assignments or during class discussions?
Discussed ideas from your readings or classes with faculty members outside of class?
In your experience at your institution during the current school year, about how often have you…
![Page 22: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/22.jpg)
nsse.iub.eduNational Survey of Student Engagement
Sample Questions
Received prompt written or oral feedback from faculty on your academic performance?
Worked harder than you thought you could to meet an instructor’s standards or expectations?
In your experience at your institution during the current school year, about how often have you…
![Page 23: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/23.jpg)
nsse.iub.eduNational Survey of Student Engagement
Sample Questions
Memorising facts, ideas, or methods…
Analysing the basic elements of an idea…
Synthesising and organizing ideas, information…
Making judgments about the value of information…
Applying theories or concepts…
During the current school year, how muchhas your coursework emphasized the following mental activities?
![Page 24: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/24.jpg)
nsse.iub.eduNational Survey of Student Engagement
Sample Questions
Examined the strengths or weaknesses of your own views on a topic or issue?
Learned something that changed the way you understand an issue or concept?
During the current school year, about how often have you done each of the following?
![Page 25: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/25.jpg)
nsse.iub.eduNational Survey of Student Engagement
Using NSSE
Results
How are we doing?
How can we improve?
![Page 26: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/26.jpg)
nsse.iub.eduNational Survey of Student Engagement
Using NSSE Results
Diagnostic purpose: analysing the undergraduate experience
Identify strengths and weaknesses, areas where improvement is needed
Stimulates conversations about quality
Informs decision-making with “actionable” information
![Page 27: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/27.jpg)
nsse.iub.eduNational Survey of Student Engagement
Many Ways to Use Results
Peer comparisons How do we measure
up to others?
Self comparison Are we improving?
Absolute standard How do we measure
up to our ideals?
Internal variation Who is least
engaged?
What to look at and how to look at it
![Page 28: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/28.jpg)
NSSE results as a catalyst for campus discussions
about teaching & learning
![Page 29: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/29.jpg)
nsse.iub.eduNational Survey of Student Engagement
Comparing Student andInstructor Perceptions
First-year students
Instructors Final-year students
Instructors0%
20%
40%
60%
80%
100%
69%
33%
62%
23%
Courses emphasised memorisation “Quite a bit” or “Very much”
Faculty Survey
![Page 30: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/30.jpg)
nsse.iub.eduNational Survey of Student Engagement
Comparing Student andInstructor Perceptions
First-year students
Instructors Final-year students
Instructors0%
20%
40%
60%
80%
100%
69%
33%
62%
23%
Courses emphasised memorisation “Quite a bit” or “Very much”
![Page 31: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/31.jpg)
Results at institutions with
several NSSE administrations show many with positive trends
![Page 32: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/32.jpg)
nsse.iub.eduNational Survey of Student Engagement
Summary of Findings
• 41% of institutions demonstrated a pattern of improvement in at least one measure for first-year students
• 28% did so for final-year students
• Trends found across many institutional types Control (public & private), Size, Research emphasis
• Ratio of positive to negative trends: 7:1
![Page 33: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/33.jpg)
nsse.iub.eduNational Survey of Student Engagement
Overview of Observed Changes: FY
Diversity Exp.
High-Impact
Integrative
Higher-Order
Supp. Campus
Stu-Faculty Int.
Active & Collab.
Academic Chall.
-10 0 10 20 30 40 50 60 70 80 90 100
Percentage with positive or negative trends
PositiveNegative
![Page 34: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/34.jpg)
nsse.iub.eduNational Survey of Student Engagement
Overview of Observed Changes: Sr
Diversity Exp.
High-Impact
Integrative
Higher-Order
Supp. Campus
Stu-Faculty Int.
Active & Collab.
Academic Chall.
-10 0 10 20 30 40 50 60 70 80 90 100
Percentage with positive or negative trends
PositiveNegative
![Page 35: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/35.jpg)
nsse.iub.eduNational Survey of Student Engagement
Cautions & Controversies
NSSE is no “magic bullet”The risks of high-stakes uses
“Participation” vs. “use”
![Page 36: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/36.jpg)
nsse.iub.eduNational Survey of Student Engagement
Cautions
NSSE is no “magic bullet”: Best used in combination with other information
NSSE’s research foundation is based on studies of US university students
Most variation is between students, not institutions
![Page 37: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/37.jpg)
nsse.iub.eduNational Survey of Student Engagement
Controversies
Confidentiality of results: Institutions control disclosure
Attaching high stakes to NSSE is risky Institutions will decline to participate if
public relations cost > institutional benefit Reliability of responses could be
compromisedMere participation is not sufficient Results must be analysed and used
![Page 38: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/38.jpg)
nsse.iub.eduNational Survey of Student Engagement
Understanding Improvement
Organizations that improve… create and nurture agreement on what is worth achieving, and they set in motion the internal processes by which people progressively learn how to do what they need to do in order to achieve what is worthwhile.
(Elmore in Fullan, 2001)
![Page 39: Nsse.iub.eduNational Survey of Student Engagement An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement](https://reader036.vdocuments.site/reader036/viewer/2022081516/551ae732550346b2288b6643/html5/thumbnails/39.jpg)
Thank you.