student engagement at wsu: what can we learn from five years of nsse information
DESCRIPTION
Office of Institutional Research. Student Engagement at WSU: What can we Learn From Five years of NSSE Information. NSSE: National Study of Student Engagement. A national survey of first year students and seniors in 4-year colleges and universities. - PowerPoint PPT PresentationTRANSCRIPT
1
Student Engagement at WSU: What can we Learn From Five years of NSSE Information
Office of Institutional Research
2
NSSE: National Study of Student Engagement
• A national survey of first year students and seniors in 4-year colleges and universities.
• It was designed to measure “engagement” or how students perceive their involvement in the total academic experience
• Engagement is defined by George Kuh as:– What students do -- time and energy devoted to
educationally purposeful activities both in and out of the classroom
– What institutions do -- using effective educational practices to induce students to do the right things
3
NSSE: National Study of Student Engagement
• First full data collection in 2000
• 1300 different 4-year schools have participated
• Averages 600+ schools for the last five years
• Survey of a random sample of freshmen and seniors conducted in spring each year
4
Questions
• Students are asked to respond to questions about their experiences at the university including academics, university services, and various out of classroom activities.
• 85 items with varying response options such as– How often have you… 1 Never to 4 very often– How would you rate… 1 very unfavorable to 7 highly
favorable– How would you evaluate…1 poor to 4 excellent– How much emphasis… 1 very little to 4 very much
5
Seven Principles of Good Practice in Undergraduate Education*
Student-faculty contact Active learning Prompt feedback Time on task High expectations Experiences with
diversity Cooperation among
students
* Chickering & Gamson, 1987; Pascarella & Terenzini, 2005
6
NSSE Indicators ofEffective Educational Practice
Level of Academic Challenge
Active & Collaborative
Learning
Enriching Educational Experiences
Student – Faculty
Interaction
Supportive Campus
Environment
The WSU ExperienceSeveral administrations
Pooling data on several administrations
Providing college level results with sufficient sample size
8
Background Information
NSSE has now been administered at WSU six times (2002, 2004, 2005, 2007, 2008 and 2009)
Although part of two recent KBOR Performance Agreements, use by the academic colleges or other areas has been minimal.
The Foundation of Excellence Project last year did use the results for the first year students.
Academic colleges have expressed concern that the sample size in a single year is insufficient for their use.
9
Thus …
The University Assessment asked us to see if we could legitimately combine several years’ data and then consider differences among the academic colleges.
10
Purposes of Current Analyses
Reviewed data over several years to identify any trends or changes.
Using the data from several survey administrations, disaggregate the WSU NSSE data to individual colleges by major with sample sizes large enough to encourage confidence in results.
Provide information about groups of items that are conceptually and empirically confirmed as related (i.e. benchmarks and scalelets).
Allow colleges and other campus groups the opportunity to use the NSSE as another component of their total outcome assessments.
Limited to seniors since most logical at the college level.
1. Validated item consistency
2. Determined differences, if any, for WSU data over administrations
Step 1: Merging data over multiple survey administrations
12
Item consistency over time:Review using NSSE materials
• Reviewed the detailed report produced by the NSSE staff for item changes over each administration since 2002.– Sufficient number of changes before 2004 that deletion
of 2002 data seemed essential.– One item changed in 2004 but this item not used in our
analysis
• The wording of the survey items and the response options on the surveys are consistent over the years since 2004.
13
Item consistency over time: Review using WSU Data
Compared each item by class and year using Chi-Square or Analysis of Variance.
Used complete responses (e.g. 1-4) and pooled responses (e.g. often/very often) as another means to validate any differences in an item over time.
Removed 2004 and repeated the analyses. No additional meaningful differences noted.
With this large number of cases, occurrence of significant differences by chance alone are highly probable.
Reviewed items for meaningful differences if statistically significant.
14
Item consistency over time: Summary
Differences in items over administrations were minimal.
Fluctuations in scores rather than progressive pattern of changed was noted for some items.
Responses from seniors were remarkably stable NSSE developers indicate that unless deliberate
and systemic university-wide interventions were made, differences should be minimal.
15
Two interesting items
E-mail as a method for communication with faculty Increased from 49% in 2004 to 72% In 2009 as reported by
freshmen. No differences for seniors
University encouraging students to vote quite a bit or very much. Increases were noted in the two presidential election years i.e.
2005 and 2009. In 2005 and 2009, freshmen reported encouragement at 30 %
and 37% respectively but only 18-19% in other years. In 2005 and 2009, seniors reported encouragement at 24% and
27% respectively but only 18-19% in other years.
1. Sample versus total seniors
2. Characteristics of Students for sample versus total seniors
Demographics
17
Comparison of Sample and Total Enrollment by College (N=2173)
College/Division Sample University*
Business 20% 20%
Education 14% 13%
Engineering 13% 11%
Fine Arts 7% 7%
Health Professions 11% 12%
LAS - Humanities 5% 4%
LAS - Math/Science 8% 7%
LAS - Social Science 21% 20%
LAS - Other 2% 2%
*Fall 2008
18
Demographics of Seniors
Sample WSU Comment
>=24 yrs 53% 56% No significant difference among the colleges.
Women 64% 56% Clear gender differences. Women predominate in HP(90%), ED(80%) & LAS SS(79%). Men predominate in EN(77%) & LAS Math/SC(56%).
Minority 17% 18% Range from 10% and 12% in Education and Fine Arts to 21% and 22% in LAS Social Sciences and LAS Math/Science respectively. Differences are not statistically different (p>.001)
International 6% 6% No significant difference among colleges
Full time 73% 63% All colleges are above 70% full time.
Working(Pay) 83% NA Range form 71% n Education to 89% in Business. All colleges above 70%.
Transfer Hours
88% 87% No difference among colleges
A few Individual Items
20
Rating for Academic Advising and Satisfaction Entire Educational Experience
Interrelated items
22
Benchmarks represent Areas of Effective Practice
Level of Academic Challenge
Active & Collaborative
Learning
Enriching Educational Experiences
Student – Faculty
Interaction
Supportive Campus
Environment
23
Scalelets versus Benchmarks
• Scalelets include a small number of items related to specific concept that have empirically validated. (Pike,2002,2006)
• 14 scalelets with 12 as subsections of the benchmarks
• Advantages over benchmarks
– More specific and meaningful concepts– More congruence between items and concept
• Benchmark and scalelet scores are converted to 0-100 scale.
• The mean for the items in each scalelet and benchmark for each student were calculated if he/she answered 3/5 or 60% of items
Benchmark: Academic Challenge
Scalelets: Course Challenge and Writing
Academic Challenge
25
Level of Academic Challenge: Items
1. Preparing for class (studying, reading, writing, doing homework or lab work, etc. related to academic program)
2. Number of assigned textbooks, books, or book-length packs of course readings3. Number of written papers or reports of 20 pages or more4. Number of written papers or reports of between 5 and 19 pages 5. Number of written papers or reports of fewer than 5 pages6. Coursework emphasizes: Analysis of the basic elements of an idea, experience
or theory 7. Coursework emphasizes: Synthesis and organizing of ideas, information, or
experiences into new, more complex interpretations and relationships8. Coursework emphasizes: Making of judgments about the value of information,
arguments, or methods9. Coursework emphasizes: Applying theories or concepts to practical problems or
in new situations 10. Working harder than you thought you could to meet an instructor's standards or
expectations11. Campus environment emphasizes: Spending significant amount of time studying
and on academic work.
26
Level of Academic Challenge: Mean Scores
27
Course challenge
How often…Worked harder than you thought you could to meet an instructor's standards or expectations WORKHARD
How often…Come to class without completing readings or assignments CLUNPREP(reverse)
To what extent have your examinations during the current school year challenged you to do your best work. EXAMS
How many hours per week spent…Preparing for class (studying, reading, writing, doing homework or lab work, analyzing data, rehearsing, and other academic activities) ACADPR01
To what extent does your institution emphasize spending significant amounts of time studying and on academic work
ENVSCHOL
28
Writing
How often have you...Prepared two or more drafts of a paper or assignment before turning it in
REWROPAP
How often have you...Worked on a paper or project that required integrating ideas or information from various sources
INTEGRAT
In current school year…Number of written papers or reports of 20 pages or more WRITEMOR
In current school year…Number of written papers or reports between 5 and 19 pages WRITEMID
In current school year…Number of written papers or reports of fewer than 5 pages WRITESML
29
Higher-Order thinking Skills
Memorizing facts, ideas, or methods from your courses and readings so you can repeat them in pretty much the same form
MEMORIZE(reverse)
Analyzing the basic elements of an idea, experience, or theory, such as examining a particular case or situation in depth and considering its components
ANALYZE
Synthesizing and organizing ideas, information, or experiences into new, more complex interpretations and relationships
SYNTHESZ
Making judgments about the value of information, arguments, or methods, such as examining how others gathered and interpreted data and assessing the soundness of their conclusions
EVALUATE
Applying theories or concepts to practical problems or in new situations
APPLYING
30
Academic Challenge: Related Scalelets
Collaborative Learning
Active Learning
Active and Collaborative Learning
32
Active & Collaborative Learning
1. Asked questions in class or contributed to class discussions
2. Made a class presentation3. Worked with other students on projects during class4. Worked with classmates outside of class to prepare
class assignments5. Tutored or taught other students (paid or voluntary)6. Participated in a community-based project (e.g.,
service learning) as part of a regular course7. Discussed ideas from your readings or classes with
others outside of class (students, family members, co-workers, etc.)
33
Active Learning
How often have you...Asked questions in class or contributed to class discussions
CLQUEST
How often have you..Made a class presentation CLPRESEN
How often have you..Participated in a community-based project (e.g. service learning) as part of a regular course
COMMPROJ
34
Collaborative Learning
How often have you...Worked with other students on projects during class
CLASSGRP
How often have you...Worked with classmates outside of class
OCCGRP
How often have you...Tutored or taught other students (paid or voluntary)
TUTOR
How often have you...Discussed ideas from your readings or classes with others outside of class (students, family members, co-workers, etc.)
OOCIDEAS
35
Active and Collaborative Learning
Course Interaction
Out of Classroom Interaction
Faculty/Staff – Student Interaction
37
Course Interaction
How often have you...Discussed grades or assignments with an instructor
FACGRADE
How often have you..Discussed ideas from your readings or classes with faculty members outside of class
FACIDEAS
How often have you..Received prompt written or oral feedback from faculty on your academic performance
FACFEED
38
Out-of-Class Interaction
How often have you...Talked about career plans with a faculty member or advisor
FACPLANS
How often have you...Worked with faculty members on activities other than coursework (committees, orientation, student life activities, etc.)
FACOTHER
How often have you...Work on a research project with a faculty member outside of course or program requirements
RES
39
Course Interaction and Outside Class Interaction
Varied Experiences
Information Technology
Diversity
Enriching Educational Experiences
41
Varied Experiences(Part I)
Have you or plan to do ... A practicum, internship, field experience, co-op experience, or clinical assignment
INTERN04
Have you, or plan to.. Community service or volunteer work
VOLNTR04
Have you, or plan to.. Participate in a learning community or some other formal program where groups of students take two or more classes together
LRNCOM04
Have you, or plan to.. Foreign language coursework
FORLNG04
Have you or do you plan to.. Study abroad
STDABR04
42
Varied Experiences(Part II)
Have you, or plan to.. Independent study or self-designed major
INDSTD04
Have you, or plan to.. Culminating senior experience (capstone course, senior project or thesis, comprehensive exam, etc.)
SNRX04
How many hours per week...Participating in co-curricular activities (organizations, campus publications, student government, fraternity or sorority, intercollegiate or intramural sports, etc.)
COCURR01
To what extent does your institution emphasize… Attending campus events and activities (special speakers, cultural performances, athletic events, etc.)
ENVEVENT
43
Technology
How often have...Used an electronic medium (listserv, chat group, Internet, instant messaging, etc.) to discuss or complete an assignment
ITACADEM
How often have.. Used e-mail to communicate with an instructor
to what extent does your institution emphasize...Using computers in academic work
ENVCOMPT
44
Diversity
How often have you...Had serious conversations with students of a different race or ethnicity than your own
DIVRSTUD
How often have you...Had serious conversations with students who are very different from you in terms of their religious beliefs, political opinions, or personal values
DIFFSTU2
To what extent has your institution encouraged contact among students from different economic, social, and racial or ethnic backgrounds
ENVDIVRS
45
Varied Experiences, Technology, Diversity
Support for Student Success
Interpersonal Environment
Supportive Campus Environment
47
Support for Student Success
To what extent does your institution emphasize… Providing the support you need to help you succeed academically
ENVSUPRT
To what extent does your institution emphasize...Helping you cope with your non-academic responsibilities (work, family, etc.)
ENVNACAD
To what extent does your institution emphasize...Providing the support you need to thrive socially
ENVSOCAL
48
Interpersonal Environment
Relationships with other students ENVSTU
Relationships with faculty members ENVFAC
Relationships with administrative personnel and offices
ENVADM
49
Support for Student Success and Supportive Interpersonal Environment
Practical Skills
General Education
Outcome Measures
51
Gains in Practical Skills
To what extent does your institution emphasize...Using computing and information technology
GNCMPTS
To what extent does your institution emphasize...Analyzing quantitative problems
GNQUANT
To what extent does your institution emphasize...Acquiring job or work-related knowledge and skills
GNWORK
52
Gains in General Education
To what extent does your institution emphasize...Writing clearly and effectively
GNWRITE
To what extent does your institution emphasize...Speaking clearly and effectively
GNSPEAK
To what extent does your institution emphasize...Thinking critically and analytically
GNANALY
To what extent does your institution emphasize...Acquiring a broad general education
GNGENLED
53
Practical Skills and General Education
54
Thank You