indiana state university assessment of general education objectives using indicators from national...
TRANSCRIPT
Indiana State University
Assessment of General Education Objectives Using Indicators From National Survey of
Student Engagement (NSSE)
Introduction to NSSE•Used nation-wide to assess how well students are engaged in their studies, with other students, and the outside community
•Used at Indiana State University as a method of assessment for planning, assessing student engagement, retention research, etc.
•Contains questions relevant to general education, especially to the objectives contained in the catalog
•Critical Thinking
•Communication Skills
•Issues of Value and Belief
•Life-long Learning
Contained within each of these elements are specific aspects of students’ learning experience in general education that can be assessed; specifically:
Critical Thinking:
• Developing Students’ Capacities for Independent Thinking
• Developing Students’ Capacities for Critical Analysis
• Developing Student’ Capacities for Reasoned Inquiry
Communication Skills
• Enhancing Students’ Writing Abilities
• Enhancing Students’ Speaking Abilities
• Enhancing Students’ Reading Abilities
• Enhancing Students’ Listening Abilities
Issues of Value and Belief
• Enhancing Students’ Capacities for Making Informed and Reasonable Choices
Life-Long Learning
• Helping Students Develop Knowledge and Skills That Encourage Participatory Citizenship
• Helping Students Develop Knowledge and Skills That Acknowledge the Value of Learning
• Helping Students Develop Knowledge and Skills That Facilitate Adaptation to Change
Purpose of This Presentation
• To view some indicators within the NSSE that inform the general education committee to what extent learning objectives are being met;
• To enable committee members to discuss what additional indicators (i.e, direct assessments) might lead to continued assessment of these objectives;
• To enable committee members to discuss the validity of specific questions on the NSSE when used to measure objectives;
• To empower committee members with information that will lead to focused action to address strengths and weaknesses discovered through assessment of general education objectives
Critical Thinking
Developing Students’ Capacities for Independent Thinking
Independent Thinking and In-Class Experiences
2.763
1.76
2.43
3.183.2
2.05
2.56
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
Asked Questionsin Class orContributed toDiscussions
Integrated Ideasin Class Paper orProject
Discussed ClassReadings withFaculty Member
Worked HarderThan Thought
1=never 2=sometimes 3=often 4=very often
•The score improved between freshmen and seniors in each category
•The highest score had to do with integration of ideas into class papers or projects; freshmen reported that they did this often
•For the question, “Asked questions in class…”, the average senior score was significantly lower than those at other doctoral intensive universities and other colleges and universities who participated in the NSSE
Independent Thinking and Cognitive Activities
2.812.96
2.78
3.09
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
MakingJ udgementsAbout the Valueof Information
Applying Theoriesor Concepts toPracticalProblems or inNew Situations
1=very little 2=some 3=quite a bit 4=very much
•Freshmen report that they apply theories and make judgments on the higher end between “some” and “quite a bit.”
•Seniors report that they engage in both “quite a bit”
•However, senior responses are significantly lower than those for other NSSE institutions
Critical Thinking and Independent Thinking
2.85
2.5
2.99
2.48
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
LearningEffectively OnYour Own
Developing aPersonal Code ofValues and Ethics
1=very little 2=some 3=quite a bit 4=very much
•Freshman and seniors rated both questions between “some” and “quite a bit”
•Senior responses are lower than freshman responses on both question items
•Senior responses are significantly lower than those of other institutions participating in the NSSE.
Critical Thinking
Developing Students’ Capacities for Critical Analysis
2.98
2.73
3.11
2.572.47
3.1
2.81
3.19
2.742.58
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
Analyzing BasicElements of AnIdea
Synthesizing andOrganizing Ideas
ThinkingCritically andAnalytically
AnalyzingQuantitativeProblems
Solving ComplexReal-WorldProblems
1=very little 2=some 3=quite a bit 4=very much
•Senior responses are higher than Freshmen responses on all question items
•On question elements having to do with thinking critically and analytically, both freshmen and seniors rate them as “quite a bit.”
•The question item having to do with solving real-world problems was rated the lowest by both freshmen and seniors
•Freshman responses are significantly lower for all institutions participating in the NSSE and all comparable doctoral intensive universities for analyzing basic elements of an idea, synthesizing and organizing ideas, and analyzing quantitative ideas
•Senior responses are significantly lower for all institutions participating in the NSSE and all comparable doctoral intensive universities for all questions in this category except for solving complex real-world problems.
Critical Thinking
Developing Students’ Capacities For Reasoned Inquiry
2.47
2.71
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
Putting TogetherConcepts FromDifferent CoursesWhen CompletingAssignments ofDuring ClassDiscussions
1=never 2=sometimes 3=often 4=very often
•Senior responses higher than freshmen responses
•Response average between “sometimes” and “often”
•Senior responses are significantly lower than comparable doctoral-intensive universities and for all institutions participating in the NSSE.
3.11 3.19
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
ThinkingCritically andAnalytically
1=very little 2=some 3=quite a bit 4=very much
•Average responses are between “quite a bit” and “very much”
•Senior responses are significantly lower for comparable doctoral-intensive universities and all universities participating in NSSE
Communication Skills
Enhancing Students’ Writing Abilities
Preparing and Revising Papers
2.55 2.47
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
Prepared 2+Drafts of Paperor AssignmentBefore Turning In
1=never 2=sometimes 3=often 4=very often
•Average responses lie between “sometimes” and “often”
•For freshmen, average responses are significantly lower than all other institutions participating in the NSSE.
Number of Written Papers
1.24
2.37
3.45
1.55
2.4
3
0
0.5
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
Number ofWritten Papers orReports of 20Pages or More
Number ofWritten Papers orReports Between5 and 19 Pages
Number ofWritten Papers orReports of FewerThan 5 Pages
1=none 2=between 1 and 4 3=between 5 and 10, 4=between 11 and 20, 5=more than 20
•Freshmen rarely wrote papers of twenty pages or more; they wrote between 1-4 papers of five to nineteen pages during their current school year, and between 5 to 10 papers of less than five pages in length
•The average response for freshmen writing papers of less than 5 pages was significantly higher than other doctoral intensive universities and all other universities participating in the NSSE
•Seniors wrote few papers of twenty pages or longer. They wrote between 1-4 papers of 5-19 pages during their current school year, and between 5-10 papers of less than five pages in length
•The average response for seniors reporting the number of assignments by paper length was significantly lower for papers of more than twenty and between 5 and 19 pages in length for doctoral intensive universities and all colleges and universities participating in the NSSE.
To What Extent Has Your Experience At This Institution Contributed To Your Knowledge, Skills, and Personal Development in Writing Clearly and Effectively?
2.95 3.01
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
Writing ClearlyAnd Effectively
1=very little 2=some 3=quite a bit 4=very much
•Both freshmen and sophomores rated this item as “quite a bit”
•There is little difference between the average response rates for either freshmen or sophomores
•The responses are average when compared to comparable doctoral institutions and other colleges and universities participating in the NSSE.
Communication Skills
Enhancing Students’ Speaking Abilities
To What Extent Has Your Experience At This Institution Contributed To Your Knowledge, Skills and Personal Development in Speaking Clearly and Effectively?
2.89 2.86
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
Speaking Clearlyand Effectively
1=very little 2=some 3=quite a bit 4=very much
•Average responses for both freshmen and seniors fell on the higher range between “some” and “quite a bit”
•For freshmen, the average response was significantly higher than doctoral intensive and all institutions choosing to participate in the NSSE
Communication Skills
Enhancing Student Reading Abilities
During the Current School Year, What Was The Number of The Following Assignments?
3.29
1.91
3.07
2.06
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
Number ofAssignedTextbooks,Books, Or Book-Length Packs ofCourse Readings?
Number of BooksRead on YourOwn For PersonalEnjoyment orAcademicEnrichment?
1=none 2=between one and four 3=between 5 and 10 4=between 11 and 20 5=more than 20
•On average, freshmen read between 5-10 assigned texts or course packets during the current school year; they did little reading on their own.
•On average, seniors tended to read between 5-10 assigned texts or course packets during the current school year; they tended to read between 1 and four books for personal or academic enrichment
•For seniors, the number of required readings and reading for personal or academic enrichment was significantly below the averages for comparable doctoral intensive universities and all colleges and universities choosing to participate in the NSSE.
About How Many Hours Do You Spend In A Typical 7-Day Week Preparing For Class?
3.563.86
11.5
22.5
33.5
44.5
55.5
66.5
77.5
8
Freshmen Seniors
Preparation
1=0 hrs. per week 2=1-5 hrs. 3=6-10 hrs 4=11-15 hrs. 5=16-20 hrs. 6=21-25 hrs. 7=26-30 hrs. 8=more than 30 hrs.
•The average response for both freshmen and sophomores indicate that both spend between 6-10 hrs. per week preparing for class.
•For seniors, the average response is significantly lower than seniors at colleges and universities choosing to participate in the NSSE
•For freshmen, the average response is significantly lower than freshmen and comparable doctoral-intensive colleges and universities, and for freshmen at colleges and universities choosing to participate in the NSSE.
Communication Skills
Enhancing Students’ Listening Abilities
2.492.34
1.76
2.58 2.512.6
2.05
2.74
0
0.5
1
1.5
2
2.5
3
Freshmen Sophomores
Worked WithOther StudentsOn ProjectsDuring Class
Worked WithClassmatesOutside of ClassTo Prepare ClassAssignmentsDiscussed IdeasFrom Readings OrClasses WithFaculty
Discussed IdeasFrom Readings ofClasses WithOthers Outside ofClass
•Average response for freshmen indicate that working with students on projects was significantly higher when compared to freshmen at comparable doctoral-intensive universities and all colleges and universities choosing to participate in NSSE.
•Average response for freshmen also indicate that discussing ideas from readings was significantly lower when compared with freshmen at colleges and universities choosing to participate in the NSSE.
•Average response rates seniors indicate a significantly lower response for working with classmates outside of class when compared to seniors as comparable doctoral-intensive colleges and universities and to all colleges and universities participating in the NSSE
•Average response for seniors were also significantly lower for discussing ideas from readings with others when compared to seniors at colleges and universities choosing to participate in the NSSE.
Issues of Value and Belief
Enhancing Students’ Capacities For Making Informed and
Reasonable Choices
2.81
2.21
2.47
2.78
1.79
2.58
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
MakingJ udgments Aboutthe Value ofInformation
Helping You CopeWith Non-AcademicResponsibilies
Solving ComplexReal-WorldProblems
•In respect to making judgments about the value of information, both freshmen and seniors rated this item on the high end between “some” and “quite a bit”
•On this item, the average response for seniors was significantly below the average response rates to other colleges and universities choosing to participate in the NSSE
•In respect to helping students cope with non-academic responsibilities, the average response for seniors is lower than that of freshmen.
•The freshmen average response is significantly higher than the response rate for comparable doctoral-intensive colleges and universities
•In respect to solving complex, real-world problems, the average response is around the mid-point between “some” and “quite a bit”
Life-Long Learning
Helping Students Develop Knowledge and Skills That Encourage Participatory Citizenship
Encouraging Participatory Citizenship: Course-Related Indicators
1.541.591.72
1.87
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
Participate inCommunity-BasedProjects As Partof A Course
Worked WithFaculty OnActivities OtherThan Coursework
•1=never 2=sometimes 3=often 4=very often
•In respect to participating in community-based project as part of a regular course, both freshman and senior responses were significantly higher than those of students at comparable doctoral-intensive colleges and universities
•In respect to working with faculty members on activities other than coursework, senior average responses were significantly higher than those at comparable doctoral-intensive colleges and universities
Encouraging Participatory Citizenship: Diversity
2.64 2.59
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
SeriousConversationsWith StudentsWho AreDifferent
1=never 2=sometimes 3=often 4=very often
•Average responses were between “sometimes” and “often”
•The average responses for freshmen was significantly lower than those for freshman at all other colleges and universities choosing to participate in the NSSE.
Encouraging Participatory Citizenship: Sense of Being Involved
1.71
2.08
1.68
2.27
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
Voting
Contributing tothe Welfare ofYour Community
1=very little 2=some 3=quite a bit 4=very much
•Responses to both questions, for both freshmen and seniors, indicates some or little voting or community activity
•Responses for voting significantly lower for freshmen and seniors for colleges and universities participating in the NSSE.
•The same holds true for “contributing to the welfare of your community”
Life-Long Learning
Helping Students Develop Knowledge and Skills that Acknowledge the Value
of Learning
1.57
2.43
1.89
2.56
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
Tutored OrTaught Students
Worked HarderThan YouThought
1=never 2=sometimes 3=often 4=very often
•Freshman average response was significantly lower than those at comparable doctoral-intensive colleges and universities and all participating in the NSSE. Senior response rates were slightly higher than both.
•Freshman and senior average responses for “worked harder than you thought” were significantly lower than the average response rates for freshmen and seniors at comparable doctoral-intensive colleges and universities and all choosing to participate in the NSSE.
Life-Long Learning
Helping Students Develop Knowledge and Skills that Facilitate Adaptation to
Change
Diversity
2.662.552.64 2.652.542.59
1
1.5
2
2.5
3
3.5
4
Freshmen Seniors
Included DiversePerspectives inClass Discussionsor WritingAssignmentsHad SeriousConversation withStudent Of ADifferent Race
Had SeriousConversationWith StudentDifferent FromYou
1=never 2=sometimes 3=often 4=very often
•Freshmen average response for “Had serious conversations with students different from you” was significantly lower when compared to freshmen and colleges and universities choosing to participate in the NSSE.
External Orientation
0.3
0.21
0.25
0.1
0
0.05
0.1
0.15
0.2
0.25
0.3
Freshmen Seniors
Do You Plan To orHave You TakenCourses InForeign Language
Do You Plan To orHave YouParticipated InStudy Abroad?
•Freshman average response for study abroad are significantly lower than those of freshman at comparable doctoral-intensive colleges and universities and all choosing to participate in the NSSE; the same is true for seniors
•The above comment is also true for foreign language coursework.
Summary—Opportunities For Growth
•Critical Thinking: Room for improvement needed in students being equipped and gaining experience in thinking critically and analytically
•Communication Skills: Need for more completion of longer papers and amount of time students put in to preparing for class. Number and quality of reading assignments also appears to be a concern.
•Values and Belief: Seniors report that they feel less equipped to deal with issues outside of academics (though freshmen rate this highly).
•Life-Long Learning: Feelings of academic challenge are lower than expected, as well as a sense of being involved and exposure to people and issues outside of students’ world view (diversity).
Summary—Positive Findings
• Critical Thinking: Improvement is evident when freshman and senior averages are compared;
• Communication Skills: Ratings for speaking skills are high when compared to other institutions;
• Issues of Value and Belief: Freshman levels are high, suggesting that this issue is addressed early. This may suggest that a strength of general education lies in its efforts to challenge students to engage students whose orientation might be different.
• Life-long Learning: The number and quality of community-oriented projects is high as well as the number of students working with faculty outside of class. This suggests an orientation toward application of knowledge outside of the classroom.
Limitations
•The NSSE compares freshman and senior responses. Can a pre- and post- test be designed to see if differences in freshman/senior responses are significant?
•The general education committee will need to discuss issues of validity with each of the NSSE questions used for this analysis. Are some questions more effective in assessing general education objectives than others?
•NSSE questions are a good source of student satisfaction and reports regarding specific educational activities (that do not require the creation of additional assessments—it is “non-invasive” in a sense). What direct measures can be used to measure general education objectives?
•Given the above discussions, what actionable goals can be set to improve on strengths and to address weaknesses discovered through this analysis?
Next Steps
• Discuss NSSE questions and their use to measure GE objectives
• Discuss what direct assessments can be used to assess GE objectives
• Discuss what NSSE questions say about what steps need to be taken to achieve general education objectives.