assessing the citizenship islo: spring 2014 · citizenship islo 1) demonstrate scientific literacy...
TRANSCRIPT
![Page 1: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/1.jpg)
Assessing the
Citizenship ISLO:
Spring 2014
FACILITATOR: KAREN WONG
WITH MUCH GRATITUDE TO THE PARTICIPATING INSTRUCTORS
SOURCE: SKYLINE COLLEGE OFFICE OF PLANNING, RESEARCH, AND INSTITUTIONAL EFFECTIVENESS
![Page 2: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/2.jpg)
Citizenship ISLO 1) DEMONSTRATE SCIENTIFIC LITERACY CONCERNING A RANGE OF
GLOBAL ISSUES.
2) ARTICULATE SIMILARITIES AND CONTRASTS AMONG CULTURES,
DEMONSTRATING KNOWLEDGE OF AND SENSITIVITY TO VARIOUS
CULTURAL VALUES AND ISSUES.
3) DEVELOP ATTITUDES CENTRAL TO LIFELONG LEARNING: OPENNESS,FLEXIBILITY, INTELLECTUAL CURIOSITY, AND A BROAD PERSPECTIVE THAT
VALUES DIVERSITY OF THOUGHT.
4) DEMONSTRATE APPROPRIATE SOCIAL SKILLS IN GROUP SETTINGS,
LISTENING AND BEING RECEPTIVE TO OTHERS’ IDEAS AND FEELINGS,
EFFECTIVELY CONTRIBUTING IDEAS, AND DEMONSTRATING LEADERSHIP
BY MOTIVATING OTHERS.
5) DEMONSTRATE COMMITMENT TO ACTIVE CITIZENSHIP.
![Page 3: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/3.jpg)
Assessment Strategy
• Eighteen faculty across the disciplines whose
courses mapped to at least two of the five
descriptors participated.
• Faculty evaluated 523 students’ work– a major
assignment, performance, or activity-- using the
relevant parts of the rubric.
• The PRIE office aggregated and disaggregated
results for discussion.
![Page 4: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/4.jpg)
Citizenship Rubric: Novice, Basic, Proficient, Advanced
• Scientific Literacy
• Scientific Method
• Understanding of the impact of decisions made by local, national and international
organizations, societies, environments and economies
• Understanding of the impact of the global economy on political decision-making
• Understanding of the interconnectedness of the world
• Participation in a global society
• Cultural Sensitivity
• Lifelong Learning
• Social Skills
• Social Interaction
• Participation
• Teamwork
• Leadership
• Active Citizenship
![Page 5: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/5.jpg)
Rubric Example: Demonstrate
commitment to active citizenship.
Indicator □ Novice □ Basic □ Proficient □ Advanced
Demonstrate commitment to active citizenship via civic engagement: having a sense of responsibility to the larger community
No commitment- student may be aware of issues but does not participate in active citizenship.
Minimal commitment-student is aware of issues; may make conscious choices and change own daily habits.
Adequate commitment-student demonstrates “basic” skill level behaviors; in addition, informs and attempts to persuade family and friends to make similar changes and/or collaborates with others to enact change.
Full commitment-student demonstrates “proficient” skill level behaviors; in addition, adopts a leadership role in enacting change.
![Page 6: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/6.jpg)
EIGHTEEN PARTICIPATING
DISCIPLINE FACULTY
Communications (3)
English (3)
English for Speakers of Other Languages (1)
World Languages (1)
Kinesiology– Varsity
Teams (2)
Administration of Justice (1)
History (1)
Paralegal Studies (1)
Political Science (1)
Biology (2)
Physics (1)
Surgical Technology (1)
Telecommunications (1)
![Page 7: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/7.jpg)
SAMPLE SIZE
REPRESENTATION
3%
53%22%
22%
Kinesiology Language Arts Social Sciences Science/ Technology
![Page 8: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/8.jpg)
Assignments/ Activities Activity
Students worked with a partner and then reflected on what they learned about their partner’s culture, and which intercultural communication competency skills might help them/did help them through this process.
Major Writing Assignment
Students researched and wrote about ecological issues; the psychological, political, social and economic contexts of crime; the causes of entrenched poverty in a third world country, etc.
Students reflected on how their personal actions and food consumption affect the world around them.
Performance and Reflection
Students who play team sports; participate in science labs; are on site for clinical observations were evaluated and/or evaluated each other’s social skills.
Exams
Students answered multiple choice test questions on their legal rights and the courts, and indicated whether they knew the information prior to taking the course.
![Page 9: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/9.jpg)
Guiding Questions
• WHICH OF THE DESCRIPTORS ARE MOST PREVALENT IN
THE COURSES THAT WERE ASSESSED?
• HOW ARE STUDENTS FARING WITH EACH?
• WHAT ARE THE IMPLICATIONS OF THESE FINDINGS?
![Page 10: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/10.jpg)
Which of the rubric descriptors are most prevalent in the
courses that were assessed?
10783
63
154179
316
234
287 287303
284
192
0
50
100
150
200
250
300
350
Scientific Method (SM) SM- Impact of Decisions SM- Impact of Global Economy
SM- World Interconnectedness SM- Global Participation Cultural Competency
Lifelong Learning Social Skills (SS) - Interaction SS- Participation
SS- Teamwork SS- Leadership Active Citizenship
![Page 11: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/11.jpg)
How are students faring?
Mastery Which criteria have the highest number of “advanced” and “proficient” scores?
Challenge(s) Which criteria have the highest number of “novice”scores?
Insights?
Concerns?
Questions?
![Page 12: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/12.jpg)
20% 18%13%
17%11%
33%26% 28%
20% 18%24%
11%
36% 40%
35%
41%
36%
43%
43%46%
40%49%
47%
48%
35%37%
46%34%
45%
19%
20%18%
22%
27% 24%
34%
9%5% 6% 8% 7% 5%
10% 8%
19%
5% 6% 7%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Overall Results
Advanced Proficient Basic Novice
![Page 13: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/13.jpg)
How are students faring
with social skills?
3326 28
20
43
4346
40
1920
18
22
510 8
19
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Social Interaction Participation Teamwork Leadership
Advanced Proficient Basic Novice
![Page 14: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/14.jpg)
Equity–
How are different groups faring?
What are the various levels of mastery for
each group?
Who, if anyone, appeared to be
disproportionately impacted?
What may account for different group’s
outcomes?
![Page 15: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/15.jpg)
Disaggregated by Gender
2013/14
Demographics
Citizenship ISLO
Demographics
Difference
Female 51 % 53 % + 2 %
Male 47 % 43 % - 4 %
Unreported 2 % 4 % + 2 %
![Page 16: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/16.jpg)
Disaggregated by Gender
22%16%
27%
37%43%
41%
19%
31%
30%
22%
11%
2%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Female (1434) Male (1151) Other/ Unreported (103)
Advanced Proficient Basic Novice
![Page 17: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/17.jpg)
Disaggregated by Ethnicity
2013/14
Demographics
Citizenship ISLO
Demographics
Difference
African American 3 % 3 %
Asian/ Pacific Islander 21 % 23 % +2 %
Filipino 17 % 18 % +1 %
Hispanic 18 % 18 %
White 19 % 12 % - 7 %
Multi-ethnic 18 % 16 % - 2 %
Unreported 3 % 5 % + 2 %
![Page 18: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/18.jpg)
Disaggregated by Ethnicity
17%20%
11%
22%18%
23%
30%25%
67%
40%54%
45%
43%
42%
49%
38%
17%
27%
29% 26%
29%27%
21%
30%
0%
12%6% 7% 10% 8%
0%
7%
0%
20%
40%
60%
80%
100%
120%
American Indian
(6)
Asian/ Pacific
Islander (576)
African
American (82)
Filipino (462) Hispanic (469) Multi-ethnic (406) White (302) Other/
Unreported (124)
Advanced Proficient Basic Novice
![Page 19: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/19.jpg)
Disaggregated by Age
2013/14
Demographics
Citizenship ISLO
Demographics
Difference
Under 18 6 % 1 % - 5 %
18-22 53 % 67 % +14 %
23-28 23 % 18 % - 5 %
29-39 11% 8 % -3 %
40-49 4 % 3 % -1%
50-59 2 % 1% -1%
60+ 1% 3 % +2 %
![Page 20: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/20.jpg)
Disaggregated by Age
31%
21%15%
27%
35%39%
32%
69%
40%49%
60% 46%
30% 43%
0%
29%28%
13%
11%
26%
24%
0%
10% 8%
0%
8%4%
1%
0%
20%
40%
60%
80%
100%
120%
Under 18 (13) 18-22 (1634) 23-28 (429) 29-39 (206) 40-49 (63) 50+ (23) Other/ Unreported
(79)
Advanced Proficient Basic Novice
![Page 21: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/21.jpg)
Positive Findings (from 9/22/14 Town Hall)
• ASSESSED STUDENTS ARE SOCIALLY COMPETENT.
• FEMALES ARE MORE HIGHLY REPRESENTED ON OPPOSITE ENDS OF
THE SPECTRUM, “ADVANCED” AND “NOVICE.” ON THE OTHER
HAND, ALL THREE GENDER CATEGORIES (INCLUDING
“OTHER/UNREPORTED”) ARE ABOUT EVENLY SPLIT BETWEEN
“NOVICE AND BASIC” AND “PROFICIENT AND ADVANCED.”
• INSTEAD OF ONLY FOCUSING ON ASSESSMENT AS THE END
PRODUCT, WE’RE MOVING IN THE DIRECTION OF USING THE
SLOAC AS A MEANS TO HAVE MORE DISCUSSIONS AROUND
CURRICULUM AND TEACHING PRACTICES, THE “INPUT” THAT
ENABLES STUDENTS TO PRACTICE AND DEVELOP THESE
COMPETENCIES.
![Page 22: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/22.jpg)
Areas for Additional Dialogue(from 9/22/14 Town Hall)
• HIGHEST PERCENTAGE OF “NOVICES” WAS UNDER
“LEADERSHIP.”
• THE RESULTS MAY BE DUE TO THE TYPES OF ASSESSMENTS AND/OR CONTEXT OF
LEADERSHIP BEING ASSESSED.
• LOWEST LEVELS OF “MASTERY,” AND HIGHEST LEVELS OF
“BASIC/NOVICE” ALL INVOLVE GLOBAL CRITERIA.
• PERHAPS GLOBAL TOPICS ARE NOT REINFORCED ENOUGH IN OUR
CURRICULUM.
• POSSIBLY THE AGE OF THE STUDENTS ALSO FACTORED INTO THE RESULT. THE
VAST MAJORITY ARE 18- 22 AND MAY NOT HAVE BEEN SUFFICIENTLY
EXPOSED TO GLOBAL TOPICS.
• ALSO THERE WERE LOW LEVELS OF REPRESENTATION OF THOSE AREAS IN THE
COURSES SAMPLED.
![Page 23: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/23.jpg)
Potential Action Plans (from 9/22/14 Town Hall)
LEADERSHIP CRITERION
Professional development-- May want to initiate more conversation about how these ISLO competencies manifest themselves within our disciplines, and techniques for both teaching and assessing each ISLO component.
GLOBAL CRITERIA
May want to offer more professional development about how we can incorporate global topics into our curriculum.
May want to launch a “One [Global] Book” initiative such that students study a common book from different disciplinary perspectives.
May want to integrate more service learning opportunities into classes.
May want to disaggregate the data according to the number of units students have taken to see if there’s a difference between the two groups (e.g., 36+ units and those with fewer).
Is there a better way to select courses so that representation of the areas is better balanced? If more courses had this as a focus, the results on the rubric might be different.
May want to prompt people to review their mapping, which could help to improve the sampling of courses used for ISLO assessments.
![Page 24: Assessing the Citizenship ISLO: Spring 2014 · citizenship islo 1) demonstrate scientific literacy concerning a range of global issues. 2) articulate similarities and contrasts among](https://reader035.vdocuments.site/reader035/viewer/2022071109/5fe475c456fa0108d4761319/html5/thumbnails/24.jpg)
Action Plan: What might you do in
response to these ISLO findings?
How is this information pertinent to your discipline/ area?
What can you do to reinforce or improve learning?
Revise how you convey the information?
Revise the assignment/ task?
Provide more opportunities for practice and feedback?
Revise assessment methodology?
Revise ISLO descriptors?