information literacy 2016-2018 (2) · literacy outcomes and achieving competency in information...

Report prepared by the Office of Institutional Effectiveness Source: Livetext by Watermark, 2019

Core Competency Data Report

PEPPERDINE UNIVERSITY

OFFICE OF INSTITUTIONAL EFFECTIVENESS

2016 – 2018

Information Literacy

Core Competencies: Information Literacy

2

Background

WASC Information on the Five Core Competencies

The five core competencies – writing, oral communication, quantitative reasoning, critical thinking, and information literacy – are critical higher-order intellectual skills for students to develop in order to be successful at school, at work, and in their private and civic lives. Accordingly, WSCUC’s institutional review process calls upon institutions to describe how the curriculum addresses each of these competencies, explain their learning outcomes in relation to the core competencies, and demonstrate the extent to which these outcomes are achieved. The 2013 Handbook of Accreditation, Criteria for Review 2.2a states: “Baccalaureate programs engage students in an integrated course of study of sufficient breadth and depth to prepare them for work, citizenship, and life-long learning. These programs ensure the development of core competencies including, but not limited to, written and oral communication, quantitative reasoning, information literacy, and critical thinking. Component 4 (Educational Quality) of the Institutional Review Process asks for institutions “to describe how the curriculum addresses each of the five core competencies, explain their learning outcomes in relation to those core competencies, and demonstrate, through evidence of student performance, the extent to which those outcomes are achieved.”

Participants in the Assessment of Information Literacy

Information literacy, as defined by AAC&U, is the ability to know when there is a need for information, to be able to identify, locate, evaluate, and effectively and responsibly use and share that information for the problem at hand.

Two of Pepperdine’s schools participated in the assessment of the Information Literacy Core Competency at Pepperdine University between fall 2016 and fall 2018: (1) Seaver College and (2) Graziadio Business School.

Eleven Undergraduate programs from Seaver College participated: (1) Art, (2) French, (3) Psychology, (4) Theater, (5) Art History, (6) Music, (7) Philosophy, (8) Social Work, (9) International Studies and Languages, (10) Nutritional Science, and (11) Business Administration. One hundred forty-seven individual undergraduate students contributed work for the assessment, in addition to the five groups of students from ART 390 who participated. This represents an appropriate sample size of Seaver College students.

One Graduate program and twenty-five students from Graziadio Business School Participated: (1) MBA: Economics.


3

Outcomes

Pepperdine Information Literacy Outcomes, based on the AAC&U Information Literacy Value Rubric Student Will:

1. Determine the Extent of Information Needed: Effectively defines the scope of the research question or thesis. Effectively determines key concepts. Types of information (sources) selected directly relate to concepts or answer research question.

2. Access the Needed Information: Accesses the information using effective well-designed search strategies and most appropriate information sources.

3. Evaluate Information and its Sources Critically: Chooses a variety of information sources

appropriate to the scope and discipline of the research question. Selects sources after considering the importance (to the researched topic) of the multiple criteria used (such as relevance to the research question, currency, authority, audience, and bias or point of view).

4. Use Information Effectively to Accomplish a Specific Purpose: Communicates, organizes and

synthesizes information from sources to fully achieve a specific purpose, with clarity and depth.

5. Access and Use Information Ethically and Legally: Students use correctly all of the following information use strategies (use of citations and references; choice of paraphrasing, summary, or quoting; using information in ways that are true to original context; distinguishing between common knowledge and ideas requiring attribution) and demonstrate a full understanding of the ethical and legal restrictions on the use of published, confidential, and/or proprietary information.

Methodology:

To assess the core competency of Information Literacy across the Institution, assessment leaders asked programs to submit scored rubrics and students’ work using the AAC&U VALUE rubric for Information Literacy. Whenever possible this data was submitted from capstone or senior level courses so that the data reflects student learning at the point of graduation. Scores were entered into LiveText and compiled by assessment staff in the Office of Institutional Effectiveness.

In keeping with best practices at our peer and aspirational schools, as well as following the AAC&U VALUE Rubric for Information Literacy, this project had two parts. The AAC&U rubric design required that we ask students to complete two assignments.


4

Assignment 1: A research journal or diary that culminates in a reflective essay about the research process.

Participants were required to write three to four brief research journal entries throughout the semester reflecting on process. We then asked students to turn those journal entries into a reflective essay that outlined the process that they followed with their research.

The purpose of the journal entries was to give the students solid material from which to draw when writing the reflective essay. The reflective essay was assessed using three dimensions of the information literacy rubric, which align with Pepperdine Information Literacy Outcomes #1 (Determine the Extent of Information Needed), #2 (Access the Needed Information), and #3 (Evaluate Information and its Sources Critically).

Assignment 2: The research paper.

The research paper was program specific. Faculty from each program decided on the student assignment and the research paper could take whatever form was appropriate for each specific discipline. The research paper was assessed using two dimensions of the information literacy rubric, which align with Pepperdine Information Literacy Outcomes #4 (Use Information Effectively to Accomplish a Specific Purpose) and #5 (Access and Use Information Ethically and Legally).

The VALUE Rubric for Information Literacy is recommended for assessing a collection of work, such as research papers, editorials, and speeches, rather than a single work. Evidence of a student’s research process can provide vital context for understanding a student’s information gathering process and further demonstrates a student’s information literacy proficiency.

This assessment covers the work of students from fall 2016 through fall 2018. Faculty instructors assessed an initial sample size of research papers and reflective essays for Seaver College and Graziadio Business School from classes offered in fall 2016 and fall 2017. To provide additional data for the Information Literacy Core Competency assessment, in Fall 2018 three librarians from Seaver College assessed research papers from three Seaver College courses offered in Fall 2017 and Spring 2018.

Each applicable dimension of the rubric was scored on a four-point scale, with 1 = benchmark, 2 and 3 = milestones, and 4 = capstone. Scores in the benchmark area primarily demonstrated a very basic understanding of information literacy and indicated a lack of competency in achieving one of Pepperdine’s five Information Literacy Learning Outcomes. Scores in the capstone area, however, indicated mastery of the material at the level of a graduate student or career professional. To achieve a ranking in the milestone or capstone categories, students needed to demonstrate a progressively complex application of information literacy skills, including defining the scope of a question, applying search strategies, choosing and applying criteria to evaluate information, organizing information effectively, and correctly using multiple information use strategies.


5

Analysis

Overall, the results of the assessment indicate that Pepperdine students are meeting the information literacy outcomes and achieving competency in information literacy. The expected benchmark for information literacy is for 75% of students in a course to reach the milestone (2) level or higher within each learning outcome assessed. Across each of the five dimensions of the VALUE rubric, the average score of the 68 Seaver College students assessed by faculty was at the milestone (3) level. The 25 Graziadio Graduate students scored slightly lower across four of the five dimensions (learning outcomes one through four) when compared to the average scores of Seaver students, but scored higher than Seaver students scored on learning outcome five. The average score of graduate students was at the milestone (3) level or at the upper end of the milestone (2) level. It is difficult to draw conclusions on the scores of graduate students due to the small sample size assessed, as well as the possibility of differing faculty expectations of information literacy competency at the graduate level. Finally, when the average scores of librarian evaluations of seventy-nine Seaver College students are considered, it is evident that all student groups in this assessment achieved nearly the same average scores on outcome four, scoring at the milestone (3) level. It is also worth noting that the average scores of students assessed by librarians on learning outcome five was at least one-half point lower than the other groups’ average scores for this dimension. The average scores of the 79 students assessed by Librarians for outcome five were between the milestone (2) and milestone (3) level, while the average scores of Seaver College students and Graziadio students were at the milestone (3) level or higher. This scoring discrepancy likely reflects the expertise of the librarian evaluators; outcome five focuses on information use strategies, and as information professionals, Pepperdine librarians may be keener in spotting these nuances in research papers. However, the average librarians’ scores for learning outcome four were in line with both the average scores of faculty-assessed Seaver students and graduate students. On average, Seaver students scored highest in learning outcomes one and five. The highest average score for graduate students was also in learning outcome five. Seaver students scored the lowest on outcome two, while graduate students scored the lowest on outcome three. While the lowest average scores for all three groups was above the milestone (2) level of accomplishment, which indicates the groups are meeting expected levels of competency, the outcomes that align with lower scores are those related to accessing needed information for Seaver students and critically evaluating information for Graziadio students.

Demographics (Gender and Ethnicity): Of the 68 students participating in the 2016-2017 assessment, the fall 2016 sample included 10 male students and 22 female students, and the spring 2017 sample included 18 male students and 18 female students. The assessment results were disaggregated by gender for five courses: FRE 380, THEA 311, MUS 355, PHIL 480, and SW 300. Male students in FRE 380 (N = 4) scored significantly lower than female students (N = 5) on both outcomes assessed (outcomes 4 and 5). Likewise, in THEA 311 male students (N = 2) scored lower than female students across all dimensions of the rubric, except for learning outcome #2, in which scores were nearly equal. Differences among gender were less evident in MUS 355, PHIL 480, and SW 300. In MUS 355, male students (N = 5) scored slightly higher than female students (N = 2) on outcomes #1 and #2, while females scored higher on outcomes #3 and #5, and male and female students scored equally on outcome #4. In SW 300, male students (N = 3) and female students (N = 5)


6

scored nearly equally on outcomes #1 and #2, while male students scored higher than female students on outcome #3. In PHIL 480, the largest class assessed that was disaggregated by gender, male students (N = 9) scored higher on outcomes #1 and #3, and female students (N = 8) scored higher on outcome #4, with male and female students scoring nearly equally on outcomes #2 and #5. When these scores are analyzed holistically, the averaged results of this assessment suggest that males score higher on outcomes #1 and #2, females score higher on outcomes #4 and #5, and male and female students score equally on outcome #3. However, due to differing sample sizes and small ranges between scores, this analysis may not apply to the student population as a whole.

Summary of assessment scores by gender and course (f = female students scored higher; m = male students scored higher).

Course

Outcome #1: Determine the Extent of Information Needed

Outcome #2: Access the Needed Information

Outcome #3: Evaluate Information and its Sources Critically

Outcome #4: Use information effectively

Outcome #5: Access and Use Information Ethically and Legally

FRE 380 N/A N/A N/A F F

THEA 311 F M F F F

MUS 355 M M F Equal scores F

PHIL 480 M Equal Scores M F Equal scores

SW 300 Equal Scores Equal Scores M N/A N/A

Regarding ethnicity, the fall 2016 assessment included four non-resident Alien students, eight Hispanic or Latino students, two Asian students, three Black or African American students, thirteen white students, and two students of two or more races. The spring 2017 assessment included six non-resident Alien students, five Hispanic or Latino students, three Asian students, one Black or African American student, eighteen white students, and three students whose Race or Ethnicity was unknown.

Scores were disaggregated by ethnicity in two courses, FRE 380 and SW 300. In FRE 380, white students’ scores nearly equaled those of minority students on learning outcome four and white students scored higher than minority students on learning outcome five. The average scores of both groups were at the milestone (2) level, indicating both groups of students met the competency. It is difficult to draw a conclusion based on ethnicity in this course due to the comparatively small sample size of white students (N= 2). For SW 300, the scores were compared among three non-resident alien students, two minority students, and three white students. Overall, white students scored higher than minority and international students on learning outcomes one, two, and three. White students reached the


7

milestone (3) level, while minority and international students scored at the upper milestone (2) level. International students scored slightly higher than minority students on learning outcomes one and three, while minority students scored higher than international students did on learning outcome two. Due to the small sample sizes of these groups and relatively small range of scores, it is unclear whether there is a statistically significant difference in scores among ethnicity.

Results disaggregated by program:

• ART 390: ART 390 included four assessors and each student was assessed multiple times. The average scores on outcomes #4 and #5 were slightly lower than the average score of Seaver students and the average Librarian evaluation scores. This course met the expected benchmark for outcome #4, with 75% of students scoring at the milestone (2) level or higher. This course did not meet the expected benchmark for outcome #5, as 65% of students scored at the milestone (2) level or higher. This may be attributed to the nature of the assignment or the fact that this was a group project. However, a high percentage of students scored at the capstone level for outcomes #4 (15%) and #5 (25%).

• FRE 380: The average scores on outcomes #4 and #5 were lower than the average score of Seaver College students and the average Librarian evaluation scores. As mentioned above, male students scored significantly lower than female students on both outcomes, and white students scored slightly higher than minority students overall. Students met expected competency levels for both outcomes in this course, with 89% of students achieving at the milestone (2) level or higher for outcome #4, and 78% of students doing so for outcome #5.

• PSYC 310: Students enrolled in PSYC 310 scored higher on outcome #4 and outcome #5 when compared to both the average scores of Seaver Students and average Librarian evaluation scores. Students met the expected benchmarks for this course, and 100% of students scored at the milestone (2) level or higher for outcome #4 and outcome #5. An especially high percentage of students scored at the capstone level; 44% reached the capstone mark on outcome #4, and 94% of students reached the capstone level on outcome #5. This high level of capstone achievement may reveal an issue with applying the rubric, as discussed below.

• THEA 311: Across all five outcomes, students enrolled in THEA 311 scored lower than the average scores of Seaver students and average scores of the Librarian evaluation. Female students scored higher than male students in all dimensions except outcome #2, as mentioned above. While the average scores of students in THEA 311 were lower, the course meets the expected benchmarks for information literacy, with the exception of outcomes #3 and #4. In these two areas, students scored just slightly below the expected benchmark of 75%. For outcome #1, 86% of students scored at the milestone (2) level or higher. 100% of students met did so for outcome #2. 72% of students met the expected benchmark for outcome #3. 71% of students met the expected benchmark for outcome #4. 86% of students met the expected


8

benchmark for outcome #5.

• ARTH 490: Students enrolled in ARTH 490 scored slightly lower than the average score of Seaver College students on outcomes #1 and #2, but higher than the average score of Seaver students and average Librarian evaluation scores on outcomes #4 and #5. Students met the expected benchmarks for this course, and 100% of students scored at the milestone (2) level or higher for all five outcomes.

• MUS 355: Across all five outcomes, students enrolled in MUS 355 scored equally or higher than the average scores of Seaver College Students and average Librarian evaluation scores. Students met the expected benchmarks for this course, and 100% of students scored at the milestone (2) level or higher for all five outcomes. A very high percentage of students achieved at the capstone level for outcomes #3 (43%) and #5 (71%). This distribution of scores at the capstone level may suggest an issue with applying the rubric, as discussed below.

• PHIL 480: Students enrolled in PHIL 480 scored higher across all five dimensions when compared to the average score of Seaver College students and the average Librarian evaluation scores. Students met the expected benchmarks for this course, and 100% of students scored at the milestone (2) level or higher for all five outcomes. More than 40% of students reached the capstone level in each of the five dimensions; 76% of students scored at the capstone level for outcome #5.

• SW 300: Students enrolled in SW 300 scored lower than the average score of Seaver College Students, but 100% of students scored at the milestone (2) level or higher for all three dimensions of the reflective essay rubric.

• ECNM 602 (Graziadio Graduate School): Students enrolled in ECNM 602 met the expected benchmark for information literacy. 96% of students met the benchmark for outcome #1, 84% for outcome #2, 96% for outcome #3, 100% for outcome #4, and 100% for outcome #5. Since this is a graduate level class, a high percentage of students scored at the capstone level, especially in outcome #5 (44%).

• INTS 455 (Librarian evaluation): Students enrolled in INTS 455 scored lower than the average scores of Seaver College and average scores of librarian evaluation. 100% of students met the expected benchmark for outcome #4, and 89% of students met the expected benchmark for outcome #5.

• NUTR 499 (Librarian evaluation): Students enrolled in NUTR 499 scored higher than the average score of Seaver College and librarian evaluation for outcome #4, and lower than the average score of Seaver College, but higher than the average librarian score for outcome #5. Students


9

met the expected benchmarks for this course, 100% of students scored at the milestone (2) level or higher for outcomes #4 and #5.

• BA 497 (Librarian evaluation): Students enrolled in BA 497 scored equally with the average scores of Seaver students on outcome #4 and lower than Seaver students but equally with scores of Librarian evaluations on outcome #5. Students met the expected benchmarks for this course, with 99% of students meeting the expected benchmark for outcome #4, and 98% for outcome #5.

Percentage of students by course who scored at the milestone (2) level or higher for each outcome:

Course

Outcome #1: Determine

the Extent of Information

Needed

Outcome #2: Access the

Needed Information

Outcome #3: Evaluate

Information and its

Sources Critically

Outcome #4: Use

information effectively

Outcome #5: Access and

Use Information Ethically and

Legally

ART 390 N/A N/A N/A 75% 65%

FRE 380 N/A N/A N/A 89% 78%

PSYC 310 N/A N/A N/A 100% 100%

THEA 311 86% 100% 72% 71% 86%

ARTH 490 100% 100% 100% 100% 100%

MUS 355 100% 100% 100% 100% 100%

PHIL 480 100% 100% 100% 100% 100%

SW 300 100% 100% 100% N/A N/A

ECNM 602 96% 84% 96% 100% 100%

INTS 455 N/A N/A N/A 100% 89%

NUTR 499 N/A N/A N/A 100% 100%

BA 497 N/A N/A N/A 99% 98%


10

In summary, ten of the twelve classes in this assessment met the expected competency level for all five learning outcomes. All courses met the expected benchmark level of 75% for at least one of the five outcomes. While ART 390 and THEA 311 did not quite meet all of the expected benchmark levels, these two courses were not far from expectations. It is important to keep in mind the context of class size when reading these numbers. Seven of the twelve courses in this assessment had fewer than ten students, which can magnify the percentage of scores; small class sizes can skew percentages. To put this in perspective, only two students in THEA 311 did not meet the competency for outcomes four and five.

The assessment had a strong sample size of 172 students plus the five ART 390 groups. This analysis indicates the University is meeting outcomes for the Information Literacy Core Competency. This assessment and analysis also indicate there are no notable programmatic areas of concern for this core competency and that the University is meeting these outcomes satisfactorily, exceeding expectations in many cases.

One opportunity for future growth is to broaden program participation in the assessment of information literacy. While this assessment featured strong representation from most of Seaver College, there is a notable absence of participation by Humanities programs, such as English or History. In addition, the assessment lacked representation from the graduate schools.

Overall, the AAC&U Information Literacy Value Rubric aligns well with the Pepperdine Information Literacy Outcomes. An area of concern with the rubric in this assessment, however, is that too many students scored at the capstone level for multiple courses. The expected level of accomplishment for the capstone level is 2% to 5% of students per course; some courses, such as PSYC 310, MUS 355, and PHIL 480, however, saw students reaching the capstone level at a rate of 70% or higher. Since Pepperdine University offers “capstone” level classes, faculty who are assessing these classes may associate the level of course with the level of achievement on the rubric. One way of adjusting this distribution of scores may be to rename the “capstone” level of accomplishment on the rubric and well as to clarify the meaning of this level as it relates to the rubric. While this skew does not affect the overall competency rates of this assessment since these students would have scored at the milestone level regardless, it does raise questions, assessors’ application of the rubric, and the appropriate distribution of accomplishment levels. Assessment staff in OIE may wish to address these issues with faculty prior to the next assessment of information literacy.


11

AAC&U Information Literacy VALUE Rubric - Dimensions Divided by Assignment

Capstone Milestones Benchmark 4 3 2 1

Determine the extent of information needed Reflective Essay

Effectively defines the scope of the research question or thesis. Effectively determines key concepts. Types of information (sources) selected directly relate to concepts or answer research question.

Defines the scope of the research question or thesis completely. Can determine key concepts. Types of information (sources) selected relate to concepts or answer research question.

Defines the scope of the research question or thesis incompletely (parts are missing, remains too broad or too narrow, etc.). Can determine key concepts. Types of information (sources) selected partially relate to concepts or answer research question.

Has difficulty defining the scope of the research question or thesis. Has difficulty determining key concepts. Types of information (sources) selected do not relate to concepts or answer research question.

Access the needed information Reflective Essay

Accesses information using effective, well-designed search strategies and most appropriate information sources.

Accesses information using variety of search strategies and some relevant information sources. Demonstrates ability to refine search.

Accesses information using simple search strategies retrieves information from limited and similar sources.

Accesses information randomly, retrieves information that lacks relevance and quality.

Evaluate information and its sources critically Reflective Essay

Chooses a variety of information sources appropriate to the scope and discipline of the research question. Selects sources after considering the importance (to the researched topic) of the multiple criteria used (such as relevance to the research question, currency, authority, audience, and bias or point of view).

Chooses a variety of information sources appropriate to the scope and discipline of the research question. Selects sources using multiple criteria (such as relevance to the research question, currency, and authority).

Chooses a variety of information sources. Selects sources using basic criteria (such as relevance to the research question and currency).

Chooses a few information sources. Selects sources using limited criteria (such as relevance to the research question).

Use information effectively to accomplish a specific purpose Research Paper

Communicates, organizes and synthesizes information from sources to fully achieve a specific purpose, with clarity and depth

Communicates, organizes and synthesizes information from sources. Intended purpose is achieved.

Communicates and organizes information from sources. The information is not yet synthesized, so the intended purpose is not fully achieved.

Communicates information from sources. The information is fragmented and/or used inappropriately (misquoted, taken out of context, or incorrectly paraphrased, etc.), so the intended purpose is not achieved.

Access and use information ethically and legally Research Paper

Students use correctly all of the following information use strategies (use of citations and references; choice of paraphrasing, summary, or quoting; using information in ways that are true to original context; distinguishing between common knowledge and ideas requiring attribution) and demonstrate a full understanding of the ethical and legal restrictions on the use of published, confidential and/or proprietary information.

Students use correctly three of the following information use strategies (use of citations and references; choice of paraphrasing, summary, or quoting; using information in ways that are true to original context; distinguishing between common knowledge and ideas requiring attribution) and demonstrates a full understanding of the ethical and legal restrictions on the use of published, confidential and/or proprietary information.

Students use correctly two of the following information use strategies (use of citations and references; choice of paraphrasing, summary, or quoting; using information in ways that are true to original context; distinguishing between common knowledge and ideas requiring attribution) and demonstrates a full understanding of the ethical and legal restrictions on the use of published, confidential and/or proprietary information.

Students use correctly one of the following information use strategies (use of citations and references; choice of paraphrasing, summary, or quoting; using information in ways that are true to original context; distinguishing between common knowledge and ideas requiring attribution) and demonstrates a full understanding of the ethical and legal restrictions on the use of published, confidential and/or proprietary information.


12

SEAVER COLLEGE

&

GRAZIADIO GRADUATE SCHOOL

&

LIBRARIAN EVALUATION


13

Combined Average rubric scores by Dimensions for Seaver & Graziadio, Fall 2016, Spring 2017, Fall 2017, Spring 2018

Findings: Seaver students scored slightly higher on all dimensions of the Reflective Essay rubric when compared to Graziadio graduate students. For Research Paper, Seaver students scored slightly higher than Graziadio graduate students and Librarian evaluation on “Use information effectively to accomplish a specific purpose” and Graziadio students had the highest score on “Access and use information ethically and legally” when compared to both Seaver and Librarian Evaluation. The N for Seaver students and Librarian were more than twice as large as Graziadio students.

3.1 2.9 3.0 3.0 3.13.0 2.72.4

2.93.4

2.92.5

0.0

1.0

2.0

3.0

4.0

Determine the extentof information needed

Access the neededinformation

Evaluate informationand its sources

critically

Use information effectively to

accomplish a specific purpose

Access and useinformation ethically

and legally

Seaver College ( N=68 students + 5 groups) Graziadio Graduate ( N=25 students) Librarian Evaluation (N=79 students)

Average Score of Seaver College

( N = 68 students + 5 groups )

Average Scores of Graziadio Graduate ( N = 25

students )

Average Scores of Librarian

Evaluation ( N = 79

students)

Determine the extent of information needed 3.1 3.0 -

Access the needed information 2.9 2.7 -

Evaluate information and its sources critically 3.0 2.4 -

Use information effectively to accomplish a specific purpose

3.0 2.9 2.9

Access and use information ethically and legally 3.1 3.4 2.5

Reflective Essay Research Paper


14

SEAVER COLLEGE Fall 2016 & Spring 2017


15

Table 1. Contributing programs for Seaver College* Fall 2016, Spring 2017

*Note: Overall sample size for Seaver College: 68 students with additional 5 groups of students contributing programs

Not all groups submitted both the reflective essay and research paper

Year Course Instructors/Assessors # of Students

Rubric Dimensions Assessed

Fall 2016

ART 390 Gretchen Batcheller, Timothy Pownall, Joseph Piasentin, Bethany Wilson

5 Groups Research Paper

FRE 380 Mina Soroosh 9 Research Paper

PSYC 310 Jessica Cail 16 Research Paper

THEA 311 Bradley Griffin 7 Reflective Essay & Research Paper

Spring 2017

ARTH 490 Lauren Kilroy – Ewbank 4 Reflective Essay & Research Paper

MUS 355 Gary Cobb 7 Reflective Essay & Research Paper

PHIL 480 Garrett Pendergraft 17 Reflective Essay & Research Paper

SW 300 Emily Scott-Lowe 8 Reflective Essay


16

Table 2. Demographics: 68 Seaver Student Population Included in Assessment (Excl. ART 3901)

Table 3. Demographics: 68 Seaver Student Population Included in Assessment (Excl. ART 390*)

1 Art 390 is a group project , no demographics data

Year Gender FRE 380 PSYC 310 THEA 311 Total

Fall 2016 Male 4 4 2 10 Female 5 12 5 22 Total 9 16 7 32

Year Gender ARTH 490 MUS 355 PHIL 480 SW 300 Total

Spring 2017 Male 1 5 9 3 18 Female 3 2 8 5 18 Total 4 7 17 8 36

Ethnicity Fall 2016 Spring 2017

FRE 380

PSYC 310

THEA 311 Total ARTH

490 MUS 355

PHIL 480

SW 300 Total

Non-resident Alien - 2 2 4 1 1 1 3 6

Hispanic or Latino 6 2 - 8 - 1 3 1 5

Asian - 2 - 2 - 3 - - 3 Black or African American 1 2 - 3 - - - 1 1

White 2 7 4 13 3 1 11 3 18

Two or more Races - 1 1 2 - - - - - Race/Ethnicity Unknown - - - - - 1 2 - 3

Total 9 16 7 32 4 7 17 8 36


17

Group Project Fine Arts Program ART 390 – Fall 2016

Average Scores by Dimension

Average Scores of ART 390

( N=7 groups)




Evaluation ( N = 79 students)

Use information effectively to accomplish a specific purpose 2.5 3.0 2.9


Findings: ART 390 did not submit student Reflective Essays. Their assessment data was performed on their Research Paper using 2 dimensions on the rubric “Use information effectively to accomplish a specific purpose” and “Access and use information ethically and legally”. The students enrolled in ART 390 scored slightly lower on both dimensions when compared to Seaver students and Librarian Evaluation.

2.5 2.4

3.0 3.12.9

2.5

0.0

1.0

2.0

3.0

4.0


Access and use information ethically and legally

Average Scores of ART 390 ( N=5 students)Average Score of Seaver College ( N = 68 students + 5 groups )Average Scores of Librarian Evaluation ( N = 79 students)

Research Paper


18

Group Project Fine Arts Program ART 390 – Fall 2016

ART 390 Scores Overall2

Level of Accomplishment Capstone Milestones Milestones Benchmark

Informat ion not Present

(4 pts) (3 pts) (2 pts) (1 pts) % of students expected

to score in each level 2-5% Between 35- 45% Between 35- 45% under 10-15% 0%

Findings by

Dimension


C ap s to ne s c or es exc eed t he es t a b l i s hed benc h ma r k . S t udent s s c or ed w i t h i n t he b enc hma r k a t t he M i l e s to n e s (3 ) l ev e l . S t udent s s c or ed b e l ow t he b enc hma r k i n t he M i l e s to ne s (2 ) l ev e l . S t udent s ’ s c or es g r ea t l y exc eed t he es t a b l i s hed benc hma r k a t t he B e n c h m ark l ev e l .


S c or es a t t he M i l e s to n e s (3 ) a nd M i l e s to ne s (2 ) l ev e l s ar e be l ow t he b enc hma r k . S t ud ent s ’ s c or es a t t he C ap s to n e a nd B e nc h m ark l ev e l s g r ea tl y exc eed t he es t a b l i s hed benc hma r k .

2 ART 390 includes four assessors. Each student is assessed multiple times.

3 (15%)

5 (25%)

8 (40%)

5 (25%)

4 (20%)

3 (15%)

5 (25%)

7 (35%)


Access and use informationethically and legally

Capstone (4 pts) Milestones (3 pts) Milestones (2 pts) Benchmark (1 pts)

Capstone Milestones Milestones Benchmark

(4 pts) (3 pts) (2 pts) (1 pts) Use information effectively to accomplish a specific purpose 15% 40% 20% 25%

Access and use information ethically and legally 25% 25% 15% 35%


19

French Program FRE 380 – Fall 2016


Average Scores of FRE 380

( N = 9 students)




Evaluation ( N = 79

students) Use information effectively to accomplish a specific purpose 2.4 3.0 2.9


Findings: FRE 380 did not submit student Reflective Essays. Their assessment data was performed on their Research Paper using 2 dimensions on the rubric “Use information effectively to accomplish a specific purpose” and “Access and use information ethically and legally”. The students enrolled in FRE 380 scored lower on both dimensions when compared to Seaver students and Librarian Evaluation.

2.4 2.2

3.0 3.12.92.5

0.0

1.0

2.0

3.0

4.0



Average Scores of FRE 380 ( N=9 students)Average Score of Seaver College ( N = 68 students + 5 groups )Average Scores of Librarian Evaluation ( N = 79 students)

Research Paper


20

FRE 380 – Fall 2016 Average Scores by Gender

Male

( N = 4 students) Female

( N = 5 students)

Use information effectively to accomplish a specific purpose 1.8 3.0

Access and use information ethically and legally 1.5 2.8

Findings: The male students in FRE 380 scored significantly lower in the two dimensions that were assessed when compared to the female students for Fall 2016. For “Use information effectively to accomplish a specific purpose,” male students averaged a 1.8 while the female students’ average was 3.0. And for the dimensions “Access and use information ethically and legally” male students was 1.5 and female students was 2.8.

1.81.5

3.0 2.8

0.0

1.0

2.0

3.0

4.0



Male ( N=4 students) Female ( N=5 students)

Research Paper


21

FRE 380 – Fall 2016 Average Scores by Ethnicity

Findings: The minority students in FRE 380 scored slightly lower in the two dimensions that were assessed when compared to the white students for Fall 2016. For “Use information effectively to accomplish a specific purpose,” minority students averaged a 2.4 while the female students’ average was 2.5. And for the dimensions “Access and use information ethically and legally” minority students was 2.1 and white students was 2.5.

2.42.1

2.5 2.5

0.0

1.0

2.0

3.0

4.0



Minority ( N=7 students) White ( N=2 students)

Minority

( N = 7 students) White

( N = 2 students)



Research Paper


22

FRE 380 – Fall 2016 FRE 380 Scores Overall





Findings by

Dimension


S t udent s s c or ed w i t h i n t he es t a b l i s hed be nc hma r k a t t he M i l e s to ne s (2 ) a nd B e n ch m ar k l ev e l s . C ap s ton e l ev e l sc or es exc eed t he es t a b l i s hed benc hma r k . S t ud ent s ’ s c or es a r e s l ig ht ly be l ow t he be nc hma r k a t t he M i l e s to n e s (3 ) l ev e l .


S t udent s s c or ed w i t h i n t he b enc hma r k a t t he M i l e s to n e s (2 ) l ev e l . S c or es a t t he B e n ch m ar k a nd C ap s ton e l ev el s a r e a bov e t he es t a b l i s hed benc h ma r k . S c or es a t t he M i l e s to n e s (3 ) l ev e l a r e be l ow t he benc h ma r k .

1 (11%)

1 (11%)

3 (33%)

2 (22%)

4 (45%)

4 (45%)

1 (11%)

2 (22%)

Use information effectively toaccomplish a specific purpose



Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)

Use information effectively to accomplish a specific purpose 11% 33% 45% 11%



23

Psychology Program PSYC 310 – Fall 2016


Average Scores of PSYC 310

( N = 16 students)




Evaluation ( N = 79



Findings: PSYC 310 did not submit student Reflective Essays. Their assessment data was performed on their Research Paper using 2 dimensions on the rubric “Use information effectively to accomplish a specific purpose” and “Access and use information ethically and legally”. The students enrolled in PSYC 310 scored higher on both dimensions when compared to Seaver students and Librarian Evaluation.

3.9 3.8

3.0 3.12.92.5

0.0

1.0

2.0

3.0

4.0



Average Scores of PSYC 310 ( N=16 students)Average Score of Seaver College ( N = 68 students + 5 groups )Average Scores of Librarian Evaluation ( N = 79 students)

Research Paper


24

PSYC 310 – Fall 2016 PSYC 310 Scores Overall





Findings by

Dimension


S t udent s ’ s c or es g r ea t l y exc eed t he es t a b l i s hed benc h ma r k a t t he C ap s to n e l ev e l . M i le s to n e s (3 ) l ev e l s c or es a l s o exc eed t he benc h ma r k . Due t o t he h i g h per c ent a g e of s c or es i n t he C ap s to n e a nd M i l e s to ne s (3 ) l ev e l s , s c or es a t t he M i l e s to ne s (2 ) l ev e l a r e f ar be l ow t he es t a b l i s hed benc hma r k , a nd n o s t ud ent s s c or ed a t t he B e n c hm ar k l ev e l .


S c or es a t t he C ap s to n e l ev e l g r ea tl y exc eed t he benc hma r k . S t udent s s c or ed f a r be l ow t he benc h ma r k a t t he M i l e s to n es (3 ) l ev e l . No s t ude nt s c or ed a t t he M i l e s to n e s (2 ) a nd t he B e nc h m ark l ev e l s .

7 (44%)

15 (94%)

8 (50%)

1 (6%)

1 (6%)Use information effectively toaccomplish a specific purpose



Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)

Use information effectively to accomplish a specific purpose 44% 50% 6% 0%



25

Theatre Program THEA 311– Fall 2016


Findings: THEA 311 submitted student Reflective Essays and Research Papers. All 5 dimensions of the rubric were used for assessment. The students enrolled in THEA 311 scored lower on all 5 dimensions when compared to Seaver students and Librarian Evaluation.

2.6 2.42.1 2.1

2.4

3.1 2.9 3.0 3.0 3.12.92.5

0.0

1.0

2.0

3.0

4.0




critically




and legally

Average Scores of THEA 311 ( N=7 students)Average Score of Seaver College ( N = 68 students + 5 groups )Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of THEA 311

( N = 7 students)



Average Scores of Librarian Evaluation

( N = 79 students)

Determine the extent of information needed 2.6 3.1

-

Access the needed information 2.4 2.9 - Evaluate information and its sources critically 2.1 3.0

-





26

THEA 311– Fall 2016 Average Scores by Gender

Findings: The female students in THEA 311 scored higher averages than males students in all dimensions except for “Access the needed information”

2.02.5

1.52.0 2.0

2.82.4 2.4 2.2

2.6

0.0

1.0

2.0

3.0

4.0

Determine theextent of

information needed



critically




and legally


Male


( N = 5 students) Determine the extent of information needed 2.0 2.8 Access the needed information 2.5 2.4 Evaluate information and its sources critically 1.5 2.4 Use information effectively to accomplish a specific purpose 2.0 2.2 Access and use information ethically and legally 2.0 2.6

Research Paper Reflective Essay


27

THEA 311– Fall 2016 THEA 311 Scores Overall

1 (14%)

1 (14%)

1 (14%)

3 (43%)

3 (43%)

3 (43%)

1 (14%)

2 (29%)

2 (29%)

4 (57%)

2 (29%)

3 (43%)

3 (43%)

1 (14%)

2 (29%)

2 (29%)

1 (14%)

Determine the extent of informationneeded

Access the needed information

Evaluate information and its sourcescritically

Use information effectively toaccomplish a specific purpose

Access and use information ethicallyand legallyCapstone (4 pts) Milestones (3 pts) Milestones (2 pts) Benchmark (1 pts)

Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)

Determine the extent of information needed 14% 43% 29% 14% Access the needed information 0% 43% 57% 0% Evaluate information and its sources critically 0% 43% 29% 29% Use information effectively to accomplish a specific purpose 14% 14% 43% 29% Access and use information ethically and legally 14% 29% 43% 14%



(4 pts) (3 pts) (2 pts) (1 pts) % of students expected to score

in each level 2-5% Between 35- 45% Between 35- 45% under 10-15% 0%

Findings by

Dimension

Determine the extent of information needed

S t u d e n t s s c o r e d a b o v e t h e e s t a b l i s h e d b e n c h m a r k a t t h e C a p s t o ne l e v e l . S c o r e s w e r e w i t h i n t h e e s t a b l i s h e d b e n c h m a r k a t t h e M i le s t on e s ( 3) a n d B en c h m a r k l e v e l s . S t u d e n t s s c o r e d b e l o w t h e b e n c h m a r k a t t h e M i le s t on e s ( 2) l e v e l .


S t u d e n t s s c o r e d w i t h i n t h e b e n c h m a r k a t t h e M i le s to ne s ( 3) le v e l . M i le s t on e s ( 2) l e v e l s c o r e s ar e a b o v e t h e e s t a b l i s h e d b e n c h m a r k . N o s t u d e n t s s c o r e d a t t h e B e n c h m a r k a n d t h e C a p s t o ne l e v e l s .

Evaluate information and its sources critically

S t u d e n t s s c o r e d w i t h i n t h e b e n c h m a r k a t t h e M i le s to ne s ( 3) le v e l . M i le s t on e s ( 2) l e v e l s c o r e s ar e b e l o w t h e e s t a b l i s h e d b e n c h m a r k . S c o r e s a t t h e B e n c h m a r k l e v e l g r e a t l y e x c e e d e d t h e e s t a b l i s h e d b e n c h m a r k . N o s t u d e n t s s c o r e d a t t h e C a p s t o n e l e v e l .


S t u d e n t s s c o r e d a b o v e t h e b e n c h m a r k a t t h e C a p s t o n e a n d B e n c h m a r k l e v e l s . T h e s c o r e s a t Mi le s t on e s ( 2) l e v e l ar e w i t h i n t h e e s t a b l i s h e d b e n c h m a r k . S c o r e s a t t h e M il e s to ne s (3 ) a r e be l o w t h e b e n c h m a r k .


T h e s c o r e s a t t h e C a p s t o n e l e v e l e x c e e d t h e e s t a b l i s h e d b e n c h m a r k . S t u d e n t s s c o r e d w i t h i n t h e e s t a b l i s h e d b e n c h m a r k a t t h e B e n c h m a r k a n d t h e M il e st on e s ( 2) l e v e l s . T h e s c o r e s a t t he M i le s t on e s ( 3) l e v e l a r e b e l o w t h e b e n c h m a r k .


28

Art History Program ARTH 490 – Spring 2017


Findings: ARTH 490 submitted student Reflective Essays and Research Papers. All 5 dimensions of the rubric were used for assessment. For Reflective Essay, the students enrolled in ARTH 490 scored slightly lower on “Determine the extent of information needed” and “Access the needed information” and scored equally on “Evaluate information and its sources critically” when compared to Seaver students. While they scored higher on all 2 dimensions of Research Paper when compared to Seaver students and Librarian Evaluation.

3.02.7

3.03.5 3.5

3.1 2.9 3.0 3.0 3.12.92.5

0.0

1.0

2.0

3.0

4.0



Evaluate informationand its sourcescritically


Access and useinformation ethicallyand legally

Average Scores of ARTH 490 ( N= 4 students)Average Score of Seaver College ( N = 68 students + 5 groups )Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of ARTH 490

( N = 4 students)




( N = 79 students)


-


-





29

ARTH 490 – Spring 2017 ARTH 490 Scores Overall

1 (33%)

2 (50%)

2 (50%)

3 (100%)

2 (67%)

1 (33%)

2 (50%)

2 (50%)

1 (33%)

1 (33%)




Use information effectively to accomplish aspecific purpose



Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)

Determine the extent of information needed 0% 100% 0% 0% Access the needed information 0% 67% 33% 0% Evaluate information and its sources critically 33% 33% 33% 0% Use information effectively to accomplish a specific purpose 50% 50% 0% 0%






Findings by

Dimension


A l l s t udent s s c or ed a t t he M i l e s to n e s (3 ) l ev e l , w h ic h f a r exc eeds t he es t a b l i s hed benc hma r k . No s t ude nt s s c or ed a t t he B e n c h m ar k , M i l e s ton e s (2 ) , or C ap s ton e l ev e l s .


S t udent s s c or ed w el l a bov e t he es t a b l i s hed benc h ma r k a t t he M i l e s to n e s (3 ) l ev e l . S t udent s s c or ed s l i g htl y be l ow t he b enc hma r k a t t he M i l e s to n e s (2 ) l ev e l . No s t udent s s c or ed a t t he B e n c hm ar k o r C ap s to n e l ev e l s .


S t udent s s c or ed a b ov e t he es t a b l i s hed benc h ma r k a t t he C ap s to n e l ev e l . S t udent s s c or ed s l ig ht l y l ow er t ha n t he be nc hma r k i n t he M i l e s to n e s (3 ) a nd M i l e s to n e s (2 ) l ev e l s . No s t udent s s c or ed a t t h e B e n c h m ark l ev e l .


S t udent s ’ s c or es g r ea t l y exc eed t he benc h ma r k a t t he C ap s to ne l ev e l . M i l e s ton e s (3 ) s c or es a l s o exc eed t h e es t a b l i s hed benc hma r k . No s t u dent s s c or ed i n t h e M i l e s to ne s (2 ) o r B e nc h m ark l ev e l s .


S t udent s ’ s c or es g r ea t l y exc eed t he benc h ma r k a t t he C ap s to ne l ev e l . M i l e s ton e s (3 ) s c or es a l s o exc eed t h e es t a b l i s hed benc hma r k . No s t u dent s s c or ed i n t h e M i l e s to ne s (2 ) o r B e nc h m ark l ev e l s .


30

Music Program MUS 355 – Spring 2017


Findings: MUS 355 submitted student Reflective Essays and Research Papers. All 5 dimensions of the rubric were used for assessment. The students enrolled in MUS 355 scored equally on “Determine the extent of information needed” and “Use information effectively to accomplish a specific purpose” when compared to Seaver students. While they scored higher on the rest 3 dimensions when compared to Seaver students and Librarian Evaluation.

3.1 3.13.4

3.03.7

3.1 2.9 3.0 3.0 3.12.92.5

0.0

1.0

2.0

3.0

4.0




critically




and legally

Average Scores of MUS 355 ( N=7 students)Average Score of Seaver College ( N = 68 students + 5 groups )Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of MUS 355

( N = 7 students)




( N = 79 students)


-


-





31

MUS 355 – Spring 2017

Average Scores by Gender

Findings: The male students enrolled in MUS 355 scored slightly higher on “Determine the extent of information needed” and “Access the needed information” and scored equally on “Use information effectively to accomplish a specific purpose”, while they scored lower on “Evaluate information and its sources critically” and “Access and use information ethically and legally” when compared to female students.

3.2 3.23.4

3.03.6

3.0 3.03.5

3.0

4.0

0.0

1.0

2.0

3.0

4.0

Determine theextent of

information needed



critically




and legally


Research PaperReflective Essay

Male ( N = 5 students )

Female ( N = 2 students)

Determine the extent of information needed 3.2 3.0 Access the needed information 3.2 3.0 Evaluate information and its sources critically 3.4 3.5 Use information effectively to accomplish a specific purpose 3.0 3.0 Access and use information ethically and legally 3.6 4.0


32

MUS 355 – Spring 2017

MUS 355 Scores Overall

1 (14%)

1 (14%)

3 (43%)

2 (29%)

5 (71%)

6 (86%)

6 (86%)

4 (57%)

3 (43%)

2 (29%)

2 (29%)











Findings by

Dimension


S t u d e n t s s c o r e d a b o v e t h e e s t a b l i s h e d b e n c h m a r k a t t h e C a p s t o ne l e v e l . S t u d e n t s ’ s c o r e s a t t h e M i l e s to ne s (3 ) l e v e l f ar e x c e e d t h eb e n c h m a r k . N o s t u d e n t s s c o r e d i n t h e B e n c h m a r k o r Mi le s to ne s ( 2)l e v e l s .


S t udent s s c or ed a b ov e t he es t a b l i s hed benc h ma r k a t t he C ap s to n e l ev e l . S t udent s ’ s c or es a t t he M i l e s to n e s (3 ) l ev e l f a r exc eed t hebenc hma r k . No s t udent s s c or ed i n t he B e n c h m ar k o r M i le s to n e s (2 )l ev e l s .


S t u d e n t s ’ s c o r e s g r e a t l y e x c e e d t h e e s t a b l i s h e d b e n c h m a r k a t t h e C a p s t o ne l e v e l . Mi le s t on e s ( 3 ) s c o r e s a l s o e x c e e d t h e b e n c h m a r k . N o s t u d e n t s s c o r e d i n t h e B e n c h m a r k o r M il e s to ne s (2 ) l e v e l s .


M i le s t on e ( 3) l e v e l s c o r e s ar e w i t h i n t h e e s t a b l i s h e d b e n c h m a r k . S t u d e n t s ’ s c o r e s e x c e e d t h e e s t a b l i s h e d b e n c h m a r k i n t h e C a p s t o ne l e v e l . S t u d e n t s c o r e s a r e b e l o w t h e b e n c h m a r k i n t h e M i l e st on e ( 2)l e v e l . N o s t u d e n t s s c o r e d a t B e n c h m a r k l e v e l .


S t u d e n t s ’ s c o r e s g r e a t l y e x c e e d t h e b e n c h m a r k a t t h e C a p s t o ne l e v e l . S t u d e n t s s c o r e d b e l o w t h e e s t a b l i s h e d b e n c h m a r k i n t h e M i le st o ne s (3 ) l e v e l . N o s t u d e n t s s c o r e d i n t h e M i l e s t on e s ( 2) o r B en c h m a rk l e v e l s .

Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)




33

Philosophy Program PHIL 480 – Spring 2017


Findings: PHIL 480 submitted student Reflective Essays and Research Papers. All 5 dimensions of the rubric were used for assessment. The students enrolled in PHIL 480 scored higher on all 5 dimensions when compared to Seaver students and Librarian Evaluation.

3.53.2 3.2 3.2

3.8

3.1 2.9 3.0 3.0 3.12.92.5

0.0

1.0

2.0

3.0

4.0




critically




and legally

Average Scores of PHIL 480 ( N=17 students)Average Score of Seaver College ( N = 68 students + 5 groups )Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of PHIL 480

( N = 17 students)




( N = 79 students)


-


-





34

PHIL 480– Spring 2017 Average Scores by Gender

Findings: The male students enrolled in PHIL 480 scored slightly higher on “Determine the extent of information needed” and “Evaluate information and its sources critically” and scored equally on “Access and use information ethically and legally”, while they scored lower on “Access the needed information” and “Use information effectively to accomplish a specific purpose” when compared to female students.

3.73.2 3.3

3.0

3.83.3 3.3

3.03.5

3.8

0.0

1.0

2.0

3.0

4.0



Evaluate informationand its sources critically




and legally


Male ( N = 9 students)

Female ( N = 8 students)


Access the needed information 3.2 3.3

Evaluate information and its sources critically 3.3 3.0





35

PHIL 480 – Spring 2017 PHIL 480 Scores Overall

Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)



9 (53%)

8 (47%)

7 (41%)

7 (41%)

13 (76%)

7 (41%)

5 (29%)

6 (35%)

7 (41%)

4 (24%)

1 (6%)

4 (24%)

4 (24%)

3 (18%)









(4 pts) (3 pts) (2 pts) (1 pts) % of students expected to

score in each level 2-5% Between 35- 45% Between 35- 45% under 10-15% 0%

Findings by

Dimension


M i l e s to ne s (3 ) l ev e l s c or es ar e w i t hi n t he es t a b l i s hed benc hma r k . S t udent s c or es g r ea t l y exc eed t he benc hma r k a t t he C ap s to n e l ev e l . M i l e s to ne s (2 ) l ev e l s c or es ar e f a r be l ow t he benc hma r k . N o s t udent s s c or ed a t t he B e n c hm ar k l ev e l .


S t udent s c or es g r ea t l y exc eed t he benc hma r k a t t he C ap s to n e l ev e l . S t udent s c or es i n t he M i l e s to n e s (2 ) a nd M i l e s to n e s (3 ) l ev e l s a r e be l ow t he b enc hma r k s . No s t u dent s s c or ed a t B e n c h m ar k lev e l .


M i l e s to ne s (3 ) l ev e l s c or es ar e w i t hi n t he es t a b l i s hed benc hma r k . M i l e s to ne s (2 ) l ev e l s c or es ar e be l ow t he be nc hma r k . S t ud ent s c or es g r eat l y exc eed t he benc h ma r k a t t he C ap s to n e l ev e l . No s t udent s s c or ed a t t he B e n c hm ar k l ev e l .


M i l e s to ne s (3 ) l ev e l s c or es ar e w i t hi n t he es t a b l i s hed benc hma r k . M i l e s to ne s (2 ) l ev e l s c or es ar e be l ow t he be nc hma r k . S t ud ent s c or es g r eat l y exc eed t he benc h ma r k a t t he C ap s to n e l ev e l . No s t udent s s c or ed a t t he B e n c hm ar k l ev e l .


S t udent s c or es g r ea t l y exc eed t he benc hma r k a t t he C ap s to n e l ev e l . M i l e s to ne s (3 ) l ev e l s c or es ar e be l ow t he be nc hma r k . No s t udent s s c or ed a t t he B e n c h m ar k o r M i l e s ton e s (2 ) l ev e ls .


36

Social Work Minor SW 300 – Spring 2017


Findings: SW 300 did not submit student Research Papers. Their assessment data was performed on their Reflective Essay using 3 dimensions of the rubric “Determine the extent of information needed”, “Access the needed information” and “Evaluate information and its sources critically”. The students enrolled in SW 300 scored lower on all 3 dimensions when compared to Seaver students.

2.8 2.62.93.1 2.9 3.0

0.0

1.0

2.0

3.0

4.0

Determine the extent ofinformation needed

Access the needed information Evaluate information and itssources critically

Average Scores of SW 300 ( N=8) Average Score of Seaver College ( N = 68 students + 5 groups )

Average Scores of SW 300 ( N = 8

students)



Determine the extent of information needed 2.8 3.1 Access the needed information 2.6 2.9 Evaluate information and its sources critically 2.9 3.0

Reflective Essay


37

SW 300– Spring 2017

Average Scores by Gender

Findings: The male students enrolled in SW 300 scored slightly higher on rubric “Access the needed information” and “Evaluate information and its sources critically”, while they scored slightly lower on “Determine the extent of information needed” when compared to female students.

2.7 2.7

3.32.8 2.6 2.6

0.0

1.0

2.0

3.0

4.0

Determine the extent of informationneeded

Access the needed information Evaluate information and its sourcescritically


Male


( N = 5 students) Determine the extent of information needed 2.7 2.8 Access the needed information 2.7 2.6 Evaluate information and its sources critically 3.3 2.6

Reflective Essay


38

SW 300– Spring 2017

Average Scores by Ethnicity

Minority

( N = 2 students) International

( N = 3 students) White

( N = 3 students) Determine the extent of information needed 2.5 2.7 3.0 Access the needed information 2.5 2.3 3.0 Evaluate information and its sources critically 2.5 2.7 3.3

Findings: The white students enrolled in SW 300 have the highest score on all 3 dimensions. International students scored slightly higher on “Determine the extent of information needed” and “Evaluate information and its sources critically” when compared to the minority students.

2.5 2.5 2.52.72.3

2.73.0 3.0

3.3

0.0

1.0

2.0

3.0

4.0


Access the needed information Evaluate information and itssources critically

Minority (N=2 students) International (N=3 students) White (N=3 students)

Reflective Essay


39

SW 300 – Spring 2017

SW 300 Scores Overall

1 (13%)

6 (75%)

5 (62%)

5 (62%)

2 (25%)

3 (38%)

2 (25%)



Evaluate information and itssources critically


Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)

Determine the extent of information needed 0% 75% 25% 0% Access the needed information 0% 62% 38% 0% Evaluate information and its sources critically 13% 62% 25% 0%





Findings by

Dimension


S t udent s c or es f a r exc ee d t he benc hma r k i n t he M i l e s to ne s (3 ) l ev e l . M i l e s to ne s (2 ) l ev e l s c or es a r e be l ow t he b enc hma r k . No s t udent s s c or ed a t t he B e nc h m ark o r C ap s ton e l ev e ls .


M i l e s to ne s (2 ) l ev e l s c or es ar e w i t hi n t he be nc hma r k . S t udent s c or es e xc eed t he benc h ma r k i n t he M i l e s to n e s (3 ) l ev e l . No s t ude nt s s c or ed a t t he B e n c h m ar k o r C ap s to ne l ev e l s .


S t udent s c or es e xc eed t he benc h ma r k i n t he M i l e s to n e s (3 ) a nd C ap s to n e l ev e ls . M i l e s to ne s (2 ) l ev e l s c or es ar e be l ow t he b enc hma r k . No s t udent s s c or ed a t t he B e nc h m ark l ev e l .


40

GRAZIADIO GRADUATE SCHOOL

Fall 2016


41

Table 1. Contributing programs for Graziadio Graduate School



Fall 2016 ECNM 602 Jared Ashworth 25 Reflective Essay & Research Paper


42

Economics Program ECNM 602 – Fall 2016


Findings: ECNM 602 submitted student Reflective Essays and Research Papers. All 5 dimensions of the rubric were used for assessment. The students enrolled in ECNM 602 scored higher on “Access and use information ethically and legally” when compared to Librarian Evaluation and scored equally on the rest 4 dimensions when compared to Graziadio students and Librarian Evaluation.

3.02.7

2.42.9

3.43.0

2.72.4

2.93.4

2.92.5

0.0

1.0

2.0

3.0

4.0




critically




and legally

Average Scores of ECNM 602 ( N = 25 students)Average Score of Graziadio Graduate ( N = 25 students)Average Scores of Librarian Evaluation ( N = 79 students)

Average Scores of ECNM 602

( N = 25 students)

Average Score of Graziadio Graduate ( N = 25

students)


( N = 79 students)


-


-





43

ECNM 602 – Fall 2016 ECNM 602 Scores Overall

Level of

Accomplishment Capstone Milestones Milestones Benchmark Informat ion not Present

(4 pts) (3 pts) (2 pts) (1 pts) % of students expected to score

in each level 2-5% Between 35- 45% Between 35- 45% under 10-15% 0%

Findings by Dimension


S t udent s c or es e xc eed t he es t a b l i s hed benc hma r k a t t he M i l e s to n e s (3 ) a nd C ap s to n e l ev e ls . B en c h m ar k a nd M i l e s to ne s (2 ) l eve l s c or es a r e f a r be l ow t he es t a b l i s hed benc hma r k .


M i l e s to ne s (3 ) l ev e l s c or es ar e w i t hi n t he be nc hma r k . C ap s to n e a nd B e nc h m ark l ev e l s c or es exc eed t he es t a b l i s hed benc hma r k . M i l e s to ne s (2 ) l ev e l s c or es ar e be l ow t he be nc hma r k .


C ap s to ne l ev e l s c or es ar e w i t hi n t he es t a b l i s hed benc h ma r k . S t udent s c or es g r ea t l y exc eed t he benc hma r k a t t he M i l e s t o n e s (2 ) l ev e l . B en c hm ar k a nd M i l e s to ne s (3 ) s c or es a r e be l ow t he benc hma r k .


S t udent s c or es e xc eed t he es t a b l i s hed benc hma r k a t t he M i l e s to n e s (3 ) a nd C ap s to n e l ev e ls . M i l e s to ne (2 ) l ev e l s c or es ar e be l ow t he benc hma r k . No s t udent s s c or ed a t t he B e n c h m ar k l ev e l .


C ap s to ne l ev e l s c or es g r ea tl y exc eed t he es t a b l i s hed benc h ma r k . M i l e s to ne s (3 ) l ev e l s c or es ar e s l ig ht ly gr ea t er t ha n t he be nc hma r k . M i l e s to ne s (2 ) l ev e l s c or es ar e f a r be l ow t he es t a b l i s hed benc hma r k . No s t udent s s c or ed a t B e n ch m ar k l ev el .

5 (20%)

6 (24%)

1 (4%)

4 (16%)

11 (44%)

17 (68%)

9 (36%)

8 (32%)

14 (56%)

12 (48%)

2 (8%)

6 (24%)

15 (60%)

7 (28%)

2 (8%)

1 (4%)

4 (16%)

1 (4%)







Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)

Determine the extent of information needed 20% 68% 8% 4%

Access the needed information 24% 36% 24% 16%

Evaluate information and its sources critically 4% 32% 60% 4%


16% 56% 28% 0%



44

LIBRARIAN EVALUATION Fall 2017 & Spring 2018


45

Table 1. Contributing programs for Graziadio Graduate School*

*Note: The librarians in Payson library retrieved students' work in their senior year to provide additional data for the Information Literacy core competency assessment. This assessment examined 2 dimensions of the value rubric on “Use information effectively to accomplish a specific purpose” and “Access and use information ethically and legally”. In total 65 students artifacts were assessed by 2 to 3 assessors.



Fall 2017 INTS 455 Jaimie Colvin, Jeremy Whitt,

Mary Elizabeth Parang 9 Research Paper

Fall 2017 NUTR 499 Jaimie Colvin, Jeremy Whitt 5 Research Paper

Spring 2018 BA 497 Jaimie Colvin, Jeremy Whitt, Marc Vinyard

65 Research Paper


46

International Studies and Languages Program INTS 455 – Fall 2017


Average Scores of INTS 455

( N = 9 students)




Evaluation ( N = 79



Findings: INTS 455 did not submit student Reflective Essays. Their assessment data was performed on their Research Paper using 2 dimensions on the rubric “Use information effectively to accomplish a specific purpose” and “Access and use information ethically and legally”. The students enrolled in INTS 455 scored lower on both dimensions when compared to Seaver students and Librarian Evaluation.

2.52.2

3.0 3.12.92.5

0.0

1.0

2.0

3.0

4.0



Average Scores of INTS 455 ( N = 9 students)Average Score of Seaver College ( N = 68 students + 5 groups )Average Scores of Librarian Evaluation ( N = 79 students)

Research Paper


47

INTS 455 – Fall 2017 INTS 455 Scores Overall





Findings by

Dimension


M i l e s to ne s (3 ) l ev e l s c or es ar e w i t hi n t he es t a b l i s hed benc hma r k . M i l e s to n e (2 ) s c or es ar e a bov e t he benc hma r k . C ap s to ne l ev e l s c or es ar e s l i g htl y h ig her t ha n t he benc hma r k . No s t udent s s c or ed a t t he B e n c h m ar k l ev e l .


B e nc h m ark l ev e l s c or es a r e w it h i n t he es t a b l i s hed benc hma r k . M i l e s to n e s (2 ) s c or es exc eed t h e be nc hma r k . M i l e s to ne s (3 ) l ev e l s c or es ar e be l ow t he es t a b l i s hed benc hma r k . C ap s to n e l ev e l s c or es a r e s l ig ht l y h ig her t ha n t he be nc hma r k .

6%

6%

39%

22%

56%

61% 11%




Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)


6% 39% 56% 0%



48

Natural Science Program NUTR 499 – Fall 2017


Average Scores of NUTR 499

( N = 5 students)




Evaluation ( N = 79

students) Use information effectively to accomplish a specific purpose

3.4 3.0 2.9


Findings: NUTR 499 did not submit student Reflective Essays. Their assessment data was performed on their Research Paper using 2 dimensions on the rubric “Use information effectively to accomplish a specific purpose” and “Access and use information ethically and legally”. The students enrolled in NUTR 499 scored higher on “Use information effectively to accomplish a specific purpose” when compared to Seaver students and Librarian Evaluation, while scored lower than Seaver students and higher than Librarian Evaluation on “Access and use information ethically and legally”.

3.4

2.83.0 3.12.92.5

0.0

1.0

2.0

3.0

4.0



Chart Title

Average Scores of NUTR 499 ( N = 5 students)Average Score of Seaver College ( N = 68 students + 5 groups )Average Scores of Librarian Evaluation ( N = 79 students)

Research Paper


49

NUTR 499 – Fall 2017 NUTR 499 Scores Overall





Findings by

Dimension


C ap s to ne l ev e l s c or es g r ea tl y exc eed t he es t a b l i s hed benc hma r k . Du e t o t he h i g h per c ent a g e of s c or es i n t h e C ap s to ne l ev e l , s c or es i n t he M i l e s to n e s (2 ) a nd M i l e s to n es (3 ) l ev e l s a r e be l ow t he benc hma r k . N o s t ude nt s s c or ed a t t he B e n c h m ar k l ev e l .


M i l e s to ne s (3 ) l ev e l s c or es gr ea tl y exc eed t h e es t a b l i s hed benc hma r k . M i l e s to n e s (2 ) l ev e l s c or es ar e be l ow t he benc hma r k . No s t udent s s c or ed a t t he B e n c h m ar k o r C ap s to ne l ev e l s .

60% 20%

80%

20%

20%

Use information effectively to accomplish a specificpurpose



Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)


60% 20% 20% 0%



50

Business Administration Program BA 497 – Spring 2018


Average Scores of BA 497 ( N = 65

students)




Evaluation ( N = 79



Findings: BA 497 did not submit student Reflective Essays. Their assessment data was performed on their Research Paper using 2 dimensions on the rubric “Use information effectively to accomplish a specific purpose” and “Access and use information ethically and legally”. The students enrolled in BA 497 scored equally with Seaver students on “Use information effectively to accomplish a specific purpose” and higher than Librarian Evaluation. While they scored equally with Librarian Evaluation and lower than Seaver Students on “Access and use information ethically and legally”.

3.02.5

3.0 3.12.92.5

0.0

1.0

2.0

3.0

4.0



Chart Title

Average Scores of BA 497 ( N = 65 students)Average Score of Seaver College ( N = 68 students + 5 groups )Average Scores of Librarian Evaluation ( N = 79 students)

Research Paper


51

BA 497 – Spring 2018 BA 497 Scores Overall





Findings by

Dimension


M i l e s to ne s (3 ) s c or es gr ea t l y exc eed t he es t a b l i s hed benc hma r k . M i l e s to n e s (2 ) a nd B e n ch m ar k l ev e l s c or es ar e f a r be l ow t he es t a b l i s hed benc hma r k . C ap s to n e l ev e l s c or es a r e a bov e t he b enc h ma r k .


M i l e s to n e s (3 ) a nd C ap s to ne l ev e l s c or es a r e wi t h i n t he es t a b l i s hed benc hma r k . M i l e s to n e s (2 ) l ev e l s c or es a r e a bov e t he es t a b l i s hed benc h ma r k . B e nc h m ark l ev e l s c or es a r e be l ow t he es t a b l i s hed benc h ma r k .

8%

4%

81%

41%

10%

53%

1%

2%

Use information effectively to accomplish a specificpurpose



Capstone (4 pts)

Milestones (3 pts)

Milestones (2 pts)

Benchmark (1 pts)


8% 81% 10% 1%


information literacy 2016-2018 (2) · literacy outcomes and achieving competency in information...

Documents