2014-2015 college of arts and sciences department reports

2014-2015 COLLEGE OF ARTS AND SCIENCES

Department Reports

Anthropology Assessment 2014 -‐ 2015 1

1

EXECUTIVE SUMMARY During 2014-‐2015 academic year, the Department of Anthropology continued our conversations on how best to integrate portfolios into our program’s assessment plan. Although we continue to collect assessment data, some of which is reported here, we have re-‐directed some of our efforts due to our ongoing Program Review (to be wrapped up in December 2014) and in light of the new 2014-‐2020 IPFW Strategic Plan, USAP goals development, and closure of the IPFW-‐Archaeological Survey. Because of these developments, we are in the process of revising our program’s mission statement, vision, and goals. As explained in last year's report, the Department of Anthropology faculty has incorporated “high impact” learning strategies across our entire curriculum beginning in Fall 2016 and we will be assessing the effectiveness of those changes in future assessment reports. With the data we currently have available, it is evident from both the direct and indirect measures we report here that we are achieving all of our program’s learning objectives. Our interim measures indicate that our students continue to gain a familiarity with different cultures (Objective 1), they are learning how to understand them holistically (Objective 3), and they are developing writing skills (Objective 4) in our core curriculum. Direct assessment methods within our capstone course that are used to address some of our learning objectives also continue to be employed: our internal exit measure demonstrates that anthropology seniors continue to be highly satisfied with our program and course content, feel they have gained all the skills and learning objectives spelled out in our assessment plan, and are satisfied with the job we are doing with advising. As explained within this report, our exit survey results are consistent with those collected by our recently completed Alumni Survey. Finally, one of our external exit measures – graduate and professional school admissions – continues to demonstrate a high rate of admission to graduate programs. The numbers of graduate school admissions are similar for both graduating seniors (37.5% [3/8] of last year's graduating seniors) and alumni (43.1%).


2

PART I. Overview of Programmatic Student Learning Objectives ANTHROPOLOGY PROGRAM MISSION The purpose of the department's program for anthropology majors is to assist them in acquiring a comprehensive and integrated knowledge base within the discipline and the skills to apply this knowledge in their professional lives or post-‐graduate education. GOALS

1. Acquire knowledge of core areas within the discipline: theory, methods, ethnography, archaeology, linguistics, and bio-‐anthropology.

2. Acquire knowledge of a broad sub-‐area within the discipline.

3. Develop the skills to analyze and apply this knowledge.

LEARNING OBJECTIVES The specific skills students should acquire in our program include the following learning objectives:

1. Achieve familiarity with different cultures in at least two regions of the world 2. Know the major anthropological approaches to understanding the human

condition 3. Be able to explain societies in a holistic manner 4. Achieve competency in writing 5. Demonstrate critical thinking 6. Acquire quantitative skills for analysis 7. Demonstrate a willingness to engage learning and scholarship as a life-‐long

endeavor These skills address Strategic Goal 1 of the IPFW Strategic Plan, Provide Innovative, Relevant and Rigorous Academic Programs. Our goals also substantially overlap with the IPFW Baccalaureate Framework as follows:

Baccalaureate Goal (BG1) Acquisition of Knowledge Baccalaureate Goal (BG2) Application of Knowledge Baccalaureate Goal (BG3) Professional and Personal Values Baccalaureate Goal (BG4) Sense of Community Baccalaureate Goal (BG5) Critical Thinking and Problem Solving Baccalaureate Goal (BG6) Communication


3

PART II. ANTHROPOLOGY PROGRAM CURRICULAR MAP Anthropology

Assessment Goals

Baccalaureate Framework Goals

BG1 BG1 BG2 BG6 BG5 BG3

Anthropology Learning Objectives

1 2 3 4 5 6 7

Anthropology Core Courses

ANTH E105 X X X X ANTH B200 X X X X X ANTH P200 X X X X ANTH L200 X X X Group A Ethnographic Survey Courses

ANTH E310 X X X ANTH E330 X X X ANTH E350 X X X ANTH E356 ANTH E398 X X X SOC S410 X X X Group B Anthropology Topics

ANTH A385* ANTH A496* ANTH B405* X X X X X ANTH B426* X X X X X X ANTH E313 X X X X X ANTH E375 X X X X ANTH E385* X X X X X ANTH E406 ANTH E445 X X X X X ANTH E455 X X X X ANTH P370 X X X X ANTH P405* X X X X ANTH P421 X X X X X ANTH P430 X X X X X Capstone Course ANTH H445 X X X X X X *Represents a course where anthropological methods are taught.


4

PART III & PART IV. ANTHROPOLOGY ASSESSMENT PLAN AND MEASURES & RESULTS Internal Interim Assessment

1. Student Ability to Perform in Group A and B Courses. Description: As an interim measure of our success in transmitting anthropology basics in our core courses, student performance in upper level Group A and Group B courses are monitored (students typically take these courses after taking the core). These grades serve only as an indirect measure, or barometer, of our achievement of learning objectives. Systematic patterns of deficiency may indicate areas where we are not adequately addressing learning objectives in our introductory courses. The faculty will report the numbers of students who receive D’s or below in these courses to the chair. At the end of the academic year, the faculty will review systematic deficiencies, if any, in the transmission of anthropology basics from the core introductory courses, and recommend changes if necessary.

During the 2014-‐2015 academic year, 81.8% (down from 90.7% during 2013-‐2014, 84.75% during 2012-‐2013) of our students in Group A courses received grades of “C” or higher, while 87.1% (up from 86.9% during 2013-‐2014, 84% during 2012-‐2013 and 83% during 2011-‐2012) of our students in Group B topical courses received grades of “C” or higher. Through faculty discussions, it was evident (as it has consistently been in the past) that in each case the students who performed poorly (below “C”) simply stopped coming to class mid-‐way through the semester and/or failed to hand in assignments, and did not reflect on any lack of preparation or delivery in the anthropology curriculum. Therefore, our ability to meet Learning Objectives 2-‐4 (familiarity with other cultures, holistic understanding, and writing) does not seem to be deficient. Indeed, we believe that these data point to the success of our assessment efforts and curricular changes over the last five years. Further, our discussion of student assessment (in class discussions, reaction papers, essay exams questions that emphasize critical thinking skills) within each faculty’s Group A and Group B courses indirectly indicates that we are also achieving our Learning Objective 5 (Demonstrate Critical Thinking).

2. Course Offerings. Every academic year, the frequency with which our courses

are taught (and therefore the frequency with which we are addressing our learning objectives) will be examined in the curriculum map. This allows us to identify inadequacies or imbalances in our ability to meet curricular needs. If a course was taught, the number of sections offered is placed in bold next to each “X” that marks a learning objective.


5

Anthropology Program Curriculum Map Anthropology

Assessment Goals

Baccalaureate Framework Goals

BG1 BG1 BG2 BG6 BG5 BG3

Anthropology Learning Objectives

1 2 3 4 5 6 7

Anthropology Core Courses

ANTH E105 16 X 16 X 16 X 16 X ANTH B200 9 X 9 X 9 X 9 X 9 X ANTH P200 2 X 2 X 2 X 2 X ANTH L200 3 X 3 X 3 X Group A Ethnographic Survey Courses

ANTH E310 1 X 1 X 1 X ANTH E313 ANTH E330 1 X 1 X 1 X ANTH E350 1 X 1 X 1 X ANTH E356 ANTH E398 1 X 1 X 1 X SOC S410 1 X 1 X 1 X Group B Anthropology Topics

ANTH A385* 1 X 1 X 1 X 1 X 1 X ANTH A496* ANTH B405* ANTH B426* 1 X 1 X 1 X 1 X 1X 1 X ANTH E313 ANTH E375 1 X 1 X 1 X 1 X ANTH E406 ANTH E445 ANTH E455 1 X 1 X 1 X 1 X 1 X ANTH P370 1 X 1 X 1 X 1 X ANTH P405* 1 X 1 X 1 X 1 X ANTH P421 1 X 1 X 1 X 1 X 1 X ANTH P430 Capstone Course ANTH H445 1 X 1 X 1 X 1 X 1 X 1 X *Represents a course where anthropological methods are taught.


6

Our faculty taught a broad array of courses that allowed us to meet our learning objectives in the Group A and B sections for the academic year 2014-‐2015. In past years, due to assessment, a lack of Group A ethnographic area course offerings was identified as creating a bottle-‐neck for our student. We continue to ensure that at least two Group A courses are offered each semester, including an on-‐line offering (ANTH E335 European Ethnography) and an evening course offering (ANTH E310 Cultures of Africa).

Internal Exit

3. Capstone. Anthropology H445: History and Theory of Anthropology has always been our capstone course. The course is intended for graduating seniors. The course is a survey of the major theories and historic developments within anthropology over the past 150 years. Students read primary works and variously take written exams and write reflective essays on these major works. Student performance is evaluated based on their performance on two exams, written essays and contributions to class discussion. In addition, a great deal of time is spent discussing the nature of graduate work and professional scholarship in the course (i.e., research design, ethics). ANTH H445 effectively ties together all of our learning objectives. A student’s grade is an effective gauge of the extent to which we have been successful in transmitting the breadth of anthropology to our students. If there emerges a pattern of low grades on specific elements (i.e., exams on knowledge, or poor performance on specific written assignments), then the faculty will address how our curriculum may not be adequately covering specific learning objectives.

Currently, four written assignments are used to assess students’ familiarity with different cultures (Objective 1), an understanding of anthropological theory (Objective 2), an ability to explain culture holistically (Objective 3), and ability to write well (Objective 4). All written assignments also assess our students’ critical thinking skills. Our expectation is that 85% or more of our graduating majors should be able to perform satisfactorily (70% or higher) on each of the written assignments. We have chosen 70% as our minimum level for "satisfactory", given that a grade below 70% -‐ in terms of how all of us determine grades – is, by definition, a lower than "average" range grade and we consider this unsatisfactory for completion of degree requirements. Ten of the total 11 (90.9%) students enrolled in ANTH H445 during Spring 2014 received "A"s (4), "B"s (3), or "C"s (3). This represents a level that is on par with historic trends in our graduating seniors' performance (compared to 100% [11/11] during 2014, 94.7% [18/19] during 2013, and 86.7% [13/15] during 2012). Based upon the written assignments, 10 of the 11 H445 students demonstrated an ability to think critically (Learning Objective 5), demonstrated a


7

familiarity with at least two different cultures (Learning Objective 1), demonstrated an acceptable understanding of anthropological theory (Learning Objective 2), and an ability to explain culture holistically (Objective 3) and an ability to write well (Objective 4). The only student who did not perform acceptably failed to show up to class approximately 50% of the time.

4. Survey of Graduating Majors. This past Spring 2015 semester we conducted our

formal questionnaire of graduating seniors within the context of the ANTH H445 capstone course. The questionnaire consists of 20 statements that ask our seniors to evaluate our program’s goals, learning objectives, advising, overall quality of instruction, and numerous other items on a 1 to 5 scale with “1” equaling a strong disagreement with the statement and a “5” representing a strong agreement with the statement. Seven additional items ask more the seniors to provide qualitative information and feedback (i.e., how can the program be strengthened? Are there courses that should be added, etc.)? Surveys were handed out to all students during the last class meeting of H445. Of the 11 students enrolled in the course, 10 of the students turned in their questionnaires (90.9% return rate).

Our graduating seniors rated all 20 items on our program’s questionnaire favorably (>4.1 on all items), and indicated that they are highly satisfied with their familiarity they received from our program’s courses on different cultures (Learning Objective 1 = 4.60), their understanding they gained from our curriculum of major anthropological approaches to understanding the human condition (Learning Objective 2 = 4.60), their ability to explain societies in a holistic manner (Learning Objective 3 = 4.80), the writing skills they achieved within our courses (Learning Objective 4 = 4.70), their critical thinking skills (Learning Objective 5 = 4.70), the quantitative skills gained from our program (Learning Objective 6 = 4.50), and their willingness to engage in learning and scholarship as a lifelong endeavor (Learning Objective 7 = 70% [7/10] of respondents intend on going on to graduate school).

External Exit

5. Graduate School Admissions. Last year, our program officially had 8 students who graduated during the 2014-‐15 academic year. Among those students, three (Brittany Kime, Sidney Flint, and Nicole Black) were accepted to graduate or professional programs (37.5%). This is consistent with historic trends for our program: our recently completed alumni survey indicates that a total of 43.1% (25/58) of our former graduates received or are pursuing graduate degrees (24% -‐ 14/58 -‐ in anthropology).

6. Alumni Survey. Although we included this data in last year's report, we include it in this years report as well for the purposes of comparability with our graduating


8

senior's exit survey. Our former majors rated all items on our program’s questionnaire favorably (>4.0 on all items), thereby indicating that they are highly satisfied with their familiarity they received from our program’s courses on different cultures (Learning Objective 1 = 4.38), their understanding they gained from our curriculum of major anthropological approaches to understanding the human condition (Learning Objective 2 = 4.70), their ability to explain societies in a holistic manner (Learning Objective 3 = 4.60), the writing skills they achieved within our courses (Learning Objective 4 = 4.36), their critical thinking skills (Learning Objective 5 = 4.66), the quantitative skills gained from our program (Learning Objective 6 = 4.43), and their willingness to engage in learning and scholarship as a lifelong endeavor (Learning Objective 7 = 61.2% of respondents either have or intend on going on to graduate school). Further, our alumni largely feel that the advising they received from our program was outstanding (4.25), and – overall – they are satisfied with the educational experience they received from their IPFW Anthropology degree (4.11). Our recently completed alumni survey also indicates that a total of 43.1% (25/58) of our former graduates received or are pursuing graduate degrees (24% -‐ 14/58 -‐ in anthropology). Further, among those who have not pursued or earned a graduate degree, an additional 19% (11/58) plan on applying to graduate school in the near future, and an additional 30% (17/58) of respondents claimed that they currently work in a job that allows them to apply knowledge gained from their anthropology degree.

7. Evaluation of achievement conducted by external visitors. There was no

evaluation of achievement conducted by external visitors during the 2014-‐2015 academic year. Feedback from external visitors is planned to occur as a part of our department's program review, which is ongoing. However, due to faculty turn over, sabbaticals, and one faculty taking a year's leave of absence, we have temporarily put our program review on hold.

8. Additional Indirect Measures

Persistence Efficiency IPFW’s Institutional Research has begun reporting data regarding the number of new and transfer majors into a program each year as well as the number of official graduates. Here, we report the data that are available for academic years 2011 -‐ 2015. During that five year period, the Department of Anthropology exhibited a 128% efficiency in the number of new and transfer majors relative to the number of graduates. By any standard, this is an excellent indicator of our program’s ability to successfully marshal our students through our program.

2015 2014 2013 2012 2011 Totals

New 7 9 9 14 12 51 Graduated 8 15 13 17 11 64


9

Efficiency 114.29% 166.67% 144.44% 121.43% 91.67% 128% PART V. CONCLUSIONS, NEXT STEPS, AND COMMUNICATION Curricular/Assessment Changes High Impact Learning Methods

As reported last year, all of our courses (both introductory and upper level) utilize many of the proven high impact learning methods (term papers, experiential learning, oral presentations, research experiences, etc.). We will be assessing the effectiveness of these changes to our curriculum in the future.

Areas to be addressed as a result of program assessment

As reported last year, our review of the program’s stated goals and learning objectives previously identified two areas of deficiency in our curriculum. Namely, the faculty agreed that our majors were not receiving any methods training in cultural anthropology and that our students were struggling or received relatively little in the way of quantitative skills necessary to adequately perform anthropological methods. Additionally, a number of our graduating seniors reporting having received relatively low scores on the GRE’s quantitative section, and we recognize that the only places in our curriculum where students gain and subsequently use quantitative skills are in ANTH B200 Introduction to Biological Anthropology, ANTH B405 Fieldwork in Bioanthropology, and ANTH B426 Human Osteology. Addressing these deficiencies requires us to take a number of sequential measures. Some of these have measures have already been taken and their impact will be assessed in future assessment reports.

Quantitative skills As a direct result of the aforementioned deficiencies we decided to make a number of changes to our program. First, beginning in Fall 2013, we added an additional degree requirement for anthropology majors to complete a social science statistics (POLS Y395, PSY 201, SOC 351) or any STAT in addition to STAT 125. Although it is too soon to determine the impact of this additional degree requirement, trends in our exit survey of graduating seniors indicates that they believe that their quantitative skills are satisfied with the quantitative skills they are gaining from our program: specifically, we report incremental improvement our graduating majors' response regarding the quantitative skills gained from our program (Learning Objective 6), which – in our survey conducted in Spring of 2013 was 4.00 and has now increased to 4.50 during among our Spring 2015 exit survey responses. We will continue to assess the effectiveness of this as new


10

majors who are required to adhere to the degree change filter through our program.

Methods training Methods training is an area that we as a faculty – as a direct result of assessment and both graduating senior exit surveys and our alumni survey results – have identified as a program deficiency. A familiarity with methods has been one of our student learning objectives since our Assessment Plan was first developed and we have taken piece-‐meal steps toward implementing such a requirement. We are planning to add a methods course requirement for new majors' degree requirement beginning in Fall 2016.

Working toward the establishment of this requirement, it has been necessary for us to first ensure that enough methods courses are available for our majors to complete the requirement. We have now have methods courses developed for three of the traditional four anthropological subdisciplines: Human Osteology represents a methods course in biological anthropology, as does the occassionally offered Fieldwork in Biological Anthropology in Peru. Two of our cultural anthropology faculty (Doug Kline and Harold Odden) recently developed ethnographic field methods courses that were first offered during Summer 2012 (ANTH A496 Ethnographic Field Work in Scotland) and Spring 2013 (Field Methods in Medical Anthropology). Further, Harold Odden's in-‐class based ethnographic field methods course (Ethnographic Methods) has been developed, but when it was offered during the Fall 2014 semester, we had difficult achieving sufficient enrollments for the course. In the archaeology subdiscipline, Dr. Richard Sutter offers an occasional ANTH P405 Archaeological Field School offering (Summer 2014) and Margaret Brown Vega recently developed and successfully taught (Fall 2014) a lab-‐based anthropological methods course (ANTH 385 GIS in Anthropology). However, Professor Brown Vega is on a year's leave of absence, so it is unclear when the course will be offered again. The occasional nature and prohibitive costs of ANTH P405 (an archaeological field school in Peru) make it unclear if we will consistently be able to offer an archaeological methods course. We should have some clarity on this once Margaret Brown Vega determines if she will return to her position here at IPFW. Last year, we reported that it was also our intention to submit a course proposal for a variable title/variable credit "Internship in Anthropology". Our decision to offer internships is a direct result of our assessment plan and exit survey results that consistently indicate that a plurality of our graduating majors feel that our program should offer internships (consistently between 25-‐35% of respondents). Since our last assessment report, we have identified five partners willing to regularly provide internships for our majors (specifically, Planned Parenthood,


11

the Allen County Department of Health, Science Central, and the African American History Museum). Additional community partners have also been identified where our majors can potentially complete internships, including The History Center and The Fort Wayne Museum of Art. We should be submitting a proposal for a variable title course by late 2015 or early 2016. It is our hope that formal internships will be available to anthropology majors during the Fall 2016 semester. Over the last two years, we have also reported that we intend to tie internships into our curricular change proposal to require our majors complete a methods course as part of their degree requirements. It was our intention to submit the methods requirement proposal by the end of the Fall 2014 semester. However, in light of the central administrations recent low tolerance of lower enrolling courses and the onerous works associated with USAP, and uncertainty regarding Margaret Brown Vega's continued employment, we have postponed submitting this proposal to the COAS Curriculum Committee. We will likely try to submit a methods requirement proposal for our majors some time during the Spring 2016 semester. As part of the major requirement, majors will be required to have methods training (either a methods course or internship course) in at least one methods course as part of their Group B topical anthropology course degree program requirements (that is, all methods courses are already counted as Group B requirements). Therefore, this additional degree requirement will not add additional credits to achieve. In the future, the report of research paper that results from their methods course will also be considered an element that they must submit as part of portfolio (see subsequent section).

Portfolios As part of our ongoing assessment discussions, the Anthropology faculty is convinced that portfolios represent a valuable means of both assessing our students as well as providing them with a more conscious awareness of the skill sets they are acquiring as part of their anthropological degree. In consultation with Kent Johnson, we are still trying to determine the best means to implement a portfolio requirement for our majors. Dr. Johnson has provided us with valuable feedback, and we are now considering having our future majors submit key artifacts (see below) to ePortfolios. However, in order to implement the portfolio requirement, the faculty feels that we must first have the methods course requirement in place. Our use of ePortfolio will require majors to upload relevant artifacts (two term papers from their Group A ethnographic courses, a narrated PowerPoint presentation from one of their upper level anthropology courses [either from Group A or Group B course], a technical report from an approved Group B methods course, a reflective statement on their degree and degree requirements, a resume, and a letter of intent for graduate school). Students


12

would be informed of this requirement during their initial meeting with their assigned degree advisor. Anthropology faculty will require majors enrolled in their upper level course to upload to ePortfolio-‐relevant artifacts that would already required as part of the course work requirements, while the reflective statement, resume, and letter of intent for graduate school would be worked on with the assistance of Career Services and turned in as part of their ANTH H445 capstone course requirement during our majors' senior year. Each artifact submitted as part of our graduating majors' ePortfolio would be evaluated independently by all departmental using rubrics (still in development in consultation with Dr. Johnson) that will align each artifact with assessment student learning outcomes. We will then, as a faculty, meet, discuss, and reflect our assessment of our graduating majors' ePortfolio to evaluate how well we are doing on the relevant learning objectives and to identify areas in our curriculum where we might need to address shortcomings. Our goal is to submit a proposal to have ePortfolios implemented as a degree requirement for our new and transfer majors entering during the 2016-‐2017 academic year. This goal, however, will depend on our ability to have our variable title course approved and our methods course requirement approved by the COAS Curriculum Committee and all subsequent committees.

ASSESSMENT REPORT

Chemistry Department

Academic Year: 2014-2015

Section 1: Student Learning Outcomes for the Program

1) Students will acquire a broad and integrated body of chemical knowledge in the areas

of structure, physicochemical transformation, synthesis and analysis, and will be capable

of critically applying these concepts to solve appropriate problems.

2) Students will be able to demonstrate effective oral and written communication of

chemical principles.

3) Students will be aware of the personal and professional expectations of the discipline

and the scientific community, in general.

4) Students will have the opportunity to acquire a Bachelor’s degree certified by the

American Chemical Society.

Section 2: Curriculum Map

A. Map of Programmatic Outcomes to Baccalaureate Framework

1) Outcome #1 covers acquisition of knowledge, application of knowledge and critical

thinking.

2) Outcome #2 covers communication.

3) Outcome #3 covers community and professional values.

B. Map of Programmatic Outcomes to Identified “core courses” in the curriculum.

The course-based curriculum map is shown in tabular form on the next page. The table is

formatted with column headings denoting outcomes and the row headings denoting core

courses; the outcomes have been broken down for outcomes 1 & 2 according to particular

subcategories. If a course covers a particular outcome, it is noted for doing this at an

introductory level (I), a developmental level (D) or a mastering level (M).

Section 3: Assessment Plan

A. Description of Department’s Assessment Model

The assessment regime employs assessment at various points during students’

coursework (interim measures) and upon graduation. These measures include embedded

quizzes, internally-developed and nationally-normed written examinations, and a

capstone course to evaluate communication skills using internally-developed rubrics.

B. Measures Used

1. Interim measures

a. The chemistry faculty will administer a departmentally-developed examination to all entering

chemistry majors early in their first semester. The exam, testing key concepts in CHM

115 and 116 (general chemistry) and 241 (introductory inorganic chemistry), will have

no impact on the student's grade in the course. (There will be no provisions for make-up

examinations.) It is a multiple choice exam that probes the student's knowledge of

chemistry prior to taking CHM 115, 116 and 241. Similarly, Chemistry 261/265

students will take a departmentally-developed multiple choice exam during the first week

of the semester to test their knowledge of organic chemistry (261-2, 265-6) prior to

taking the courses at IPFW. The students will again be assured that their performance

will have no repercussions on any course grades.

Chemistry majors taking CHM 321 (analytical chemistry) will then be given an exam during

the first week of the semester (typically the fall semester of their junior year) composed

of exactly the same questions that appeared on the two above examinations. The exams

will be scored and the student's performances on the examinations will be compared. By

comparing each student's performance on the `pre-course' and `post-course' examinations,

the department will have a quantitative assessment of how well students have learned the

course material and, thus, the exams can provide feedback to the chemistry faculty on

how successfully we have taught course material.

The departmental Assessment Committee will be charged with looking critically at students'

scores on the examinations and providing feedback to the entire department.

b. All incoming majors will be required to enroll in CHM 194 (Freshman Seminar). A portion of

the course content will cover the personal and professional expectations of the discipline

and the greater scientific community. Aggregate performance on embedded quizzes

evaluating student comprehension of these expectations will be reviewed.

2. Internal measures at or after graduation

Students will be required to complete a capstone course (CHM 496/497) during the academic

year in which they intend to graduate. Two of the course expectations will be 1) to generate a

research paper based on either several articles from refereed, discipline-appropriate journals or

an independent, “for-credit” research project under the direction of a faculty mentor and, 2) give

an oral presentation of the paper’s contents. Department faculty members will evaluate these

papers and the accompanying presentations using internal rubrics.

3. External measures at or after graduation

a. The annual American Chemical Society (ACS) report and departmental alumni surveys will be

reviewed.

b. The Diagnostic of Chemical Undergraduate Knowledge (DUCK), a standardized exam

developed by the ACS, will be an expectation for the CHM 496/497 capstone course. All

students in that course will be required to take this exam in the last semester of the year in which

they plan to graduate.

C. Rubrics or Evaluation Metrics Descriptions

1) Analysis of the pre-test and post-test results will be performed biannually beginning in 2017. This

analysis will determine the average and standard deviations for the pre-test and post-test results and

the means will be compared for statistically significant difference; the benchmark for this analysis will be demonstration of significant difference at the 90% confidence level.

2) Weighted averages for the papers and oral presentations generated for CHM 497 will be calculated

from the rubrics filled out by department faculty members; the numerical scaling will extend from

“1” for Poor to “4” for Excellent. (The rubrics are provided at end of this report.) This analysis will

be performed in odd-numbered years for students from the current year and from the year prior. The benchmark for both the paper and the presentation will be an aggregate average that is 3 or better.

3) Aggregate student performance on the DUCK exam will be compared with nationally normed data

determined by the American Chemical Society. This analysis will be performed in even-numbered

years for students from the current year and from the year prior. The benchmark for this will be aggregate performance at or above the 50th percentile.

4) Aggregate student performance on questions embedded in CHM 194 quizzes that test

comprehension of the personal and professional expectations will be analyzed biannually, starting in

2016; the benchmark for this metric will be 70% correct.

D. Description of Plan for Disseminating and Using Findings for Programmatic Learning

Improvement

If an outcome fails to meet its benchmark, a more detailed analysis of the measures for the

outcome will be performed by the Curriculum & Assessment Committee. The goal of this

analysis will be to identify the outcome measures that are responsible for the underperformance,

discover any patterns present that might link two or more of the measures, and recommend a

course of action (modification of the program, the measures or the assessment process, etc.)

These recommendations will then be brought to the entire department for consideration.

Section 4: Assessment Results

A. Current Year Assessment Findings

We had nine graduates in 2014-2015; six of these students received ACS-certified degrees.

The Committee focused on the data from CHM 497 (initial results from the DUCK examination

and assessment of the student papers and oral presentations).

The aggregate average of the five students taking the DUCK examination at the end of CHM 497

was 33.8 correct responses out of 60 questions. This compares to a national average of 31.5

correct responses and ranks at the 60th percentile. This exceeds the benchmark of a minimum

performance at the 50th percentile. Additional analysis of questions with the worst performance

by our students indicated that these were also the questions garnering poor performance

nationally.

Assessment of the oral presentations was made by faculty members attending them, using the

rubrics developed previously; the aggregate weighted average for these presentations is 3.05,

which exceeds the benchmark of 3. Additional analysis identified the main issue in this group of

presentations to be a lack of citation of the sources used in them.

Assessment of the written papers was made by members of the Assessment Committee, using the

rubric developed previously; the aggregate weighted average for these written papers is 2.91,

which falls below the benchmark of 3. Additional analysis identified that the averages in all

categories fell between 2.70 and 2.90 except for the correct citation of sources used in the papers.

B. Proposed Changes to Address Findings

Realizing the number of students taking CHM 497 is small, the Committee is hesitant to propose

major changes based on these results; however, the averages for the written presentations do

appear problematic. The Committee will suggest a modification to advising practices within the

Department to encourage that both ENG W131 and ENG W233 be taken as soon as can be

scheduled.

C. Prior Year Assessment Findings and Description of Changes Made

No specific findings or actions regarding curriculum were recommended in last year’s

report.

D. Assessment Findings for Curricular Changes Made

No specific findings or actions regarding curriculum were made.

Conclusions, Next Steps, and Communication

Initial analysis of data from the first group of students to take the revised capstone course (CHM

496/497) indicates a deficiency in technical writing skills; the Committee recommends that the

Department adopts a revised advising strategy to encourage students to take ENG W131 and

ENG W233 as early as a student’s schedule permits.

The Committee will continue to evaluate the data generated from the CHM 497 capstone course,

as well as the data generated by the first offering of CHM 194 (Freshman Orientation). We will

also continue to review the Department’s assessment plan.

CHM 49600/49700 research paper assessment rubric

POOR FAIR GOOD EXCELLENT

Organization of topics and synthesis of ideas (25%)

Incoherent with no

relationships shown.

Partially coherent with

several relationships not

addressed.

Mostly coherent with a

few relationships not

addressed.

Ideas/topics compiled

coherently to show

relationships among

them.

Breadth and depth of treatment (15%)

Number of topics is

insufficient and their

coverage is marginal.

Too many topics with

marginal coverage or

too few topics with

excessive coverage.

Some imbalance

between number of

topics and depth of their

coverage.

Number and coverage

of topics are

appropriate and

balanced.

Clarity and consistency of logic (15%)

Ideas/topics and

conclusions are unclear;

no logical focus is

discernible.

Ideas/topics and

conclusions are

somewhat clear; focus

is discernible, but logic

is unclear.

Ideas/topics and

conclusions are mostly

clear with a discernible,

logical focus.

Ideas/topics and

conclusions are stated

clearly, with a

consistent, logical

focus.

Support of conclusions by cited data (15%)

No sources cited are

comparable to refereed

journal articles.

Some sources cited are


journal articles.

Most sources cited are


journal articles.

All sources cited are


journal articles.

Clarity and quality of writing (30%)

Narrative is confusing

in many places due to

grammatical or spelling

errors.

Narrative is confusing

in a few places due to


errors.

Narrative is easily

understood with some

minor grammatical or

spelling errors.

Narrative is easily

understood with no


errors.

CHM 49700 oral presentation assessment

POOR FAIR GOOD EXCELLENT

Organization of topics and synthesis of ideas (20%)

Incoherent with no

relationships shown.

Partially coherent with

several relationships not

addressed.

Mostly coherent with a few

relationships not addressed.

Ideas/topics compiled

coherently to show

relationships among them.

Breadth and depth of treatment (15%)

Number of topics is

insufficient and their

coverage is marginal.

Too many topics with

marginal coverage or too few

topics with excessive

coverage.

Some imbalance between

number of topics and depth

of their coverage.

Number and coverage of

topics are appropriate and

balanced.

Clarity and consistency of logic (15%)

Ideas/topics and conclusions

are unclear; no logical focus

is discernible.


are somewhat clear; focus is

discernible, but logic is

unclear.


are mostly clear with a

discernible, logical focus.


are stated clearly, with a

consistent, logical focus.

Support of conclusions by cited data (15%)

No sources cited are


journal articles.

Some sources cited are


journal articles.

Most sources cited are


journal articles.

All sources cited are


journal articles.

Clarity and quality of the presentation (25%)

Presenter speaks with

inadequate volume or clarity,

and visual aids are confusing

and not employed

effectively.

Presenter frequently speaks

with inadequate volume or

clarity; and visual aids are

somewhat confusing or not

employed effectively.

Presenter sometimes speaks

with inadequate volume or

clarity; and visual aids are

adequately prepared and are

used effectively.

Presenter consistently speaks

with adequate volume and

clarity, and visual aids are

well prepared and used in an

effective manner.

Ability to address seminar questions (10%)

Presenter identifies all

questions as beyond his/her

understanding and makes no

attempt to answer them.

Presenter identifies most



attempt to answer those

questions.

Presenter identifies some



attempt to answer those

questions.

Presenter attempts to answer

all questions, even those

identified as beyond his/her

understanding.

ASSESSMENT REPORT FOR COMMUNICATION GRADUATE PROGRAM FALL 2015

Section 1: Student Learning Outcomes for the Program The learning goals for the Professional Communication Master's Program strive:

● To offer a balanced approach between communication theory and communication practice.

● To integrate various approaches to the study of communication, including rhetorical, theoretical, and critical analysis.

● To offer three different kinds of courses: A range of research methods courses, courses from a communication theory core, and courses in an area of specialization.

● To prepare students accordingly in one of the following broadly defined areas: ○ Rhetoric ○ Media studies ○ Interpersonal and organizational communication.

Section 2: Curricular Maps B. Map of Programmatic SLO’s to identified “core courses” in the curriculum

Theory Core Courses Preparing Students In:

Rhetoric COM 518 Theories of Persuasion COM 521 Theories of Rhetoric

Media Studies COM 527 Introduction to Cultural Studies COM 559 Current Trends in Mass Communication Research

Interpersonal and Organizational

COM 512 Theories of Interpersonal Communication COM 520 Small Group Communication COM 574 Organizational Communication

Section 3: Assessment Plan A. Description of Department’s Assessment Model – How is the department assessing student progress to Programmatic SLO at key common points in matriculation to degree? We plan to revise our recently piloted version of a scale and use it to assess non-course degree requirement work by students (comprehensive exams, professional projects, and theses). We will assess all of the non-course degree requirements from our graduating students until the program determines otherwise. B. Measures Used Assess the paper on the basis of a 100 point scale below, using sliders to select the

number which best indicates whether the paper demonstrates the criteria for each objective. Please use only criteria that apply to the paper. Additional comments may be added on the bottom of the sheet. To demonstrate an understanding of and the ability to apply research methods relevant to the student's particular area of study, the paper:

Figure 1. Questions 1 and 2 To demonstrate an understanding of the multi-dimensional nature of the discipline, the paper:

Figure 2. Questions 3 and 4 To demonstrate performance of and/or critical competence in communication skills relevant to the student’s particular area of study, the paper:

Figure 3. Questions 5 and 6.

Figure 4. Questions 7, 8, and 9. C. Rubrics or Evaluation Metrics Descriptions See Section 3.B. D. Description of Plan for Disseminating and Using Findings for Programmatic Learning Improvement Once the plan is complete and findings have been recorded, the data will be analyzed to extract conclusions about meeting the student learning outcomes. The findings, conclusions, and recommended future directions to reach these outcomes will be shared online with faculty in the department as well as stakeholders (beyond faculty) to allow for feedback and collaborative work to leverage and enhance the department’s performance. Section 4: Assessment Results A. Current Year Assessment Findings

To pilot the assessment rubric, we had eight coders apply the rubric to 8 term papers from three courses from fall 2014. We chose to use term papers since the new non-course degree requirements only became part of our graduate bulletin in fall 2015. We found the current rubric to be an acceptable way of assessing graduate work. Each paper was scored by two different coders and we report the average disagreement by question here.

Question Number Degree of disagreement

1 14.91

2 13.83

3 9.91

4 10.79

5 12

6 10.667

7 30.08

8 13.29

9 12.41

The rubric had coders rate graduate student term papers on a 100 point scale with 0 = Very Bad, 50 = Neither good nor bad, and 100 = Very Good. On average, coders differed in their ratings by 14.21 points. Coders disagreed on several items on whether or not the items were applicable to the paper. This occurred six times in the pilot. The average rating for the graduate student papers was 78.48. Based on our scale, this average rating falls on the high end of “fair.” We would like to work toward raising the average into the “Good” range (above 80) but we cannot suggest any curricular changes based on these findings. The rubric applied to these term papers was designed to be applied to non-course degree requirement work (e.g. graduate theses). For example Q2, which asks for coders to assess the effectiveness of the paper to apply a research method, does not apply to many of the term papers. We report the average ratings across coders by paper and question here.

Paper No.

Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Mean

1 51.67 49.67 86.67 86.67 81.67 85 65 94.33 92.67 77.04

2 45 45 67.5 67.5 70 80 67.5 75 52.5 63.33

3 50 52 90 95 82.5 90 93 95 95 82.5

4 40 40 87.5 90 85 90 70 90 90 75.83

5 75 75 71.5 70 67.5 72.5 60 52.5 45 65.44

6 89 89.5 92 92 89.5 91.5 64.5 94 93.5 88.38

7 85.5 82.5 82 86.5 86.5 87.5 83.5 78.5 84.5 84.11

8 85 80 91.5 92.5 88.5 90 91.5 95 93.5 89.72

Mean 65.15 64.21 83.58 85.02 81.40 85.81 74.38 84.29 80.83

B. Proposed Changes to Address Findings We need clearer directions on how to apply each of the items on the rubric to graduate work. While the small degree of difference on the 100 point scale is encouraging that this is a useful and reliable tool for assessing graduate work, we would ideally like to see fewer disagreements about whether or not a rating can be applied to a particular text. That said, possibly the rubric was inconsistently applied to these texts because the items were designed to be applied to work such as theses and comprehensive exam responses. The disagreements about applicability may disappear when using the texts for which the rubric was designed. Regardless, we will revisit and revise the directions to the rubric in light of these findings. C. Prior Year Assessment Findings and Description of Changes Made No assessment was conducted in the previous year. D. Assessment Findings for Curricular Changes Made Because this was a pilot and because we did not assess the non-course degree requirements texts, we do not have suggestions for curricular changes. Section 5: Conclusions, Next Steps, and Communication Conclusions: We present this report as an interim measure. Starting with students admitted for fall 2015, our graduate students will complete one of three non-course graduation requirements: a thesis, a professional project, or comprehensive examinations. We will enact a revised assessment rubric using those graduation requirements as assessment artifacts. Next Steps: Accordingly, this is our last assessment report until a critical mass of students who entered our program in fall 2015 or later has completed one of the above-mentioned graduation requirements. Communication: Please see Section 3D and Section 4B above.

1 Assessment Report - 2015 Communication Sciences and Disorders Department B.S. Degree – Communication Sciences and Disorders 1. Student Learning Outcomes for CSD degree

The following SLOs are matched with the University Ways of Knowing.

a. Students will compare typical communication to disordered communication. (A1, 2; B4, 5, and 7)

b. Students will explain concepts from basic communication sciences and disorders topics including those of acoustics, phonetics, anatomy and physiology of speech and hearing, speech, voice and fluency, language and cognitive processes associated with language development and disorders. (A1, 2, 3; B4, 5, 6, and 7)

c. Student will demonstrate the ability to screen hearing, speech and language. (A2, B5, 7)

d. Students will demonstrate correct use of speech and language assessment tools. (A1, 2, 3; B4, 5, 7)

e. Students will analyze assessment results and create intervention plans. (A1, 2, 3; B4, 5, 6, 7)

f. Students will implement appropriate intervention plans for specific clients. (A1, 2; B4, 5, 7) 2. Curricular Maps (See attached) 3. Assessment Plan In addition to specific Interim and Internal measures (see below) Communication Sciences and Disorders assesses student learning through common assessments at the course level evaluating student progress to programmatic SLO’s at key curricular points. In addition, a common research project is assessed as described in 4A1. Assessment of professional skills is accomplished through evaluations of “pre-professional” experiences as described in Section B1. The Department Assessment Report for 2014 was not done due to the illness of the department chairperson. 4. Interim Assessment Measure

A. Academic Scholarship Measures 1. Research/Culmination paper mid-way

Goal: Students demonstrate appropriate research skills and oral, written, application of

2 communication sciences and disorders models for assessment and treatment, and group interaction effectiveness in their progress toward professional preparedness with 80% scoring good or excellent on the measure described below. This assessment meets the Ways of Knowing A, 2, and 3 and B4, 5, 6 and 7.

In past years projects for the course CSD 42000 Introduction to Developmental Speech and Language Disorders has been designated to provide measures of academic scholarship because it is typically taken after the fundamental courses in the department sequence are successfully completed-about mid-way. Using a random selection process, at least 40% of the papers are independently evaluated by three faculty to determine clarity and depth of the chosen topic, accurate research methods used and reported, and application or conclusions appropriate to the findings of the research about the topic. The evaluation determined the number of papers that were judged to be excellent, good, fair and poor. See Appendix A. This particular project met all of the areas for the Ways of Knowing used by the university.

In the spring of 2012 that course was taught by a new Assistant Professor who had already resigned her position in the department and who, by doing so, made herself unavailable for participation in these activities. In the spring of 2013 the course was taught by an LTL and in 2014 the course was taught by a new assistant professor. Since the evaluation of the papers has been done over the course of the summer, it was not feasible to ask and associate faculty member to participate and the new assistant professor was just beginning her employment. For spring 2016 Assistant Professor Pam Reese has agreed to make one of her writing assignments in CSD 42000 available as an assessment measure. Due to illness of the department chairperson this type of assessment was not done for the 2014 or 2015 assessment report. 2. With-in Course measures. Annual assessment rubrics for these measures have yet to be determined. However, an example of what could be included is the project from CSD 42000 Introduction to Developmental Speech and Language Disorders. A group project was designed to introduce students to the application of theory, assessment tools and findings, and intervention models. The case study group project includes provided assessment information about a client to determine if additional and different assessment information is needed (if so, determine how to collect that information), analyze what areas of deficit are present, design an intervention plan for the client and finally, explain the process to classmates. This project would meet the department SLOs b, d, e and f.

B. Internal Measures 1. Pre-professional Experience

Goal: Eighty percent or more of graduating seniors will successfully engage in at least

3 one culminating pre-professional experience as noted below in a, b and c. These capstone experiences meet the following areas of the Ways of Knowing: A1, 2, and 3, B4, 5, and 7.

a. CSD 54900 - Clinical Practicum Summary Goal 1: Ninety percent of students will meet or make sufficient progress on their self-selected clinical goals. The Ways of Knowing areas in this assessment are: A 2 and 3; B4, 5, 6 and 7. Fourteen seniors participated. Students chose the goals in consultation with their clinical supervisor. Twelve students were matched with a single client, and one with two clients for a total of 14 clients served. Baseline data on student performance was taken and performance data was updated throughout the semester. Typically, the student clinician needed to display targeted professional behavioral skills over a specific number of consecutive therapy sessions (usually 3) in order to meet their goals. Final data collection was taken during the semester at the point when they reached their goals; however, data collection continued in order to determine if the clinician continued to maintain or progress in the specific skill. A list of 25 professional behavioral skills has been developed by clinic supervisors over the years. Examples of the skills on this list are: Acceptable writing skills for clinical documentation Clarity of explanations and instructions Provision of appropriate (reinforcing or corrective) and consistent feedback to clients Elicitation of adequate client responses and participation Accurate recording of session data Use of standard professional English in the clinical setting Skills to be measured were selected from the list of 25 by the student and supervisor at the beginning of the clinic assignment. Baseline data was taken through observed clinical sessions and reviewed and updated at mid-term, continued through the end of the semester and scored again. Of the 34 targeted skills, each student and her/his supervisor selected an average of three for which improvement was needed. Baseline and post-training evaluations of the skills chosen for measurement showed that 100% of the students improved on all of the skills selected. Most students continued to work on their goals to exceed a higher criteria throughout the semester. The 90% goal for achieving sufficient progress in clinical skills was met. Goal 2: Ninety percent of the clients or their families will agree that they were satisfied with the progress made; that they would recommend the clinic to others; and if previous therapy was experienced at a different locality, that the experience at IPFW was as

4 good or superior by comparison.

Client Feedback Form Results - CSD 549 & 399, Spring 2015 (14/15 surveys returned)

Key: SA = strongly agree A = agree D = disagree SD = strongly disagree SA A D SD Q1 I was informed of test results and therapy goals (n = 14 ) 12 2 Q2 I had appropriate opportunities for conferences with the clinician (n = 14 ) 12 2 Q3 I used the opportunity to observe (n = 12 n/a = 2) 12 0 Q4 I am satisfied with the therapy & progress achieved in time allowed (n = 14) 11 2

Superior Good Fair Poor Q6 If received previous speech therapy at a different facility, was this superior, good, fair, poor? (n = 9) no previous therapy received; (n=5)

5 4

Yes No Unsure Q7 Would you recommend program to others? (n = 14) 14

Client Survey Results All fourteen respondents strongly agreed or agreed (100%) that they were satisfied with the progress they made during therapy. And, all fourteen clients said ‘yes’ they would recommend the clinic to others. Five participants who had had therapy elsewhere rated the experience at IPFW as “superior”, four as “good” as previous speech therapy treatment received by licensed speech-language pathologists.

b. AAC Theater Camp Goal: CSD 39900 in conjunction with CSD 40500: Students will participate in the preparation and delivery of a weekend theater camp for children who use augmentative or alternative communication devices. This experience is similar to, but separate from, the clinical practicum experience. The Ways of Knowing areas met with this experience include: A1 and 2; B4, 5, and 7. In the Fall of 2015 seven seniors participated in this experience in which seven disabled youths who use speech synthesizers to communicate rehearsed and performed a play which was present to the public at the end of the weekend (Friday evening to Sunday evening) camp.

c. Directed Study-CSD 590 Goal: 90% successful completion of an independent project developing research skills in a topic area of interest. All the Ways of Knowing areas are met with this project.

5 Three seniors developed successful research projects, one for the Honor’s Certificate. Two projects were presented at the Student Research Symposium March 2015. The project Hearing Screening pass/fail rates among Head Start Children, won first place honors from Sigma Xi for presentations in science and third place for the symposium as a whole. There was 100% successful completion of Directed Study projects. Summary Twenty-four seniors graduated in December 2014 and spring 2015. Seventeen of these (64%) participated in at least one capstone experience, six students participated in two experiences. While the head-count goal of 80% was missed, it is important to note that seven of our 22 graduates finished their coursework in December. CSD54900 (clinical practicum) and CSD39900 (gerontology clinic) are offered to seniors only in spring semester. A past limitation on capstone experiences for CSD students has been the number of clinically certified faculty who can supervise the clinical experiences or number of faculty who can develop research plans with students. Because of the increase in CSD majors in the past few years enrollment in these courses has been limited to seniors with GPA’s of 3.5 or above due to a shortage of faculty supervisors 5. External Measures-Exit Surveys: at exit and three years after

A. Exit survey of December 2014 - Spring 2015 graduates A survey was conducted to determine the perceived merits of the department as judged by the most recent graduates once formally departed IPFW. In fall 2014 and spring 2015 all graduating seniors were asked to participate in an exit survey. Fifteen of 25 forms were completed. Goal: Overall satisfaction of 100% 1. Strengths: 1. Academic classes: all respondents expressed satisfaction with courses and instructors calling them excellent or ‘great’. All enjoyed the smaller class sizes and hands on opportunities. 2. Clinic: Those who participated in clinical experiences and judged it as a highlight of their college experience. 3. Student life: Students felt the bond with others and enjoyed the friendly atmosphere; 12 participated in four activities 4. Academic advising: Students believed they were well served by their advisors and commented on how each cared for them and their academic progress.

6

5. Dept. facilities: Students were reasonably satisfied with the area, the appreciated workspace, most called the space cozy. Many commented on that it should be larger. 6. Special dept. meetings: All appreciated the value of the graduate preparation seminar. 7. NSSLHA (Speech and Hearing Club/ASL-PAH): Few commented, but indicated they enjoyed the Club experiences.

All commented on the academic rigor and the friendliness of the department both faculty and students which made their academic experience at IPFW meaningful and pleasant. B. Annual Tally of Graduates and Graduate School Acceptance class of 2014-15 Goal: Ninety percent graduate. December 2014 and May 2015 Graduates, N = 24 of 25 96% The one who did not graduate is retaking courses to improve her overall GPA and plans to graduate in the following year. Goal: Eighty percent acceptance of those who apply to graduate school Programs in speech-language pathology Applicants = 12 Acceptances = 10 (83%) Other MA-level programs Audiology Clinical Doctorate Applicants = 3 Acceptances = 3 (100%) Library and Information Sciences Applicants = 1 Acceptances = 1 (100%) Total applicants, 2014-15 graduates, 16 of 25 (64% of graduates) Total acceptances 14/16 (88% of applicants) The goal of 90% graduating seniors was met and has been met for the past 5 years. The 80% acceptance into graduate school goal was met. Of the two graduates who were not accepted into graduate school one found work as a Behavioral Therapist for the Autism Center and plans to reapply to graduate school for fall 2017. The other graduate works as a speech-language pathology aid in a school system in Elkhart.

7 The competition for places in MA programs is increasingly keen. For example the average GPA of applicants accepted into programs is typically over 3.7 while all MA granting programs have five or more applicants for each of their available places. CSD is proud of the accomplishments and success of the majors. C. Alumni Survey Goal: To determine the impact of particular departmental activity as now perceived by graduates three years post-graduation.

These surveys are sent out every three years to graduates of the program who finished their degrees at least three years ago. The next year to collect alumni surveys is spring 2017 for classes of 2014, 2015 and 2016. Special Student Opportunities

Language and Literacy Project GOAL: CSD 399: Students will take a class on language and literacy and participate in planning and providing reading tutoring in a literacy camp held in Summer II for children from the Fort Wayne community who are emergent or struggling readers. This experience is similar to but separate from, the clinical practicum experience. The Ways of Knowing areas met with this experience include: A1 and 2, B4, 5, and 7. In the summer of 2014 (four students) and 2015 (five students) participated in this experience in which 5 children (2014) and seven children (2015) attended a four day a week camp during the month of July in which they participated in large group read aloud, shared reading, small group literacy activities and Writer’s Workshop. This new opportunity has been offered two times so the department may consider making it a choice for the department capstone experience in the future with appropriate rubrics for assessment included. The American Sign Language Choir continued this past year. This group is open to any student on campus, although it is helpful if the student has or is taking CSD 181-Introduction to American Sign Language. A group of eight CSD students performed at the IPFW graduation ceremonies in May 2014. The Sign Choir is another way for students to socialize and to participate in meaningful pre-professional activities with their peers. 6. Conclusions and Next Steps A. Improvements 1. The interim writing project in CSD 42000 should be revisited to determine if this is still the desired vehicle to measure progress and accomplishment. If the project is used, more specific rubrics for the success of the students’ work should be developed. The rubric should coordinate with the Department Learning Objectives one and two.

8

2. SLOs from courses can be used as within course measurements. Specific rubrics should be developed according to the department overall goals and the university goals to document student growth. 3. The capstone opportunities are challenging for the faculty due to the small size and lack of time to add on another experience for the students. Two projects that of the AAC Camp and the Language Literacy Camp should develop rubrics for assessment. Earlier awareness of the opportunity to do the 590 Independent experience was commented on by the graduates. Again, this is challenging for the faculty since they are all teaching, doing research and advising at capacity. For those graduating in December, the capstone experience must be discussed earlier so they have time to participate if they desire to do so. 4. The core of the assessment program is course embedded assessments, aligned to programmatic SLOs and assessment of professional practice. Indirect measures presently include formal student feedback. In addition students are encouraged to complete a capstone experience (discussed in 3 above) which serves as an additional assessment. 5. Assessment measures should be revisited to determine which are still meaningful and what other measures may achieve better information to assist with curricular changes and reflect the strength of the department. The small size of the faculty creates logistical challenges for programmatic assessment; however, the department will be working with IPFW’s Assessment Office and Director in the spring to build on current strengths in an effort to improve the assessment plan.

B. Next Steps

The caring faculty and family like atmosphere was commented on by almost every graduating senior. The high graduation rate speaks well of the satisfaction students feel as they complete their bachelor’s degree. The excellent acceptance into graduate school rate also speaks well of the academic and clinical experiences. From the exit surveys most students commented on the aging equipment and lack of space particularly when classes are larger. Future plans should include expanding the facilities and upgrades on the audiological equipment. Many commented on the need to implement the Master’s degree in speech-language pathology which now may be a real possibility for the future. When this takes place, there would need to be expansion of the facilities as well as more faculty. Further work on the department assessment plan needs to done to align with the 2020 plan. Systematic methods and rubrics for documenting the data collected in courses imbedded assessments need to be developed and included.

9 Specific student learning objectives for each course need to be redone to reflect the 2020 assessment goals. CSD 11500 has met that task this fall. See Appendix B. The faculty will work with the Assessment Director to develop a common rubric for the Capstone project(s) to guide programmatic innovation and improvement. Submitted by, Lucille Hess, Ph.D., CCC-SLP Interim Department Administrator November 2015

10 APPENDIX A Criteria for Evaluation of Writing and Research Paper (CSD42000) Excellent Good Fair Poor Thesis Clearly stated Clearly Stated Adequately Unclear Concise Appropriate stated Lacks depth Appropriate for new Lacks depth Information Wordy Poorly stated Development of Thesis Ideas Clear, organized Clear-lacks some Clear, lacks Unclear, un- Appropriate flow organization organization organized Use of Sources Accurate, no Accurate with few Accurate with Many errors Errors minor errors some major Errors Appropriate Appropriate, Lacks major Limited use, Good, depth major sources sources lack major Sources Style Accurate APA Accurate APA Errors in APA Did not Style style style follow APA Mechanics Under 5 errors Over 5 errors, Over 10 errors, Over 15, Less than 10 major grammar major Grammar Errors

11 APPENDIX B

CSD 115 Student Learning Objectives/Goals

The student will demonstrate when and to whom to refer persons who have communication disorders. The student will demonstrate strategies to facilitate communication with persons who have communication disorders. The student will describe characteristics of typical/atypical speech, language, hearing and swallowing. The student will describe how to effectively include a person with communication disorders in conversation and social situations.

Department of Communication 2015 Undergraduate Assessment Report Section 1: Student Learning Outcomes for the Program Student learning outcomes for all graduates of the Department of Communication:

1. Articulate the historical traditions of the discipline 2. Demonstrate an awareness of and skillful use of technologies relevant to their major 3. Explain communication concepts and theories relevant to their major 4. Explain, apply, and evaluate the processes involved in productive conflict in the contexts

(e.g. interpersonal, small group, organizational, mediated, public) relevant to their major 5. Demonstrate awareness of diverse perspectives 6. Read, speak, write, and listen competently

In addition to the six outcomes for all graduates, Media and Public Communication (COMM) graduates will be able to:

7. Identify and analyze the interrelation among media economics and relevant institutions and agencies

8. Critically analyze media and public communication 9. Identify and analyze instances of the interdependent relations between media and society 10. Demonstrate a basic understanding of the terminology of mediated and public

communication 11. Identify and analyze the form, structure and techniques of mediated or public texts in

their entirety, and consider how they function in a larger context In addition to the six outcomes for all graduates, Interpersonal and Organizational Communication (COMI) graduates will be able to:

7. Evaluate interpersonal and/or group interactions 8. Communicate competently (effectively, appropriately, ethically) interpersonally and/or in

groups

2

Section 2: Curricular Maps

A. Map of Programmatic Student Learning Outcomes to Baccalaureate Framework, and B. Map of Programmatic Student Learning Outcomes to identified “core courses” in the

curriculum * All courses are COM courses, and “core courses” appear in bold font and are highlighted Student learning outcomes for all graduates of the Department of Communication: Student Learning Outcome

Baccalaureate Framework Goal(s)

Classes where this Outcome Should be Met*

1. Articulate the historical traditions of the discipline

Acquisition of Knowledge 300

2. Demonstrate an awareness of and skillful use of technologies relevant to their major

Acquisition of Knowledge 120, 308, 331, 332, 334, 337, 480

3. Explain communication concepts and theories relevant to their major

Acquisition of Knowledge 212, 248, 253, 300, 303, 310, 318, 324, 325, 330, 410, 422

4. Explain, apply, and evaluate the processes involved in productive conflict in the contexts (e.g. interpersonal, small group, organizational, mediated, public) relevant to their major

Application of Knowledge Critical Thinking and Problem Solving Communication

212, 303, 316, 318, 410, 471

5. Demonstrate awareness of diverse perspectives

Critical Thinking and Problem Solving Communication Personal and Professional Values A Sense of Community

212, 300, 303, 310, 312, 314, 316, 318, 330, 338, 410, 422

6. Read, speak, write, and listen competently

Application of Knowledge 212, 300, 303, 308, 310, 314, 316, 318, 325, 333, 334, 410, 471

3

Section 2: Curricular Maps (continued from the previous page) In addition to the six outcomes for all graduates, Media and Public Communication (COMM) graduates will be able to: Student Learning Outcome Baccalaureate Framework

Goal(s) Classes where this Outcome Should be Met*

7. Identify and analyze the interrelation among media economics and relevant institutions and agencies

Acquisition of Knowledge Personal and Professional Values

250

8. Critically analyze media and public communication

Application of Knowledge Critical Thinking and Problem Solving Communication

248, 250, 253, 318, 330, 338, 401, 421, 422

9. Identify and analyze instances of the interdependent relations between media and society

Acquisition of Knowledge Personal and Professional Values A Sense of Community

250, 330, 421, 422

10. Demonstrate a basic understanding of the terminology of mediated and public communication

Acquisition of Knowledge 248, 253, 318, 332, 421

11. Identify and analyze the form, structure and techniques of mediated or public texts in their entirety, and consider how they function in a larger context

Acquisition of Knowledge Application of Knowledge

248, 253, 318, 333, 334, 401, 421

In addition to the six outcomes for all graduates, Interpersonal and Organizational Communication (COMI) graduates will be able to: Student Learning Outcome Baccalaureate Framework

Goal(s) Classes where this Outcome Should be Met*

7. Evaluate interpersonal and/or group interactions

Application of Knowledge Personal and Professional Values A Sense of Community Critical Thinking and Problem Solving Communication

212, 310, 320

8. Communicate competently (effectively, appropriately, ethically) interpersonally and/or in groups

Application of Knowledge A Sense of Community Communication

212, 310, 320, 410

4


A. Description of Department’s Assessment Model: How is the department assessing student progress to Programmatic SLO at key, common points in matriculation to degree?

• Portfolios: Per the Department of Communication Assessment document, “Undergraduate programmatic assessment will take place every other year with the evaluation of undergraduate portfolios required in the final capstone course (COM 480/499). These portfolios will be gathered each semester, by randomly drawing 25% of portfolios from the final capstone course (COM 480/499) (stratified to match major and/or concentration proportions for that year). Beginning in Spring, 2011, we will assess the portfolios each year to refine the portfolio instructions and assessment rubrics. In Spring, 2013, we will begin an every other year rotation of having faculty assess the portfolio.” (The entire Department of Communication Assessment document is available as Appendix A.)

• Alumni surveys and employer surveys: Per the Department of Communication Assessment document, “The assessment committee will conduct graduate and undergraduate alumni and alumni employer surveys every five years. This external exit measure will seek opinions as to the degree of accomplishment of the overall goals of the curricula.” (The entire Department of Communication Assessment document is available as Appendix A.)

B. Measures Used

• Portfolios: Per the Department of Communication Assessment document, “Portfolios will be assessed according to the objectives listed in the portfolio manual for each major/concentration. The faculty will use a rubric designed to evaluate how well the portfolios show mastery of the objectives.” (The entire Department of Communication Assessment document is available as Appendix A. Also, the portfolio assessment forms with rubrics are available as Appendix D.)

• Alumni surveys and employer surveys: Alumni surveys were conducted via Qualtrics. The alumni survey was available from June 29, 2015 through October 2, 2015. COM alumni (who graduated between May 2011 and May 2015) were contacted twice via email, if a current email address was available, and asked to complete the alumni survey. In addition, a link to the alumni survey was posted on the Department of Communication’s official Facebook page. Within this alumni survey, there were questions that asked about the alum’s employer. If contact information was provided by the alum, the employer received an employer survey. (The alumni survey is available as Appendix B, and the employer survey is available as Appendix C.)

5

Section 3: Assessment Plan (continued from the previous page)

C. Rubrics or Evaluation Metrics Descriptions • Portfolios: Portfolios were assessed using rubrics with the following ratings and

definitions: (Please note: Based on the Department of Communication’s 2014 assessment report, the COAS Assessment Committee made the following recommendation, “Written evidence of newly adopted definitions for rubric labels”, and the below directly addresses this recommendation.)

o 5: Advanced: far on or ahead in development or progress; adept o 4: Proficient: a high degree of competence or skill; expertise o 3: Competent: the ability to do something sufficiently; demonstrates

basic understanding o 2: Substandard: work doesn’t demonstrate/doesn’t meet learning

objective; deficient in one or more ways o 1: Unacceptable: Work fails to meet basic standards; minimal o M: Missing submission o I: Inappropriate submission

(Please note: Based on the Department of Communication’s 2014 assessment report, the COAS Assessment Committee made the following recommendation, “The benchmark measures the department will use to evaluate the successfulness of the programs. Set a standard for your department (i.e. 8/10 as a B on portfolios) and report the number of positive scores as well as means”, and the below directly addresses this recommendation.)

o Benchmark to evaluate the successfulness of the programs: The following was stated in the 2013 Communication Assessment Report, “Students must have a mean of three (competent) on the rubric and earn a four (proficient) in at least half of the student learning outcomes for their major.” For this year’s report, the wording was fine-tuned to accurately reflect assessment procedures. Therefore, this benchmark now reads as follows, “The Communication Department faculty strives for students to earn a mean of three (competent) on the rubric and earn a four (proficient) in at least half of the student learning outcomes for their major.” (Please refer to part “a” of section 4.)

(The portfolio assessment forms, including the definitions for rubrics’ labels, are available as Appendix D.)

6


C. Rubrics or Evaluation Metrics Descriptions (continued from the previous page) • Alumni surveys and employer surveys: The alumni and employer surveys both

asked questions directly related to the Student Learning Outcomes. o More specifically, the following Student Learning Outcomes are directly

related to the following questions on the alumni survey:

Student Learning Outcome Question on the Alumni Survey Demonstrate awareness of diverse perspectives

Select the number that indicates the extent to which your coursework in communication helped you to demonstrate an awareness of diverse perspectives.

Read, speak, write, and listen competently

Select the number that indicates the extent to which your coursework in communication helped you develop effective reading, speaking, writing, and listening skills.

Critically analyze media and public communication

Select the number that indicates the extent to which your coursework in communication helped you to critically analyze media and public communication.

Identify and analyze instances of the interdependent relations between media and society

Select the number that indicates the extent to which your coursework in communication helped you to identify and analyze instances of the interdependent relations between media and society.

Evaluate interpersonal and/or group interactions

Select the number that indicates the extent to which your coursework in communication helped you to evaluate interpersonal and/or group interactions.

Communicate competently (effectively, appropriately, ethically) interpersonally and/or in groups

Select the number that indicates the extent to which your coursework in communication helped you to communicate competently interpersonally and/or in groups.

7


C. Rubrics or Evaluation Metrics Descriptions (continued from the previous page) • Alumni surveys and employer surveys: (continued from the previous page)

o The following Student Learning Outcomes are directly related to the following questions on the employer survey:

Student Learning Outcome Question(s) on the Employer Survey Read, speak, write, and listen competently

This employee possesses the oral communication skills necessary to perform her/his current position well. This employee possesses the written communication skills necessary to perform her/his current position well. This employee possesses the listening skills necessary to perform her/his current position well.


This employee possesses the small group communication skills necessary to perform her/his current position well.

Demonstrate awareness of diverse perspectives

This employee possesses the ability to adapt and work in a diverse environment.

Demonstrate an awareness of and skillful use of technologies relevant to their major

This employee possesses the technical and/or computer knowledge and skills necessary to perform her/his current position well.

(The alumni survey is available as Appendix B, and the employer survey is available as Appendix C.)

D. Description of Plan for Disseminating and Using Findings for Programmatic Learning

Improvements • Portfolios: The 2015 Department of Communication Undergraduate Assessment

report, with recommendations from the assessment committee, will be available to COM full-time faculty. Feedback from COM full-time faculty regarding improvement will be considered and/or implemented.

• Alumni surveys and employer surveys: The 2015 Department of Communication Undergraduate Assessment report, with recommendations from the assessment committee, will be available to COM full-time faculty. Feedback from COM full-time faculty regarding improvement will be considered and/or implemented.

8


A. Current Year Assessment Findings • Portfolios:

(Please note: Based on the Department of Communication’s 2014 assessment report, the COAS Assessment Committee made the following recommendation, “Additional analysis of the raw data you gather”, and the below directly addresses this recommendation.) * Means were rounded to the nearest tenth of a percent. Furthermore, missing submissions (M) and inappropriate submissions (I) were factored as zero (0). Student Learning Outcome (SLO)

Mean (out of 5.0) for COMM graduates*

Mean (out of 5.0) for COMI graduates*

Mean (out of 5.0) for COMM and COMI graduates*

Benchmark: Students earn a mean of three (competent) on the rubric Benchmark met?

Articulate the historical traditions of the discipline

3.1 3.8 3.8 Mean: 3.6 Benchmark met: Yes

Demonstrate an awareness of and skillful use of technologies relevant to their major


Explain communication concepts and theories relevant to their major


Explain, apply, and evaluate the processes involved in productive conflict in the contexts (e.g. interpersonal, small group, organizational, mediated, public) relevant to their major

3.5 3.6 4 Mean: 3.7 Benchmark met: Yes

Demonstrate awareness of diverse perspectives

2.7 3.6 3 Mean: 3.1 Benchmark met: Yes

Read, speak, write, and listen competently


9

Section 4: Assessment Results (continued from previous page)

A. Current Year Assessment Findings (continued from previous page) • Portfolios: (continued from previous page)

Student Learning Outcome (SLO)





Identify and analyze the interrelation among media economics and relevant institutions and agencies

3.7 N/A (SLO only applicable to COMM graduates)

3.3 Mean: 3.5 Benchmark met: Yes

Critically analyze media and public communication


3 Mean: 3.6 Benchmark met: Yes

Identify and analyze instances of the interdependent relations between media and society



Demonstrate a basic understanding of the terminology of mediated and public communication



Identify and analyze the form, structure and techniques of mediated or public texts in their entirety, and consider how they function in a larger context



Evaluate interpersonal and/or group interactions

N/A (SLO only applicable to COMI graduates)

3.6 3 Mean: 3.3 Benchmark met: Yes

10


A. Current Year Assessment Findings (continued from previous page) • Portfolios: (continued from previous page)

Student Learning Outcome (SLO)






N/A (SLO only applicable to COMI graduates)

3.8 3.8 Mean: 3.8 Benchmark met: Yes

Overall, this portfolio is assessed as (Please note: since this is not a student learning outcome, this was not included in the below benchmark.)


Benchmark: Students earn a four (proficient) in at least half of the student learning outcomes for their major Benchmark met?

Number of outcomes that earned at least a four: 2 Total number of student learning outcomes: 11 Benchmark met: No



11


A. Current Year Assessment Findings (continued from previous page) • Alumni surveys and employer surveys:

(Please note: Based on the Department of Communication’s 2014 assessment report, the COAS Assessment Committee made the following recommendation, “Additional analysis of the raw data you gather”, and the below directly addresses this recommendation.) * Means were rounded to the nearest tenth of a percent.

Alumni Survey Question directly related to Student Learning Outcome




Mean for all COM graduates*

Select the number that indicates the extent to which your coursework in communication helped you demonstrate an awareness of diverse perspectives.

4.9 4.1 5.0 4.7

Select the number that indicates the extent to which your coursework in communication helped you develop effective reading, speaking, writing, and listening skills.

4.7 4.3 5.0 4.7

Select the number that indicates the extent to which your coursework in communication helped you to critically analyze media and public communication.

4.7 N/A (survey question only applicable to COMM graduates)

5.0 4.9

Select the number that indicates the extent to which your coursework in communication helped you to identify and analyze instances of the interdependent relations between media and society.

4.4 N/A (survey question only applicable to COMM graduates)

5.0 4.7

12


A. Current Year Assessment Findings (continued from previous page) • Alumni surveys and employer surveys: (continued from previous page)

Alumni Survey Question directly related to Student Learning Outcome




Mean for all COM graduates*

Select the number that indicates the extent to which your coursework in communication helped you to evaluate interpersonal and/or group interactions.

N/A (survey question only applicable to COMI graduates)

4.7 4.5 4.6

Select the number that indicates the extent to which your coursework in communication helped you to communicate competently interpersonally and/or in groups.

N/A (survey question only applicable to COMI graduates)

4.4 5.0 4.7

Employer surveys No data provided. Only one alum provided employer contact information. This employer was contacted, and asked to complete a survey. The survey was not returned by the employer.

13


B. Proposed Changes to Address Findings • Portfolios: The Department of Communication Undergraduate Assessment

Committee has four suggestions to address the findings. First, the Department of Communication Undergraduate Assessment Committee will continue to recommend to instructors of the portfolio courses (COM 120, COM 308 and COM 480/499) as well as all faculty teaching courses that might provide portfolio artifacts that they clarify to students the objectives and the types of items most suitable for the portfolios. Secondly, the Department of Communication Undergraduate Assessment Committee will recommend to all faculty teaching courses that might provide portfolio artifacts that they clearly/directly explain to students how/why a specific assignment meets a specific student learning outcome. Third, in an effort to ensure an understanding of the assessment forms, the Department of Communication Undergraduate Assessment Committee will “fine-tune” the explanations of some of the student learning outcomes. (The explanations of the student learning outcomes are provided in Appendix D). Finally, in an effort to ensure an understanding of the assessment forms, prior to the assessment of the portfolios, the Department of Communication Undergraduate Assessment Committee will discuss the portfolio assessment forms with rubrics (Appendix D) with the faculty members assessing the portfolios.

• Alumni surveys and employer surveys: The Department of Communication Undergraduate Assessment Committee has four suggestions to address the findings. First, the wording of the question asking for the alum’s name needs to be revised to clearly/directly remind them that the survey is no longer anonymous if a name is provided. Secondly, while posting a link on Facebook increased visibility and responses, that strategy could have inadvertently compromised the data, due to the fact that anyone (including non-alumni) could have potentially completed a survey. Future surveys should be directly targeted towards known alumni. Third, while the response rate to the alumni survey increased greatly since the last time one was conducted (addressed in section 4 part c), additional strategies could be implemented to increase future response rates (i.e. maintain stronger/ongoing connection with alumni, offer incentives to complete the alumni survey, etc.). Finally, the Department of Communication Undergraduate Assessment Committee will establish benchmarks to measure the graduates’ perceptions of the Communication Department’s successfulness.

14


C. Prior Year Assessment Findings and Description of Changes Made • Portfolios: Since portfolios were last assessed (fall 2013), the Department of

Communication Undergraduate Assessment Committee amended the rubric to include a new rating scale. (Previously, a 10-1 scale was used.) Secondly, the Department of Communication Undergraduate Assessment Committee provided written definitions of the rubrics and established benchmarks. (Provided in part “c” of section 3.)

• Alumni surveys and employer surveys: Since the alumni surveys and employer surveys were last conducted, the Department of Communication Undergraduate Assessment Committee made three key changes. First, when the alumni surveys and employer surveys were last conducted (included in the fall 2011 assessment report), the alumni were contacted once via a post card sent by the United States Postal Service. The response rate for the alumni survey was 5%, and the response rate for the employer survey was 0%. When the alumni survey was conducted this year, of the 268 alumni, 28 responded. This was 10.4% response rate; this was more than double the earlier response rate. In an effort to achieve this, alumni received two emails and the survey was posted multiple times on the Department of Communication’s official Facebook page. Secondly, when the alumni surveys and employer surveys were last conducted (included in the fall 2011 assessment report), the alumni survey was only available for approximately two weeks. When the alumni survey was conducted this year, the availability was increased to three months. The final change was focused on the survey questions themselves. More specifically, since the alumni survey was last conducted (included in the fall 2011 assessment report), the student learning outcomes were updated. When the alumni survey was conducted this year, alumni were asked survey questions directly related to the updated student learning outcomes.


• Portfolios: The undergraduate curriculum was revised for the fall 2014 semester. More specifically, the Communication major is composed of four new concentrations, and the student learning outcomes for each of these concentrations were updated. As a result, the portfolio assessment forms will be updated accordingly. These updated portfolio assessment forms will be used by the instructors in the portfolio classes (COM 120, COM 308, and COM 499).

• Alumni surveys and employer surveys: The undergraduate curriculum was revised for the fall 2014 semester. More specifically, the Communication major is composed of four new concentrations, and the student learning outcomes for each of these concentrations were updated. As a result, the alumni surveys and employer surveys will be updated accordingly. More specifically, both surveys will ask questions directly related to the updated student learning outcomes.

15

Section 5: Conclusion, Next Steps, and Communication (Please note: Based on the Department of Communication’s 2014 assessment report, the COAS Assessment Committee made the following recommendation, “Conclusions based upon portfolio scores and other assessments”, and the below directly addresses this recommendation.) Based on this assessment report, the Department of Communication Undergraduate Assessment Committee recommends the following conclusions, next steps, and communication:

1. Student learning outcomes: The Department of Communication undergraduate curriculum was revised for the fall 2014 semester. More specifically, the Communication major is composed of four new concentrations, and the student learning outcomes for each of these concentrations were updated. As a result, the portfolio assessment forms, alumni survey questions, and employer survey questions must be updated accordingly.

2. Portfolios: Based on this assessment report, the Department of Communication Undergraduate Assessment Committee recommends the following four conclusions, next steps, and communication. First, Department of Communication Undergraduate Assessment Committee will continue to recommend to instructors of the portfolio courses (COM 120, COM 308 and COM 480/499) as well as all faculty teaching courses that might provide portfolio artifacts that they clarify to students the objectives and the types of items most suitable for the portfolios. Secondly, the Department of Communication Undergraduate Assessment Committee will recommend to all faculty teaching courses that might provide portfolio artifacts that they clearly/directly explain to students how/why a specific assignment meets a specific student learning outcome. Third, in an effort to ensure an understanding of the assessment forms, the Department of Communication Undergraduate Assessment Committee will “fine-tune” the explanations of some of the student learning outcomes. (The explanations of the student learning outcomes are provided in Appendix D). Finally, in an effort to ensure an understanding of the assessment forms, prior to the assessment of the portfolios, the Department of Communication Undergraduate Assessment Committee will discuss the portfolio assessment forms with rubrics (Appendix D) with the faculty members assessing the portfolios.

3. Alumni surveys and employer surveys: Based on this assessment report, the Department of Communication Undergraduate Assessment Committee recommends the following four conclusions, next steps, and communication. First, the wording of the question asking for the alum’s name needs to be revised to clearly/directly remind them that the survey is no longer anonymous if a name is provided. Secondly, while posting a link on Facebook increased visibility and responses, that strategy could have inadvertently compromised the data, due to the fact that anyone (including non-alumni) could have potentially completed a survey. Future surveys should be directly targeted towards known alumni. Third, while the response rate to the alumni survey increase greatly since the last time one was conducted (addressed in section 4 part c), additional strategies could be implemented to increase future response rates (i.e. maintain stronger/ongoing connection with alumni, offer incentives to complete the alumni survey, etc.). Finally, the Department of Communication Undergraduate Assessment Committee will establish benchmarks to measure the graduates’ perceptions of the Communication Department’s successfulness.

16

Appendix A: Department of Communication Assessment Document

ASSESSMENT OF ACADEMIC ACHIEVEMENT DEPARTMENT OF COMMUNICATION

The faculty of the Department of Communication acknowledges the desirability of the systematic review of student academic achievement. Such review will allow identifying strengths and weaknesses of current curricula and pedagogy as well as developing and implementing curricular and pedagogical efforts to maximize the identified strengths and minimize the identified weaknesses. The academic accomplishments of graduate students in Professional Communication will be assessed by graduate faculty review of the synthesis paper required of all students who complete the curriculum. This internal exit assessment will be administered by the Director of Graduate Study with assistance from graduate faculty. They will review the papers and prepare a biannual collective summary of the extent to which the papers demonstrate accomplishment of the goals for the graduate program as articulated in “Mission and Goals -- Department of Communication” approved September 13, 1993. Each four years the collective summaries will be reviewed by the Director of Graduate Study and curricular and/or pedagogical adaptations will be presented for consideration to the faculty of the department. Undergraduate programmatic assessment will take place every other year with the evaluation of undergraduate portfolios required in the final capstone course (COM 480/499). These portfolios will be gathered each semester, by randomly drawing 25% of portfolios from the final capstone course (COM 480/499) (stratified to match major and/or concentration proportions for that year). Beginning in Spring, 2011, we will assess the portfolios each year to refine the portfolio instructions and assessment rubrics. In Spring, 2013, we will begin an every other year rotation of having faculty assess the portfolio. Portfolios will be assessed according to the objectives listed in the portfolio manual for each major/concentration. The faculty will use a rubric designed to evaluate how well the portfolios show mastery of the objectives. Finally, in an effort to ensure understanding of the assessment forms, the Department of Communication Undergraduate Assessment Committee will discuss the portfolio assessment forms with rubrics (Appendix D) prior to the faculty members assessing the portfolios. The assessment committee will conduct graduate and undergraduate alumni and alumni employer surveys every five years. This external exit measure will seek opinions as to the degree of accomplishment of the overall goals of the curricula. The committee will prepare a recommendation based on the responses. Every five years the assessment devices and adaptations will be reviewed by the assessment committee. The committee will prepare recommendations for general and specific modifications of the assessment procedures. Internal interim and external exit assessments will begin Spring, 1994. The entire process will function by June 30, 1995. Approved 11/8/93 Amended 3/28/94 Amended 4/24/2001 Amended 4/18/2003 Amended and approved 4/29/09 Amended and Approved 10/11/13 Amended and Approved 3/25/15

17

Appendix B: Alumni Survey

IPFW Department of Communication Alumni Survey

The purpose of this survey is to assess graduates’ experiences in the Department of Communication at IPFW. This survey should take less than 10 minutes to complete. Your feedback is valued, and your answers will be kept completely confidential. Click here for the survey. URL: URL for the survey itself: https://purdue.qualtrics.com/SE/?SID=SV_9WWBDHSxdCOQASN Please answer the following questions based on your experience in the Department of Communication at IPFW. Your answers will be kept completely confidential.

1. What was your major? a. Media/Public (COMM) b. Interpersonal/Organizational (COMI) c. Media/public and Interpersonal/organizational (COMM and COMI) d. MA/MS Professional Communication

Questions for all alumni (COMM, COMI, and MA/MS)

2. Select the number that indicates the extent to which your coursework in communication has helped you professionally.

5 4 3 2 1 Helped a lot Did not help at all 3. Select the number that indicates the extent to which your coursework in communication

helped you to reach your professional goals. 5 4 3 2 1

Helped a lot Did not help at all

4. Select the number that indicates the extent to which your coursework in communication helped you develop effective analytical skills.

5 4 3 2 1 Helped a lot Did not help at all

5. Select the number that indicates the extent to which your advisor helped you reach your

academic goals. 5 4 3 2 1


6. In what specific ways did the Communication Department support your personal and/or professional goals? (open-ended question)

18

Appendix B: Alumni Survey (continued from previous page)

7. In what specific ways could the Communication Department have better supported your personal and/or professional goals? (open-ended question)

Questions for COMM and COMI alumni

8. Select the number that indicates the extent to which your coursework in communication helped you to demonstrate an awareness of diverse perspectives.


helped you develop effective reading, speaking, writing, and listening skills. 5 4 3 2 1

Helped a lot Did not help at all Questions for COMM alumni

10. Select the number that indicates the extent to which your coursework in communication

helped you to critically analyze media and public communication. 5 4 3 2 1


11. Select the number that indicates the extent to which your coursework in communication helped you to identify and analyze instances of the interdependent relations between media and society.


Questions for COMI alumni

10. Select the number that indicates the extent to which your coursework in communication helped you to evaluate interpersonal and/or group interactions.


helped you to communicate competently interpersonally and/or in groups. 5 4 3 2 1


19

Appendix B: Alumni Survey (continued from previous page) Questions for MA/MS Professional Communication alumni


helped you to apply research methods relevant to your area of study. 5 4 3 2 1


11. Select the number that indicates the extent to which your coursework in communication helped you to integrate your knowledge of communication theory, communication research, and applied communication skills.



helped you to develop an understanding of the multi-dimensional nature of the discipline of Communication.



helped you to demonstrate performance of and/or critical competence in communication skills relevant to your particular area of study.


Questions for all alumni (COMM, COMI, and MA/MS) Name (optional) ________________________________________________________________ Current Position ________________________________________________________________ Does your current position utilize the knowledge/skills you learned in your degree? Yes Somewhat No

Your participation and insights on the previous questions are appreciated and valued. Your employer may be able to provide additional insights into IPFW’s Communication Program and the skills/knowledge gleaned from the program. Therefore, please consider answering the last two questions. May we contact your supervisor with a brief survey? Yes No If yes, please provide your supervisor’s name and contact information (i.e. business address, phone, and/or email).

20

Appendix C: Employer Survey Dear _______ _______________, an alumnus/alumna of the Department of Communication at Indiana University Purdue University Fort Wayne, has provided your name to us as his/her employer or supervisor. As a department, we value assessment both inside and outside the classroom. With that said, we are gathering data about alumni and their career preparation after graduation. This is where you come in. We value your insight regarding how this alumnus/alumna demonstrates competencies central to our degree programs in his/her work (i.e. outside the classroom). This is where the true mark of an education can be understood and realized. We ask for a few minutes of your time and your honest feedback. By completing this brief survey you help us to continually improve our degree programs that we think, will in turn, benefit the employers of our future graduates. This assessment process also helps to maintain the integrity of the degrees granted at IPFW and specifically the Communication Department. We welcome your insights. Please consider participating in the development of our program. Please click on the link below to complete the survey. This should take no more than five minutes. Your answers will be kept strictly confidential. Thank you very much for you time, expertise, and insights. We appreciate your participation. Sincerely, The Assessment Committee Melissa Dietrich Sarah LeBlanc Kevin Stoller Company Name:___________________________ Your Name: ________________________ Type of Business:___________________________ Employee/Evaluee’s Position:______________________

21

Appendix C: Employer Survey (continued from previous page) ______________________________________________________________________________ Please select the most appropriate number following the below scale: Strongly Agree Agree Neutral Disagree Strongly Disagree Not Relevant 5 4 3 2 1 N/R ______________________________________________________________________________ This employee possesses the oral communication skills 5 4 3 2 1 N/R necessary to perform her/his current position well: This employee possesses the written communication skills 5 4 3 2 1 N/R necessary to perform her/his current position well: This employee possesses the listening skills necessary to 5 4 3 2 1 N/R perform her/his current position well: This employee possesses the small group communication 5 4 3 2 1 N/R skills necessary to perform her/his current position well: This employee possesses the ability to adapt and work in a 5 4 3 2 1 N/R diverse environment: This employee possesses the technical and/or computer 5 4 3 2 1 N/R knowledge and skills necessary to perform her/his current position well: This employee possesses the critical thinking skills 5 4 3 2 1 N/R necessary to perform her/his position well: This employee demonstrates the problem solving and 5 4 3 2 1 N/R decision-making skills necessary to perform her/his current position well: This employee demonstrates a sufficient amount of 5 4 3 2 1 N/R knowledge, in and across the discipline of her/his profession, necessary to perform her/his current profession well: Based on this employee’s ability to communicate, how prepared do you think they are to succeed in their current position? Based on this employee’s ability to communicate, what do you think could have been done to better prepare them for their current position? Is there anything else you would like to share regarding this employee’s ability to communicate?

22

Appendix D: Portfolios Assessment Forms with Rubrics (Ratings and Definitions)

Communication: Media and Public Major Portfolio Assessment Form

Student Name: _________________________________________________________________

The student has been evaluated on the manner in which his or her portfolio effectively and professionally addresses each of the following communication area learning outcomes.

Rating legend: 5: Advanced: far on or ahead in development or progress; adept 4: Proficient: a high degree of competence or skill; expertise 3: Competent: the ability to do something sufficiently; demonstrates basic understanding 2: Substandard: work doesn’t demonstrate/doesn’t meet learning objective; deficient in one or more ways 1: Unacceptable: Work fails to meet basic standards; minimal M: Missing submission I: Inappropriate submission

LEARNING OUTCOMES RATING Be able to articulate the historical traditions of the discipline: The student has demonstrated that s/he is able to articulate the history and development of communication as a field of study; understanding of the development of theories and methods; fundamental background in the literature and chronology of communication studies, as well as key historical figures

5 4 3 2 1 M I

Be aware of and skillful in the use of new technologies relevant to your major; The student has demonstrated that s/he is aware of and skillful in the use of new technologies relevant to her/his major. The student has demonstrated general computer literacy including the ability to use software programs, online databases, and Internet resources; ability to present ideas professionally through media including word processing and presentation software (i.e., PowerPoint), creation of website or web-based software, photography, and audio-video-film production.

5 4 3 2 1 M I

23

Appendix D: Portfolios Assessment Forms with Rubrics (Ratings and Definitions) (continued from previous page)


(continued from previous page)

LEARNING OUTCOMES RATING Be able to explain communication concepts and theories relevant to your major; The student has demonstrated a conceptual understanding of the communication concepts including knowledge of various genres and major theories. Areas of theoretical study may include mass communication, nonverbal, group, cultural and gender, rhetoric, interpersonal, and persuasion.

5 4 3 2 1 M I

Be able to explain, evaluate and apply the processes involved in productive conflict in the contexts (interpersonal, small group, organizational, mediated, public) relevant to the major; The student has demonstrated the ability to analyze diverse perspectives involved in a conflict and respond in a competent manner.

5 4 3 2 1 M I

Demonstrate awareness of diverse perspectives; The student has demonstrated awareness of diverse perspectives (i.e., in terms of cultural, ethnic, sex, gender, religious, or sexual orientation differences).

5 4 3 2 1 M I

Be a competent reader, speaker, writer, and listener (a course artifact may mean one or more of these). The student has demonstrated his/her ability to comprehend written and/or mediated text, deliver an effective oral presentation (with appropriate organization, support, and delivery), write an effective paper (well organized, coherent argument, appropriate supported), and listen effectively (exhibits active listening skills with appropriate responses).

5 4 3 2 1 M I

Identify and analyze the interrelation among media economics and relevant institutions and agencies; The student has demonstrated understanding of how two or more of the media industries and some agencies (i.e. companies) within those industries influence each other, compete with one another, and support one another and perhaps other agencies (i.e. governmental).

5 4 3 2 1 M I

24




LEARNING OUTCOMES RATING Critically analyze media and public communication; The student has demonstrated the ability to critically analyze messages in media (possibly including but not limited to commercials, advertisements, narratives, technical components, etc.) and/or messages in public communication (possibly including but not limited to public address, campaigns, news media, political punditry, public opinions, etc.).

5 4 3 2 1 M I

Identify and analyze instances of the interdependent relations between media and society; The student has demonstrated the ability to critically analyze how media influence society, and individuals in that society, and has demonstrated the ability to critically analyze how society and individuals in that society influence the media industries.

5 4 3 2 1 M I

Demonstrate a basic understanding of the terminology of mediated and public communication; The student has demonstrated an understanding of terminology used within mediated and/or public communication and the student has demonstrated an understanding of terminology used to analyze mediated and/or public communication.

5 4 3 2 1 M I

Identify and analyze the form, structure, and techniques of mediated or public texts in their entirety, and consider how they function in a larger context. The student has demonstrated the ability to critically analyze the form, structure, and techniques in media [possibly including but not limited to narrative structure, production techniques (i.e. shot composition, lighting technique, mise-en-scene, etc.), genre, etc.] and messages in public communication (possibly including but not limited to presentational structure and/or style, persuasive/rhetorical techniques, etc.).

5 4 3 2 1 M I

Overall, this portfolio is assessed as:

5 4 3 2 1 M I

_______________________________ _______________________

Faculty Signature Date

_______________________________ _______________________


25


Communication: Interpersonal/Organizational Major Portfolio Assessment Form

Student Name: _________________________________________________________________




5 4 3 2 1 M I


5 4 3 2 1 M I

Be able to explain communication concepts and theories relevant to your major; The student has demonstrated a conceptual understanding of the communication concepts including knowledge of various genres and major theories. Areas of theoretical study may include mass communication, nonverbal, group, cultural and gender, rhetoric, interpersonal, and persuasion.

5 4 3 2 1 M I

26




LEARNING OUTCOMES RATING Be able to explain, evaluate and apply the processes involved in productive conflict in the contexts (interpersonal, small group, organizational, mediated, public) relevant to the major; The student has demonstrated the ability to analyze diverse perspectives involved in a conflict and respond in a competent manner.

5 4 3 2 1 M I


5 4 3 2 1 M I

Be a competent reader, speaker, writer, and listener (a course artifact may mean one or more of these). The student has demonstrated his/her ability to comprehend written and/or mediated text, deliver an effective oral presentation (with appropriate organization, support, and delivery), write an effective paper (well organized, coherent argument, appropriately supported), and listen effectively (exhibits active listening skills with appropriate responses).

5 4 3 2 1 M I

Evaluate interpersonal and/or group interactions (a course artifact may meet either context); Student has demonstrated the ability to evaluate key components of an interpersonal interaction (i.e., perspective taking, self-monitoring, ways to deal with conflict, assertiveness) or group interaction (member roles, leadership behavior, productive conflict management, task/relational/procedural roles).

5 4 3 2 1 M I

27




LEARNING OUTCOMES RATING Communicate competently (effectively, appropriately, ethically) interpersonally and/or in groups (a course artifact may meet either context). The student has demonstrated the ability to consider him/herself and “other” in the interpersonal or group interaction and to be effective (in reaching instrumental and relational goals), appropriate (for the situation and the goals) and ethical.

5 4 3 2 1 M I


5 4 3 2 1 M I

_______________________________ _______________________


_______________________________ _______________________


28


Communication: Interpersonal/Organizational and Media/Public Major Portfolio

Assessment Form

Student Name: _________________________________________________________________




5 4 3 2 1 M I


5 4 3 2 1 M I

Be able to explain communication concepts and theories relevant to your major; The student has demonstrated a conceptual understanding of the communication concepts including knowledge of various genres and major theories. Areas of theoretical study may include mass communication, nonverbal, group, cultural and gender, rhetoric, interpersonal, and persuasion.

5 4 3 2 1 M I

29



Assessment Form (continued from previous page)

LEARNING OUTCOMES RATING Be able to explain, evaluate and apply the processes involved in productive conflict in the contexts (interpersonal, small group, organizational, mediated, public) relevant to the major; The student has demonstrated the ability to analyze diverse perspectives involved in a conflict and respond in a competent manner.

5 4 3 2 1 M I


5 4 3 2 1 M I

Be a competent reader, speaker, writer, and listener (a course artifact may mean one or more of these). The student has demonstrated his/her ability to comprehend written and/or mediated text, deliver an effective oral presentation (with appropriate organization, support, and delivery), write an effective paper (well organized, coherent argument, appropriately supported), and listen effectively (exhibits active listening skills with appropriate responses).

5 4 3 2 1 M I

Identify and analyze the interrelation among media economics and relevant institutions and agencies; The student has demonstrated understanding of how two or more of the media industries and some agencies (i.e. companies) within those industries influence each other, compete with one another, and support one another and perhaps other agencies (i.e. governmental).

5 4 3 2 1 M I

Critically analyze media and public communication; The student has demonstrated the ability to critically analyze messages in media (possibly including but not limited to commercials, advertisements, narratives, technical components, etc.) and/or messages in public communication (possibly including but not limited to public address, campaigns, news media, political punditry, public opinions, etc.).

5 4 3 2 1 M I

Identify and analyze instances of the interdependent relations between media and society; The student has demonstrated the ability to critically analyze how media influence society, and individuals in that society, and has demonstrated the ability to critically analyze how society and individuals in that society influence the media industries.

5 4 3 2 1 M I

30



Assessment Form (continued from previous page)

LEARNING OUTCOMES RATING Demonstrate a basic understanding of the terminology of mediated and public communication; The student has demonstrated an understanding of terminology used within mediated and/or public communication and the student has demonstrated an understanding of terminology used to analyze mediated and/or public communication.

5 4 3 2 1 M I

Identify and analyze the form, structure, and techniques of mediated or public texts in their entirety, and consider how they function in a larger context. The student has demonstrated the ability to critically analyze the form, structure, and techniques in media [possibly including but not limited to narrative structure, production techniques (i.e. shot composition, lighting technique, mise-en-scene, etc.), genre, etc.] and messages in public communication (possibly including but not limited to presentational structure and/or style, persuasive/rhetorical techniques, etc.).

5 4 3 2 1 M I

Evaluate interpersonal and/or group interactions (a course artifact may meet either context); Student has demonstrated the ability to evaluate key components of an interpersonal interaction (i.e., perspective taking, self-monitoring, ways to deal with conflict, assertiveness) or group interaction (member roles, leadership behavior, productive conflict management, task/relational/procedural roles).

5 4 3 2 1 M I

Communicate competently (effectively, appropriately, ethically) interpersonally and/or in groups (a course artifact may meet either context). The student has demonstrated the ability to consider him/herself and “other” in the interpersonal or group interaction and to be effective (in reaching instrumental and relational goals), appropriate (for the situation and the goals) and ethical.

5 4 3 2 1 M I


5 4 3 2 1 M I

_______________________________ _______________________


_______________________________ _______________________


1

Graduate Studies Committee (Roberts, Cain, Rumsey)

Department of English and Linguistics

Assessment 2014–15

Introduction: Student Learning Outcomes Statement

Students who complete the Master of Arts in English (36 hours) will be able to

• Demonstrate the knowledge of one specific area of study in English (Literature; Language and Linguistics; Writing Studies)

• Demonstrate an understanding and appreciation of the diverse scholarly approaches to major issues in the study of literature, language and linguistics, or rhetoric/composition/writing

• Write professional papers that demonstrate critical and analytical thinking and other necessary skills for independent research and writing

Students who complete the Master of Arts for Teachers of English (36 hours) will be able to

• Demonstrate knowledge of those areas of study in English that are relevant to their development as teachers at the secondary or college level

• Demonstrate an understanding and appreciation of the diverse scholarly approaches to major issues in the study of literature, language and linguistics, or rhetoric/composition/writing

• Write professional papers that demonstrate critical and analytical thinking and other necessary skills for independent research and writing

The assessment plan for the English & Linguistics graduate program include both direct and indirect measures:

Direct assessment measures Indirect assessment measures

• Evaluation of collected sample student essays from M.A. core courses and 700-level graduate seminars.

• Exit surveys of graduating students.

• Surveys of current graduate students.

• Tracking the success of program alumni.

2

Assessment of Sample Graduate Essays

Each year, the committee reviews sample graduate essays using a three-question rubric based on our graduate program student learning outcomes. For all responses, 5 indicates strongly agree and 1 indicates strongly disagree. The sample essays examined by the committee this year were collected using the program’s policy of taking sets of essays from MA core courses and 700-level seminars.

Q1. The essay demonstrates the knowledge of [one specific area of study].

Q2. The essay demonstrates an understanding and appreciation of diverse scholarly approaches to the major issues in the study of [relevant area of study].

Q3. The essay demonstrates critical and analytical thinking and other necessary skills for independent research and writing.

Assessment Results: Sample Graduate Essays

For the 2014-2015 academic year, essays were collected from 5 graduate courses from the 2014-2015 academic year, following the department’s policy of collecting essays from MA core and seminar courses. These courses were: B501 Professional Scholarship in Literature (Bassett), B605 Critical Theory (Lin), B753 Poetry of the American West (Kalamaras), B780 East-West Influences (Lin), C780 Workplace and Organizational Writing (Rumsey). Papers from C780 were then excluded from assessment because the course was taught by a member of the graduate studies committee. Out of a total of 24 papers, that left 15 papers available, of which 12 (50%) were randomly selected for evaluation and scoring based on the department’s assessment rubric for graduate student essays. These 12 papers were drawn from: B501, B605, B753, and B780. The committee evaluated each essay and assigned scores based on the rubric, using the following benchmarks:

• 1.5 or below indicates poor quality • 1.51 – 2.5 indicates below average quality • 2.51 – 3.5 indicates average quality • 3.51 – 4.5 indicates above average quality • 4.51 – 5.0 indicates excellent quality

3

Essay Q1 Q2 Q3 A 4.83 4.50 4.66 B 4.93 4.93 4.93 C 3.66 2.83 3.50 D 4.93 4.93 4.93 E 3.66 3.10 3.66 F 4.16 4.33 4.33 G 4.60 4.26 4.60 H 4.80 4.96 5.00 I 4.43 4.20 4.66 J 3.66 3.33 3.33 K 4.00 4.10 4.16 L 3.50 3.66 3.33 Overall Average 4.26 4.09 4.25

MA Exit Survey

In the past, the committee has solicited exit surveys from graduating graduate students. For this year’s report, an online exit survey was emailed to the 13 graduating graduate students. We received a total of 2 responses, for a response rate of 15.4%. Note: unlike in past years, this year’s exit survey was incorporated as part of the department’s survey of all graduating students, both undergraduate majors and master’s graduates. The low rate of return, along with the desirability of returning to some form of separate survey of graduating master’s degree students, will be discussed below in the Conclusions and Recommendations section.

Assessment Results for Exit Survey

1. When did you graduate?

Spring 2015

May 2015

2. Which degree did you earn?

MA 2

MAT 0

4

TENL Certificate (Graduate)

0

3. What was your concentration?

Language 0

Literature 1

Writing 1

4. Did you complete a minor? [Not applicable to graduate students]

5. If so, which one? [Not applicable to graduate students]

6. Did you complete a second major? [Not applicable to graduate students]

7. Are you currently employed?

Yes 1

No

No response 1

8. If so, where? What is your title?

IPFW – Coordinator of Composition, Communication, and Supplemental Instruction

No response

9. Are you currently working in your area of study?

Yes 1

No

No response 1

5

10. Do you have plans to pursue additional education? If yes, please select which option(s) you are most interested in.

No 1

MA/MAT Program 0

Ph.D. Program 1

JD 0

Other (Please explain) 0

11. Feel free to provide any additional comments/testimonial you may have

Am also teaching English Comp at Ivy Tech.

No response

Assessment Results: Survey of Current Graduate Students

Surveys were distributed in hardcopy to students taking Fall 2015 graduate-only courses (B605, B753, C517, and C780). Surveys and reminders were then emailed to 19 students currently enrolled, or taking courses as temporary students, in the MA or MAT program, in order to ensure all students had a chance to respond. Twelve surveys were returned, for a response rate of 63.2%. This is a marked improvement over the response rate from last year’s assessment report, when only 23.3% of current students returned the survey. This is likely due to our new method of distributing the survey in hardcopy during graduate-only courses, the benefits of which will be discussed below in the Conclusions and Recommendations section.

What is your area of concentration?

Literature 3

Writing Studies 8

Language and Linguistics 1

6

For each of the following statements, 5 indicates strongly agree and 1 indicates strongly disagree. NR indicates no response: 5 4 3 2 1 NR Average (n=12)

Program requirements provide you the opportunity to gain a satisfactory knowledge of your area of concentration.

5 6 1 4.25

5 4 3 2 1 NR Average (n=12)

The MA degree requirements afford you the opportunity to develop an understanding of major issues and approaches in your areas of study.

5 6 1 4.25

5 4 3 2 1 NR Average (n=12)

You are taught or encouraged to develop the research skills needed for graduate-level work in English.

4 6 2 4.00

5 4 3 2 1 NR Average (n=12)

You are taught or encouraged to develop the writing skills needed for graduate-level work in English.

5 4 1 2 4.00

5 4 3 2 1 NR Average (n=11)

The graduate advisor keeps you informed of your progress toward the degree and assists you in completing requirements in a timely manner.

6 2 2 1 1 4.18

5 4 3 2 1 NR Average (n=6)

Online/hybrid courses you have taken have provided satisfactory alternatives to traditional classroom learning.

3 1 2 6 3.16

7

What are your plans upon finishing your degree? (Students were to select all that applied) pursuing a doctorate in English or Linguistics 6

pursuing a law degree

pursuing a degree in library and information sciences

pursuing another degree: 1

working as a writer; as a researcher; as a journalist 7

working in publishing/editing/copywriting 7

working as an elementary or secondary school teacher 1

working as an instructor at a community college or university

10

working as a teacher abroad 4

working as a librarian/archivist 1

working in public relations, advertising, or marketing 4

working in academic administration, arts administration, or business management

1

working in government or public service

working for a nonprofit organization 3

other: 1

Please use the back of this sheet to provide any written comments or to make any suggestions for improving the program. (4 of 12 students responded to this request – 33.3%.) Very pleased with the opportunities afforded. Hopeful a doctorate becomes an option in the future. There needs to be more writing classes where we learn to write in different professional genres. Theory of Rhetoric is great and all but there need to be more applicable classes that teach skills.

8

We have a phenomenal faculty in comp/rhet yet not as many courses listed (granted it has gotten better). Cross-listed courses are less effective. I feel satisfied with my experiences so far.

Assessment Results: Tracking the success of program alumni:

In the last three years, our program has graduated 42 students: 40 M.A. and 2 M.A.T. Of those, we have been able to track the academic and/or career paths of 35 program alumni (83.3%). Our data show that they are involved in fields and pursuits which are related to their degrees, as follows (note: several alumni are involved in multiple fields):

Accepted into Ph.D. programs 2

Accepted into Law School 1

University or College Instructors 19

English Teachers at Secondary Schools 11

Writers/Editors 4

Business or Academic Administration 5

Conference Presentations (Note: this total includes current students as well as alumni.)

20

9

Conclusions and Recommendations Based on the survey and essay scores of this limited sample, our students demonstrate above-average quality in their work and in their own assessment of our program. Averaged essay scores to each of the three queries from our rubric (4.26, 4.09, and 4.25) place our students’ performance in the above-average range of our benchmark (3.51 – 4.5). Likewise, student responses to the survey of current graduate students are also within the above-average range, with the exception of the response to a query concerning online and hybrid courses (3.16), which falls within the average benchmark range (2.51 – 3.5): this response with be discussed further in the recommendations below. Students’ concerns continue to focus on course diversity and scheduling, as well as on the availability of graduate-student-only courses. Seminar diversity and course scheduling remain issues that are complicated by fluctuating enrollment numbers, by the availability of faculty, and by other staffing concerns. However, the program has begun to broaden the availability of writing studies seminars beyond C780 with the introduction of a new seminar, C723 Literacy & Family History Research. Also, the new M.A. requirements, which took effect in Fall 2014, increase the number of required seminars from two to three, thereby requiring students to take at least one additional non-cross-listed course. The 2013-2014 assessment report recommended the program should undertake a revision of the exit survey and should explore alternative ways of collecting surveys from students. Response: As explained above, this year’s exit survey was incorporated as part of the department’s survey of all graduating students, both undergraduate majors and master’s graduates. It was hoped that this shorter survey would encourage a greater response than in previous years, but this in fact did not occur. The very low rate of return, coupled with the smaller amount of information gathered by this survey, indicate that a return to a separate, more focused graduate student exit survey is needed, along with new strategies to increase the rate of response. Such strategies will be discussed in the recommendations section below. On the other hand, the number of returned current student surveys increased from the previous year’s 23.3% to 63.2%, as a direct result of distributing the survey in hardcopy to students taking Fall 2015 graduate-only courses, and then following up that distribution with an emailed survey and reminder to all current graduate students.

The 2013-2014 assessment report also recommended the program should continue and complete the survey of other graduate programs to find and implement solutions to current assessment issues and to discover how our methods align with other comparable programs. Response: We have requested information on assessment methods from English graduate programs at the following seventeen universities: Ball State, Boise State, Central Oklahoma, Cleveland State, CUNY - Staten Island, Eastern Michigan, IUPUI, Nebraska at Omaha, New Orleans, Northern Kentucky, Oakland (MI), Portland State, South Alabama, Texas at El Paso, Wichita State, Wright State, and Youngstown State. Of these seventeen programs, six responded to our request. (See Appendix A for full responses.) This response rate (35.3%) is symptomatic of the larger assessment problem of low response rates to student surveys, suggesting that there is little incentive at any level – undergraduate, graduate, or professional – for anyone outside of an institution to respond to such requests for assessment information. Furthermore, when considering those six programs which did respond, there does not seem to be a consistent assessment plan among our peer M.A. programs: one program uses mailed exit surveys, one uses current student surveys, one uses exit interviews with an on-site survey, while two use the M.A. thesis, two collect sample papers or portfolios of graduate student work, and two track the success of their graduates. One does no formal assessment at all. In light of the wide variance in these programs’ assessment methods, our own assessment plan appears to be at the very least above average. We do both

10

exit surveys and current student surveys, collect sample essays for evaluation by committee via a standard rubric, and attempt to track the success of our program alumni.

The 2013-2014 assessment report also recommended the program should continue our initiative to trace the career paths of our program alumni, to publicize their successes as a recruitment tool, and to increase current students’ awareness of possible careers they could pursue with their degrees. Response: Last year, the program undertook an initiative to trace as many of our alumni as possible so that we can provide accurate information about how they have used their master’s degrees in their chosen fields. Findings: The English & Linguistics Master of Arts program granted 262 degrees between 1978 and spring of 2015. Of these, 173 were Master of Arts degrees and 89 were Master of Arts for Teachers degrees. We continue to update this data, and at present we have career and contact information for all but 71 of our alumni. Now that we have a working database of alumni information, we are in a position to make use of that data for purposes of publicizing our program, recruitment, and current student advising. Strategies for using this data are discussed in the recommendations section below. See Appendix B for detailed information for the last three years.

The 2013-2014 assessment report also recommended the program should examine the effects of the 2014 changes to M.A. degree requirements, especially on the ability of students to enroll more easily in the courses they need to complete the program in a timelier manner. Response: New M.A. requirements, which went into effect for Fall 2014, reduced the number of core course from four to two, increased the required number of area concentration courses from four to five, increased the required number of 700-level seminars from two to three, and streamlined the literature concentration requirements. These changes appear to have had the following positive effects:

• All students during the 2014-2015 academic year were able to enroll in the core courses which they needed.

• More faculty have expressed interest in teaching a greater variety of seminars. • Students with literature concentrations have been able to utilize a wider variety of courses to

complete the concentration requirements.

11

Response to the COAS Assessment Committee’s review of the 2013-2014 assessment report: • “A discussion of continued efforts to increase participation in student surveys”: As explained

above, the program seeks assessment data via two kinds of student survey: 1) the exit survey of graduating students and 2) the current student survey. Participation in each kind of survey was widely different, with a 15.4% response rate for exit surveys and a 63.2% response rate for current student surveys. The greater level of participation in current student surveys can be directly attributed to distribution of that survey in hardcopy to students in graduate-only courses. As the responses to our query of other graduate programs suggest, exit surveys are notoriously difficult to obtain from students who have left the institution and who may no longer have any contact with the program. The answer seems clear: face-to-face interactions and direct hardcopy distribution will dramatically increase participation. Suggestions for how the program can accomplish this are discussed in the recommendations below.

• “More clearly identify the benchmark(s) used to draw conclusions about meeting learning outcomes”: The Assessment Rubric for Sample Graduate Essays has been revised to include a range of benchmark scores:

• 1.5 or below indicates poor quality • 1.51 – 2.5 indicates below average quality • 2.51 – 3.5 indicates average quality • 3.51 – 4.5 indicates above average quality • 4.51 – 5.0 indicates excellent quality

The current student survey also uses a 5 point scale, and so these benchmarks are also applicable to students’ evaluations of the program via surveys as well as to the committee’s evaluation of sample student essays.

Response to the COAS Assessment Committee’s Request for Additional Desirable (but not required) Report Features:

• “discussion of the reliability and validity of the measure(s) used”: Our direct assessment via evaluation of sample student essays is our most reliable and valid form of assessment. The collection of essays from multiple core courses and seminars assures that we have an adequate sample of papers, and the process of committee evaluation employs the opinions of three faculty members on each sample essay. Furthermore, essays produced by students in these courses are precisely the kinds of professional research and writing tasks which the program learning outcomes are designed to measure. Reliability and validity of student surveys can be compromised by a low rate of response (addressed in the recommendations below); however, the higher response rate on current student surveys does demonstrate a consistent set of responses and concerns among our students. Finally, tracking the success of our alumni gives us insight into the continuing success of our program, as the data gathered indicates the multiple academic and career paths our graduates have been able to pursue. The greatest indication of the reliability and validity of our assessment process can be found in the consistency of the findings across multiple assessment measures. For example, the graduate studies committee’s above-average ratings via the assessment rubric closely match the above-average ratings of the program from the current student surveys.

• “[indicate] whether the measures used are direct or indirect”: See the explanation of our assessment plan on page 1 of this report.

• “identify examples of successful programs and student learning”: For examples of the successes of our students and alumni, see Appendix B.

12

This report has the following recommendations for the program to pursue in the coming year:

1. Methods for improving the return of and also increasing the information gathered by exit surveys should be implemented. This will require the following:

a. A revision of the exit survey. The graduate studies committee and the graduate director should undertake these revisions in order to have the survey ready for Spring 2015 graduates. Such revisions should streamline the survey and make sure it is in line with the learning outcomes and assessment benchmarks.

b. Most importantly, the rate of participation must be increased. As mentioned above, the solution would seem to include face-to-face interactions and direct hardcopy distribution of the exit survey. To this end, the graduate director should institute a policy of exit interviews coupled with hardcopy distribution and onsite completion of the exit survey. Methods might include either one-on-one interviews with individual students or a simultaneous meeting with all graduating students to discuss their ideas about the program. Whichever method is used, students will need to be assured that their responses to exit surveys will remain anonymous.

2. After graduating 42 students in the last three years, our current enrollment numbers are down. The program should explore ways to recruit new students and increase enrollments. Such methods may include contacting area English departments which do not offer master’s degrees to advertise our program, increasing awareness of the program on campus, reaching out to our alumni (see below), and asking department faculty to discuss the program with promising undergraduate English majors.

3. Now that the program has a more extensive database of alumni, we should make use of that information in recruitment, advising, and program visibility. We have begun to incorporate alumni success information into the new graduate student orientation meeting which takes place each fall. The graduate studies committee, the graduate director, and the faculty should explore additional ways to make use of this data, such as including alumni stories in the department newsletter, inviting successful alumni to visit campus and talk with current students, publicizing alumni success to the campus, to current students, to prospective students, and to the community.

4. Scores on the survey question regarding online/hybrid courses were relatively lower than scores for other aspects of the program, indicating lower student satisfaction. The statement “Online/hybrid courses you have taken have provided satisfactory alternatives to traditional classroom learning” elicited a score of 3.16, which places it in the average range, but well below other aspects of the program. The graduate director should survey faculty who have taught online/hybrid courses and gather information about their experiences and practices in these courses, along with ideas for making them more attractive and useful to future students.

Appendix A: Comparisons of Graduate Program Assessment Methods

Grad Director Email

Ball State

Joyce Huff [email protected]

For our undergraduate programs, we select a random sample of essays, and a committee scores them according to a 3-question rubric, as you do. However, for our graduate programs, we use the “natural” endpoints – the MA project or thesis and the Ph.D. comprehensive exam and dissertation. For each of these, committee members fill out the rubric, rating each student on three Student Learning Outcomes. The rubric is on Googledocs and the entire committee agrees on the rating. We feel that these points represent the culmination of each student’s learning.

New Orleans

Robert Shenk [email protected]

We do an exit questionnaire, attempting to get graduating students to complete the questionnaire before they leave campus (relatively successful). We don't do the regularly general survey; we do have something like your 12-essay survey (but don't do it for the half of the students who did the thesis option instead). It sounds like you have a very good mix. Beyond that, we also track students once they have graduated, having gotten good data on maybe half the graduates from the last 15 years — to see what fields they are in: how many have gone on for the PhD, how many teach K-12 somewhere, how many teach or work at universities in some capacity, how many become professional writers (one of our concentrations, how many are in other business positions where they can use their writing skills, and how many go into fields not necessarily related to their Master's degree.

Portland State

Graduate Coordinator [email protected]

We have no formal “assessment” strategy in our English M.A. program. Our Department has a Literary Studies Committee, a Curriculum Committee, and each strand of our MA/MS program has its own committee and a director, who keeps close tabs on the success of his or her program, the quality of its exit exams or theses, and its graduates. Plus, we have a first-rate faculty who invest a great deal of time into our graduate students, their writing and their academic success. Graduates from our English MA program, for example, have gone on to prestigious Ph.D. programs at Stanford, Cornell, Fordham, Ohio, all the UC system schools, Univ. of Wisconsin, U Washington, Notre Dame, Univ. of York, etc. I think we feel our track record on this is a decent assessment. Our students go on to law school as well as business and teaching careers, besides the Ph.D. We do regularly re-examine our exit exam system through our committees, and tweak it accordingly. Our program is not so large nor so well-funded that we can do more, nor would I care to. We have, however, tried with varying success, to have feedback from our graduates just at or after their graduation, using a survey/questionnaire. I’m reassuming this position after a break of several

mailto:[email protected]



Grad Director Email

years, so if we make some changes, I’d like to get back to that post-graduation survey more formally, and would love a newsletter that goes to graduates and would receive news from them. The Graduate Coordinator keeps a general log of the achievements of our graduates, and many of them send us information on their lives after PSU that fills out this picture, thus helping us to see how we are doing.

South Alabama

Ellen Burton Harrington [email protected]

All students in both concentrations (creative writing and literature) write a thesis, so we assess each thesis using numerical ratings in four areas specific to each concentration and asking each rater to comment anecdotally on the strengths and weakness of the thesis. There are two to three faculty members rating each thesis. In addition to the general survey that our Graduate School emails to all graduate students each fall, our chair meets briefly with each graduating student for a quick exit interview each May which is accompanied by a quick and anonymous questionnaire completed on site. (Students who come to these interviews get a free departmental t-shirt as an incentive.)

Texas at El Paso

Tom Schmid [email protected]

Our undergraduate literature program does something very similar to the assessment you outlined for your graduate program (random selection of papers evaluated according to a brief rubric) but so far we have not implemented an annual assessment for our M.A. program in literature. All of our graduate degree programs were recently evaluated extensively, however, for a UT system mandate, and that evaluation included gathering data on faculty teaching loads and scholarly productivity, enrollment numbers, employment of students post-graduation, and so forth. I think we will likely looking at having to implement some kind of annual evaluation soon

Youngstown State

Steve Brown [email protected]

The university requires the department to have four student learning outcomes. We are required to assess two each year. We have two measures, one indirect and one direct. The indirect one is a Survey Monkey for graduating MA students. It asks some demographic questions and then asks the students to rate their proficiency on each of the four outcomes. The direct measure is a reading of all graduation portfolios (7-10 a year) by the graduate committee, which is a seven-person group representative of the department. Our program allows an individual student to submit a portfolio or a thesis, depending on goals. We have left assessment of theses to the thesis committees, and the university doesn't seem bothered by that. The portfolio as a requirement has always been somewhat controversial, or at least problematic, which is why we try to stay focused on assessing samples of portfolios. Last year, which was the first time we completely implemented this version of the system, we found a gap between what the students thought they could do and what we saw them doing, which has led us to work on revising the outcomes this year. As for getting information for students once they leave, yes, a challenge. It's something we'd like to do, and something the state would like to know, but we've found no answers.

Number of institutions queried: 17 Number of responses: 6 35.3% response rate





Appendix B: Recent Graduate Student & Alumni Successes The English & Linguistics Master of Arts program granted 262 degrees between 1978 and spring of 2015. Of these, 173 were Master of Arts degrees and 89 were Master of Arts for Teachers degrees. We continue to update this data, and at present we have career and contact information for all but 71 of our alumni. The following list presents examples of recent (2013-2015) graduate student and alumni successes, including career information, publications, conference presentations, and academic service. Calla Andrews (M.A. 2014), Adjunct English Professor and Coordinator of Student Support Services,

Grace College. Darleen Baker (M.A. 2013), Instructor, Ancilla College; Instructor, Ivy Tech-Kokomo. Meghann Bassett (M.A. 2015), Adjunct Instructor, Indiana Institute of Technology. Ana Boman (M.A. 2013 & TENL graduate certificate), Assistant Professor, English as a Second

Language, Trine University; collaborating with the English as a Foreign Language program (EFL) at the Universidad de Guanajuato, Mexico on the impact of technology on intercultural communication. Presenter: "Impact of Technology on Intercultural Communication in English as a Second Language," 2013 Fort Wayne Teaching Conference.

Jessey Britt (M.A. 2015), English Teacher, Northside High School. Joshua Brock (M.A. 2013), Adjunct Professor of English and Assistant Baseball Coach, Manchester

University. Nancy Button, Presenter: “Writing the Smaller Stories: Research Practices in Family History Writing,”

2015 IPFW English & Linguistics Graduate Student Colloquium. Deakin Chipps (M.A. 2014), English Teacher, East Allen University (East Allen Public Schools). Brittany Cowley (M.A.T. 2013), English Teacher, Lakeland High School. Gloria Diaz, "Letters to Karla," 2015 TransWritLarge Conference. Megan (Timmerman) Disque (M.A. 2013), English Teacher, East Noble High School. Amanda Dreher (M.A. 2014), Limited Term Lecturer, IPFW. Presenter: “Sufi Garden and Zen

Mountaintop: The Mysterious Consciousness of Lessing's and Pirsig's First-Person Narrators,” 2014 Louisville Conference on Literature and Culture since 1900; and 2014 IPFW English & Linguistics Graduate Student Colloquium.

Shannon Eichenhauer (M.A. 2013), Adjunct Instructor, Vincennes University; English Teacher, East Allen University (East Allen Public Schools); Ph.D. student at Ball State University. Presenter: "Teaching Under Prepared Students. . . . Wait I Am One of Them," 2012 Fort Wayne Teaching Conference.

Shannon Elward (M.A. 2014), Limited Term Lecturer, IPFW; poems accepted for publication in Calibanonline; co-edited (and co-published) with Ben Larson a new literary magazine, Map Points. Indiana Americorps Vista for ICHE 2015.

Kristine Frye (M.A. 2015), Coordinator of Composition and Communication, IPFW. Veronica Gabet (M.A. 2013), Adjunct Professor, Indiana Institute of Technology. Katie Harris (M.A. 2013), English Teacher, Heritage Jr./Sr. High School. Melissa Hirsch (M.A. 2013), English Teacher, Northrop High School. Presenter: “Teaching the

Transition: Addressing Learning Styles and Levels,” 2012 Fort Wayne Teaching Conference. Stephen Horstman (M.A. 2015), Adjunct Instructor, Hillsborough Community College (Tampa, FL). Darren Hunt (M.A. 2013), Adjunct Faculty, Academic Skills Advancement, Ivy Tech Northeast. Richard Kemery (M.A. 2013), Limited Term Lecturer, IPFW; Information Technology Consultant, Ilion

Consulting; Help Desk Analyst, Lutheran Health Network. Presenter: “‘Wipe Away the Debt’: Quantum Theory in ‘BioShock Infinite,’” 2014 Northeast Modern Language Association.

Zack Kruse (M.A. 2015), forthcoming article: "Steve Ditko: Violence and Romanticism in the Silver Age" in Studies in Comics 5.2; Ph.D. student at Michigan State University. Presenter: “The Right to Kill: Steve Ditko and the Death of the Silver Age,” 2013 Northeast Popular/American Culture Association Conference; and 2014 IPFW English & Linguistics Graduate Student Colloquium. He was also the 2014 NEPCA area chair for Comics and Graphic Novels.

Ben Larson (M.A. 2013), Limited Term Lecturer, IPFW; poems published in Calibanonline ; co-edited (and co-published) with Shannon Elward a new literary magazine, Map Points.

Timothy Leonard, Sigma Tau Delta Midwestern Student Representative 2014-2015. Presenter: “The Influence of Charles Dickens on Sensation Fiction,” 2014 Western Illinois University Graduate Organization and Sigma Tau Delta Conference; Presenter: “Trans-Atlantic Partnerships in 19th Century Periodicals: The Preliminary Approach,” 2015 IPFW English & Linguistics Graduate Student Colloquium.

Leslie Mackey (M.A. 2013), completing Ph.D. at University of Minnesota. Presenter: “From Walls to Paper: Defining Design Literacy and Establishing Textual Meaning through Spatial Manipulation,” 2013 Conference on College Composition and Communication.

Kristi Manduka (M.A. 2013), English Teacher, Warsaw Community High School. Aaron McClaskey, Editor of Confluence 2014-2015; student representative on the COAS dean search

committee in fall 2014/spring 2015. Chelsie McCorkle (M.A. 2015), Adjunct Faculty, University of St. Francis. Presenter: “Deconstructing

Dualism in Harrison’s ‘The Woman Lit by Fireflies’: Inhabiting Liminal Space in Theme and Form,” 2015 IPFW English & Linguistics Graduate Student Colloquium.

Carla Overdahl, Co-Presenter, “Teachers and Students at the Threshold: Exploring Habits of Mind and College Readiness in the Dual Credit Composition Classroom,” 2015 Indiana Teachers of Writing Conference.

Madison Prall, "Maintaining Professional and Ethical Boundaries during Mobile Consultations," 2015 East Central Writing Centers Association Conference.

Colleen Reimer (M.A. 2013), Policy and Procedure Specialist, Vera Bradley. Presenter: “Anonymous Who?: Changing Face, Changing Identity, Changing Response,” 2012 East Central Writing Center Association Conference.

Jacqueline Reynolds (M.A. 2013), Limited Term Lecturer, IPFW; Program Assistant for Graduate Studies, TENL, and Appleseed Writing Project, English & Linguistics, IPFW. Presenter: “Without Empathy: Flannery O’Connor’s Representation of the Imperfect,” 2013 South Atlantic Modern Language Association Conference.

Jack Schroeder (M.A. 2014), Limited Term Lecturer, IPFW. Chad Simpson (M.A. 2014), Credit Analyst, Sumitomo Mitsui Banking Corporation (NYC). Craig Skinner (M.A. 2013), Limited Term Lecturer, IPFW; Teaching Assistant, Norwell High School

(W131). Naomi Stephens (M.A. 2013), former English Teacher, Concordia Lutheran High School; Adjunct

Instructor, Luzerne County Community College (St. Nanticoke, Pennsylvania). Presenter: “Swindled, Forged, and Ruined: Debt-Motivated Suicides in Sensation Fiction,” 2012 Midwest Modern Language Association Conference.

Brittany Straub (M.A.T. 2013), Special Education Teacher, Homestead High School; winner of NBC33's Excellence In Education Award (April 2015).

Sarah Styf (M.A. 2013), English Teacher, Concordia Lutheran High School. Rebecca Ward-Davies (M.A. 2013), attending the University of Georgia-Athens Law School; Instructor

of Writing, Athens Technical College. Muyun Yin (M.A. 2015), Editor/Blogger at Trendy New York; Events Manager and China

Communications, EB5 Investors Magazine (Irvine, CA). Presenter: “The Taoist Water and the Byzantium Fire: The Dialectic of the Finite and the Infinite in the Poetry of Arthur Sze and W.B. Yeats,” 2014 Louisville Conference on Literature and Culture since 1900; and 2014 IPFW English & Linguistics Graduate Student Colloquium.

Emerald Yorke (M.A. 2015), Presenter: “Playing the Hunger Game (and Winning): Suppressed Appetites and Social Criticism in Oliver Twist and The Hunger Games,” 2015 IPFW English & Linguistics Graduate Student Colloquium.

ENG Undergraduate COAS Assessment Report 2014-‐15 (submitted Nov. 2015)

SECTION 1: SLOs

ENG Undergraduate Program General Student Learning Outcomes (not concentration-specific)

1. Students will demonstrate the ability to write critically, precisely, and persuasively, especially about topics relevant to the major field and selected concentration.

2. Students will demonstrate an ability to communicate knowledge of literary and linguistic conventions and traditions, especially those of America and England.

3. Students will demonstrate the ability to use research tools and methods appropriate to their selected concentration.

SECTION 2: ENG CURRICULAR MAPS

A. Map of ENG Programmatic SLOs to Baccalaureate Framework Baccalaureate Framework

1. Acquisition of Knowledge: Students will demonstrate breadth of knowledge across disciplines and depth of knowledge in their chosen discipline. In order to do so, students must demonstrate the requisite information skills and technological competencies.

2. Application of Knowledge: Students will demonstrate the ability to integrate and apply that knowledge, and, in so doing, demonstrate the skills necessary for life-long learning.

3. Personal and Professional Values: Students will demonstrate the highest levels of personal integrity and professional ethics.

4. A Sense of Community: Students will demonstrate the knowledge and skills necessary to be productive and responsible citizens and leaders in local, regional, national, and international communities. In so doing, students will demonstrate a commitment to free and open inquiry and mutual respect across multiple cultures and perspectives.

5. Critical Thinking and Problem Solving: Students will demonstrate facility and adaptability in their approach to problem solving. In so doing, students will demonstrate critical-thinking abilities and familiarity with quantitative and qualitative reasoning.

6. Communication: Students will demonstrate the written, oral, and multimedia skills necessary to communicate effectively in diverse settings.

General ENG SLOs mapped to the BF numbers above

1. English majors demonstrate literary, historical, linguistic, and rhetorical conventions and traditions of English through critically sound oral and written expression reflective of this integration of curriculum material.


2. Students use analytical and rhetorical skills to produce persuasive, critically precise essays and projects that reveal an integration of research skills with the acquired curriculum.

3. Students demonstrate through peer review of written work/projects and sound use of sources in research essays a respect for their colleagues and for the intellectual property used in their research. Student respect for class attendance and for critical engagement in dealing with secondary sources reflect personal integrity and a responsible acquisition of ethical values in literary and rhetorical studies.

4. Students come to recognize diverse communities and beliefs through literary studies that expose them to a multitude of heterogeneous voices. Students write essays and respond verbally to questions that have abiding historical and culture significance (e.g. consequences of war, racism, nationalism, personal bias).

5. Students integrate literary and cultural analysis of a broad range of literary texts to produce critical, analytical writing that responds to questions of genre, character analyses, literary style, and historical significance of various American and British texts.

6. Students integrate written, oral, and multimedia skills to produce rhetorically sound essays, original creative works, and literary analyses that reflect rhetorical precision, clarity of thought and critical understanding of a wide range of historical, cultural, and ethnic texts and situations.

B. Map of ENG Programmatic SLOs to Identified “Core Courses” GENERAL ENG SLOs (not concentration-specific)

A) Students will demonstrate the ability to write critically, precisely, and persuasively, especially about topics relevant to the major field and selected concentration.

B) Students will demonstrate an ability to communicate knowledge of literary and linguistic conventions and traditions, especially those of America and England.

C) Students will demonstrate the ability to use research tools and methods appropriate to their selected concentration.

ENG CORE COURSES ASSESSED FOR AY 2014-15 ENG L202: Literary Interpretation ENG L371: Critical Practices

ENG Core Courses ENG L202 ENG L371 ENG SLO letter

A, B, C A, B, C

♦ ENG L202: Literary Interpretation. The course description in the current IPFW

Bulletin describes L202 thus: “Close analysis of representative texts (poetry,


drama, fiction) designed to develop art of lively, responsible reading through class discussion and writing of papers, including a documented research paper. Attention to literary design of critical method.” As a class approved by COAS to fulfill the writing requirement, by definition this class requires substantial opportunity for students to demonstrate precise, persuasive, and polished critical analysis of literary texts through the production of formal papers. Because the course description requires the production of a research paper, students will demonstrate their ability to utilize research tools such as electronic academic databases in order to garner peer-reviewed articles. Additionally, as this class is a de facto “Introduction to Literature” which covers literature from diverse periods, national origins, and genres, students will demonstrate their knowledge of literary and linguistic conventions and traditions through the production of their formal papers. As a class required of all majors relatively early in their academic careers, L202 provides an excellent opportunity to assess ENG SLOs.

♦ ENG L 371: Critical Practices. As an upper-division/capstone class, L371 obliges students to demonstrate an even more sophisticated understanding of literary and linguistic conventions as manifest in literary theory. The formal papers written in this class require students to demonstrate precise, persuasive, and polished critical analysis of these theories in their active application to representative literary texts. As an upper-division class, there is an expectation that research will be involved in the production of papers, through which students will demonstrate their ability to utilize research tools such as electronic academic databases in order to garner peer-reviewed articles. Because L371 is the major capstone class, it provides an excellent opportunity to assess ENG SLOs at a late point in majors’ academic careers.

SECTION 3: ENG UNDERGRADUATE ASSESSMENT PLAN 2014-15

A. ENG Undergraduate Assessment Model:

1. ENG undergraduate assessment will consist of two metrics: the assessment of artifacts from two core classes, and a survey distributed to graduating seniors soliciting their opinions regarding whether or not ENG met departmental goals. A four-year cycle has been developed for future assessment which will rely on the assessment of artifacts from the capstone class, ENG L371, and another set of artifacts from a rotation of ENG’s three concentrations (literature, language, and writing), to return to an assessment of ENG L202 in the fourth year.

2. The artifacts assessed for AY 2014-15 were taken from ENG L371 and ENG L202. Because these are required classes, these courses provide an excellent opportunity to obtain data about student success at an early, gateway stage (L202) and at a later (capstone) point in the path towards graduation.

3. The commentary from the COAS Assessment Committee suggested the assessment of clearer department SLOs. In order to address this, we decided to


assess the general (not concentration-specific) SLOs as set forth by our department (see https://www.ipfw.edu/departments/coas/depts/english/about/). These SLOs apply to all concentrations within the major, and as such provide a solid base-line for the assessment of departmental success as a whole.

4. An assessment rubric is utilized in the assessment of program goals (see rubric in Section 3.C below). Successful accomplishment of SLOs is determined by an average score of 2 or higher on a 4-point scale by 75% or more of the artifacts in all three SLOs.

5. All assessment artifacts are made anonymous, with student names, etc removed. 6. Representative/statistically valid samples need to be taken from each course

assessed. In the case of ENG L202, artifacts were taken from multiple sections of the class.

7. Five faculty representing all concentrations from within ENG assessed the artifacts. This number of assessors will help ensure the statistical reliability of the average scores produced in assessment.

B. Measures Used This year’s assessment was undertaken exactly as described above in the Assessment Model. In the case of ENG L202, a total of 21 essays taken from three sections of the class offered in Fall 14 and Spring 15 were assessed. Only one section of ENG L371 was assessed, providing a total of 13 papers. Anonymous assessment artifacts were read by five assessors and scored on a 4 point scale utilizing a purpose-designed rubric. Success in meeting program goals was defined as 75% or more of assessed artifacts scoring an average of 2 or higher (on a 4-point scale) on the purpose-designed program assessment rubric for each of the 3 program goals. These goals were clearly met. The survey of graduating seniors was conducted by means of a Qualtrics poll. Responses are scored utilizing a one to five scale, with one indicating “strongly disagree” and five indicating “strongly agree.”

C. ENG Undergraduate Program Assessment Rubric

1) Does the student demonstrate the ability to write critically, precisely, and persuasively, especially about topics relevant to the major field and selected concentration?

Excellent (4) Good (3) Needs Work (2) Unsatisfactory

(1)

2) Does the student demonstrate an ability to communicate knowledge of literary and linguistic conventions and traditions, especially those of America and England?



(1)

3) Does the student demonstrate the ability to use research tools and methods appropriate to their selected concentration?


(1)

Note: the classifications below take into account the specific level (200-, 300-level, etc.) of the class in which the assessment artifact was composed. Success in meeting program

goals is defined as a score of 2 or above for each of the program goals above. EXCELLENT: Student demonstrates exceptional depth and nuance in analysis, using particularly well-chosen examples to illustrate her/his points. Scholarly sources are sufficient in number, particularly well-chosen, thoroughly analyzed, and gracefully integrated into the project. GOOD: Student engages in solid analysis, though with less nuance than above, using solid examples in sufficient number to support/illustrate her/his points. Scholarly sources are sufficient in number, well-chosen, analyzed sufficiently, and integrated into the project. NEEDS WORK: Student engages in analysis but not with great depth and complexity. Some examples are provided, but could be greater in number, better chosen, or analyzed with greater precision. Sources are sufficient in number, but possibly not the most relevant, and/or are less thoroughly analyzed and are introduced abruptly. UNSATISFACTORY: Student engages in minimal or no analysis. Examples are excessively general or non-existent. Insufficient number of scholarly sources or sources are of insufficient scholarly quality and go unanalyzed.

D. Description of Plan for Disseminating and Using Findings for Programmatic Learning Improvement

One of the strengths of ENG’s assessment model is that it relies upon the active participation of faculty from all of the concentrations offered in the major. This affords the opportunity for the assessment committee members to report back to the faculty in


their individual concentrations and to thus use assessment data at a “local level.” In addition, the ENG Undergraduate Assessment Report will be provided to the ENG chair for further distribution and discussion among the department as a whole, as well as with other appropriate stakeholders. SECTION 4: ASSESSMENT RESULTS (see also attached assessment data for more detail)

A. Current Year Assessment Findings Success in meeting program goals for assessed artifacts was defined as 75% or more of the artifacts scoring 2 or higher (on a 4-point scale) on the purpose-designed program assessment rubric for each of the 3 program goals. These goals were clearly met. In ENG L202, 100% of the assessed artifacts scored an average of 2 or higher for SLOs #1 and #2, while 86% scored 2 or higher for the third SLO. In ENG L371, 100% of the average scores for all three SLOs were 2 or higher. The data collected from the survey of graduating majors suggests that in the eyes of its graduates ENG has been successful in its goals. In a total of twenty-five questions ENG scored a four or better (on a five-point scale) in twenty-one of them, an 84% success rate. No question scored less than a 3.58 on the five point scale. While the length of the survey prevents a point-by-point analysis here, ENG can take particular satisfaction in the high marks that we received in some very important categories. When asked to comment on whether they agreed that their “instructors were good teachers” ENG received a score of 4.60 out of 5. Similarly, ENG received a score of 4.55 out of 5 when students were asked “If you had known what you would experience as an English major, would you still have chosen our department?” And finally, 90% of respondents answered “yes” in response to the question “Do you feel that your IPFW degree helped prepare you for your future or current studies or employment?” This is a strong endorsement of our major (and the liberal arts in general) by our students, underscoring the practical value of the skills taught in ENG. See Appendix 1 below for complete assessment data, including scores for assessment artifacts and complete survey results.


While clearly ENG general undergraduate SLOs are being achieved with great success, the only category where less than 100% of artifacts received an average score of 2 or more was observed in ENG L202. This was SLO number three, centered on utilizing research tools and methods. Given that the success rate in this category was 86% (well above the 75% rate established to determine success in the category), any suggestions


made here would be simply be aimed at improvement of an already high level of success, not as suggestions intended to remedy a failure. It should also be noted that the data collected in the capstone class (ENG L371) achieved a 100% success rate in this as well as the other 2 categories, suggesting that whatever weaknesses that students exhibit in early stages of their careers are effectively addressed by the training that they receive as they make progress in their degree. Nonetheless, some potential strategies could be examined to foster further success in research as deemed appropriate by individual instructors, including building additional time into syllabi to be spent instructing students about such resources as Helmke’s electronic databases and Document Delivery Service, “staging” researched papers (building them from an initial assignment to produce an annotated bibliography, for example), and requiring review of rough drafts of research papers with an eye to improving the scores on the final draft. No doubt L202 instructors already incorporate many of these strategies into their classes, but periodic reassessment of one’s assignments and pedagogy can be worthwhile. Regarding the survey results, the four lowest scores center on a few common themes. The lowest score on the survey was 3.58 out of 5 when students were asked to evaluate “advice received for career goals.” In related fashion, ENG received scores of 3.85 and 3.7 out of 5 respectively in response to the questions “my career preparation was adequate” and “my preparation for further (graduate) studies was adequate (if applicable).” The only other question to elicit an average score below 4 out of 5 was one asking students to rate their “experiences or opportunities as a major outside of courses,” which elicited a score of 3.94. While none of these scores indicated dissatisfaction (less than a score of 3), ENG will seek to improve these results from the “neutral” average response. In some respects these results are surprising given that 90% of students answered “yes” in response to the question “Do you feel that your IPFW degree helped prepare you for your future or current studies or employment?” Nor was it an indictment of faculty advising in general, as the score for the “advice given for selection of courses” was 4.37 out of 5; similarly, a score of 4.3 out of 5 resulted when students were asked if “instructors were helpful outside of class.” What this may indicate is a desire for more formal opportunities to discuss career pathways with ENG faculty. It may be worthwhile for ENG to discuss at a department level the institution of more formal or frequent workshops for students on topics such as applying to graduate school. As a largely commuter campus where students are often working one or more jobs, IPFW does not have the same built-in opportunities (or funding) taken for granted at residential colleges to foster the esprit de corps that comes from extracurricular activities. In order to improve students’ “experiences or opportunities as a major outside of courses,” ENG could discuss further opportunities to involve students in extracurricular activities. For example, additional department-sponsored events such as student readings or performances perhaps could be instituted. In addition, were additional funding made available to do so, student participation in academic conferences could be further


encouraged. This not only would help professionalize majors (preparing them for the expectations of their future employment), but it also would help to provide students with lines for their CVs that would help them in their graduate school applications.


Last year’s assessment focused on different outcomes than this year’s: the COAS Assessment Committee’s evaluation of last year’s report suggested greater precision was required in both the wording of SLOs and the criteria utilized to assess them. In addition, (by the ENG undergraduate assessment committee’s own admission) the survey distributed to graduating seniors did not elicit enough responses to be statistically valid: only four replies were forthcoming. To remedy this, AY 2014-15’s assessment focused on the departmental SLOs established for all ENG majors irrespective of concentration, utilizing a purpose-designed assessment rubric to evaluate each of the three general SLOs. In addition, this year a much larger sample size was returned for the survey distributed to graduating seniors: AY 2014-15 garnered responses from a total of twenty-two students. Last year’s assessment had assessment artifacts reviewed by at least four assessors drawn from three different ENG subfields (e.g. literature, writing, linguistics). A total of fourteen artifacts drawn from ENG L371 were assessed. Assessors utilized a rubric to assess the following criteria on a one to five scale, where one was “strongly disagree” and five was “strongly agree,” with an average score of three being deemed a successful demonstration of competence in the outcome:

1. Rhetorical content: The paper/project addresses audience and task. 2. Disciplinary content: What raters look for as evidence of knowledge of

disciplinary content varies each year, depending on the discipline and specific assignment. This year the criterion was specified as “The paper/project shows evidence of metacognition.”

3. Genre awareness: The paper/project demonstrates knowledge of the conventions of the genre.

The average scores were:

Rhetorical Content: 4.19

Disciplinary Content: 4.39

Genre Awareness: 3.99

The following is a quote from last year’s analysis of this data:


Given that a score of 3 is the benchmark for competence in the major, the essays as a whole met or exceeded the departmental standards for competence. As for the specific papers, all the papers were deemed competent or above in the areas of Rhetorical Content and Disciplinary Content, while 11 out of 13 papers in the area of Genre Awareness were deemed competent or above. The committee is pleased with these results.

Last year’s ENG undergraduate assessment committee had this to say regarding last year’s survey of graduating seniors:

We developed a survey to give to graduating seniors each year and administered it to our winter 2014 graduates. We will give the survey to the remaining graduates at the end of the Spring term. Since we only have four replies so far, the specific results are inconclusive but are included in this report to establish precedent and to document for this year the nature of our survey.

We will offer little in the way of analysis of the survey this year because of the small sample size. We can say that in all questions at least three out of four students were satisfied with their education in our Department. However, in a few areas one out of the four indicated some degree of dissatisfaction (a score of 2 on a scale of 1 to 5). These areas need to be monitored in the future to see if they are something we need to address. Specifically, one student disagreed (score=2) with the statement “My career preparation was adequate” and one student did not feel challenged by the courses.

The results of the AY 2013-14 survey are reproduced below:

Qualtrics Survey

Questions and Results

Q1: In what term is your graduation?

# Answer Response % 1 Fall 2014 4 100% 2 Spring 2015 0 0% 3 Summer 2015 0 0% 4 Other 0 0% Total 4 100%

Q2: What was your area of concentration?

# Answer Response % 1 Literature 1 25% 2 Writing 1 25% 3 Teaching Certification 1 25% 4 Language 1 25% 5 Communication Media 0 0% Total 4 100%


Q3: Please rate the advising you were provided during your undergraduate career: (1=Not requested or received, 2=Not at all helpful, 3=Not very helpful, 4=Somewhat helpful, 5=Very helpful)

# Answer Min Value Max Value Average Value Standard Deviation Responses

1 Advice given for selection of courses 4.00 5.00 4.75 0.50 4

2 Advice received for educational goals 1.00 5.00 3.50 1.73 4

3 Advice received for career goals 1.00 5.00 3.50 1.73 4

Q4: How would you rate each of the following? (1=Poor, 2, 3=Average, 4, 5=Excellent)


1 Your course experiences as a major 4.00 5.00 4.75 0.50 4

2 Your experiences or opportunities as a major outside of courses

3.00 5.00 4.00 0.82 4

Q5: How would you rate the quality of the instruction you received as an English major? (1=Poor, 2, 3=Average, 4, 5=Excellent)

# Answer Min Value Max Value Average Value Standard Deviation Responses 1 Core courses 4.00 5.00 4.75 0.50 4

2 Concentration Courses 4.00 5.00 4.75 0.50 4

Question 6: To what extent do you agree or disagree with the following statements about courses: (1=Strongly disagree, 2=Disagree, 3=Neither agree nor disagree, 4=Agree, 5=Strongly agree)


1 In my major, the core courses I wanted were available

4.00 5.00 4.50 0.58 4

2 In my major, the concentration courses I wanted were available

4.00 5.00 4.50 0.58 4

3 In my major, my instructors were good teachers

4.00 5.00 4.75 0.50 4

4 In my major, instructors were knowledgeable 5.00 5.00 5.00 0.00 4

5 In my major, my instructors were helpful outside classes

4.00 5.00 4.75 0.50 4


Question 7: To what extent do you agree or disagree with the following statements about your education as a major: (1=Strongly disagree, 2=Disagree, 3=Neither agree nor disagree, 4=Agree, 5=Strongly agree)


1 Department academic standards met my expectations

4.00 5.00 4.50 0.58 4

2 Quality of the courses met my expectations 4.00 5.00 4.50 0.58 4

3 My career preparation was adequate 2.00 5.00 4.00 1.41 4

4

My preparation for further (graduate) studies was adequate (if applicable)

3.00 5.00 3.75 0.96 4

5 Quality of the faculty met my expectations 4.00 5.00 4.75 0.50 4

Question 8: In general, how would you rate your overall experience as a major in the Department of English and Linguistics? (1=Poor, 2, 3=Average, 4, 5=Excellent)


1 4.00 5.00 4.75 0.50 4

Question 9: Would you recommend IPFW's Department of English and Linguistics to a friend or relative considering an English major? (1=No, definitely not, 2, 3=Maybe, 4, 5=Yes, without reservations)


1 4.00 5.00 4.75 0.50 4 Question 10: Looking back on your experience at IPFW, would you say you were challenged by the courses you took for your major? (1=Never, 2, 3=Sometimes, 4, 5=Always)

# Answer Min Value Max Value Average Value Standard

Deviation Responses

1 2.00 5.00 4.25 1.50 4

Question 11: Using the scale below, please answer each of the following questions: (1=Definitely not, 2=Probably not, 3=Maybe, 4=Probably yes, 5=Definitely yes)



1

If you had known what you would experience as an English major, would you still have chosen our department?

4.00 5.00 4.75 0.50 4

2 Would you still have selected the same concentration?

3.00 5.00 4.25 0.96 4

Question 12: What is most likely to be your principal activity upon graduation? (Check all that apply.)

# Answer Response % 1 Full-time employment 1 25% 2 Part-time employment 2 50% 3 Second bachelor's degree 0 0% 4 Master's degree 2 50% 5 Specialist degree (J.D., Ed. S., etc.) 0 0% 6 Medical degree (M.D., D.D.S, etc.) 0 0% 7 Doctorate (Ph.D., Ed.D., etc.) 0 0% 8 Other 0 0%

Question 13: Do you feel that your IPFW degree helped prepare for your future or current studies or employment?

# Answer Response % 1 Yes 4 100% 2 No 0 0% Total 4 100%

Question 14: If it has taken you more than 4 years to graduate, please indicate all of the reasons why:

1 text reply:

“I only took as many credits as I thought I could handle, but still be a full time student, so most of my semesters were 12 credit hours. I also added a minor and a certificate on top of my degree, so that meant more credit hours to take, which took a little longer than four years to complete.”

Question 15: Please feel free to leave additional comments:

2 text replies:

“Being an English major has been the most fulfilling and rewarding experience I have ever had. I used to be a psychology major before changing my major and I am very glad I


did. I finally feel like I know what I can do with my life that won't be something forced on me to make money, but something I can be happy doing while still being able to support myself. I found a passion I didn't know I had until spending my time as an English major, with the professors I've met that inspired and challenged me.”

“Thank you! This has been a lovely journey!”

D. Assessment Findings for Curricular Changes Made Not applicable. SECTION 5: CONCLUSIONS, NEXT STEPS, AND COMMUNICATION All of the assessment evidence underscores that the ENG undergraduate program goals are clearly being met. The total and average scores for the classes assessed were well above the range defined as demonstrating success in achieving program goals, with the capstone demonstrating a 100% success rate in all three SLOs. The establishment of general ENG SLOs as the benchmark by which future assessment will take place will allow greater opportunity for consistent analysis of department data. This will allow meaningful longitudinal study over multiple years. The next steps will be to begin collecting assessment artifacts from the next set of classes in the assessment cycle. This report (and the COAS Assessment Committee’s response to it) will be distributed to the ENG department chair and the ENG Undergraduate Committee for further discussion at the committee and department level (as appropriate). Appendix 1: Individual ENG Undergraduate Program Assessment Artifact Data COMPLETE DATA FOR ENG L371 ASSESSMENT: 1. Does the student demonstrate the ability to write critically, precisely, and persuasively, especially about topics relevant to the major field and selected concentration?


# Question Unsatisfactory Needs Work Good Excellent Total

Responses Mean

1 Essay #1 0 0 1 4 5 3.80 2 Essay #2 0 1 4 0 5 2.80 3 Essay #3 0 0 3 2 5 3.40 4 Essay #4 0 0 2 3 5 3.60 5 Essay #5 0 0 3 2 5 3.40 6 Essay #6 0 0 4 1 5 3.20 7 Essay #7 0 3 1 1 5 2.60 8 Essay #8 1 3 1 0 5 2.00 9 Essay #9 0 0 4 0 4 3.00

10 Essay #10 0 0 3 2 5 3.40

11 Essay #11 0 4 1 0 5 2.20

12 Essay #12 0 0 3 2 5 3.40

13 Essay #13 0 1 2 2 5 3.20

2. Does the student demonstrate an ability to communicate knowledge of literary and linguistic conventions and traditions, especially those of America and England?


Responses Mean


10 Essay #10 0 1 2 2 5 3.20

11 Essay #11 0 3 2 0 5 2.40

12 Essay #12 0 1 3 1 5 3.00

13 Essay #13 0 0 2 3 5 3.60


Statistic

Essay #1

Essay #2

Essay #3

Essay #4

Essay #5

Essay #6

Essay #7

Essay #8

Essay #9

Essay #10

Essay #11

Essay #12

Essay #13

Min Value 2 3 3 3 2 3 2 2 3 2 2 2 3

Max Value 4 4 4 4 4 4 4 3 4 4 3 4 4

Mean 3.40

3.20

3.80

3.40

3.20

3.60

2.80

2.40

3.40

3.20

2.40

3.00

3.60

Variance

0.80

0.20

0.20

0.30

0.70

0.30

0.70

0.30

0.30

0.70

0.30

0.50

0.30

Standard Deviation

0.89

0.45

0.45

0.55

0.84

0.55

0.84

0.55

0.55

0.84

0.55

0.71

0.55

Total Responses

5 5 5 5 5 5 5 5 5 5 5 5 5

3. Does the student demonstrate the ability to use research tools and methods appropriate to their selected concentration?


Responses Mean


10 Essay #10 0 0 3 2 5 3.40

11 Essay #11 1 1 3 0 5 2.40

12 Essay #12 0 1 2 2 5 3.20

13 Essay #13 0 0 2 3 5 3.60


Statistic

Essay #1

Essay #2

Essay #3

Essay #4

Essay #5

Essay #6

Essay #7

Essay #8

Essay #9

Essay #10

Essay #11

Essay #12

Essay #13

Min Value 2 2 3 3 3 3 3 2 3 3 1 2 3

Max Value 4 3 4 4 4 4 4 3 4 4 3 4 4

Mean 3.60

2.80

3.80

3.60

3.40

3.40

3.20

2.20

3.20

3.40

2.40

3.20

3.60

Variance

0.80

0.20

0.20

0.30

0.30

0.30

0.20

0.20

0.20

0.30

0.80

0.70

0.30

Standard Deviation

0.89

0.45

0.45

0.55

0.55

0.55

0.45

0.45

0.45

0.55

0.89

0.84

0.55

Total Responses

5 5 5 5 5 5 5 5 5 5 5 5 5


COMPLETE DATA FOR ENG L202 ASSESSMENT: 1. Does the student demonstrate the ability to write critically, precisely, and persuasively, especially about topics relevant to the major field and selected concentration?


Responses Mean


10 Essay #10 0 2 3 0 5 2.60

11 Essay #11 0 2 3 0 5 2.60

12 Essay #12 0 2 3 0 5 2.60

13 Essay #13 1 0 3 1 5 2.80

14 Essay #14 1 1 3 0 5 2.40

15 Essay #15 0 0 3 2 5 3.40

16 Essay #16 0 2 2 1 5 2.80

17 Essay #17 0 1 0 4 5 3.60

18 Essay #18 3 0 1 1 5 2.00

19 Essay #19 0 2 2 1 5 2.80

20 Essay #20 0 4 1 0 5 2.20

21 Essay #21 0 2 2 1 5 2.80

2. Does the student demonstrate an ability to communicate knowledge of literary and linguistic


conventions and traditions, especially those of America and England?


Responses Mean


10 Essay #10 0 2 3 0 5 2.60

11 Essay #11 0 2 3 0 5 2.60

12 Essay #12 0 1 4 0 5 2.80

13 Essay #13 0 1 3 1 5 3.00

14 Essay #14 1 0 4 0 5 2.60

15 Essay #15 0 0 4 1 5 3.20

16 Essay #16 0 2 2 1 5 2.80

17 Essay #17 0 1 1 3 5 3.40

18 Essay #18 2 2 0 1 5 2.00

19 Essay #19 0 2 2 1 5 2.80

20 Essay #20 0 2 3 0 5 2.60

21 Essay #21 0 1 3 1 5 3.00


Statistic

Essay #1

Essay #2

Essay #3

Essay #4

Essay #5

Essay #6

Essay #7

Essay #8

Essay #9

Essay #10

Essay #11

Essay #12

Essay #13

Essay #14

Essay #15

Essay #16

Essay #17

Essay #18

Essay #19

Essay #20

Essay #21

Min Value

1 2 3 2 2 2 2 1 2 2 2 2 2 1 3 2 2 1 2 2 2

Max Value

4 4 4 4 4 3 3 4 3 3 3 3 4 3 4 4 4 4 4 3 4

Mean

2.80

3.20

3.20

3.00

2.80

2.60

2.60

3.00

2.40

2.60

2.60

2.80

3.00

2.60

3.20

2.80

3.40

2.00

2.80

2.60

3.00

Variance

1.70

0.70

0.20

0.50

0.70

0.30

0.30

1.50

0.30

0.30

0.30

0.20

0.50

0.80

0.20

0.70

0.80

1.50

0.70

0.30

0.50

Standard Deviation

1.30

0.84

0.45

0.71

0.84

0.55

0.55

1.22

0.55

0.55

0.55

0.45

0.71

0.89

0.45

0.84

0.89

1.22

0.84

0.55

0.71

Total Responses

5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5


3. Does the student demonstrate the ability to use research tools and methods appropriate to their selected concentration?


Responses Mean


10 Essay #10 0 1 3 1 5 3.00

11 Essay #11 1 0 3 1 5 2.80

12 Essay #12 1 1 3 0 5 2.40

13 Essay #13 0 3 2 0 5 2.40

14 Essay #14 1 0 4 0 5 2.60

15 Essay #15 0 1 4 0 5 2.80

16 Essay #16 1 2 1 1 5 2.40

17 Essay #17 1 0 0 4 5 3.40

18 Essay #18 3 1 0 1 5 1.80

19 Essay #19 1 2 2 0 5 2.20

20 Essay #20 1 3 1 0 5 2.00

21 Essay #21 1 2 1 1 5 2.40


Statistic

Essay #1

Essay #2

Essay #3

Essay #4

Essay #5

Essay #6

Essay #7

Essay #8

Essay #9

Essay #10

Essay #11

Essay #12

Essay #13

Essay #14

Essay #15

Essay #16

Essay #17

Essay #18

Essay #19

Essay #20

Essay #21

Min Value

2 1 2 2 2 2 1 1 1 2 1 1 2 1 2 1 1 1 1 1 1

Max Value

4 4 4 4 4 3 2 4 2 4 4 3 3 3 3 4 4 4 3 3 4

Mean

2.80

2.80

2.80

2.80

3.00

2.40

1.80

3.00

1.80

3.00

2.80

2.40

2.40

2.60

2.80

2.40

3.40

1.80

2.20

2.00

2.40

Variance

1.20

1.70

0.70

0.70

0.50

0.30

0.20

1.50

0.20

0.50

1.20

0.80

0.30

0.80

0.20

1.30

1.80

1.70

0.70

0.50

1.30

Standard Deviation

1.10

1.30

0.84

0.84

0.71

0.55

0.45

1.22

0.45

0.71

1.10

0.89

0.55

0.89

0.45

1.14

1.34

1.30

0.84

0.71

1.14

Total Responses

5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5


COMPLETE DATA FOR AY 2014-15 GRADUATING SENIOR SURVEY: 1. In what term is your graduation?

# Answer Response % 1 Fall 2014 4 18% 2 Spring 2015 17 77% 3 Summer 2015 0 0% 4 Other 1 5% Total 22 100%

Other 2017 Statistic Value Min Value 1 Max Value 4 Mean 1.91 Variance 0.37 Standard Deviation 0.61 Total Responses 22 2. What was your area of concentration?

# Answer Response % 1 Literature 12 55% 2 Writing 4 18%

3 Teaching Certification 3 14%

4 Language 3 14%

5 Communication Media 0 0%

Total 22 100% Statistic Value Min Value 1 Max Value 4 Mean 1.86 Variance 1.27 Standard Deviation 1.13 Total Responses 22


3. Please rate the advising you were provided during your undergraduate career:

# Answer Min Value Max Value Average Value

Standard Deviation Responses

1

Advice given for selection of courses

0.00 5.00 4.37 1.16 19

2

Advice received for educational goals

1.00 5.00 4.16 0.96 19

3

Advice received for career goals

0.00 5.00 3.58 1.54 19

4. How would you rate each of the following?

# Answer Min Value Max Value

Average Value


1 Your course experiences as a major

2.00 5.00 4.30 0.86 20

2

Your experiences or opportunities as a major outside of courses

3.00 5.00 3.94 0.94 18

5. How would you rate the quality of the instruction you received as an English major?

# Answer Min Value Max Value

Average Value


1 Core courses 2.00 5.00 4.35 0.88 20

2 Concentration Courses 2.00 5.00 4.50 0.89 20


6. To what extent do you agree or disagree with the following statements about courses:

# Answer Min Value

Max Value

Average Value


1

In my major, the core courses I wanted were available

3.00 5.00 4.48 0.60 21

2

In my major, the concentration courses I wanted were available

3.00 5.00 4.43 0.68 21

3

In my major, my instructors were good teachers

3.00 5.00 4.60 0.60 20

4

In my major, instructors were knowledgeable

3.00 5.00 4.67 0.58 21

5

In my major, my instructors were helpful outside classes

2.00 5.00 4.30 0.80 20


7. To what extent do you agree or disagree with the following statements about about your education as a major :



1

Department academic standards met my expectations

3.00 5.00 4.35 0.75 20

2

Quality of the courses met my expectations

2.00 5.00 4.25 0.91 20

3

My career preparation was adequate

2.00 5.00 3.85 1.23 20

4

My preparation for further (graduate) studies was adequate (if applicable)

2.00 5.00 3.70 1.08 20

5

Quality of the faculty met my expectations

2.00 5.00 4.40 0.88 20

8. In general, how would you rate your overall experience as a major in the Department of English and Linguistics at IPFW?



1 3.00 5.00 4.30 0.66 20


9. Would you recommend IPFW's Department of English and Linguistics to a friend or relative considering an English major?



1 2.00 5.00 4.30 0.98 20 10. Looking back on your experience at IPFW, would you say you were challenged by the courses you took for your major?



1 1.00 5.00 4.20 1.15 20 11. Using the scale below, please answer each of the following questions:

# Answer Min Value

Max Value

Average Value


1

If you had known what you would experience as an English major, would you still have chosen our department?

2.00 5.00 4.55 0.89 20

2

Would you still have selected the same concentration?

1.00 5.00 4.10 1.07 20


12. What is most likely to be your principal activity upon graduation? (Check all that apply.)

# Answer Response %

1 Full-‐time employment 11 55%

2 Part-‐time employment 3 15%

3 Second bachelor's degree

0 0%

4 Master's degree 12 60%

5 Specialist degree (J.D., Ed. S., etc.)

0 0%

6 Medical degree (M.D., D.D.S, etc.)

0 0%

7 Doctorate (Ph.D., Ed.D., etc.)

1 5%

8 Other 0 0% Other Statistic Value Min Value 1 Max Value 7 Total Responses 20 13. Do you feel that your IPFW degree helped prepare for your future or current studies or employment?

# Answer Response % 1 Yes 18 90% 2 No 2 10% Total 20 100%

Statistic Value Min Value 1 Max Value 2 Mean 1.10 Variance 0.09 Standard Deviation 0.31 Total Responses 20


14. If it has taken you more than 4 years to graduate, please indicate all of the reasons why: Text Response I only took as many credits as I thought I could handle, but sill be a full time student, so most of my semesters were 12 credit hours. I also added a minor and a certificate on top of my degree, so that meant more credit hours to take, which took a little longer than four years to complete. While I did transfer schools, I don't believe that was a reason it took me five years instead of four. There were more credits required than I could cram into four years with my job(s) and while I was trying to get experience that would look good to employers -‐ such as internships. I wound up needing so many extra credits that I just decided to add a second major, Sociology, which I discovered I had a greater passion for than English Literature studies. In the end, I'm deeply grateful for that experience even if I was angry about my predicament when I discovered it. worked full time and raised kids Decided to add on a minor and certificate on top of balancing my work schedule throughout college. I was ill-‐prepared upon entering college. I had a poor work ethic and did not make school a priority. Full-‐time employment In my very first week as a freshman there was a death in my family that I took extremely hard. I started skipping classes and not doing my homework. I nearly failed out my freshman year. I had to write an essay explaining why I should be readmitted. I decided to take a year off after that and regain myself and put myself back on the right track. Once I returned I was in an accident and broke my back leaving me in a immobilized state. Going to classes was no longer an option until I had finished my required therapy. So I have to take another year off. Even when I returned I returned in a wheelchair. After that it was a struggle to get all the classes I needed into a schedule that worked for me so I have had to finish my degree as a part time student. Moving, personal issues I only took 12 credit hours a semester instead of fifteen Failed some classes freshman year Switched majors I took my time due to also having a job. Life Statistic Value Total Responses 13


15. Please feel free to leave additional comments: Text Response Being an English major has been the most fullfilling and rewarding exerience I have ever had. I used to be a psychology major before changing my major and I am very glad I did. I finally feel like I know what I can do with my life that won't be something forced on me to make money, but something I can be happy doing while still being able to support myself. I found a passion I didn't know I had unitl spending my time as an English major, with the professors I've met that inspired and challenged me. Thank you! This has been a lovely journey! My experience as an English major at IPFW has been excellent overall, but I want to especially thank Dr. Lewis Roberts for his outstanding children's literature classes, and Dr. Sara Webb-‐Sunderhaus for her outstanding advising and support. At my first university, we were required to take four classes per semester (they didn't go on credit hours as they were a private university and didn't have to) -‐ transferring here was a huge shock when I discovered that I would have had to take five classes each semester (and would have had to have had that all along) if I wanted to graduate on time. It's unreasonable, I think, in today's environment where almost no student can go to school without having at least one job, plus all the other expectations (internships, volunteer work) for them to get a career. As wonderful as it would be for education to be the goal of higher education, most people are here to get a better job than they would have had without it. They don't want to be trapped in debt for the rest of their lives with a useless degree (and to be clear: my degrees are not useless, I truly believe that, but the high costs for the low potential for a job due to not being able to get out-‐of-‐class experiences because they were too busy going to school and working jobs to get applicable experiences makes it hard to see that sometimes). There has to be more of a focus on helping these students go places with their degrees. Especially in the English department. People for some reason think that English is useless and pointless and that only slackers get English degrees. Ok, it's not just some reason. Its because 1. they don't know what possible careers an English major could get (and many English majors can't answer this question) and 2. English graduates wind up lost . My Sociology program had a capstone class that not only showed us some options, it taught us how to /get/ those options and make them our future. It's great that we learn how to research literature, talk about literature, explore literature. Now what are we going to do with it, and how are we going to get those jobs? My professors and classes have taught me how to use my early struggles to become an even better student and person than I would have been without those early failures. Statistic Value Total Responses 5

ASSESSMENT REPORT

DEPARTMENT OF POLITICAL SCIENCE, 2014-2015

General Information: political science majors graduated from December 2014 through August 2015. This number is in keeping with past

numbers of degrees awarded (between 11 and 16 from 2001 to 2014). These figures indicate that majors are continuing to graduate at an

acceptable rate. The department currently had approximately 50 majors seeking B.A. degrees. (This figure does not include i4 students

seeking a BA in Economics or those pursuing Political Science as a second major. A number of students seeking the BA in Economics are

pursuing a second major in Political Science which is facilitated by the fact that the Political Science Department administers both

programs).

Key components for continuing assessment for all majors for 2013-2014: writing course required of all majors, statistics course required of

all majors, and senior seminar (capstone course) required of all majors. Also included in this assessment are a curriculum map of courses

taught with reference to areas covered, and some detailed materials addressing enrollment issues in the department.

A. Learning Goals 1. To know: Identify (basic) and explain (advanced) key terms and concepts in the major fields of the discipline. (BF Goal: Depth of

knowledge in your field.) a. Political Thought and Philosophy

b. American government and politics

c. Comparative government

d. International Relations

e. Quantitative and Qualitative Methods

[Major learning goals and other objectives covered by courses noted in the Curriculum Map]

How Assessed:

The department already requires majors to pass, with a C- or better, a statistics course and the departmental second-semester writing course

(or its equivalent) in order to meet the requirements for a major in Political Science. Passage of the required courses demonstrates success in

this area

1) Twelve students (of 15 initially enrolled) completed the course in Spring Semester 2015 which is similar to results in past semesters.

2) Eleven of twelve students successfully passed the statistics course in the Fall Semester with a grade of C- or better.

The department recently incorporated a change in requirements for majors that will require them to take at least one course in the four areas

of American government and politics, comparative government, political thought and philosophy, and international relations. This change

ensures that students will meet this objective of knowledge in all major areas of political science. There has been no indication that this

requirement for breadth of knowledge in the discipline has created any difficulties for majors since sufficient courses in each of the four

areas are routinely offered.

Assessment in the ability to demonstrate the ability to fulfill this broad knowledge requirement of the department learning goals by the need

to earn at least a C- in any course that counts towards the major. The requirement has meant that the department has to offer courses in each

of the four areas and to the extent possible more than one course in each area to provide students with opportunities to meet this

requirement. Difficulties have been eased to some extent since some courses are offered in the TV Studio classroom permitting students to

register for the on-line versions of the courses.

2. For 2013-14, the focus was on item C below.

a. Write/communicate clearly and effectively. (BF Goal: Effective communication skills in multiple media.)

Detailed evaluations for one section of POLS Y207 and one section of Y490 are attached b. Use quantitative and qualitative research tools

appropriately. (BF Goal: Resourceful critical thinking and problem solving.)

c. Research and analyze political issues and engage in problem solving. (BF Goal: Resourceful critical thinking and problem solving.)

How Assessed: Writing skills are assessed by analysis of the work done for the departmental writing course, by analysis of the papers

written for the senior seminar, and by individual reports on majors in other upper-division courses with research papers.

1) There were 8 students in the senior seminar in the spring. Seven of the eight completed the fall course although two did relatively poorly

in terms of framing their research questions well. Eleven students in the writing course demonstrated the ability to prepare a basic research

paper.

2) The department has created a rubric by which it can assess the skills of majors preparing papers in other upper-level courses focusing on

content, organization, grammar and structure; use of sources and citation; and synthesis of information. The rubrics developed in some cases

are similar to those used in the evaluation of courses for the new General Education program.

3) The department statistics course will continue to be used to guarantee the use of quantitative and qualitative research skills (which also

can be seen in the senior seminar papers).

4) The papers from the senior seminar require students to demonstrate the ability to research and analyze political issues and engage in

problem solving. The results from the one sections of Political Science Y490 is attached.

5) Ethical and professional standards are determined by issues of academic dishonesty or attempted plagiarism in a given year. In 2013-2014

there were no reported cases of such violations by majors taking courses in the department.

d. Behave ethically and professionally in keeping with disciplinary standards for personal integrity, academic honesty, respect for diversity,

and civil dissent and discourse. (BF Goals: Citizenship and leadership in diverse communities and Personal integrity and ethical

action.)

3.* To be prepared for: (BF Goal: Lifelong application of knowledge using appropriate technologies.)

a. Employment in government/public service/political system or related areas.

b. Graduate study/law school.

c. Becoming active and involved citizens and leaders in the local community, the nation, and beyond. (BF Goal: Citizenship and

leadership in diverse communities.) d. As part of the program review presently under way, the department will survey previous graduates determining how effectively the

department has prepared them for life-long learning. Majors have clearly done well in areas a and b as indicated by assessment reports over

the years, including the current one.

B. Enrollment Issues The department has been faced with declining enrollments in the last few years. An analysis of enrollment patterns for all upper division

courses indicates that enrollment has been relatively constant in those courses. That situation could change as the effects replacing the old

general education Area VI requirement with the capstone requirement are felt. In the introductory level 100 courses, there has been a clear

drop off in enrollments over time for Political Science 103 noted last year that has continued for the current assessment period. The

department has continued its efforts to match the current resources to the current demand levels. The department has further reduced the

number of overload internet courses previously offered and has offered more online course/studio courses in the on-load category. The

department will be offering POLS Y103 via the internet to Wabash High School in the upcoming year to provide a dual credit opportunity

(and hopefully to recruit students). The department is also experimenting with dual credit offerings by a tenured faculty member at

Homestead High School. The department has introduced a number of 200 level courses that might have greater appeal among non-majors

who are seeking more introductory level courses on various topics. The department will also be experimenting with some different patterns

of days and time for offering courses. This combination of efforts has helped to keep enrollments at a reasonable level. (The report from

last year is provided for context.)

C. Student Successes and Accomplishments The graduation rate for the department is on part for graduating majors within six year. Of 15 identified majors in Blueprint, 13 of them

graduated in six years or less while one with a double major in Computer Science and Economics took eight years and another took 7 years

to complete the degree which would appear to reflect a pattern of taking only a few courses in the initial years. Previous successes in terms

of the graduation of majors have led to a decline in the total number of majors.

Appendix 1 Curriculum Map Template

DEPARTMENT: Political Science PROGRAM: Bachelor of Arts Date: Fall 2014 and Spring 2015 Dept. Chair: James M. Lutz Courses Objectives

Political Thought and Phil

Amer Govt & Politics

Com- parative Gov’t

Intern’l Relations

Quant/ Qual Methods

Write/ Communi- cate

Use of Re search Tools

Analysis & Prob- solving

Professional standards

Prep for Employment

Prep for Grad/ Law school

Prep for Citizen- ship

Y101 I I I I I I I I I

Y103 I E I R I R R E

Y105 E R E R R R R R E

Y109 I I E I I I

Y150 I E R R R R R

Y207 I I I I E E E E I I I

Y211 R I E R R R E R

Y301

R R R E E

R R E I

Y304 E R E E E I

Y307 E R R R R R

Y317 E E R R E R I R R

Y200/401 Film

E E R R R R

Y200/401 Terrorism

R E I I E R–200 E-401

R R R

Y200/Y401 Utopias

E I I E R-200 E-401

E R R R E

Y397Intervention I E I I E

E E I

Y200/Y401 Family Law

I E I E R-200 E-401

E I R E R

Y200/401 Comp Pol Bahavior

I E R E R-200 E-401

R R E I

Y208/Y401 Scandals

E R R R R-200 E-401

R R R R R

Y252/Y401 Sports & Pub Policy

E I E R-200 E-401

R R R R R

Y285/401 Politics and Science

I E I I R R-200 E-401

R I I E R

Y275/401 Film

E E R R

R R

Y203/401 E E E R-203 E-401

R R R R E

Y340 I E R E ER E E R R I

Y339 I E R E E E E R R I

Y360 E R R E E E R E I

Y376 I I I E R R R E R I R

Y375 I E R E E E R R R R R

Y378 E R R R E R R R E

Y381 E E E R R R R R E

Y382 E E E R R R R R E

Y395 E R E E I I I

Y490 (Toole) E R E E E E E R E R

Y490 (Wolf) R R E E E E R R E R

I- Introduced in course E- Emphasized in course R- Reinforced in course

Appendix 2: Overview of POLS Y395

Political Science Y395: Quantitative Analysis (Methods and Statistics Course)

Twelve Students Registered for Course from first test to end of course

6 Students demonstrated a high level of proficiency in course material/knowledge of scope and methods in political science

1 Student demonstrated a good level of proficiency in course material/ knowledge of scope and methods in political science

4 Students demonstrated an acceptable level of proficiency in course material/ knowledge of scope and methods in political science

1 Student failed to demonstrate an acceptable level of proficiency in the course material/ knowledge of scope and methods in political

science

The completion rate for the required course for majors was very positive, especially when 50% of the class demonstrated a high level

of proficiency at the end of the course. Only one student failed to achieve an acceptable level which is better than the completion rate

in past semesters.

Appendix 3: Completion of Degrees

Time for Completion of Degree: Political Science and Economics

Time from first course for IPFW (or date that transfers were accepted) to awarding of degree

Graduate 1: 6 years, Economics/Computer Science double major

Graduate 2: 6 years, transfer credits

Graduate 3: 4 years,

Graduate 4: 4 years,



Graduate 7: 8 years

Graduate 8: 4 years

Graduate 9: 4 years

Graduate 10: 3.5 years, transfer credits

Graduate 11: 7 years





13 of 15 graduates in Political Science and Economics finished in six years or less after entering IPFW

1

Assessment Report

Spring 2015 Political Science Y207

Prof. James Toole

According to the Department of Political Science Assessment Plan, one of the assessment

measures used for the B.A. in Political Science is an interim measure evaluating student projects

for Y207 in terms of progress toward achieving the goals of the program.

One of the learning goals of the Political Science B.A. program is goal 2(c), demonstration of the

ability to research and analyze political issues and to engage in problem solving. Spring 2015

research papers were evaluated to determine how well they met this particular program goal.

Of the fifteen students who began Y207 in Spring 2015, twelve completed the fifteen-page final

project. In this assessment, the finished projects were evaluated based on how well each student

posed and answered his or her individual analytical research question. In Y207, each paper is

expected to pose a research question that remains unresolved and thus has more than one

plausible answer. The paper then is expected to solve the political problem addressed in the

question by using logic and evidence to determine which alternative answer seems most

persuasive. Because the accumulation and use of evidence is more a focus of departmental

learning goal 2(b) and is accordingly assessed in years other than this one, the present assessment

is focused instead on the clarity and logic of the question and its ultimate answer.

Projects were evaluated on the following scale:

5: The research question is clear, significant, and unresolved; it is thoughtfully described

and placed intelligently into theoretical, practical, and (where appropriate)

historical context. The answer to the question is highly persuasive, based on

unusually sound reasoning.

4: The research question is clear, significant, and unresolved; it is clearly described and

placed well into some context. The answer to the question is convincing, based on

good reasoning.

3: The research question is useful but is not as clear, significant, or unresolved as it could

be; it is described adequately and put into some context. The answer to the

question is only somewhat convincing; some of its reasoning is weak.

2: The research question is unclear or poorly conceived; it is not well described and the

description lacks context. The answer to the question is unconvincing, based on

weak or inappropriate reasoning.

1: The research question is unclear or not posed at all; if posed, it is poorly described and

not put into any useful context. The question, such as it is, is barely or not at all

answered.

2

Results of the assessment are as follows:

Score Number of students receiving the score

5 3

4 7

3 1

2 1

1 0

This year, well more than half of the student projects posed and answered an analytical question

in a very good or excellent way (earning a 4 or a 5). One earned a middle score and another

scored below the midpoint. None received the lowest score. This pattern suggests that the

learning outcome was quite well met.

This semester’s assessment scores are much better than those achieved the last time that this

learning objective was assessed, in Spring 2012. This might have been because the 2012

assessment alerted me to possible problems, leading me to focus more on this learning objective

as I taught; it might also have been because the students came into this class with better skills. As

data are collected in future assessments of learning objective 2(c), answers to this kind of

question may become clearer. For the time being, this assessment suggests that current teaching

practices directed toward instruction in learning outcome 2(c) are working well. As always, I will

continue to monitor student progress in meeting the learning outcome during future iterations of

the course.

1

Assessment Report

Spring 2015 Political Science Y490

Prof. James Toole

According to the Department of Political Science Assessment Plan, one of the assessment

measures used for the B.A. in Political Science is an internal measure at exit evaluating student

projects for Y490 in terms of progress toward achieving the goals of the program.

One of the learning goals of the Political Science B.A. program is goal 2(c), demonstration of the

ability to research and analyze political issues and to engage in problem solving. Spring 2015

Y490 research papers were evaluated to determine how well they met this particular program

goal.

Of the eight students who began Y490 in Spring 2015, seven completed the final project. In this

assessment, the finished projects were evaluated based on how well they posed a clear,

significant, and unresolved research question and on how well they answered that question using

appropriate logic and evidence. This method of assessing program learning goal 2(c) was

selected because asking and answering an appropriate question is essential to both good social

science analysis and problem-solving in general.

Projects were evaluated on the following scale:

5: The research question is clear, significant, and unresolved; it is thoughtfully described

and placed intelligently into theoretical, practical, and (where appropriate)

historical context. The answer to the question is highly persuasive, based on

unusually sound logic and on extensive, detailed, and appropriate evidence.

4: The research question is clear, significant, and unresolved; it is clearly described and

placed well into some context. The answer to the question is convincing, based on

good logic and on a good amount of detailed, appropriate evidence.

3: The research question is useful but is not as clear, significant, or unresolved as it could

be; it is described adequately and put into some context. The answer to the

question is only somewhat convincing; some of its logic is flawed and some

important supporting evidence is lacking.

2: The research question is unclear or poorly conceived; it is not well described and the

description lacks context. The answer to the question is unconvincing, based on

weak logic and on inadequate or inappropriate evidence.

1: The research question is unclear or not posed at all; if posed, it is poorly described and

not put into any useful context. The question, such as it is, is barely or not

answered, and the answer has little or no logic or evidence to support it.

Results of the assessment are as follows:

2

Score Number of students receiving the score

5 1

4 2

3 1

2 1

1 2

This year more than half of the student projects scored at the midpoint or above, and nearly half

engaged in analysis and problem solving in a very good or excellent way (earning a 4 or a 5).

One scored just below the midpoint, but two of the seven did a very poor job of achieving the

learning outcome (earning only a 1).

The poor scores earned by three of the seven projects are likely due in part to the complexity and

length of the project, which at 30 pages is a longer paper than most students ever will have

written at IPFW. It is not unusual for students in all sections of Y490 to wither at the end of the

long research and writing process, and the two whose projects received a 1 did just that: they

gave up during the second part of the semester, in the end choosing not to do the extensive

revisions that I recommended to them well in advance of the paper due date. The student who

scored a 2 acted very differently; while the final version of her paper was not very good, she

worked extremely hard—continuing to revise until the very end—to make her paper as good as

she could make it.

We as a department have long regularly discussed the teaching of Y490, which rotates among us

on a three-year cycle, and we are aware of the challenges that our students face in completing the

project. We tend to agree, however, that such challenges are essential to the post-graduate

success of our students, many of whom will need to perform at high levels in their jobs or

graduate programs. Given the nature of the project, it seems hard to avoid some low scores in

every Y490 section. Still, the number of students who score at the midpoint or above continues

to remain fairly robust, and graduates regularly thank us for pushing them to achieve more in

Y490 than they thought possible.

As always, we will continue to monitor assessments of Y490, learning what we can from them to

further improve student success.

Appendix 6: POLS Y103 Enrollment Data and Analysis

Year AP/CLEP Ivy Tech Community College Trine Total

1972 0 0 0 1 0 1

1976 0 0 0 1 0 1

1979 0 0 0 1 0 1

1982 0 0 0 1 0 1

1984 0 0 0 1 0 1

1985 0 0 1 2 0 3

1986 0 0 1 1 0 2

1987 0 0 0 3 0 3

1988 0 0 0 2 0 2

1989 0 0 0 2 0 2

1990 0 0 0 5 0 5

1991 0 0 0 3 0 3

1992 0 0 3 0 0 3

1993 0 0 0 1 2 3

1994 0 0 0 2 2 4

1995 0 3 7 21 1 32

1996 0 5 16 37 1 59

1997 0 6 20 90 6 122

1998 1 21 29 112 5 168

1999 1 15 22 100 6 144

2000 1 26 24 112 12 175

2001 0 36 24 109 11 180

2002 2 31 30 91 11 165

2003 2 69 23 146 19 259

2004 3 73 21 106 18 221

2005 4 69 21 112 20 226

2006 9 76 42 94 5 226

2007 4 67 17 96 16 200

2008 10 76 29 82 17 214

2009 11 112 37 80 14 254

2010 7 101 17 59 13 197

2011 8 113 25 70 12 228

2012 15 111 17 75 12 230

2013 24 91 19 51 12 197

Only partial data available for fall of 2014.

Trends

1970s 0 0 0 3 0 3

1980s 0 0 2 12 0 14

1990-1994 0 0 3 11 4 18

1995-1999 2 50 94 360 19 525

2000-2004 8 235 122 564 71 492

2005-2009 38 400 146 464 72 1120

2010-2013 54 416 78 255 49 852

Analysis of the Decline in Political Science Enrollments (POLS Y103)

There has been a major decline in the number of hours in political science courses taken by

students at IPFW. The most significant drop in course enrollments has been in POLS Y103

(Introduction to American Government). An analysis of 3,617 transfers with prior credit for

POLS Y103 indicates some of the sources of this marked decline.

In the 1970s and 1980s, only a handful of students enrolled at IPFW with prior credit for POLS

Y103. On average, less than two students a year registered with prior credit. All of the students

had credit transfer from some other university or community college. In these decades the

department offered multiple sections of POLS Y103 including off campus locations offered in

conjunctions with Continuing Studies.

In the first half of the 1990s, there was a slight increase, but by the last half of the 1990s there

was a major increase, resulting in an average of 100 transfers per year. Most credits were

transferred from other universities. Community colleges also provided transfer credit, including

for the first time Ivy Tech campuses. AP credit or CLEP credit was not a major source of credit

transfers for the course. It is not clear from the available data where the credit actually

originated. Credit from dual credit courses from high schools in Indiana (or other states) could

have transferred into other universities, community colleges or IVY Tech; once on the transcripts

at these places they would then have been transferred to IPFW when students entered. Trine

University has been actively developing links to some schools providing for dual credit. The

numbers for Trine University listed in the accompanying table would combine both individuals

who took Introduction to American Government on a Trine campus, those who came to Trine

with AP credit, or those who received dual credit in high school and with Trine University.

It was not possible to determine how many students earned credit in the 1990s through

Collegiate Connection since such courses appear as IPFW courses (and would not have

decreased overall enrollments in POLS Y103).

From 2000 onward, obviously larger numbers of students entered IPFW with prior credit for

POLS Y103. While still small in total numbers, AP credit became more important. IVY Tech

provided increasingly larger numbers of such transfer credits, eventually becoming the largest

single source of such credits. It is likely that many students at various IVY Tech campuses

initially enrolled with dual credit from higher school programs. Others took the basic course on

campus to take advantage of lower tuition. The numbers for Trine University also went up in this

period, suggesting that dual credit programs might have had an increasing impact. By 2010,

IPFW dual credit programs were in place for POLS Y103, but the numbers would have been

small (and, like Collegiate Connection enrollments, they would not show up in transfer figures).

Overall, from 2005 on, slightly more than 200 students on average annually entered IPFW with

credit for POLS Y103. These numbers mean that there would be a reduction in six to seven

fewer sections of POLS Y103 offered (on-campus or at other locations).

The reduction in demand reflects in part more transfers from other universities to IPFW. State

efforts to make IVY Tech a community college have been very successful in reducing the need

for the course at IPFW. There are very few indications that dual credit offerings have directly

limited demand, but it is very likely that some dual credit effects are present through the

acceptance of the credits by other universities.

The analysis of the transfers suggests the source of enrollment declines. Another issue which

cannot be addressed from this data is the quality of the teaching for these courses and whether

reasonable learning objectives have been met through the process. As is well known, students

are likely to benefit more when taught by university qualified instructors.

In addition to the very obvious effects of transfer students have credit for POLS Y103, additional

factors might have contributed to decreased demand. Changes in requirements for teaching

certificates by the State of Indiana have resulted in decreased need for education degrees. POLS

Y103 was often a requirement (either specifically or frequently taken from a limited range of

courses) for education degrees especially in the area of secondary education social studies.

Finally, there have been a smaller number of political science majors in recent years after an

upsurge, but the number of majors would seem to have had a limited impact on demand for the

POLS Y103 course.

2014-2015 Geosciences Assessment Report

Geosciences Department Assessment Report 2014-2015 1. Student Learning Outcomes We have begun the process of re-evaluating student learning outcomes with the aim of creating a proper assessment model that captures students throughout their coursework.

1.A. Summary of SLOs for all programs The following preliminary list summarizes and categorizes departmental program Student Learning Outcomes for coding into the course map. For the purposes of meaningful assessment, some of these learning outcomes are in need of more specificity. For example, SLO 1, a central core of knowledge, comprises the bulk of course content, and each course covers a small part of the SLO, so assessment will require breaking “core knowledge” into manageable bites. Similarly, in 2B “synthesize” is a common buzzword that has lost all real meaning, but can be translated into something that is concretely assessable. Refinements will be discussed over the next year.

As currently constituted, most sub-outcomes are “progressive –inclusive”: achievement of one implies achievement of all previous suboutcomes on the list (e.g. 4D implies and builds on 4A-4C, etc.). Some sub-outcomes cannot be put into a progression and are listed as “discrete non-progressive” (e.g. 10C does not necessarily imply or build on 10A)

SLO 1. have acquired a central core of geological knowledge (progressive-inclusive sub list) 1A. command a basic core of knowledge appropriate to the discipline and course-level

1B. have acquired a central core of geological knowledge SLO 2. Integrating subdisciplines of Geosciences (progressive-inclusive sub list) 2A. understand aspects of the interconnectedness among geological subdisciplines

2B. Be able to synthesize and integrate interconnectedness among geological sub-disciplines SLO 3. Have demonstrated proficiency in ancillary sciences applied to geology SLO 4. Be able to apply appropriate mathematical solutions to quantifiable problems (progressive-inclusive sub list)

4A. limited arithmetic solutions 4B. limited algebraic and other mathematical solutions

4C. rigorous algebraic solutions 4D. rigorous statistical, algebraic, and calculus-based solutions SLO 5. Be able to draw inferences about geological phenomena not encountered in course work (progressive-inclusive sub list) 5A. limited inferences 5B. general inferences SLO 6. Have demonstrated ability to solve field problems (progressive-inclusive sub list)

6A. simple field problems 6B. fundamental field problems 6C. quantifiable complex field problems SLO 7. Reading and writing in Geology (progressive-inclusive sub list) 7A. read technical information appropriate to the discipline and course-level

7B. read and comprehend short technical reports 7C. write short technical reports 7D. read and comprehend technical papers of moderate length and complexity


7E. review and evaluate geologic research (advanced specialties-discrete non-progressive sub-sublist) 7E1. (Students completing the environmental geology option) should show particular strength in

courses appropriate for future work and study in environmental geology, hydrogeology, waste control, and other related disciplines.

7E2. (Students completing the geology option) should show particular strength in courses appropriate for future work and study in resource exploration, field geology, structural geology, paleontology, and other related disciplines.

7F. write a thesis or other summary of original ideas SLO 8. Be able to develop and apply multiple working hypotheses to solve problems (discrete non-progressive

sublist) 8A. to geologic problems. 8B. to non-geologic problems.

SLO 9. Be prepared for further education or employment 9A. in technical or nontechnical fields, not necessarily related to geology, or

9B. Be prepared for advanced study in graduate school, or in technical and nontechnical fields, as a professional geologist

SLO 10. View Geosciences in a broader context (discrete non-progressive sublist) 10A. understand the value of a lifetime spent observing the natural world 10B. understand the importance of the Earth Sciences in rendering social, political and personal decisions

in a modern society 10C. have a broad perspective that places their science training within the context of the arts, humanities,

and social sciences

1.B. SLOs for General Education Courses SLO 1A. command a basic core of knowledge appropriate to the discipline and course-level SLO 5A. show the ability to correctly evaluate situations not explicitly covered in course work SLO 4A. be able to apply limited, arithmetic solutions to quantifiable problems SLO 7A. be able to read technical information appropriate to the discipline and course-level SLO 10A. understand the value of a lifetime spent observing the natural world SLO 10B. understand the importance of the Earth Sciences in rendering social, political and personal decisions in a modern society

1.C. SLOs for Minor In Geology SLO 1B. have acquired a central core of geological knowledge SLO 2A. understand aspects of the interconnectedness among geological subdisciplines SLO 4B. be able to apply limited algebraic and other mathematical solutions to quantifiable problems SLO 5A. have the ability to draw limited inferences about geological phenomena not encountered in course work SLO 6A. be able to evaluate simple field problems SLO 7B. be able to read and write short technical reports

1.D. SLOs for Bachelor of Arts With Major in Geology SLO 1B. have acquired a central core of geological knowledge SLO 2A. understand aspects of the interconnectedness among geological subdisciplines SLO 4C. be able to apply rigorous algebraic solutions to quantifiable problems SLO 5B. have the ability to draw inferences about geological phenomena not encountered in course work SLO 6B. be able to solve fundamental field problems SLO 7D. be able to read and write technical papers of moderate length and complexity SLO 8B. bring rigorous thinking evaluation processes to nongeological situations SLO 10C. have a broad perspective that places their science training within the context of the arts, humanities, and social sciences


SLO 9A. be prepared for employment in technical or nontechnical fields, not necessarily related to geology

1.D. Bachelor of Science in Geology SLO 1B. have acquired a central core of geological knowledge SLO 7E. demonstrate an understanding of selected advanced specialties SLO 7E1. Students completing the environmental geology option should show particular strength in courses

appropriate for future work and study in environmental geology, hydrogeology, waste control, and other related disciplines. SLO 7E2. Students completing the geology option should show particular strength in courses appropriate for future work and study in resource exploration, field geology, structural geology, paleontology, and other related disciplines.

SLO 2B. Ability to synthesize and integrate interconnectedness among geological and related disciplines SLO 3. have a background that includes a working knowledge of ancillary sciences, including physics, chemistry,

and biology SLO 4D. be able to apply rigorous statistical, algebraic, and calculus-based solutions to quantifiable problems SLO 5B. demonstrate an ability to draw inferences about geological phenomena not encountered in course work SLO 6C. be able to solve complex field problems SLO 7D, 7F. be able to read and write technical papers of moderate length, and be prepared to write a thesis or

other summary of original ideas SLO 8B. be able to bring rigorous thinking and evaluation processes to nongeological situations SLO 9B. be prepared for advanced study in graduate school or for employment in technical and non-technical

fields, possibly as a professional geologist

2. Curricular Maps 2.A. Map of SLO’s to Baccalaureate Framework There are some points on the framework that we cover in our courses and other requirements that perhaps are not featured adequately in our stated SLOs. For example, there is a great emphasis on verbal presentation and defence, but this is not written as an SLO. This issue will be under discussion.

Baccalaureate Framework Program’s Student Learning Objective

Acquisition of Knowledge: Students will demonstrate breadth of knowledge across disciplines and depth of knowledge in their chosen discipline. In order to do so, students must demonstrate the requisite information seeking skills and technological competencies.

SLO 1, 2, 3

Application of Knowledge: Students will demonstrate the ability to integrate and apply that knowledge, and, in so doing, demonstrate the skills necessary for life-long learning.

SLO 2, 3, 4, 5, 6, 7, 8

Personal and Professional Values: Students will demonstrate the highest levels of personal integrity and professional ethics.

SLO 9, 10

A Sense of Community: Students will demonstrate the knowledge and skills necessary to be productive and responsible citizens and leaders in local, regional, national, and international communities. In so doing, students will demonstrate a commitment to free and open inquiry and mutual respect across multiple cultures and perspective.

SLO 8, 9, 10

Critical Thinking and Problem Solving: Students will demonstrate facility and adaptability in their approach to problem solving. In so doing, students will demonstrate critical thinking abilities and familiarity with quantitative and qualitative reasoning.

SLO 4, 5, 6, 7, 8

Communication: Students will demonstrate the written, oral, and multimedia skills necessary to communicate effectively in diverse settings.

SLO 7


2.B. Preliminary Map of SLO’s to core courses Given some problems with the specificity of the SLOs, this map is under development. A main goal of the current year is to implement assessment methods that dovetail well with current teaching practices and that do not distract from teaching and mentoring.

Course Core SLOs Instruments ((AST A100 - The Solar System Cr. 3 OR GEOG G107 - Physical Systems of the Environment Cr. 3 OR GEOL G100 - General Geology Cr. 3-5 OR GEOL G210 - Oceanography) AND GEOL L100 - General Geology Laboratory) OR GEOL G103 - Earth Science: Materials and Processes (includes lab)

M BA BSG

1A, 2A, 4A, 5A, 7A, 10A, 10B

General education assessments online Additional metrics under review

MA 15300 - Algebra and Trigonometry I BA 4C Assessed in core courses

MA 15400 - Algebra and Trigonometry II BAo 4C Assessed in core courses

MA 22700, 22800 - Calculus for Technology I,II BSGo 4D Assessed in core courses

MA 16500, 16600 - Analytic Geometry and Calculus I, II

BSGo 4D Assessed in core courses

PPOL E272 - Introduction to Environmental Sciences BSGeo STAT 30100 - Elementary Statistical Methods I BAo

BSGg BSGeo

4D Assessed in other courses

CHM 11100 - General Chemistry BAo 3 Assessed in other courses

CHM 11500 - General Chemistry BAo BSG

3 Assessed in other courses

CHM 11600 - General Chemistry BSG 3 Assessed in other courses

PHYS 21800, 21900 - General Physics I, II BSGo 3 Assessed in other courses

PHYS 15200 – Mechanics, 251- Heat, Electricity, and Optics

BSGo 3 Assessed in other courses

GEOL G104 - Earth Science: Evolution of the Earth

M BA BSG

1, 2, 3, 4B, 5A, 7B, 9A,B 10A,B

Instruments under review

GEOL G211 - Introduction to Paleobiology

M BSG

1,2,3, 4C, 5B, 6A,7D, 7E2, 9A,B, 8A, 10A,B


GEOL G221 - Introductory Mineralogy

BA BSG

1,2,3, 4D, 5B, 7D, 7E2, 8A, 9A,B, 10A,B


GEOL G222 - Introduction to Petrology

BSG 1, 2,3, 4D, 5B, ,7D, 7E2, 8A, 9A,B, 10A,B



GEOL G323 - Structural Geology

Mo BSG

1, 2,3, 4D 5B, 6B, 7D, 7E2, 8A, 9A,B, 10A,B


GEOL G334 - Principles of Sedimentology and Stratigraphy

Mo BSG

1, 2,3, 4D, 5B, 6B, 7D, 7E2, 8A, 9A, 10A,B


GEOL G410 - Undergraduate Research in Geology BA BSG

3, 4D, 5B, 6C, , 7E, 7F, 8A, 9A,B, 10A,B


GEOL G429 - Field Geology in the Rocky Mountains (Field Camp)

BSGg 2, 3, 4D, 5B, 6C, ,7D, 7E2, 8A, 9A,B, 10


GEOG G315 - Environmental Conservation BSGe 1,2,3, 4B, 5B, 6B, 7D, 7E1, 8B, 9A,B, 10A,B


GEOL G300 - Environmental and Urban Geology BSGe Mo

1,2,3, 4D, 5B, 6C, 7D, 7E1, 8B, 9A,B, 10A,B


GEOL G451 - Principles of Hydrogeology BSGe 1,2,3, 4D, 5B, 6C, 7D, 7E1, 8A, 9A,B, 10A,B


GEOG G237 - Mapping Our World Mo 1,2,3, 4D, 5B, 6C, 7D, 7E1, 8B, 9A,B, 10A,B


2.C. Appendix to curricular maps: program requirements. Geology Minor

• GEOL G100/L100 General Geology with Lab OR GEOL G103 - Earth Science: Materials and Processes

• GEOL G104 - Earth Science: Evolution of the Earth • GEOL G211 - Introduction to Paleobiology • GEOG G237 - Mapping Our World OR GEOL G323 - Structural Geology • GEOL G300 - Environmental and Urban Geology OR GEOL G334 - Principles of Sedimentology

and Stratigraphy • Additional credits chosen from two GEOL/GEOG courses at the 200 level or higher: 6 Credits

BA Geology Introductory Earth Science Course

• ((AST A100 - The Solar System Cr. 3 OR GEOG G107 - Physical Systems of the Environment Cr. 3 OR GEOL G100 - General Geology Cr. 3-5 OR GEOL G210 - Oceanography) AND GEOL L100 - General Geology Laboratory) OR GEOL G103 - Earth Science: Materials and Processes (includes lab)


Core and Concentration (Major) Courses • MA 15300 - Algebra and Trigonometry I • MA 15400 - Algebra and Trigonometry II OR STAT 30100 - Elementary Statistical Methods I • CHM 11100 - General Chemistry OR CHM 11500 - General Chemistry • GEOL G104 - Earth Science: Evolution of the Earth Cr. 3. • GEOL G221 - Introductory Mineralogy • GEOL G410 - Undergraduate Research in Geology • Additional 200-level or above courses from BIOL, CHM, CS, ENGR, GEOL (excluding G410) MA or

PHYS Credits: 9 (Max of 3 credits outside GEOL/GEOG. Maximum 9 credits at the 200 level.)

B.S.G. Geology Introductory Earth Science Course

• ((AST A100 - The Solar System Cr. 3 OR GEOG G107 - Physical Systems of the Environment Cr. 3 OR GEOL G100 - General Geology Cr. 3-5 OR GEOL G210 - Oceanography) AND GEOL L100 - General Geology Laboratory) OR GEOL G103 - Earth Science: Materials and Processes (includes lab)

Core Courses • MA 22700 - Calculus for Technology I AND MA 22800 - Calculus for Technology II) OR (MA 16500

- Analytic Geometry and Calculus I AND MA 16600 - Analytic Geometry and Calculus II) • (PHYS 21800 - General Physics AND PHYS 21900 - General Physics II) OR (PHYS 22000 - General

Physics AND PHYS 22100 - General Physics) OR (PHYS 15200 - Mechanics AND PHYS 25100 - Heat, Electricity, and Optics)

• CHM 11500 - General Chemistry AND CHM 11600 - General Chemistry • GEOL G104 - Earth Science: Evolution of the Earth • GEOL G211 - Introduction to Paleobiology • GEOL G221 - Introductory Mineralogy • GEOL G222 - Introduction to Petrology • GEOL G323 - Structural Geology • GEOL G334 - Principles of Sedimentology and Stratigraphy • GEOL G410 - Undergraduate Research in Geology

Environmental Geology Option Courses • GEOG G315 - Environmental Conservation • GEOL G300 - Environmental and Urban Geology • GEOL G451 - Principles of Hydrogeology • STAT 30100 - Elementary Statistical Methods I OR PPOL E272 - Introduction to Environmental

Sciences • Additional 200-level or above courses from BIOL, CHM, CS, ENGR, GEOG, MA or PHYS Credits: 9 • (Max 3 credits outside GEOL/GEOG)Minimum 3 credits at the 400-level (excluding GEOL G410).

Geology Option Courses • Field camp experience (e.g., GEOL G429) • Additional 200 level or above courses from BIOL, CHM, CS, ENGR, GEOG, GEOL, MA or PHYS

Credits: • (Max 3 credits ourside GEOL/GEOG) ) • STAT 30100 - Elementary Statistical Methods I


3. Departmental Assessment Model

3.A. Description of Assessment Model Challenges The challenge at this point is developing a departmental Assessment model to assess student learning at key points. To this end we have begun consulting with the Office of Assessment under the Vice Chancellor for Academic Affairs, and will work with that office over the coming year.

We have instituted a routine assessment for general education courses which addresses campus-wide General Education SLOs specifically, so does not completely address questions of Minors, BA or BSG outcomes.

A similar program has not yet been instituted to capture the Student Learning Outcomes for the Geology Minor, BA or BSG at key points in their progress through the program.

The task ahead is to track SLOs through a student’s course path and to generate a cumulative list of SLOs that we expect a student to attain at any given point in each program. The assessment strategy will also require identification of the best courses from which to consistently capture students at various stages of the programs. Then we must identify appropriate targets.

Courses that might be good targets for assessment G100/G103, along with other 100 level general education courses, required for the minor, the BA and the BSG. These courses might serve as a pretest

G211 serves as an early midpoint course for BSG students. It is also the last common course that ALL minors must take.

G221 captures all BSG and BA students at late midpoint

G334 is the last content course (including other courses as pre or corequisites) that captures All BSG students and some of the minors

GEOG G237 captures the remaining minors at the same point.

G410 is a capstone that captures all majors

Field camp experience (e.g., GEOL G429) is an ultimate capstone for the BSG which is taught by outside schools. This might provide an opportunity for unbiased assessment of student learning in the BSG. The difficulty is developing and delivering a rubric to be used consistently by a range of outside instructors from a range of outside institutions.

3.B. Measures Used Our current model is to track numbers of student presentations at scientific conferences as a way of capturing outcomes 7E, 7F, and 8.

We also track number of students who work in the department as a measure of engagement, without reference to specific learning outcomes. Student work is funded by departmental funds and by research grants.


3.C. Rubrics or Evaluation Metrics Descriptions Proposed methods include questions delivered as either exam questions or as separate surveys.

Rubrics may be developed for evaluation of lab technical reports and capstone posters, theses, and slide presentations.

Reading comprehension may also be evaluated by reading samples with accompanying question sets.

3.D. Plan for Using Findings for Improvement Courses in the department have been established by long practice in the field of geology and each contains a more or less standard set of learning outcomes. Furthermore, with the exception of introductory/general education courses, each course is consistently taught by only one faculty member, all at full load.

Beyond changing out or rearranging courses in a program, adjustments to improve assessment results would be restricted to

Since courses and programs have already been significantly rearranged in the last two years, we would probably restrict changes to negotiations of SLO distribution among courses or possibly evaluation of methods and assignments.

For example, spreadsheet (Microsoft Excel) practice was added to the G211 curriculum in the Fall semester of 2015 when it was found that some Spring 2015 G410 students were not taking advantage of the full functionality of this computer software to efficiently process data for their research. This outcome problem was recognized informally, and the change was instituted informally as well. We hope that the exercise of conducting assessments formally will help make the process of curricular (per course) improvement more trackable. Finding such unexpected shortfalls in educational coverage may also result in adjustments to the SLOs themselves.

4. Assessment Results 4.A. Current Year Assessment Findings While we are working on instruments to directly measure SLOs, we report on both student employment in the department, and student involvement in research activities, including the IPFW Student symposium and scientific conferences sponsored by the Geological Society of America. The presentation of research projects provides a measure of outcomes 4, 5, 6 7, and 8. Currently we are measuring success at presenting at actual scientific conferences as evidence of competence.

Students employed within the department

Name Fall 14

Spri 15

Sum 15

Fall 15

F 1 1 1 1 H 1 1 1 S 1 1 1 Y 1 1 1 M 1 1 1 B 1


G 1 R 1 1 K 1 1 R 1 1 M 1 S 1 A 1 C 1 F 1 K 1 L 1 S 1 P total 7 9 4 12

Student Research Presentations Presentations at the 2015 Spring Student Research Symposium at IPFW (17 students, 17 presentations)

Oh, the Well Waters are Different From the Same Aquifer; Are They Really? Sarah Budd and Tessa Matthews. http://opus.ipfw.edu/stu_symp2015/11/

3D Reconstruction of the Vascularity of a Stegosaurus Dorsal Plate and an Alligator Scute. Dan Deifenbaugh, Benjamin Aeschliman, Paul Barrett, Charlotte Brassey, Shoji Hayashi, and Beomjin Kim. http://opus.ipfw.edu/stu_symp2015/76/

A Hands-on Approach: Making Geology Labs Accessible to the Visual Impaired. Sarah Fischer, Dawn Stager, and Tessa Matthews http://opus.ipfw.edu/stu_symp2015/19/

Can carbon isotopes constrain high-resolution stratigraphy of Ordovician shallow water facies in the Cincinnati, Ohio Region? Rebecca L. Freeman, Sarah Fischer, Thomas J. Schramm, Carlton E. Brett, Sasha L. Mosser, Michael Blair, and Suvankar Chakraborty. http://opus.ipfw.edu/stu_symp2015/20/

Bottoms Up: A Sedimentological Analysis of the Manus-Only Coffee Hollow Sauropod Tracks. Crystal Harter, James O. Farlow, Everett E. Deschner, and Richard Solis. http://opus.ipfw.edu/stu_symp2015/32/

River Sieve Analysis of Soil Samples at Indiana-Purdue University Fort Wayne. Shelby Johnston. http://opus.ipfw.edu/stu_symp2015/35/

Small Town Water Quality and Health Concerns, Nitrate Levels Increasing in the Midwest, Northeast Indiana, USA. Brittany Kime. http://opus.ipfw.edu/stu_symp2015/38/

Independent Research Project as an Integral Part of Geology Courses Provides Undergraduates the Opportunity to Solve Real World Problems. Brittany Kime, Tessa Matthews, Sara Budd, and Jessie Reeder. http://opus.ipfw.edu/stu_symp2015/37/

Weathering of Salem Limestone. Collin Lambert. http://opus.ipfw.edu/stu_symp2015/44/ Progressive Mylonitization of a Paleozoic Granite in the Cordillera Oriental, NW Argentina. Daniel

Maassel. http://opus.ipfw.edu/stu_symp2015/17/ Determining Water Quality At Indiana University-Purdue University Fort Wayne (IPFW) Using Well Data.

Tessa Matthews and Sarah K. Budd. http://opus.ipfw.edu/stu_symp2015/49/

http://opus.ipfw.edu/stu_symp2015/11/












The Microstructure of Pyrite Blackening in Fossil Shells. Paul O'Malley, Anne S. Argast, and Carlton Brett. http://opus.ipfw.edu/stu_symp2015/52/

The Graf Phosphatic Fauna: Is It Dwarfed? Jessie Reeder and Anne S. Argast. http://opus.ipfw.edu/stu_symp2015/57/

Osteology of Miocene fossil fishes from the Nevada Test Site. Michael Stoller and Robert B. Gillespie Ph.D. http://opus.ipfw.edu/stu_symp2015/65/

Does Sedimentary Phosphate Reflect Sediment Maturity? Amanda Straw. http://opus.ipfw.edu/stu_symp2015/66/

Sedimentology of the "Utica" below Cincinnati. Simon Whitehouse and Collin Lambert. http://opus.ipfw.edu/stu_symp2015/75/

Are Intertidal Sediments Biased With Respect to Carbon Isotopes? Ross Yeater, Rebecca Freeman, Anne S. Argast, and Sarah Fisher. http://opus.ipfw.edu/stu_symp2015/74/

Presentations at the 2014 Fall Geological Society of America Meeting, Vancouver, British Columbia. 19–22 October 2014 (7 Students, 8 presentations)

ISIORHO, Solomon A., BUDD, Sarah K., MATTHEWS, Tessa A., KIME, Brittany, and REEDER, Jessie . INDEPENDENT RESEARCH PROJECT AS AN INTEGRAL PART OF GEOLOGY COURSES PROVIDES UNDERGRADUATES THE OPPORTUNITY TO SOLVE REAL WORLD PROBLEM https://gsa.confex.com/gsa/2014AM/webprogram/Paper247839.html

REEDER, Jessie L. and ISIORHO, Solomon A. WATER QUALITY AND THE INVASION OF THE ST. JOSEPH RIVER, FORT WAYNE, INDIANA, BY CYANOBACTERIA. https://gsa.confex.com/gsa/2014AM/webprogram/Paper241322.html

BUDD, Sarah K., ISIORHO, Solomon A. and MATTHEWS, Tessa A. OH, THE WELL WATERS ARE DIFFERENT FROM THE SAME AQUIFER; ARE THEY? https://gsa.confex.com/gsa/2014AM/webprogram/Paper244034.html

MATTHEWS, Tessa A., BUDD, Sarah K. and ISIORHO, Solomon A. DETERMINING THE WATER QUALITY AT INDIANA UNIVERSITY-PURDUE UNIVERSITY FORT WAYNE (IPFW) USING WELL DATA https://gsa.confex.com/gsa/2014AM/webprogram/Paper239953.html

KIME, Brittany and ISIORHO, Solomon A. SMALL TOWN WATER QUALITY AND HEALTH CONCERNS, NITRATE LEVELS INCREASING IN THE MIDWEST, NORTHEAST INDIANA, USA https://gsa.confex.com/gsa/2014AM/webprogram/Paper240402.html

HARTER, Crystal, DATTILO, Benjamin, FARLOW, James O., DESCHNER, E. Everett and SOLIS, Richard, BOTTOMS UP: A SEDIMENTOLOGICAL ANALYSIS OF THE MANUS-ONLY COFFEE HOLLOW SAUROPOD TRACKS. https://gsa.confex.com/gsa/2014AM/webprogram/Paper249964.html

O'MALLEY, Paul, DATTILO, Benjamin, ARGAST, Anne, BRETT, Carlton E., THE MICROSTRUCTURE OF PYRITE BLACKENING IN FOSSIL SHELLS. https://gsa.confex.com/gsa/2014AM/webprogram/Paper249726.html







https://gsa.confex.com/gsa/2014AM/webprogram/Paper247839.html








PINAN LLAMAS, Aranzazu, LOPEZ, Jose Pablo, MAASSEL, Daniel, ACOSTA NAGLE, Ana and BUDD, Sarah K., PROGRESSIVE MYLONITIZATION OF A PALEOZOIC GRANITE IN THE CORDILLERA ORIENTAL, NW ARGENTINA. https://gsa.confex.com/gsa/2014AM/webprogram/Paper247148.html

Presentations at the Geological Society of America North Central Section Meeting, Madison, Wisconsin, May 19-20, 2015. (4 students, 4 presentations)

MATTHEWS, Tessa A, and ISIORHO, Solomon A., WHAT’S IN THE PHREATIC AQUIFER AT INDIANA UNIVERSITY-PURDUE UNIVERSITY FORT WAYNE (IPFW)? https://gsa.confex.com/gsa/2015NC/webprogram/Paper255557.html

DATTILO, Benjamin, REEDER, Jessie, FREEMAN, Rebecca L., ARGAST, Anne, ORDOVICIAN SMALL SHELLY FAUNA FROM THE ELGIN MEMBER OF THE MAQUOKETA: ECOLOGICALLY DWARFED OR TAPHONOMICALLY BIASED? https://gsa.confex.com/gsa/2015NC/webprogram/Paper256054.html

KIME, Brittany and ISIORHO, Solomon A., WATER QUALITY ALONG A RIVER PROFILE IN RURAL AGRICULTURAL SETTING IN THE MIDWEST, NORTHEAST INDIANA. https://gsa.confex.com/gsa/2015NC/webprogram/Paper255452.html

MCMICHAEL, Kaitlin N., and ISIORHO, Solomon A. IS THE HIGH MPN OF THE ZOO’S POND WATER TIED TO ITS GROWING WATERFOWL POPULATION? https://gsa.confex.com/gsa/2015NC/webprogram/Paper255888.html

4.B. Proposed Changes to Address Findings These findings do not warrant any changes in curriculum. We are in the process of improving the assessment process itself.

4.C. Prior Year Assessment Findings and Changes The significant findings of the previous year were that the assessment process needed improvement with more precise metrics for Student Learning Outcomes.

No significant changes were made to the curriculum based on prior year assessment findings.

The Main thrust of current changes is to improve the assessment process itself, and the SLOs and their curricular maps are the principle change to date. New assessment metrics and instruments, being developed with the help of the Office of Assessment should be well defined for next year’s assessment report.

4.D. Assessment Findings for Curricular Changes Made Major changes to the curriculum have been made recently. No further changes are immediately indicated.

5. Conclusions • The current assessment report presents a significant revision to the programmatic SLOs and

how these are mapped onto courses. • Students are performing well as evidenced by high participation in research presented both on

campus and at scientific conferences.


https://gsa.confex.com/gsa/2015NC/webprogram/Paper255557.html




INTL COAS Assessment Report 2014-‐15 (submitted Nov. 2015)

SECTION 1: SLOs

INTL Program Student Learning Outcomes 1. Students will demonstrate the ability to think critically about major international

issues. 2. Students will demonstrate an appreciation of the histories and cultures of other

nations and the various means used to promote and maintain normal relations among them.

3. Students will demonstrate an understanding of the impact of individual decisions on the world and world events on the individual.

SECTION 2: INTL CURRICULAR MAPS

A. Map of INTL Programmatic SLOs to Baccalaureate Framework Baccalaureate Framework

1. Acquisition of Knowledge: Students will demonstrate breadth of knowledge across disciplines and depth of knowledge in their chosen discipline. In order to do so, students must demonstrate the requisite information skills and technological competencies.

2. Application of Knowledge: Students will demonstrate the ability to integrate and apply that knowledge, and, in so doing, demonstrate the skills necessary for life-long learning.

3. Personal and Professional Values: Students will demonstrate the highest levels of personal integrity and professional ethics.

4. A Sense of Community: Students will demonstrate the knowledge and skills necessary to be productive and responsible citizens and leaders in local, regional, national, and international communities. In so doing, students will demonstrate a commitment to free and open inquiry and mutual respect across multiple cultures and perspectives.

5. Critical Thinking and Problem Solving: Students will demonstrate facility and adaptability in their approach to problem solving. In so doing, students will demonstrate critical-thinking abilities and familiarity with quantitative and qualitative reasoning.

6. Communication: Students will demonstrate the written, oral, and multimedia skills necessary to communicate effectively in diverse settings.

INTL PROGRAM SLOS

A. Students will demonstrate an appreciation of the histories and cultures of other nations and the various means used to promote and maintain normal relations among them.

B. Students will demonstrate an understanding of the impact of individual decisions on the world and world events on the individual.

C. Students will demonstrate the ability to think critically about major international issues.


Baccalaureate Framework Number

1 2 3 4 5 6 INTL SLO letter

A, B, C A, B, C A, B, C A, B, C A, B, C A, B, C

1. As an inherently interdisciplinary field, INTL requires students to demonstrate a breadth of knowledge in a variety of disciplines (including POLS, HIST, ECON, SOC, ENG, PHIL, etc.) and international settings. For example, a non-Western course distribution requirement ensures that an exclusively Eurocentric approach will not be taken. Students will demonstrate requisite informational skills and technological competencies through the research for and the successful completion of papers and other assignments in INTL-approved classes, as well as through their navigation of systems like Blackboard. These assignments will require the use of academic databases, the use of primary sources, etc. as appropriate to the individual class and assignment.

2. INTL students will demonstrate the ability to integrate and apply knowledge through the successful completion of papers and other assignments in INTL-approved classes. In the case of INTL I200, for example, students are required in formal written assignments to bring to bear the theories and concepts covered in class on a series of International Activities at IPFW and the wider community, such as the recent Omnibus Lecture given by retired four-star admiral Eric Olsen.

3. Students will demonstrate the highest levels of personal integrity and professional ethics through the careful citation of sources in their INTL assignments. Similarly, in constructive peer response groups and in active and respectful class participation students demonstrate personal integrity and ethical treatment of others.

4. Because of the breadth and depth provided by the INTL curriculum, students will be exposed to a truly diverse cross-section of cultures and nation-states from around the world, both Western and non-Western. The INTL language requirement ensures that INTL students will not simply study cultures at a remove, but experience for themselves these individual societies’ unique ways of viewing the world as embodied in their language, history, and culture. This important practical experience will do much to prevent an insular worldview, a skill essential in this era of rapid globalization. Finally, the topics central to INTL (war, diplomacy, globalization, nationalism, racism, etc.) all lead students to demonstrate an understanding of the importance of mutual respect and free and open inquiry. INTL’s stated goal to cause students to reflect on the connection of the individual to issues of international concern (and vice-versa) is intended to foster a nuanced understanding of the “global village” and their place within it.

5. Because of the varied disciplines involved in INTL, students will necessarily demonstrate facility and adaptability in their approach to problem solving: students will complete assignments in courses requiring quantitative reasoning (such as ECON E430: International Economics), and qualitative reasoning (such


as in POLS Y109: Intro to International Relations). Students will demonstrate critical thinking as they analyze and apply political theory, and engage in cultural, historical, and literary analysis of societies worldwide.

6. Because of the INTL language requirement, INTL students will demonstrate their ability to communicate in written and oral form not only in English, but in other modern languages as well. As a result of the diverse courses and disciplines involved in the INTL certificate, students develop and integrate written, oral, and multimedia skills to critically analyze a wide array of historical, political, and cultural topics, preparing them to productively participate in whatever diverse setting in which they might find themselves.

B. Map of INTL Programmatic SLOs to Identified “Core Courses” INTL PROGRAM SLOS

A. Students will demonstrate an appreciation of the histories and cultures of other nations and the various means used to promote and maintain normal relations among them.

B. Students will demonstrate an understanding of the impact of individual decisions on the world and world events on the individual.

C. Students will demonstrate the ability to think critically about major international issues.

INTL CORE COURSES INTL I200: Introduction to International Studies INTL I208: International Cinema INTL I330: Cultural Crossroads INTL I155: Intro to Language and Culture in Near Eastern and East Asian Studies (hopefully to be offered in Fall 16)

INTL Core Courses INTL I200 INTL I208 INTL I330 INTL I155 INTL SLO letter

A, B, C A, B, C A, B, C A, B, C

Note: Because they have not been offered in the past several years and no recent assessment data is available for them, this report will not focus on INTL I330 and INTL I155, instead foregrounding INTL I200 and INTL I208. The inability to regularly offer these classes evinces the need for additional faculty resources for INTL.

♦ INTL I200: Introduction to International Studies Because of the unique multi-disciplinary and team-taught nature of this class, students can easily demonstrate success in all three INTL SLOs from this one class alone. By definition, all INTL classes will address “major international issues” in formal assignments. In addition, because every section of I200 incorporates an introduction to


international relations, INTL students are required to analyze relations among modern nation-states in a formal paper as well as exams. The first half of the semester is spent covering the “big picture” theoretical models central to POLS and INTL such as realism, International Relations liberalism, and critical theory as well as topics like globalization. The second half of the semester focuses on individual “case studies” on which these theoretical models can be brought to bear. For example, the case study focusing on Northern Ireland affords the opportunity for students to critically examine (in formal papers, exams, and class discussion) the manner in which the British government’s decision to remove political prisoner status from those charged with political offenses had a profound impact on the individuals charged. This led to the individual decisions of Bobby Sands and other political prisoners in Long Kesh to protest in a variety of ways, from standing for election to the British and Irish parliaments, to going on hunger strike to the death. These deaths on hunger strike and the successful election of prisoners to parliament profoundly changed British and Irish history and politics; for example, by ending opposition parties’ historic electoral abstentionism, the prisoners began the movement towards participation in the Northern Irish peace process and, later, in constitutional politics. A case study like this, in short, is an excellent demonstration of the “impact of individual decisions on the world and world events on the individual.”

♦ INTL I208: International Cinema Like I200, I208 by definition requires students to critically engage with major international issues in formal papers, exams, and in class discussion. In addition, film provides an excellent medium (in both form and content) to discuss cultural and aesthetic norms. Special sections of I208 focusing on one country/culture afford the opportunity to analyze these issues in additional depth, complimenting the breadth of analysis in other INTL-approved classes. Other sections focused on issues such as cinematic depictions of colonialism around the world around obviously center more exclusively on the (often predatory) relationships among nations.

SECTION 3: INTL ASSESSMENT PLAN 2014-15

A. INTL Assessment Model:

1. As per the COAS “Best Practices” document, a minimum of two members of the INTL steering committee shall assess the material, with a third reader present for adjudication of markedly disparate assessment scores (greater than 1 point on a 4 point scale). The assessment shall be conducted in a group meeting format for the specific purpose of assessment, to allow norming and discussion of other issues as required.

2. An assessment rubric needs to be utilized in assessment of program goals (see attached rubric).

3. All assessment artifacts will be made anonymous, with student names, etc


removed. 4. When possible, INTL instructors will not assess their own assignments, but

instead will assess artifacts produced in other instructors’ classes / portions of a team-taught class.

5. Representative/statistically valid samples need to be taken from each course assessed. In the case of team-taught classes like INTL I200, artifacts should be taken in equal amount from all instructors.

6. In every academic year INTL I200 will be used as one of the courses to generate assessment data. As the only specifically required course in the INTL certificate and as a class offered in both Fall and Spring semesters, it provides an ideal opportunity to provide a benchmark, one that will provide statistically reliable numbers. All INTL I200 instructors will be asked to provide electronic copies of designated assessment materials to the INTL director.

7. Other INTL-prefixed classes will be assessed as and when offered. At present there are only four: INTL I200 (Intro. to International Studies), INTL I208 (International Cinema), INTL I330 (Cultural Crossroads) and INTL I155 (Intro to Language and Culture in Near Eastern and East Asian Studies). Due to insufficient faculty resources, these classes are offered with a varying degree of regularity. For AY 2014-15, INTL I200 was the only INTL-prefixed class offered.

8. A Qualtrics survey is currently being developed for distribution among AY 2014-15 graduates who earned the INTL certificate. It is hoped that this survey will shed light on students’ perceptions of their experiences, and is intended to compliment the assessment of classes such as INTL I200 (which is often taken relatively late in students’ academic careers).

9. A mechanism will be developed to collect data from 300-400 level INTL-certified classes. Past assessment models relied on students submitting the papers voluntarily outside of class, resulting in small data pools (only 4 of 28 INTL students provided the requested material in AY 2008-9). One possibility is the development of an INTL capstone course in place of one of the other 3 credit requirements. (The INTL Program Review, currently underway, will investigate this further.)

B. Measures Used This year’s assessment was undertaken exactly as described above in the INTL Program Assessment Plan. The only INTL-prefixed courses offered during this year were assessed: the 36 assessment artifacts were taken from 2 sections of INTL I200 (Intro. to International Studies), offered in Fall 14 and Spring 15. Anonymous assessment artifacts were read by a minimum of 2 assessors and scored on a 4 point scale utilizing a purpose-designed rubric. In AY 14-15 a third-reader adjudication was required only four times for the program assessment. This consistency of scores argues in favor of my policy of utilizing assessors who (as members of the INTL Committee) are not only familiar with the INTL Program Goals, but are also active teachers in the INTL certificate as well. For this year’s assessment of I200, samples were taken from both the Fall 2014 and Spring 2015 sections of the class. As I200 is a team-taught class, artifacts produced by


students for all eight of the instructors were represented in equal numbers in the sample. A section of I200 is capped at 40: a total of 36 artifacts were assessed. No other INTL-prefixed classes were taught in AY 2014-15. Success in meeting program goals was defined as 75% or more of assessed artifacts scoring 2 or higher (on a 4-point scale) on the purpose-designed program assessment rubric for each of the 3 program goals. These goals were clearly met.

C. INTL Program Assessment Rubric

1) Does the student demonstrate the ability to think critically about major international issues?


(1)

2) Does the student demonstrate an appreciation of the histories and cultures of other nations and the various means used to promote and maintain normal relations among them?


(1)

3) Does the student understand the impact of individual decisions on the world and world events on the individual?


(1)

Note: the classifications below take into account that INTL I200 is an introductory / Gen

Ed course. EXCELLENT: Student demonstrates exceptional depth and nuance in analysis, using particularly well-chosen examples to illustrate her/his points. GOOD: Student engages in solid analysis, though with less nuance than above, using solid examples in sufficient number to support/illustrate her/his points.


NEEDS WORK: Student engages in analysis but not with great depth and complexity. Some examples are provided, but could be greater in number, better chosen, or analyzed with greater precision. UNSATISFACTORY: Student engages in minimal or no analysis. Examples are excessively general or non-existent.


One of the strengths of INTL’s assessment model is that it relies upon the frequent and active participation of the faculty who are members of the INTL Steering Committee and who regularly teach classes like INTL I200. Additionally, because we assess together in the same room, once the assessment is complete we have important opportunities to discuss “in the moment” the assessment model itself, the form that class assignments/assessment artifacts take, as well as any other issues that arise. For instance, as a result of last year’s COAS assessment, the instructor who most regularly teaches INTL I208 plans to revise some of the paper assignments to more directly address some of the INTL program goals. The INTL Steering Committee and the other faculty who teach INTL I200 value collaboration in general, and frequently tailor their classes / portions of team-taught classes to compliment other INTL offerings. In addition to the regular informal contact that many INTL faculty have due to many of our offices being located on the second floor of the Liberal Arts building, the INTL Steering Committee meets at least twice a year in addition to assessment meetings. As Director of INTL, I regularly update INTL faculty about the program, and supply them via email the INTL COAS and General Education assessment reports, as well as the INTL USAP report, soliciting comments and suggestions for improvement from the faculty. In addition, as INTL Director, I have regular contact with the chairs of the departments most central to INTL, including POLS, HIST, ILCS, COM, and ENG, and communicate with them about topics relevant to INTL. SECTION 4: ASSESSMENT RESULTS (see also attached assessment data for more detail)

A. Current Year Assessment Findings Success in meeting program goals was defined as 75% or more of assessed artifacts scoring 2 or higher (on a 4-point scale) on the purpose-designed program assessment rubric for each of the 3 program goals. These goals were clearly met. In INTL I200, 100% of the assessed artifacts scored 2 or higher, with an average score of 3.37 for all


three of the program goals (3.14, 3.82, and 3.20 for Goals 1, 2, and 3 respectively). See Appendix 1 below for complete assessment data.


The strong successes evident in the assessment results of AY 2014-15 indicate that previous changes have had an impact, and that the strategies implemented should continue and expand. For example, I will encourage INTL I200 instructors to develop and revise rubrics for assignments. The real issue exposed in this year’s assessment for COAS is the under-resourcing of the INTL program in general. The fact that the certificate cannot count on regular INTL-prefixed course offerings other than INTL I200 puts the program at a disadvantage not only in terms of garnering assessment data, but more importantly, in terms of having a truly vibrant range of classes specific to the certificate—particularly classes under the Director of INTL’s direct supervision. For example, INTL I208 is most regularly taught by a French professor in International Languages and Culture Studies (ILCS); because French recently lost its only Continuing Lecturer, the remaining 2 tenured/tenure track faculty in the department cannot teach regularly for other programs. Similarly, the two instructors who are the best candidates to teach INTL I155 are the only full-time faculty in the Japanese and Arabic language programs, and as such have limited ability to teach anything but language courses. Likewise, History and Political Science have no faculty who teach East Asian content, and so cannot lend a hand for purposes of this class. This is one of the frustrations in being a certificate that relies upon other departments—especially when the INTL certificate supposedly values diverse course content, as evidenced by its 3-credit non-Western distribution requirement. Even my own participation in teaching in the INTL program is limited somewhat by my own obligations to my home department. While I teach one section of INTL I200 in both Fall and Spring semesters, the rest of my teaching is in English. I hope to remedy this somewhat by developing and International Literature class that would have an INTL prefix rather than an ENG one—however, this would require the ability to forgo teaching an equivalent class for English. These proposed changes would require some serious systemic alterations not just within INTL but would possibly require investment from COAS in the form of resources so that current faculty can teach more INTL classes. In the short term, because it is the only class that all INTL students are required to take, assessment of INTL I200 is essential to getting a read on the success of the INTL certificate. It’s central place in INTL assessment will remain. However, other measures need to be developed to give a fuller picture of the INTL certificate. In order to address this, I am currently developing a Qualtrics survey for distribution among AY 2014-15 graduates who earned the INTL certificate. It is hoped that this survey will shed light on students’ perceptions of their experiences, and is intended to


compliment the assessment of classes such as INTL I200 (which is often taken relatively late in students’ academic careers). In addition, I hope to develop a mechanism to collect data from 300-400 level INTL-certified classes in other programs/departments. I plan to approach the Director of Women’s Studies to investigate the possibility of using WOST W301: International Perspectives on Women for further INTL certificate assessment. Previous INTL Directors met with very limited success in their attempts to do this, perhaps in part because past assessment models relied on students submitting the papers voluntarily outside of class, resulting in small data pools (only 4 of 28 INTL students provided the requested material in AY 2008-9, the last year an attempt was made to obtain assessment artifacts from non-INTL prefixed courses). Another possibility is the development of an INTL capstone course in place of one of the other 3 credit requirements. (The INTL Program Review, currently underway, will investigate this and other options further.)

C. Prior Year Assessment Findings and Description of Changes Made AY 2013-14’s assessment was undertaken exactly as described above in the INTL Program Assessment Plan (see Section 3.A above). The only two INTL-prefixed courses offered during 2013-14 were assessed: INTL I200 (Intro. to International Studies) and INTL I208 (International Cinema). Anonymous assessment artifacts were read by a minimum of 2 assessors and scored on a 4 point scale utilizing a purpose-designed rubric. A third-reader adjudication was required only once for the program assessment, and that for only a single program goal for the artifact in question. This consistency of scores argues in favor of my policy of utilizing assessors who (as members of the INTL Committee) are not only familiar with the INTL Program Goals, but are also active teachers in the INTL certificate as well. In the case of I200, samples were taken from both the Fall 2013 and Spring 2014 sections of the class. As I200 is a team-taught class, artifacts produced by students for all eight of the instructors were represented in equal numbers in the sample. A section of I200 is capped at 40: a total of 36 artifacts were assessed. Only one section of I208 was offered in AY 2013-14, with a total final enrollment of 13 students. To produce the most reliable data possible with this small a sample size, assessment artifacts from all 13 students in I208 were assessed. AY 2014-15 witnessed a marked improvement over AY 2013-14: while certainly INTL was successful in AY 13-14 with 83.3% of assessment artifacts scoring 2 or higher, AY 2014-15 saw 100% success. Some of this improvement simply reflects that AY 14-15 was a year with more than our fair share of exceptional students: some of these students have gone on to top-shelf law schools, graduate schools, etc. Likewise, AY 2013-14 might be said to have had a couple unusually unsuccessful students who skewed the average results. This argues for a multi-year approach to assessment data rather than looking only at an individual year in isolation.


However, the majority of the improved results in AY 2014-15 reflect the active revision of course assignments by INTL I200 instructors after last year’s assessment. These revisions were made not just for purposes of COAS assessment, but rather to improve student success in general. Some faculty who previously had not used rubrics for INTL I200 assignments developed them—and importantly, provided those rubrics to students in advance so that there was no ambiguity in either assignment parameters or grading methodology. As a result, faculty and students were more likely to be on the same page with regard to course expectations. In the case of last year’s assessment of INTL I208, it was determined by the INTL committee that the artifacts being assessed did not uniformly provide an opportunity to evaluate whether INTL Program Goals 1 or 3 were being met. It was NOT the case that students were asked to address issues related to those goals and they failed to do so: instead, because of the wide range of focal choices available to the students as an inherent part of the assignment used to generate this assessment artifact, some papers simply were not expected to address all 3 INTL program goals. For this reason, the INTL committee decided to mark those papers utilizing a score of NA (Not Applicable) rather than the numeric score of the rubric. It was felt that this would more accurately represent the situation: it was not a failure to achieve a program goal that could be measured by the assignment, but rather an indication that the goal was not an inherent part of the particular assignment option chosen by the student. The NA percentage for Goal 1 in INTL I208 was 76.9%, and for Goal 3 it was 46.2%. All I208 artifacts addressed Goal 2. In future semesters, work will be undertaken in INTL I208 to develop assignments that address more directly a larger number of the INTL SLOs. Because INTL I208 was not taught in AY 2014-15 the results of this analysis are yet to be determined.

D. Assessment Findings for Curricular Changes Made Not applicable. Future curricular changes may be made once the Program Review is completed. SECTION 5: CONCLUSIONS Many of the conclusions below were anticipated in the questions upon which the current INTL Program Review focuses.

♦ All of the assessment evidence underscores that the INTL program goals are clearly being met. In both AY 2013-14 and AY 2014-15 the total and average scores for the classes assessed were well above the range defined as demonstrating success in achieving program goals, with this past year demonstrating a 100% success rate.

♦ The relative consistency of the scores (few adjudications required) argues in favor of my policy of utilizing assessors who (as members of the INTL Committee) are not only familiar with the INTL Program Goals, but are also active teachers in the INTL certificate as well. In addition, these multiple readings provide greater


reliability in the assessment numbers than the previous assessment model (which utilized only one reader and was not done with anonymous artifacts, etc).

♦ However, while assessment of INTL I200 and other INTL-prefixed courses provides a very good baseline to examine program success, other assessment artifacts need to be gathered from other sources to ensure greater opportunity to assess the achievement of all program goals. In the short term, this might be accomplished by collecting artifacts from upper-division INTL-approved classes offered by other departments such as WOST or POLS. Unfortunately, because of the atomized nature of the INTL certificate—relying as it does almost exclusively on cross-listed courses—this is perhaps easier said than done, as classes in other departments are not supervised by / directly beholden to the INTL Director. It may prove difficult to ensure the gathering of statistically significant numbers of assessment artifacts.

♦ Another option is the creation of a required INTL portfolio either inside or outside of an INTL-prefixed capstone class. Unfortunately, with the recent movement to restrict any additional classes beyond the 120 credit limit, this seems a less workable solution: preliminary opinion of the INTL Steering Committee argued fairly strongly against such a capstone for the certificate. In addition, it seems likely that some of the popularity of the INTL certificate originates precisely in its flexibility in terms of accepted offerings and its lack of a required capstone course. In fact, anecdotal evidence suggests that many students who are awarded the INTL certificate sign up for it relatively late in their IPFW careers, and indeed, many take Intro to International Studies in their senior year. In any case, the logistics of staffing and scheduling a 400-level INTL capstone class would need to be approached with care and forethought—it at very least would require a renegotiation of the INTL Director’s expected teaching obligations, currently divided between INTL and ENG (3 courses per year in English, 2 sections per year of Intro to International Studies).

♦ In a related manner, program goals need to be reassessed and reflected in the assessment mechanism. For example, while INTL requires 3 credits of non-Western subject matter, the current program goals do not mention this non-Western requirement, and do not measure the program’s success in fostering an understanding of non-Western cultures / the interaction among western and non-western political/social groupings. These issues are being studied as part of the current INTL Program Review.

♦ Additional attention will be paid in the future to develop assignments/artifacts in INTL-prefixed and cross-listed classes that more explicitly address multiple program goals.

INTL is currently engaged in its Program Review, and among the questions being addressed are the following (selected list as relevant to COAS Program assessment): Mission


• What is the relationship of the INTL mission statement to COAS / IPFW missions (as manifest in the new Strategic Plan for example)?

• Are current INTL program objectives current to the standards in the field at peer institutions (focusing only on certificate programs in the cases where majors/minors are offered)?

• Are current INTL program objectives in accord with IPFW’s Strategic Plan? • Are INTL program objectives being met, and in what ways might assessment of

program objectives be improved? Curriculum

• Is the current INTL curriculum / course distribution current to the standards in the field at peer institutions (focusing only on certificate programs in the cases where majors/minors are offered)?

• To what extent should current INTL course distributions (and courses therein) be revised?

Analysis of these issues will no doubt produce changes that will need to be reflected in a revised assessment plan. In particular, as mentioned above, INTL’s non-Western distribution requirement needs to be represented in its mission statement, program goals, and assessment plan. Additionally, in order to address a broader range of INTL program goals, a mechanism to assess non-INTL-prefixed classes needs to be developed and implemented (especially targeting upper-division courses). The potential development of an INTL portfolio or upper-division INTL capstone course will be explored. Finally, additional attention will be paid in the future to develop assignments/artifacts in INTL-prefixed and cross-listed classes that more explicitly address multiple program goals. Appendix 1: Individual INTL Program Assessment Artifact Data Program goals:

1. Does the student demonstrate the ability to think critically about major international issues?

2. Does the student demonstrate an appreciation of the histories and cultures of other nations and the various means used to promote and maintain normal relations among them?

3. Does the student understand the impact of individual decisions on the world and world events on the individual?

INTL I200 Artifact Program Goal 1 Goal 2 Goal 3


1-F2014 4/4= 4 4/4= 4 4/4= 4 2-F2014 3/3= 4 3/3= 3 3/3= 4 3-F2014 4/4= 4 4/4= 4 4/4= 4 4-F2014 4/4= 4 4/4= 4 4/4= 4 5-F2014 3/3= 3 2/2= 2 3/3= 3 6-F2014 4/2/2= 2.7 3/2/2= 2.3 4/2/3= 3 7-F2014 3/2= 2.5 2/2= 2 3/3= 3 8-F2014 3/2= 2.5 3/2= 2.5 3/2= 2.5 9-F2014 3/3= 3 3/2= 2.5 3/3= 3 10-F2014 4/3= 3.5 4/3= 3.5 4/3= 3.5 11-F2014 3/2= 2.5 3/2= 2.5 3/2= 2.5 12-F2014 3/3= 3 3/3= 3 3/3= 3 13-F2014 3/3= 3 4/3= 3.5 3/3= 3 14-F2014 4/3= 3.5 4/3= 3.5 4/3= 3.5 15-F2014 3/3= 3 3/3= 3 3/3= 3 16-F2014 4/3= 3.5 4/3= 3.5 4/3= 3.5 17-F2014 4/4= 4 4/4= 4 4/4= 4 18-F2014 4/4= 4 4/4= 4 4/4= 4 19-S2015 3/3= 3 3/2= 2.5 3/2= 2.5 20-S2015 4/4= 4 4/4= 4 4/4= 4 21-S2015 3/3= 3 3/2= 2.5 3/3= 3 22-S2015 3/3= 3 3/3= 3 3/3= 3 23-S2015 3/4/3= 3.3 3/4/2= 3 3/4/3= 3.3 24-S2015 3/3= 3 3/3= 3 3/3= 3 25-S2015 3/3= 3 3/2= 2.5 3/3= 3 26-S2015 2/3/3= 2.7 3/4/3= 3.3 3/4/2= 3 27-S2015 3/3= 3 3/4= 3.5 3/3= 3 28-S2015 4/2/3= 3 4/2/3= 3 4/2/3= 3 29-S2015 3/2= 2.5 3/3= 3 3/3= 3 30-S2015 3/2= 2.5 2/2= 2 3/2= 2.5 31-S2015 2/2= 2 3/3= 3 3/3= 3 32-S2015 3/3= 3 3/3= 3 3/3= 3 33-S2015 3/3= 3 3/3= 3 3/3= 3 34-S2015 3/3= 3 3/3= 3 3/3= 3 35-S2015 3/3= 3 3/3= 3 3/3= 3 36-S2015 4/3= 3.5 4/3= 3.5 4/3= 3.5 Averages: 3.14 3.82 3.20 % avg. scores greater than/ equal to 2: 100%, 100%, 100%

Department of Mathematical Sciences

Major Program Assessment Report 2014-‐15

November 12, 2015

Section 1: Student Learning Outcomes for the Program

Goals and outcomes for all majors The Department of Mathematical Sciences has adopted the following outcomes for the goals of all majors, with assessment to be done in the named course(s): Program Goal G1. Students should be able to reason mathematically.

G1 outcome 1. Students will demonstrate an understanding of the calculus: The differential and integral calculus of one and multiple variables, infinite series, the geometry of Euclidean space, and theorems of Green, Gauss, and Stokes. [MA 263] G1 outcome 2. Students will demonstrate an understanding of elementary linear algebra: Linear transformations, finite dimensional vector spaces, matrices, determinants, systems of linear equations. [MA 351] G1 outcome 3. Students will demonstrate understanding of high-‐level topics such as sets, logical inference, induction, recursion, counting principles, binary relations, vectors and matrices, elementary graphs, and algorithm analysis. [MA 175]

Program Goal G2. Students should be good problem solvers.

G2 outcome 1. Students will demonstrate the ability to translate real-‐world or discipline-‐specific problems into mathematical language, and the solutions of mathematical problems into ordinary language. [courses are option specific] G2 outcome 2. Students will demonstrate the ability to choose, apply, and adapt appropriate strategies to solve diverse problems. [courses are option specific]

Program Goal G3. Students should be able to understand and apply mathematical concepts to other disciplines.

G3 outcome 1. Students will understand basic applications of the calculus to the physical sciences and engineering, and be able to use appropriate techniques in various contexts. [MA 263] G3 outcome 2. Students will understand basic applications of linear algebra and be able to use appropriate techniques in various contexts. [MA 351]

Goals and outcomes for specific options Goals and Outcomes for the Teaching option The department has already approved goals and outcomes for all options.

The department has already approved goal G4 for the Teaching option Program Goal G2. Students should be good problem solvers.

G2 outcome 1. Students will demonstrate the ability to Translate real-‐world or discipline-‐specific problems into mathematical language, and the solutions of mathematical problems into ordinary language. [MA 460] G2 outcome 2. Students will demonstrate the ability to choose, apply, and adapt appropriate strategies to solve diverse problems. [MA 460]

Program Goal 4: Students will be prepared to deliver necessary mathematics content to high school students.

G4 outcome 1: Students will demonstrate mathematical knowledge necessary for high school teaching by scoring at or above the minimum score for the CASA exam. [assessment via CASA exam] G4 outcome 2: Students will demonstrate their knowledge of geometry, especially those concepts typically taught in high school geometry courses. [MA 460] G4 outcome3: Students will demonstrate their ability to implement lessons successfully by completing CEPP Key Assessment 6 and receiving an acceptable score for certification. [EDUC M448] G4 outcome 4: Students will demonstrate the ability to teach by receiving scores of Target Acceptable for all items assessed for student teaching. [EDUC M480]

Goals and Outcomes for the Statistics option The department has already approved goals and outcomes for all options. The department has already approved goal G4 for the Statistics Option. Program Goal G2. Students should be good problem solvers.

G2 outcome 1. Students will demonstrate the ability to Translate real-‐world or discipline-‐specific problems into mathematical language, and the solutions of mathematical problems into ordinary language. [STAT 511] G2 outcome 2. Students will demonstrate the ability to choose, apply, and adapt appropriate strategies to solve diverse problems. [STAT 511]

Program Goal G4. Students should have mastered fundamental concepts of statistics up to the level of multiple regression and analysis of variance.

G4 outcome 1. Students should be able to make statistical inference by applying hypothesis tests and confidence intervals. [STAT 517] G4 outcome 2. Students should be able to apply calculus to solve certain probability and estimation problems. Students should be able to distinguish between different probability models and apply them in problem solving. [STAT 516]

G4 outcome 3. Students should be able to fit a regression models, perform a diagnostic analysis, and make appropriate inferences from data. [STAT 512] G4 outcome 4. Students should be able to design simple experiments and carry out the statistical analysis and interpretation. [STAT 514]

Goals and Outcomes for the Actuarial Sciences option The department has already approved goals and outcomes for all options. The department has already approved goal G4 for the Actuarial Sciences Option. Program Goal G2. Students should be good problem solvers.

G2 outcome 1. Students will demonstrate the ability to translate real-‐world or discipline-‐specific problems into mathematical language, and the solutions of mathematical problems into ordinary language. [STAT 511] G2 outcome 2. Students will demonstrate the ability to choose, apply, and adapt appropriate strategies to solve diverse problems. [STAT 511]

Program Goal G4. Students should have sufficient preparation in calculus, linear algebra, probability, and statistics to pass the preliminary actuarial science examinations and obtain Validation by Educational Experience credit from the Society of Actuaries.

G4 outcome 1. Students should be able to make statistical inference by applying hypothesis tests and confidence intervals. [STAT 517] G4 outcome 2. Students should be able to apply calculus to solve certain probability and estimation problems. Students should be able to distinguish between different probability models and apply them in problem solving. [STAT 516] G4 outcome 3. Students should be able to fit a regression models, perform a diagnostic analysis, and make appropriate inferences from data. [STAT 512]

Goals and Outcomes for the (Pure) Mathematics option The department has already approved goals and outcomes for all majors. The department has already approved goal G4 for the Pure Math option. Program Goal G2. Students should be good problem solvers.

G2 outcome 1. Students will demonstrate the ability to Translate real-‐world or discipline-‐specific problems into mathematical language, and the solutions of mathematical problems into ordinary language. [MA 441, MA 453] G2 outcome 2. Students will demonstrate the ability to choose, apply, and adapt appropriate strategies to solve diverse problems. [MA 441, MA 453]

Program Goal G4. Students should know and be able to apply fundamental concepts of advanced undergraduate mathematics.

G4 outcome 1. Students will demonstrate knowledge of concepts of higher mathematics, including logic and proof techniques, functions, relations, and the real numbers as a complete ordered field. [MA 305] G4 outcome 2. Students will demonstrate the ability to solve a variety of differential equations [including first order differential equations, higher order linear differential equations, systems of first order equations, and basic PDEs]. [MA 363] G4 outcome 3. Students will demonstrate knowledge of the fundamentals of real analysis [theory of functions of a real variable, theory of differentiation and Riemann integration, sequences and series of functions, uniform convergence]. [MA 441] G4 outcome 4. Students will demonstrate knowledge of the fundamental properties of the constructs of abstract algebra [groups, rings, the integers, polynomials, and fields]. [MA 453]

Goals and Outcomes for the Computing option The department has already approved goals and outcomes for all options. Although plans reported last year included the completion of goals and outcomes specific to the Computing option, the Department Academic Affairs Committee has not yet been able to determine locations in the major, contained within departmental courses that appropriately reflect the computing focus of the option. Courses containing a specific computing focus are not housed in the Mathematical Sciences Department. As an alternative, the Mathematical Modeling course (MA 314) is being considered, along with other possibilities, as a source of appropriate assessment data. The role of MA 314 throughout the Mathematics program is currently also being considered. Thus, completing Computing option goals and outcomes is dependent upon the result of these considerations. Goals and Outcomes for the Business option The department has already approved goals and outcomes for all options. Goal 4: Students will develop and apply knowledge in the fundamentals of business with emphasis in mathematical finance and business models.

G4 Outcome 1: Students will be able to convert between different types of interest rates. G4 Outcome 2: Students will be able to calculate the present value or accumulated value of a set of cash flows, an annuity or a perpetuity. G4 Outcome 3: Students will be able to calculate the internal rate of return given a set of cash flows and their present values. G4 Outcome 4: Students will be able to calculate a loan payment amount and an amortization schedule. G4 Outcome 5: Students will be able to calculate the price, duration and convexity of a bond. G4 Outcome 6: Students will be able to calculate the value, duration and convexity of a series of cash flows. G4 Outcome 7: Students will be able to understand the relationship between the spot rate curve, the yield curve and the forward rates. In particular, they should be able to calculate one given one of the others. G4 Outcomes 1-‐7 will be assessed in MA 27300.

G4 Outcome 8: Students will be able to develop and use mathematical models that describe business problems. [MA 31400]

Section 2: Curricular Maps A. Map of Programmatic SLO’s to Baccalaureate Framework

Mathematical Sciences SLO’s to Baccalaureate Framework Map

B1: ACQ K B2: APP K B3: PPV B4: SOC B5: CTPS B6: COM Goal 1 X X Outcome 1 x x Outcome 2 x Outcome 3 x Goal 2 X X Outcome 1 x x Outcome 2 x x Goal 3 X X X Outcome 1 x x x Outcome 2 x x Goal 4 S, A, M M Outcome 1 T, S, A, M A Outcome 2 T, M S, A S, A Outcome 3 M S, A S, A T Outcome 4 M S S T Legend X,x All Math Majors T Teaching Option

S Statistics Option M Mathematics Option A Actuarial Sciences Option Baccalaureate Framework Objectives:

Acquisition of Knowledge ACQ K Students will demonstrate breadth of knowledge across disciplines and depth of knowledge in their chosen discipline. In order to do so, students must demonstrate the requisite information-‐ seeking skills and technological competencies.

Application of Knowledge APP K Students will demonstrate the ability to integrate and apply that knowledge, and, in so doing, demonstrate the skills necessary for life-‐long learning.

Personal and Professional Values PPV

Students will demonstrate the highest levels of personal integrity and professional ethics.

A Sense of Community SOC Students will demonstrate the knowledge and skills necessary to be productive and responsible citizens and leaders in local, regional, national, and international communities. In so doing, students will demonstrate a commitment to free and open inquiry and mutual respect across multiple cultures and perspectives.

Critical Thinking and Problem Solving CTPS Students will demonstrate facility and adaptability in their approach to problem solving. In so doing, students will demonstrate critical-‐thinking abilities and familiarity with quantitative and qualitative reasoning.

Communication COM Students will demonstrate the written, oral, and multimedia skills necessary to communicate effectively in diverse settings.

B. Map of Programmatic SLO’s to Identified “core courses” in the curriculum

Mathematical*Sciences*Curriculum*Map

Major&Option&Codes: ALL=All&Math&Majors ACT=Actuarial& STAT=Statistics PURE=Pure&Math TCH=Math&Teaching

MA&175&(discrete) MA&263&(multiv.) MA&305&(Intro.&Proof)MA&351&(lin.&Alg.) MA&363&(diff.&eq.) MA&441&(Real&An.)Goal&1Outcome&1 ALLOutcome&2 ALLOutcome&3 ALL

Goal&2Outcome&1 PUREOutcome&2 PURE

Goal&3Outcome&1 ALLOutcome&2 ALL

Goal&4Outcome&1 PUREOutcome&2 PUREOutcome&3 PUREOutcome&4

MA&453&(alg.) MA&460&(geom.) STAT&511 STAT&512 STAT&514 STAT&516Goal&1Outcome&1Outcome&2Outcome&3

Goal&2Outcome&1 PURE TCH ACT,&STATOutcome&2 PURE TCH ACT,&STAT

Goal&3Outcome&1Outcome&2

Goal&4Outcome&1Outcome&2 TCH ACT,STATOutcome&3 ACT,STATOutcome&4 PURE STAT

STAT&517 EDUC&M448 EDUC&M480Goal&1Outcome&1Outcome&2Outcome&3



Goal&4Outcome&1 ACT,STATOutcome&2Outcome&3 TCHOutcome&4 TCH

Mathematical*Sciences*Curriculum*Map

Major&Option&Codes: ALL=All&Math&Majors ACT=Actuarial& STAT=Statistics PURE=Pure&Math TCH=Math&Teaching

MA&175&(discrete) MA&263&(multiv.) MA&305&(Intro.&Proof)MA&351&(lin.&Alg.) MA&363&(diff.&eq.) MA&441&(Real&An.)Goal&1Outcome&1 ALLOutcome&2 ALLOutcome&3 ALL

Goal&2Outcome&1 PUREOutcome&2 PURE

Goal&3Outcome&1 ALLOutcome&2 ALL

Goal&4Outcome&1 PUREOutcome&2 PUREOutcome&3 PUREOutcome&4

MA&453&(alg.) MA&460&(geom.) STAT&511 STAT&512 STAT&514 STAT&516Goal&1Outcome&1Outcome&2Outcome&3

Goal&2Outcome&1 PURE TCH ACT,&STATOutcome&2 PURE TCH ACT,&STAT


Goal&4Outcome&1Outcome&2 TCH ACT,STATOutcome&3 ACT,STATOutcome&4 PURE STAT

STAT&517 EDUC&M448 EDUC&M480Goal&1Outcome&1Outcome&2Outcome&3



Goal&4Outcome&1 ACT,STATOutcome&2Outcome&3 TCHOutcome&4 TCH

Section 3: Assessment Plan A. Description of Department’s Assessment Model

At the end of each semester, the Department Assessment Committee sends out a request letter to instructors of all courses participating in the assessment process. The request letter clearly states the goal(s) and student learning outcome(s) to be measured. The instructors then return a report, along with a copy of their final and a map of the problems used to measure the designated learning outcomes. The Assessment Committee collects these reports and summarizes the data. All student artifacts are maintained by the faculty.

Samples of instructor reports are provided, per previous request of the COAS committee.

B. Measures Used

The measurements used in assessing student performance are instructor and/or curricular group selected questions to specifically assess designated outcomes, which are embedded in a project or final exam of each of the key courses. Instructors evaluate students’ performance on those specific questions, and the number of students meeting the goal(s), i.e., scoring at least 70% on learning goal designated problems, are reported to the department. These results are collected and analyzed by the Assessment Committee.


The departmental aspirational goal is that 80% or more of students will score at least 70% on those designated questions selected to directly address outcomes corresponding to each course in the assessment plan. It should be emphasized the 80% goal was originally set as a stretch goal. As previously collected data sets have been for relatively small numbers of department majors, an aggregate average has been computed to summarize all data collected since the beginning of the current assessment plan in Spring 2011.

Reflecting upon the fact that some courses now have data from up to 10 separate courses, the Assessment Committee has begun exploring methods of time series analysis. A guiding philosophy behind the effort is that, while aggregate averages have important summarizing value, the investigation of baseline trends and responses to curricular interventions may also become evident from application of signal versus noise analysis to student success data tracked over time. In order that conclusions drawn are based upon a strong statistical foundation, at least 30 data points are generally needed. The committee is continuing to work with departmental expertise toward quantifying the balance of conclusions drawn from fewer data points and relative degrees of significance in such conclusions. The results will provide a robust analysis of assessment data of student success, from both the perspective of identifying statistically meaningful trends as well as drawing valuable information on variability in the data.

To provide a sense of current work on such time series analyses, data concerning Goal 1 Outcome 3 (MA 175), relevant to students in all options, will be discussed as a specific example. Starting with data from Summer 2011, there are 10 data points representing sections held in Fall, Spring, and Summer terms. The current long-‐term average is around 51% of students reported succeeding at a 70% level or above on this outcome, which is below the aspirational goal of 80%. However, the long-‐term average includes very low percentages from early stages in programmatic assessment efforts, and these alone present difficulty for even overachieving

future sections to overcome in the long-‐term average.

Plotting the data in order from oldest to most recent, the following time series graph is obtained.

Here, the vertical axis is shown as percentage of students achieving the goal, and the horizontal axis simply lists the 10 data points from individual courses, from 1 (oldest) to 10 (most recent). The impact of the initial data points on the aggregate is now easily observable, as are variations from term to term. One source of variation that warrants attention may be seen in the dramatic peak at data point 6, which represents a report from a course with 1 department major enrolled who satisfied the outcome. If, on the other hand, that single student had not been successful, the data point would show a dramatic drop to 0 at data point 6.

Reports representing a small number of students have a limited number of possible outcomes. This has a dramatic and somewhat artificial effect on time series variability. In order to factor out the effect and allow better analysis of an overall trend, two alternative approaches have been introduced based on (1) a three-‐year moving average, and (2) a weighting method, both with the goal of tempering the impact of reports for a small number of students.

A three-‐year moving average is often applied in situations where individual data points are obtained from populations of varying size, as well as when there is interest in balancing out the influence of isolated high or low points (such as the 100% results reported for a class enrolling 1 department major). Each data point is replaced with an average computed for aggregated student results in the corresponding course and previous two courses. (In the case of the first two data points, the average is done over just the courses available. Alternatively, the time series could be started with the third course.) Such an analysis conducted in the case of MA 175 data results in the following graph.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10

The peak previously observed at 100% for the course representing 1 department major is now effectively smoothed out, and a more discernible baseline trend may be observed.

The second type of time series considered by the Assessment Committee is one where data points for each course are replaced by the reported success percentage multiplied by a weighting factor, determined according to the population of students represented in that data point. The weighting factor is designed intentionally to introduce elements of randomness, reflecting a successful practice commonly used to isolate a signal existing within noisy data.

Intuitively, if there are more students contained in a reported percentage, then this represents an outcome from a population of students that had many ways the resulting percentage could have come out lower than the 80% target. For example, with a single student contained in the report, 1/2 = 0.5 of the possible outcomes are below 80%, while in a report containing results for 30 students, 24/31 or roughly 0.774 of the possible outcomes are below 80%. Another way to look at such a fraction is it provides a measure of the probability a reported result could have come out below the 80% target if student results were purely random. Thus, such a probability provides a way of weighting for number of students represented in a reported percentage. For example, the 100% success in a report for 1 student can be weighted as 0.5 (100%) = 50% while a result of 100% in a report for 30 students can be weighted as 0.774 (100%) = 77.4%.

As a way of normalizing such weighted percentages against our target, allowing for easier comparison with the target, the weighted percentages are divided by a weighted version of the 80% target for a hypothetical report containing 30 students. Presented as a percentage, which may be thought of as a “student success index”, a graph of the same data for MA 175 rescaled according to such an approach now appears as follows.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10

In this graph, the target can be thought of a 100%. It turns out, however, the weighting used is not exactly normalizing, and results above 100% are possible. The committee is continuing to investigate such an approach, in consultation with departmental expertise.

Again, a more discernable baseline trend emerges from the data. It is important to note the approach used here intentionally introduces randomness as a way of drawing out information about an existing signal, e.g., student success trend, as well as variability. Rather than try to control for all sources of variation, the analyses provides a powerful means of investigating trends in data that inherently contain sources of variation, which can reasonably be modeled as random.


Results from an academic year will be discussed by the entire department during a special departmental meeting early in the Fall semester, with the intention of discussing results of previous curricular interventions, implementing changes for the following academic year, and reviewing methods of data analysis utilized.

Recommendations for curricular changes, based upon results following previous changes, data analyses, and feedback from the department, will be communicated from the Assessment Committee to the entire department and, subsequently, to the various curricular committees.

Continuing communication of program assessment policies, practices, data, and reports will be maintained on a secure department shared, online location. Immediate access will be thus provided for all departmental purposes, from individual instructor course planning to departmental considerations of curricular planning and assessment.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10


A. Current Year Assessment Findings

A cursory view of the data presented above suggests some form of intervention might be considered in MA 175. On the other hand, a related question is what type of statistical significance may be quantified for such an observation based upon a time series analysis for 10 data points. Both directions will be presented for departmental consideration as a result of the analyses of assessment data.

Although there are currently just four data points for MA 263 concerning Goal 1 Outcome 1 and Goal 3 Outcome 1, both moving average and success index approaches described above reveal an emerging baseline trend that are reflective of the department target, as may be seen in the following graphs.

Goal 1 Outcome 1 (MA 263) – Three-‐year Moving Average

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4

Goal 1 Outcome 1 (MA 263) – Weighted, Success Index Approach


0%

20%

40%

60%

80%

100%

120%

1 2 3 4

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4


Concerning Goal 1 Outcome 2, there is now adequate data from MA 351 (Linear Algebra) to begin investigating a baseline trend. Despite some variation observed, success with this goal appears on target from both methods of time series analysis. It is interesting to note the Three-‐year Moving Average results smooth out and in some sense reveal variations predominantly due to number of students contained in the individual data reports.

0%

20%

40%

60%

80%

100%

120%

1 2 3 4



On the other hand, both methods of analysis for Goal 3 Outcome 2 below (again in Math 351), provides reason to begin further study into the wide variation observed, as may be seen in the

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7 8 9 10

0%

20%

40%

60%

80%

100%

120%

140%

1 2 3 4 5 6 7 8 9 10

graphs below. The same caveats regarding degree of statistical significance in these observations, presented previously in the context of MA 175 above, apply here also.



0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1 2 3 4 5 6 7

0%

20%

40%

60%

80%

100%

120%

140%

1 2 3 4 5 6 7


As included in the discussion of the previous item on assessment findings, consideration of changes to address the baseline trend observed concerning MA 175 and the variation observed in MA 351 are suggested.

In parallel, quantification of the statistical significance which can be attributed to time series analyses based upon less than an optimal number of data points is necessary alongside any consideration of implementing changes at this point.


From the point of view of aggregated averaged data considered last year by the Assessment Committee, interventions in MA 351 appeared necessary. Discussions involving the relevant departmental curriculum committee, instructors, and other faculty resulted in a change of text, which was deemed better suited to the curriculum and student audience.

In the previous departmental assessment report, a consideration of possible changes to MA 305 is also suggested. Assessment data for this course still only consists of two course reports. Thus, it will be necessary to obtain at least two more data points prior to reporting on progress in such direction.


Although qualification of conclusions drawn here are needed concerning statistical significance, the data does suggest changes made in MA 351 have had a positive impact. The trend, along with further consideration of variation observed in the data will warrant continuing study.

Section 5: Conclusions, Next Steps, and Communication

As suggested in the work reported above, the following steps appear to provide necessary steps for advancing program quality and further promoting student success. These will form continuing agenda items for the department through ongoing work of the Assessment Committee as well as other committees/groups according to departmental structure and as deemed appropriate by department faculty.

• Complete option specific outcomes for the Computing Option, begin collection of new Goal 2 designated course assessments (Teaching and Pure Mathematics options), and determine the future role of MA 314 as a potential project based, capstone course in the departmental program assessment model.

• Review approved department goals and assessments to ensure consistency with current practices.

• Provide for both accessibility and security of all departmental assessment policies, regularly scheduled data collections, analyses, and reports.

• Automate the communication and collection of required assessment data reports from

instructors of courses designated in approved student learning outcomes. • Develop the central role of the Department Assessment Committee in providing for the flow of

data-‐driven programmatic recommendations and follow-‐up analyses. • Continue to develop both aggregate and time series analyses, toward supporting conclusions

based upon a foundation of corresponding statistical theories of data analysis. • Conduct further investigation of the baseline trend suggested in MA 175 data, variation

observed in MA 351 data, and related potential directions for curricular improvement. Note: Designated departmental curriculum committees are preparing the assessments of general education courses. The reports will be uploaded on the Gen Ed site on Vibe.

Appendices – Last year’s assessment report, COAS committee response letter, sample instructor reports

2013-2014 Assessment Report

Department of Mathematical Sciences

As was done in last year’s assessment report we shall provide two parts, an assessment of the Bachelor of Science in Mathematics Undergraduate Program and an update on the progress of the assessment of the quantitative reasoning part of the General Education Program at IPFW. Individual course reports will be filed on the General Education vibe site by December 31, 2014. Assessment of the BS in Mathematics Program The primary mission of the Department of Mathematical Sciences is to help prepare our majors for employment in business and industry, teaching in secondary schools, or study for advanced degrees. In 2010-2011 we approved a departmental assessment process, the assessment procedures (AAC 2010.2) and the common goals (AAC 2010.3) for all math majors, as well as option-specific goals for our Statistics and Actuarial Science options (AAC 2010.4 and AAC 2010.5). The option-specific goals for our pure mathematics option (AAC 2012.2) were passed in Fall 2012 and for our math teaching were passed by the Department in Spring 2014. Our original plan to draft the option-specific goals for the computing and the business options was delayed by the recent changes to the computer science and the business minors required by the respective programs. To reflect the recommendations of the COAS Assessment committee, the Academic Affairs Committee combined the entire assessment plan into one document (see attached), which will be amended once the option-specific goals of the two programs above are accepted in the Spring 2015 semester. The complexity of defining option-specific goals stems primarily from the fact that significant part of the measurable outcomes is in courses located in other departments. We are exploring the utilization of our capstone course Introduction to Mathematical Modeling MA 314 to measure appropriate outcomes. Below we address concerns raised in the COAS letter from January 14, 2014 (see attached). Concern: The data provided is minimal. Indeed, while teaching the most credit hours of all departments on campus, we have a modest number of majors in each of the core courses used for assessment. We continue

to accumulate data. Since almost half of our majors transfer from other departments or colleges, we are considering assessing the learning outcomes for all students in the core classes, MA 175, MA 263/MA 261, and MA 351, as many of these students are potential majors. We would like to solicit the advice of the COAS Assessment committee. The three goals we assess are that students should be (i) able to reason mathematically; (ii) be good problem solvers; and (iii) be able to apply mathematical concepts to other disciplines. Concern: Standards set by the Department are not met. The Department is concerned with the students’ performance in MA 305. We have recently changed the prerequisite for MA 30500 to allow MA 27500 or MA 17500. We are considering whether our students would be better served with some changes in MA 30500 and/or MA 17500 or MA 27500. Material in MA 17500 and MA 27500 is the first opportunity for students to begin learning abstract mathematical problem solving, proof techniques, and proof writing. MA 30500 is the second. Our goal remains the same. Another concern is the lower performance on Outcome 2 of Goal 3, application of mathematical concepts to other disciplines, measured in the linear algebra course MA 351. This is an important service course for science and engineering majors. We are considering some curriculum review to benefit our math majors. Concern: Include a clear explanation of the process and description of the test or artifacts used by the professors who complete the outcomes evaluations. The main tool used in assessing the student performance is specific questions to assess the designated outcomes embedded in the final exam in each of the key courses. The Assessment Committee has developed procedures for streamlining the reporting process of the primary data. A request letter is sent to instructors of the courses participating in the assessment process. They return a report with a copy of their final and a map of the problems used to measure the designated learning outcomes. The Assessment Committee of the department summarizes the reports. Students’ artifacts are kept by the faculty. Although most of the data are still sparse, the cumulative information for some of the goals allows us to start meaningful analysis. The attached spreadsheet summarizes our currently available data. Specific percentages of students meeting each goal are compiled. We stress that these percentages, individually, are not statistically meaningful. Referring to the spreadsheet, the bottom row compiles data for each course over time, the rightmost column compiles outcomes for each goal, and the lower right box has the totals. Analysis: The best conclusions we can draw from the data are summarized above, with regard to MA 305 and MA 351 outcomes. In summary, the development of the assessment process of the Department of Mathematical Sciences is well under way; the collection of data to assess the common

IV. Response Letter to Last Year’s Assessment Report

goals for all options is in its fourth year. We expect to reach critical mass in most outcomes of the common goals and close the loop in Fall 2015. The option-specific goals for the three options housed entirely in the department as well as the math teaching option are adopted and data is being collected. Recent changes in business and computer sciences minors have delayed the adoption of option-specific goals for the Business and the Computing options. The Academic Affairs Committee is exploring the utilization of the Capstone Course MA 314 into our assessment process. Update on A.3 - Quantitative Reasoning General Education assessment Early in the Fall 2013 semester we developed course assessment plans for the following courses: MA 101 – Mathematics for Elementary Teachers I; MA 153 – Algebra and Trigonometry I; MA 159 – Precalculus; MA 165 – Analytic Geometry and Calculus I; MA 168 – Mathematics for the Liberal Arts Student; MA 227 – Calculus for Technology; MA 229 – Calculus for the Managerial, Social, and Biological Sciences I; STAT 125 – Communicating with Statistics. Unlike the assessment of our majors, which is housed in the (department) Assessment Committee, the assessment of the General Education Quantitative Reasoning learning objectives for the eight courses above is housed in the corresponding Curriculum Committees (Elementary Concepts – MA 101; Precalculus – MA 153, 159; Calculus – MA 165, 227, 229; Discrete and Computational Mathematics – MA 168; Statistics – STAT 125). The collection of data started in the Fall 2013 semester. The curriculum committees are in the process of summarizing the collected data for 2013-2014 academic year and assessing the corresponding learning outcomes for each of the courses. Assessment reports will be filed by December 31, 2014. Sincerely,

Peter Dragnev, Professor and Chair Department of Mathematical Sciences Indiana University-Purdue University Fort Wayne, Fort Wayne, IN 46805 Tel. (260) 481-6382 Email: [email protected]

Section 1: Student Learning Outcomes for the Program (SD 98-22 rev. Appendix D Section I) The student learning outcomes are included below. These may be broken down into large categories: 1) Thinking like a physicist, 2) Communication, 3) Scientific ethics, and 4) Physics Content. The latter is determined at the course level. Every course has its own outcomes of which approximately 75% are expected to be achieved.

Program Outcomes

1. Thinking like a physicist – Having a physicist’s habits of mind. a. Forming arguments:

i. Students can construct an argument supported by conceptual and/or quantitative reasoning and reach an evidence based conclusion.

ii. Students can critique arguments and separate arguments that are evidence based from arguments without quantitative or qualitative support.

b. Solving Problems i. ****Students should be able to use physical principles explored in a class

to solve problems in successive classes and outside of classroom domains.

ii. Students can analyze a physical situation and extract the relevant information from distracters as part of their problem solving process.

iii. Students should be able estimate solutions of complex problems by applying physically justifiable approximations.

c. Demonstrating Technical Skills i. Students will be able to independently use and interpret results using

electronic devices including digital millimeters, data acquisition devices, and oscilloscopes.

ii. Students can independently design, analyze, and construct simple electrical circuits.

iii. Students can develop computational models to represent physical circumstances and/or solve complex problems with non-analytical solutions.

iv. **Students can independently troubleshoot experimental setups and computational models that produce results that are not consistent with our physical understanding and/or expectations.

v. Students in experimental concentrations will be able to… 1. …properly handle, clean, mount, store sensitive laboratory

components. 2. …build and optimize experimental setups such as optical,

electronic and mechanical systems. 3. … acquire data through interfaced data acquisition systems.

4. … design amplification circuits when confronted with weak signals or current limited outputs.

d. Devising experiments and/or creating models i. Students can design and perform experiments to test physical principles

and/or models. ii. The students will be capable of properly analyzing data obtained from

their experiment/model iii. Students can develop analytical models using physical principles to

describe the expected or measured outcome of an experiment. iv. The students will form evidence based conclusions from their analysis of

their experiment/model? 2. Communicate with appropriate audience in a variety of modes

a. Students will express physical principles, results, and/or conclusions in oral presentations, poster presentations, and written papers to an audience of their peers and faculty.

b. Students will express physical principles, results and/or conclusions in oral presentations and poster sessions in a manner that is comprehensible to an audience with less technical background.

c. Students will critique oral presentations, poster presentations and written papers presented by their peers.

3. Scientific Ethics a. The students recognize when it is acceptable to disregard a measured value and

when it cannot be disregarded. b. The students will properly cite other investigator’s work and will only include

authorship of work to those people who made significant contributions. c. The students will learn of the importance of peer review towards the scientific

practice. This includes understanding that information obtained through peer review is confidential material and cannot be used for competitive gain (until published and then only what is published).

4. Physics Content

a. The students demonstrate basic understanding of Newtonian mechanics at an appropriate level.

b. The students demonstrate basic understanding of Electricity and Magnetism at an appropriate level

c. The students demonstrate basic understanding of modern physics topics at an appropriate level

d. The students demonstrate basic understanding of thermodynamics and statistical mechanics at an appropriate level.

e. The students demonstrate basic understanding of quantum mechanics at an appropriate level.

f. The students demonstrate basic understanding of wave phenomena and optics at an appropriate level.

Section 2: Curricular Maps A. Map of Programmatic SLO’s to Baccalaureate Framework (Appendix D, Section II)

Goals BF1 - Acquisition

BF2 - Application

BF3 - Values

BF4 - Community

BF5 – Critical Thinking

BF6- Communicate

1. Thinking like a Physicist x x x x

2. Communicate x x 3. Scientific Ethics x x

B. Map of Programmatic SLO’s to Identified “core courses” in the curriculum (Appendix D, Section III)

Course Thinking Like a Physicist Communicate Scientific Ethics

a.i

a.ii

b.i

b.ii

b.iii

c.i

c.ii

c.iii c.iv

c.v.1

c.v.2

c.v.3

c.v.4

d.i

d.ii

d.iii

d.iv a b c a b c

152 - Phys I 2 2 1 2 2 L 1 L L 1 1 2 1 2 2 (L) 1 1

251 - Phys II 3 3 3 3 2 2 1 1C 1 (L) 1 1 2 1 2 C 1 1 2 (L) 1 1

305 - Math Meth. 2 2 2 1 1 1 C 1 1 2 E 2 2 C 1 1 1

310 - Class Mech. 2 2 2 2 2 2 C 1 1 E 1 2 E 2 2 C 1 1 1

312 - E & M I 2 2 2 2 2 2 C 1 1 E 1 2 E 2 2 C 1 1 1

322 - Optics 2 2 2 2 2 2 c 1 1 e 1 2 e 2 2 1 1 1

325 - Computational 2 2 2 2 2 2 2 1 e 1 e 1 1 1

342 - Modern 2 2 2 2 2 2 c 2 1 e 1 e 1 1 C 1 1 1

343 - Modern Lab 3 3 3 3 2 3 2 1 1 1 1 1 3 3 3 2 3 2 2 3 2 2

345 - Optics Lab 2 2 1 2 1 2 1 1 1 1 1 2 2 2 1 1 1 1 2 1 1

346 - Advanced Lab 3 3 3 3 2 3 3 2 2 2 2 2 1 3 3 3 2 3 3 3 3 3 3

361 - Electronics 2 2 2 2 1 2 2 1 c 1 1 1 1 2 2 2 2 c 1 1 1

413 - E & M II 2 2 2 2 2 2 c 1 1 e 1 2 e 2 2 c 1 1 1

418 - Thermo 3 3 3 3 3 1 1 3 c 1 1 e 2 3 e 3 3 c 1 1 2 2 1

442 - Quantum 3 3 3 3 3 1 1 3 c 1 1 e 2 3 e 3 2 c 1 1 2 2 1

480 - Capstone 3 2 3 3 2 2 2 2 2 1 2 1 1 2 2 2 2 3 3 3 3 3 2

There were three raters. If a rater did not know, they left it blank. Ideally, everyone in the department would have completed this, but that did not happen yet.

L – completed in laboratory, C – completed through computational project, E completed through experimental project. Every upper level class must have a computational and an experimental project which the students must communicate about in some form or other.

Section 3: Assessment Plan A. Description of Department’s Assessment Model – How is the department assessing student

progress to Programmatic SLO at key common points in matriculation to degree (Appendix D, Section IV)

The physics department is supposed to use multiple measures to assess student progress. The data that we nominally collect is:

1. the Force Concept Inventory standardized test in the first semester physics classes, 2. the DIRECT and the Concept Survey of Electricity and Magnetism standardized tests in the

second semester introductory physics classes; 3. artifacts consisting of student computational and experimental projects and associated

papers; 4. senior thesis proposal presentations, senior thesis final presentation, senior thesis paper;

and 5. senior exit interview. 6. faculty discussion of our observations of students (we have 50 ish students, it is not too

difficult to keep track of them all. 7. student survey on completion of course outcomes in each course.

B. Measures Used (Appendix D, Section IV)

We relied mostly on two evaluators reading the papers by all physics majors in the modern physics course, the faculty discussion, the senior thesis and senior exit interviews.


Rubrics for the measurements have not yet been developed. It is something we are working on. At present, we rely heavily on discussion of what we see. There is a great deal of agreement between the raters.

D. Description of Plan for Disseminating and Using Findings for Programmatic Learning Improvement The physics program is always under development. Every time we talk about what the students are doing, or what we feel we are failing at, we endeavor to find a way to correct the situation.

Section 4: Assessment Results A. Current Year Assessment Findings (Appendix D, Section V)

The current year findings are not surprising. They are based upon 1) our readings of the student computational and experimental reports; 2) Student senior thesis presentations; 3) Faculty discussions.

First, the student communication skills are not where we would like them to be. The writing is mostly a “book-report” format. This was from two readers on student project papers from our Modern Physics class, which is at the end of a student’s sophomore year. While we did not have a rubric for rating the paper, the readers were in agreement. The first group of senior theses (4 in total) were not as good a data source as we might have hoped. There were issues with students doing what the faculty considered an acceptable level of work and not necessarily taking the course as seriously as desired. The department discussions about the program identified a number of issues which need to be addressed.

1. The classroom experience is vastly different depending upon who teaches the class. This is especially important when considering introductory classes. The laboratory experiences are also inconsistent.

2. The computational and experimental projects are not being taken as seriously as they should be by faculty nor are they being used to reinforce proper communication skills.

3. There is significant concern that students get overwhelmed by the number of projects. 4. We need rubrics for rating student papers 5. We need to create a collection of possible projects with standardized formats 6. Students need to have greater guidance about how to complete and write their projects. 7. Course outcomes are not being put on the syllabus nor is there much effort to survey the

students about whether outcomes are being met. 8. Senior thesis is not being taken as seriously as it should be by students. 9. From the curriculum map, we can see that we are putting too many demands on the

laboratory classes which not every student takes. In particular the issues of scientific ethics and communication. There are also issues on acquisition of technical skills not necessarily being completed in the laboratories as they should.

B. Proposed Changes to Address Findings (Appendix D, Section V)

Addressing each of the points listed above:

1) We need to work more closely to develop a set of effective laboratory and course practices. This is not without challenges because some of the faculty are deeply set in their ways and will likely not be willing to change. Laboratories need to updated and be brought into a more consistent format.

2) Departmentally, we need to create the policy that projects must be included in every course and that there needs to be sufficient time for the student work to be revised on comment.

3) To address this we need to make certain the projects are listed at the beginning of the semester on the syllabus so that students are aware of the projects from the beginning.

4) We will create a committee to create appropriate rubrics. 5) We will create a repository for all of the projects created by different faculty. From this

we can start to assemble a standardized format for the projects.

6) When we create the repository, we will also create a better framework to guide students to learning appropriate writing style.

7) Course outcomes must be on every syllabus. Department policy. 8) Currently the Senior Thesis course is designed as 3 credits in the first semester and 0

credits in the second semester. The courses are pass fail only. We felt that being pass fail meant resulted in some students being lackadaisical compared with others, and no way to reward the harder working students, or to really punish the students who worked less hard. To make the students take this with the requisite seriousness, we will change the courses to 2 hours the first semester with a grade and 1 credit the second semester also with a grade.

9) This will be addressed, in part, by the formalization of the class projects. However, it also means that advanced laboratories need to be revised to fulfill the desired outcomes.

C. Prior Year Assessment Findings and Description of Changes Made (Appendix D, Section IV)

Last year, we did not really make any findings because we were in the usual cycle of revising our assessment plan, which we continue to revise.

D. Assessment Findings for Curricular Changes Made Appendix D, Section V)

Nope.

Section 5: Conclusions, Next Steps, and Communication (Appendix D, Section V and Section VI) I absolutely hate writing these reports, but I see the value in doing it. I apologize for not being timelier. Maybe next year. Hope springs eternal.

Psychology Assessment 1

Assessment Report 2014-15 Bachelor’s Degree in Psychology

I. Student Learning Outcomes for the Program 1. Students will demonstrate knowledge of the major theoretical approaches,

findings, and historical trends in psychology. 2. Students will demonstrate the ability to understand the major research methods in

psychology, including design, data analysis, and interpretation. 3. Students will demonstrate the ability to think critically and to use the scientific

approach to understanding behavior. 4. Students will demonstrate an understanding of the values that are the

underpinnings of psychology as a science (e.g., value of empirical evidence, tolerance for ambiguity, ethics in research, teaching, and practice).

5. Students will demonstrate the ability to apply concepts, information, and skills learned in psychology courses to their lives and work.

6. Students will demonstrate the ability to effectively locate and evaluate sources of information.

7. Students will demonstrate the ability to express themselves effectively in the discourse of the discipline.

8. Students will demonstrate the ability to understand people from a diverse range of backgrounds and varying demographic characteristics such as age, race, disability, sexual orientation, class, ethnicity, religion, and cognitive abilities.

9. Students will demonstrate the ability to make decisions about future employment or graduate education.

II. Curricular Maps

A. Map of Programmatic SLO’s to Baccalaureate Framework

Baccalaureate Framework Element PSY BA Learning Objectives

Acquisition of Knowledge: Students will demonstrate

breadth of knowledge across disciplines and depth of

knowledge in their chosen discipline. In order to do

so, students must demonstrate the requisite

information seeking skills and technological

competencies.

1. Students will demonstrate knowledge of the major theoretical approaches, findings, and historical trends in psychology. 2. Students will demonstrate the ability to understand

and use the major research methods in psychology,

including design, data analysis, and interpretation.

Application of Knowledge: Students will demonstrate

the ability to integrate and apply that knowledge, and,

in so doing, demonstrate the skills necessary for life‐

long learning.

5. Students will demonstrate the ability to apply concepts, information, and skills learned in psychology courses to their lives and work. 6. Students will demonstrate the ability to effectively

locate and evaluate sources of information.

Personal and Professional Values: Students will

demonstrate the highest levels of personal integrity

and professional ethics.

4. Students will demonstrate an understanding of the values that are the underpinnings of psychology as a science (e.g., value of empirical evidence, tolerance for ambiguity, ethics in research, teaching, and practice)


A Sense of Community: Students will demonstrate the

knowledge and skills necessary to be productive and

responsible citizens and leaders in local, regional,

national, and international communities. In so doing,

students will demonstrate a commitment to free and

open inquiry and mutual respect across multiple

cultures and perspectives.

8. Students will demonstrate the ability to understand

people from a diverse range of backgrounds and

varying demographic characteristics such as age, race,

disability, sexual orientation, class, ethnicity, religion,

and cognitive abilities.

Critical Thinking and Problem Solving: Students will

demonstrate facility and adaptability in their approach

to problem solving. In so doing, students will

demonstrate critical thinking abilities and familiarity

with quantitative and qualitative reasoning.

3. Students will demonstrate the ability to think critically and to use the scientific approach to understanding behavior. 2. Students will demonstrate the ability to understand

and use the major research methods in psychology,

including ethical standards, design, data analysis, and

interpretation.

Communication: Students will demonstrate the

written, oral, and multimedia skills necessary to

communicate effectively in diverse settings.

7. Students will demonstrate the ability to express

themselves effectively in the discourse of the

discipline.

B. Map of Programmatic SLO’s to Core Courses in the Curriculum

Course 1

Knowlge

2

Method

3

Think

4

Values

5

Apply

6

Locate

7

Express

8

Diversty

9

Career

120 X X X X

140 X X X X

201 X X

203 X X X X X

235 X X X

240 X X X

314 X X X X X X X

329 X X X X X X X

350 X X X X

416 X X X X X X

420 X X X X X X

441 X X X X X X X

490 X X X X X X X X

540 X X X X X


III. Assessment Plan

Program Learning Objectives

1. Knowledge of the major theoretical approaches, findings, and historical trends in psychology

Major Field Test

2. Ability to understand and use the major research methods in psychology, including ethical standards, design, data analysis, and interpretation”

Major Field Test, Senior Papers

3. Ability to think critically and to use the scientific approach to understanding behavior

Senior Papers

4. Students will demonstrate an understanding of the values that are the underpinnings of psychology as a science (e.g., value of empirical evidence, tolerance for ambiguity, ethics in research, teaching, and practice)

Psychological Reasoning Test (under development)

5. Ability to apply concepts, information, and skills learned in psychology courses to their lives and work

Survey of Majors, Alumni Survey

6. Ability to effectively locate and evaluate sources of information Senior Papers

7. Ability to express themselves effectively in the discourse of the discipline

Senior Papers

8. Ability to understand people from a diverse range of backgrounds and varying demographic characteristics such as age, race, disability, sexual orientation, class, ethnicity, religion, and cognitive abilities


9. Ability to make decisions about future employment or graduate education


A. Description of Assessment Model

The Psychology Department spreads the administration and analysis of assessment measure over a four-year period. The measures include an exit exam, assessment of writing, assessment of critical thinking (psychological reasoning), and a survey of current majors (and of alumni at the time of program review). Two of the measures are administered at all or almost all levels of the major (Survey of Majors; Psychological Reasoning Test), two are administered at or near the time of graduation (Senior Papers; Major Field Test), and one is administered after graduation (Alumni Survey). The learning objectives assessed by each of the measures are shown in the table above.

B. Measures i. Major Field Test in Psychology (Measures learning objectives 1 and 2)

The Major Field Test (MFT) in Psychology is a nationally-normed test measuring content knowledge in the discipline of psychology, constructed and scored by the Educational Testing Service (ETS). It generates one overall score, and four subscores: 1. learning & cognition; 2. sensation, perception, physiological, comparative & ethology; 3. clinical, abnormal personality, and 4. developmental


& social. It also provides reports for six assessment indicators, for the purposes of program outcomes assessment: 1. memory and thinking; 2. sensory and physiology; 3. developmental; 4. clinical and abnormal; 5. social; and 6. measurement and methodology. All graduating psychology majors are required to take the Major Field Test at the end of the semester in which they are to graduate. They must score above a certain minimum (2 SD below the national mean on the test) in order to be certified to graduate.

ii. Evaluation of Senior Papers (Measures learning objectives 3, 5, and 6)

All graduating seniors are required to submit research papers from two upper-level courses required in the major, PSY 32900-Psychobiology and PSY 41600-Cognitive Psychology. A sample of these papers is selected randomly from those collected since the last assessment of papers. Papers are evaluated by two independent raters using a rubric developed by the department. Student names are removed and each paper is assigned a number by the department secretary before the papers are given to the raters.

iii. Survey of Majors (Measures learning objectives 5, 8, and 9)

The department conducts a survey of at least 35% of currently enrolled majors every four years and at the time of departmental program review (typically done every seven years). Should the normal four-year cycle for the survey of current students lead to the survey being done two years in a row, it will be postponed one year to match the program review cycle. This survey assesses students’ evaluation of many aspects of their education, and their satisfaction with the program and services (especially advising) provided by the department. The survey is modified each time it is administered in order to gain information needed by the department at that time, but will always include items to measure learning outcomes 5, 8, and 9.

iv. Survey of Alumni (Measures learning objectives 5, 8, and 9)

At the time of departmental program review (typically on a 7-year cycle) the department undertakes a survey of alumni. This survey is constructed to meet departmental needs at the time of program review, in addition to measuring learning objectives 5, 8, and 9. Data from this survey may be included in the department’s assessment report at the time it is conducted.

v. IPFW Psychological Reasoning Test (Measures learning objective 4)

The IPFW Psychological Reasoning Test was developed to measure students’ ability to critically evaluate claims about psychological phenomena on the basis


of the adequacy of evidence presented. Some of the items are from tests of psychological reasoning obtained from other universities and others are “home-grown” items developed by our department. The test was administered for the first time to students in targeted classes at different levels of the major (PSY 140, 203, 329/416) in fall 2013, as well as to students about to graduate at the end of fall 2013 and spring 2014. In the following year, two independent raters evaluated test results with a scoring rubric developed by the department; the ratings are currently under analysis.

C. Rubrics or Evaluation Metrics Descriptions i. Major Field Test (MFT) in Psychology

The MFT-Psychology is administered at 357 institutions nationwide. The mean scores in different content areas for our department are compared with those for all psychology departments in the national sample as well as with departments at comparable institutions. We set our benchmark for mean scores to be in the top half of psychology programs nationally, as this seems reasonable given the variety of institutions and range of admissions selectivity in the national sample, and the top third of psychology programs at institutions comparable to our own.

ii. Evaluation of Senior Papers

A sample of research papers from two upper-level courses, PSY 32900-Psychobiology and PSY 41600-Cognitive Psychology is evaluated using a rubric developed by the department. The rubric has four categories: strength of thesis, integration of evidence, future directions or applications, and scientific writing style. Interrater reliabilities for these categories are .79, .81, .80, and .85, respectively, using the G(q,k) reliability estimator. (This reliability estimator is based on generalizability theory, which proposes that modelling the contribution of rater main effects on variability in scores allows a more accurate estimate of interrater reliability than is possible with traditional measures of interrater reliability, such as Pearson r, that do not account for rater effects.) The rubric is attached at the end of this report.

iii. Survey of Majors

As evidence that our program is achieving success on learning outcomes 5, 8, and 9, we expect to see majority ratings of 4 or 5 on a 5-point scale of perceived excellence or satisfaction on items related to these learning outcomes. We also expect to see ratings increase as class standing increases from the freshmen to senior level.


iv. Survey of Alumni

Again, majority ratings of 4 or 5 on a 5-point scale of perceived excellence or satisfaction on items related to learning outcomes 5, 8, and 9 will be used as evidence of successful attainment.

vi. IPFW Psychological Reasoning Test

This test is still in development, but we will take as evidence of success mean scores that are above the midpoint of the range of possible scores and an increase in scores over the course of the major from PSY 140 to PSY 203 to PSY 329/416 and finally at graduation.

D. Description of Plan for Disseminating and Using Findings for Programmatic

Learning Improvement The report of each year’s assessment findings will be presented to the department as a whole and proposed adjustments to the program will be considered as needed to address any concerns with attainment of specific learning outcomes.

IV. Assessment Results

A. Current Year Assessment Findings Our assessment measure this year was the Survey of Majors, in which we focused on learning objective 9-Ability to make decisions about future employment or graduate education, and to a lesser extent learning objective 5-Ability to apply concepts, information, and skills learned in psychology courses to their lives and work and 8-Ability to understand people from a diverse range of backgrounds and varying demographic characteristics such as age, race, disability, sexual orientation, class, ethnicity, religion, and cognitive abilities. Items on the survey asked about effectiveness of specific department efforts to promote career planning, perceptions of attainment of skills and characteristics that employers desire, perceptions of acquisition of knowledge of different areas of human behavior, and acquisition of skills foundational to an education at IPFW. The survey was taken by 124 psychology majors at the end of the spring semester of 2015 (total number of majors was approximately 400), but for unknown reasons, 17 participants did not advance past a certain point near the beginning of the survey. As shown in the following tables, students in our program believe that they are effectively attaining learning objective 8 (ability to understand a diversity of people).


The two items on this objective have mean ratings of 4 or above on the 5-point scale, and a clear majority of ratings are either 4 or 5. There is a noticeable increase in ratings of these items between freshmen and sophomores and thereafter, the ratings remain relatively high (note, however, that there are only 4 freshmen in this analysis). Similarly, students in our program perceive high attainment of learning objective 5 (application of psychological concepts to work and lives), with mean ratings slightly above 4 and the majority of ratings either 4 or 5. For these items, there is a steady increase in ratings from the freshman to senior level. Mean ratings of items related to learning objective 9 (ability to make plans for future professional lives) tend to be more clustered around the midpoint of the 5-point scale, and ratings of 4 or 5 are generally not in the majority. Only ¼ of the participants had attended one of the department’s informational seminars on graduate school and even fewer, 1/6, had attended our seminar on bachelor’s level careers, but perceptions of those who had attended either of these were on average positive. The least favorable ratings are for the extent to which career plans had been discussed at an advising appointment, and although mean ratings of this item increase from the freshman to senior level, they remain below the midpoint of the scale even for seniors.


Percent of ratings at bottom, middle, and top of 5-point rating scale and mean ratings for individual items.

Survey Item Learning Objective n % Bottom

Ratings-1 or 2 % Middle Ratings-3

% Top Ratings-4 or 5

Mean Rating (SD)

To what extent do you perceive that the Psychology program has increased your awareness of how differences in backgrounds and identities of individuals can affect their values, beliefs, and behaviors?

8 124 5.6% 12.9% 81.5% 4.23 (.99)

Rate the department’s efforts to address the needs of students with diverse backgrounds and identities.

8 124 5.6% 16.9% 77.4% 4.00 (.88)

To what extent have you discussed career plans during an academic advising appointment?

9 124 37.9% 23.4% 38.7% 3.03 (1.28)

Have you attended a graduate school seminar put on by the Psychology Department?

9 Yes = 34 No = 90

To what extent did the graduate school seminar you attended increase your knowledge of future educational options?

9 34 1470% 32.4% 52.9% 4.29 (.94)

Have you attended a career seminar put on by the Psychology Department?

9 Yes = 22 No = 102

To what extent did the career seminar you attended increase your knowledge of career options?

9 17 29.4% 41.2% 29.4% 3.94 (.90)

To what extent do you have a clear plan for your career/educational future after graduation?

9 107 23.4% 20.6% 56.1% 3.49 (1.18)

To what extent has the psychology program contributed to your development of a plan for the future?

9 107 14% 27.1% 58.8% 3.63 (1.08)

Have you considered doing (or actually done) a field placement (e.g., PSY 480) as a means of exploring career interests?

9 107 Taken = 20 Considering=51 Not familiar =9 Not interested = 27

To what extent have your psychology classes


contributed to your development of the skills and characteristics below that are desired by employers…

Communication skills (writing, public speaking, social skills).

7 107 9.4% 15.9% 74.8% 3.93 (.96)

Motivation/influence (commitment to lifelong learning, ability to meet challenges).

107 9.4% 15.0% 75.7% 3.99 (.99)

Teamwork skills (ability to work with others).

107 10.3% 25.2% 64.5% 3.79 (.99)

Leadership abilities (ability to lead groups, communicate, work effectively with others).

107 15.9% 16.8% 67.3% 3.72 (1.03)

GPA/academic skills (broad education, good grades).

107 4.6% 16.8% 78.5% 4.11 (.88)

Interpersonal skills (interacts well with others, has social skills).

107 7.4% 16.8% 75.7% 4.00 (.92)

Flexibility/adaptability (can cope with change, tolerate stress).

107 8.4% 16.8% 74.8% 4.03 (.97)

Technical skills (computer skills, internet skills).

107 15.9% 39.3% 44.9% 3.45 (1.03)

Honesty/integrity (ethics, personal responsibility)

107 9.4% 15.9% 74.8% 3.98 (.99)

Analytic skills/problem-solving skills (reasoning and thinking abilities).

3 107 4.7% 14.0% 81.3% 4.21 (.86)

Ability to write proposals and reports.

7 107 14.0% 16.8% 69.2% 3.91 (1.15)

Ability to identify problems and suggest solutions based on research findings and knowledge of psychology.

3and 5 107 5.6% 16.8% 77.6% 4.07 (.93)

Ability to conduct interviews.

107 30.9% 27.1% 42.1% 3.16 (1.28)

Ability to do statistical analyses.

2 107 16.8% 14.0% 69.1% 3.79 (1.18)

Ability to design and carry out research projects

2 107 10.3% 19.6% 70.0% 3.92 (1.10)

To what extent have your psychology classes contributed to your knowledge of the following areas of human behavior…

Knowledge of attitude formation and change.

1 107 7.4% 16.8% 75.7% 4.03 (.94)


Knowledge of how people think, solve problems, and process information.

1 107 4.7% 9.3% 86.0% 4.25 (.81)

Knowledge of group dynamics and structure.

1 107 9.4% 15.9% 74.7% 4.05 (1.02)

Knowledge of how the physical environment influences our feelings and actions.

1 107 4.6% 8.4% 87.0% 4.26 (.84)

To what extent have your psychology classes to date contributed to your development of the skills below considered to be foundational to an education at IPFW?

Application of Knowledge (integration and application of knowledge in psychology, and skills for life-long learning).

5 107 4.7% 16.8% 78.5% 4.15 (.93)

Personal and Professional Values (personal integrity and professional ethics).

4 107 8.4% 15.0% 76.7% 4.06 (.99)

Sense of Community (knowledge and skills to be a productive and responsible citizen and a leader in the local community and beyond; a commitment to free and open inquiry and mutual respect across multiple cultures and perspectives).

107 13.1% 26.2% 60.7% 3.72 (1.06)

Critical Thinking and Problem (facility and adaptability in problem solving; critical-thinking abilities and familiarity with quantitative and qualitative reasoning).

3 107 6.5% 13.1% 80.4% 4.18 (.93)

Communication (written, oral, and multimedia skills necessary to communicate effectively in diverse settings).

7 107 3.7% 18.7% 77.5% 4.14 (.88)

Acquisition of Knowledge (depth of knowledge in psychology and information-seeking skills).

1 107 3.8% 15.9% 80.3% 4.27 (.93)


Mean ratings (and standard deviations) for selected items that assess learning objectives 5, 8, and 9 as a function of class standing.


To increase students’ perceptions regarding the development of their career plans, we plan to establish as a requirement for graduation that students meet at least once during their senior year with a career counselor at IPFW’s Career Services. They may work with the career counselor on any aspect of career development (e.g., exploration of career interests, help with a resume, graduate school application, or interview skills). Department faculty will meet with Career Services staff to develop an understanding of the nature of these career appointments for our students. In addition, we have submitted to our college a proposal for a .5 professional advisor in psychology for freshman and sophomore majors so that faculty can focus their advising efforts on juniors and seniors, who are at the point when advising related to career planning becomes more critical. The

Survey Item Learning Objective

Freshmen n = 4

Sophomores n = 13

Juniors n = 22

Seniors n = 68

To what extent do you perceive that the Psychology program has increased your awareness of how differences in backgrounds and identities of individuals can affect their values, beliefs, and behaviors?

8 2.50 (1.29)

4.31 (.75)

4.50 (.67)

4.32 (.87)

Rate the department’s efforts to address the needs of students with diverse backgrounds and identities.

8 2.75 (.50)

4.23 (.73)

4.00 (.62)

4.10 (.87)

To what extent have you discussed career plans during an academic advising appointment?

9 2.25 (.50)

2.69 (1.11)

2.82 (1.37)

3.25 (1.30)

To what extent do you have a clear plan for your career/educational future after graduation?

9 2.75 (1.71)

3.54 (1.05)

3.68 (1.13)

3.46 (1.19)

To what extent has the psychology program contributed to your development of a plan for the future?

9 2.50 (1.00)

4.00 (.91)

3.73 (.94)

3.59 (1.12)

To what extent have your psychology classes contributed to your development of ability to identify problems and suggest solutions based on research findings and knowledge of psychology.

3and 5 3.00 (.82)

3.77 (1.30)

3.95 (.95)

4.24 (.79)

To what extent have your psychology classes to date contributed to your development of knowledge (integration and application of knowledge in psychology, and skills for life-long learning).

5 3.00 (1.63)

3.85 (.90)

4.23 (1.02)

4.25 (.82)


department also may consider ways to facilitate career-related presentations by alumni employed in the local community, perhaps hosted by our student organization, Psi Chi.

C. Prior Year Assessment Findings and Description of Changes Made Changes were not needed based on last year’s assessment findings as our benchmarks were met.

D. Assessment Findings for Curricular Changes Made NA

V. Conclusions, Next Steps, and Communication This year’s assessment findings suggest that our program is effectively meeting learning objectives related to understanding of the diversity of individuals and application of psychological concepts to work and lives. However, the learning objective related to the ability to make decisions about future careers is not being attained at the level we would like. This report will be presented and discussed at a department meeting. In spring of 2016, we plan to implement the requirement for graduating seniors to meet with a Career Services counselor. We also plan to complete development of our new measure of psychological critical thinking and analyze data collected to date by next fall.

Submitted by Carol Lawton, Chair, Psychology


IPFW DEPARTMENT OF PSYCHOLOGY‐WRITING RUBRIC CODEBOOK

CRITERION A – STRENGTH OF THESIS

Literature review papers will always have a topic, or subject matter, but a topic or subject is NOT the same thing as a “thesis.” The thesis of a paper articulates what the author wishes to convey about the topic through the literature being reviewed in the paper—it might be thought of as the paper’s “central theme” or purpose. A thesis is more specific than a topic, and serves to organize the paper, and set the reader’s expectations for what the remainder of the paper will demonstrate or investigate. The thesis should be evident before the literature review begins. (Note that citations used to support the introduction of the topic would not constitute the beginning of the literature review.) 0 – The paper does not have a thesis; while the topic of the paper may be clear, there is no central theme or main idea regarding the topic that makes clear to the reader what the literature to be reviewed will demonstrate to the reader. If the reader must infer what the thesis is, or if the review of the literature begins before a thesis is made clear, a code of “0” is to be used.

Example: “Posttraumatic stress disorder was first recognized as a psychiatric disorder when it was introduced into the DSM‐III in 1980. It was considered controversial at the time because it emphasized an outside event rather than a weakness within the individual as a cause of the disorder.” The paper continues with a history of the disorder, followed a description of symptoms and causes.

Discussion of Example: There is no statement upfront of the purpose of the literature review; the reader wonders what issue is being addressed with this description of a disorder

1 – The paper seems to have a thesis that appears early in the paper, but it is somehow unclear. The reader is left with only a vague sense of what the paper is going to show (see Example 1) or the thesis statement is too broad (see Example 2).

Example 1: “The purpose of this paper was to look at journal articles that used a similar task to study memory in order to examine certain brain processes.” The opening paragraph goes on to describe a model of working memory and subsequent paragraphs describe tasks used to study different components of working memory.

Discussion of Example 1: There is a statement of the purpose of the literature review but it is so vague that the reader has little idea of the specific issue to be addressed (e.g., nature of the brain processes underlying a particular component of working memory). The vagueness could have been avoided if the author had specified which memory task and which brain processes were to be reviewed.

Example 2: “The purpose of this paper is to discover a greater understanding of the biology of motivation in the brain regarding various brain structures and neurotransmitters, and that this information can be helpful in future research on this topic.”


Discussion of Example 2: There is no way to do a focused literature on such a broad topic; the topic is more appropriate for a textbook.

Example 3: “There are several factors that may affect depression such as genetic predisposition, abnormal neurotransmitter levels in the brain, and an increased production of cortisol.”

Discussion of Example 3: Here, the elements of the thesis are specified (not vague) and any one by itself might set the stage for a focused review. However, there is such a large body of research on each of the three elements specified that it would be virtually impossible to have a focused review that unites all elements.

2 – The paper has a thesis that is conveyed early in the paper (i.e., in the first few paragraphs), and gives the reader a clear sense of what the upcoming literature review will demonstrate. Ideally, the thesis of a literature review makes some evaluative statement or assertion regarding the body of literature that is to be reviewed—however, this ideal element is not required for a code of “2” to be assigned. Note that the writer may make their thesis‐‐the point of their paper‐‐ very clear without necessarily having to phrase it as "the purpose of this paper is..." or "the thesis of this paper is..." In this case, the writer should go on to support/develop the idea; it should clearly be the "backbone" of the paper.

Example: “This paper will review evidence for the idea that individuals have different preferences for ways of processing information, known as learning styles, and that these preferences should be taken into account by educators to maximize learning by their students.”

Discussion of Example: This statement clearly specifies the issue that the paper seeks to address. Note: An even stronger example, based on the one above would be to add an assertion about the literature to be reviewed: “This review will show that this popular notion is lacking empirical support.”

CRITERION D – INTEGRATION/SYNTHESIS OF EVIDENCE

The purpose of a literature review is to make some substantive or meaningful point(s)

or argument(s) or assertion(s) about a topic, supported with evidence‐‐i.e., findings

from the literature review. The manner in which the author presents their evidence

ought to reflect critical, objective, scientific thinking, as well as an ability to pull

together, synthesize, and/or integrate research findings and conceptual arguments.

Where inconsistencies are found in the literature, the author should point them out,

and if appropriate (i.e., supported by past research) should offer possible explanations

for their existence that can be traced to the evidence on the table. This synthesis or

evaluation may take place in a separate section from the summaries of individual

works' findings, e.g., near the end of the literature review after presentation of all the

evidence, or ideally, it may be directly integrated with the summaries as the literature

review proceeds.


0 ‐ Previous works are presented in a "laundry list" type fashion (i.e., study by study or source by

source), WITHOUT ANY connections being drawn between any of the sources.

Example: Marks and Johnson (2012) studied the relation between exercise and experienced stress in a

sample of community‐dwelling older adults. Using survey methods, they found…..

Franklin and Thompson (2004), using a sample of 367 college students and an experimental approach

determined that exercise of at least 30 minutes’ duration causes a drop in blood pressure, over a six‐

month period….

Jetts and Manning (2007) look at the relations among age, employment status and exercise in women

over 50….

Discussion of Example: Essentially, each piece of literature is presented (and may be done so quite

well), but there’s no attempt to relate one study to another.

1 ‐ Attempts to make connections between sources under review are inconsistent or unclear, or

occur without making some larger, meaningful point. The reader is left hanging insofar as what to

make of the connections being drawn. A “1” might also be used if the connections made are

superficial, or the larger point the author is trying to make seems superficial.

Example: “Both studies were similar in that they looked at the arousal effect. But there is also a

difference between the two, the study by Roberts and Feld (2009) looked at neurological changes from

using brain games. Another similarity between the two studies was they both used brain‐training

games to see if they affected people’s cognitive skills. A big difference between the two studies was

the fact that the participants in one study were college students and in the other study they used

elderly participants. Also, both studies had the same rationale for their hypotheses.”

Discussion of Example: The connections here are not used to make larger points and so have no

meaning. The only meaningful connection is found in the last paragraph of the paper where the

author states that both studies found that brain training results in improvement on a specific task but

there is no transfer to new tasks.

2 ‐ Meaningful connections are consistently made among previous works (again, ideally occurring

throughout the paper)‐‐whether those connections be the ways in which sources are similar or

different (e.g., in scope, sampling, methodology, findings). These connections allow the author to

make larger, summary points/statements, which might be (but not necessarily) the conclusions. .

The reader can see why these connections are being made, without having to draw his/her own

inferences. (When the integration/synthesis is presented at or near the end of the paper, to

warrant a code of “2,” it will need to be well‐fleshed out, tying in the majority of the sources that

have been reviewed.)

Example: “Although both Strayer and Johnston (2001) and Shutko and Tijerina (2011) rendered similar

deficits in driving performance due to distraction, they had different explanations for their findings.


Strayer and Johnston contributed the results to diversion of cognitive resources away from the driving

task while Shtko and Tijerina contributed the findings primarily to looking away from the road rather

than cognitive distraction. Both explanations, however, agree that attentional factors are key to

understanding the effects of distracted driving.”

Discussion of Example: The author compares and contrasts the findings and conclusions of two

studies to make a larger point. Note that this example suffers from awkward ("rendered") and

inappropriate ("contributed the results") wording. This weakness would be picked up in the coding of

Criterion H later on‐‐the issue of integration/synthesis here is very good, despite the awkward use of

words.

CRITERION F ‐‐ CONSIDERATION OF FUTURE DIRECTIONS/APPLICATIONS

A goal of scientific writing is to point to new directions for pursuit of basic knowledge

on the topic or practical applications of the knowledge gained thus far. Suggestions for

future research or possible applications should be clearly tied to the focus of the

literature review, and stem from the evidence presented. Addressing both future

directions and applications is not necessary.

0 – Neither future directions nor practical applications are present or those that are presented are

not meaningful: not tied to any specific findings reviewed or so garbled as to not be understood.

1 – Future directions and/or practical applications are discussed, but the ties to the findings

reviewed are weak or unclear, or the future directions and/or applications themselves are stated

incompletely. A “1” should also be used if the future directions and/or applications are vague; that

is, they offer little in the way of concrete suggestions beyond the obvious.

Example 1: “Future research should examine whether there are cultural differences in the effects of

distributed practice for learning new material. Teachers could use this knowledge to change the

methods they use with different groups of students and improve their learning.”

Discussion of Example 1: The future direction is relatively clear but it is not well tied to the previous

literature review. The review did not present any evidence for group differences in effects of

distributed practice; rather, it focused on the effectiveness of distributed practice for learning

different types of information.

Example 2: “Although it is commonly assumed that women and men differ in the way that they solve

problems, the articles reviewed in this paper do not support this view. When third variables are taken


into account, gender differences are reduced or eliminated. Gender differences in problem solving may

in fact be diminishing over time. Problem solving should remain a significant focus of research because

it is fundamental to human cognition.”

Discussion of Example 2: The conclusion too general and there is no indication of what the next steps

in this area of research would be, in light of the evidence that was presented in the review.

2 ‐ Future directions for continued research and/or practical applications are presented clearly and

explicitly, and the reader can easily see how or why the author would suggest these directions or

real‐world applications because they seem natural outgrowths of the findings presented in the

preceding literature review. Furthermore, the future directions and/or practical applications reflect

some originality of thought on the part of the author—that is, they go beyond simply reiterating or

summarizing the main points of the literature review.

Example: “Future research on prosocial videogames could examine some of the factors known to

affect helping behavior in real‐world situations, such as the similarity between helper and person

needing help. Prosocial videogames targeted at a specific age group or gender might increase

empathy more if they depict characters who are similar in age or gender to the player than if they

depict characters who are very different from the player. It is important to continue research in this

area in order to maximize the positive impact of prosocial media.”

Discussion of Example: The suggestion for future research relates directly to findings described in the

literature review and the author seeks to extend those findings in a new direction.

CRITERION I – SCIENTIFIC WRITING STYLE

Scientific writing is characterized by a number of features that distinguish it from other styles of

writing. In addition to striving for maximum clarity in expression, the tone of a literature review paper

ought to be formal (as opposed to casual/colloquial, slang) and objective (factual, no personal

anecdotes).The sources of information for ideas presented ought to be documented. The literature

under review should be scholarly and derived from scientific sources. The author should not overly rely

on direct quotations from sources (i.e., “lazy writing”) to make his/her points.

0 – Subjective (i.e., personal, anecdotal, opinion), statements are present, or the use of any non‐

scholarly sources in the literature review itself as a main source or piece of evidence (as opposed to

the use of a non‐scholarly source to illustrate a point or an example, e.g., in the opening paragraph)

would warrant a code of “0.” When the tone of the paper is mostly informal (e.g.,

colloquialisms/slang frequently appear) or numerous factual statements are made without attribution

to a source, a “0” should be assigned. A preponderance of direct quotations (i.e., “lazy writing”)

would warrant a code of “0.”


1 ‐ Occasional use of a non‐formal tone (e.g., a conversational or colloquial tone), or the occasional

appearance of factual statements that lack attribution to their source(s) would be coded as “1.”

Occasional instances of lazy writing warrant a code of “1.” Also, if the tone is formal and objective but

substance is lacking or clarity is lacking (e.g., weak sentence construction that interferes with reader’s

understanding), a “1” should be used. Note: It is sometimes the case in scientific writing that the word

“we” can be used, particularly when discussing findings or drawing conclusions, so it is important to

consider the context in which the word is used.

2 – The paper’s tone is clear, formal, and also objective – that is, points are made logically and

grounded in objective evidence that is ideally (but perhaps not always) attributed to sources that are

scholarly. (Note: Sometimes a student author may fail to provide a citation for information, usually when used to introduce the topic. If it is questionable as to whether that information is common knowledge and if this failure to cite occurs only infrequently, this should not rule out a score of 2.)

1

Department of Philosophy Religious Studies Minor Annual Assessment Report 2015 revised 11/9/15 Orienting Information Program Description Since 2003, the Department of Philosophy has offered an undergraduate minor in religious studies. The description of the minor as found in both the 2015–2016 IPFW Undergraduate Bulletin as well as in associated marketing, advising, and other relevant materials runs as follows:

Transdisciplinary in approach and global in scope, the minor in religious studies provides students with a firm grounding in the academic study of religion through the informed investigation of the phenomenon of religion in diverse historical, social, cultural, and political contexts. Coupling the acquisition of broad knowledge of the world’s religious traditions-past and present-with a critical apprehension of the methods and tools scholars use to study them, the minor in religious studies can serve as an excellent complement to a variety of majors, including anthropology, general studies, history, philosophy, political science, and sociology.

As articulated in the 2015–2016 IPFW Undergraduate Bulletin, the requirements for the minor run as follows:

Students may earn a minor in religious studies by completing 18 credits as distributed below with a grade of C- or better in each course. At least 6 credits must be earned as resident credit at the 200 level or above. Substitutions for these courses may be made with the approval of the department program coordinator. Core Courses Credits: 9 REL 11200 - Religion and Culture Cr. 3. REL 23000 - Religions of the East Cr. 3. REL 23100 - Religions of the West Cr. 3. Religious Traditions Credits: 3 REL 30000 - Religions of the Ancient World Cr. 3. REL 30100 - Islam Cr. 3 REL 30200 - Christianity Cr. 3 REL 30500 - Judaism Cr. 3. REL 30600 - Hinduism Cr. 3. REL 30700 - Buddhism Cr. 3.

2

REL 31100 - African Traditional Philosophy and Religion Cr. 3. Thematic and Comparative Credits: 3 REL 29300 - Topics in Religious Studies Cr. 3. REL 31200 - The Black Religious Experience Cr. 3 REL 31400 - Religion and Violence Cr. 3. REL 31500 - Religion and Women Cr. 3. REL 32100 - Religion and the Civil Rights Movement Cr. 3 REL 32300 - Religion and Popular Culture Cr. 3. REL 38100 - Islam and Modernity Cr. 3 REL 40100 - Studies in Sacred Texts Cr. 3 REL 40200 - Mysticism Cr. 3. REL 49300 - Undergraduate Seminar Cr. 3 REL 49500 - Individual Readings in Religious Studies Cr. 3. Cognate Credits: 3 ANTH E455 - Anthropology of Religion Cr. 3. ANTH E462 - Anthropological Folklore Cr. 3. CLAS C205 - Classical Mythology Cr. 3. CLAS C405 - Comparative Mythology Cr. 3-4. FOLK F252 - Folklore and the Humanities Cr. 3. FOLK F305 - Asian Folklore Cr. 3. FOLK F352 - Native American Folklore Cr. 3. HIST C392 - History of Modern Near East Cr. 3. HIST C393 - Ottoman History Cr. 3. PHIL 20600 - Philosophy of Religion Cr. 3. PHIL 30200 - History of Medieval Philosophy Cr. 3. PHIL 43100 - Contemporary Religious Thought Cr. 3. SOC S313 - Religion and Society Cr. 3. Total Credits: 18 There are presently a total of 25 active students pursuing the minor (52% of which are students in the College of Arts and Sciences) and 18 students who have declared the minor but who are presently inactive. The program is presently supported by two 1.00 FTE tenured faculty members as well as one 1.00 FTE Continuing Lecturer who typically devotes .5 of his teaching load to supporting REL courses each academic year. In keeping with the department’s focus on service teaching in support of various university (general education), college (distribution), and other non-departmental curricula, roughly 94–96% of the total credit hours generated by REL courses are taken by non-minors. The majority of these credits are generated in general education courses, which have their own robust assessment plans, reporting protocols, and such like. As such, it is essential to note that this required program-specific assessment report focuses solely on the program as such. That is, it focuses on students who have completed the minor, the minority, rather than on the experience of those many other students who have taken the core courses, the majority. As to the latter, the relevant general education assessment reports for REL 11200, REL 23000,

3

and REL 23100 are publically available on Vibe and easily accessible to the assessment committee should they wish to view them. It should be stressed, however, that the processes informing the preparation of those reports represent a separate aspect of the program’s present positioning within a wider context of intersecting curricula and, as such, is not reduplicative of the information presented herein. Section 1 Student Learning Outcomes for the Program

1. Students who have completed the minor will demonstrate that they have a critical apprehension of the methods and tools scholars use to study religion.

2. Students who have completed the minor will demonstrate that they have acquired broad knowledge the world’s religious traditions—past and present.

3. Students who have completed the minor will demonstrate that they have successfully engaged in the informed investigation of the phenomenon of religion in diverse historical, social, cultural, and political contexts.

Section 2 Curricular Maps Map of Programmatic Student Learning Outcomes to Baccalaureate Framework REL SLO1 > BF3, BF4, BF6 REL SLO2 > BF1, BF2, BF3, BF4 REL SLO3 > BF2, BF3, BF4, BF5 Map of Programmatic Student Learning Outcomes to Identified “Core Courses” in the Curriculum REL 11200 > REL SLO1 REL 23000 > REL SLO2 REL 23100 > REL SLO2 Section 3 Assessment Plan Description of Department’s Assessment Model Early in the spring 2012 term, a twenty-five multiple choice question “Religious Studies Knowledge Test” was developed by department faculty teaching courses under the REL prefix to assess how well the department’s religious studies program is doing in meeting basic curricular goals for those students who are actively pursuing the undergraduate minor in religious studies. This test assesses how well students have retained key information from the three “core courses” required for the minor, namely REL 11200, REL 23000, and REL 23100, and is made available online through the Purdue Qualtrics web-based survey software.

4

In the context of the regular updating of advising records for students actively pursuing the minor in religious studies which is done at the beginning and the end of each academic term, including the summer session, the religious studies program coordinator identifies those students who have completed the course requirements for the minor. These students are then sent a generic invitation via email asking them to take the knowledge test through the Qualtrics site. The email addresses used are students’ listed IPFW email addresses. The email used runs as follows:

Dear IPFW Religious Studies Minor, As you have recently completed the minor in religious studies, we would like to request that you take 10–15 minutes of your time to complete a brief “religious studies general knowledge” test online. This test is an anonymous instrument used by the religious studies program to assess how well we are communicating basic knowledge in the core courses for the minor. All students who have completed the minor have been invited to take the test. In this test, you will be asked to respond to a total of twenty-five multiple choice questions covering basic knowledge from REL 11200, “Religion and Culture”, REL 23000, “Religions of the West”, and REL 23100, “Religions of the East”. Scores on the test are evaluated only in the aggregate and are intended solely to help us improve our teaching of the core courses and, by extension, the academic quality of the religious studies minor. Please follow the link below my signature line to take the test. Congratulations again on completing the minor! Sincerely, Erik S. Ohlander, Ph.D. Associate Professor of Religious Studies Department of Philosophy

IPFW Measures Used The above mentioned religious studies knowledge test runs as follows: 1. Functionalist definitions of religion tend to define it mainly: a. In terms of how it operates in human life. b. In terms of its historical origin. c. In terms of the ideas that religious people find important. d. In terms of belief in supernatural beings.

5

2. Karl Marx famously defined religion as: a. The opiate of the masses. b. A universal obsessional neurosis. c. The daughter of hope and fear. d. The great fairy tales of conscience. 3. In his Elementary Forms of Religious Life, Emile Durkheim’s primary data set is: a. The practices, rituals, and beliefs of the Ancient Greeks and Romans. b. The tradition of Hasidic Judaism in Eastern Europe. c. The totemic system of the Aborigines of Australia. d. Protestants and Catholics in Western Europe. 4. According to Mircea Eliade, the study of religion should begin with the recognition that: a. Religion absorbs and reflects back the values of different socio-economic classes. b. To understand religion is to understand society, and vice versa. c. Religion is a category sui generis, a phenomenon unique in and of itself. d. To understand religion one must first understand the human psyche. 5. In his theory of religion, Clifford Geertz posits that ethos and worldview: a. Are synthesized primarily through science. b. Are synthesized primarily through ritual. c. Are synthesized primarily through magic. d. Are synthesized primarily through theology. 6. Judaism, Christianity, and Islam are all considered: a. Monotheistic religious traditions. b. Henotheistic religious traditions. c. Polytheistic religious traditions. d. Nontheistic religious traditions. 7. How many books does the Torah contain? a. Four. b. Five. c. Seven. d. Twenty-three. 8. The Jewish holiday of Yom Kippur is most readily associated with: a. Commemorating the rededication of the Temple in Jerusalem. b. Raucous celebrations and street festivals. c. Atoning for and seeking repentance from sins. d. Remembering the story of the Exodus.

6

9. Which of the following is a classic text of Rabbinic Judaism? a. The Zohar. b. The Shivhei Ha-Besht. c. The Kuzari. d. The Babylonian Talmud. 10. The Synoptic Gospels are: a. Matthew, Mark, Luke. b. Mark, Luke, John. c. Matthew, Mark, Thomas. d. Mark, Luke, Peter. 11. The most contentious debates in early Christianity were over matters of: a. Liturgy. b. Christology. c. Ontology. d. Eschatology. 12. The Protestant Reformation took place in which century? a. The fourteenth century. b. The fifteenth century. c. The sixteenth century. d. The seventeenth century. 13. Which of the following is not one of the Five Pillars of Islam: a. Canonical Prayer (salat). b. Fasting during Ramadan (sawm). c. Pilgrimage to Mecca (hajj). d. Marriage (nikah). 14. In Islam, the Hadith refer to: a. The recorded sayings and doings of the Prophet. b. The opening chapter of the Qur’an. c. Acts which are forbidden. d. A mountain near the city of Mecca. 15. The majority of Muslims in the world today are: a. Salafi. b. Sunni. c. Shiite. d. Sufi.

7

16. Which of the following Asian religious traditions has the largest number of adherents? a. Buddhism. b. Hinduism. c. Taoism. d. Shinto. 17. Which of the following concepts is typically not associated with Hinduism? a. Karma. b. Dharma. c. Moksha. d. Nirvana. 18. The popular Hindu scripture known as the Bhagavad Gita is actually part of: a. The Lotus Sutra. b. The Ramayana. c. The Rig Veda. d. The Mahabharata. 19. In Buddhism, which of the following is the first of the Four Noble Truths: a. Suffering ceases when attachment to desire ceases. b. All life is suffering. c. Suffering arises from attachment to desires. d. Freedom from suffering is possible by following the eight-fold path. 20. The two major divisions of Buddhism are: a. Nichiren and Vajrayana. b. Vedanta and Yoga. c. Shaivism and Vaishnavism. d. Theravada and Mahayana. 21. Which of the following concepts is most closely associated with Jainism? a. Nonviolence (ahimsa). b. One divine constant (ek onkar). c. Devotional faith (bhakti). d. Suffering caused by transience (dukkha). 22. Which of the following influential religious teachers is associated with Sikhism? a. Bodhidharma. b. Guru Nanak. c. Lao Tzu. d. Shankara.

8

23. Which of the following concepts is not associated with Confucianism? a. Proper etiquette (li). b. Altruism (ren). c. Emptiness (kong). d. Filial piety (xiao). 24. In Taoist thought, yin and yang are typically symbolized as: a. Dark and Light. b. Land and Sea. c. Good and Evil. d. Knowledge and Ignorance. 25. The kami are associated with which of the following East Asian religious traditions? a. Shinto. b. Confucianism. c. Taoism. d. Chondogyo. Rubrics or Evaluation Metrics Descriptions Only one of the four possible answers for each multiple choice question in the test is correct. An incorrect answer on any question is considered a potential deficiency in the relevant “core course” to which it applies (REL 11200, REL 23000, or REL 23100). Description of Plan for Disseminating and Using Findings for Programmatic Learning Improvement The intention of this assessment mechanism is to use aggregate results gathered from multiple tests over time to identify those knowledge areas in which students might be interpreted as displaying deficiencies so that adjustments in course material(s), teaching strategies, and so forth can be made. The results are gathered from the responses to invitations sent and then analyzed by the program coordinator, after which recommendations have been made to relevant faculty who are responsible for teaching the courses in question. Section 4 Assessment Results Current Year Assessment Findings On 1/4/15 a total of four invitations to take the test were sent to eligible students. One of the four invitees responded and took the test. The response rate was thus 25%. The response ran as follows:

9

Respondent answering correctly: 100%. 1. Functionalist definitions of religion tend to define it mainly: a. In terms of how it operates in human life. b. In terms of its historical origin. c. In terms of the ideas that religious people find important. d. In terms of belief in supernatural beings. Respondent answering correctly: 100%. 2. Karl Marx famously defined religion as: a. The opiate of the masses. b. A universal obsessional neurosis. c. The daughter of hope and fear. d. The great fairy tales of conscience. Respondent answering correctly: 100%. 3. In his Elementary Forms of Religious Life, Emile Durkheim’s primary data set is: a. The practices, rituals, and beliefs of the Ancient Greeks and Romans. b. The tradition of Hasidic Judaism in Eastern Europe. c. The totemic system of the Aborigines of Australia. d. Protestants and Catholics in Western Europe. Respondent answering correctly: 100%. 4. According to Mircea Eliade, the study of religion should begin with the recognition that: a. Religion absorbs and reflects back the values of different socio-economic classes. b. To understand religion is to understand society, and vice versa. c. Religion is a category sui generis, a phenomenon unique in and of itself. d. To understand religion one must first understand the human psyche. Respondent answering correctly: 100%. 5. In his theory of religion, Clifford Geertz posits that ethos and worldview: a. Are synthesized primarily through science. b. Are synthesized primarily through ritual. c. Are synthesized primarily through magic. d. Are synthesized primarily through theology.

10

Respondent answering correctly: 100%. 6. Judaism, Christianity, and Islam are all considered: a. Monotheistic religious traditions. b. Henotheistic religious traditions. c. Polytheistic religious traditions. d. Nontheistic religious traditions. Respondent answering correctly: 100%. 7. How many books does the Torah contain? a. Four. b. Five. c. Seven. d. Twenty-three. Respondent answering correctly: 100%. 8. The Jewish holiday of Yom Kippur is most readily associated with: a. Commemorating the rededication of the Temple in Jerusalem. b. Raucous celebrations and street festivals. c. Atoning for and seeking repentance from sins. d. Remembering the story of the Exodus. Respondent answering correctly: 100%. 9. Which of the following is a classic text of Rabbinic Judaism? a. The Zohar. b. The Shivhei Ha-Besht. c. The Kuzari. d. The Babylonian Talmud. Respondent answering correctly: 100%. 10. The Synoptic Gospels are: a. Matthew, Mark, Luke. b. Mark, Luke, John. c. Matthew, Mark, Thomas. d. Mark, Luke, Peter.

11

Respondent answering correctly: 100%. 11. The most contentious debates in early Christianity were over matters of: a. Liturgy. b. Christology. c. Ontology. d. Eschatology. Respondent answering correctly: 100%. 12. The Protestant Reformation took place in which century? a. The fourteenth century. b. The fifteenth century. c. The sixteenth century. d. The seventeenth century. Respondent answering correctly: 100%. 13. Which of the following is not one of the Five Pillars of Islam: a. Canonical Prayer (salat). b. Fasting during Ramadan (sawm). c. Pilgrimage to Mecca (hajj). d. Marriage (nikah). Respondent answering correctly: 100%. 14. In Islam, the Hadith refer to: a. The recorded sayings and doings of the Prophet. b. The opening chapter of the Qur’an. c. Acts which are forbidden. d. A mountain near the city of Mecca. Respondent answering correctly: 100%. 15. The majority of Muslims in the world today are: a. Salafi. b. Sunni. c. Shiite. d. Sufi.

12

Respondent answering correctly: 100%. 16. Which of the following Asian religious traditions has the largest number of adherents? a. Buddhism. b. Hinduism. c. Taoism. d. Shinto. Respondent answering correctly: 100%. 17. Which of the following concepts is typically not associated with Hinduism? a. Karma. b. Dharma. c. Moksha. d. Nirvana. Respondent answering correctly: 100%. 18. The popular Hindu scripture known as the Bhagavad Gita is actually part of: a. The Lotus Sutra. b. The Ramayana. c. The Rig Veda. d. The Mahabharata. Respondent answering correctly: 100%. 19. In Buddhism, which of the following is the first of the Four Noble Truths: a. Suffering ceases when attachment to desire ceases. b. All life is suffering. c. Suffering arises from attachment to desires. d. Freedom from suffering is possible by following the eight-fold path. Respondent answering correctly: 100%. 20. The two major divisions of Buddhism are: a. Nichiren and Vajrayana. b. Vedanta and Yoga. c. Shaivism and Vaishnavism. d. Theravada and Mahayana.

13

Respondent answering correctly: 100%. 21. Which of the following concepts is most closely associated with Jainism? a. Nonviolence (ahimsa). b. One divine constant (ek onkar). c. Devotional faith (bhakti). d. Suffering caused by transience (dukkha). Respondent answering correctly: 100%. 22. Which of the following influential religious teachers is associated with Sikhism? a. Bodhidharma. b. Guru Nanak. c. Lao Tzu. d. Shankara. Respondent answering correctly: 100%. 23. Which of the following concepts is not associated with Confucianism? a. Proper etiquette (li). b. Altruism (ren). c. Emptiness (kong). d. Filial piety (xiao). Respondent answering correctly: 100%. 24. In Taoist thought, yin and yang are typically symbolized as: a. Dark and Light. b. Land and Sea. c. Good and Evil. d. Knowledge and Ignorance. Respondent answering correctly: 100%. 25. The kami are associated with which of the following East Asian religious traditions? a. Shinto. b. Confucianism. c. Taoism. d. Chondogyo. Proposed Changes to Address Findings Based on the above results, the program director would like to make the following recommendations to faculty teaching REL 11200, REL 23000, and REL 23100:

1. Based on the results, it would appear that student command of core knowledge presented in REL 11200 is more than passable, maintaining parity with the scores

14

noted in the previous deployment of the assessment mechanism (i.e. the annual assessment report for 2014). No changes are recommended.

2. Based on the results, it would appear that student command of core knowledge presented in REL 23000 is more than passable, with the unusually low scores on questions nos. 10–12 witnessed in the previous deployment of the assessment mechanism (i.e. the annual assessment report for 2014) being ameliorated here. No changes are recommended.

3. Based on the results, it would appear that student command of core knowledge presented in REL 23100 is more than passable, with the unusually low score on question no. 18 witnessed in the previous deployment of the assessment mechanism (i.e. the annual assessment report for 2014) being ameliorated here. No changes are recommended.

Prior Year Assessment Findings and Description of Changes Made On 1/6/14 a total of four invitations to take the test were sent to eligible students. The response rate was 0%. On 5/27/14 a total of eight invitations were sent to eligible students. The response rate was 25%. As shown below, the results gathered from the responses gathered from invitations sent have been analyzed by the program coordinator and recommendations made to relevant faculty in conjunction with the preparation and dissemination of the annual assessment report for 2014. By way of reminder, the combined results of all attempts on the “Religious Studies Knowledge Test” which were collected from invitations sent to eligible students for that reporting period run as follow: Respondents answering correctly: 100%. 1. Functionalist definitions of religion tend to define it mainly: a. In terms of how it operates in human life. b. In terms of its historical origin. c. In terms of the ideas that religious people find important. d. In terms of belief in supernatural beings. Respondents answering correctly: 100%. 2. Karl Marx famously defined religion as: a. The opiate of the masses. b. A universal obsessional neurosis. c. The daughter of hope and fear. d. The great fairy tales of conscience.

15

Respondents answering correctly: 100%. 3. In his Elementary Forms of Religious Life, Emile Durkheim’s primary data set is: a. The practices, rituals, and beliefs of the Ancient Greeks and Romans. b. The tradition of Hasidic Judaism in Eastern Europe. c. The totemic system of the Aborigines of Australia. d. Protestants and Catholics in Western Europe. Respondents answering correctly: 100%. 4. According to Mircea Eliade, the study of religion should begin with the recognition that: a. Religion absorbs and reflects back the values of different socio-economic classes. b. To understand religion is to understand society, and vice versa. c. Religion is a category sui generis, a phenomenon unique in and of itself. d. To understand religion one must first understand the human psyche. Respondents answering correctly: 100%. 5. In his theory of religion, Clifford Geertz posits that ethos and worldview: a. Are synthesized primarily through science. b. Are synthesized primarily through ritual. c. Are synthesized primarily through magic. d. Are synthesized primarily through theology. Respondents answering correctly: 100%. 6. Judaism, Christianity, and Islam are all considered: a. Monotheistic religious traditions. b. Henotheistic religious traditions. c. Polytheistic religious traditions. d. Nontheistic religious traditions. Respondents answering correctly: 100%. 7. How many books does the Torah contain? a. Four. b. Five. c. Seven. d. Twenty-three.

16

Respondents answering correctly: 100%. 8. The Jewish holiday of Yom Kippur is most readily associated with: a. Commemorating the rededication of the Temple in Jerusalem. b. Raucous celebrations and street festivals. c. Atoning for and seeking repentance from sins. d. Remembering the story of the Exodus. Respondents answering correctly: 100%. 9. Which of the following is a classic text of Rabbinic Judaism? a. The Zohar. b. The Shivhei Ha-Besht. c. The Kuzari. d. The Babylonian Talmud. Respondents answering correctly: 50%. 10. The Synoptic Gospels are: a. Matthew, Mark, Luke. b. Mark, Luke, John. c. Matthew, Mark, Thomas. d. Mark, Luke, Peter. Respondents answering correctly: 50%. 11. The most contentious debates in early Christianity were over matters of: a. Liturgy. b. Christology. c. Ontology. d. Eschatology. Respondents answering correctly: 50%. 12. The Protestant Reformation took place in which century? a. The fourteenth century. b. The fifteenth century. c. The sixteenth century. d. The seventeenth century.

17

Respondents answering correctly: 100%. 13. Which of the following is not one of the Five Pillars of Islam: a. Canonical Prayer (salat). b. Fasting during Ramadan (sawm). c. Pilgrimage to Mecca (hajj). d. Marriage (nikah). Respondents answering correctly: 100%. 14. In Islam, the Hadith refer to: a. The recorded sayings and doings of the Prophet. b. The opening chapter of the Qur’an. c. Acts which are forbidden. d. A mountain near the city of Mecca. Respondents answering correctly: 100%. 15. The majority of Muslims in the world today are: a. Salafi. b. Sunni. c. Shiite. d. Sufi. Respondents answering correctly: 100%. 16. Which of the following Asian religious traditions has the largest number of adherents? a. Buddhism. b. Hinduism. c. Taoism. d. Shinto. Respondents answering correctly: 100%. 17. Which of the following concepts is typically not associated with Hinduism? a. Karma. b. Dharma. c. Moksha. d. Nirvana.

18

Respondents answering correctly: 50%. 18. The popular Hindu scripture known as the Bhagavad Gita is actually part of: a. The Lotus Sutra. b. The Ramayana. c. The Rig Veda. d. The Mahabharata. Respondents answering correctly: 100%. 19. In Buddhism, which of the following is the first of the Four Noble Truths: a. Suffering ceases when attachment to desire ceases. b. All life is suffering. c. Suffering arises from attachment to desires. d. Freedom from suffering is possible by following the eight-fold path. Respondents answering correctly: 100%. 20. The two major divisions of Buddhism are: a. Nichiren and Vajrayana. b. Vedanta and Yoga. c. Shaivism and Vaishnavism. d. Theravada and Mahayana. Respondents answering correctly: 100%. 21. Which of the following concepts is most closely associated with Jainism? a. Nonviolence (ahimsa). b. One divine constant (ek onkar). c. Devotional faith (bhakti). d. Suffering caused by transience (dukkha). Respondents answering correctly: 100%. 22. Which of the following influential religious teachers is associated with Sikhism? a. Bodhidharma. b. Guru Nanak. c. Lao Tzu. d. Shankara.

19

Respondents answering correctly: 100%. 23. Which of the following concepts is not associated with Confucianism? a. Proper etiquette (li). b. Altruism (ren). c. Emptiness (kong). d. Filial piety (xiao). Respondents answering correctly: 100%. 24. In Taoist thought, yin and yang are typically symbolized as: a. Dark and Light. b. Land and Sea. c. Good and Evil. d. Knowledge and Ignorance. Respondents answering correctly: 100%. 25. The kami are associated with which of the following East Asian religious traditions? a. Shinto. b. Confucianism. c. Taoism. d. Chondogyo. Based on the above results, the program director made the following recommendations to faculty teaching REL 11200, REL 23000, and REL 23100 in conjunction with the required program assessment report which was submitted in 2014:

1. Based on the results, it would appear that student command of core knowledge presented in REL 11200 is more than passable, with the unusually low scores on questions nos. 3 and 4 witnessed in the previous assessment report having been completely ameliorated here.

2. Based on the results, it would appear that student command of core knowledge presented in REL 23000 is more than passable; however questions nos. 10–12 witnessed unusually low scores. As questions 9 and 12 witnessed unusually low scores in the previous assessment report, the instructor of the course should continue to endeavor to reconfirm that the relevant material is effectively addressed in the appropriate place in the course.

3. Based on the results, it would appear that student command of core knowledge presented in REL 23100 is more than passable; however question no. 18 witnessed an unusually low score. At the same time, it should be noted that the scores on question nos. 17, 19 and 23 improved substantially from those noted in the previous assessment report. The instructor of the course should continue to

20

endeavor to reconfirm the relevant material is effectively addressed in the appropriate place in the course.

Relevant faculty were asked to take these recommendations into account when planning their upcoming offering(s) of the course(s) in question. It should be noted that these recommendations took into account the relevant matters adduced in the required assessment report submitted in 2013. Assessment Findings for Curricular Changes Made This information is already embedded in the response to two of the required reporting areas above. See above s.v. “Proposed Changes to Address Findings” as well as s.v. “Prior Year Assessment Findings and Description of Changes Made.” Conclusions, Next Steps, and Communication This information is already embedded in the response to one of the required reporting areas above. See above s.v. “Proposed Changes to Address Findings.” As in previous years, this report will be shared with relevant faculty.

1

Sociology Department Assessment 2015 Report

Index

Section 1: Student Learning Outcomes for the Program ------------------------------------ p. 2

Section 2: Curricular Maps:

Program Assessment as Evaluated by Performance in Core Courses ----------- p. 3

Section 3: Assessment Plan ------------------------------------------------------------------- p. 4

Section 4: Assessment Results ---------------------------------------------------------------- p. 5

Program Assessment for SOC-S351 Social Statistics, Spring 2015 ------------ p. 5

Program Assessment of SOC-S470 Senior Seminar, Fall 2014 ----------------- p. 8

Undergraduate Assessment – Report on Exit Student Survey 2014 ------------- p. 11

Undergraduate Assessment – Report on Student Alumni Survey 2015 --------- p. 16

Section 5: Conclusion -------------------------------------------------------------------------- p. 19

2

Section I: Student Learning Outcomes for the Program

The Sociology Department Assessment Report has three components: upper level

required course assessment, exit survey results for student graduating, and alumni survey results.

Two course assessments (SOC-S 351 and SOC-S 470) indicated that the required courses met the

program objectives assigned. The exit student survey results demonstrated senior students’ high

level of satisfaction with the sociology undergraduate program. The students valued knowledge

and skills that they earned from the sociology program and also acknowledged faculty’s

involvement and guidance through advising/mentoring for their course work and future careers.

However, the students did note areas of improvement related to providing students with: more

diverse course offerings to choose as their electives and more opportunities to participate in

research activities such as data collection and analysis. Similarly, the alumni survey results

showed students’ positive experience through the undergraduate program but also suggested that

the Department should provide guidance for students by clearly showing how sociology would

lead students to their future job opportunities and career after their graduation.

Given the assessment results, the Sociology Undergraduate Program Committee has three

recommendations to improve the sociology program. First, the Department is recommended to

offer more various courses from the existing courses. Second, the Department is recommended

to discuss strategies for developing career awareness. Third, faculty members are encouraged to

continue advising their students through their teaching courses and creating/getting involved

with sociology-related campus events and activities.

To improve the assessment plan, changes in assessment plan are suggested. Because the

changes suggested in the previous assessment (2013 – 2014) helped the Department improve the

3

collection of exit student survey, the Department is suggested to discuss redesigning alumni

questions to improve alumni survey implementation.

Section 2: Curricular Maps

This was the third year that the Department assessed the program by analyzing student

performance in the achievement of program learning objectives that were to be “emphasized” in

the four upper level required courses. Table 1 lists each of the program learning objectives for

each of the required courses.

Table 1

Undergraduate Learning Objectives and Course Responsibilities

S161 S340 S351 S352 S470

Identify Theoretical Perspectives I E R

Engage in Theoretical

Analyses

I E R

Theoretically Interpret Social Issues I E R

Demonstrate Use of Quant / Qual

Methodologies

E

Evaluate Different Research Methods I E R

Interpret Results of Data Gathering E I

Demonstrate Use of Statistical

Techniques

E

Demonstrate use of Statistical Software E I

Critically Evaluate Theoretical

Arguments

I E R

Develop Evidence Based Arguments I R

Critically Evaluate Published Research E E E R

Write a Research Paper R R

Develop Oral Research Report E

4

Demonstrate Mastery of Ethical

Standards Related to Human Research

Subjects

E R

Demonstrate Understandings of ASA

Professional Ethical Standards

I E

I = Introduced, E = Emphasized, R = Reinforced

The four core upper level courses were assessed by the individual faculty using a variety of

instruments. The Department has no standard instruments that are used for each of the required

courses. The individual instructor determines the best way to assess student performance relative

to the emphasized objectives.


Based on the Department’s document of the Undergraduate and Graduate Program Assessment

(January 2013), this report was prepared. The following statements indicate the sociology

undergraduate program assessment plan.

1. Program assessment will include course level assessment for all core courses within the

undergraduate program. The faculty who are responsible to teach specific core courses

with the exception of the S161 – Principles of sociology course will meet and develop

assessment instruments that will measure the level of success that students have achieved

in meeting the leaning objectives that have been defined in the curriculum map for the

course.

a. The method of assessment may include student responses to objective

questions, short answer, essay or written report or term paper that is included

as part of the course requirements.

b. If written work a grading rubric for assessing the student work and will be

developed to assess student performance relative to the course objectives.

c. All faculty members who teach the core courses will be required to issue a

brief report to the chair that will include a description of the level of success

of students in meeting the learning objectives for the course.

2. Survey of graduating seniors - Students in their final semester of study at IPFW will be

surveyed to self-report on the degree to which they have achieved the learning objectives

for the program. They will also be asked about areas of improvement in the program

including advising, teaching methodologies, and course offerings. The survey instrument

will be sent electronically and by regular mail.

5

3. Survey of alumni - A survey of graduates will be conducted on the third anniversary of

their graduation. The survey instrument will be sent electronically and by regular mail to

measure student satisfaction with the program and their self-report of skills and

knowledge areas that have been important to their future education and employment.

4. The assessment report will include information on the undergraduate program assessment

and based on the information make recommendations for program improvement and/or

improvement in the instruments of assessment.

5. The assessment report will be sent to faculty prior to submission to the COAS

Assessment Committee for comment and suggested changes.


Below is a summarization of the instructor’s assessment of student performance as it

relates to the program learning objectives. In this report, S351 and S470 assessments are

presented. Although S340 was offered in Fall 2014, the Department was not able to obtain the

assessment report because the faculty who taught the course retired in December 2014.

Program Assessment for S351 Social Statistics, Spring 2015

S351 Social Statistics is a core course for the Sociology undergraduate program. The

course learning objectives are stated below and the key indicates the level of mastery of the

learning objective.

The instructor used a pre-test/post-test assessment method using the following open-

ended questions found in Table 1.0 Pre-test/Post-test Questions for S351 below with each

question aligned to a particular learning objective or set of learning objectives as provided in

Table 1.1 S351 Social Statistics Learning Objectives and Level of Mastery. Because the

expected level of performance is C or better to complete the course for credit in sociology, the

instructor use the same threshold for performance on the pre-test/post-test. It is expected that

80% of students will succeed at 70% or better on each learning objective by the end of the course.

Pre-test and post-test scores are reported in Table 1.2 S351 Social Statistics Pre-test/Post-test.

6

Table 1.0 Pre-test/Post-test Questions for S351

Objective 1: What is a data set? Describe a data set structure.

Objective 1 & 2: What is multivariate analysis? When do you use it? What conclusions can you

draw? Give a couple examples.

Objective 2: Explain ANOVA? When do you use it and what does it tell you?

Objective 2: Write three hypotheses.

Objective 2: What are the two types of t-tests? How are they similar and different?

Objective 2: What is a categorical variable?

Objective 2: What is the regression equation? Explain the elements of the equation and how you

use each.

Objective 2: What is a p-value? Explain thoroughly.

Objective 2: What is a scale? How do you create one? Why do we use them?

Objective 1, 2 & 3: What is univariate analysis? When do you use it? How do you do it in SPSS?

What conclusions can you draw? Give a couple examples.

Objective 1, 2 & 3: What is bivariate analysis? When do you use it? How do you do it in SPSS?

What conclusions can you draw? Give a couple examples.

Objective 3: How do you complete ANOVA in SPSS?

Objective 3: How do you complete a t-test in SPSS?

Objective 3: What is SPSS and how do you open a data set and obtain descriptive statistics on a

variable?

Objective 3: Define recode. Give an example. Explain when/why you would recode.

Objective 3: Define reverse code. Give an example. When/why would you reverse code?

From the results the course instructor concluded that while the complete class failed every

learning objective on the pretest, the post-test demonstrated that over 95% of students achieved

learning objectives 1, 2, and 3 with at least with a performance of 70% or better.

Regarding Objective 4, it was assessed by requiring students to collect scholarly journal

articles using each statistical technique taught in the course from which they had to write

7

summaries and critiques in their journals for the course. Over 95% of students completed the task

with 70% or better on average.

Table 1.1 S351 Social Statistics Learning Objectives and Level of Mastery

Undergraduate Learning Objectives and Course Responsibilities Level of Mastery

Objective 1: Interpret Results of Data Gathering E

Objective 1: Interpret Results of Data Gathering E

Objective 3: Demonstrate use of Statistical Software E

Objective 4: Critically Evaluate Published Research E

I = Introduced, E = Emphasized, R = Reinforced

Table 1.2 S351 Social Statistics Pre-test/Post-test

Undergraduate

Learning

Objectives and

Course

Responsibilities

Level of

Mastery

Pre-test

Post-test Was goal

met?

Intervention

required?

Changes in

Assessment

Needed?

Objective 1:

Interpret Results

of Data

Gathering

E 0% of

students

achieved

70% or

better

100% of

students

achieved

70% or

better

Yes None No changes

needed.

Objective 2:

Demonstrate

Use of

Statistical

Techniques

E 0% of

students

achieved

70% or

better

Over 95%

of

students

achieved

70% or

better

Yes None No changes

needed.

Objective 3:

Demonstrate

use of Statistical

Software

E 0% of

students

achieved

70% or

better

Over 95%

of

students

achieved

70% or

better

Yes None No changes

needed.

Objective 4:

Critically

Evaluate

E Not

assessed

in pre-test.

Over 95%

of

students

Yes None No changes

needed.

8

Published

Research

achieved

70% or

better

Program Assessment Report on S470 Senior Seminar – Fall 2014

The S470 Senior Seminar is the capstone course for the major. In terms of the learning

objectives for this course, two skill areas are emphasized; Develop Oral Research Report and

Demonstrate Understandings of ASA Professional Ethical Standards. These two learning

objective that are listed as emphasized is quite modest and doesn’t really provide an opportunity

for a more comprehensive assessment of student performance given its capstone nature.

However, the skill areas for reinforcement as noted by the learning objectives for the major are

extensive. The previous year’s assessment focused principally on the two emphasized areas. In

this year’s course design and implementation, the instructor focused more on the assessment of

skills represented in the reinforced areas. In particular, the instructor designed the capstone

course to teach and assess students’ abilities in theoretical analysis and the research design. The

assessment of student performance in these areas was based on my observations of classroom

discussion, student oral report, design team reports, individual student written exercises and

report, and student’s own self –assessment that was incorporated in the student course

evaluations.

Assessment of student performance in the theoretical analysis area were based on the

evaluation of student presentations and discussions in the portion of the course that had the

students analyze a classic work in sociology, The Sociological Imagination by Mills. The major

areas identified in the program learning objectives for reinforcing theoretical skills were in the

students’ ability to identify theoretical perspectives, engage in theoretical analysis, critically

9

evaluate theoretical arguments, and theoretically interpret social issues. Students were assigned

chapters of Mills work to do an oral presentation highlighting the central components of his

analysis of the discipline and his application of theory to the social conditions of his day. In

general, the students’ performance were adequate. Some students were more able to interpret this

work than others. Although most students demonstrated basic skills in the identification of

theoretical models and their application and to theoretically interpret social issues, their ability to

critically evaluate theoretical arguments was generally weak. Part of the problem resulted from

their ability to understand the text. However, there also was a lack of depth of theoretical

knowledge to provide the necessary background to understand Mill’s critique and to consider a

critique of his work. Nevertheless, in student evaluations, the all nine students enrolled in the

course agreed (both strongly agree and agree) with the following statements; “This course

provided me with an opportunity to identify and critically evaluate theoretical perspectives and

theory based arguments,” “This course provided you with an opportunity to use sociological

theory to interpret social issues and problems,” and “This course provided you an opportunity to

engage in sociological theoretical analyses.”

In regard to the methods skills to be reinforced, the instructor focused on the students’

ability to evaluate different research methods. Students were broken into teams to develop a

series of recommendations on the design of a research project. The instructor presented an

applied research question regarding the university administration’s need to assess student,

employee, and alumni perceptions of the benefits of the Division I Athletic Program. The student

teams (of three students each) made recommendations and reported on basic design issues;

quantitative vs. qualitative, population and sampling issues, identification of concepts embedded

in the research question, operationalization of variables, and questionnaire design. Overall, two

10

out of the three teams of students were able to determine the appropriateness of their chosen

design either as a qualitative or quantitative research design and contributed to the

operationalization of variables in the development of the questionnaire. One team of students had

more problems determining the appropriate design given the research question and had problems

in the operationalization of the variables. However, approximately half the students had

problems in identifying concepts embedded within the research question. This may been in part a

result of the specific research questions the students were asked to investigate and the limited

time to explore the topic.

In separate exercise, students were also assessed in the application of the ASA’s ethical

standards related to human subject research. Students were asked to apply the ASA Ethical

Standards to a series of research scenarios. All students were in general successful in the passing

this exercise. In student evaluations, all students either responded strongly agree or agree to the

statements “This course provided you with an opportunity to evaluate different research methods

for strengths and weaknesses” and “The Ethics Exercise assignment provided you a good

opportunity to demonstrate your understanding of ASA Professional Ethical Standards.”

The last area of assessment in the area of reinforcing knowledge was in the development

of a research paper. Students conducted library research on a topic of interest, developing a

research question from their literature review and developed a research design for answering the

research question. The majority of students successfully completed the assignment. Problem

areas for several of the students was in making the transition to designing the research to answer

the research question that they had chosen. Students had less problems conducting a review of

the literature and developing a research question. Although, some students had problems

integrating the literature into a concise review. But when came to design work, they had

11

problems going through the steps in designing the research. In the student evaluations, all

students either responded strongly agree or agree to the statement that “The final report in this

course, the Sociological Analysis and Research Design Report, was an effective way for you to

demonstrate your skills in defining a research topic, develop a research question based on the

literature and design a research project based on the question posed.”

Based on the results of student performance in this course, the instructor would suggest

that students be provided with more opportunity to write research papers which require them to

develop a research design of the research questions that they develop from their literature

reviews. Possible this could be accomplished with having more of our students enrolling in

S495, the directed research courses which would provide them with a guidance experience the

literature review and research design. The instructor would also suggest that the S470 course be

more standardized in the manner in which it is taught. This was my first time teaching this

course, and the instructor borrowed some of the framework from how it was taught it previously

but also incorporated other elements to better assessed student theory and methods skills. The

assessment instruments used for this course should also be standardized along with the areas of

assessment. This course as the capstone course, should provide for an opportunity for the

Department faculty to better assess the strengths and weaknesses of the program. However, this

can only be done if the instruments of assessment and course content become more standardized.

Exit Survey Results 2014

To ask students to report their experiences related to the sociology undergraduate

program and areas for improvement in the program, the Department of Sociology administered

Exit Surveys to nine students who were enrolled in SOC-S470 Senior Seminar. The surveys were

12

conducted in December 2014. The surveys were distributed in class, and all nine students

participated in the surveys. The survey instruments contained three parts: 1. Learning objectives,

2. Experience related to the sociology program, and 3. Suggestions that students like to make for

improving the sociology program. Part 1 and 2 were asked by Likert-scale questions, and Part 3

was asked by open-ended questions.

Part 1 asked how well they felt they were able to accomplish each of 16 learning

objectives set for the course of SOC-S470 (see p.3 in this report). Graph 1 summarizes results of

Part 1. Five Likert-scale was used, indicating Level 1 as “Unable to accomplish” through Level 5

“Fully accomplish.”

Graph 1

0 1 2 3 4 5 6 7 8 9 10

Demonstrate an understanding of the ASA Code of Ethics

Understand the ethical standards as related to the use of…

Successfully present a research report

Successfully write a research paper

Critically evaluate published research

Develop evidence based on arguments

Critically evaluate theoretical arguments

Use a statistical software package

Correctly select the appropriate statistical technique…

Interpret social science data

Critically evaluate different research methodologies in…

Demonstrate the use of qualitative methodologies in the…

Demonstrate the use of quantitative methodologies in…

Theoretically interpret social issues

Engage in theoretical analysis

Identify, define and explain the three major theoretical…

unable to accomplish (Level 1) Level 2 Level 3 Level 4 Fully accomplish (Level 5)

13

Overall, the majority of students indicated they were able to accomplish learning objectives set

for the course of Senior Seminar, rating from 4 to 5. All nine students rated “Fully accomplish

(level 5)” on the following three objectives: “Develop evidence based on arguments,”

“Successfully write a research paper,” and “Understand the ethical standards as related to the use

of human subjects in research.” On the following six objectives, “Theoretically interpret social

issues,” “Critically evaluate different research methodologies in sociological research,”

“Critically evaluate theoretical arguments,” “Critically evaluate published research,”

“Successfully present a research report,” and “Demonstrate an understanding of the ASA Code

of Ethics,” all the students rated 4 or above.

Interestingly, students’ responses on statistical knowledge and skills varied. On the

learning objective, “Use a statistical software package,” one out of nine students indicated “fully

accomplished (5),” as level of accomplishment, six students reported 4, one rated 3, and another

one indicated 2. On the objective, “Correctly select the appropriate techniques when analyzing

data,” one student indicated “fully accomplished (5),” seven students responded 4, and one

student rated 2.

Part 2 focused on student experience related to the sociology undergraduate program and

asked to rate the following items: Advising within the Department , Frequency of course

offerings, Diversity of course offerings, Availability of faculty, and Sociology Club activities.

Graph 2 summarizes Part 2 results. Five Likert-scale was used, indicating Level 1 as “Poor

quality” through Level 5 “Excellent quality.”

14

Graph 2

In the item, “Advising within the Department ,” four students rated “Excellent,” two students

rated “level 4,” and three indicated “level 3.” In the item, “Frequency of course offerings,” three

students indicated “Excellent,” another three reported 4, two indicated 3, and one student

indicated 2. In the item, “Diversity of course offerings,” six out of nine students reported

“Excellent (5),” two students indicated 4, and one student rated 3. In the item, “Availability of

faculty,” four students indicated “Excellent (5),” three students reported 4, and two students rated

3. Finally, in the item “Sociology Club activities,” two students indicated “Excellent (5),” two

rated 4, and five students assessed 3.

Part 3 asked students to provide specific comments on the following items: Advising,

Frequency of course offerings, Diversity of course offerings, Faculty availability, What are the

overall strengths of the program?, What are the overall weaknesses of the program?, Would you

like to see more or less writing assignments in our program?, Was something not covered in the

0 1 2 3 4 5 6 7 8 9 10

Sociology club activities

Availability of faculty

Diversity of course offerings

Frequency of course offerings

Advising within the department

Poor (Level 1) Level 2 Level 3 Level 4 Excellent (Level 5)

15

core courses (writing, methods, stats, capstone) that you would have liked to cover?, Any

additional comments.

In the item, “Advising,” overall, students showed positive responses about faculty

advising. A student reported “great,” and another said “continue to help students view their

options and obtain the most out of their course choices.” One student indicated “I feel advisors

(with the exception of a few) need to help the student plan better to get the most out of IPFW.”

In the item, “Frequency of course offerings,” a student described “continue to be flexible with

students and their schedules.” Two students suggested to offer more methods and stats courses.

One student indicated “stats offered during more periods,” and another wrote “I would have

liked stats to be available every semester.” In the item, “Diversity of course offerings,” students’

responses were mixed. Some students indicated a high level of satisfaction of course offerings.

One student wrote “I love the diversity of sociology. I feel versed in a number of areas because

of the program (politics, economic, etc).” In contrast, other students suggested more choices. A

student described “I would have liked separate course offerings for macro and micro theories.

They are the core of the discipline and a strong grasp is essential.” Another student said “I’d like

to see more things pertaining to education, personally but maybe trying more with other areas

(WOST, Communication, Education, English, whatever) could be a way for soc students to study

more diverse offerings.” In the item, “Availability of faculty,” mostly students were satisfied

with faculty availability. A student reported “Faculty is great. I really only had availability issues

with my advisor,” and another said “most were very gracious with their time.”

According to student respondents, strengths of the program were: “Knowledge and

enthusiasm of faculty,” “faculty support and guidance,” “a good learning experience,” and “the

different types of sociology offered,” “online classes, knowledgeable professors, variety of

16

course offerings,” “the professors who are passionate about what they teach” and “I loved al the

professors I had because they were engaging. They made me get involved not just for my grade

but because I was always able to find things I cared about.” Six out of none indicated none for

weaknesses of the sociology program. Three students reported weaknesses of the program by

saying: “Different for fulltime workers to schedule all required courses,” “professors who do not

engage in the learning process – they seem to be here just for a job or the money,” and “The

once-a-year required courses was by far the worst/most difficult part to deal with, especially

when pre-reqs within the Department were featured in (writing is a spring class but required for

basically all the other courses).”

In the question, “Would you like to see more or less writing assignments in our courses?”

a student responded “I think professors should be more hands-on with students with the writing

assignments.” Another student wrote “In some classes, I would say less and more hands on

activities, but others were balanced well.” “less – all 3 classes I had were all writing assignments

for grades – could get overwhelming at times.” In the question, “Was something not covered in

the core courses (writing, theory, methods, stats, capstone) that you would have liked to cover?”,

a student indicated “Everything was covered more on writing and theory than on methods and

stats. Possibly add more methods/stats material.” One student mentioned “methods – very

unorganized and didn’t learn all of what I expected.”

Finally, some students wrote additional comments such as “Try to get students more

engaged in department activities,” and “really enjoyed this program.” One student wrote: “Thank

you for allowing me to come back to finish my degree, I feel like I am officially mentally and

academically ready to begin graduate work.”

17

Sociology Alumni Survey Results 2015

To ask alumni to report their experiences related to the sociology undergraduate program

and areas for improvement in the program, the Department of Sociology administered Alumni

Survey to alumni who graduated in the 2012 and 2013 academic year in August 2015. Because

only one of the eight alumni responded to an online survey, we decided to conduct a mail survey

in September 2015. As a result, three of the eight alumni completed a mail survey, and the

response rate was about 38 percent. The survey instrument was composed of 9 questions in total,

four of which were intended to collect demographic data related to the alumni (Questions 1-4):

current position, affiliation/employer, year of degree completion, and enrollment status at IPFW.

According to the survey results, all three alumni were employed. At IPFW, two alumni were full

time students, and one indicated both part time/full time student.

The remaining items contained Likert-scale and open-ended question. The followings are

Likert-scale questions (Five Likert-scale: 1. Great value/Very satisfied, 2. Some value/Satisfied,

3. Little value/Dissatisfied, 4. No value/Very dissatisfied, 5. Not sure):

Question 5: Please indicate the values of the skills and knowledge you developed as a

sociology major in preparing you in your current position.

Question 7: Please indicate your level of satisfaction with the undergraduate sociology

program at IPFW in preparing you for your current job.

Question 8: Overall, how satisfied are you with the quality of the sociology program at

IPFW?

In addition, the following two items were open-ended questions:

Question 6: Please list specific skills or areas of knowledge that they acquired as a

sociology major which has been useful to them in their current positions.

18

Question 9: Please indicate any suggestions that you would like to make that may

improve the quality of the sociology program at IPFW.

To answer Question 5, two students found “some value” while one indicated “great

value.” The survey asked to list specific skills or areas of knowledge that they acquired as a

sociology major which has been useful to them in their current positions (Question 6). One

student indicated “data collection, observation techniques, and operationally defining terms.”

Another student reported “reading, writing speaking understanding people, understanding

societal factors, becoming a certified mediator.” The last student described “ability to think

critically and problem solve.”

Regarding Question 7, three different responses were chosen: “very satisfied,” “very

dissatisfied,” and “not sure.” Among the three alumni students, two indicated their overall

satisfaction of the sociology program at IPFW was “satisfied” while one indicated “very

dissatisfied (Question 8).” The survey further asked the alumni students to indicate any

suggestions that they would like to make that may improve the quality of the sociology program

at IPFW (Question 9). Two out of three students suggested that the Department create more

specific job opportunities related to the sociology field in the curriculum. One student mentioned

that “Internship opportunities and interview training would be helpful,” and another suggested

“more electives and opportunity to do research.”

19

Section 5: Conclusion

Overall, two upper level core course assessments (SOC-S 351 and SOC-S 470) indicated

that the required courses met the program objectives assigned. The exit student survey results

also revealed positive learning experiences of students who were enrolled in the course of SOC-S

470. Almost all students reported that they were able to accomplish the learning objectives set

for SOC-S470. They acknowledged their understanding of sociological theory and ability of

evaluating published research and writing/presenting a research paper. However, a few students

showed less confident with quantitative analyses (using statistical software and conducting a

statistical analysis) than with theoretical understanding and application.

The exit survey results demonstrated students’ positive experiences through the sociology

undergraduate program. The majority of students rated 4 or above on advising, course offerings,

and faculty availability. However, they did request more frequent course offerings of core

courses and more diverse courses for electives. Some students suggested that the sociology

program should more often offer research methods and statistics courses. Others noted that they

would have liked to have more hands-on opportunities where students could engage in research

activities such as data collection and analysis. Given these responses, the Department will need

to discuss redesigning course offerings in order to provide more research activities for students.

The alumni survey results revealed mixed responses regarding students’ experiences

through the sociology undergraduate program. They valued knowledge and skills that they were

able to gain from the sociology undergraduate program. However, their levels of satisfaction

varied, rating from “very dissatisfied” to “very satisfied.” Specifically, the alumni respondents

noted that the Department should clearly show how sociology would lead students to their future

job opportunities and career after their graduation. To respond to this suggestion, the Department

20

is recommended to discuss strategies for enhancing career awareness among majors and minors.

For examples, the Department can encourage faculty to incorporate a career awareness session in

their teaching courses and also host a career service event for majors and minors. In addition to

the curriculum redesign/development, the Department will continue to encourage faculty to meet

and mentor individual students in order to learn about their interests, expectations, and goals.

Given the results from the core course assessment, exit student survey, and alumni survey,

the Sociology Undergraduate Program Committee has three recommendations to improve the

assessment plan and curriculum. First, the Department is recommended to discuss to offer more

various courses from the existing courses. For example, the Department can generate more

interdisciplinary courses by cross-listing courses with other Department s and programs. Second,

the Department is recommended to clearly show students how sociology will lead them to their

future professional jobs/career by providing specific examples and guidance. Creating more

opportunities for student research and presentation will be helpful for sociology students.

Offering the courses such as internship and service learning will increase diverse course

offerings and hand-on opportunities for students. Third, faculty members are encouraged to

continue advising and mentoring their students through their teaching courses and get involved

with sociology-related activities such as sociology club. Students appeared to appreciate faculty

advising and engaging, and therefore, faculty’s involvement is indeed critical for students to

succeed and retain in the sociology program.

Discussion for improving an assessment plan

The exit student survey results and the alumni surveys were very useful for us to not only

identify specific areas for improvements in our curriculum but also learn how well our program

has helped students connect job opportunities and develop their professional careers. However, a

21

low response rate has been key in conducting surveys. In the previous report (2013-2014), the

Department Undergraduate Committee proposed that the exit student survey be completed at the

conclusion of the S470, Senior Seminar course, in order to increase graduating student’s

participation in the exit survey. For this report, we were able to obtain all students’ responses

enrolled in the course of senior seminar. However, we experienced difficulty for obtaining

alumni responses this year. We first conducted online surveys to all eight alumni students but

received only one response from them. To increase the response rate, we mailed surveys in

September 2015. As a result, we were able to obtain three responses out of eight alumni. To gain

more responses from alumni, how to conduct alumni surveys needs to be discussed within the

Department. In addition, redesigning alumni survey questions will be needed to fully understand

students’ experiences through the sociology program and their directions after they graduated

from our program.

2014-2015 college of arts and sciences department reports

Documents