chapman university...chapman university annual learning outcomes assessment report 2013-2014 degree...
TRANSCRIPT
CHAPMAN UNIVERSITY
Annual Learning Outcomes Assessment Report 2013-2014
Degree Program Information
Academic Degree Program (Major)
Psychology
Degree (BS, BA, BFA, MFA, MBA, etc.)
B.A.
Department/ School Crean College of Health and Behavioral
Sciences
Number of students currently enrolled (as majors) In the program
411
Contact Person
Name (Person coordinating program’s assessment effort)
Steven Schandler Ph.D.
Title Professor of Psychology (tenured)
E-mail address [email protected]
CHAPMAN UNIVERSITY
Learning Outcome #1 I. Process: Student Learning Outcome Description
SLO 1: KNOWLEDGE OF CORE PSYCHOLOGICAL
FIELDS
A. Identify the current and historical core content of and
what is known therein.
B. Differentiate the various areas of Psychology and identify
what is known in each.
C. In a chosen topic area in Psychology demonstrate clear
understanding of operational definitions in that area and use
those to clarify what is and what is not presently known, and
what is the certainty of each type of knowledge.
D. In a chosen topic area in Psychology, integrate what is
presently known to explicate the status of this area of
knowledge, design further inquiry and conclude the present
and possible future impact of this knowledge on society.
E. Evaluate research skills as adequate for a career in
science.
F. Develop an understanding of Psychology sufficient to
develop and support the pursuit of post-graduate and/or
professional goals.
Supports University Theme (Some or
all of the program’s learning outcomes
must support at least two of the university’s
strategic themes.)
Themes: Internationalization, Personalized Education, Faculty/Student Research, Interdisciplinarity, or Student Writing
Describe how the theme is supported by the learning outcome.
1. This is a core element of the educational process. It
supports literally every stated University Theme.
2. Outcomes E and F particularly emphasize the use of
Personalized Education and Student Research.
Supports WASC Core Competency (Please indicate whether this outcome
1. Objectives A and B meet WASC core competencies of
CHAPMAN UNIVERSITY supports any of WASC’s core competencies) • Oral Communication • Written communication • Information Literacy • Quantitative Reasoning • Critical Thinking
information literacy and written communication.
2. Objective C meets WASC core competency of
quantitative reasoning
3. Objectives C, D and E meet WASC core competencies of
critical thinking, information literacy, and written and oral
communication.
4. Objective F meets WASC core competency of critical
thinking.
Where is the outcome published for students? • Syllabi (list course numbers)
• Website • Handbook
1. Information regarding the Psychology program and each
course learning objectives is available from the Chapman
University Learning Outcomes Assessment website at:
http://www.chapman.edu/academics/learning-at-
chapman/_files/assessment-report-
pdfs/bapsychology_aloar.pdf
2. In terms of Outcomes A and B, the Psychology 496, 497
and 498 Senior Project course syllabi specify that all
graduating seniors, as part of the capstone experience,
complete the GRE Major Field Test (MFT) in Psychology
(see LOA Table 1). When the test scores are returned from
ETS, each student is given their individual scores along with
the national averages and the results are explained to all, in
terms of graduate applications and career planning. The need
for the accumulation of course specific knowledge is
published in every one of the 39 psychology undergraduate
syllabi.
3. Outcomes C and D are presented and discussed in the
syllabi for the research methods courses (Psychology 204,
204L, and 304) and for each Senior Project course
(Psychology 496, 497, or 498). From the first course
meeting, students in each class are informed of the
expectation of these outcomes and the means of achieving
them.
4. Outcome E is specified in the syllabi for Psychology 496:
Senior Research Project and Psychology 497: Senior Thesis.
5. Outcome F is addressed early in the Psychology Major
core curriculum and continuously through matriculation.
CHAPMAN UNIVERSITY Early in their pursuit of the major, student must decide: a) Is
the Psychology Major correct for them?; and, if so, then b)
How will they use their education to pursue personal and
professional objectives after completion of their
undergraduate studies. To assist the addressing of these
questions, Outcome F is addressed during regular meetings
with the student's academic advisor and through a student
planning workshop presented prior to registration each
semester sponsored by our chapter of the Honors Society in
Psychology (Psi Chi). The Psychology Program office
maintains an extensive literature library of materials from the
American Psychological Association about the major and
about post-graduate academic and career planning.
Evidence of Learning • capstone project
• presentation • performance • course-embedded exam • assignment • standardized test • portfolio
1. The Psychology MFT assesses Outcome A with the
general psychology knowledge overall score based on a
national comparison with all students taking the test in the
academic year. Outcome B is assessed by the level of
knowledge in the major areas of Psychology: 1) Learning /
Cognition, 2) Perception / Physiological, 3)
Clinical/Abnormal / Personality, and 4) Developmental /
Social in the same manner as the overall score is determined.
The students are sampled in a formal ETS testing session,
and 84 graduating seniors were tested in 2012-2013.
2. As presented in LOA Tables 2, 3 and 4, the Senior
Projects Options are graded by strict rubrics specifically
structured to each respective course that serves the option,
providing highly detailed evidence for Outcomes C and D.
3. Outcome E is assessed by the nationally-developed and
normed Standardized Assessment of Information Literacy
Skills (SAILS®) instrument (Kent State University
©, 2014).
This is the first year that this instrument was been used by
the Psychology Program.
Background Note: In the previous Psychology ALOA
report, it was proposed that, beginning this 2013-2014
Academic Year, Outcome E would be assessed with the
Undergraduate Research Students Self-Assessment (URSSA)
instrument. This instrument was selected because it was
being used by other programs in Psychology’s “home” at that
time, the Schmid College of Science, and because the
instrument was capable some tailoring to specific program
CHAPMAN UNIVERSITY emphases. However, for the assessment of the Psychology
Program, the instrument possesses two major limitations:
First, it only evaluates science knowledge and laboratory
research skills. Second, because the instrument is designed
for specific programs, it possesses no standardization or
national normative data. Comparisons can only be made
within the same program on data collected across years.
The Fall 2013 announcement of the move of the
Psychology Program from the Schmid College to the new
Crean College of Health and Behavioral Sciences in June
2014 resulted in a rethinking of the assessment of the
evaluation of research skills as adequate for a career in
science (SLO1.E.). Within the new College, Psychology is
an applied science focusing on the use of evidence derived
from the scientific method to develop interventions that both
maintain the health of society and treat individuals whose
health is compromised. This forms the basis for the current
overarching tenet of the discipline and the Chapman
Psychology curriculum which is evidence-based practice.
While the curriculum provides a basic course with laboratory
training in research methods, the upper division process,
applied and capstone courses predominantly focus on
interpreting and critiquing the recent empirical research
literature supporting current topics and trends. Therefore,
evaluations of research skills related to a career in
Psychology should focus more on the location and use of
research information rather than on its collection in the
laboratory. With the assistance of the Leatherby Library
staff, and especially Education Librarian Carolyn Radcliff
whose help is gratefully acknowledged, the SAILS®
instrument was identified. This instrument is fully
concordant with the applied science curriculum of the
Psychology Program and it possesses standardization and
normative data allowing comparisons with other programs
throughout the United States.
SAILS®
is based on the Association of College and
Research Libraries (ACRL) Information Literacy
Competency Standards for Higher Education. The
instrument yields evaluations of undergraduate student
proficiency in the areas of:
Research strategy development
Searching basics, including Boolean operators and
CHAPMAN UNIVERSITY strategies for expanding and limiting searches
Information retrieval
Basics of information evaluation
Finding tool selection and use
Basic understanding of source documentation
Basic understanding of the role of the academic library
in information literacy
Advanced searching, including the use of controlled
vocabularies
Advanced understanding of source documentation
Basic and advanced understanding of intellectual
property issues involved in the use of information
As this nationally used instrument is expressly designed to
evaluate research conceptualization and skills, its data are
directly reflective of the learning outcome.
4. Outcome F is assessed by administration of the program-
designed, Psychology Student Life Objectives and Goals
(PSLOG) survey to all Psychology Majors in the Fall
semester of their senior year (see LOA Table 5). Data
consists of the percentage of students applying for graduate
education in Psychology or an allied field is compared to the
national average of this variable as published by the
American Psychological Association and the Association for
Psychological Science. Collecting and Analyzing the Data How did you select the sample?
What was your sample size (number of students)?
Provide the percentage of the sample size as compared to the relevant population.
How did you assess the student work/data collected?
Possible Tools: rubric, exam questions, portfolio samples.
Attach all assessment tools.
1. The MFT was administered to all the students in Senior
Research and in Senior Thesis (Psychology 496, 497, n=17)
and to a selected sample of students in Senior Seminar
(Psychology 498, n=36). The use of selected sampling for
the latter was due to the greatly increased number of Fall and
Spring semester 498 course sections which would have
necessitated multiple purchases and administrations of the
MFT. This was not feasible given the lack of available test
proctors and budget constraints. However, statistical
sampling theory does indicate that a sample of 60% of a
homogenous (all Psychology Majors) populations is
sufficiently representative.
Scoring of the MFT is performed by the Educational
Testing Service, and the results are sent to the senior project
instructor in charge of test security for Outcomes A and B.
As this instrument measures both overall knowledge of
Psychology and knowledge in what are arguably the four
CHAPMAN UNIVERSITY major areas of the discipline, the instrument and scores are
directly reflective of Learning Outcomes A. and B.
2. Outcomes C and D are scored using the senior project
grading rubric. For purposes of score reliability, Psychology
496 research projects are independently scored by a
committee of three Psychology Program faculty. For
Psychology 496 (Individual Research, n=2), Psychology 497
(Thesis, n=15) and 498 (Senior Seminar in Psychological
Topics, n=72). Select sections of the grading rubric are
separately scored by the senior project instructor and a
trained graduate student course assistant. These sections are
Hypothesis, Operational Definitions, Organization,
Reference Quality, and APA format. As the senior capstone
project is directly reflective of the student’s ability to
formulate a question of controversy in the field; answer that
question by locating, analyzing and integrating the
information from the field’s literature; and formally present
the results both orally and in writing; the capstone data are
directly linked to these learning outcomes.
3. Outcome E is measured by the students' proficiency score
on the Standardized Assessment of Information Literacy
Skills (SAILS®) instrument. The assessment is completed by
the student online, and a score report is provided to the test
administrator by instrument publisher. (SAILS®) was
administered to all seniors completing Psychology 497:
Senior Thesis (n=15) during Spring 2014. Senior Thesis was
selected because, of any of the capstone options, this one-
year course demands from the most academically-
accomplished and academic career-oriented Psychology
Majors the most precise, efficient and sustained literature
research skills in producing the thesis. For this first
administration, a randomly selected sample of students
enrolled in Psychology 498: Senior Seminar in Psychological
Topics (n=27) also completed the instrument. Though this is
a one-semester course, the final project is an advanced
annotated review of literature on specific topic. Further,
students enrolled in 498 represent the academic performance
most reflective of the majority of Psychology Majors.
The score report consists of item-by-item performance for
each student together with an overall proficiency percentile
score for the group and a list of overall scores from 50 other
United States institutions similar to Chapman. According to
CHAPMAN UNIVERSITY the instrument publisher, a score at or above the 70
th
percentile demonstrates proficiency in information literacy
reflected by:
Research strategy development
Searching basics, including Boolean operators and
strategies for expanding and limiting searches
Information retrieval
Basics of information evaluation
Finding tool selection and use
Basic understanding of source documentation
Basic understanding of the role of the academic library
in information literacy
A score at or above the 85th
percentile further includes
mastery of:
Advanced searching, including the use of controlled
vocabularies
Advanced understanding of source documentation
Basic and advanced understanding of intellectual
property issues involved in the use of information
Students scoring “well-above” the 85th
percentile can also
demonstrate:
Advanced information evaluation
4. Outcome F is scored by the percentage of Psychology
seniors applying to graduate school for further education in
this major or allied areas. These percentages are compared
to previous year’s seniors as well as to national averages.
This is the method of assessing this objective recommended
for use by American Psychological Association and the
Association for Psychological Science. Expected Levels of Achievement
What was your target(s) for student performance for this outcome? (This should tie to the methods in which you assessed the students and collected and analyzed data in the section above.)
1. Regarding Outcomes A and B: The content of the
Psychology Major is designed to fully conform to the
curricular content of the American Psychological Association
Guidelines for the Undergraduate Psychology Major Version
2.0 (August, 2013). Not unexpectedly, the GRE Major Field
Test in Psychology is designed to measure the four key
content areas of the curriculum indicated in the APA
guidelines. Since publication of the most recent 2006 MFT
scale norms in 2006 for U.S. public and private institutions
(n=7,557), Chapman Psychology graduating seniors have for
CHAPMAN UNIVERSITY every year achieved mean MFT scores above the national
averages for overall scores and all four subtest areas (see
LOA Table 1). These findings were expected to continue for
2013-2014.
2. Assessment criteria for the capstone Senior Project
Options are presented in LOA Tables 2, 3, and 4. Outcome
C is assessed for students completing the Option 1:
Individual Research Project (LOA Table 2) by scores in the
column specifying "Problem", "Reference Quality" and
"Reference Coverage". The achievement criterion is a score
of 9.0 or higher (90th percentile).
For students completing Senior Project Option 2: Thesis
(LOA Table 3) assessment of Outcome C is presented in
scores for the "Hypothesis", "Operational Definitions",
"Reference Quality", and "Reference Coverage". The
assessment criterion is a score of 9.0 or higher (90th
percentile).
Outcome C assessment of students completing the one
semester Senior Project Option 3: Senior Seminar in special
topics in Psychology and completing a similar first semester
of the two- semester Option 2: Thesis (LOA Table 3) is
based on scores in "Hypothesis", "Operational Definitions",
"Reference Quality" and "Reference Coverage". The
expected level of achievement for the first three areas is
8.5/10. Relative to the other Senior Project options, this is a
lower criterion reflecting the review/developmental nature of
the project rather than the definitive test of a hypothesis. The
expected level of achievement for "Reference Coverage" is
4.5/5.0. The category is given less value than in the other
option because the number of reference annotations allowed
is limited.
3. Assessment criteria for the Psychology Program Capstone
Senior Project Options are presented in LOA Tables 2, 3, and
4. Outcome D is assessed for students completing the Option
1: Individual Research Project (see LOA Table 2) by scores
in the columns specifying "Results" and "Discussion". The
expected level of achievement is a score of at least 9.0/10.0.
This expectation was met for 2013-2014.
For students completing Options 2: Thesis, Outcome D
assessment is found in LOA Table 3 in the column
specifying "Logic" and "Conclusion". The expected level of
achievement is a score of at least 9.0/10.0. This expectation
CHAPMAN UNIVERSITY was exceeded for 2013-2014.
The assessment of Outcome D for students completing
Option 3: Senior Seminar is found in the "Annotation
Quality" column of LOA Table 4. The expected level of
achievement in this category is a score of 16.0/20.0. This
expectation was exceeded for 2013-2014. Relative to the
other Senior Project options, this is a lower criterion
percentile level reflecting the review/developmental nature of
the project rather than the definitive test of a hypothesis.
4. Assessment of Outcome E began during this, 2013-2014
Academic Year using a selected administration of the
Standardized Assessment of Information Literacy Skills
(SAILS®) instrument. With no previous information
available regarding the literacy proficiency of Chapman
Psychology Majors, two general predictions were made: a)
Psychology Program Senior Project students would show an
overall proficiency score substantially higher than the
national norm. The rationale for this predictions is based on
the Psychology Program’s unique requirement that all majors
(rather than the usual select group of honors students) must
complete a substantive literature-based research project for
their capstone; b) Due to the relatively higher literacy
research demands of the two-semester thesis course, students
enrolled in Senior Thesis (Psychology 497) would show
higher proficiency scores than students enrolled in the one-
semester Senior Seminar (Psychology 498).
5. Outcome F is assessed using Psychology Student Life
Objectives and Goals (PSLOG) survey data (LOA Table 5)
of the percentage of graduating seniors applying to graduate
school in Psychology and allied areas such as school
psychology, social work, etc. and the percentage of
graduating seniors who are undecided about their career
objective. According to the American Psychological
Association, the national average for psychology students
pursuing graduate education immediately after completion of
the bachelor degree is 10-11%. Further, a national average
of 30% of psychology students is undecided about their post-
graduation career plans. Locally, the data are a bit more
optimistic. Information from surrounding institutions in the
area (CSU Long Beach, Fullerton and San Marcos; UC
Irvine and Riverside; USC) is that at least 20% of graduating
Psychology students pursue graduate education immediately
CHAPMAN UNIVERSITY upon graduation. The Psychology faculty expectation It is
expected that Psychology Program graduating seniors will
better these national and local findings (i.e., more seeking
graduate education; fewer undecided about a career path).
The rationale for this expectation is based on the relatively
high levels of academic and career counseling emphasized
and provided by the Psychology Faculty both within the
content of all Psychology courses and during face-to-face
student advising. Graduate school and career planning are
also promoted by multiple workshops sponsored throughout
the academic year by our chapter of the Honors Society in
Psychology (Psi Chi).
As for specific expectations, PSLOG data from 2012-2013
showed more than half of graduating Psychology seniors
were either accepted or planning to apply to graduate
programs. Fewer than 13% were undecided about their
career path. It was expected that at least similar findings
would occur for Psychology seniors graduating in 2014.
CHAPMAN UNIVERSITY
Learning Outcome #2 I. Process: Student Learning Outcome Description
SLO 2: CRITICAL REASONING
A. Identify historical and current major issues of discussion
and analysis in Psychology.
B. Demonstrate ability to construct and critically analyze
complex arguments, and distinguish good reasoning from
bad.
C. Evaluate lay and professional literature related to
psychological issues and distinguish appropriate and valid
information from specious and flawed information.
D. Organize and construct a formal critique of a major
psychological issue.
Supports University Theme (Some or
all of the program’s learning outcomes
must support at least two of the university’s
strategic themes.)
Themes: Internationalization, Personalized Education, Faculty/Student Research, Interdisciplinarity, or Student Writing
Describe how the theme is supported by the learning outcome.
1. Personalized Education
2. Student Research
3. Student Writing
Supports WASC Core Competency (Please indicate whether this outcome supports any of WASC’s core competencies.) • Oral Communication • Written communication • Information Literacy • Quantitative Reasoning • Critical Thinking
1. Objectives A, B, C and D meet WASC core competencies
of critical thinking, information literacy, and written and oral
communication.
Where is the outcome published for students? • Syllabi (list course numbers)
• Website • Handbook
1. Information regarding the Psychology program and each
course learning objectives is available from the Chapman
University Learning Outcomes Assessment website at:
http://www.chapman.edu/academics/learning-at-
chapman/_files/assessment-report-
pdfs/bapsychology_aloar.pdf
CHAPMAN UNIVERSITY 2. Psychology 201- Critical Thinking. Psychology is the
only university major to require a specific core course in
critical thinking early in the student's matriculation through
the major. In its 27th year of being offered every semester,
the purpose and goals of the course Psychology 202: Critical
Thinking is stated generally in the University Catalog and
specifically in the course syllabus.
3. Senior Project Capstone. The syllabus of every
Psychology senior project capstone course (Psychology 496,
497, 498) also states that Critical Thinking is the essential
component evaluated in the development of the project
question/hypothesis, the evaluation of related literature, and
logical interpretation of findings.
Evidence of Learning • capstone project
• presentation • performance • course-embedded exam • assignment • standardized test • portfolio
Objectives A – D:
1. Psychology 201- Critical Thinking. Evidence is collected
from instructors of the course Psychology 201: Critical
Thinking. This is a lower-division, required course in the
Psychology Major Core. The course is has a strict
enrollment cap and employs a seminar format. Essay
examinations assess the student’s identification and
understanding of critical thinking and reasoning principles,
types, processes, and strengths and limitations. Through
several directed assignments, the student also develops
literature-supported arguments either in support of or in
refutation of current controversial theories of human
behavior. The arguments are presented in both written form
and in the form of verbal participation in course discussions
and debates.
2. Senior Project Capstone. Complete data from the Senior
Project capstone courses are provided by the respective
instructors (refer to LOA Tables 2, 3, 4). For Psychology
496: Senior Research Project (LOA Table 2) the critical
thinking component consists of an evaluation of the
"Problem", "Organization", "Method", and "Discussion"
categories. For Psychology 497: Senior Thesis (LOA Table
4) the critical thinking component consists of an evaluation
of the “Hypothesis”, “Logic”, “Organization”, and
“Conclusion” categories. For Psychology 498: Special
Topics in Psychology: Literature Research and Critique
(LOA Table 5) the critical thinking component consists of an
evaluation of the “Hypothesis”, “Organization”, and
CHAPMAN UNIVERSITY “Annotation” categories.
Collecting and Analyzing the Data How did you select the sample?
What was your sample size (number of students)?
Provide the percentage of the sample size as compared to the relevant population.
How did you assess the student work/data collected?
Possible Tools: rubric, exam questions, portfolio samples Attach all assessment tools.
Objectives A – D:
1. Psychology 201- Critical Thinking. Nine sections of
Psychology 201: Critical Thinking were offered during
Academic 2013-2014. Total student enrollment was 193.
Critical Thinking is evaluated by overall course grades;
grades on selected critical reasoning assignments; student
course evaluations; and discussions with the course
instructors. The evaluation of data from a critical thinking
course is considered the most direct method of assessing
achievement of the Psychology Program’s critical thinking
learning outcome objective. That the course content is based
on nationally promulgated curricula for teaching and
assessing critical thinking and reasoning further validates the
direct connection between the course data and this Student
Learning Outcome.
2. Senior Project Capstone. Performance in Psychology
Senior Project courses (Psychology 496, n=1; Psychology
497, n=15; Psychology 498, n=72) is evaluated by collecting
and averaging the data from each of the categories specified
in Evidence of Learning.
Expected Levels of Achievement
What was your target(s) for student performance for this outcome? (This should tie to the methods in which you assessed the students and collected and analyzed data in the section above.)
Objectives A – D:
1. Psychology 201- Critical Thinking. Psychology majors
must pass Psychology 201: Critical Thinking with average
overall performance on examinations and critical reasoning
assignments above the 80th
percentile. Included in this
criterion is the demonstration of understanding of the
principal critical thinking skills used to evaluate scientific
and nonscientific evidence and the use of these skills to
support or challenge, through written and verbal expression,
current theories in the discipline.
2. Senior Project Capstone. Within the senior project
options (Psychology 496, 497, 498), the development of an
operationalized question/hypothesis; the selection and critical
review of relevant literature; and the logical interpretation of
findings and presentation of a conclusion account for 50% of
the project grade. All senior project students are expected to
achieve at least the 80th percentile in these areas.
CHAPMAN UNIVERSITY
Learning Outcome #3 I. Process: Student Learning Outcome Description
SLO 3: WRITING
A. Identify the writing format of the American Psychological
Association (APA).
B. Recognize when writing is and is not in conformance with APA
format.
C. Demonstrate ability to write in APA format and critique the
written format used by others.
D. State and operationally define a formal hypothesis and produce
a written thesis/research report that applies knowledge of critical
reasoning, accurately interprets behavioral science and related
sources, and communicates in writing a balanced account and
definitive conclusion of whether hypothesis is proven or not
proven. Supports University Theme (Some or
all of the program’s learning outcomes
must support at least two of the university’s
strategic themes.)
Themes: Internationalization, Personalized Education, Faculty/Student Research, Interdisciplinarity, or Student Writing
Describe how the theme is supported by the learning outcome.
1. Student Writing
2. Personalized Education
Supports WASC Core Competency (Please indicate whether this outcome supports any of WASC’s core competencies) • Oral Communication • Written communication • Information Literacy • Quantitative Reasoning • Critical Thinking
1. Objectives A, B, C meet WASC core competencies of
information literacy and written communication.
2. Objective D meets WASC core competencies of critical
thinking, information literacy, and written and oral
communication.
Where is the outcome published for students? • Syllabi (list course numbers)
• Website • Handbook
1. Information regarding the Psychology program and each
course learning objectives is available from the Chapman
University Learning Outcomes Assessment website at:
http://www.chapman.edu/academics/learning-at-
chapman/_files/assessment-report-
pdfs/bapsychology_aloar.pdf
CHAPMAN UNIVERSITY 2. As specified in the course syllabi, every Psychology
course, including the elective individual study, internship,
and fieldwork practicum courses, requires the production of
written work in the publication format of the American
Psychological Association. The work takes the form of
literature reviews and critiques, topic-focused term papers,
and/or laboratory or field journals.
3. As specified in the course syllabi, the mandatory senior
capstone courses for students majoring in Psychology require
the successful production of either a complete research report
and manuscript (Psychology 496); the production of a 50-70
page thesis using the literature and pilot studies to test an
operationally defined hypothesis (Psychology 497); or a
comprehensive annotated literature review and critical
analysis of a topic in psychology and/or related areas
(Psychology 498).
4. As specified in the University Course Catalog, graduation
with Psychology Program honors requires completion of
independent research. Further, program advisors and
workshops presented by the University's chapter of the
International Honors Society in Psychology (Psi Chi),
encourage all Psychology students seeking post-
baccalaureate academic education to complete and present
research at local, regional, and national venues. Presentation
of this research requires the writing of abstracts, presentation
papers, and publications.
Evidence of Learning • capstone project
• presentation • performance • course-embedded exam • assignment • standardized test • portfolio
Objectives A – C:
1. As presented in LOA Table 6, the first element of
evidence is that 100% of the offered 100-400 level courses
have graded written components. Beginning in the first,
introductory psychology course, submitted written work
must be formatted APA style or it is not accepted.
Objective D:
1. A. Capstone Course Writing: Evaluation Criteria for
Senior Research Project (Psychology 496). Research
manuscript guidelines of the American Psychological
Association.
a. Problem: Development of the problem to be addressed by
the research project using literature and logic; statement of
hypothesis; contribution of the proposed test of the
CHAPMAN UNIVERSITY hypothesis(ses) to the discipline and to society.
b. Detail: Use of specific data, constructs and studies in the
development of the problem; degree to which the empirical
studies are analyzed and presented to support testing of the
research hypothesis. Operational definitions of all
components of hypothesis. Inclusion of key definitions
necessary to comprehend reviewed articles.
c. Organization: Structure of the research project; use of
major and subsections to effectively present the research
problem, hypothesis, method, and the interpretation and
implications of the findings.
d. Method: Presentation of subjects, apparatus and
procedures in sufficient detail and with sufficient rationale to
allow comprehension and replication of the research project.
Adherence to all applicable ethical standards for the
conducting of the research project.
e. Results: Presentation of methods for collecting and
analyzing the research data. Selection and use of correct
statistical analysis(ses). Presentation of outcomes using both
descriptive and, if warranted, inferential statistics.
f. Discussion: Clarity of the conclusion derived from the
analysis of the data. Discussion of study strengths and
limitations. Discussion of impact of the findings on the
discipline and on society. Presentation of proposal for future
studies.
g. Reference Quality: Quality of the sources used in the
thesis (including recency of references and journal quality).
h. Reference Coverage: Degree to which the thesis
references address the problem and the hypothesis.
i. Care of Preparation: Absence of grammatical and
typographical errors.
j. APA Style: Conformance to the publication standards of
the American Psychological Association.
1. B. Capstone Course Writing: Written Evaluation Criteria
for Senior Thesis (Psychology 497):
a. Hypothesis: Development of the problem to be addressed
by the thesis hypothesis; statement of hypothesis;
contribution of the proposed test of the hypothesis(ses) to the
discipline and to society.
b. Definitions: Operational definitions of all components of
hypothesis. Inclusion of key definitions necessary to
comprehend reviewed articles.
c. Logic: Development of the problem to be addressed by the
CHAPMAN UNIVERSITY hypothesis; test of the hypothesis using empirical literature;
manner in which the literature findings are used to support or
refute the hypothesis.
d. Conclusion: Clarity of the conclusion derived from the
analysis of the literature; impact of the findings on the
discipline and on society; proposal for future studies.
e. Detail: Use of specific data, constructs and studies in the
development of the problem; degree to which the empirical
studies are analyzed and presented to support or refute the
hypothesis.
f. Organization: Structure of the thesis; use of major and
subsections to effectively present the thesis problem,
hypothesis, analysis of the literature, and the interpretation
and implications of the findings.
g. Reference Quality: Quality of the sources used in the
thesis (including recency of references and journal quality).
h. Reference Coverage: Degree to which the thesis
references address the problem and the hypothesis.
i. Care of Preparation: Absence of grammatical and
typographical errors.
j. APA Style: Conformance to the publication standards of
the American Psychological Association.
1. C. Capstone Course Writing: Evaluation Criteria for
Senior Seminar in Psychological Topics (Psychology 498):
a. Hypothesis: Development of the problem to be addressed
by the thesis hypothesis; statement of hypothesis;
contribution of the proposed test of the hypothesis(ses) to the
discipline and to society.
b. Operational Definitions: Operational definitions of all
components of hypothesis. Inclusion of key definitions
necessary to comprehend reviewed articles.
c. Organization: Structure of the thesis; use of major and
subsections to effectively present the thesis problem,
hypothesis, analysis of the literature, and the interpretation
and implications of the findings.
d. Annotation Quality: Selection and summary of relevant
research that effectively addresses the project hypothesis or
question. Use of specific data, constructs and studies in the
development of the problem; degree to which the selected
studies are analyzed and presented to address the hypothesis
or the question.
g. Reference Quality: Quality of the sources used in the
thesis (including recency of references and journal quality).
CHAPMAN UNIVERSITY h. Reference Coverage: Degree to which the thesis
references address the problem and the hypothesis.
i. APA Style: Conformance to the publication standards of
the American Psychological Association.
j. Care of Preparation: Absence of grammatical and
typographical errors. Use of proper headings and page
breaks.
3. Evidence for quality of research writing is based on the
number of abstracts and subsequent posters, oral
presentations, and papers accepted at local, regional, and
national/international venues. This information is presented
in LOA Table 7.
Collecting and Analyzing the Data How did you select the sample?
What was your sample size (number of students)?
Provide the percentage of the sample size as compared to the relevant population.
How did you assess the student work/data collected?
Possible Tools: rubric, exam questions, portfolio samples Attach all assessment tools.
Objective A – C:
1. The rubrics for scoring written coursework are course-
dependent and presented in each course syllabi. Overarching
is the degree of concordance between submitted written
coursework and APA publication standards. Some
instructors grade the submitted written work on the degree of
concordance. Other instructors will not accept/grade written
work that is not fully concordant with APA standards.
Objective D:
1. Senior project written work is scored independently by at
least two and up to four Psychology professors using the
100-point rubrics presented in LOA Tables 2-4.
2. Research abstracts are evaluated by the faculty member
supervising the student's research. The faculty member will
not allow a research abstract, report, and/or manuscript to be
submitted unless the writing is sufficient to at least support
an expectancy of serious review and acceptance.
Expected Levels of Achievement
What was your target(s) for student performance for this outcome? (This should tie to the methods in which you assessed the students and collected and analyzed data in the section above.)
Objective A – C:
1. On average, the Psychology course written components
account for 20% of total course grade. It is expected that
students will achieve at least "C" performance in this area.
Objective D:
1. For all Psychology senior project capstone courses the
final written project accounts for at least 60% of the final
course grade. Written projects completed during
development of the final project account for at least 30% of
the final course grade. Therefore, with the exception of a
CHAPMAN UNIVERSITY 10% oral presentation component, the senior project grade is
largely based on the written component. It is expected that
all seniors will earn at least a "B" grade in their senior project
course.
2. It is expected that every student completing Individual
Research (Psychology 499), the Senior Project Research and
Thesis Options (Psychology 496 and 497, respectively) and
those seniors seeking an "A" grade in Senior Seminar in
Psychological Topics (Psychology 498) will submit an
abstract to and present at the Schmid College of Science and
Technology Student Research Conference. For students
pursuing doctoral-level graduate work, it is expected that
they will also submit an abstract to and present at least one
regional or national professional conference.
CHAPMAN UNIVERSITY
Learning Outcome #4 I. Process: Student Learning Outcome Description
SLO 4: GLOBAL PERSPECTIVE
A. Identify and/or observe the role of personal and cultural
diversity on the behavior of an individual and recognize the
necessity and advantages of this information and its impact
on individuals and societies.
B. Distinguish the key behavioral factors associated with
personal and cultural diversity and construct an intervention
to assess the contribution of these factors to maximize
behavioral effectiveness or to treat behavioral dysfunction.
C. Produce a capstone project that investigates cultural and
diversity differences in the student chosen topic, if they exist,
and that proposes differences in conceptualization or
intervention when needed.
Supports University Theme (Some or
all of the program’s learning outcomes
must support at least two of the university’s
strategic themes.)
Themes: Internationalization, Personalized Education, Faculty/Student Research, Interdisciplinarity, or Student Writing
Describe how the theme is supported by the learning outcome.
As specified in the catalogue, Chapman University has a
major commitment to the theme of globalization and the
global citizen.
Supports WASC Core Competency (Please indicate whether this outcome supports any of WASC’s core competencies) • Oral Communication • Written communication • Information Literacy • Quantitative Reasoning • Critical Thinking
Where is the outcome published for students? • Syllabi (list course numbers)
• Website • Handbook
1. Information regarding the Psychology program and the
learning objectives of each course is available from the
Chapman University Learning Outcomes Assessment
website at:
http://www.chapman.edu/academics/learning-at-
chapman/_files/assessment-report-
CHAPMAN UNIVERSITY pdfs/bapsychology_aloar.pdf
2. In terms of Outcome A, there are cross-cultural and
diversity issues addressed in every Psychology course
syllabus. Primary and peer academic advising and special
workshops sponsored by the Program’s Chapter of the
International Honors Society in Psychology (Psi Chi)
encourage majors to study Psychology abroad for an even
more thorough exposure to the world-wide similarities and
differences in psychological issues.
3. In terms of Outcome B, the Psychology Curriculum
presented in the University course catalog and in the
Psychology Program Sheet published on the program
website, specify that majors must complete either
Psychology 328: Abnormal Psychology or Psychology 330:
Child Abnormal Psychology as part of their process course
requirements. These courses contain sections that explicitly
address the diagnosis and treatment of persons with diverse
personal and cultural backgrounds. Psychology Majors are
encouraged to take a least one of the Psychology cross-
cultural and/or diversity classes that are also part of the
General Education Global Citizen Cluster. The specific
courses are Psychology 341: Cross-Cultural Psychology;
Psychology 344: Psychology of Gay, Lesbian and Bisexual
Experience; and Psychology 355: Diversity in Marital and
Family Relationships. For these courses, an understanding of
global and/or diversity issues in Psychology is the major
theme of the course content with emphasis placed on
directing students to incorporate diversity and/or global
issues into their capstone project.
4. In terms of Outcome C, for the 2013-2014 Academic
Year Psychology majors completing the Senior Project
capstone courses of Psychology 497: Senior Thesis and
Psychology 498: Senior Seminar in Special Topics are
required by the syllabi to include cross-cultural and diversity
research in any chosen topic that has such research in
existence. Evidence of Learning • capstone project
• presentation • performance • course-embedded exam
1. Outcome A: In the Spring of 2009, the University
requested that all academic programs initiate assessment of
global education in their majors. In the 2009-2010 Academic
Year an assessment survey (see LOA Table 8) was added to
CHAPMAN UNIVERSITY • assignment • standardized test • portfolio
the Psychology Assessment Program that solicited from
every senior their global studies/diversity experience and
their opinion of whether their global studies/diversity
experience added to the student’s 1) General Education and
2) Education in the Psychology Major. The instrument also
obtains student ratings on a 7-point Likert scale with "7"
indicating that their global studies/diversity experience
substantially added to their General Education and
Psychology studies.
2. Data for Outcomes B and C are collected from the
capstone courses Psychology 497a/b: Senior Thesis and
Psychology 498: Special Topics. This assessment began this
this year (Academic 2013-2014). Relative to their chosen
topic, the student must provide evidence of their search of
the literature for cross-cultural and diversity research.
Collecting and Analyzing the Data How did you select the sample?
What was your sample size (number of students)?
Provide the percentage of the sample size as compared to the relevant population.
How did you assess the student work/data collected?
Possible Tools: rubric, exam questions, portfolio samples Attach all assessment tools.
This is an end of matriculation assessment. Therefore, the
2013-2014 sample was all Psychology seniors enrolled in
Senior Thesis (Psychology 497, n=15) and Senior Seminar
(Psychology 498, n=72). This sample of 87 students
constituted 89% of all graduating Psychology seniors (n=96).
The remaining 11% of Psychology seniors will complete
their projects during the Summer of 2014.
1. To assess Outcome A, the data are scored as the number
(and therefore percentage) of students who indicate they have
gone overseas for an educational experience as well as the
number of unit completed overseas, while enrolled at
Chapman University.
Beginning in the 2010-2011 Academic Year additional
Outcome A data were collected consisting of the student’s
assessment of the impact of global studies/diversity
experience on their General Education and their education in
Psychology. Statistical analysis, when required, consists of
one-way analysis of variance with a posteriori comparisons.
2. Beginning Academic 2013-2014, data for Outcomes B
and C are collected from the theses submitted from students
in the one-year duration Psychology 497: Senior Thesis and
from the topic review and analysis papers submitted by
students in the Psychology 498: Special Topics course. All
students were evaluated on whether they had completed the
diversity/cross-cultural course cluster of Psychology 341,
CHAPMAN UNIVERSITY 344 and 355. Referring to LOA Tables 3 and 4, a
"Diversity" category is added to the Senior Project Option 2:
Thesis and Option 3: Special Topic Seminar assessment
templates. Thesis topics and completed theses are reviewed
for focus on diverse populations and/or global, diverse
cultural perspectives. Criteria were at least one specific
section in thesis introduction identified as discussing
diversity and/or cultural issues and at least 30% of reviewed
empirical studies in the results using diverse and/or cross-
cultural subject populations. The 30% content criterion
derives from a similar criterion applied by the American
Psychological Association to articles submitted for
publication in the Association’s sponsored journals.
Expected Levels of Achievement
What was your target(s) for student performance for this outcome? (This should tie to the methods in which you assessed the students and collected and analyzed data in the section above.)
1. In terms of Outcome A, it is expected that the Psychology
program will place in the top half of Chapman University in
the percentage of students taking advantage of study abroad
programs. It is also expected that the self-evaluation of
impact will be above a mean of 5.0 on a 7-point Likert type
scale of impact of the study abroad experience on the
student’s 1) General education; and 2) Education in
Psychology.
2. In terms of Outcome B, we investigated if there is a
difference in topic choice by students completing the Senior
Project capstone courses who have and have not taken one or
more of the Psychology 341, 344 and/or 355 courses. It is
expected that students who have taken a course in the
Psychology Global Studies/Diversity cluster will be more
likely to choose a senior project topic with cross-cultural or
diversity implications than those who have not. Further, the
likeliness of pursuing a Psychology senior project topic rich
in diverse and/or cross-cultural content will be directly
related to the number of courses taken from the Psychology
341, 344, 355 cluster.
3. In terms of Outcome B and C, the Psychology senior
project instructors evaluate every senior project relative to
whether the topic specifically addresses an empirically-
verified cross-cultural and/or diversity issue. The
expectation is that at least 50% of the Senior Thesis and
Senior Seminar topics will explicitly and empirically address
a cross-cultural and/or diversity issue. This criterion was
derived from discussions of expectations for the
CHAPMAN UNIVERSITY undergraduate Psychology curriculum presented by the
American Psychological Association.