individual and group assessment€¦ · web viewresearch and program evaluation - studies that...

CALIFORNIA STATE UNIVERSITY NORTHRIDGEMichael D. Eisner College of Education

Department of Educational Psychology & Counseling

EPC 601 – Individual and Group Assessment

FALL 2008

Wednesdays, 4:00 p.m. – 6:50 p.m.ED 1121

Terry D. Piper, Ph.D.University Hall [email protected] Hours: By Appointment

Conceptual Framework

Michael D. Eisner College of Education

Regionally focused and nationally recognized, the Michael D. Eisner College of Education is committed to Excellence and Innovation. Excellence in the acquisition of professional knowledge, skills, and dispositions is evidenced by the growth and renewal of ethical and caring professionals – faculty, staff, candidates – and those they serve. Innovation occurs through collaborative partnerships that represent communities of diverse learners who engage in creative and reflective thinking. To this end we continually strive to achieve the following competencies and values that form the foundation of the Conceptual Framework:

• Excellence in professional and academic preparation • Collaborative partnerships

• Evidence of growth and renewal • Communities of diverse learners

• Ethical and caring professionals • Creative and reflective thinking

Nature of the Course

This course focuses on assessment and evaluation of college students and programs from an operational perspective rather than a clinical or psychological perspective. Its primary purpose is to help practitioners learn how to conduct assessment and evaluation to enhance student affairs practice and institutional effectiveness.

Participants in this course will explore the multiple methods [data gathering techniques] used in assessment and evaluation, understand the distinction between qualitative and quantitative methodologies of inquiry, gain practical experience in designing, data collection, data analysis, and reporting on assessment results.

mailto:[email protected]

EPC 601 • Piper • Fall 2008 page 2 of 13

Course Learning Outcomes

1. Students will understand the evolution and purpose of assessment in student affairs work. [CACREP Core standard: II.K.7.a]

2. Students will understand assessment standards and ethics as well as how to apply them in assessment practice. [Core standards: II.K.7.i, II.K.8.f]

3. Students will be able to articulate and execute an assessment plan at the program/service, unit or divisional level. [Core standards: II.K.7.b, II.K.7.c, II.K.7.g, II.K.8.b, II.K.8.d; Student Affairs Specialty standards: B.4, B.5]

4. Students will be able to identify and differentiate the various purposes/types of assessment and be able to apply the appropriate type(s) based upon the goal(s) of the assessment effort. [Core std.: II.K.8.d; Student Affairs Specialty standard: B.5]

5. Students will be able to identify the key characteristics of quantitative and qualitative methodology and the assumptions underlying the methodology. [Core standards: II.K.7.f, II.K.8.b]

6. Students will be able to articulate sampling approaches for both methodologies and to choose an appropriate sample size and selection method. [Core standard: II.K.7.g]

7. Students will have the ability to identify the types of data/information needed, data collection method, and data analysis based upon the goal(s) of the assessment. [Core standards: II.K.7.c, II.K.8.c]

8. Students will be able identify strengths and weaknesses of established assessment instruments and select the most appropriate for the goal(s) of the assessment. [Core standard: II.K.7.g]

9. Students will be familiar with basic psychometric properties, statistical methods, and computer based analysis options. [Core stds.: II.K.7.c, II.K.7.d, II.K.7.e, II.K.8.c]

10. Students will be able to analyze and interpret data using the appropriate methods. [Core standards: II.K.7.c, II.K.7.f]

11. Students will be able to develop assessment surveys that are valid and reliable. [Core stds.: II.K.7.b, 11.K.7.d, II.K.7.e; Student Affairs Specialty standard: B.5]

12. Student will be able to develop appropriate interview questions and protocol for conducting individual and group interviews. [Core stds.: II.K.7.f, II.K.7.g, II.K.8.b]

13. Students will be able to discuss the use of benchmarking and standards. [Student Affairs Specialty standard: B5]


2001 CACREP Standards Addressed in EPC 601 (Assessment in Student Affairs)

• (Core Standards)

7. ASSESSMENT - studies that provide an understanding of individual and group approaches to assessment and evaluation, including all of the following:

a. historical perspectives concerning the nature and meaning of assessmentb. basic concepts of standardized and non-standardized testing and other assessment

techniques including norm-referenced and criterion-referenced assessment, environmental assessment, performance assessment, individual and group test & inventory methods, behavioral observations, and computer-managed and computer-assisted methods;

c. statistical concepts, including scales of measurement, measures of central tendency, indices of variability, shapes and types of distributions, and correlations;

d. reliability (i.e., theory of measurement error, models of reliability, and the use of reliability information);

e. validity (i.e., evidence of validity, types of validity, and the relationship between reliability and validity;

f. age, gender, sexual orientation, ethnicity, language, disability, culture, spirituality, and other factors related to the assessment and evaluation of individuals, groups, and specific populations;

g. strategies for selecting, administering, and interpreting assessment and evaluation instruments and techniques in counseling; and

i. ethical and legal considerations.

8. RESEARCH AND PROGRAM EVALUATION - studies that provide an understanding of research methods, statistical analysis, needs assessment, and program evaluation, including all of the following:

b. research methods such as qualitative, quantitative, single-case designs, action research, and outcome-based research;

c. use of technology and statistical methods in conducting research and program evaluation, assuming basic computer literacy;

d. use of research to improve counseling effectiveness; andf. ethical and legal considerations.

• (Student Affairs Specialty) CONTEXTUAL DIMENSIONS of Student Affairs [B.4, B.5] 4. methods of needs assessment that are applicable to college student populations; and

5. systematic assessment techniques that are applicable to higher education environments.

Some of these standards are applied in other courses as well.


Course Expectations

1. Readiness to Engage in Learning and Teaching

A) Readiness to engage means that you have completed the assigned reading; have developed a point of view on the author’s perspective; and, will share the meaning you have made of the reading during classroom and/or assessment group discussion.

B) Learning and Teaching emerge from your active participation. In other words you bring your insights and contribute them to the class discussion. It also means engaging actively with the thoughts of your colleagues – listening carefully, responding openly to, and making connections among others’ contributions.

2. You will attend class unless it is impossible to do so.

Much of the practical application of the assessment knowledge you will be developing will occur during the class. If you miss a class you will miss at least one component of the assessment process. It will be difficult to develop assessment skill without the opportunity to learn from doing.

3. Application of Group Work Skills

You will be assigned to an assessment planning group. Group work can be challenging due to differences in learning styles, interpersonal styles, commitment level, etc. As individuals who have begun developing counseling skills including working with groups I expect the group to utilize their collective skills in the development and maintenance of an effective group process.

4. Required assignments and due dates are described below.

You are expected to submit your papers electronically to me by the time class begins on the due date. I reserve the right to alter or eliminate credit for late assignments that had not been negotiated in advance. If you are unable to meet a deadline, it is your responsibility to request an extension in writing by providing a justification and an alternate due date. All assignments should be in APA style, typed/word processed, double-space (for hard copy), using non-sexist language (when possible). As a professional you will be expected to express your ideas clearly and concisely in writing; therefore, spelling and grammatical errors will be taken into consideration in the grading. You will receive feedback on your papers electronically.

Methods of Instruction:

The following methods of instruction are used in teaching and learning from this course: lecture, structured classroom exercises and out-of-class assignments that promote further critical thinking and analytical skill development within a small group environment. Active class participation by students is expected.


Final Report Process

1. You will keep a record of the group’s assessment planning process. The record will reflect your thinking about the phases of assessment process whether or not you agree with the direction of the group.

2. The format for the report will reflect the Phases of Assessment that you will learn about through the discussion, the instructor’s presentations, and your group work. The phases of assessment constitute the framework for the course and the topical outline for your report.

3. For each phase you will summarize your assessment group’s process, considerations, and decisions.

4. After reporting on the group’s work you will indicate the extent to which you agree or disagree and your rationale.

5. If you disagree you will provide your preferred alternative.

6. Work is to be done individually and confidentially – you should not share nor discuss your report with others.

7. You will be submitting your report electronically and may be asked to submit portions throughout the semester so that I can comment.

8. The report will be evaluated on the quality/complexity of your thinking, the extent to which you integrate your thinking with that of the authors you have read (required and non-required), the accurate use of terminology (you are expected to use terms as we have defined them in class), thoroughness, and overall assessment plan.

9. Quality not quantity should be your guiding principle.

Evaluation of Work:

Classroom Participation* 20%Assessment Group Peer Evaluation 20%Two Exams 20% eachAssessment Project Report 20%

* Classroom participation will be evaluated based upon Course Expectation No. 1.

A plus/minus grading system will be utilized in determining grades. Each assignment will be based on a 100-point scale: 93 – 100 A

90 – 92 A-88 – 89 B+83 – 87 B80 – 82 B-78 – 79 C+73 – 77 C72 and below: not passing for a graduate course


Course OutlineDate TopicWeek 1August 27

Introduction to course syllabus and to assessment history, purpose, and terminology.

Week 2September 3

Assessment Ethics and Good Practice

Week 3September 10

Deciding when to and when not to assessPhases of AssessmentStating an assessment question

Week 4September 17

Purposes (types) of assessment

Week 5September 24

Where to get information and other issues to be considered.

Week 6October 1

Methodology vs. Methods

Week 7October 8

Qualitative and Quantitative Sampling

Week 8October 15

EXAMCollecting Qualitative Data

Week 9October 22

Pilot interview protocolShare experiences

Week 10October 29

Analyzing Qualitative Data Collecting

Week 11November 5

Collecting Quantitative Data Selecting or Developing Instruments

Week 12November 12

Develop and Pilot SurveyShare experiences

Week 13November 19

Analyzing Quantitative Data

Week 14November 26

Preparing Assessment Reports

Week 15December 3

Benchmarking Standards

Week 16December 10

Assessment group presentations

Week 17December 17

Final Exam


Required Reading:

Select reading on the electronic reserve. Password is: 4891

There are no required texts.

August 27

National Association of Scholars (2008, July 15). Rebuilding campus community: The wrong imperative. Retrieved August 4, 2008, from http://www.nas.org/polStatement.cfm

Hersh, R. H., & Keeling, R. P. (2008, August 1). 'Liberal education': Lessons beyond the classroom. Chronicle of Higher Education, 54(47), A64. Electronically downloaded from http ://chronicle.com/weekly/v54/i47/47a06401.htm (and sent to you electronically on Aug. 13 from Dr. Simon)

Ewell, P. T. (2002). An emerging scholarship: A brief history of assessment. In T. W. Banta (Ed.), Building a scholarship of assessment (pp. 3-25). San Francisco: Jossey-Bass.

Davis, B. G. (1994). Demystifying assessment: Learning from the field of evaluation. In J. S. Stark & A. Thomas (Eds.), Assessment and evaluation, ASHE Reader Series, (pp 45-57). Boston: Pearson

Schuh, J. H., & Upcraft, M. L. (1998). Facts and myths about assessment in student affairs. About Campus, 3(4), 2-8.

Upcraft, M. L., & Schuh, J. H. (2002, March-April). Assessment vs. research: Why we should care about the difference. About Campus, 16-20.

September 3

ACPA (2006). ASK Standards. Washington D.C.: Author. (skim)

Banta, T. W., Lund, J. P., Black, K. E., & Oblander, F. W. (1996). Assessment in practice: Putting principles to work on college campuses (pp. 1-63). San Francisco: Jossey-Bass. (Introduction, Principle and Principle in Context sections only)

Cannon, H. J. (1985). Ethical problems in daily practice. In H. J. Cannon, & R. O. Brown (Eds.), Applied ethics in student affairs. New Directions in Student Services. No. 30, San Francisco: Jossey-Bass.Note: Contains cases studies referenced by the Kitchener chapter.

Kitchener, K. S. (1985). Ethical principles and ethical decision making in student affairs. In H. J. Cannon, & R. O. Brown (Eds.), Applied ethics in student affairs. New Directions in Student Services, 30, San Francisco: Jossey-Bass.

http://chronicle.com/weekly/v54/i47/47a06401.htm

http://www.nas.org/polStatement.cfm


September 10

Bresciani, M. J., Zelna, C. L., & Anderson, J. A. (2004). Assessing student learning and development: A handbook for practitioners (pp. 8-18). Washington, D.C.: National Association for Student Personnel Administrators.

Woosley, S. A. & Knerr, A. R. (2005, June 21, October 12, November 23). Common missteps in assessment building blocks and how to avoid them (Part I, II, III). Retrieved August 22, 2006, from http://www.naspa.org/membership/mem/nr/PrinterFriendly.cfm?id=1502.

Upcraft, M. L., & Schuh, J. H. (2000). Assessment in student affairs. In M. J. Barr & M. K. Desler (Eds.). The handbook of student affairs administration (2nd ed., pp. 257-262). San Francisco: Jossey-Bass.

September 17

Upcraft, M. L. & Schuh, J. H. (1996). Assessment in student affairs. San Francisco: Jossey-Bass. pp 113-120, 126-132, 148-154, 157-158, 166-170, 189-200.

September 24

Review prior readings to identify factors to be considered in assessment planning.

Evans, N. J. (1985). Needs assessment methodology: A comparison of results. Journal of College Student Personnel, 26(2), 107-144.

October 1

Schuh, J. H. & Upcraft, M. L. (2001). Assessment practice in student affairs: An application manual. San Francisco: Jossey-Bass. (pp. 27-41)

Patton, M. J. (1991). Qualitative research on college students: Philosophical and methodological comparisons with quantitative approach. Journal of College Student Development, 32, 389-396.

Manning, K. (March 1992). A rationale for using qualitative research in student affairs. Journal of College Student Development, 33, 132-136.


October 8

Jones, R. J., Torres, V., & Arminio, J. (2006). Negotiating the complexities of qualitative research in higher education (chapter 3). New York: Routledge. (skim)

Sampling methods for quantitative assessment: http://www.statpac.com/surveys/sampling.htm

Random number calculator http://stattrek.com/tables/random.aspx#calculator

October 15

Popham, W. S. (1993). Focus groups: A potent education qualitative data-gathering procedure (pp. 194-215). Needham Heights, MA: Simon & Schuster.

Bresciani, M. J., Zelna, C. L., & Anderson, J. A. (2004). Assessing student learning and development: A handbook for practitioners (pp. 48-57). Washington, DC: National Association for Student Personnel Administrators.

October 22

TBD

October 29

TBD

November 5

Raphael, A. & Lloyd, J. M. (February 25, 2005). The use of self-report in assessment: What is the literature saying? NETRESULTS 2/25/2005. Retrieved July 2005, from http://www.naspa.org/membership/mem/nr/PrinterFriendly.cfm?id=1475.

Ory, J. C. (1994). Suggestions for deciding between commercially available and locally developed assessment instruments. In J. S. Stark & A. Thomas (Eds.) Assessment & program evaluation. ASHE Reader Series. Boston: Pearson Custom Printing.

Rea, L. M., & Parker, R. A. (1997). Designing and conducting survey research: A comprehensive guide (pp. 45-64). San Francisco: Jossey-Bass.

Molasso, W. R. (Fall 2005). Ten tangible and practice tips to improve student participation in web surveys. Student Affairs Online, (6) 4. Retrieved January 11, 2006, from http://studentaffairs.com/ejournal/Fall_2005/ StudentParticipantinWebSurveys.htm.


November 12

TBD

November 19

Astin, A. W. (1993). Assessment for excellence: The philosophy and practice of assessment and evaluation in higher education (pp. 94-127) (American Council on Education Series on Higher Education). Westport, CT: Oryx Press.

Popham, W. J. (1993). Educational evaluation (pp. 261-282). Needham Heights, MA: Simon & Schuster.

November 26

Worthen, B. R. (1987). Educational evaluation (pp. 341-368). White Plains, NY: Longman.

December 3

CSUN Department of Institutional Research web site http://irqry.csun.edu:8080/openweb/interim/index.html

http://www.cas.edu: History of CAS – CAS Context – Self-Study Process (skim)

http://www.wascsenior.org/wasc (skim the accreditation process)

December 10

None


Other resources that you might find useful in developing your understanding of assessment:

Broido, E. M., & Manning, K. (2002, July/August). Philosophical foundations and current theoretical perspectives in qualitative research. Journal of College Student Development 43(3), 434 – 445.

Bryan, W. A., & Mullendore, R. H. (1991). Operationalizing CAS Standards for program evaluation and planning. In W. A. Bryan, R. B. Winston, Jr., & T. K. Miller (Eds.), New directions for student services: Using professional standards in student affairs (no. 53). San Francisco: Jossey-Bass.

Council for the Advancement of Standards for Student Services/Development Programs (CAS) (2003). The book of professional standards for higher education. Washington, DC: CAS.

Desler, M. K. (2000). Translating theory and assessment results to practice. In M. Barr, M. K. Desler, & Associates (Eds.), The handbook of student affairs administration. San Francisco: Jossey-Bass.

Erwin, T. D. (1991). Assessing student learning and development. San Francisco: Jossey- Bass.

Erwin, T. D., Scott, R. L., & Menard, Jr, A. J. (1991). Student outcome assessment in student affairs. In T. K. Miller, R. B. Winston, Jr. and Associates (Eds.), Administrative leadership in student affairs: Actualizing student development in higher education (pp. 741-763). Muncie, IN: Accelerated Development.

Hanson, G. R. (1982). New directions for student services: Measuring student development (no. 20). San Francisco: Jossey-Bass.

Hood, A. B., & Johnson, R. W. (1997). Assessment in counseling: A guide to the use of psychological assessment procedures. Alexandria, VA: American Counseling Association.

Huebner, L. A., & Lawson, J. M. (1990). Understanding and assessing college environments. In D. G. Creamer and Associates (Eds.), College Student Development: Theory and practice for the 1990s (pp. 127-151). Alexandria, VA: American College Personnel Association.

Hyman, R. E., Beeler, K. J., & Benedict, L. G. (1994). Outcome assessment and student affairs: New roles and expectations. NASPA Journal, 32 (1).

Jacobi, M, Astin, A. & Ayala, Frank, Jr. (1987). College student outcome assessment: A talent development perspective (ASHE-ERIC Higher Education Report, No. 7). Washington, D.C.: Association for the Study of Higher Education.


Jacoby, B., & Thomas, Jr., W.L (1991). Professional standards and the accreditation process. In W. A. Bryan, R. B. Winston, Jr., & T. K. Miller (Eds.), New directions for student services: Using professional standards in student affairs (no 53, pp. 19-28). San Francisco: Jossey-Bass.

Jones, S. R. (2002,July/August). (Re)Writing the word: Methodological strategies and issues in qualitative research. Journal of College Student Development, (43)3, 461-473.

King, P. M. (1990). Assessing development from a cognitive developmental perspective. In D. G. Creamer and Associates (Eds.), College Student Development: Theory and practice for the 1990s (pp. 81-98). Alexandria, VA: American College Personnel Association.

Kuh, G.D. (1982). Purposes and principles of needs assessment in student affairs. Journal of College Student Personnel, 23(3), 202-209.

Kuh, G. D., Whitt, E. J. & Shedd, J. D. (1987). Student affairs work, 2001: A paradigmatic odyssey. Washington, DC: American College Personnel Association.

Kuh, G. D., & Andreas, R. E. (September, 1991). It’s about time: Using qualitative methods in student life studies. Journal of College Student Development, 32, 397-405.

Mable, P. (1991). Professional standards: An introduction and historical perspective. In W. A. Bryan, R. B. Winston, Jr., & T. K. Miller (Eds.), New directions for student services: Using professional standards in student affairs (no. 53, pp. 5-18). San Francisco: Jossey-Bass.

Magolda, P. M. (2003, November/December). Saying good-bye: An anthropological examination of a commencement ritual. About Campus, 44(6), 779-796.

Miller, T. K., & Winston, Jr., R. B. (1990). Assessing development from a psychosocial perspective. In D. G. Creamer and Associates (Eds.), College Student Development: Theory and practice for the 1990s (pp. 99-126). Alexandria, VA: American College Personnel Association.

Palomba, C. A., & Banta, T. W. (1999). Assessment essentials: Planning, implementing and improving assessment in higher education. San Francisco: Jossey-Bass.

Pascarella, E. T., & Upcraft, M. L. (1999). Assessment strategies for student learning imperative. In E. J. Whitt (Ed.), Student learning as student affairs work: Responding to our imperative [monograph series 23, pp. 63-79]. Washington, DC: National Association of Student Personnel Administrators.

Pope, R. L., Reynolds, A. L., & Mueller, J. A. (2004). Multicultural competence in student affairs (pp. 98-119). San Francisco: Jossey-Bass.

Posavac, E. J., & Carey, R. G. (1989). Program evaluation: Methods and case studies. Englewood Cliffs, NJ: Prentice Hall.


Upcraft, M. L., & Schuh, J. H. (2002). Assessment vs. research: Why we should care about the difference. About Campus: Enriching the student learning experience, 7(1), 16-20.

Winston, Jr., R. B., & Miller, T. K. (1994). A model for assessing developmental outcomes related to student affairs programs and services. NASPA Journal, 32 (1),

Winston, R. B., Jr., & Moore, W. S. (1991). Standards and outcome assessment: strategies and tools. In W. A. Bryan, R. B. Winston, Jr., & T. K. Miller (Eds.), New directions for student services: Using professional standards in student affairs (no. 53, pp. 63-82). San Francisco: Jossey-Bass.

Witkin, R. B., & Altschuld, J. W. (1995). Planning and conducting needs assessment: A practical guide. Thousand Oaks, CA.: Sage.

Worthen, B. R., Sanders, J. R., & Fitzpatrick, J. L. (1997). Program evaluation: Alternative approaches and practical guidelines. New York: Longman Publishers.

individual and group assessment€¦ · web viewresearch and program evaluation - studies that...

Documents