approaches to assessment of the sophomore-year experience: a panel discussion

Approaches to Assessment of the Sophomore-Year Experience:

A Panel Discussion

Institute on Sophomore Student SuccessColumbia, SC

April 12-14, 2013 Jennifer R. Keup

Session Layout

• Brief assessment primer: Jennifer Keup• Panel of assessment colleagues to start

discussion:– Facilitator: Jennifer Keup, National Resource

Center– Panelists:• Laurie Schreiner, Azusa Pacific University• Jimmie Gahagan, USC• David Sneed, Belmont University

ASSESSMENT AND THE SOPHOMORE-YEAR EXPERIENCE:

STARTING THOUGHTS AND

INITIAL IDEAS

What is Assessment?

• “The systematic collection, review & use of information about educational programs for the purposes of improving student learning and development.”

• “Any effort to gather, analyze, and interpret evidence, which describes institutional, divisional, or agency effectiveness.”

• “The process of gathering & discussing information from multiple & diverse sources in order to develop a deeper understanding of what students know, understand, & can do.”

CREATEACTION

GATHER LEARNFROM

USE

Is sophomore assessment different than other student assessment?

GENERALLY, NO. Good assessment

practices are

constant, but…

• Specific sample• Developmentally

appropriate outcomes• Can be one point,

starting point, or end point

• One point in process

Systematic Cycle of Assessment

• Identify learning outcomes• Gather evidence• Document what you already know• Identify appropriate methods of measurement• Acknowledge methodological limitations but focus

on results• Interpret evidence• Implement change

Identify Learning Outcomes• Characteristics:– Focuses on product rather than process– Must be measurable– Detailed and specific– Includes action verbs– Aspirational– Meaningful (i.e., must be useful not just interesting)

• Think-Pair-Share: What are some examples of sophomore learning outcomes?

Examples of Outcomes• Persistence• Analytical & critical thinking skills• Knowledge of university requirements• Interpersonal/Communication skills• Creation of a sense of community• Identity exploration and development• Develop educational career goals/Declare a major• Practical competence• Knowledge integration and application

How to involve faculty?

Examples of Outcomes (cont.)

• Multicultural competence• Civic engagement/Democratic citizenship• Ability to identify, seek, & utilize organizational

resources and student programs• Physical health & emotional wellness• Leadership skills• Moral and ethical development• Cognitive complexity• Academic achievement

Any others?

Gather Evidence• Needs assessment• Utilization data• Criterion-referenced– What are the criteria?

• Benchmarking– Who are your true or aspirational peers?– How to get their data?

• Trends• Value added

INPUTS

ENVIRONMENTS

OUTCOMES

• What data do you already have?– Who is collecting it?– Is it being used? If so, for what purposes?– Where is it being housed?– Are there plans for future data collection?

• How can you use data that has already been collected for your assessment needs?

• Challenge: List as many institutional sources of data on sophomores as you can.

Document What You Already Know

“Data lust turns into data dust”

• Admissions/Registrar’s data• Institutional participation in national surveys of first-

year students• Program/course evaluations• Utilization statistics• Program review reports• Satisfaction surveys• Campus/student profile• Focus group findings

Potential Sources of Data

Potential Data Sources (cont.)

• Home-grown IR surveys• SA departmental surveys, focus groups, or student

interviews• Self-study processes/CAS• Accreditation reports• Grants• Strategic planning• Student & faculty interview transcriptions• Other sources?

A series of important decisions

Identify Appropriate Methods of Measurement

• Formative assessment – Normative assessment• Quantitative methods – Qualitative methods• Direct measures – Indirect measures• National (“off-the-shelf”) instrument – Locally developed

instrument

Remain connected to learning outcomes

Acknowledge Methodological Limitations but Focus on Results

• Do what you can to minimize limitations but do not perseverate in the assessment planning stage

• There is always something to learn from assessment activities and findings

• Good assessment will typically raise more questions than it answers

• There is no “perfect” assessment plan; JUST GET STARTED & KEEP LEARNING!

Interpret Evidence

“You can’t fatten a pig by weighing it.”

• What conclusions can you draw from the data? How did you do?

• What other questions do the data raise? What other information might you need?

• What are the implications for practice or what policy decisions would you make?

Interpret Evidence

• Don’t interpret in isolation– Form an assessment committee– Ad hoc groups to process results– Benefits:

• More holistic thinking• Create buy in• Tool for dialogue

• Use multiple lenses (i.e. benchmarking, longitudinal, value added, quantitative, qualitative, etc.)

Implement Change• Purpose of assessment is to CREATE ACTION to:– Continue effective practice, or– Initiative change and improvement

• Effective dissemination of results– Who needs to know this information?– How do they need to know the information?

• Changes will be connected to new outcomes and assessment plans

Panelists

• Introduce themselves• Identify their role/experience with

sophomore-year assessment• Indicate the biggest challenge they see to

sophomore-year assessment• Best advice they know/have received with

respect to sophomore-year assessment• Open up for questions from audience

Discussion Questions: Challenges & Strategies

What are the biggest challenges to assessment of second-year students and

sophomore initiatives?

What are some strategies for overcoming these challenges?

Discussion Questions:Identifying Outcomes

How do I determine the best outcomes on which to focus for assessment of

sophomores?

Which outcomes would generate the greatest “buzz” on our campus?

Discussion Questions

Access to Population: How do I most effectively access sophomores for

assessment activities?

Instruments & Methods: What assessment tools already exist that are appropriate for

assessment of the second year?

Discussion Questions

Existing Data: What is our most important data point that we are likely not using?

Measurement: How do I create meaningful measures of complex outcomes (e.g.,

multicultural competence, leadership, civic engagement)?

Career Planning

Undergraduate Research

Learning Community

Class Events

Leadership Development

Discussion Questions:Indirect vs. Direct Measures

How do I best “sell” assessment outcomes generated from indirect measures?

How do I identify efficient direct measures for what is currently measured indirectly?

Discussion Questions:Creation of an Assessment Plan

Who are key campus partners with whom I should be communicating about

assessment?

How do I effectively integrate sophomore-year assessment efforts to a larger cycle

that includes the first-year and senior-year data collection efforts?

ReferencesAstin, A.W. (1991). What Matters in College. San Francisco:

Jossey-Bass.Gahagan, J., Dingenfelder, J., & Pei, K. (2010). A Faculty and

Staff Guide to Creating Learning Outcomes. Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience and Students in Transition.

Gansemer-Topf, A.M. & Pattengale, J.A. (2010). Evaluating Second-Year Programs. In M.S. Hunter, B.F. Tobolowsky, J.N. Gardner, S.E. Evenback, J.A. Pattengale, M.A. Schaller, L.A. Schreiner & Associates, Helping Sophomores Succeed (pp. 234-247).

Other Assessment ResourcesBresciani, M.J., Zelna, C.L., & Anderson, J.A. (2004). Assessing Student

Learning and Development. Washington, DC: National Association of Student Personnel Administrators.

Friedman, D.B. (2012). The First-Year Seminar: Designing, Implementing, and Assessing Courses to Support Student Learning and Success: Vol. V. Assessing the First-Year Seminar. Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience and Students in Transition.

Maki, P. (2004). Assessing for Learning: Building a Sustainable Commitment Across the Institution. Sterling, VA: Stylus.

Strayhorn, T.L., Creamer, D.G, Miller, T.K. (2006). Frameworks For Assessing Learning And Development Outcomes. Washington, DC: Council for the Advancement of Standards in Higher Education.

Upcraft, M.L. & Schuh, J.H. (1996). Assessment in Student Affairs: A Guide for Practitioners. San Francisco: Jossey-Bass.

Other questions for discussion?

Thanks!