california community colleges registrars and admissions officers (caccrao) development of meaningful...
TRANSCRIPT
California Community Colleges Registrars and Admissions Officers
(CACCRAO) Development of Meaningful Learning Outcomes
Dotti Cordell BS, MPH, BSNDirector of Student Health Services
San Diego City CollegeMay 4, 2009
rekindle
reorient
REMEMBER relearn rechargerecharge REEXAMINEREEXAMINE
reviewreview A Challenge to ...A Challenge to ...
RECONNECTRECONNECT
revisitrevisit restore recommitrecommit REDISCOVERREDISCOVER
And Restart?...And Restart?... redirectredirect
replenishreplenish rechallenge recall rejuvenate
recapture
Presentation Objectives Review the process of
Outcomes Assessment Clarify misconceptions
relating to outcomes in Student/Administrative Services
Explain the Six Column Model utilized at SD City College Equip you with an
organized systematic knowledge base in this sometimes unfamiliar or uncomfortable territory
Outline potential trouble spots
Choice and Attitude
I hope that you can turn what might be a negative external mandate into a positive internal challenge.
I challenge you to view assessment as an opportunity for admission program improvement and a means to seek the best possible way to serve students.
Do You Feel A Time-Crunch?
What do I need to know?What does my department
need to know?What do I do?
Don’t Panic
You and your department are probably already doing it. You just may not know it. Or you may not be documenting the process.
Or you think there is a perfect way to do it and you are afraid to jump in.
Or it does not seem “natural” and seems to be a large additional work load.
Quality: What Is It?
Quality of service: the concept does not possess substance or mass. Cannot place quality on the table to
analyze or study. Any definition of quality almost always
seems deficient Quality is in the eye of the beholder. Quality definitions reflect the tastes
and biases of those doing the defining.
Quality
A quality admissions program may be one in which services for students faculty and administration increases the likelihood of desired outcomes and is consistent with current professional knowledge
11
Performance
+ ServiceWrong Things Done RightInefficient admission process or system but people do an excellent job of entering the data
Right Things Done Right
Wrong ThingsDone WrongInefficient system to enroll students and staff using system make many errors entering the data
Right ThingsDone WrongState-of-the-art equipment used incorrectly by staff
(-)
P
R
O
C
E
S
S
+(-)
QualityQualityNaked assertions of high quality
admissions, records, evaluations, counseling or any college department or program is no longer sufficient. Consumers, society, accrediting agencies and campus administrators hold us accountable to provide objective, quantifiable standards for quality student and service outcomes.
Admissions Assessment and Improvement
You know, WE know, that admission departments help maintain the institution and are ESSENTIAL to operations.
Assessment will help demonstrate that each staff member in your admissions/ registration departments are endeavoring to do their jobs better every day.
Assessment Principles
Assessment is NOT a one-time activity; it is evolutionary, ongoing, incremental.
Over time, assessment efforts should become more comprehensive, systematic and integrated.
Assessment efforts must be compatible with the institution's mission
Diane Ramirez of College of the Desert and Merrill Deming at Crafton Hills College
How Can You/We Do It All?
FERPABudget Crises Increasing
Demand for Services
Day-to-Day Operations OUTCOMES
MEASUREMENT?
What is Assessment?
Assessment is an ongoing process aimed at understanding/improving student learning and campus services and processes It serves to clarify our values Sets criteria and standards for learning and services We gather data to see how well performance
matches standards Assessment is deciding what YOU as THE subject
matter experts value and wish to measure and then using the resulting information to document and improve performance
Key Questions
What are we trying to do? What is it that admissions should accomplish? How is success defined? How successful are our students? How well have they learned?
How do we know? How well are we performing our functions?
How do we know? How do we use that information to continually
improve? Does the improvements we make work and do we
maintain the gain?Adapted From M. Bresciani, Director of Assessment SDSU and from CUPR Guidelines
The answers to these questions form the essence of what is called assessment.
Student Learning OutcomesDefinition
The knowledge, skills, abilities, attitudes, habits of mind or competencies that students have at the completion of a
Course Program Activity or
interaction The presence and quality of
which can be measured and evaluated.
Categorization of Student Learning Outcomes
Complex Cognitive Skills Knowledge Acquisition Interpersonal
Development Social Development Academic Achievement Persistence Practical Competence Civic Responsibility
National Assoc. of Student Personnel Administrators, Upcraft 1999
Assessment to Improve Admissions
Processes/Services Redesign services based upon best
practices Use information technology to improve
access to information and to support decision-making
Improve workforce knowledge and skills Develop effective teams within and
outside the department(s) Coordinate referrals Measure performance and outcomes
Why Do We Use AnAssessment Framework?
Model provides common language and a consistent framework
Allows celebration of small wins Demonstrates accomplishments
Is readily explained, one page Model clearly shows linkages with larger
institutional competencies and college mission
San Diego City College’s Six Column Model
Credit to Jim Nichols – We have adapted his work with over 380 colleges and are using an expansion of his model.
Columnar model--helps individuals see linkages between levels of organization.
City College Six Column Assessment Model
Column 1: Statement of Institutional Purpose—for City this is our Mission and our Institutional Priorities
Column 2: Institutional Competencies Communication/Interpersonal Skills Critical thinking Analyses/Computation Cultural Sensitivity/Global Awareness Information Management/ Literacy Personal Responsibility Civic and Environmental Responsibility
Column 3: Programmatic Student Learning Outcomes Column 4 : Means of Assessment and the Criteria for Success Column 5: Summarizes data collected (Results) Column 6: Designated to demonstrate how you will use
results to improve the program
Note: Linkages between the 6 columns is essential
Mission
Your departmental mission is your reason for being, the motivation for doing this work, the soul of the department
Column One
To guide City faculty and staff in formulating outcomes, column one is comprised of the City College mission statement and our college priorities
Column Institutional Competencies
Communication/Interpersonal Skills
Critical thinking
Analyses/Computation
Cultural Sensitivity/Global Awareness
Information Management/ Literacy
Personal Responsibility
Civic and Environmental Responsibility
Admissions/Registration 2009 Outcome
College Mission and Priorities
COLUMN 2:
Institutional CompetenciesCommunication/Interpersonal SkillsCritical thinkingAnalyses/ComputationCultural Sensitivity/Global AwarenessInformation Management/ LiteracyPersonal Responsibility Civic and Environmental Responsibility
Column :Development of Departmental/Programmatic SLO’s
Departmental staff members convene to discuss and reach consensus on the focus of assessment for the coming semester/year.
Developing Programmatic Student Learning OR Administrative Unit
Outcomes
Collect and review documents that describe the program (catalog, your program’s mission)
Examine outcome statements from similar programs at other institutions
Review assessment criteria of WASC accrediting body
Review criteria from your professional organization--CACCRAO
Collect and review departmental materials
Brainstorm with your staff and utilize the short list/long list concept
Long and Short List Concept
Now choose 1-2 to study this semester/year. Have your department collaboratively discuss and reach consensus on what is important THIS year or semester:
1.2.
Make a LOOONG list of all the learning or administrative outcomes of importance to your department:1. 2. 3.…..42.
Writing Outcomes Uses clear, observable and measurable verbs
CompileCompile Arrange Arrange Classify Classify AnalyzeAnalyze Identify Identify Operate Operate DesignDesign Solve Write Solve Write ApplyApply Differentiate Calculate Differentiate Calculate DemonstrateDemonstrate Formulate Formulate Compose Compose ExplainExplain Predict Predict Assess Assess CompareCompare Estimate Estimate Critique Critique
The following verbs are relatively vague states of mind. They are The following verbs are relatively vague states of mind. They are very difficult/ impossible to measure:very difficult/ impossible to measure: Know Understand Appreciate Learn
Credit to Gavilan College for this content
Admissions/Registration 2009 Outcome
College Mission and Priorities
Institutional Competencies
Communication/Interpersonal Skills
Critical thinking
Analyses/Computation
Cultural Sensitivity/Global Awareness
Information Management/ Literacy
Personal Responsibility
Civic and Environmental Responsibility
COLUMN 3
Programmatic Learning or Administrative Outcome
After an encounter in the Admissions office or through the website students will _____________________________________________
Or the Admissions Dept. will_______________________________________________
Column 4—Means of Assessment and Criteria for Success
“Rubber hits the road” in this column Here you are defining the specifics of
measurement and precisely what you meant in column 3. You are describing the context for the outcome. What do you mean that you want the student to be better at availing him/herself of campus resources? Able to navigate the catalog?
It is in this column you must be very clear on the specifics of measurement, and at what point you will consider yourself successful. 78% of the students will…..etc..
Admissions/Registration 2009 Outcome
College Mission and Priorities
Institutional Competencies
Communication/Interpersonal Skills
Critical thinking
Analyses/Computation
Cultural Sensitivity/Global Awareness
Information Management/ Literacy
Personal Responsibility
Civic and Environmental Responsibility
COLUMN 3
Programmatic Learning or Administrative Outcome
After an encounter in the Admissions office or through the website students will _____________________________________________
Or the Admissions Dept. will decrease turnaround time for ____ filing.
COLUMN 4:
Means of Assessment and Criteria for Success
Using the 2009 Admissions satisfaction survey 72% of students will demonstrate 3.5 or above on the satisfaction scale of the admissions dept.’s ability to assist them with ________.
Using computer data from_____the Admissions dept. will demo a 12.6% decrease in turn-around time for ______ in the 2009-2010 school year.
Column 5: Log results obtained and compare to the criteria for success that the department or program set for itself
Column 6: Compare how actual results stacked up against the criteria for success. Will the department rotate the outcome off the list as “accomplished” and not problematic, or does it see the issue as an opportunity for programmatic improvement?
What are the departmental actions to improve? NOW you have “closed the loop” and the entire goal
of this assessment effort comes full cycle: you now have data and actions to support program improvement efforts.
Admissions/Registration 2009 Outcome
College Mission and Priorities
Institutional CompetenciesCommunication/Interpersonal Skills
Critical thinking
Analyses/Computation
Cultural Sensitivity/Global Awareness
Information Management/ Literacy
Personal Responsibility
Civic and Environmental Responsibility
COLUMN 3
Programmatic Learning or Administrative Outcome
After an encounter in the Admissions office or through the website students will _____________________________________________
Or the Admissions Dept. will decrease turnaround time for ____ filing.
COLUMN 4:
Means of Assessment and Criteria for Success
Using the 2009 Admissions satisfaction survey 72% of students will demonstrate 3.5 or above on the satisfaction scale of the admissions dept.’s ability to assist them with ________.
Using computer data from_____the Admissions dept. will demo a 12.6% decrease in turn-around time for ______ in the 2009-2010 school year.
COLUMN 5: Results
Using the 2009 Admissions satisfaction survey 64% of students demonstrated 3.5 or above on the satisfaction scale of the admissions dept.’s ability to assist them with ________.
Using computer data from_____the Admissions dept. demo’ed a 4% decrease in turn-around time for ______ in the 2009-2010 school year.
COLUMN 6: Use of Results for Program Improvement
With these results, we have decided to:
Do nothing because our findings are so good. We will move on to something else.
OR
We will do ________ to improve our findings and will re-measure again in _____ 2009/2010.
Column 6: Closing the LoopKey Questions
How well are we achieving the admissions/registrar program or department goals?
How do we know? How can we use
data/information to continually improve the processes and systems of the department?
Student Learning Outcomes And Administrative Unit Outcomes:
Clarification: FAQ’s
What is a student learning outcome (SLO)?Knowledge, skills, abilities, attitudes, habits of mind that students have at the completion of a series of activities or interactions--the presence and quality of which can be measured and evaluated
What is an AUO (Administrative Unit Outcome)?An AUO describes what an “Administrative Unit” such as your Admissions Depts. will accomplish or do in the provision of their services. The focus of AUOs is services. How you know you are accomplishing the purpose of Admissions, Records, Registration? THE FOCUS IS ON IMPROVEMENT IN SERVICES PROVIDED TO SUPPORT STUDENT LEARNING.
Why is it important to include student support services in this work?Key to understanding the importance of this initiative is the premise that the entire campus supports and is crucial to student learning. As such it requires a commitment of the entire campus as part of a comprehensive Institutional Effectiveness program
Why should I engage in outcome work if I’m not in an instructional area of the campus?Student learning experiences most definitely do not just occur in the classroom. Some of the most important learning on campus occurs outside of the classroom. It is important that the campus participate to demonstrate learning outside the classroom and the efficiency of campus operations.
Similarities between SLOs and AUOs
Both focus on improvement (in either student learning or administrative functioning)
Both contribute to and are part of the quality picture of the entire campus
Both use assessment data to effect improvement
Examples Of Programmatic Outcomes— Admissions
Admissions will shorten the response time for _____from 3 days to 2.5 days .
Over the counter requests for __________will be delivered in ____minutes/hours versus ____minues/hours.
94% of students will articulate a high level of satisfaction with the overall registration process as evidenced by an increase from 2.7 to 3.5 on a 5 point satisfaction scale.
Or 85% of students will demonstrate a high( remember to define what you mean by “high”) level of satisfaction on exit interviews from the admissions dept.
Admissions office complaints or dissatisfaction reports will decrease from ___ per semester to <2 per semester.
Admissions professionals will contribute to the overall campus commitment to a diverse student body by increasing_____________.
The Admissions department will institute the following interventions this semester to increase the % students having a valid ID card prior to “x” date of the semester from ___% to ___%:
Admissions will demonstrate an increase in students successfully( how is successfully defined?) navigating the college catalogue from ___% to ___% by______________________________.
To improve the learning outcome related to services referral and campus networking---As reported on an exit questionnaire from the Admissions department, students will accurately report 3 of 5 departments necessary to meet their needs and educational goals.
Admissions will provide accurate class enrollment data_____________.
Key Foundational Assessment Factors
at City College Not all outcomes are student learning outcomes, some may be
administrative outcomes which are essential to, and form the foundation upon which, student learning and success are built
This whole process is directed towards incremental program improvement, not individual evaluation
The “program” is as the department/area/service defines it. Key themes emerged at City on learning outcomes and Student
Services: Students learn about other campus resources from those of us in Student Services Students learn from Student Services professionals how to better advocate for
themselves Self-advocacy Self-assertiveness Networking
To have buy-in, sustainability, and assessment of value-- it must be an inclusive staff/faculty driven process: a grassroots effort
Values driven You and your department are the subject matter experts —what do
YOU feel is important to study in your department? Part of daily/weekly/monthly/semester work--not a set-aside project
Student Services
The campus is our classroom. All of us are teachers and each of us is
essential to the college mission and the work and learning that occurs daily with students.
Assessment is good---it demonstrates for us the vital role that each of us play in the educational process.
How To Start What do you and your dept. value or feel is important
to do well in your area? Seek consensus in your area on choosing this issue to
study. Pick one for now. Keep it simple
Write it down Decide on a measure
Observe students? Survey?
Collect data Act on the results. Write it down.
Direct and Indirect Measures
Satisfaction surveys Provide feedback on student opinions Indirect measure of assessment
Measure what the student feels or thinks about the program
Move away from satisfaction surveys Optimal to use satisfaction measures in conjunction with other measures
The assessment effort seeks to find what the students know, have learned or can demonstrate Direct measures of learning
Tips in Creating a Survey Limit number of questions
Include explicit directions
Check reading level
Eliminate having two questions built into one
Pilot the survey
Consider alternative methods of distribution/collection
Consider whether anonymity is an issue
Lessons from the Field A thousand mile march really does begin with your first
step.
Change is rarely convenient. The only who likes it is a wet baby.
Who are the assessment champions? Persons willing to be accountable to keep the process going forward? Set meetings? Keep notes? HOPE IS NOT A PLAN.
Focus, Focus, Focus. Start with just one question, one form, one anything!
KNOW how you will know that what you have done is an improvement. Design for incremental improvement, not publication or perfection.
Don’t get hung up on names: outcomes, objectives, competencies, goals, SLO’s, AUO’s, program definition. You know what is important to your dept.
Sometimes you just have to jump in and DO IT!
You are doing assessment much more than you think…you just are not taking credit for it.
Contact Information
Dotti CordellSan Diego City College1313 Park Blvd. Rm. A116San Diego, CA 92101(619) 388-3903Email: [email protected]