overview of assessment

ADMINISTRATIVE AND EDUCATIONAL SUPPORT UNITS

ASSESSMENT WORKSHOP

2009-2010 CYCLE 2010-2011 Cycle

Del Mar CollegeRisè KnightMay 2010

OVERVIEW OF ASSESSMENT

PURPOSES OF ASSESSMENT

1. To improve – The assessment process should provide feedback to determine how the unit can be improved.

2. To inform – The process should inform decision-makers of the contributions and impact of the administrative unit to the development and growth of students.

3. To prove – The assessment process should demonstrate what the unit is accomplishing to students, faculty, staff, and outsiders.

4. To support – The process should support for campus decision-making activities such as program review and strategic planning, as well as external accountability activities such as accreditation.

When developing the assessment plan for your unit, always refer back to the four main functions of assessment .

INTRODUCTION to DEVELOPING an ASSESSMENT PLAN

Clarify the focus and content. Remember the purpose.

1. What are you trying to do?2. How well are you doing it?3. Using the answers to the first two questions, how

can you improve what you are doing?4. What and how does your unit contribute to the

development and growth of students?5. How can the student learning experience be

improved?

A Systematic Approach to Developing the Administrative Assessment Plan

1. Organize for assessment. Identify key players, committees and structure. All staff should assume the responsibility for designing, implementing and carrying out the assessment process.

2. Define the mission. The missions should be a broad statement of the unit’s direction and should reflect each unit’s contribution to the educational and career paths of students who encounter the unit specifically.

Systematic approach Cont.

3. Define the Outcomes for the unit. The outcomes usually focus more on process and student development than learning. The outcomes will focus on what the department is going to do and what its impact will be on student and key stakeholders.

4. Inventory existing and needed assessment measures. Identify , list, and describe all available approaches that can be used to conduct assessment. What methods are needed to provide you with the necessary information for assessment?

Systematic approach cont.

5. . Identify the achievement target for each outcome. Determine the standards expected from your services.

6. How will assessment results be used for unit improvement? The results and information gained should be distributed to the appropriate parties to achieve continuous improvement. Theses results may justify professional development, travel, equipment, personnel, facilities, etc.(Budget).

Systematic approach cont.

7. Establish a schedule for the above steps. 8. Submit the assessment plan for review to your

respective divisions and committees.9. Implement the plan, collect data, review data at

appropriate intervals and continue to update where needed.

10. Provide feedback to key members of the administrative unit so that data driven improvements can be made on a continuous basis.

FOCUS

DECISION MAKING IS THE FOCUS OF

ASSESSMENT

SACS REQUIREMENT

Del Mar College submits Detailed Assessment Reports (WEAVE) as supporting documentation for compliance with SACS Core Requirement 2.5 and Comprehensive Standard 3.3.1 and 3.4.2.

Core Requirement 2.5

The institution engages in ongoing, integrated, and institution-wide research-based planning and evaluation process that incorporate a systematic review of programs and services that (a) results in continuing improvement, and (b) demonstrates that the institution is effectively accomplishing its mission(Institutional Effectiveness)

Comprehensive Standard 3.3.1 – Institutional Effectiveness

The institution identifies expected outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves these outcomes; and provides evidence of improvement based on analysis of those results.

Comprehensive Standard 3.4.2

The institution’s continuing education, outreach, and service programs are consistent with the institution’s mission. …What evidence exists that demonstrates that continuing education, outreach, and public service programs are regularly evaluated in relation to the institution’s mission?

2009-2010 Assessment Reports

Assessment Reports = SACS Documentation Findings Action Plans AnalysisAnnual ReportReports demonstrate our institution is

effectively accomplishing our mission.

2009-2010 CycleFINDINGS

Findings Guidelines Findings are merely the quantifiable data, without any

analysis, that result when the measures listed in the assessment plan are completed.

Specific numbers are essential for reporting findings. The actual percentage (%) or numbers that resulted

from the measures are the focus of findings. The sample size (the “n”) should be reported as well, since it provides context for the results.

Findings should be phrased the same as the measures are to show the direct relationship.

What to do…..

Target At least 50% of all WebCT users will attend at least one upgrade training session in 2009-10 academic year.

Findings 60% of all WebCT users (n=100) attended at least one upgrade training session during 2009-10.

Target IT will conduct ten campus-wide workshops for DataTel conversion in June and July of 2010.

Findings Eight campus-wide workshops for DataTel conversion were conducted in June and July 2010.

What not to do……

Target There will be a minimum of 10% increase in overall satisfaction between 08-09 and 09-10 on the student satisfaction survey.

Findings There was less than 10% increase in overall satisfaction between 08-09 and 09-10.

Corrected: Findings There was a 5% increase in overall satisfaction

between 08-09 and 09-10.(08-09: n=25, mean =3.82) (09-10: n=27, mean = 4.01)

ACTION PLANS An action plan needs to be developed for each

measure, even if the achievement target was met (unless you are not assessing that outcomes any longer).

The action plan is developed at the end of the assessment year cycle and implemented the following assessment year (Exception)

An action is an organized activity you undertake to help your unit more effectively achieve its intended outcomes.

In WEAVE, Action Plans are entered after “Findings”

Action Plan cont.

If a desired target is not achieved, an action should be planned to address that shortcoming.

If a desired target is achieved and you wish to expand upon or increase the target level, you should plan an action to address the outcomes.

If you have achieved your target and wish to “move on”, you no longer need to write an action plan. You will not assess that outcome in the next year.

ACTION TRACKING

2009-10 Action Plans will be found in 2010-11 Assessment Reports as “Tracking”.

Indicate the status of each as Planned, Finished, On-hold, or Terminated.

Enter a comment regarding each ACTION in the notes filed. The NOTES in “Action Tracking” serves as an opportunity to “close the loop” and provides documentation of improvement over time.

ANALYSIS

Achievement Summary/Analysis (under the assessment tab)

You are asked to respond to two broad analysis questions about your strengths and your weaknesses. The third question refers to budget implications. Within those broad questions, you should use additional questions to explain how your data has informed your decisions and your assessment plan for the next year.

Analysis 1st Question

What specifically did your assessments show regarding proven strengths or progress you made on outcomes/objectives?

• What objectives did you accomplish in the previous year?

• What led to the unit’s success in accomplishing them?

• Where else could these successes be duplicated in your unit or division?

Analysis 2nd Question

What specifically did your assessments show regarding any outcomes/objectives that will require continued attention?

• What objectives/outcomes did you NOT accomplish in the previous year?

• What might have contributed to the unit’s inability to accomplish them?

• What could be done differently to improve the results of this objective in the coming year?

Analysis 3rd Question

How will your assessment be used by the unit to justify professional development, travel, equipment, personnel, facilities, etc.? (Budget)

• What improvements require additional funding?

• Should resources be utilized differently to achieve better results?

“Progress on Outcomes”

The answers to theses questions are compiled into a college-wide “Progress on Outcomes” report. “Progress on Outcomes” reports will demonstrate a culture of evidence-based planning and budgeting. The reports will be used as supporting documentation for compliance with SACS Comprehensive Standard 3.3.1.

ANNUAL REPORT GUIDELINES

The Annual Report asks you to reflect on and communicate about several categories.

These reports become the history of Del Mar College ‘s administrative accomplishments for the year. The information provided here should communicate what you want everyone to know about your area.

Executive Summary

This should be a paragraph that provides a high-level summary of the previous assessment (academic) year. This may include information not contained in the assessment plan (such as staff changes, grants received, or benchmarks met) but should apply some focus to assessment and its results.

Contributions to the Institution

Again, this paragraph or list of bullet points may include information not included in the assessment plan. It may present accomplishments in daily operations or focus on the objectives undertaken in the assessment plan.

Highlights

This can be a list of bullet points that emphasize the high (or low) points, and challenges and solutions of the previous year.

List any special awards, accomplishments, or events within your unit.

You may include conferences or professional development attended by staff.

Teaching Activities

Unless your unit does a significant amount of teaching to students or staff, you do not need to provide any information in this field. You may enter “N/A”

If your unit provided professional development you may include it here.

Research and Scholarly Activities

Unless your unit does a significant amount of research and publication, or if you received a grant for administrative research, you do not need to provide any information in this field. You may enter “N/A.” If this section does apply to your unit, please tell what research and grants were executed and what results were, and/or what scholarships was accepted for publication.

Public/Community Service

If your unit is involved in public or community service, please describe those activities and highlight the accomplishments in this area, otherwise enter “N/A.”

International Activities

If these are separate from the mission of your department, discuss any international activities undertaken by your department, if applicable. You may enter “N/A” if your unit did not participate in international activities this year.

Challenges

Explain in at least one paragraph or bullet points the challenges encountered in the past administrative year, and the actions taken to overcome them. If there are anticipated challenges for the coming year, they can be identified here.

2010-2011 Assessment Cycle– Effective administrative unit assessment is viewed as a

comprehensive, systematic and continuous activity, a means of self- improvement and used as a management tool.

• The Three Steps of Assessment 1(WEAVE components italicized) • 1) Articulate your goals for your unit.• ( Outcomes/Objectives) • 2) Gather evidence about how well the unit is meeting the goals. • (Measures, Targets, & Findings) • 3) Use the information for improvement • (Analysis, Action Plans, Action Tracking, Annual Report)

Articulate GoalsOutcomes/Objectives

Outcomes are statements that describe the desired quality (timeliness, accuracy, responsiveness, etc.) of key functions and services within the administrative unit.

Operational outcomes define exactly what the services should promote.

Goals are broad statements, while outcomes are precise, specific, and clear.

Goal – Outcomes RelationshipGoal 1 Goal 2 Goal 3

Outcome 1 Outcome 1 Outcome 1

Outcome 2 Outcome 2 Outcome 2

Outcome 3 Outcome 3

Outcome 4

Outcomes 5

Outcomes/Objectives cont.

A program does not need to state all possible outcomes, but it should try to articulate those that are fundamental

5 minimum (for 2010-11), 10 maximum Programs reevaluate outcomes/objectives at the

beginning of the assessment cycle. The unit mission should be reevaluated periodically.

Outcomes/Objectives should be rotated but a minimum of 5 assessed each year

Gather EvidenceMeasures and Targets

Check to see that the measure matches the objective/outcome – “apples to apples”

Measures need to clearly identify the content that addresses the outcomes.

Measures should provide useful data.

To ensure a high level of accuracy, two assessment measures should be used for each outcome, combining direct and indirect.

Targets need to show criteria for success for each outcome

Targets need to show criteria for success for each student learning outcome.

Good Example

Outcome Hypothetical Administrative Unit will demonstrate increased timeliness in terms of processing student requests for the 2010-11 academic year.

Measure A computerized log will track the date and time of each student request and the date and time it is resolved. The time between request and resolution will be compared with 2009-10 log.

Achievement Target The HAU will improve student request process time by an overall average of 10% .

Not a good example

Outcome Hypothetical Administrative Unit will demonstrate increased timeliness in terms of processing student requests. (when??)

Measure Number of students served by HAU will be tracked for three semesters. (Not useable information, does not match the outcome).

Target Students will be satisfied with response time to their request. (How much? How will you know?)

Use the Information for ImprovementMAKE ASSESSMENT MEANINGFUL!!

(Analysis, Action Plans, Action Tracking, Annual Report)Assessment Plans and the Action Plans developed in the

Assessment Report will tell what you will do in the future. Assessment contacts need to ensure all unit staff know the unit’s action plans, outcomes, and measures and are carrying out and reporting assessment activities throughout the academic year . The entire staff should be involved in the analysis of the assessment data to recommend changes for improvement. The goal is to keep it simple and to use the assessment process for better decision making.

REMINDERS!!!!

To Complete the 2009-2010 Assessment ReportReport Findings in WEAVEOnline.Each measure should have an action plan, either

new or one in tracking. Remember to add notes to the ones in tracking in order to document progress.

Complete Achievement Summary/Analysis by answering the Analysis Questions.

Complete the Annual/Special Reporting section.

REMINDERS!!!

For the 2010-2011 Assessment PlanArticulate Outcomes (remember to check your

VP’s goals for your area)Five outcomes for 2010-11Use appropriate measuresSet achievement targets

Submit for Review

SUBMIT Assessment Reports for 2009-10 for review by July1, 2010.

SUBMIT Assessment Plans for 2010-11 for review by July 15, 2010.

Email me when you have completed these so that I may review immediately. This is very important.

Call for HELP!Risè E. Knightrknight1@delmar.edu438-4608 cell698-1102 office (on campus every other two weeks, available off campus via

email and cell)

Dr. Irma Woodsiwoods@delmar.edu698-1461

Technical WEAVEOnlineJane Haasjhaas@delmar.edu698-1208