THIS FORM MAY NOT BE REPLICATED

AND UNDER NO CIRCUMSTANCES CAN THE LANGUAGE BE ALTERED

BOG, California Community Colleges Chancellor's Office - 6870

,,,,;-.,,,,., 'M"@lli ,%,%SSW, .f'.Wiitl''

~;~C)firr_;

91

Student Success and Support Program (formerly Matriculation) California Community College Common Assessment Initiative

RFA# 13 - 083

Grant o.nr,oon-ionT 3 - 083 - 001

Funding Fiscal Year

2013-14 Total Amount Encumbered $ 8,000,000-----------11

This grant is made and entered into, by and between, the BOG, California Community Colleges Chancellor's Office and the aforementioned district, hereafter referred to as the Grantee. The grant shall consist of this Grant Agreement face sheet and the Grantee's application, with all required forms. The RFA Specification and the Grant Agreement Legal Terms and Conditions (Articles I, Rev. 10/10 and II, Rev. 4/08), as set forth in the RFA Instructions are incorporated into this grant by reference.

The total amount payable for this grant shall not exceed the amount specified above as "Amount Encumbered".

The term of this grant shall be from December 1, 2013 to June 30, 2014. The Final Report must be submitted within 60 days of the grant end date.

Funding under this grant is contingent upon the availability of funds, and is subject to any additional restrictions, limitations or conditions enacted in the state budget and/or Executive Orders that may affect the provisions, term, or funding of this agreement in any manner.

Project Director: Total Grant Funds Requested: $ 8,000,000

Barry Gribbons, Ph.D. Total Match Funds, (if applicable):

Date: I 12 Prfnt Name/Title of Person Signing: District Address: 26455 Rockwell Canyon Road, Santa Clarita, CA 9

Project Monitor: Agency Address: 1102 Q Street, Suite 4554

Bonnie Edwards Sacramento, CA 95811-6539 --------11 Item: Object of Expenditure Chapter Statute Fiscal Year Amount

I 6870 - 101 - 0001 (10) 5218 - 151 - 21111 20 2013 2013-14 $ 8,000,000

Total Amount Encumbered : $ 8,000,000

Signature, Accounting Manager (or Authorized Designee) Budgeted funds are available for the period and purpose of the expenditures stated above.

Date: Signature, Executive Vice Chancellor (or authorized Designee)

Date: Print Name/Title of Person Signing: Steve Bruckman, Executive Vice Chancellor

Grant Face Sheet Form - Single Year Grant Revised 5/10/2013

Chancellor's Office District: Santa Clarita CCD California Community Colleges College: College of the Canyons

RFA Number: 13-083

Grant Agreement No.:

E Proposal ID No.: Funding Status: Fiscal Year:

Funding Source(s): California Community Colleges Student Services and Special Programs and Technology, Research and Information Systems Divisions

Project Title: California Community College Common Assessment Initiative

Institution: College of the Canyons Address: 26455 Rockwell Canyon Road

City: Santa Clarita State: CA Zip+4: _;;9....;.1...;;..3=-55~-------

College President (or authorized Oesignee)

Name: Dr. ~C .Van Hook., . , . Title: Chancellor Signature: ~ (;jn i)v.lf Date: _0--c...;;...;;to_b_e_r_10__._2_0_1_3____________ Phone: ( 661) 362-3400 Fax: (661) 259-9261 E-Mail Address: dianne.vanhook@canyons.edu

Responsible Administrator (Appropriate Program Area)

Name: Dr. D~. Van Hook Title: Chancellor

Signature: ~ &, M IJ- Date: --0--c...;;...;;to__b___e_r_10__._2_0_1_3____________ Phone: (661) 362-3400 Fax: (661) 259-9261 E-Mail Address: dianne.vanhook@canyons.edu

Project Director

Name: Bar~Ph.D. Title: Assistant Superintendent/VP Institutional Development Signature: ~~ Date: -=0-=c~to~b...;;.e.;.,_r..;..1O..;...i....;;;2;;;..;;0;._1--3____________

Phone: (661) 362-5500 Fax: (661) 362-5600 E-Mail Address: barry.gribbons@canyons.edu

Business Officer Name: Ms. Sharlene Coleal ,/"1 ,,, Title: Assistant Superintendent/VP Business Services

Signature: C:5il1l1,f;:,,1( t L(klz)~4/ Date: October 10, 2013 Phone: (661) 362-3405 Fax: (661) 362-5405 E-Mail Address: sharlene.coleal@canyons.edu

Application/Grant Writer Name: Ms. Theresa Zuzevich Title: Director, Grants Development

Signature: -rv-vJ~)C,-,f 1-:'JLL/tptjt n1:lr) Date: _O_c_to_b_e_r_1_0_2_0_1_3____________ Phone: (661) 362-3644 L/ ''-f"ax: (661) 362-5600 E-Mail Address:theresa.zuzevich@canyons.edu

Chancellor's Office District: Santa Clarita CCD California Community Colleges College: College of the Canyons

RFA Number: 13-083

APPL/CATION CONSO TIUM

DATA SHEET

-'1 Please check here if this proposal is a consortium project Complete the following information for each college of the consortium. Use additional sheets if required. Attach this form directly behind the Contact Page.

District/College or Organization: San Joaquin Delta College Address: 5151 Pacific Ave City: Stockton State: CA Zip+4: 92507-6304

Project Contact: Kathy Hart. Ph.D., Superintendent/President Phone: 209-954-5151

Amount of dollars contributed to project by the district/college: $Q

Role of district/college in the consortium design: Co-hosting data for the common assessment initiative.

Technical and data su12port.

District/College or Organization: Irvine Valley College Address: 5500 Irvine Center Drive City: Irvine State: CA Zip+4: 92618-0301

Project Contact: Craig Ha~ardz Ph.D., Director of Planning. Research, and Accreditation Phone:

949-451-5788 Amount of dollars contributed to project by the district/college: $Q Role of district/college in the consortium design: Service on grant steering committee and in assessment and placement stakeholder groups. Collaboration with assessment vendors and the (@One Program to ensure adequate and relevant professional development programs are developed and administered. Collaboration with external evaluator to refine and implement the research plan. Local IT support for implementation of the assessment tests at Irvine Valley College.

District/College or Organization: Cabrillo Community College District Address: 6500 Soquel Drive

City: Aptos State: CA Zip+4: 95003-3119

Project Contact: Terrence Willett, Director of Planning, Research and Knowledge S~stems Phone:

831-4 77-5656 Amount of dollars contributed to project by the district/college: $Q Role of district/college in the consortium design: Cabrillo's Planning and Research Office and Information Technology department will participate in the Qlanning and implementation of the Common Assessment Initiative.

Applic:c1tion Consortium Data heet (Continued) RFA Number: 13-083

District/College or Organization: The RP Group Address: 2600 Tenth Street, Suite 435 City: Berkeley State: CA Zip+4: 94710-3105 Project Contact: Michael Howe, Executive Director Phone: 510-527-8500 Amount of dollars contributed to project by the district/college: $.Q. Role of district/college in the consortium design: The RP Group will lead the evaluation of the initiative.

District/College or Organization: Niemi Brown LLC Address: 8001 Irvine Center Drive. Suite 400 City: Irvine State: CA Zip+4: 92618-2956 Project Contact: David Niemi, Ph.D., President Phone: 949-754-2891 Amount of dollars contributed to project by the district/college: $.Q. Role of district/college in the consortium design: Item development and validation and development of an adaptive test engine, in addition to consulting on related activities.

District/College or Organization:___ Address:-City:_ State:_ Zip+4:_ Project Contact:___ Phone:___ Amount of dollars contributed to project by the district/college: $__ Role of district/college in the consortium design:_

District/College or Organization:____ Address:-City:_ State:__ Zip+4:_ Project Contact:___ Phone:___ Amount of dollars contributed to project by the district/college: $__ Role of district/college in the consortium design:_

Application Abstract (narrative form)

College of the Canyons (COC) will lead and coordinate the development and implementation of a single common assessment test for math, English and English as a Second Language for use by all California community colleges.

COC will establish two oversight committees to work with stakeholders across the state to develop assessment specifications, review existing assessments, create test items (if needed), validate the assessment items, identify a test delivery system, pilot and roll-out the assessment.

The project will improve the effectiveness and accuracy of assessment and placement through robust research support from community college partners, the RP Group and additional consultants. Activities will increase student awareness of the importance of placement testing, including development of a pretest designed to improve placement.

By July 2015, the common assessment will be available to all 112 colleges, with implementation at a minimum of 80, reducing assessment costs across the state by 40%. The project will provide a secure data warehouse to serve as a central repository for K-12 and higher education assessment data, including , cut scores and multiple measures. A co-location for full redundancy will ensure system availability set- ' up.

The project will reduce the number of retests given to students and increase the portability of assessment tests across the system. Of colleges using the common assessment test, 25% will expand placement policies to accept results from other colleges. The common assessment will also improve placement, reducing the percentage of students who place into remedial English, math and English as a Second Language by 25%.

COLLEGE OF THE CANYONS

California Community College Common Assessment Initiative (RF A Specification #13-083)

Table of Contents

Face Sheet

Contact Page

Consortium Data Sheet

Application Abstract

Table of Contents

Narrative

Need

Objectives

Procedures/ Activities

Performance Outcomes/Evaluation Design

Work Plan

Project Management/Institutional Commitment

Organizational Chart

Overall Feasibility of Project

Dissemination

Budget Summary

Budget Detail

Letters of Support/Commitment

Attached

Attached

Attached

Attached

1

4

8

14

42

67

68

72

73

Attached

Attached

Attached

SANTA CLARITA COMMUNITY COLLEGE DISTRICT

College of the Canyons

An application to The California Community College's Chancellor's Office

California Community College Common Assessment Initiative (RFA Specification #13-083)

Introduction College of the Canyons (COC) is pleased to present this proposal to the California Community Colleges Chancellor's Office to provide assistance to all California community colleges in the creation of a common assessment for English, math and English as a Second Language.

Need Assessment for course placement is the process of identifying a student's academic skills and

deficienq.es in order to accurately place a student into appropria~e courses. California's

community colleges are faced daily with the task of accurately assessing incoming students who

enter college at a wide range of readiness levels and with diverse academic and training interests.

Currently, students are assessed in math, English and English as a Second Language.

Assessment affects college placement, which ultimately affects how quickly students can earn a

certificate and degree as well as their likelihood of finishing at all. Students who place into basic

skills courses are less likely to complete their educational path than those students who place into

college-level courses.

According to the California Community Colleges' Chancellor's Office, recent studies in

California have found that over 83 percent of incoming community college students place into

basic skills mathematics (with 38 percent placing two or more levels below college-level) and 72

percent place into basic skills English (with 61 percent placing two or more levels below college

level).1 Two main challenges faced by students taking an assessment test include lack of

uniformity among assessments and cut scores by colleges and the lack of preparation for the

assessment by the student.

Legislation

The Seymour-Campbell Matriculation Act was enacted in 1986 and required colleges to help

students define their education goals and provide services to help them achieve their goals.

Services included admissions, orientation, assessment and advisement. In 2012, the Seymour

Campbell Student Success Act (Senate Bill 1456) was passed that not only stressed the

importance of the support services for students, but developed a funding formula for colleges

based on use of a common assessment to be established by the Board of Governors. In addition,

Assembly Bill 743 provides funding for SB 1456, which will begin in the 2013/2014 fiscal year.

1Venezia, A., Bracco, K.R., and Nodine, T. (2010). One Shot Deal? Students' Perceptions of Assessment and Course Placement in California's Community Colleges. WestEd. Web.

1

http:deficienq.es

While the Seymour-Campbell Matriculation Act gave the Board of Governors responsibility to validate and approve all assessments, each district or college has the responsibility for selection and correlation to the course placement. In 2008, a report by the Consultation Council Taskforce on Assessment identified approximately 30 different assessments being used at that time and most colleges did not accept placement results from other California community colleges. This decentralized approach has been criticized for generating additional costs to the system due to retesting and inconsistent messages to high school students regarding academic standards. The passage of Senate Bill 1456 and Assembly Bill 743 aim to identify/create a common assessment tool to be used by all colleges.

Lack ofuniformity

As mentioned, there are approximately 30 different assessments being used across the 112 California community colleges. According to a recent study by W estEd, each college sets their own test score levels, meaning that students can be placed at different levels in different colleges.2 This creates confusion and frustration for students. Among the issues students face are: \ \

inconsistent policies across the colleges - this may include time to get scores and how soon scores are explained, which results in delayed registration dates;

lack of clarity about policies for retaking the assessment - this includes can it be retaken and if so, as well as how to go about retaking the assessment;

confusion about multiple measures and how to challenge the course placement; and frustration about achieving educational goals.

Multiple measures are additional criteria, usually questions embedded in the assessment that colleges use to calculate placement. Though required by state policy, there are no criteria as to what the multiple measures must be. Therefore, like the assessment exams, these measures vary from college to college, with most students not even knowing what they are or what significance they hold. These criteria may include, but is not limited to:

hours worked outside of class; high school transcripts; student aspirations; and level ofpreparedness for the assessment.

When students realize their placement into basic skills courses could equate to a year or more of coursework before they can take college-level courses, they feel stuck, frustrated and many drop out. Students don't want to have to spend the additional time to get their degree so they can begin their career.

2 Venezia, A., Bracco, K.R., and Nodine, T. (2010). One Shot Deal? Students' Perceptions of Assessment and Course

Placement in California's Community Colleges. WestEd. Web.

2

The goals of the Common Assessment System are to:

Develop and use centralized and integrated technology solutions to support the

assessment and placement activities of the California Community Colleges.

Help colleges provide more students with robust and effective assessment and placement tools.

Expand the underlying systems of data and research and improve access to information necessary to support a comprehensive statewide assessment and placement system in support of students, faculty and staff.

The proposed project builds on the latest educational advancements regarding assessments already happening at California community colleges. Long Beach City College is implementing a new approach to assessing student readiness for college-level coursework. Rather than solely relying on the assessment, they are also reviewing high school transcript data for first-time students from the Long Beach Unified School District. 3 Long Beach City College believes that they will now place 800-900 students into transfer~level English, as opposed to the only 100 currently being placed there by the assessment tes~. In math, at least 20% of students will be placed one or more levels higher using high school transcripts instead ofjust the assessment test.

Grossmont-Cuyamaca Community College District has already implemented this approach with positive results. Ninety-five percent of students with A's and B's in lih grade English at one of their local high schools were being placed into basic skills English based only on the assessment test. The District began to place students in courses based on grades instead, placing those students with A's or B's into transfer-level English. Eighty-six percent of those students successfully complete transfer-level English on their first try.4

Lack of preparation

Many students in California's community colleges experience assessment as a single event, not a process that they began preparing for in high school. Because of this attitude, many students are unprepared for the assessment, do not take it seriously, and place into basic skill-level courses.

An example of one college's efforts to address the lack of preparation is Santa Monica College. Santa Monica College partnered with their local high school district, Santa Monica-Malibu Unified School District and created the Prep2Test program to help students prepare for the placement test. Launched in 2011, the Prep2Test program consists of 2 online videos that not only explain the importance of the placement test to students, but serve as an orientation as well. This program has helped increased the percentage of students placing into college-level math and English courses by approximately 9 percent. 5

3 RP Group. (2013). Student Transcript-Enhanced Placement Study (STEPS). RP Group. Web.

4 RP Group. (2013). Student Transcript-Enhanced Placement Study (STEPS). RP Group. Web.

5

Santa Monica Lookout Staff. (2013). Santa Monica College Wins Prestigious Assessment Award. Santa Monica

Lookout. Web.

3

Scope of the Problem

This is a statewide problem affecting the majority of first-time students entering California community colleges, especially those who are first generation, low income, ethnic minority, English language learner, reentry student or historically place into remedial courses.

Target Groups

The populations to be served by this program include first-time students and faculty and staff at all California community colleges.

Objectives

To achieve the goals of the Common Assessment Initiative, College of the Canyons will achieve the following required objectives by implementing strategies and activities. Please see the Work Plan for a more detailed discussion of the activities. All objectives will be evaluated for recommendations for improvement and recommendations will be implemented as appropriate.

1

' Objective 1 Governance:

Objective lA, ensure program success through effective governance structures, will be addressed through the creation of two oversight committees comprised of state and local California Community College, K-12 and higher-ed leadership from the State Academic Senate, Academic Affairs, Student Services, Communications and Technology as well as consortium partners Cabrillo College, Irvine Valley College, San Joaquin Delta College, the RP Group and Niemi Brown LLC. The Common Assessment Steering Committee (CASC), working with the project director, will provide oversight for the project with the Common Assessment Advisory Committee (CAAC) providing input on the major activities of the project. These committees will serve as the governance structures that will ensure success. This will be measured through committee meetings and minutes.

Objective lB, the Common Assessment Program must operate legally, effectively and satisfy the program and operational goals of the State, colleges and students, will be addressed by convening the committees and other stakeholders, including the Chancellor's Office to review all items associated with the project, including reviewing and approving contracts and MOUs. This will be measured through the number of contracts, MOU s and other legal documents approved.

Objective 2 Assessment and Placement:

Objective 2A, develop a common test for English, math and English as a Second Language for

all California Community Colleges, addresses the need for a common assessment. The

committees will work with stakeholders across the state to develop specifications for the

assessment (including alignment with K-12 Common Core Standards), review existing

assessments, create test items (if needed), validate the assessment items, identify a test delivery

system and pilot and roll-out the assessment. This objective will be measured through the

specifications and items created, identification of an assessment and platform and a successful

pilot at 15 colleges with eventual roll-out to all colleges. The change expected is a single-use

4

assessment available to all California community colleges, reducing the number of times a student must retake the assessment as they transfer colleges.

Objective 2B, reduce the number ofretests given to community college students and increase the portability of assessment tests across the system, addresses the need for a common assessment. This objective will be measured through the number of students attending multiple colleges, taking multiple tests each year, and the number of colleges adapting the assessment test and expanding their placement policies to accept results from other colleges. The change expected is a single-use assessment available to all California community colleges, reducing the number of times a student must retake the assessment as they transfer colleges.

Objective 2C, increase the effectiveness and accuracy of test placement for students placing at or below college level, addresses the need for improved student placement. The objective will be measured through improved placement, stronger validity redaction to course completion as well as differential impact for subgroup less then assessment prior to new common assessment. The change expected is tmproved placement for students, particularly among stijldents who are Latino, African American, Asian, low-income, English language learners, and those who traditionally place into remedial courses.

Objective 2D, lower remediation rates for California Community College students and increase the initial placement level for students, addresses the need for improved student placement. The objective will be achieved by administering a pre-test for students designed to improve placement and measured by the number of students accessing test-prep materials online. The change expected is improved placement for students.

Objective 2E, increase awareness of the importance ofplacement tests and improve student preparation, addresses the need for improved student placement. The objective will be measured through improved placement, stronger validity redaction to course completion as well as differential impact for subgroup less then assessment prior to new common assessment, based on marketing the importance of the assessment and the pre-test. The change expected is better student preparation and improved test placement for students, particularly among students who are Latino, African American, Asian, low-income, English language learners, and those who traditionally place into remedial courses.

Objective 2F, assist colleges with efforts to improve local assessment and placement practices, addresses the need for improved student placement. The objective will be measured through improved placement, stronger validity redaction to course completion as well as differential impact for subgroup less than the assessment prior to the new common assessment. The change expected is improved test placement for students.

Objective 2G, reduce the cost of assessment related activities at the district/college level, addresses the need for a common assessment. The objective will be measured through the number of colleges using the test and the number of students assessed as well as the average cost of the tests. The change expected is lower costs for test placement for colleges.

5

Objective 2H, leverage multiple measures data and research to lower the number of assessment tests issued and to improve placement, addresses the need for a common assessment and the need for improved student placement. The objective will be measured through the number of colleges using the test and the number of students assessed as well as the average cost of the tests. The change expected is lower costs for assessments at each college.

Objective 3 Professional Development:

Objective 3A, provide faculty and staff with the training and skills necessary to be successful in the common assessment environment, addresses the need for a common assessment and training to understand how to better use the test and data. COC will partner with the RP Group and the @One Program to ensure adequate, and relevant professional development programs are developed and administered. The objective will be measured through the number of workshops offered and the number of attendees at each workshop. Workshops will be offered online and in person. The change expected is increased adaptation of the assessment at all colleges.

\

Objective 4 Data Warehouse, Research and Validation Support:

Objective 4A, provide a secure data warehouse to serve as a central repository for K-12 and higher education assessment data, including multiple measures, addresses the need for a secure platform and location to host the assessment and multiple measures data. The objective will be measured through co-locations for full redundancy that meet designed specifications for a platform that will be piloted before statewide roll-out. The change expected is a secure location for all colleges to use for placement data.

Objective 4B, improve assessment and placement through robust research, addresses the need for research and validation support. The objective will be measured through a partnership with CalPASS Plus, Terrance Willett (Cabrillo College), the RP Group, Niemi Brown LLC and CASC to conduct the research and validation support. The change expected is improved test placement for students.

Objective 4C, support colleges by providing validation assistance as appropriate, addresses the need for a common assessment and training. The objective will be measured through research studies on cut scores and multiple measures. The change expected is satisfaction with the common assessment.

Objective 5 Technical:

Objective SA, leverage technology to standardize and enhance the user (student, faculty, staff) experience. In addition to improving testing services and features and accessibility, addresses the need for a system platform for use by all colleges that will support community college and K-12 student data, including high school transcript data. This objective also addresses the need for a system that is compatible with the Chancellor's Office, MIS and local SIS. The objective will be measured through the development of specifications for the platform and the creation of the

6

platform as well as the satisfaction of the users. The change expected is colleges using the common assessment and using high school transcript data to assist in placement.

Objective SB, statewide technology system integration and standards will be a guiding principle throughout the lifecycle of the project, addresses the need for a secure location for data. The objective will be measured through a secure environment that meets system integration specifications for consistent data and technical standards for transferring data. The change expected is an increase in colleges using the system.

Objective SC, existing and new statewide programs will be leveraged to achieve objectives, addresses the need for a common assessment. The objective will be measured through MOUs with Cal-PASS Plus and CCCT to support system integration and the data warehouse for reporting, validation, assessment/research and K-12 data. The change expected is an increase in colleges using the system.

Objective 6 Program Evaluation:

Objective 6A, the various elements of the Common Assessment program will have clearly stated and measurable metrics for evaluation purposes, addresses the need to develop metrics for evaluation. The college will partner with the RP Group to ensure that a realistic set ofmetrics for performance tracking and expert evaluation services is established. The objective will be measured through evaluations, survey reports and recommendations implemented. The change expected is an increase in colleges using the system.

Objective 7 Planning:

Objective 7 A, adequate planning activities will be developed, followed and monitored to ensure progress toward milestones, addresses the need for milestones to ensure the project stays on schedule. The objective will be measured through quarterly reports and milestones reached. The change expected is a common assessment available to all colleges.

Objective 8 Communication:

Objective 8A, the Common Assessment Program will be marketed to current and potential community college students, K-12 and other educational institutions through effective channels of communication, addresses the need to communicate awareness of the program to all stakeholders. COC's award-winning Public Information Office will lead this effort. The objective will be measured by evaluating the effectiveness of the communication plan by the number of students reached. The change expected is increased awareness of the common assessment test, particularly among students who are Latino, African American, Asian, lowincome, English language learners, and those who traditionally place into remedial courses.

Objective 8B, information related to the Common Assessment Program will be easily accessible and clearly understood, addresses the need to ensure all marketing materials are easy to understand. The objective will be evaluated through surveys of focus groups, including faculty, community college students, K-12 students and parents and UC/CSU personnel. The change

7

expected is an improved common assessment that is accessible to all students, especially those who are first-generation, low income, ethnic minority, English language learner, reentry students and those who historically place into remedial courses.

Procedures/ Activities

To achieve the goals of the Common Assessment Initiative, College of the Canyons will employ the following activities. Please see the Work Plan for a more detailed discussion of the activities. All activities will be evaluated for recommendations for improvement and recommendations will be implemented as appropriate.

Objective 1 Governance:

Procedure 1 A, establish a governance structure comprised of state and local California Community College, K-12 and higher-ed leadership from the State Academic Senate, Academic Affairs, Student Services, Communications, and Technology, achieves the governance objective by creating, oversight committees that will work with the project director. A Common Assessmen~ Steering Committee (CASC), made up of those indivi~uals stated above, along with partners from Cabrillo College, San Joaquin Delta College, Irvine Valley College, the RP Group and Niemi Brown will work with the project director to provide oversight for the project. This committee will ensure the governance structures that will promote success.

Procedure lB, regularly convene and effectively manage the Common Assessment Advisory Committee and agenda, achieves the governance objective by creating a Common Assessment Advisory Committee (CAAC), which will provide input on major activities of the project. The committee will also have sub-committees that will work on specific strands of the project. This committee will ensure the governance structures that will promote success.

Procedure 1 C, review, establish, modify and vet policy and process related items to ensure the Common Assessment Program can operate legally, effectively and achieve program objectives, achieves the governance objective by creating policy and procedures that will ensure the project operates legally. All MOUs, contracts and other correspondence will be approved as needed by the committee and the Chancellor's Office Legal Affairs Division. This oversight will ensure the governance structures that will promote success.

Objective 2 Assessment and Placement:

Procedure 2A, collaborate with faculty, staff and other committees, such as the Statewide Academic Senate, across the California Community College System to develop and vet requirements for the common assessment tests and assessment/placement system, achieves the assessment and placement objective by collaborating with stakeholders to create a common assessment. This procedure is vital to the project, as it creates the specifications needed for the assessment test; reviews current off-the-shelf tests, including Accuplacer, EAP, MDTP and CTEP; develops test items, in conjunction with K-12 Common Core Standards, for the exam if there is no comprehensive off-the-shelf test; validates all items for the tests, with a minimum of two forms per assessment to measure evidence ofpredictive validity and show support for measures and differential impact; development of a test delivery system which may be a platform

8

already created, modifying an existing platform or creating a new platform; a pilot at 15 colleges with evaluation and recommendations for improvement; followed by the roll-out at all colleges. Each step will be measured through evaluation and pilot testing. This procedure will allow for a common assessment that all colleges will use.

Procedure 2B, facilitate the development and adoption of processes and standards to increase test portability across the system, achieves the assessment and placement objective by reducing the number of times a student attending multiple colleges will take an assessment test. This procedure will allow student placement data, along with multiple measure data, including high school transcript data, to be uploaded into the assessment system platform so as students transfer community colleges, colleges can access student data and placement results so the student can be properly placed. A common assessment test will allow students to finish faster by not having to retake assessment test that vary at each college.

Procedure 2C, lead and coordinate the development and implementation of a single common test for the three curricular areas of\math, English and English as a Second Language, achi~ves the assessment and placement objective by assessing available off-the-shelf tests to see if any of the tests will assess the specifications that were designed for each subject area. The assessments will be validated to measure evidence of predictive validity and show support for measures and differential impact. If no test meets the specifications, item development will begin to create a common assessment. This procedure will allow for a common assessment that all colleges will use.

Procedure 2D, ensure the common assessment tests are available for use at all community colleges, achieves the assessment and placement objective by improving student placement. The assessment exam will provide a more accurate assessment of the student particularly among students who are Latino, African American, Asian, low-income, English language learners, and those who traditionally place into remedial courses. A common assessment test will allow students to finish faster by being more accurately placed into courses.

Procedure 2E, negotiate vendor contracts so that test develop and administration costs are borne centrally and not by individual colleges, achieves the assessment and placement objective by making assessment testing more cost effective. Historically, the California Community College Chancellor's Office spends approximately $8 million a year on assessments for each college. This procedure will allow the leveraging of all colleges, instead of each college negotiating on their own; to obtain the best price possible for the test, and reduce the total cost about 40%. This procedure will allow for a common assessment that all colleges can use.

Procedure 2F, define and negotiate test develop and administrative costs and features within the confines of the available common assessment budget, achieves the assessment and placement objective by making assessment testing more cost effective. This procedure will allow the leveraging of all colleges, instead of each college negotiating on their own, to obtain the best price possible for the test. This procedure will allow for a common assessment that all colleges can use.

9

Procedure 2G, ensure the common tests and delivery platform are regularly reviewed and enhanced to meet the needs of the community colleges, achieves the assessment and placement objective by working with colleges one-on-one to implement the common assessment and evaluate how the system works with campus websites and student information systems. Recommendations for improvement will be implemented as appropriate. This procedure will allow for a common assessment that all colleges can use and will be satisfied with.

Procedure 2H, establish and maintain a working relationship with K-12 and higher education partners to address test content, placement and inter-segmental alignment, including alignment with K-12 Common Core Standards. The Common Assessment Program will partner with the Cal-PASS Plus Project to address internal and inter-segmental alignment, achieves the assessment and placement objective by aligning the common assessment with all educational partners, including K-12 and CSU/UC. Working with these partners will allow for more accurate test content as well as data provided, including high school transcript data. This procedure will allow for a more accurate placement\ of students.

Procedure 21, identify, develop and implement reporting tools and services to facilitate and streamline assessment and placement at the colleges, achieves the assessment and placement objective by creating a platform that will allow for delivery, reporting and data storage. This allows multiple measure data, including high school transcripts, to be stored and accessed by colleges as needed, to reduce the number of times a student needs to take the assessment test. It also allows for reporting tools to streamline the assessment and placement at the college. This procedure will allow for more accurate placement of students.

Procedure 2J, incorporate effective collection and analysis ofmultiple measure data points to improve placement and potentially reduce the need to assess students via the common tests, achieves the assessment and placement objective by reducing the number of students who place into remedial courses, particular! y among students who are Latino, African American, Asian, low-income, English language learners, and those who traditionally place into remedial courses. Multiple measure data will be evaluated and used more effectively to accurately place students into courses. By using common multiple measures data, such as high school transcript data, students will be more accurately placed into courses and can reduce their time to completion.

Procedure 2K, administer a student "pre-test" that could be delivered via the internet to prospective students before taking the actual assessment tests, achieves the assessment and placement objective by reducing the number of students who place into remedial courses. Many students do poorly on placement tests because they are not prepared and don't realize how important they are. Test preparation materials, including a pre-test, will be created to help students be prepared and succeed on the test. This allows students to be more accurately placed into courses and can reduce their time to completion.

10

Objective 3 Professional Development:

Procedure 3A, partner with assessment vendors and the @One Program to ensure adequate, and relevant professional development programs are developed and administered, achieves the professional development objective by creating in-person and online workshops to train stakeholders in the common assessment. Training topics will include technical components such as reliability and validity ( content, construct, criterion-related and consequential); differential impact; multiple measures; and recent relevant research studies for assessment in general and the common assessment system in particular. Badges will be earned for completion ofworkshops. By hosting the workshops in different formats, more stakeholders will be able to attend and be trained on the new assessment, making it more 'effective.

Objective 4 Data Warehouse, Research and Validation Support:

Procedure 4A, partner with Cal-PASS Plus to provide a secure central assessment and multiple measures data warehouse, achieves the data warehouse, research and validation support objective by developing specifications for a delivery system, data warehouse, reporting requirements, loc;ally setting cut scores and identifying multiple measure~, system integration and placement decisions. This warehouse will be co-located for full redundancy to ensure system availability. Having a system that is consistently available and has data on a centralized platform will help colleges be more effective with their placement tests.

Procedure 4B, partner with Cal-PASS Plus to conduct research and validation support functions, achieves the data warehouse, research and validation support objective by identifying areas of improvement and implementing those changes. As the system is piloted and rolled-out, it will be constantly evaluated to ensure it is benefitting all colleges, works with educational partners such as Cal-PASS Plus as well as ensuring correct student placement. This will allow students to be properly placed, reducing the number of times a student takes a placement test and possibly faster completion due to proper placement.

Procedure 4C, implement sufficient standards to allow for the development of optimization algorithms of placement success based on test score and highest level of subject taken, achieves the data warehouse, research and validation support objective by assisting colleges with identifying and implementing cut scores and multiple measure data to more accurately combine data for student placement. Again, this will allow students to be properly placed, reducing the number of times a student takes a placement test and possibly faster completion due to proper placement.

Objective 5 Technical:

Procedure 5A, organize a committee to define system requirements and develop system design and implementation plans, achieves the technical objective by identifying system specifications that include CCCID, DDO, DED, system integration with CCCCO MIS, local SIS and Cal-PASS Plus. The system needs to be compatible with all these systems to ensure correct and secure data storage between the systems, including high school transcript data. Correct data will lead to proper placement of students.

11

Procedure 5B, consistent data and technical standards will be developed, implemented and maintained to enable the automated transfer of data and systems integration. The program will follow the technology standards and recommendations approved by the Chancellor's Office and the Systemwide Architecture Committee (SAC), including but not limited to the Postsecondary, Educational Standards Committee (PESC), Federated Identity Management, and Service Oriented Architecture, achieves the technical objective by creating specifications for data element and protocols for transferring information in a secure environment that meets system integration and follows SAC, including PESC, FIM and SOA. This ensures secure data storage that meets the technology standards set by the various state agencies.

Procedure 5C, the Common Assessment Program will utilize the Systemwide Student Account I Identifier (CCCID) (a CCC Technology Center project) to enable single sign on functionality across the System. The California Community Colleges will participate in the InCommon Federation (InCommonFederation.org) for single sign-on (SSO) using Shibboleth's Identity Provider and Service Provider software (shibboleth.intemet2.edu). Students and staff will use their local college logins, to access statewide web services such CCCApply, Education Planning Program and the CVC E*-pansion Project. This procedure achieves the technicai objective by identifying system specifications to be compatible with all these systems to ensure correct and secure data storage between the systems. This allows for student and faculty access to local student information systems.

Procedure 5D, the Common Assessment Program will support the transfer of relevant student academic and support data into and out of the system, including but not limited to, Chancellor's Office MIS, local SIS, Statewide data warehouse (Cal-PASS Plus), achieves the technical objective by developing specifications for the delivery system and accurate reporting. Online training modules will be created for both items. Accurate data and reporting is essential to this project and student placement. Ensuring both will help the project be successful and encourage colleges to use the common assessment system.

Procedure 5E, faculty and staff should have the ability to easily identify and access assessment information as appropriate for job role and function, achieves the technical objective by making it easy for faculty and staff to access the information they need. Evaluations ofuser satisfaction by job role will be assessed and changes will be implemented as needed to ensure an easy system for all to use.

Procedure 5F, develop and maintain a web based presence for the Common Assessment Program, achieves the technical objective by having a website available for the project. Visitors will be able to find online training, help, contact information, evaluations and FAQs to help them with their needs. This website will be evaluated for satisfaction and changes made as needed to ensure the website stays helpful.

Procedure 5G, the Common Assessment Program will partner with the Cal-PASS Plus Program to provide data warehouse services, reporting and validation tools and to conduct assessment/multiple measures research and K-12 outreach. This partnership will be formalized

12

http:shibboleth.intemet2.eduhttp:InCommonFederation.org

through a memorandum ofunderstanding (MOU) and/or through a sub-grantee arrangement, including statement of work and budget items. This activity achieves the technical objective by having a data warehouse for this project. As stated earlier, the data warehouse will provide a delivery system, data warehouse, reporting requirements, locally setting cut scores and identifying multiple measures, system integration and placement decisions. This warehouse will be co-located for full redundancy to ensure system availability. Having a system that is consistently available and has data on one centralized platform will help colleges be more effective with their placement tests.

Procedure 5H, the Common Assessment Program will partner with the CCC Technology Center to provide development and support integration with the statewide student portal. This partnership will be formalized through a memorandum of understanding (MOU) and/or through a sub-grantee arrangement, including statement of work and budget item. This activity achieves the technical objective by ensuring system integration with a variety ofpartners, including the student portal. This will be a home for the pre-test as well as the marketing materials explaining the importance of the assessment.

Procedure 5I, hardware and bandwidth demands will allow for test taking and administration high bandwidth and low bandwidth locations, achieves the technical objective by ensuring the system is available to all colleges, regardless of their location. Specifications will be created and the system will be designed to work with low and high bandwidth locations.

Objective 6 Program Evaluation:

Procedure 6A, develop metrics to evaluate the impact ofprogram components on student, faculty, administrative and technical success and effectiveness, achieves the program evaluation objective by completing a needs documentation study. This study will help identify which elements of the program will have measurable metrics for evaluation purposes. Evaluations will be done throughout the project and recommendations will be discussed and implemented as appropriate. Ongoing evaluation will allow for all aspects of the project to be evaluated as well as find the needs for improvement as the project is being created, not after it is finished. Semiannual reports will be written and disseminated.

Objective 7 Planning:

Procedure 7 A, milestones for all project areas will be developed, tracked and communicated, achieves the planning objective by creating a project management chart to track each milestone. By tracking each milestone, the group will constantly know where in the process a specific milestone is. To help with the need for tracking, quarterly reports will be done for each milestone so there is accountability to completing the milestone in a timely manner, which will help ensure the project as a whole is completed on time.

Procedure 7B, the implementation plan for the initiative will be comprehensive, strategic and inclusive of all statewide project dependencies as appropriate, achieves the planning objective by having a planned pathway to complete the project. The implementation plan will be consistently updated to account for any changes that may occur in the program timeline. This will also

13

guarantee there is accountability to completing the milestone in a timely manner, which will help ensure the project as a whole is completed on time.

Procedure 7C, the project will develop and follow a plan for ongoing monitoring and sustaining activities, achieves the planning objective by evaluating all parts of the project. As recommendations are made and discussed, activities will be adjusted as appropriate. All reports and evaluations will be disseminated. Again, this will also guarantee there is accountability to completing each milestone in a timely manner, which will help ensure the project as a whole is completed on time.

Objective 8 Communication:

Procedure 8A, develop a comprehensive communication plan for the Common Assessment Program, achieves the communication objective by creating marketing materials for the project. Marketing will target faculty and staff to make them aware of the common assessment and training available and students to make them aware of the importance of the assessment and the av'ailability of the pre-test. This will allow for improved student placement.

' '

The project team will conduct surveys and focus groups of stakeholders, including faculty, community college students, K-12 students, K-12 parents and UC/CSU members, to evaluate communication materials, achieves the communication objective by confirming the accessibility and clarity of marketing materials to the various groups, particularly among students who are Latino, African American, Asian, low-income, English language learners, and those who traditionally place into remedial courses. Changes will be made as appropriate based on the evaluation feedback.

Performance Outcomes/Evaluation Design The Research and Planning Group for California Community Colleges (RP Group) has been engaged by the Santa Clarita Community College District to evaluate the proposed statewide Common Assessment Program. The RP Group will draw from more than 20 years of experience in evaluation, research, and the complexities of the California Community College (CCC) system to design and implement an evaluation that provides reliable quantitative and qualitative information that both informs progress achieved toward intermediate and final outcomes as well as contextualizes this information with perspectives obtained from key stakeholders.

Specifically, the RP Group's evaluation of the project will:

1. Generate information that key stakeholders need to make informed decisions throughout the project implementation process.

2. Utilize a rigorous technical design that is built on a platform ofknowledge about effective practices and integration of quantitative and qualitative research to create a multifaceted picture ofprogress toward outcomes.

3. Produce useful and meaningful information that not only guides implementation and enables the project leadership groups to make informed decisions, but also offers

14

guidance to others within and beyond the CCC system in the design and development of online systems of instruction.

A description of the overarching themes, methods, and context of the evaluation plan is provided below, with the specific activities and outcomes to be assessed detailed in the Evaluation Design Table. It is important to note, however, that this evaluation plan represents a starting point and initial framework for the project; as project objectives, outcomes, and budget details are adjusted post-award in collaboration with the Santa Clarita Community College District and the Chancellor's Office, the.evaluation will be correspondingly modified.

Key Evaluation Components

The proposed evaluation includes several interactive components, each of which is described in detail below.

Needs Documentation , 1 ' \

U~ing the Common Assessment Initiative RFA as a starting point, the RP Group will work with Common Assessment Program leadership and others develop an in-depth understanding of the quality and extent of the need for a statewide assessment system. For example, data to be collected will include:

1111 The number of students across the state who go through the placement process more than once as a result of changing colleges that don't accept the placement from other colleges;

1111 The variability in colleges' placement processes, including tests used, cut scores, and multiple measures;

111 The overall cost of placement tests to colleges; and 111 The aggregate cost of assessment to the system, including development of a metric of

quantitative cost savings to the system and state for improved placement.

To conduct this phase of the evaluation, multiple methods will be used, such as surveys of colleges, the Chancellor's Office Management Information System, and interviews with college staff.

Environmental Scan

For the Common Assessment Program to successfully achieve its goals, it must be guided by research into how other colleges, districts, and states have designed and implemented assessment and placement programs that facilitate student success and promote equity. The RP Group has extensive experience conducting literature reviews and environmental scans that inform large research initiatives; moreover, the RP Group has pioneered a new approach that extends this research component to run throughout the life of the project rather than simply providing an initial foundation of information. With this method, questions that arise as the design and implementation processes advance can be included in the environmental scan, with the appropriate expertise consulted for the duration of the project.

15

The RP Group will contribute this breadth ofknowledge and relevant experience to the expertise of the Common Assessment Program leadership, Cal-PASS, the Statewide Academic Senate, Student Success and Matriculation Professionals, and others in a collaborative effort that enables the project to leverage successful strategies at work around the country for the benefit of this project.

Evaluation Design and Implementation

At the foundation of the RP Group's work are the evaluation questions that will drive the evaluation. Based on a careful study of the RF A, the Santa Clarita Community College District and Cal-PASS team has collaborated with the RP Group to identify the following evaluation questions that will drive the evaluation. These questions address key aspects of the project's implementation approach, outcomes, and systems level impact:

1. Student success. How does a statewide common assessment impact student success? What is the impact of the new system on accuracy of placement and time to completion? How does a common assessment benefit students attending multiple colleges or transferring to ~me or more college during their course of study? What ls the interlocking impact of standardizing assessment processes and encouraging the use .ofmultiple measures on student outcomes of all kinds?

2. Equity of impact. How does this new system affect students across all groups? If reductions in the number of students placed in remediation are achieved, how are achievement gaps among various student groups affected? If increased use of multiple measures is achieved, how will placement of students from different backgrounds and/or with different academic backgrounds be impacted?

3. College structures, models, and funding. How will standardization of assessment and/or increased use ofmultiple measures impact colleges structurally and financially? What changes to counseling, academic advising, and other student supports will occur as a result of this project? How will the use of a common assessment system impact colleges' relationships with one another and their shared students?

As the RP Group conducts the specific assessments described in the Evaluation Design Table, short-term and long-term findings will be examined in light of these broader questions. In this way, data gathered and analyzed throughout the project will be used not just to inform the achievement of a particular objective, but also to provide insight into progress toward the larger goals of colleges, districts, and the State of California.

Assessment ofProgram Implementation

In addition to evaluating project outcomes in the context of the questions described above, the RP Group will also track the implementation of the Common Assessment Program. This process will assess the degree to which project activities were executed as planned and, when changes were made, identify the factors that led to course corrections/adjustments. Particular focus will be placed on assessing the degree to which feedback from stakeholders is being used to improve the project, including the Common Assessment Advisory Committee, the Common Assessment Steering Committee, faculty members, and prospective students, especially those who belonging to traditionally underserved ethnic/racial, income, and academic groups.

16

Dissemination to Internal and External Stakeholders

The final core component of the RP Group evaluation will be dissemination ofkey findings to internal and external stakeholders. Internal dissemination will be built into the evaluation design, ensuring that findings generated by the evaluation are shared in a timely manner with those who are in position to act on these findings. Because the project is large in scope and comprises individuals working in different teams and committees, the RP Group will participate in steering committee meetings and provide quarterly debriefings to this group and other project stakeholders as appropriate. At each debriefing, the evaluation team will provide an update on progress achieved toward each major outcome, and individuals will have the opportunity to ask questions, offer suggestions, and make requests. In this process, the RP Group will draw on its extensive experience translating research methods and language into information that is easily comprehensible and relevant to the intended audience.

To disseminate project findings externally, the RP Group will collaborate with Common Assessment Program lead~rship in using print, online, and other media formats to1broadly share critical information that arises from the project. Then, the RP Group will leverage its extensive network throughout the CCC system and beyond to conduct presentations, convenings, workshops, and more in order to help stakeholders inside and outside of the CCC system understand what these findings mean to them and how they can be used to improve student success.

Benefit of Outcomes

All outcomes and impact of the project will be systematically assessed at the:

College,

District,

Regional, and

State level.

This will include, but is not limited to:

Improvements in placement,

Increased student success and educational goal completion,

Reduction in the numbers of students assessed,

Financial savings, and

Economic impact ofnot only the cost savings, but more importantly the economic impact of greater student goal completion.

In addition to these impacts, all performance indicators listed in the Evaluation Design Chart, will be systematically evaluated at each level. Some of the project structure indicators ( e.g. committees formed, etc.) measuring implementation may have limited interest at the college and district level. However, indicators measuring implementation of activities at the college level and all outcomes will have greater relevance at all levels. Nonetheless, the evaluation will systematically consider all performance at all levels rather than predetermining or delimiting

17

analyses to only some levels to ensure that intended and unintended consequences are considered to the greatest extent possible.

THE BIGGER PICTURE: STRENGTHENING CALIFORNIA STUDENTS' PATH TO SUCCESS

In addition to conducting the evaluation of the Common Assessment Program, the RP Group is partnering with applicants to the Chancellor's Office's Education Planning and Online Education grants. While the development and roll-out of each of these three projects will be distinct and unique in many ways, all three projects are also intimately connected-each designed to bring quality improvement and standardization to critical aspects of the student experience and, in doing so, both broaden and clear away obstacles on students' pathway to success.

If given the opportunity to conduct evaluations for all three projects, the RP Group will work to cultivate coordination among the projects so that findings from each one inform and improve the others. In this way, the RP Group hopes to bring to California community colleges, and the California Community College system, a new level of illumination with respect to tackling the biggest obstacles to the success of our students.

Evaluation Design Table

18

Objective Procedures/ Activities Performance Outcomes

Timeline Responsible Person

Methods

lA. Governance - Ensure program success through effective governance structures.

lA. Establish a governance structure comprised of state and local California Community College, K12 and higher-ed leadership from the State Academic Senate, Academic Affairs, Student Services, Communications, and Technology. (RF A Procedure 1 A)

-

lAl Form a Common Assessment Steering Committee (CASC), with eight (8) members, including California community college faulty, deans and vice chancellors; State Academic Senate; Academic Affairs; Student Services; Communication; Technology; California Department of Education; K-12 school districts; and CSU/UCs.

lAl Common Assessment Steering Committee Formed

lAl Jan 2014

lAl Project Director

lAl Project records

1A2 CASC to conduct monthly meetings via CCCConfer and quarterly in-person.

Regularly convene and effectively manage the Common Assessment Advisory Committee and agenda. (RF A Procedure lB)

1A2 12 CASC meetings held annually with __ minutes of meeting available.

1A2 Jan 2014 and monthly meetings ongoing

1A2 Project Director

1A2 Project records (CASC minutes)

19

1A3 Form a Common Assessment Advisory Committee (CAAC) that will meet every other month to review and provide input on major strands of the project including specifications, test review, item development (if needed), item pilot analyses (if needed), test or scale development (if needed), delivery systems, integration with other systems, professional development and communications. Subcommittees will be formed to focus on major activities.

1A3 CAAC and subcommittees

--

1A3 Jan 2014

1A3 Project Director

1A3 Project records

1A4 CAAC meetings every other month with the sub-committees meeting in the off months.

1A4 12 CAAC/ subcommittee meetings held annually with minutes of meeting available

1A4 Jan 2014 and ongoing

1A4 Project Director

1A4 Project records (CASC minutes)

lB. Governance - The Common lB. Review, establish, modify and Assessment Program must operate vet policy and process related items legally, effectively and satisfy the to ensure the Common Assessment program and operational goals of Program can operate legally, the State, colleges and students. effectively and achieve program

objectives (RF A Procedure 1 C)

20

lBl lBl lBl lBl lBl Committees and stakeholders, Approved ProjectDec 2013 and Project including the Chancellor's Office policies, ongoing Director records Legal Affairs Division; Chancellor's procedures, Office divisions not already engaged, MOUsand district legal counsel and other contracts by experienced experts in large-scale respective assessment programs will review and stakeholders. approve items such as MOUs and contracts as needed.

2A. Assessment and Placement 2A

Develop a common test for English, Collaborate with faculty and staff

math and English as a Second across the California Community

Language for all California College system to develop and vet

Community Colleges. requirements for the common

assessment tests and

This will include: assessment/placement system (RF A

Procedure 2A) Reviewing specifications needed for assessment;

- - Evaluating off-the-shelf tests; Developing items (if needed); GI Validation of items;

Test delivery system; Pilot; and Rollout . e 2Al 2Al 2Al

Develop detailed specifications for 2Al 2Al

Project Project the course sequences in math,

Specifications Dec 2013 -Jan 2014 records ( test

English and English as a Second Director,for math,

specification Language that also are in alignment

AcademicEnglish and s)

with Common Core Standards. Senate, NB English as a Consulting,

Language. Second

IVC

21

2A2 Issue and RFP collecting existing commercially available and college developed tests.

2A2 A minimum of 6 off-the shelf placement tests collected

--

2A2 Jan 2014 - Dec 2014

2A2 Project Director, Student Success and Matriculatio n Professional s (SSMP), andCCCCO

2A2 Project records (RFP, Tests)

2A3 Evaluate commercially available placement tests including Accuplacer, EAP, MDTP, CTEP, and ICY A (ESL) and college developed tests relative to the detailed specifications and other criteria such as technical quality, costs, packaged delivery systems, etc.

2A3 A minimum of 6 off-the shelf placement tests evaluated

2A3 Jan 2014 - June 2014

2A3 Project Director, SSMP,RP Group, CASC

2A3 Project records (content validity and other test evaluation materials)

2A4 2A4 2A4 2A4 2A4 Contract with an entity to purchase the selected test (if selected).

Test purchase agreement

Aug 2014 Project Director and cccco

Project records ( test purchase agreement)

2A5 Develop new test if no test is selected using faculty from across California Community Colleges

2A5a Minimum of5(J. items per requisite skill

2A5a July 2014-Dec. 2014.

2A5a Project Director, Faculty, NB Consulting

2A5a Project records ( test items)

2A5b 2A5b 2A5b 2A5b Minimum of July 2014-Dec. Project Project 1000 items per 2014. Director, records ( test subject area Faculty, NB

Consulting items)

22

2A6 Develop scales from pools of items with adaptive testing algorithms conducting corresponding IR T and other analyses, including a study of adaptive tests vs. static forms.

2A6a English, Math, and ESL scales

2A6a Jan 2015 -Mar 2015

2A6a Project Director, CASC

2A6a Project records (scales)

2A6b 2A6b 2A6b 2A6b Adaptive Jan 2015 -Mar 2015 NB Project testing Consulting records algorithms (algorithms) 2A6c Study comparing adaptive test _ ~ and static forms.

2A6c Jan 2015 -Mar 2015

2A6c Project Director, IVC, Cabrillo College, NB

2A6c Placement tests, cccco MIS data, Cal-PASS Plus data

2A7 Validation of items (if needed)

2A7 Minimumof2 forms per assessment to measure evidence of predictive validity and show support for measures and differential impact.

2A7 Jan 2014-Dec 2014

2A7 Project Director, IVC, Cabrillo College, NB Consulting

2A7 Project records ( validation studies)

23

2A8 2A8 2A8 2A8 2A8 Pilot new assessments if needed New March 2015 -June Project Project receiving feedback assessments in

English, Math, and ESL

2015 Director, NB Consulting, IVC,RP Group

records (placement tests, surveys, student focus groups)

2A9 2A9 2A9 2A9 2A9 Conduct Reliability and Validity Validated June 2015 -Aug Project Placement Studies, including criterion ( concurrent and predictive) and differential impact as needed.

assessments in English, Math, and ESL

-

2015 Director, IVC,NB Consulting, Cabrillo College

tests, cccco MIS data, Cal-PASS Plus data

2B. Assessment and Placement 2B. Facilitate the development and Reduce the number of retests given adoption of processes and standards to community college students and to increase test portability across the increase the portability of system. (RF A Procedure 2B) assessment tests across the system.

2Bl Building from a relevant test measuring common requisite know ledge in English, Math, and ESL, develop systems allowing college defined cut-scores, multiple measures, and methods for combining measures per objectives and activities in 2F, 2, 3A, 4 and 8 below.

2Bla Decrease in number of students, attending multiple colleges, taking multiple tests each year by25%.

2Bla July 2014 and yearly

2Bla Project Director, RP Group

2Bla cccco MIS data, surveys

2Blb 2Blb 2Blb 2Blb 90% of colleges Jan 2015 and Project Project adapting the ongoing Director, records assessment test. CASC

24

2Blc Increase the number of colleges who will accept the scores from another college by 25 percent.

2Blc Jan 2015 and ongoing

2Blc Project Director, CASC

2Blc Survey/inter view of SSMP

2C Assessment and Placement fucrease the effectiveness and accuracy of test placement for students placing at or below college level.

2C. Ensure the common assessment tests are available for use at all community colleges. (RF A Procedure 2D)

2Cl 2Cl 2Cl 2Cl 2Cl Conduct a study comparing 20 percent March 2015 - June Cabrillo CA scores, placements at 15 sample colleges prior to new assessment, concurrent to new assessment, and after statewide roll-out.

improved placement, stronger validity related to course -completion. Differential impact for subgroup less then assessment prior to new common assessment.

2017 College, IVC,RP Group, and Project Director

cccco MIS data, Cal-PASS Plus data, focus groups

2D. Assessment and Placement 2D.fucorporate effective collection Lower remediation rates for and analysis of multiple measure California Community College data points to improve placement students and increase the initial and potentially reduce the need to placement level for students. assess students via the common tests.

(RF A Procedure 2J)

25

2Dl Conduct study of initial placement rates of students prior to use of common assessment ( at the 15 pilot colleges) and after implementation of common assessment (for all colleges) including the use of multiple measures data.

2Dl Reduce percentage of placement into remedial English, math and English as a Second Language by 20%

2Dl March 2015 and ongoing

2Dl Project Director, RP Group, Cabrillo College, IVC

2Dl Interview SSMP

2E. Assessment and Placement 2E. Administer a student "pre-test" Increase awareness of the that could be delivered via the importance ofplacement tests and internet to prospective students improve student preparation. before taking the actual assessment

tests. (RF A Procedure 2K) 2El Develop test preparation materials and sample tests available through the placement system.

2Ela 100,000 students annually will use the test prep materials by 2017.

2Ela June 2017

2Ela Project Director

2Ela Project records

2Elb 2Elb 2Elb 2Elb 90 percent of March 2015 to June RP Group Online students will report the test preparation materials and - helpful.

2017 student survey

2E2 2E2 2E2 2E2 2E2 Develop marketing campaign for 90 percent of March 2015 to June Project Project colleges on the importance of placement tests and preparation.

colleges will implement marketing campaign.

2017 Director, CASC, PIO.

records

26

2F. Assessment and Placement Assist colleges with efforts to improve local assessment and placement practices.

2F.Ensure the common tests and delivery platform are regularly reviewed and enhanced to meet the needs of the community college. (RFAProcedure 2G) 2Fl Work with colleges one-by-one in implementing the common assessment, including determining cut scores, multiple measures and marketing materials.

2Fla 80 colleges using the common assessment test.

2Fla June 2017

2Fla Project Director, RP Group

2Fla Project records, student focus groups, interviews

2Flb 2Flb 2Flb 2Flb 100,000 --students assessed.

June 2017 Project Director, RP Group

Project records

2Flc Description of cut scores and multiple measures identified.

2Flc June 2017

2Flc Project Director, RP Group

2Flc Surveys and Project records ( cut score report)

2F2 Conduct a study of the placement of students prior to and after use of the common assessment including use of online test preparation, multiple measures, and cut scores.

2F2a Improve placement by 25 percent.

2F2a June 2017

2F2a Project Director, RP Group

2F2a cccco MIS data, Survey

2F2b 2F2b 2F2b 2F2b Recommendati ons for improvement implemented.

June 2017 RP Group Interview SSMP, Student focus groups

27

2F3 Integration with campus websites and student information systems.

2F3 80 colleges websites with common assessment integrated.

2F3 June 2017

2F3 Project Director, RP Group

2F3 Project records (website)

2G. Assessment and Placement Reduce the cost of assessment related activities at the district/ college level.

2G.Negotiate vendor contracts so that test develop and administration costs are borne centrally and not by individual colleges. (RF A Procedure 2E)

Define and negotiate test development and administrative costs and features within the confines of the available common assessment budget. (RF A Procedure 2F)

-

2Gl 2Gl 2Gl 2Gl 2Gl Develop agreements allowing use of selected or developed tests at no additional cost for use by colleges

Tests available for no cost per use by colleges

June 2015 Project Director, cccco

Project records

2G2 2G2 2G2 2G2 2G2 Conduct a study of the total cost systemwide and per college, pre-and post-common assessment.

Reduce overall cost of assessment by 25%

June 2017 Project Director, CASC, Evaluators

Project records, Survey SSMP

2H. Assessment and Placement Leverage multiple measures data and research to lower the number of assessment tests issued and to improve placement.

2H. Identify, develop and implement reporting tools and services to facilitate and streamline assessment and placement at the colleges. (RF A Procedure 2I)

28

2Hl 2Hl 2Hl 2Hl 2Hl Develop delivery and reporting Platform July 2014 - June Project Project platform that draws on K-12 transcript and other data from Cal-PASS Plus.

containing K12 and Cal-PASS Plus data.

2015 Director, NB Consulting, Cabrillo College

records (CA platform)

2H2 Conduct a study and disseminate recommendations on strategies to leverage multiple measure data to lower number of assessments and improve placement.

2H2 Specific strategies that lower multiple measure data, number of elements and improve placements.

2H2 July 2015

2H2 Project Director, Cabrillo College, IVC

2H2 Literature review, CA Placement data, Cal-PASS Plus data

3A. Professional Development 3A. Partner with assessment vendors Provide faculty and staff with the and the @One Program to ensure training and skills necessary to be adequate, and relevant professional successful in the common development programs are -assessment environment. development and administered.

(RFA Procedure 3A) 3Al

Training programs offered will be offered online and in-person, and include technical components such as aspects of reliability and validity ( content, construct, criterion-related and consequential); differential impact; multiple measures; and recent relevant research studies for assessment in general and the common assessment system in particular. Badges will be earned for completion of workshops.

3Ala 3 professional development workshops offered per topic.

3Ala Jan 2014 for design and June 2014 will be offering

3Ala IVC, Project Director, RP Group

3Ala Project records (PD archived training)

29

- 3Alb 3Alb 3Alb 3Alb 90 participants Jan 2014 for design IVC, Project Project per topic. and June 2014 will

be offering Director, RP Group

records (PD sign in)

3Alc 90 badges ( certificates of completion) earned per topic.

3Alc Jan 2014 for design

and June 2014 will be offering

3Alc Project Director, RP Group

3Alc Project records (PD completion)

3Ald 95%of participants satisfied with the professional development.

3Ald Jan 2014 for design and June 2014 will be offering

3Ald Project Director, RP Group

3Ald Survey

3Ale 3Ale 3Ale 3Ale Recommendati Jan 2014 for design Project Survey ons for changes and June 2014 will Director, RP implemented. be offering Group

4A. Data Warehouse, Research and Validation Support - Provide a secure data warehouse to serve as a central repository for K-12 and higher education assessment data, including multiple measures.

4A.Partner with Cal-PASS Plus to provide a secure central assessment and multiple measures data warehouse (RF A Procedure 4A)

- -

30

4Al Develop specifications for delivery system, data warehouse, reporting requirements, locally setting cut scores, locally setting methods for identifying multiple measures, system integration, ability to incorporate multiple measures ( such as employment expectations and high school transcript data) and placement decision.

4Ala Secure, central assessment and multiple measures data warehouse created.

-

4Ala Jan 2014-Mar 2014

4Ala Project Director, CAAC Subcommitt ee,San Joaquin Delta College (SJDC), Cabrillo College, NB Consulting

4Ala Project records (CA system, Cal-PASS Plus data)

4Alb Co-location for full redundancy to ensure system availability setup.

4Alb July 2014-Dec 2014

4Alb Project Director, CAAC Subcommitt ee, San Joaquin Delta College (SJDC)

4Alb Project records

4A2 Working with Cal-PASS Plus, develop new or existing storage and test delivery and data.

4A2 Storage and test delivery and data integrated with Cal-Pass Plus - -~

4A2 Jan 2014- Mar 2014

4A2 Project Director, CAAC Subcommitt ee, San Joaquin Delta College (SJDC)

4A2 Project records (Cal-PASS Plus)

31

4A3 4A3 4A3 4A3 4A3 Pilot delivery system 15 Colleges

using the new system

June 2017 Project Director, CAAC Subcommitt ee, San

Project records (platform use)

Joaquin Delta College (SJDC)

4A4 4A4 4A4 4A4 4A4 Roll-out statewide. 80 Colleges

using the new system.

-

July 2017 Project Director, CAAC Subcommitt ee, San

Project records (platform use)

Joaquin Delta College (SJDC)

4B. Data Warehouse, Research and Validation Support - Improve assessment and placement through robust research.

4B. Partner with Cal-PASS Plus to conduct research and validation support functions. (RF A Procedure 4B) 4Bl 4Bl 4Bl 4Bl 4Bl Working with Cal-Pass Plus, Terrance Willett (Cabrillo College), the RP Group and CASC, review and adjust research agenda and statewide and local validation studies and research studies evaluating use of common assessment and the impact in placement of students statewide.

Research and local validation studies with action implications.

Mar 2014 to June 2017

Cal-PASS Plus, Cabrillo College, IVC and the RP Group

CA data, Cal-PASS Plus data, cccco MIS data

32

4C. Data Warehouse, Research and Validation Support Support colleges by providing valuable assistance as appropriate.

4C. Implement sufficient standards to allow for the development of optimization algorithms of placement success based on test score and highest level of subject taken (RF A Procedure 4C) 4Cl Conduct studies of cut scores and methods for combing data for placement of students, including use of high school transcript data.

4Cl Algorithms for placement include cut scores and multiple measures.

4Cl July 2015 and ongoing

4Cl Project Director, Cabrillo College, IVC

4Cl CA scores, Cal-PASS Plus

4C2 Work with initial colleges in implementing cut scores and multiple measures that identifies student placement.

4C2 80 colleges implementing algorithms.

4C2 July 2015 and ongomg

4C2 Project Director, Cabrillo College, RP Group

4C2 Project records

5A. Technical - Leverage 5Al. Organize a committee (CAAC 5Al 5Al 5Al 5Al technology to standardize and Subcommittee) to define system CAAC - Jan 2014 and meet Project Project enhance the user ( student, faculty, requirements and develop system Subcommittee every other month. Director and records staff) experience. In addition to design and implementation plans. formed. CAAC (minutes) improving testing services and (RFA Procedure 5A) Subcommitt features and accessibility. ee

33

5A2.The Common Assessment Program will utilize the Systemwide Student Account/ Identifier (CCCID) (a CCC Technology Center project) to enable single sign on functionality across the System. The California Community Colleges will participate in the InCommon Federation. (InCommonF ederation.org) for single sign-on (SSO) using Shibboleth's Identity Provider and Service Provider software (shibboleth.intemet2.edu). Students and staff will use their local college logins to access statewide web services such CCCApply, Education Planning Program and the CVC Expansion Project. (RF A Procedure SC)

The Common Assessment Program will support the transfer of relevant student academic and support data into and out of the system, including but not limited to, Chancellor's Office MIS, local SIS, Statewide data warehouse (Cal-PASS Plus). (RFA Procedure 5D)

5A2 System specifications that include CCCID,DDO, DED, system integration with CCCCO MIS, -local SIS, and Cal-PASS Plus.

- ---~

5A2 Jan 2014 and meet every other month.

5A2 Project Director and CAAC Subcommitt ee

5A2 Project records (system specification s)

5A3 Faculty and staff should have the ability to easily identify and access assessment information as appropriate for job role and function. (RF A Procedure SE)

5A3a Specifications for reporting by role and methods for accuracy.

5A3a Jan 2014 and ongoing

5A3a Project Director and CAAC Subcommitt ee, SSMP

5A3a Project records (system specification s)

34

5A3b 5A3b 5A3b 5A3b Delivery Jan 2014 and Project Project system with ongomg Director and records reporting for CAAC (system specified job Subcommitt specification roles. ee, SSMP s 5A3c

Online training modules for system-wide use

5A3c Jan 2014 and ongoing

5A3c Project Director and CAAC Subcommitt ee, SSMP

5A3c Project records (online training archives)

5A3d 90 percent satisfaction with using the -system by job role.

5A3d July 2015

5A3d Project Director and CAAC Subcommitt ee, SSMP

5A3d Online survey

5A4 Develop and maintain a web based presence for the Common Assessment Program (RF A Procedure SF)

5A4a Website developed and available.

5A4a July 2015

5A4a Project Director and CAAC Subcommitt ee

5A4a Project records (website)

5A4b 5A4b 5A4b 5A4b 90 percent satisfaction of website by type of user.

July 2015 RP Group Online survey

SAS Hardware and bandwidth demands will allow for test taking and administration high bandwidth and low bandwidth locations. (RF A Procedure 51)

5A5 Specifications for high and low bandwidth access.

5A5 June 2014

5A5 Project Director, SJDC and CAAC Subcommitt ee

5A5 Project records (system specification s)

35

SB. Technical Statewide technology system integration and standards will be a guiding principle throughout the lifecycle of the project.

SB. Consistent data and technical standards will be developed, implemented and maintained to enable the automated transfer of data and systems integration. The program will follow the technology standards and recommendations approved by the Chancellor's Office and the Systemwide Architecture Committee (SAC), including but not limited to the Postsecondary, Educational Standards Committee (PESC), Federated Identity Management, and Service Oriented Architecture. (RF A Procedure 5B)

5B Common Assessment System specifications for data element and protocols for transferring information in a secure environment that meets system integration and follows SAC, including PESC, FIM and- SOA.

5B Jan 2014 and