Developing a Developing a Next Generation Common Placement Next Generation Common Placement Assessment for the C6 ConsortiumAssessment for the C6 Consortium
1
Assessment for the C6 ConsortiumAssessment for the C6 Consortium
Barbara Hioco, Ed.D, CVHECRyen Hirata, Reedley College-Willow InternationalSaundra McGlothlin, MSW, West Hills College CoalingaMai Meidinger, Merced CollegeMatthew Schultz, PhD, McCann Associates
I. Significance of Common AssessmentII. Historical Overview & Background
III. The Drawing Board: Guidelines
Presentation OverviewPresentation Overview
III. The Drawing Board: Guidelines
IV.The Process
V. Conclusion
Historical Overview & BackgroundHistorical Overview & Background
3
The Process of Developing a Next Generation Common Placement Assessment
for the C6 Consortium
▶ Bakerfield*▶ Cerro Coso*▶ College of the Sequoias▶ Columbia▶ Fresno City*▶ Merced*▶ Modesto*
C6 ConsortiumC6 Consortium
▶ Modesto*▶ Porterville*▶ Reedley – Willow Intl*▶ SJ Delta*▶ Taft▶ WHC Coalinga*▶ WHC Lemoore*
*Identify active participants
�Address Central Valley Colleges’ placement test concerns
�New state legislation and mandates
Why Common Assessment? Why Common Assessment?
�New state legislation and mandates
�No definitive timeline for a statewide common assessment (at that time)
What are Current Test Challenges?What are Current Test Challenges?�Inaccurate placement of students
�High costs
�Not compatible with college data systems
�No diagnostic test
�No college curriculum alignment
�No common placement between sites/colleges
�Little to no support with cut score setting
�Overall, colleges are not satisfied with current test
▶ Community colleges in the central valley
▶ Instructional faculty (subject matter experts)
CollaborationCollaboration
▶ Assessment/Matriculation specialists
▶ Administration
▶ Test vendor
The Drawing Board: GuidelinesThe Drawing Board: Guidelines
8
The Process of Developing a Next Generation Common Placement Assessment
for the C6 Consortium
Drawing BoardDrawing Board
� Framework
� Development process
Functionality� Functionality
� Adaptability
� Common Cut
Test FrameworkTest Framework▶Web-based▶ Adaptive & diagnostic▶ Accessible: individuals with disabilities▶ Timeline goal: 2 hours▶ Common cut scores (Uh Oh!)▶ Adaptability (How?) ▶ Price point▶ Data warehouse
Development process typically used by vendors producing test for CCC.
Designed to be “off the shelf” tests
Primarily Adaptive (diagnostic available for additional cost)
Diagnostic typically not part of course placement
Current Vendor Development ProcessCurrent Vendor Development Process
Subject matter solely determined by test developer (really unknown to faculty/staff/college)
Developed to address general subject matter and High School standardized tests/core standards
No commonality: All cut scores vary between colleges
Background questions set by test developer and are customizable
C6 created a new development process based on faculty input and community college curriculum.
Placement test will include adaptive and diagnostic test portions
Adaptive and diagnostic will be used to determine course placement
Subject matter reviewed and selected by faculty within specified discipline
C6 Development ProcessC6 Development Process
Subject matter reviewed and selected by faculty within specified discipline (i.e., English-R/W, Math, and ESL)
Developed based on community college curriculum standards (C-ID) and course objectives
Common cut scores for transfer level English and degree level math
Background questions developed by CC to have standard and set specific questions
C6 Test FunctionalityC6 Test Functionality
▶Score availability, portability, comparability
▶Web-based
▶Reduce self-selection of appropriate test
▶Seamless transition ▶Seamless transition
▶Accessible for all students
▶Compatible with college data systems
▶Provides data for reports (i.e., MIS, SS, & validation)
Test Common FeaturesTest Common Features
▶The test questions, criteria, scoring
▶The required background questions
▶The required data on results page▶The required data on results page
▶Common cut scores (transfer level English; degree applicable math only)
Test CustomizationTest CustomizationWhat can be customized?
▶Slide background design
▶Additional background questions
▶Information & comments on results page▶Information & comments on results page
▶Local cut scores (except transfer level English & degree applicable math)
Cut ScoresCut Scores▶ Determined by C6 faculty
▶ Common cut scores: Transfer level English and degree level math
▶ Common cut scores accepted by all participating colleges
▶ Local cut scores determined by individual colleges to align with local curriculum
The ProcessThe Process
17
The Process of Developing a Next Generation Common Placement Assessment
for the C6 Consortium
Determine AssignmentsDetermine AssignmentsC6 & College Faculty/Staff/ Admin. McCann
Utilize Faculty/Staff input for test development
Provide a test Platform: “College Success”
Identify objectives and test items related to curriculum objectives (C-ID)
Provide test question banks
Determine required background questions to be used as standard
Create a new test based on C6 needsto be used as standard
Determine common cut scores for transfer level English and associate degree level Math
Provide automated scoring for test, including AET (Automated Essay Scoring)
Collect and house test data
1. Identify project guidelines
2. Create test flow
3. Identify adaptable test components (Blue-Prints)
4. Identify diagnostic test components (Blue-Prints)
5. Compose a working draft of test
Project OutlineProject Outline
6. Collect data and analyze
7. State Chancellor’s Office approval
8. Establish common cut scores & local cut scores
9. Pilot test
10. Continuous field testing & psychometric monitoring
� 2012▶ C6 began discussions with McCann
▶ C6 Open forum
▶ C6 Placement test development workshop
Project ActivitiesProject Activities
� 2013▶ Established lead faculty to plan/coordinate pilot project
▶ Pilot project test development
▶ Pilot test platform developed by McCann
▶ Data collection and review
Test Flow & Blue Print DevelopmentTest Flow & Blue Print Development
English/ESL Writing/Reading
Math
� Faculty/Staff driven
� Aligned with curriculum/course (C-ID)
Test ComponentsTest ComponentsTest Subject Method of Test Descriptors
Adaptive Engl/ESL Writing Essay Writing Sample Utilize Automated Essay Scoring (AES) Technology
Engl/ESL Reading Multiple Choice • 300+ items per bank
• Items calibrated • Items calibrated using 3 parameter IRT Math Multiple Choice
Diagnostic ESL Writing Mutiple Choice Provides score reporting on competency of specific objectives
Engl/ESL Reading Multiple Choice
Math Multiple Choice
Test SequenceTest Sequence
ADAPTIVE TEST
LINEAR DIAGNOSTIC TEST
PLACEMENT
C6 Test: Latest VersionC6 Test: Latest Versionhttp://college.measuredsuccess.com/
24
What Next?What Next?
� Data collection for validation
� Submit for approval from the Chancellor’s Office
� Set cut scores
� Continuous field testing & ongoing psychometric monitoring
ConclusionConclusion
29
The Process of Developing a Next Generation Common Placement Assessment
for the C6 Consortium
Test Comparison: Current vs. C6Test Comparison: Current vs. C6
Characteristics of current products Characteristics of C6
All colleges use different placement test products
All colleges can use a common placement test product
Primarily Adaptive (diagnostic available for additional cost)
Placement test will include adaptive and diagnostic test portions
Diagnostic typically not part of course Adaptive and diagnostic will be used to Diagnostic typically not part of course placement
Adaptive and diagnostic will be used to determine course placement
Subject matter solely determined by test developer (really unknown to faculty/staff/college)
Subject matter reviewed and selected by faculty within specified discipline (i.e., English-R/W, Math, and ESL)
Developed based on general subject matter and High School standardized tests/core standards
Developed based on community college curriculum standards (C-ID) and course objectives
No commonality: All cut scores vary between colleges
Common cut scores for transfer level English and degree level math
Background questions set by test developer
Background questions developed by CC to have standard and site specific
BENEFITS OF C6 PROJECT
Faculty
drive
Utilized the
Creation of
Provide a
uniform
Uniform
interface for
Provide
Facilitate
Improve
availability, portability,
Vision:
Utilize our drive
n test developme
nt
the expertise of professionals
on of a new test
model
m assessmen
t instrument
for stude
nt testing and administration
de faculty with class data
ate cost savin
gs
bility, comparability of data
between
colleges
e our experience
& expertise
32
The Process of Developing a Next Generation Common Placement Assessment
for the C6 Consortium