![Page 1: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/1.jpg)
Jasmine CareyCDE Psychometrician
Interpreting Science and Social Studies Assessment
Results
September 2014
![Page 2: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/2.jpg)
CMAS Science and Social Studies Reports
2
![Page 3: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/3.jpg)
3
Performance Level: Top two levels are indicators of being on track for college and careerDistinguished Command Strong CommandModerate Command Limited Command
Types of Scores Appearing on CMAS Reports
![Page 4: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/4.jpg)
4
CMAS Performance Levels and Policy ClaimsDistinguished Command
Students performing at this level demonstrate a distinguished command of the concepts, skills, and practices embodied by the Colorado Academic Standards for Science or Social Studies assessed at their grade level. They are academically well prepared to engage successfully in further studies in this content area.
Strong Command
Students performing at this level demonstrate a strong command of the concepts, skills, and practices embodied by the Colorado Academic Standards for Science or Social Studies assessed at their grade level. They are academically prepared to engage successfully in further studies in this content area.
Moderate Command
Students performing at this level demonstrate a moderate command of the concepts, skills, and practices embodied by the Colorado Academic Standards for Science or Social Studies assessed at their grade level. They will likely need academic support to engage successfully in further studies in this content area.
Limited Command
Students performing at this level demonstrate a limited command of the concepts, skills, and practices embodied by the Colorado Academic Standards for Science or Social Studies assessed at their grade level. They will likely need extensive academic support to engage successfully in further studies in this content area.
![Page 5: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/5.jpg)
5
Social Studies: Approximately 15% of students scored at the Distinguished or Strong Command level
Science: Approximately 30% of students scored at the Distinguished or Strong Command level
State Level 2014CMAS Performance Level
Results
![Page 6: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/6.jpg)
6
Overall scale scoreComparisons within year and across years can be made
Subscale scoreComparisons within year and across years can be made
Percent correctComparisons within year can be made
Types of Scores Appearing on Reports (cont.)
![Page 7: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/7.jpg)
7
Total Test Standard Level (Reporting Category) Prepared Graduate Competency (PGC) Grade Level Expectation (GLE) Item Type
State District School Subgroups Individual student
Levels for Reporting
![Page 8: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/8.jpg)
8
Student Performance Report Intended for Parents and Teachers
Content Standards Roster School and District level
Performance Level Summary School and District Level
Item Analysis Report School and District level
Only if the number of students is greater than 15
CMAS Science and Social Studies Reports
![Page 9: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/9.jpg)
9
Individual Student Performance Intended audience is Parents and TeachersReports individual student performance down to the PGC/GLE level
Scale ScoresOverall “Reporting Category”
Content Standards and Scientific Inquiry/Nature of Science Item Type (constructed response and selected response)
Percent CorrectPGCGLE, if there is more than one GLE in a PGC
CMASStudent Performance Report
![Page 10: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/10.jpg)
10
This graph shows the overall scale score for the assessment and the performance levelRange of 300 to 900Arrows around students scores are the standard error of measurement
CMAS Overall Scale Score and Performance Level
![Page 11: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/11.jpg)
11
CMAS Subscale Performance
1 SD Around State Average
![Page 12: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/12.jpg)
12
Prepared Graduate Competencies and Grade Level ExpectationsNot shown if only one GLE in a PGC
CMAS PGCs and GLEs
Text for PGCs and GLEs
![Page 13: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/13.jpg)
13
Percent correct scores are based on the particular items given in that year’s testCannot be compared across years
Can compare to District and State to spot trendsBased on a small number of items (between 6 and 8)
Don’t necessarily hit the whole breadth of a topic Should not be the sole basis for instructional adjustments
CMAS Percent Correct Scores
![Page 14: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/14.jpg)
14
Nearly half the points on the CMAS assessments come from Constructed-Response items This is a scale score and can be compared across years
CMAS Selected-Response and Constructed-Response
![Page 15: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/15.jpg)
15
School level report For the Spring 2014 administration the District will receive all their
school rosters in a single PDF Includes student performance on:
Overall assessmentContent StandardsPGCs/GLEs
CMAS Content Standards Roster
![Page 16: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/16.jpg)
16
Key: Lists performance levels with scale score
ranges Explains Content Standards Performance
Symbols School Summary
# and % of Students in school at each Standards Performance Indicator
CMAS Content Standards Roster(Page 1)
![Page 17: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/17.jpg)
17
Scale Scores by State, District , and School
Individual Student Performance
CMAS Content Standards Roster(Page 1)
![Page 18: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/18.jpg)
18
Page 2 reports the PGC and GLE Performance The header includes the points possible and the average % for
State, District, and SchoolPGC and GLE numbering is the same as on the SPR
The language and numbering on the SPR will be included in the Interpretive Guide for easy reference
Content Standards Roster(Page 2)
![Page 19: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/19.jpg)
19
State, District, and School levels Percent of students meeting each performance level
No Scores are NOT included in the denominator for the % in each level
Disaggregated by groupGenderRace/Ethnicity Language ProficiencyDisability, etc.
CMAS Performance Level Summary
![Page 20: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/20.jpg)
20
Includes data for the state on the district level report and state and district on school reports
Average Scale Scores Performance Levels Distribution
Performance Level Summary
![Page 21: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/21.jpg)
21
School, District, and State level Performance on individual items
1 point items: % of students answering correctly2-3 point items: average points earned divided by the number of
possible points Includes item information
CMAS Item Analysis Report
![Page 22: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/22.jpg)
22
Graph: Each item for State, District,
and SchoolOrdered by Difficulty from Most
to Lease Difficult (at the State level)
Label on the bottom has the Standard and GLE
CMAS Item Analysis Report(Page 1)
![Page 23: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/23.jpg)
23
Item InformationOrdered by Difficulty - matches the order on Page 1 Location on the test – Section/Item numberRelated PGC and GLE Item type
CMAS Item Analysis Report(Page 2)
![Page 24: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/24.jpg)
CoAlt Science and Social Studies Reports
24
![Page 25: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/25.jpg)
25
4 point items (selected response)4 points – answered correctly independently3 points – answered correctly with an additional prompt2 points – answered correctly after being told the correct answer 1 point – answered incorrectly after being told the correct answer0 points – did not respond to the question
2 point items (in groups of 3)2 points – answered correctly 1 points – answered incorrectly0 points – did not respond to the question
CoAlt Scoring
![Page 26: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/26.jpg)
26
CoAlt measures both entry skills and content Points available for demonstration of entry skills: On a 17 item
test (all but 8th grade Science)36 points can be earned with no evidence of content knowledge These points allow students to demonstrate consistency in entry
skills (responding when asked a question and repeating an answer) CoAlt is intended to measure content knowledge
All performance levels above Exploring require evidence of some content knowledge
Cut scores for the performance levels are at the higher end of the point/scale range due to the way the items are scored
Interpretation of Overall CoAlt Scores
![Page 27: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/27.jpg)
27
CoAlt has four performance levelsNoviceDeveloping Emerging Exploring
Novice and Developing are considered ready to continue academic study in this area
Inconclusive Designation Students who attempt the assessment, but do not respond to any
items
CoAlt Performance Levels
![Page 28: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/28.jpg)
28
CoAlt Performance LevelsCoAlt Performance Levels and Policy Claims
Novice Students performing at this level demonstrate a solid understanding of the concepts and skills represented by the Extended Evidence Outcomes (EEOs) of the Colorado Academic Standards (CAS). They are academically well prepared to engage in further studies in the content area with appropriate supports.
Developing Students performing at this level demonstrate a foundational understanding of concepts and skills represented by the EEOs of the CAS. They are academically prepared to engage in further studies in the content area with appropriate supports.
Emerging Students performing at this level demonstrate a limited understanding of concepts and skills represented by the EEOs of the CAS. They will likely need moderate academic supports to engage successfully in further studies in the content area.
Exploring Students performing at this level demonstrate an initial understanding of concepts and skills represented by the EEOs of the CAS. They will need extensive academic supports to engage successfully in further studies in the content area.
![Page 29: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/29.jpg)
29
In Social Studies, approximately 40% of students are in the Developing or Novice level
In Science, approximately 42% of students are in the Developing or Novice Level
State Level 2014CoAlt Performance Level Results
![Page 30: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/30.jpg)
30
Student Performance Summary Intended for Parents and Teachers
Content Standards Roster School and District level
Performance Level SummaryDistrict Level Only
CoAlt Science and Social Studies Reports
![Page 31: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/31.jpg)
31
Individual Student Performance Scale Score
Overall test score Percent Correct
Content Standards
CoAltStudent Performance Report
![Page 32: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/32.jpg)
32
Scale ScoreRange of 1 to 250 First 100 are compressedArrows around student scores are the SEM
CoAlt Overall Scale Score
![Page 33: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/33.jpg)
33
Compares student percent correct to state average percent correctPoints earned and points possible are listedContent Standard Descriptions are from the Colorado Academic
Standards and are the same as CMAS
CoAlt Content Standard Performance
![Page 34: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/34.jpg)
34
Average Overall Scale Score and Content Standards % Correct for State, District, and School
Individual Student level
CoAlt Content Standards Roster
![Page 35: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/35.jpg)
35
Only at the District level Includes the Inconclusive designation
Those students are in the denominator for the Performance Level %
CoAlt Performance Level Summary
![Page 36: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/36.jpg)
36
Schools and Districts will receive Electronic Files – available through PearsonAccess
Student Data File Summary File
Electronic PDF reports – available through PearsonAccess Performance Level Summary Reports Content Standards Roster Reports Student Performance Reports Item Analysis Reports
Hard copy reports – Sent to DACs Student Performance Reports (two copies)
CDE will provide sample letters for use with parents and staff that can be modified as districts deem appropriate.
Format ofCMAS and CoAlt Results
![Page 37: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/37.jpg)
37
Only scaled scores can be compared across yearsOverall, Content Standards and Item Type for CMASOverall only for CoAlt
Scores cannot be compared across gradesNo vertical scaling
Percent scores cannot be compared across years, grades or PGCs/GLEs Items in a PGC/GLE will change between years
The Usual Disclaimers
![Page 38: Jasmine Carey CDE Psychometrician Carey_j@cde.state.co.us Interpreting Science and Social Studies Assessment Results September 2014](https://reader030.vdocuments.site/reader030/viewer/2022032606/56649e875503460f94b8a96e/html5/thumbnails/38.jpg)
Questions?
38