the impact of emerging trends in assessment / testing on educational technologies & vice versa

The Impact of Emerging TrendsThe Impact of Emerging Trends in Assessment in Assessment // Testing on Testing on Educational Technologies Educational Technologies

&& Vice Versa Vice Versa

The Impact of Emerging Trends in Assessment/ The Impact of Emerging Trends in Assessment/ Testing on Educational Technologies Testing on Educational Technologies & Vice Versa& Vice Versa

David E. W. MottDavid E. W. MottTests for Higher StandardsROSworks LLC

Presented at the VSTE Regional Technology IntegratorPresented at the VSTE Regional Technology IntegratorMini-ConferencesMini-Conferences inin AlbemarleAlbemarle County,County, FranklinFranklin County,County, && the City of Virginia Beach, VA, July 20, 22, the City of Virginia Beach, VA, July 20, 22, && 27, 2010 27, 2010

07/19/2010 © David E. W. Mott, ROSworks, LLC

3

About Me (so you know where I’m coming from)

Started as a psychologist.Was at the VA DOE for 17+ years, in what is now

called Assessment and Reporting. Supervisor of Test Development (and

psychometrics – test equating, etc.) Administrator of the VSAP (ITBS, SAT-9)

Got with Stuart Flanagan to do SOL Math tests – then English, Science, & H/SS.

Developed ROS so that teachers didn’t have to hand score tests.


4

Commercial Message —

Helping schools put the pieces of the assessment puzzle together.

TE S T S F O R H I G H E R ST A N D A R D S

PROVIDE FOCUS + FACILITATE ACHIEVEMENT


5

Commercial Message —


6

Session Abstract Even though Virginia will not be adopting the Common Core State Standards nor participating in one of the two assessment consortia, many national trends in assessment technologies will greatly impact the technological landscape. It appears that many forces surrounding the assessment processes will place new and unique demands on school technologies and on technologists. Some examples will be the greatly increased need for assistive technologies, advanced assessments modes requiring ever more complex computer environments, needs for increased data monitoring and data verification, artificial intelligence technologies deployed to the desk top, and many others. Technologists will be expected to support the rest of the educational community. We will illustrate some of these themes using the ROS software as a presentation platform.


7

PrologNEVER FORGET

TheW Y T I W Y G

Principle

What You Test Is What You Get


8

The Examination – Large room, Examinees (seated apart) some looking desperate, Invigulators, Hourglass, Closed Doors

The ExaminationThe Examination !!as shown by the Ghost of Testing Pastas shown by the Ghost of Testing Past

NotNot that much has changed, has it?that much has changed, has it?

07/19/2010 page rev. 12/06/2010

© David E. W. Mott, ROSworks, LLC

9

The National Situation The Common Core National Standards in ELA and Math, K – 12

have been adopted by 44 states, as of December 6, 2010. NOT adopted by VA!

(VA is surrounded.) A follow-on to these

Standards is an assess-ment program.

Three consortia of states — the SMARTER Balanced Assess- ment Consortium [31 states], the Partnership for the Assessment of Readiness for College and Careers (PARCC) [26 states], andState Consortium on Board Examinations Systems (SCOBES) 12 states].

Of course,VA is in none! As of 12/06/2010


10

SMARTERThe SMARTER Balanced Assessment Consortium.

(SMARTER) [Fiscal agent – State of Washington] {31 states} A high-quality, balanced multi-state assessment system Options for customizable system components while also

ensuring comparability of high-stakes summative test results across States.

A variety of item types and performance events to measure the full range of the CCSS and to ensure accurate assessment of all students, including students with disabilities, English learners, and both low- and high-performing students.


11

SMARTER – Details 1

Summative TestsA Common CCSS-based computer adaptive summative assessments that make use of technology-enhanced item types and teacher-developed and scored performance events

Interim/Benchmark Assessments Computer adaptive — reflecting learning progressions or

content clusters Classroom Formative Assessments

Research-supported instructionally sensitive tools, processes, and practices developed by State educators to improve teaching and increase learning


12

SMARTER – Details 2

Teacher supportFocused ongoing support to teachers through professional development opportunities and exemplary instructional materials linked to the CCSS

An online reporting and tracking system To provide information about student progress toward college- and career-readiness and about strengths and limitations in what students know and can do at each grade level

Cross-State communications network To inform stakeholders about SMARTER activities and ensure common focus on college- and career-readiness for all


13

SMARTER – Summary

High-tech computer testing for adaptive testing with innovative, complex item types

High-tech collaboration for assessment developing, scoring, results sharing

High-tech classroom support systems

High-tech shared databases to store, crunch, and report assessment results, including longitudinal information


14

PARCCThe Partnership for the Assessment of Readiness

for College and Careers (PARCC) [Fiscal agent – State of Florida] {26 states} A comprehensively designed assessment system Strategic use of selected-response items, constructed-

response items, technology-enhanced items, and performance events with an emphasis on problem solving, analysis, synthesis, and critical-thinking skills (HOTS)

Providing accurate assessment of all students — including students with disabilities, English learners, and low- and high-performing students


15

PARCC – Details and Summary

Summative TestsThe end-of-year assessments (after 9/10 of instruction) in ELA and Math will be computer-enhanced and incorporate computer-scorable items to assess higher order skills.

Interim/Benchmark Assessments In ELA and Math 3-8 & EOC, students will take focused assessments after ¼ & ½ of instructional time. Then an extended and engaging performance-based task at the ¾ point. In ELA/Literacy, students will identify or read relevant research materials and compose written essays, publicly present the results, and answer questions or debate, so teachers can assess speaking and listening skills.

Results All back in time for inclusion on Report Cards!


16

SCOBES The State Consortium on Board Examinations Systems

(SCOBES) [Fiscal agent – State of Kentucky] {12 states} An assessment system for high school students

designed around existing rigorous products of a number of Board Examination System providers aligned to the Common Core State Standards

They will rely heavily on essays and constructed-response questions and including coursework and assigned projects

There will be separate Lower Division (leading directly to community or technical college transfer, certification, or associate degree programs or to Upper Division course of study as a preparation for entry into university or industry certification programs


17

National Situation Summary The SMARTER Balanced Assessment Consortium

(SBAC) supports the development and implemen-tation of learning and assessment systems to radically reshape the education enterprise in participating States in order to improve student outcomes.

Fully implement the summative assessments in grades 3–8 and high school by 2014 –15 school year.

SCOBES if successful will restructure the basic conduct of secondary education in participating states.


18

Humor

Time flies like an arrow;

fruit flies like a banana.


19

Humor




20

Humor




21

Humor



The point is what will happen to education in the near future will require of us just such a change of reference.


22

So, why worry!

As Virginia is a state that has so far declined to adopt theCommon Standards and is also not participating in any of the three assessment consortia, logically Virginia educators should Not be affected by the Common Standards or by the assessments.

Alfred E. Newman © Mad Magaxine & Norman Mingo


23

So, why worry!

Not True! It seems likely that Virginia will have to appear to be just as innovativeand forward-looking as any of the participating states.

Alfred E. Newman © Mad Magaxine & Norman Mingo


24

So, why worry!

Please, some Discussion

Alfred E. Newman


25

A look to the almost presentThe Virginia Grade Level Assessments

(VGLA) have been in use in Virginia for several years as an alternate SOL assessment for student who cannot take the regular SOL tests even with accommodations.

The VGLA have been low tech assessments.

This is about to change —


26

The VMAST programThe Virginia Modified Achievement

Standards Test in math for grades 3-8 and Algebra I will be introduced during 2011-2012. Reading assessments will be introduced in the following year.

Although these tests are designed for special populations, we can look at them to see how some future enhanced Virginia SOL tests may look.


29

At a recent testing conference

A paper was presented about an artificial world science assessment.

A avatar representing a student is placed in an artificial world containing a lake with an overgrowth of algae.

The avatar has access to various scientific instruments (pH, dissolved-oxygen meters, etc.)

The task is to discover why the algae are growing by performing the necessary experiments.


30

Two examples of what can go wrongg with technology.

The first involves batteries on 4-function calculators.

The second incident involves the Wyoming online testing program and revolves around a testing company well-known in VA.


31

So what does this mean for YOU?

More collaboration for you, as the central staffs have neither the expertise nor the personnel to do it all.

More money needed for technology.Ubiquitous computers.Many disruptions because of tech-

nology breakdowns – many of which you will be expected to “fix”.


32

Graphical Presentation of the SMARTER Proposal


33

Graphical Presentation of the PARCC Proposal


34

Graphical Presentation of the SCOBES Proposal


35

Final Discussion


36

Questions, comments, requests for copies, etc.

David E. W. Mott, Ph.D.

Tests for Higher Standards /

ROSworks, LLC

866-724-9722, 804-282-3111

dem@ rosworks.com or go to the blog www.thoughtsonassessment.com

the impact of emerging trends in assessment / testing on educational technologies & vice versa

Documents

assessment technologies

assessment testing

assessment consortia

assessment processes

assessment of readiness

accurate assessment

consortia of states

llccommercial message